Iterative regularization for learning with convex loss functions
From MaRDI portal
Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces) (46E22) Nonparametric regression and quantile regression (62G08) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05) Convex programming (90C25)
Abstract: We consider the problem of supervised learning with convex loss functions and propose a new form of iterative regularization based on the subgradient method. Unlike other regularization approaches, in iterative regularization no constraint or penalization is considered, and generalization is achieved by (early) stopping an empirical iteration. We consider a nonparametric setting, in the framework of reproducing kernel Hilbert spaces, and prove finite sample bounds on the excess risk under general regularity conditions. Our study provides a new class of efficient regularized learning algorithms and gives insights on the interplay between statistics and optimization in machine learning.
Recommendations
- On regularization algorithms in learning theory
- Learning the kernel function via regularization
- On convergence of kernel learning estimators
- Regularization techniques and suboptimal solutions to optimization problems in learning from data
- Learning rates for the kernel regularized regression with a differentiable strongly convex loss
Cited in
(23)- Modified Fejér sequences and applications
- Perturbed iterate SGD for Lipschitz continuous loss functions
- From inexact optimization to learning via gradient concentration
- Kernel-based maximum correntropy criterion with gradient descent method
- Kernel gradient descent algorithm for information theoretic learning
- Online pairwise learning algorithms with convex loss functions
- A new analytical approach to consistency and overfitting in regularized empirical risk minimization
- Consistency of learning algorithms using Attouch-Wets convergence
- Regularization techniques and suboptimal solutions to optimization problems in learning from data
- Correction to "The Importance of Convexity in Learning With Squared Loss" [Sep 98 1974-1980]
- Gradient descent for robust kernel-based regression
- Distributed learning with regularized least squares
- The importance of convexity in learning with squared loss
- Graph-dependent implicit regularisation for distributed stochastic subgradient descent
- Generalization performance of multi-pass stochastic gradient descent with convex loss functions
- scientific article; zbMATH DE number 7306907 (Why is no real title available?)
- Coefficient-based regularization network with variance loss for error
- Analysis of Online Composite Mirror Descent Algorithm
- Block coordinate type methods for optimization and learning
- Pairwise learning problems with regularization networks and Nyström subsampling approach
- Sparse online regression algorithm with insensitive loss functions
- Consistent learning by composite proximal thresholding
- Iterative regularization in classification via hinge loss diagonal descent
This page was built for publication: Iterative regularization for learning with convex loss functions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2810891)
