Consistent learning by composite proximal thresholding
From MaRDI portal
Publication:681492
Abstract: We investigate the modeling and the numerical solution of machine learning problems with prediction functions which are linear combinations of elements of a possibly infinite-dimensional dictionary. We propose a novel flexible composite regularization model, which makes it possible to incorporate various priors on the coefficients of the prediction function, including sparsity and hard constraints. We show that the estimators obtained by minimizing the regularized empirical risk are consistent in a statistical sense, and we design an error-tolerant composite proximal thresholding algorithm for computing such estimators. New results on the asymptotic behavior of the proximal forward-backward splitting method are derived and exploited to establish the convergence properties of the proposed algorithm. In particular, our method features a convergence rate in objective values.
Recommendations
- Proximal algorithms in statistics and machine learning
- A consistent algorithm to solve Lasso, elastic-net and Tikhonov regularization
- Iterative regularization for learning with convex loss functions
- Elastic-net regularization in learning theory
- Proximal Newton-type methods for minimizing composite functions
Cites work
- A consistent algorithm to solve Lasso, elastic-net and Tikhonov regularization
- A distribution-free theory of nonparametric regression
- A forward–backward splitting algorithm for the minimization of non-smooth convex functionals in Banach space
- A variational formulation for frame-based inverse problems
- Accelerated and inexact forward-backward algorithms
- Alternating proximal regularized dictionary learning
- An iterative thresholding algorithm for linear inverse problems with a sparsity constraint
- Convergence rate analysis of several splitting schemes
- Convex analysis and monotone operator theory in Hilbert spaces
- Convexity and well-posed problems
- Elastic-net regularization in learning theory
- Gradient methods for minimizing composite functions
- scientific article; zbMATH DE number 1804115 (Why is no real title available?)
- scientific article; zbMATH DE number 1807400 (Why is no real title available?)
- scientific article; zbMATH DE number 4164577 (Why is no real title available?)
- scientific article; zbMATH DE number 1968250 (Why is no real title available?)
- scientific article; zbMATH DE number 3192366 (Why is no real title available?)
- Inexact and accelerated proximal point algorithms
- Learning from examples as an inverse problem
- On the mathematical foundations of learning
- Proximal Thresholding Algorithm for Minimization over Orthonormal Bases
- Regularization and Variable Selection Via the Elastic Net
- Regularization networks and support vector machines
- Regularized learning schemes in feature Banach spaces
- Ridge Regression: Biased Estimation for Nonorthogonal Problems
- Signal Recovery by Proximal Forward-Backward Splitting
- Some properties of regularized kernel methods
- Sparsity in penalized empirical risk minimization
- Statistics for high-dimensional data. Methods, theory and applications.
- Strong Convergence of Block-Iterative Outer Approximation Methods for Convex Optimization
- Sums and Gaussian vectors
- Viscosity Solutions of Minimization Problems
- Wavelet thresholding for some classes of non–Gaussian noise
Cited in
(14)- Monotone operator theory in convex optimization
- Modified Fejér sequences and applications
- Isotonicity of the proximity operator and mixed variational inequalities in Hilbert spaces
- Solving composite fixed point problems with block updates
- Applying FISTA to optimization problems (with or) without minimizers
- Quasi-Nonexpansive Iterations on the Affine Hull of Orbits: From Mann's Mean Value Algorithm to Inertial Methods
- Isotonicity of the proximity operator and stochastic optimization problems in Hilbert quasi-lattices endowed with Lorentz cones
- Proximal activation of smooth functions in splitting algorithms for convex image recovery
- Generalized support vector regression: duality and tensor-kernel representation
- Alternating proximal regularized dictionary learning
- Consistent approximations in composite optimization
- scientific article; zbMATH DE number 7733450 (Why is no real title available?)
- The geometry of monotone operator splitting methods
- Isotonicity of proximity operators in general quasi-lattices and optimization problems
This page was built for publication: Consistent learning by composite proximal thresholding
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q681492)
