Regularized learning schemes in feature Banach spaces

DOI10.1142/S0219530516500202MaRDI QIDQ4594821zbMATH OpenOpenAlexWikidataFDO

Authors Patrick L. Combettes, Saverio Salzo, Silvia Villa

Publication date 24 November 2017

Published in Analysis and Applications (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1410.6847

zbMATH Keywords

Banach spacesconsistencyempirical riskfeature mapregularizationrepresenter theoremreproducing kernelstatistical learningtotally convex function

Mathematics Subject Classification ID

Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces) (46E22) Applications of functional analysis in probability theory and statistics (46N30) Probability theory on linear topological spaces (60B11) Nonparametric regression and quantile regression (62G08) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: This paper proposes a unified framework for the investigation of constrained learning theory in reflexive Banach spaces of features via regularized empirical risk minimization. The focus is placed on Tikhonov-like regularization with totally convex functions. This broad class of regularizers provides a flexible model for various priors on the features, including in particular hard constraints and powers of Banach norms. In such context, the main results establish a new general form of the representer theorem and the consistency of the corresponding learning schemes under general conditions on the loss function, the geometry of the feature space, and the modulus of total convexity of the regularizer. In addition, the proposed analysis gives new insight into basic tools such as reproducing Banach spaces, feature maps, and universality. Even when specialized to Hilbert spaces, this framework yields new results that extend the state of the art.

Recommendations

• Regularized learning in Banach spaces as an optimization problem: representer theorems
• Generalized representer theorems in Banach spaces
• When is there a representer theorem? Reflexive Banach spaces
• The learning rates of regularized regression based on reproducing kernel Banach spaces
• A unifying representer theorem for inverse problems and machine learning

Cites work

• 10.1162/153244303321897690
• A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines
• A distribution-free theory of nonparametric regression
• A family of minimax rates for density estimators in continuous time
• Accelerated and inexact forward-backward algorithms
• Bregman distances, totally convex functions, and a method for solving operator equations in Banach spaces
• Characteristic inequalities of uniformly convex and uniformly smooth Banach spaces
• Continuity properties of projection operators
• Convex analysis and measurable multifunctions
• Convex analysis and monotone operator theory in Hilbert spaces
• Convex functions, monotone operators and differentiability.
• Convexity and well-posed problems
• Elastic-net regularization in learning theory
• Fast learning rate of multiple kernel learning: trade-off between sparsity and smoothness
• Functional data analysis.
• scientific article; zbMATH DE number 5968915 (Why is no real title available?)
• scientific article; zbMATH DE number 1804115 (Why is no real title available?)
• scientific article; zbMATH DE number 1807400 (Why is no real title available?)
• scientific article; zbMATH DE number 3884766 (Why is no real title available?)
• scientific article; zbMATH DE number 3853749 (Why is no real title available?)
• scientific article; zbMATH DE number 5654889 (Why is no real title available?)
• scientific article; zbMATH DE number 3937791 (Why is no real title available?)
• scientific article; zbMATH DE number 49190 (Why is no real title available?)
• scientific article; zbMATH DE number 48239 (Why is no real title available?)
• scientific article; zbMATH DE number 50401 (Why is no real title available?)
• scientific article; zbMATH DE number 3465097 (Why is no real title available?)
• scientific article; zbMATH DE number 3576139 (Why is no real title available?)
• scientific article; zbMATH DE number 3626044 (Why is no real title available?)
• scientific article; zbMATH DE number 1332320 (Why is no real title available?)
• scientific article; zbMATH DE number 490143 (Why is no real title available?)
• scientific article; zbMATH DE number 1950575 (Why is no real title available?)
• scientific article; zbMATH DE number 1968250 (Why is no real title available?)
• scientific article; zbMATH DE number 1540658 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• scientific article; zbMATH DE number 2208228 (Why is no real title available?)
• scientific article; zbMATH DE number 3220452 (Why is no real title available?)
• Inequalities in Banach spaces with applications
• Integral representation of dominated operations on spaces of continuous vector fields
• Introduction to nonparametric estimation
• Iterative averaging of entropic projections for solving stochastic convex feasibility problems
• Kernel methods in machine learning
• Kernel regression with functional response
• Learning multiple tasks with kernel methods
• Learning rates of least-square regularized regression
• Learning Theory
• Learning Theory
• Local Rademacher complexities
• Modern methods in the calculus of variations. L^p spaces
• On Learning Vector-Valued Functions
• On the mathematical foundations of learning
• On uniformly convex functionals
• On uniformly convex functions
• Optimal learning rates for least squares regularized regression with unbounded sampling
• Proximal Thresholding Algorithm for Minimization over Orthonormal Bases
• Quantitative Stability of Variational Systems: I. The Epigraphical Distance
• Regularization and Variable Selection Via the Elastic Net
• Regularized learning in Banach spaces as an optimization problem: representer theorems
• Reproducing kernel Banach spaces for machine learning
• Reproducing kernel Banach spaces with the ^1 norm
• Reproducing kernel Banach spaces with the \(\ell^{1}\) norm. II: Error analysis for regularized least square regression
• Solving support vector machines in reproducing kernel Banach spaces with positive definite functions
• Some properties of regularized kernel methods
• Spaces of operator-valued functions measurable with respect to the strong operator topology
• Sparsity in penalized empirical risk minimization
• Statistics for high-dimensional data. Methods, theory and applications.
• Strong Convergence of Block-Iterative Outer Approximation Methods for Convex Optimization
• Structured sparsity and generalization
• Sums and Gaussian vectors
• Support Vector Machines
• Totally convex functions for fixed points computation and infinite dimensional optimization
• Two oracle inequalities for regularized boosting classifiers
• Universal kernels
• Variational Analysis in Sobolev and BV Spaces
• VECTOR VALUED REPRODUCING KERNEL HILBERT SPACES AND UNIVERSALITY
• VECTOR VALUED REPRODUCING KERNEL HILBERT SPACES OF INTEGRABLE FUNCTIONS AND MERCER THEOREM
• Vector-valued reproducing kernel Banach spaces with applications to multi-task learning
• Viscosity Solutions of Minimization Problems
• Wavelet thresholding for some classes of non–Gaussian noise
• Weak convergence and empirical processes. With applications to statistics

Cited in

(16)

• Bias reduction in variational regularization
• The global convergence of the nonlinear power method for mixed-subordinate matrix norms
• Understanding neural networks with reproducing kernel Banach spaces
• An efficient collocation algorithm with SSP-RK43 scheme to solve Rosenau-KdV-RLW equation
• A unifying representer theorem for inverse problems and machine learning
• Concentration of weakly dependent Banach-valued sums and applications to statistical learning methods
• The Random Feature Model for Input-Output Maps between Banach Spaces
• Error analysis of the kernel regularized regression based on refined convex losses and RKBSs
• Thresholding gradient methods in Hilbert spaces: support identification and linear convergence
• Sufficient ensemble size for random matrix theory-based handling of singular covariance matrices
• Generalized support vector regression: duality and tensor-kernel representation
• Explicit representations for Banach subspaces of Lizorkin distributions
• Sparse machine learning in Banach spaces
• Optimal Algorithms for Stochastic Complementary Composite Minimization
• Consistent learning by composite proximal thresholding
• Regularized learning in Banach spaces as an optimization problem: representer theorems

Describes a project that uses

Uses Software

• fda (R)

This page was built for publication: Regularized learning schemes in feature Banach spaces

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4594821)