A unifying representer theorem for inverse problems and machine learning
From MaRDI portal
Publication:2231644
Applications of functional analysis in optimization, convex analysis, mathematical programming, economics (46N10) Linear operators and ill-posed problems, regularization (47A52) Numerical solutions of ill-posed problems in abstract spaces; regularization (65J20) Learning and adaptive systems in artificial intelligence (68T05)
Abstract: The standard approach for dealing with the ill-posedness of the training problem in machine learning and/or the reconstruction of a signal from a limited number of measurements is regularization. The method is applicable whenever the problem is formulated as an optimization task. The standard strategy consists in augmenting the original cost functional by an energy that penalizes solutions with undesirable behavior. The effect of regularization is very well understood when the penalty involves a Hilbertian norm. Another popular configuration is the use of an -norm (or some variant thereof) that favors sparse solutions. In this paper, we propose a higher-level formulation of regularization within the context of Banach spaces. We present a general representer theorem that characterizes the solutions of a remarkably broad class of optimization problems. We then use our theorem to retrieve a number of known results in the literature---e.g., the celebrated representer theorem of machine leaning for RKHS, Tikhonov regularization, representer theorems for sparsity promoting functionals, the recovery of spikes---as well as a few new ones.
A novel higher-level formulation of regularization is proposed in the context of Banach spaces. A general representer theorem for characterizing the solutions of a broad class of optimization problems is proven and then shown to be employable for rediscovering various known similar results from the literature.
Recommendations
- Regularized learning in Banach spaces as an optimization problem: representer theorems
- Representer Theorems in Banach Spaces: Minimum Norm Interpolation, Regularized Learning and Semi-Discrete Inverse Problems
- Sparse regularized learning in the reproducing kernel Banach spaces with the \(\ell^1\) norm
- Generalized representer theorems in Banach spaces
- When is there a representer theorem? Nondifferentiable regularisers and Banach spaces
Cites work
- A mathematical introduction to compressive sensing
- A representer theorem for deep kernel learning
- A representer theorem for deep neural networks
- A theorem on duality mappings in Banach spaces
- An Introduction to Banach Space Theory
- Compressed sensing
- Continuous-Domain Solutions of Linear Inverse Problems With Tikhonov Versus Generalized TV Regularization
- Exact support recovery for sparse spikes deconvolution
- From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images
- Generalized Mercer kernels and reproducing kernel Banach spaces
- scientific article; zbMATH DE number 1804115 (Why is no real title available?)
- scientific article; zbMATH DE number 45848 (Why is no real title available?)
- scientific article; zbMATH DE number 48239 (Why is no real title available?)
- scientific article; zbMATH DE number 3533448 (Why is no real title available?)
- scientific article; zbMATH DE number 1022519 (Why is no real title available?)
- scientific article; zbMATH DE number 1022658 (Why is no real title available?)
- scientific article; zbMATH DE number 1376935 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 3220706 (Why is no real title available?)
- scientific article; zbMATH DE number 3227378 (Why is no real title available?)
- scientific article; zbMATH DE number 3240665 (Why is no real title available?)
- scientific article; zbMATH DE number 3295108 (Why is no real title available?)
- scientific article; zbMATH DE number 3314644 (Why is no real title available?)
- Inverse problems in spaces of measures
- Irrotational or divergence-free interpolation
- Kernel methods in machine learning
- Kernels for vector-valued functions: a review
- Kernels, regularization and differential equations
- MultiDimensional Sparse Super-Resolution
- On 'best' interpolation
- On Learning Vector-Valued Functions
- On representer theorems and convex regularization
- Regularization algorithms for learning that are equivalent to multilayer networks
- Regularization methods in Banach spaces.
- Regularization networks and support vector machines
- Regularization theory in image restoration-the stabilizing functional approach
- Regularized learning in Banach spaces as an optimization problem: representer theorems
- Regularized learning schemes in feature Banach spaces
- Representer Theorems for Sparsity-Promoting <inline-formula> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math> </inline-formula> Regularization
- Reproducing kernel Banach spaces for machine learning
- Semi-reproducing kernel Hilbert spaces, splines and increment kriging
- Some results on Tchebycheffian spline functions and stochastic processes
- Sparsity and incoherence in compressive sampling
- Sparsity of solutions for variational inverse problems with finite-dimensional data
- Spline solutions to L\(^1\) extremal problems in one and several variables
- Splines are universal solutions of linear inverse problems with generalized TV regularization
- Super-resolution from noisy data
- Super-resolution of point sources via convex programming
- Support recovery for sparse super-resolution of positive measures
- Theory of Reproducing Kernels
- Towards a Mathematical Theory of Super‐resolution
- Vector-valued reproducing kernel Banach spaces with applications to multi-task learning
- When is there a representer theorem? Vector versus matrix regularizers
Cited in
(43)- Functional estimation of anisotropic covariance and autocovariance operators on the sphere
- Convex optimization in sums of Banach spaces
- Sparse regularized learning in the reproducing kernel Banach spaces with the \(\ell^1\) norm
- When is there a representer theorem? Vector versus matrix regularizers
- Value regularization and Fenchel duality
- Regularized learning schemes in feature Banach spaces
- Banach space representer theorems for neural networks and ridge splines
- On the extremal points of the ball of the Benamou-Brenier energy
- A superposition principle for the inhomogeneous continuity equation with Hellinger–Kantorovich-regular coefficients
- Debiased machine learning of global and local parameters using regularized Riesz representers
- scientific article; zbMATH DE number 7626740 (Why is no real title available?)
- Two-Layer Neural Networks with Values in a Banach Space
- Parameter choices for sparse regularization with the ℓ1 norm *
- What Kinds of Functions Do Deep Neural Networks Learn? Insights from Variational Spline Theory
- On representer theorems and convex regularization
- Minimum norm interpolation in the \(\ell_1(\mathbb{N})\) space
- Generalized representer theorems in Banach spaces
- Neural network approximation
- Explicit representations for Banach subspaces of Lizorkin distributions
- A generalized conditional gradient method for dynamic inverse problems with optimal transport regularization
- Sparse machine learning in Banach spaces
- Linear inverse problems with Hessian-Schatten total variation
- Optimal learning
- A duality approach to regularized learning problems in Banach spaces
- The Geometry of Sparse Analysis Regularization
- Regularization method for the generalized moment problem in a functional reproducing kernel Hilbert space
- Functions with bounded Hessian-Schatten variation: density, variational, and extremality properties
- On the determination of Lagrange multipliers for a weighted Lasso problem using geometric and convex analysis techniques
- Distributional extension and invertibility of the k-plane transform and its dual
- Weighted variation spaces and approximation by shallow ReLU networks
- Splitting method for support vector machine in reproducing kernel Banach space with a lower semi-continuous loss function
- From kernel methods to neural networks: a unifying variational formulation
- Compositional function spaces for deep learning
- Conditional gradients for total variation regularization with PDE constraints: a graph cuts approach
- A unified framework for solving ill-posed problems using regularized conjugate gradient, Broyden-Fletcher-Goldfarb-Shanno (BFGS), and limited-memory BFGS
- Computerized tomography and reproducing kernels
- Optimal interpolation in Hardy and Bergman spaces: a reproducing kernel Banach space approach
- Efficient algorithms for regularized nonnegative scale-invariant low-rank approximation models
- Convergence analysis of the discretization of continuous-domain inverse problems
- Function-space optimality of neural architectures with multivariate nonlinearities
- A sparse optimization approach to infinite infimal convolution regularization
- Adaptive vector-valued splines for the resolution of inverse problems
- Fast and Cheap Covariance Smoothing
This page was built for publication: A unifying representer theorem for inverse problems and machine learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2231644)
