A representer theorem for deep kernel learning
From MaRDI portal
Abstract: In this paper we provide a finite-sample and an infinite-sample representer theorem for the concatenation of (linear combinations of) kernel functions of reproducing kernel Hilbert spaces. These results serve as mathematical foundation for the analysis of machine learning algorithms based on compositions of functions. As a direct consequence in the finite-sample case, the corresponding infinite-dimensional minimization problems can be recast into (nonlinear) finite-dimensional minimization problems, which can be tackled with nonlinear optimization algorithms. Moreover, we show how concatenated machine learning problems can be reformulated as neural networks and how our representer theorem applies to a broad class of state-of-the-art deep learning methods.
Recommendations
- scientific article; zbMATH DE number 1804115
- When is there a representer theorem? Vector versus matrix regularizers
- A representer theorem for deep neural networks
- When is there a representer theorem? Nondifferentiable regularisers and Banach spaces
- When is there a representer theorem? Reflexive Banach spaces
Cites work
- A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines
- Approximation by superpositions of a sigmoidal function
- Approximation of bi-variate functions: singular value decomposition versus sparse grids
- Deep learning
- Error Estimates for Multivariate Regression on Discretized Function Spaces
- Multiple kernel learning algorithms
- On Learning Vector-Valued Functions
- Optimal quasi-Monte Carlo rules on order 2 digital nets for the numerical integration of multivariate periodic functions
- Reproducing kernels of generalized Sobolev spaces via a Green function approach with distributional operators
- Scattered Data Approximation
- Sparse grids
- Stability of kernel-based interpolation
- Support Vector Machines
- Theory of Reproducing Kernels
Cited in
(23)- scientific article; zbMATH DE number 7626740 (Why is no real title available?)
- Deep restricted kernel machines using conjugate feature duality
- Multiresolution kernel matrix algebra
- Do ideas have shape? Idea registration as the continuous limit of artificial neural networks
- What Kinds of Functions Do Deep Neural Networks Learn? Insights from Variational Spline Theory
- A representer theorem for deep neural networks
- Data-Driven Kernel Designs for Optimized Greedy Schemes: A Machine Learning Perspective
- Kernel analysis of deep networks
- Reproducing property of bounded linear operators and kernel regularized least square regressions
- Learning rates for the kernel regularized regression with a differentiable strongly convex loss
- Towards Gaussian process for operator learning: an uncertainty aware resolution independent operator learning algorithm for computational mechanics
- Mehler’s Formula, Branching Process, and Compositional Kernels of Deep Neural Networks
- Application of deep kernel models for certified and adaptive RB-ML-ROM surrogate modeling
- Statistical inference using regularized M-estimation in the reproducing kernel Hilbert space for handling missing data
- Compositional function spaces for deep learning
- Multiscale scattered data analysis in samplet coordinates
- Analysis of structured deep kernel networks
- Deep networks for system identification: a survey
- Spectral complexity of deep neural networks
- Learning deep kernels in the space of dot product polynomials
- Be greedy and learn: efficient and certified algorithms for parametrized optimal control problems
- Kernel-based linear system identification: when does the representer theorem hold?
- A unifying representer theorem for inverse problems and machine learning
This page was built for publication: A representer theorem for deep kernel learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5381118)
