Generalized Mercer kernels and reproducing kernel Banach spaces

DOI10.1090/MEMO/1243MaRDI QIDQ5383917zbMATH OpenOpenAlexWikidataFDO

Authors Yuesheng Xu, Qi Ye

Publication date 20 June 2019

Published in Memoirs of the American Mathematical Society (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1412.8663

zbMATH Keywords

machine learningsupport vector machinespositive definite kernelsreproducing kernel Banach spacesgeneralized Mercer kernelssparse learning methods.

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Research exposition (monographs, survey articles) pertaining to computer science (68-02) Research exposition (monographs, survey articles) pertaining to numerical analysis (65-02) Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces) (46E22) Research exposition (monographs, survey articles) pertaining to functional analysis (46-02) Numerical approximation and computational geometry (primarily algorithms) (65Dxx)

Abstract: This article studies constructions of reproducing kernel Banach spaces (RKBSs) which may be viewed as a generalization of reproducing kernel Hilbert spaces (RKHSs). A key point is to endow Banach spaces with reproducing kernels such that machine learning in RKBSs can be well-posed and of easy implementation. First we verify many advanced properties of the general RKBSs such as density, continuity, separability, implicit representation, imbedding, compactness, representer theorem for learning methods, oracle inequality, and universal approximation. Then, we develop a new concept of generalized Mercer kernels to construct $p$-norm RKBSs for $1leqpleqinfty$. The $p$-norm RKBSs preserve the same simple format as the Mercer representation of RKHSs. Moreover, the $p$-norm RKBSs are isometrically equivalent to the standard $p$-norm spaces of countable sequences. Hence, the $p$-norm RKBSs possess more geometrical structures than RKHSs including sparsity. The generalized Mercer kernels also cover many well-known kernels, for example, min kernels, Gaussian kernels, and power series kernels. Finally, we propose to solve the support vector machines in the $p$-norm RKBSs, which are to minimize the regularized empirical risks over the $p$-norm RKBSs. We show that the infinite dimensional support vector machines in the $p$-norm RKBSs can be equivalently transferred to finite dimensional convex optimization problems such that we obtain the finite dimensional representations of the support vector machine solutions for practical applications. In particular, we verify that some special support vector machines in the $1$-norm RKBSs are equivalent to the classical $1$-norm sparse regressions. This gives fundamental supports of a novel learning tool called sparse learning methods to be investigated in our next research project.

Recommendations

• Reproducing kernel Banach spaces for machine learning
• Construction of Pairs of Reproducing Kernel Banach Spaces
• On reproducing kernel Banach spaces: generic definitions and unified framework of constructions
• Solving support vector machines in reproducing kernel Banach spaces with positive definite functions
• Support vector machines in reproducing kernel Hilbert spaces versus Banach spaces

Cites work

• scientific article; zbMATH DE number 1804115 (Why is no real title available?)
• scientific article; zbMATH DE number 3168073 (Why is no real title available?)
• scientific article; zbMATH DE number 3901506 (Why is no real title available?)
• scientific article; zbMATH DE number 45848 (Why is no real title available?)
• scientific article; zbMATH DE number 3533448 (Why is no real title available?)
• scientific article; zbMATH DE number 1022658 (Why is no real title available?)
• scientific article; zbMATH DE number 1950576 (Why is no real title available?)
• scientific article; zbMATH DE number 5055767 (Why is no real title available?)
• scientific article; zbMATH DE number 3185938 (Why is no real title available?)
• 10.1162/1532443041827925
• A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines
• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
• A Multilinear Singular Value Decomposition
• A proximity algorithm accelerated by Gauss-Seidel iterations for L1/TV denoising models
• An EM algorithm for wavelet-based image restoration
• An Introduction to Banach Space Theory
• An affine scaling methodology for best basis selection
• An iterative thresholding algorithm for linear inverse problems with a sparsity constraint
• Approximation of high-dimensional kernel matrices by multilevel circulant matrices
• Average case approximation: convergence and tractability of Gaussian kernels
• Capacity of reproducing kernel spaces in learning theory
• Classes of Semi-Inner-Product Spaces
• Construction of nearest points in the \(L^ p\), p even, and \(L^ \infty\) norms. I
• Ergodic convergence to a zero of the sum of monotone operators in Hilbert space
• From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images
• Generalized native spaces
• Green's functions with applications
• Kernel techniques: From machine learning to meshless methods
• Learning Theory
• Linear integral equations
• Meshfree approximation methods with Matlab. With CD-ROM.
• Multi-step fixed-point proximity algorithms for solving a class of optimization problems arising from image processing
• Multiscale methods for Fredholm integral equations
• Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage
• On dimension-independent rates of convergence for function approximation with Gaussian kernels
• On spectral learning
• On the mathematical foundations of learning
• On the weak convergence of an ergodic iteration for the solution of variational inequalities for monotone operators in Hilbert space
• Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ ¹ minimization
• Orthogonality and Linear Functionals in Normed Linear Spaces
• Power series kernels
• Proximity algorithms for image models: denoising
• Proximity algorithms for the L1/TV image denoising model
• Radial Basis Functions
• Regularized learning in Banach spaces as an optimization problem: representer theorems
• Reproducing kernel Banach spaces for machine learning
• Reproducing kernel Banach spaces with the ^1 norm
• Reproducing kernel Banach spaces with the \(\ell^{1}\) norm. II: Error analysis for regularized least square regression
• Reproducing kernels of Sobolev spaces via a Green kernel approach with differential operators and boundary operators
• Reproducing kernels of generalized Sobolev spaces via a Green function approach with distributional operators
• Sampling in reproducing kernel Banach spaces
• Scattered Data Approximation
• Semi-Inner-Product Spaces
• Shannon sampling and function reconstruction from point values
• Solving support vector machines in reproducing kernel Banach spaces with positive definite functions
• Stable Computations with Gaussian Radial Basis Functions
• Stable evaluation of Gaussian radial basis function interpolants
• Stable signal recovery from incomplete and inaccurate measurements
• Support Vector Machines
• Support vector machines in reproducing kernel Hilbert spaces versus Banach spaces
• Support-vector networks
• Tensor Decompositions and Applications
• Theory of Reproducing Kernels
• Universal kernels
• When is there a representer theorem? Vector versus matrix regularizers

Cited in

(42)

• scientific article; zbMATH DE number 7626740 (Why is no real title available?)
• Positive definite multi-kernels for scattered data interpolations
• Understanding neural networks with reproducing kernel Banach spaces
• Learning with reproducing kernel Banach spaces
• Weighted p-regular kernels for reproducing kernel Hilbert spaces and Mercer theorem
• Regularization method for the generalized moment problem in a functional reproducing kernel Hilbert space
• Error analysis of the kernel regularized regression based on refined convex losses and RKBSs
• A homotopy method for multikernel-based approximation
• A two-step fixed-point proximity algorithm for a class of non-differentiable optimization models in machine learning
• Multi-task learning in vector-valued reproducing kernel Banach spaces with the \(\ell^1\) norm
• Reproducing kernel Banach spaces with the ^1 norm
• A novel kernelized total Bregman divergence-based fuzzy clustering with local information for image segmentation
• Kernel-based learning methods for stochastic partial differential equations
• A duality approach to regularized learning problems in Banach spaces
• Minimum norm interpolation in the \(\ell_1(\mathbb{N})\) space
• Reproducing kernel Hilbert spaces associated with analytic translation-invariant Mercer kernels
• Vector-valued reproducing kernel Banach spaces with applications to multi-task learning
• Margin error bounds for support vector machines on reproducing kernel Banach spaces
• Approximation from noisy data
• Generalized representer theorems in Banach spaces
• Regularization in a functional reproducing kernel Hilbert space
• When is there a representer theorem? Reflexive Banach spaces
• Solving support vector machines in reproducing kernel Banach spaces with positive definite functions
• Sparse support vector machines in reproducing kernel Banach spaces
• Sparse regularized learning in the reproducing kernel Banach spaces with the \(\ell^1\) norm
• Support vector machine classifiers by non-Euclidean margins
• Sparse machine learning in Banach spaces
• On reproducing kernel Banach spaces: generic definitions and unified framework of constructions
• Generalized support vector regression: duality and tensor-kernel representation
• The weighted norm for some Mercer kernel matrices
• Reproducing kernel Banach spaces for machine learning
• Sparse Deep Neural Network for Nonlinear Partial Differential Equations
• Quasi-interpolation using generalized Gaussian kernels
• Native spaces and generalization of Wu functions
• Splitting method for support vector machine in reproducing kernel Banach space with a lower semi-continuous loss function
• Banach space representer theorems for neural networks and ridge splines
• Support vector machines in reproducing kernel Hilbert spaces versus Banach spaces
• Parameter choices for sparse regularization with the ℓ1 norm ^*
• A scalable optimization approach for the multilinear system arising from scattered data interpolation
• Hermitian operators and isometries on symmetric operator spaces
• A unifying representer theorem for inverse problems and machine learning
• Optimal interpolation in Hardy and Bergman spaces: a reproducing kernel Banach space approach

Describes a project that uses

Uses Software

• rbf_qr
• TwIST
• Matlab

This page was built for publication: Generalized Mercer kernels and reproducing kernel Banach spaces

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5383917)