Kernel methods in machine learning

DOI10.1214/009053607000000677zbMATH Open1151.30007arXivmath/0701907OpenAlexW3101749733WikidataQ93514201 ScholiaQ93514201MaRDI QIDQ930651FDOQ930651

Bernhard Schölkopf, Thomas Hofmann, Alex J. Smola

Publication date: 1 July 2008

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We review machine learning methods employing positive definite kernels. These methods formulate learning and estimation problems in a reproducing kernel Hilbert space (RKHS) of functions defined on the data domain, expanded in terms of a kernel. Working in linear spaces of function has the benefit of facilitating the construction and analysis of learning algorithms while at the same time allowing large classes of functions. The latter include nonlinear functions as well as functions defined on nonvectorial data. We cover a wide range of methods, ranging from binary classifiers to sophisticated methods for estimation with structured data.

Full work available at URL: https://arxiv.org/abs/math/0701907

Recommendations

• A tutorial on kernel methods for categorization
• Kernel methods: a survey of current techniques
• scientific article; zbMATH DE number 5957287
• scientific article; zbMATH DE number 1314294
• Kernel methods and machine learning

zbMATH Keywords

machine learninggraphical modelssupport vector machinesreproducing kernels

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Kernel functions in one complex variable and applications (30C40)

Cites Work

• Graphical Models, Exponential Families, and Variational Inference
• Title not available (Why is that?)
• Support-vector networks
• Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Atomic Decomposition by Basis Pursuit
• Title not available (Why is that?)
• Learning Theory and Kernel Machines
• Consistent independent component analysis and prewhitening
• Ridge Regression: Biased Estimation for Nonorthogonal Problems
• Theory of Reproducing Kernels
• Robust Statistics
• Smoothing spline ANOVA for exponential families, with application to the Wisconsin epidemiological study of diabetic retinopathy. (The 1994 Neyman Memorial Lecture)
• Some results on Tchebycheffian spline functions and stochastic processes
• An introduction to support vector machines and other kernel-based learning methods.
• 10.1162/153244303768966085
• On measures of dependence
• Title not available (Why is that?)
• Title not available (Why is that?)
• What is Projection Pursuit?
• Title not available (Why is that?)
• Title not available (Why is that?)
• A Projection Pursuit Algorithm for Exploratory Data Analysis
• Algorithmic Learning Theory
• An algorithm for information structuring and retrieval
• RELATIONS BETWEEN TWO SETS OF VARIATES
• Kernel methods in machine learning
• On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
• Title not available (Why is that?)
• Convex Analysis
• Projection pursuit
• On the influence of the kernel on the consistency of support vector machines
• Exploratory Projection Pursuit
• Title not available (Why is that?)
• Canonical analysis of several sets of variables
• Scattered Data Approximation
• Title not available (Why is that?)
• The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
• Theoretical foundations of the potential function method in pattern recognition learning
• Metric spaces and completely monontone functions
• Title not available (Why is that?)
• 10.1162/153244303321897690
• Title not available (Why is that?)
• Generalized Iterative Scaling for Log-Linear Models
• Convexity, Classification, and Risk Bounds
• Title not available (Why is that?)
• Positive definite functions and generalizations, an historical survey
• Monotone Funktionen, Stieltjessche Integrale und harmonische Analyse
• Numerical recipes in C++. The art of scientific computing
• Semi-Infinite Programming: Theory, Methods, and Applications
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Advances in large-margin classifiers
• Reducing multiclass to binary: A unifying approach for margin classifiers
• Linear and Nonlinear Separation of Patterns by Linear Programming
• Estimation of dependences based on empirical data. Transl. from the Russian by Samuel Kotz
• Support vector machines are universally consistent
• Estimating the support of a high-dimensional distribution
• 10.1162/15324430260185628
• Rademacher penalties and structural risk minimization
• Title not available (Why is that?)
• Theory of Classification: a Survey of Some Recent Advances
• Convergence de la répartition empirique vers la répartition théorique
• Title not available (Why is that?)
• Training invariant support vector machines
• On the difficulty of approximately maximizing agreements.
• Generalized quantile processes
• Nonlinear canonical analysis and independence tests
• Restricted canonical correlations
• Title not available (Why is that?)
• Scale-sensitive dimensions, uniform convergence, and learnability
• Title not available (Why is that?)
• A PAC-Bayesian margin bound for linear classifiers
• On a kernel-based method for pattern recognition, regression, approximation, and operator inversion
• Title not available (Why is that?)
• A unifying information-theoretic framework for independent component analysis
• Title not available (Why is that?)
• Maximal margin classification for metric spaces
• Learning Theory and Kernel Machines
• On optimal nonlinear associative recall
• Functions that preserve families of positive semidefinite matrices
• Learning Theory

Cited In (only showing first 100 items - show all)

• Choosing shape parameters for regression in reproducing kernel Hilbert space and variable selection
• On the Numerical Rank of Radial Basis Function Kernels in High Dimensions
• Applied Koopman theory for partial differential equations and data-driven modeling of spatio-temporal systems
• On the positive semi-definite property of similarity matrices
• An efficient implementation for spatial-temporal Gaussian process regression and its applications
• Function replacement vs. kernel trick
• Distribution-free uncertainty quantification for kernel methods by gradient perturbations
• Eigendecompositions of transfer operators in reproducing kernel Hilbert spaces
• Solving higher order nonlinear ordinary differential equations with least squares support vector machines
• Machine learning with kernels for portfolio valuation and risk management
• Title not available (Why is that?)
• Self-Assignment Flows for Unsupervised Data Labeling on Graphs
• Generalization bounds for regularized portfolio selection with market side information
• The Choquet kernel on the use of regression problem
• Prediction of magnetization dynamics in a reduced dimensional feature space setting utilizing a low-rank kernel method
• Kernel learning backward SDE filter for data assimilation
• Stochastic nonlocal damage analysis by a machine learning approach
• A feasible \(k\)-means kernel trick under non-Euclidean feature space
• Learning time-stepping by nonlinear dimensionality reduction to predict magnetization dynamics
• Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques
• \textsf{StreaMRAK} a streaming multi-resolution adaptive kernel algorithm
• Sparse feature map-based Markov models for nonlinear fluid flows
• Kernel-based parameter estimation of dynamical systems with unknown observation functions
• Computing functions of random variables via reproducing kernel Hilbert space representations
• Unsupervised assignment flow: label learning on feature manifolds by spatially regularized geometric assignment
• Sufficient dimension reduction for average causal effect estimation
• Consistency of empirical Bayes and kernel flow for hierarchical parameter estimation
• A decision-making machine learning approach in Hermite spectral approximations of partial differential equations
• A mean field game inverse problem
• A bag-of-paths framework for network data analysis
• The Big Data Newsvendor: Practical Insights from Machine Learning
• Adaptive machine learning-based surrogate modeling to accelerate PDE-constrained optimization in enhanced oil recovery
• Polymorphic uncertainty quantification for engineering structures via a hyperplane modelling technique
• Randomized approximate class-specific kernel spectral regression analysis for large-scale face verification
• Direct symbol decoding using GA-SVM in chaotic baseband wireless communication system
• Inverse optimization with kernel regression: application to the power forecasting and bidding of a fleet of electric vehicles
• Sparse Cholesky Factorization by Kullback--Leibler Minimization
• Kernel machines for current status data
• Fisher-regularized support vector machine
• Fast generation of Gaussian random fields for direct numerical simulations of stochastic transport
• Polynomial degree reduction in the \(\mathcal{L}^2\)-norm on a symmetric interval for the canonical basis
• Augmented direct learning for conditional average treatment effect estimation with double robustness
• Interactions Between Kernels, Frames, and Persistent Homology
• Title not available (Why is that?)
• Mathematical programming formulations for piecewise polynomial functions
• The theory and application of penalized methods or reproducing kernel Hilbert spaces made easy
• Near-optimal coresets of kernel density estimates
• Mixture of multivariate Gaussian processes for classification of irregularly sampled satellite image time-series
• A kernel approach to multi-task learning with task-specific kernels
• Learning in high-dimensional feature spaces using ANOVA-based fast matrix-vector multiplication
• Samplets: construction and scattered data compression
• A unifying representer theorem for inverse problems and machine learning
• A regression perspective on generalized distance covariance and the Hilbert-Schmidt independence criterion
• Quantum mean centering for block-encoding-based quantum algorithm
• Functional analysis techniques to improve similarity matrices in discrimination problems
• Title not available (Why is that?)
• On some transformations of high dimension, low sample size data for nearest neighbor classification
• Kernel machines with missing covariates
• Title not available (Why is that?)
• On the Schoenberg transformations in data analysis: theory and illustrations
• Kernel methods in system identification, machine learning and function estimation: a survey
• A reproducing kernel Hilbert space approach to high dimensional partially varying coefficient model
• Sparse kernel deep stacking networks
• Supervised classification and mathematical optimization
• Reproducing kernel almost Pontryagin spaces
• 10.1162/1532443041827934
• Incorporating auxiliary information for improved prediction using combination of kernel machines
• Model selection in kernel ridge regression
• ASKIT: an efficient, parallel library for high-dimensional kernel summations
• Restricted kernel canonical correlation analysis
• Tensors in computations
• Kernel method based on non-linear coherent states in quantum feature space
• Generalized Kalman smoothing: modeling and algorithms
• Functional-bandwidth kernel for support vector machine with functional data: an alternating optimization algorithm
• 10.1162/153244303322533205
• Covariate balancing propensity score by tailored loss functions
• Continuum versus discrete networks, graph Laplacians, and reproducing kernel Hilbert spaces
• Binary vectors for fast distance and similarity estimation
• On minimum distribution discrepancy support vector machine for domain adaptation
• Kernel machines with missing responses
• Super-exponential decay rates for eigenvalues and singular values of integral operators on the sphere
• Title not available (Why is that?)
• Application and comparison of kernel functions for linear parameter varying model approximation of nonlinear systems
• Segmentation of brain tissues from magnetic resonance images using adaptively regularized kernel-based fuzzy \(C\)-means clustering
• The theory of the quantum kernel-based binary classifier
• Title not available (Why is that?)
• Bayesian calibration for large‐scale fluid structure interaction problems under embedded/immersed boundary framework
• A Hilbert Space Embedding for Distributions
• Generalized Mahalanobis depth in the reproducing kernel Hilbert space
• A novel Kalman filter formulation for improving tracking performance of the extended kernel RLS
• Selection of time instants and intervals with support vector regression for multivariate functional data
• Committee polyhedral separability: complexity and polynomial approximation
• Learning low-dimensional nonlinear structures from high-dimensional noisy data: an integral operator approach
• Title not available (Why is that?)
• Title not available (Why is that?)
• A nested heuristic for parameter tuning in support vector machines
• Online multikernel learning based on a triple-norm regularizer for semantic image classification
• Two-phase iteration for value function approximation and hyperparameter optimization in Gaussian-kernel-based adaptive critic design
• Preference disaggregation and statistical learning for multicriteria decision support: A review
• Kernel discriminant analysis and clustering with parsimonious Gaussian process models

Uses Software

• Kernel-Machines
• PDCO
• FastICA

This page was built for publication: Kernel methods in machine learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q930651)