Kernel methods in machine learning

DOI10.1214/009053607000000677zbMATH Open1151.30007arXivmath/0701907OpenAlexW3101749733WikidataQ93514201 ScholiaQ93514201MaRDI QIDQ930651FDOQ930651

Bernhard Schölkopf, Thomas Hofmann, Alex J. Smola

Publication date: 1 July 2008

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We review machine learning methods employing positive definite kernels. These methods formulate learning and estimation problems in a reproducing kernel Hilbert space (RKHS) of functions defined on the data domain, expanded in terms of a kernel. Working in linear spaces of function has the benefit of facilitating the construction and analysis of learning algorithms while at the same time allowing large classes of functions. The latter include nonlinear functions as well as functions defined on nonvectorial data. We cover a wide range of methods, ranging from binary classifiers to sophisticated methods for estimation with structured data.

Full work available at URL: https://arxiv.org/abs/math/0701907

Recommendations

• A tutorial on kernel methods for categorization
• Kernel methods: a survey of current techniques
• scientific article; zbMATH DE number 5957287
• scientific article; zbMATH DE number 1314294
• Kernel methods and machine learning

zbMATH Keywords

machine learninggraphical modelssupport vector machinesreproducing kernels

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Kernel functions in one complex variable and applications (30C40)

Cites Work

• Graphical Models, Exponential Families, and Variational Inference
• Title not available (Why is that?)
• Support-vector networks
• Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Atomic Decomposition by Basis Pursuit
• Title not available (Why is that?)
• Learning Theory and Kernel Machines
• Consistent independent component analysis and prewhitening
• Ridge Regression: Biased Estimation for Nonorthogonal Problems
• Theory of Reproducing Kernels
• Robust Statistics
• Smoothing spline ANOVA for exponential families, with application to the Wisconsin epidemiological study of diabetic retinopathy. (The 1994 Neyman Memorial Lecture)
• Some results on Tchebycheffian spline functions and stochastic processes
• An introduction to support vector machines and other kernel-based learning methods.
• 10.1162/153244303768966085
• On measures of dependence
• Title not available (Why is that?)
• Title not available (Why is that?)
• What is Projection Pursuit?
• Title not available (Why is that?)
• Title not available (Why is that?)
• A Projection Pursuit Algorithm for Exploratory Data Analysis
• Algorithmic Learning Theory
• An algorithm for information structuring and retrieval
• RELATIONS BETWEEN TWO SETS OF VARIATES
• Kernel methods in machine learning
• On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
• Title not available (Why is that?)
• Convex Analysis
• Projection pursuit
• On the influence of the kernel on the consistency of support vector machines
• Exploratory Projection Pursuit
• Title not available (Why is that?)
• Canonical analysis of several sets of variables
• Scattered Data Approximation
• Title not available (Why is that?)
• The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
• Theoretical foundations of the potential function method in pattern recognition learning
• Metric spaces and completely monontone functions
• Title not available (Why is that?)
• 10.1162/153244303321897690
• Title not available (Why is that?)
• Generalized Iterative Scaling for Log-Linear Models
• Convexity, Classification, and Risk Bounds
• Title not available (Why is that?)
• Positive definite functions and generalizations, an historical survey
• Monotone Funktionen, Stieltjessche Integrale und harmonische Analyse
• Numerical recipes in C++. The art of scientific computing
• Semi-Infinite Programming: Theory, Methods, and Applications
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Advances in large-margin classifiers
• Reducing multiclass to binary: A unifying approach for margin classifiers
• Linear and Nonlinear Separation of Patterns by Linear Programming
• Estimation of dependences based on empirical data. Transl. from the Russian by Samuel Kotz
• Support vector machines are universally consistent
• Estimating the support of a high-dimensional distribution
• 10.1162/15324430260185628
• Rademacher penalties and structural risk minimization
• Title not available (Why is that?)
• Theory of Classification: a Survey of Some Recent Advances
• Convergence de la répartition empirique vers la répartition théorique
• Title not available (Why is that?)
• Training invariant support vector machines
• On the difficulty of approximately maximizing agreements.
• Generalized quantile processes
• Nonlinear canonical analysis and independence tests
• Restricted canonical correlations
• Title not available (Why is that?)
• Scale-sensitive dimensions, uniform convergence, and learnability
• Title not available (Why is that?)
• A PAC-Bayesian margin bound for linear classifiers
• On a kernel-based method for pattern recognition, regression, approximation, and operator inversion
• Title not available (Why is that?)
• A unifying information-theoretic framework for independent component analysis
• Title not available (Why is that?)
• Maximal margin classification for metric spaces
• Learning Theory and Kernel Machines
• On optimal nonlinear associative recall
• Functions that preserve families of positive semidefinite matrices
• Learning Theory

Cited In (only showing first 100 items - show all)

• Functional analysis techniques to improve similarity matrices in discrimination problems
• Title not available (Why is that?)
• On some transformations of high dimension, low sample size data for nearest neighbor classification
• Kernel machines with missing covariates
• Title not available (Why is that?)
• On the Schoenberg transformations in data analysis: theory and illustrations
• Kernel methods in system identification, machine learning and function estimation: a survey
• A reproducing kernel Hilbert space approach to high dimensional partially varying coefficient model
• Sparse kernel deep stacking networks
• Supervised classification and mathematical optimization
• Reproducing kernel almost Pontryagin spaces
• 10.1162/1532443041827934
• Incorporating auxiliary information for improved prediction using combination of kernel machines
• Model selection in kernel ridge regression
• ASKIT: an efficient, parallel library for high-dimensional kernel summations
• Restricted kernel canonical correlation analysis
• Tensors in computations
• Kernel method based on non-linear coherent states in quantum feature space
• Generalized Kalman smoothing: modeling and algorithms
• Functional-bandwidth kernel for support vector machine with functional data: an alternating optimization algorithm
• 10.1162/153244303322533205
• Covariate balancing propensity score by tailored loss functions
• Continuum versus discrete networks, graph Laplacians, and reproducing kernel Hilbert spaces
• Binary vectors for fast distance and similarity estimation
• On minimum distribution discrepancy support vector machine for domain adaptation
• Kernel machines with missing responses
• Super-exponential decay rates for eigenvalues and singular values of integral operators on the sphere
• Title not available (Why is that?)
• Application and comparison of kernel functions for linear parameter varying model approximation of nonlinear systems
• Segmentation of brain tissues from magnetic resonance images using adaptively regularized kernel-based fuzzy \(C\)-means clustering
• The theory of the quantum kernel-based binary classifier
• Title not available (Why is that?)
• Bayesian calibration for large‐scale fluid structure interaction problems under embedded/immersed boundary framework
• A Hilbert Space Embedding for Distributions
• Generalized Mahalanobis depth in the reproducing kernel Hilbert space
• A novel Kalman filter formulation for improving tracking performance of the extended kernel RLS
• Selection of time instants and intervals with support vector regression for multivariate functional data
• Committee polyhedral separability: complexity and polynomial approximation
• Learning low-dimensional nonlinear structures from high-dimensional noisy data: an integral operator approach
• Title not available (Why is that?)
• Title not available (Why is that?)
• A nested heuristic for parameter tuning in support vector machines
• Online multikernel learning based on a triple-norm regularizer for semantic image classification
• Two-phase iteration for value function approximation and hyperparameter optimization in Gaussian-kernel-based adaptive critic design
• Preference disaggregation and statistical learning for multicriteria decision support: A review
• Kernel discriminant analysis and clustering with parsimonious Gaussian process models
• A novel Frank-Wolfe algorithm. Analysis and applications to large-scale SVM training
• A classification method for binary predictors combining similarity measures and mixture models
• Convex optimization in sums of Banach spaces
• Title not available (Why is that?)
• A novel extended kernel recursive least squares algorithm
• 10.1162/15324430260185646
• Filtering with State-Observation Examples via Kernel Monte Carlo Filter
• Nonparametric likelihood based estimation of linear filters for point processes
• Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity
• On b-bit min-wise hashing for large-scale regression and classification with sparse data
• Some remarks on diffusion distances
• Continuous-variable nonlocality and contextuality
• Gradient-Based Kernel Dimension Reduction for Regression
• A regularization approach to the many instruments problem
• A kernel learning framework for domain adaptation learning
• Structural, Syntactic, and Statistical Pattern Recognition
• Regularized learning schemes in feature Banach spaces
• Estimating the support of a high-dimensional distribution
• Ensemble Kalman filtering for non-linear likelihood models using kernel-shrinkage regression techniques
• Identifying outliers using multiple kernel canonical correlation analysis with application to imaging genetics
• Kernel techniques: From machine learning to meshless methods
• Title not available (Why is that?)
• SVM kernel by electric network
• Kernel methods: a survey of current techniques
• Title not available (Why is that?)
• Kernel methods in machine learning
• Soft and hard classification by reproducing kernel Hilbert space methods
• Invariant kernel functions for pattern analysis and machine learning
• Learning Bounds for Kernel Regression Using Effective Data Dimensionality
• \(L^p\) error estimates for approximation by Sobolev splines and Wendland functions on \(\mathbb R^d\)
• Choosing shape parameters for regression in reproducing kernel Hilbert space and variable selection
• On the Numerical Rank of Radial Basis Function Kernels in High Dimensions
• Applied Koopman theory for partial differential equations and data-driven modeling of spatio-temporal systems
• On the positive semi-definite property of similarity matrices
• An efficient implementation for spatial-temporal Gaussian process regression and its applications
• Function replacement vs. kernel trick
• Distribution-free uncertainty quantification for kernel methods by gradient perturbations
• Eigendecompositions of transfer operators in reproducing kernel Hilbert spaces
• Solving higher order nonlinear ordinary differential equations with least squares support vector machines
• Machine learning with kernels for portfolio valuation and risk management
• Title not available (Why is that?)
• Self-Assignment Flows for Unsupervised Data Labeling on Graphs
• Generalization bounds for regularized portfolio selection with market side information
• The Choquet kernel on the use of regression problem
• Integrated kernels and their properties
• Prediction of magnetization dynamics in a reduced dimensional feature space setting utilizing a low-rank kernel method
• Kernel learning backward SDE filter for data assimilation
• Stochastic nonlocal damage analysis by a machine learning approach
• A feasible \(k\)-means kernel trick under non-Euclidean feature space
• Learning time-stepping by nonlinear dimensionality reduction to predict magnetization dynamics
• Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques
• \textsf{StreaMRAK} a streaming multi-resolution adaptive kernel algorithm
• Sparse feature map-based Markov models for nonlinear fluid flows
• Kernel-based parameter estimation of dynamical systems with unknown observation functions

Uses Software

• Kernel-Machines
• PDCO
• FastICA

This page was built for publication: Kernel methods in machine learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q930651)