Kernel methods in machine learning

DOI10.1214/009053607000000677zbMATH Open1151.30007arXivmath/0701907OpenAlexW3101749733WikidataQ93514201 ScholiaQ93514201MaRDI QIDQ930651FDOQ930651

Authors: Thomas Hofmann, Bernhard Schölkopf, Alex J. Smola 

Publication date: 1 July 2008

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We review machine learning methods employing positive definite kernels. These methods formulate learning and estimation problems in a reproducing kernel Hilbert space (RKHS) of functions defined on the data domain, expanded in terms of a kernel. Working in linear spaces of function has the benefit of facilitating the construction and analysis of learning algorithms while at the same time allowing large classes of functions. The latter include nonlinear functions as well as functions defined on nonvectorial data. We cover a wide range of methods, ranging from binary classifiers to sophisticated methods for estimation with structured data.

Full work available at URL: https://arxiv.org/abs/math/0701907

Recommendations

• A tutorial on kernel methods for categorization
• Kernel methods: a survey of current techniques
• scientific article; zbMATH DE number 5957287
• scientific article; zbMATH DE number 1314294
• Kernel methods and machine learning

zbMATH Keywords

machine learninggraphical modelssupport vector machinesreproducing kernels

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Kernel functions in one complex variable and applications (30C40)

Cites Work

• Graphical models, exponential families, and variational inference
• Title not available (Why is that?)
• Support-vector networks
• Regularization algorithms for learning that are equivalent to multilayer networks
• Gaussian processes for machine learning.
• Title not available (Why is that?)
• Title not available (Why is that?)
• Atomic Decomposition by Basis Pursuit
• Title not available (Why is that?)
• Kernels and regularization on graphs.
• Consistent independent component analysis and prewhitening
• Ridge Regression: Biased Estimation for Nonorthogonal Problems
• Theory of Reproducing Kernels
• Robust Statistics
• Smoothing spline ANOVA for exponential families, with application to the Wisconsin epidemiological study of diabetic retinopathy. (The 1994 Neyman Memorial Lecture)
• Some results on Tchebycheffian spline functions and stochastic processes
• An introduction to support vector machines and other kernel-based learning methods.
• 10.1162/153244303768966085
• On measures of dependence
• Title not available (Why is that?)
• Title not available (Why is that?)
• What is Projection Pursuit?
• Title not available (Why is that?)
• Title not available (Why is that?)
• A Projection Pursuit Algorithm for Exploratory Data Analysis
• Algorithmic Learning Theory
• An algorithm for information structuring and retrieval
• RELATIONS BETWEEN TWO SETS OF VARIATES
• Kernel methods in machine learning
• On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
• Title not available (Why is that?)
• Convex Analysis
• Projection pursuit
• On the influence of the kernel on the consistency of support vector machines
• Exploratory Projection Pursuit
• Title not available (Why is that?)
• Canonical analysis of several sets of variables
• Scattered Data Approximation
• Title not available (Why is that?)
• The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
• Theoretical foundations of the potential function method in pattern recognition learning
• Metric spaces and completely monontone functions
• Title not available (Why is that?)
• 10.1162/153244303321897690
• Title not available (Why is that?)
• Generalized Iterative Scaling for Log-Linear Models
• Convexity, Classification, and Risk Bounds
• Title not available (Why is that?)
• Positive definite functions and generalizations, an historical survey
• Monotone Funktionen, Stieltjessche Integrale und harmonische Analyse
• Numerical recipes in C++. The art of scientific computing
• Semi-Infinite Programming: Theory, Methods, and Applications
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Advances in large-margin classifiers
• Reducing multiclass to binary: A unifying approach for margin classifiers
• Linear and Nonlinear Separation of Patterns by Linear Programming
• Estimation of dependences based on empirical data. Transl. from the Russian by Samuel Kotz
• Support vector machines are universally consistent
• Estimating the support of a high-dimensional distribution
• 10.1162/15324430260185628
• Rademacher penalties and structural risk minimization
• Large margin methods for structured and interdependent output variables
• Theory of Classification: a Survey of Some Recent Advances
• Convergence de la répartition empirique vers la répartition théorique
• Title not available (Why is that?)
• Training invariant support vector machines
• On the difficulty of approximately maximizing agreements.
• Generalized quantile processes
• Nonlinear canonical analysis and independence tests
• Restricted canonical correlations
• Title not available (Why is that?)
• Scale-sensitive dimensions, uniform convergence, and learnability
• Title not available (Why is that?)
• A PAC-Bayesian margin bound for linear classifiers
• On a kernel-based method for pattern recognition, regression, approximation, and operator inversion
• Title not available (Why is that?)
• A unifying information-theoretic framework for independent component analysis
• Title not available (Why is that?)
• Maximal margin classification for metric spaces
• Learning Theory and Kernel Machines
• On optimal nonlinear associative recall
• Functions that preserve families of positive semidefinite matrices
• Learning Theory

Cited In (only showing first 100 items - show all)

• Functional analysis techniques to improve similarity matrices in discrimination problems
• Title not available (Why is that?)
• On some transformations of high dimension, low sample size data for nearest neighbor classification
• Similarity, kernels, and the fundamental constraints on cognition
• Kernel machines with missing covariates
• Semigroup kernels on measures.
• Title not available (Why is that?)
• Characterizing the function space for Bayesian kernel models
• On the Schoenberg transformations in data analysis: theory and illustrations
• Kernel methods in system identification, machine learning and function estimation: a survey
• A reproducing kernel Hilbert space approach to high dimensional partially varying coefficient model
• Sparse kernel deep stacking networks
• Supervised classification and mathematical optimization
• Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity
• Reproducing kernel almost Pontryagin spaces
• 10.1162/1532443041827934
• Incorporating auxiliary information for improved prediction using combination of kernel machines
• Gradient-based kernel dimension reduction for regression
• ASKIT: an efficient, parallel library for high-dimensional kernel summations
• Restricted kernel canonical correlation analysis
• Tensors in computations
• Kernel methods and machine learning
• Kernel method based on non-linear coherent states in quantum feature space
• Generalized Kalman smoothing: modeling and algorithms
• Functional-bandwidth kernel for support vector machine with functional data: an alternating optimization algorithm
• 10.1162/153244303322533205
• Covariate balancing propensity score by tailored loss functions
• Operator-valued kernels for learning from functional response data
• Continuum versus discrete networks, graph Laplacians, and reproducing kernel Hilbert spaces
• Binary vectors for fast distance and similarity estimation
• On minimum distribution discrepancy support vector machine for domain adaptation
• Kernel machines with missing responses
• Super-exponential decay rates for eigenvalues and singular values of integral operators on the sphere
• Kernels for vector-valued functions: a review
• Title not available (Why is that?)
• Segmentation of brain tissues from magnetic resonance images using adaptively regularized kernel-based fuzzy \(C\)-means clustering
• Learning the kernel matrix with semidefinite programming
• The theory of the quantum kernel-based binary classifier
• A Hilbert Space Embedding for Distributions
• Generalized Mahalanobis depth in the reproducing kernel Hilbert space
• On representing and generating kernels by fuzzy equivalence relations
• A novel Kalman filter formulation for improving tracking performance of the extended kernel RLS
• Selection of time instants and intervals with support vector regression for multivariate functional data
• Committee polyhedral separability: complexity and polynomial approximation
• Learning low-dimensional nonlinear structures from high-dimensional noisy data: an integral operator approach
• Title not available (Why is that?)
• A tutorial on kernel methods for categorization
• Title not available (Why is that?)
• A nested heuristic for parameter tuning in support vector machines
• Preference disaggregation and statistical learning for multicriteria decision support: A review
• An asymptotically minimax kernel machine
• A novel Frank-Wolfe algorithm. Analysis and applications to large-scale SVM training
• Learning with boundary conditions
• Title not available (Why is that?)
• Probability product kernels
• Kernel Methods in Computer Vision
• 10.1162/15324430260185646
• Diffusion kernels on statistical manifolds
• Nonparametric likelihood based estimation of linear filters for point processes
• Some remarks on diffusion distances
• Continuous-variable nonlocality and contextuality
• A regularization approach to the many instruments problem
• A kernel learning framework for domain adaptation learning
• Structural, Syntactic, and Statistical Pattern Recognition
• Regularized learning schemes in feature Banach spaces
• Estimating the support of a high-dimensional distribution
• Framework for kernel regularization with application to protein clustering
• Ensemble Kalman filtering for non-linear likelihood models using kernel-shrinkage regression techniques
• Identifying outliers using multiple kernel canonical correlation analysis with application to imaging genetics
• An efficient multiple kernel learning in reproducing kernel Hilbert spaces (RKHS)
• Kernel techniques: From machine learning to meshless methods
• Title not available (Why is that?)
• SVM kernel by electric network
• Kernel methods: a survey of current techniques
• Title not available (Why is that?)
• Kernel methods in machine learning
• Soft and hard classification by reproducing kernel Hilbert space methods
• Learning Bounds for Kernel Regression Using Effective Data Dimensionality
• \(L^p\) error estimates for approximation by Sobolev splines and Wendland functions on \(\mathbb R^d\)
• Filtering with state-observation examples via kernel Monte Carlo filter
• Choosing shape parameters for regression in reproducing kernel Hilbert space and variable selection
• Applied Koopman theory for partial differential equations and data-driven modeling of spatio-temporal systems
• On the positive semi-definite property of similarity matrices
• An efficient implementation for spatial-temporal Gaussian process regression and its applications
• Function replacement vs. kernel trick
• Distribution-free uncertainty quantification for kernel methods by gradient perturbations
• Eigendecompositions of transfer operators in reproducing kernel Hilbert spaces
• Solving higher order nonlinear ordinary differential equations with least squares support vector machines
• A representer theorem for deep neural networks
• Model selection in kernel ridge regression
• Machine learning with kernels for portfolio valuation and risk management
• Title not available (Why is that?)
• On \(b\)-bit min-wise hashing for large-scale regression and classification with sparse data
• Generalization bounds for regularized portfolio selection with market side information
• Interactions between kernels, frames, and persistent homology
• The Choquet kernel on the use of regression problem
• Integrated kernels and their properties
• Application and comparison of kernel functions for linear parameter varying model approximation of nonlinear systems
• Prediction of magnetization dynamics in a reduced dimensional feature space setting utilizing a low-rank kernel method
• Kernel learning backward SDE filter for data assimilation

Uses Software

• Kernel-Machines
• PDCO
• FastICA

This page was built for publication: Kernel methods in machine learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q930651)