Asymptotic normality of support vector machine variants and other regularized kernel methods

DOI10.1016/J.JMVA.2011.11.004zbMATH Open1352.62056arXiv1010.0535OpenAlexW1964319030MaRDI QIDQ765834FDOQ765834

Authors: Robert Hable 

Publication date: 22 March 2012

Published in: Journal of Multivariate Analysis (Search for Journal in Brave)

Abstract: In nonparametric classification and regression problems, regularized kernel methods, in particular support vector machines, attract much attention in theoretical and in applied statistics. In an abstract sense, regularized kernel methods (simply called SVMs here) can be seen as regularized M-estimators for a parameter in a (typically infinite dimensional) reproducing kernel Hilbert space. For smooth loss functions, it is shown that the difference between the estimator, i.e. the empirical SVM, and the theoretical SVM is asymptotically normal with rate $sqrtn$. That is, the standardized difference converges weakly to a Gaussian process in the reproducing kernel Hilbert space. As common in real applications, the choice of the regularization parameter may depend on the data. The proof is done by an application of the functional delta-method and by showing that the SVM-functional is suitably Hadamard-differentiable.

Full work available at URL: https://arxiv.org/abs/1010.0535

Recommendations

• Consistency of Support Vector Machines and Other Regularized Kernel Classifiers
• Consistency and convergence rates of one-class SVMs and related algorithms
• Statistical performance of support vector machines
• Consistency of support vector machines using additive kernels for additive models
• Some properties of regularized kernel methods

zbMATH Keywords

asymptotic normalitynonparametric regressionsupport vector machinefunctional delta-methodHadamard-differentiabilityregularized kernel method

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Asymptotic properties of nonparametric inference (62G20) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Cites Work

• Weak convergence and empirical processes. With applications to statistics
• Asymptotic Statistics
• Title not available (Why is that?)
• Support Vector Machines
• Title not available (Why is that?)
• Real Analysis and Probability
• Title not available (Why is that?)
• Title not available (Why is that?)
• Consistency and robustness of kernel-based regression in convex risk minimization
• On robustness properties of convex risk minimization methods for pattern recognition
• Title not available (Why is that?)
• Optimal rates for the regularized least-squares algorithm
• An Introduction to Banach Space Theory
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• A Bahadur representation of the linear support vector machine
• Title not available (Why is that?)
• Optimal aggregation of classifiers in statistical learning.
• Title not available (Why is that?)
• Measure and integration theory. Transl. from the German by Robert B. Burckel
• Capacity of reproducing kernel spaces in learning theory
• Fast rates for support vector machines using Gaussian kernels
• Statistical performance of support vector machines
• A new concentration result for regularized risk minimizers
• Title not available (Why is that?)
• On qualitative robustness of support vector machines
• Regularization in kernel learning
• Smooth \(\epsilon \)-insensitive regression by loss symmetrization
• Bouligand derivatives and robustness of support vector machines for regression
• Any Discrimination Rule Can Have an Arbitrarily Bad Probability of Error for Finite Sample Size
• Title not available (Why is that?)
• Title not available (Why is that?)

Cited In (7)

• Model selection in kernel ridge regression
• Non-asymptotic Analysis of $\ell_1$-norm Support Vector Machines
• On asymptotic properties of hyperparameter estimators for kernel-based regularization methods
• Asymptotic linear expansion of regularized M-estimators
• Deterministic error analysis of support vector regression and related regularized kernel methods
• On conditional risk estimation considering model risk
• Testing subspace restrictions in the presence of high dimensional nuisance parameters

This page was built for publication: Asymptotic normality of support vector machine variants and other regularized kernel methods

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q765834)