A Large Dimensional Analysis of Least Squares Support Vector Machines

DOI10.1109/TSP.2018.2889954MaRDI QIDQ4628262zbMATH OpenWikidataFDO

Authors Zhenyu Liao, R. Couillet

Publication date 6 March 2019

Published in IEEE Transactions on Signal Processing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1701.02967

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Quadratic programming (90C20) Applications of mathematical programming (90C90) Random matrices (probabilistic aspects) (60B20)

Abstract: In this article, a large dimensional performance analysis of kernel least squares support vector machines (LS-SVMs) is provided under the assumption of a two-class Gaussian mixture model for the input data. Building upon recent advances in random matrix theory, we show, when the dimension of data $p$ and their number $n$ are both large, that the LS-SVM decision function can be well approximated by a normally distributed random variable, the mean and variance of which depend explicitly on a local behavior of the kernel function. This theoretical result is then applied to the MNIST and Fashion-MNIST datasets which, despite their non-Gaussianity, exhibit a convincingly close behavior. Most importantly, our analysis provides a deeper understanding of the mechanism into play in SVM-type methods and in particular of the impact on the choice of the kernel function as well as some of their theoretical limits in separating high dimensional Gaussian vectors.

Recommendations

• scientific article; zbMATH DE number 1843268
• Large-scale linear support vector regression
• A sparse least squares support vector machine
• Efficient computations for large least square support vector machine classifiers
• Benchmarking least squares support vector machine classifiers
• Two improvements for least squares support vector machines

Cited in

(9)

• A matrix-free smoothing algorithm for large-scale support vector machines
• scientific article; zbMATH DE number 7008335 (Why is no real title available?)
• scientific article; zbMATH DE number 2110305 (Why is no real title available?)
• scientific article; zbMATH DE number 1782574 (Why is no real title available?)
• A comparative study on large scale kernelized support vector machines
• A random matrix approach to neural networks
• Massive data discrimination via linear support vector machines
• scientific article; zbMATH DE number 1843268 (Why is no real title available?)
• scientific article; zbMATH DE number 7415098 (Why is no real title available?)

This page was built for publication: A Large Dimensional Analysis of Least Squares Support Vector Machines

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4628262)