Blessing of dimensionality: mathematical foundations of the statistical physics of data
From MaRDI portal
Publication:5154201
DOI10.1098/RSTA.2017.0237zbMATH Open1470.82004arXiv1801.03421OpenAlexW3099661174WikidataQ52647993 ScholiaQ52647993MaRDI QIDQ5154201FDOQ5154201
Alexander N. Gorban, I. Tyukin
Publication date: 4 October 2021
Published in: Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences (Search for Journal in Brave)
Abstract: The concentration of measure phenomena were discovered as the mathematical background of statistical mechanics at the end of the XIX - beginning of the XX century and were then explored in mathematics of the XX-XXI centuries. At the beginning of the XXI century, it became clear that the proper utilisation of these phenomena in machine learning might transform the curse of dimensionality into the blessing of dimensionality. This paper summarises recently discovered phenomena of measure concentration which drastically simplify some machine learning problems in high dimension, and allow us to correct legacy artificial intelligence systems. The classical concentration of measure theorems state that i.i.d. random points are concentrated in a thin layer near a surface (a sphere or equators of a sphere, an average or median level set of energy or another Lipschitz function, etc.). The new stochastic separation theorems describe the thin structure of these thin layers: the random points are not only concentrated in a thin layer but are all linearly separable from the rest of the set, even for exponentially large random sets. The linear functionals for separation of points can be selected in the form of the linear Fisher's discriminant. All artificial intelligence systems make errors. Non-destructive correction requires separation of the situations (samples) with errors from the samples corresponding to correct behaviour by a simple and robust classifier. The stochastic separation theorems provide us by such classifiers and a non-iterative (one-shot) procedure for learning.
Full work available at URL: https://arxiv.org/abs/1801.03421
Recommendations
statisticsartificial intelligencelinear separabilityextreme pointsstatistical physicsFisher's discriminantapplied mathematicsensemble equivalencemeasure concentration
Cites Work
- Learning deep architectures for AI
- Pattern classification.
- Extensions of Lipschitz mappings into a Hilbert space
- An elementary proof of a theorem of Johnson and Lindenstrauss
- Title not available (Why is that?)
- Probability Inequalities for Sums of Bounded Random Variables
- On the mathematical foundations of learning
- The concentration of measure phenomenon
- Title not available (Why is that?)
- Title not available (Why is that?)
- Title not available (Why is that?)
- Concentration of measure and isoperimetric inequalities in product spaces
- Title not available (Why is that?)
- Title not available (Why is that?)
- Generic Hamiltonian dynamical systems are neither integrable nor ergodic
- A simplified neuron model as a principal component analyzer
- Title not available (Why is that?)
- Clustering. A data recovery approach.
- Training a Support Vector Machine in the Primal
- Measure-preserving homeomorphisms and metrical transitivity
- Is the \(k\)-NN classifier in high dimensions affected by the curse of dimensionality?
- Data Mining
- Concentration property on probability spaces.
- Statistical mechanics of learning
- David Hilbert and the axiomatization of physics (1894--1905)
- Title not available (Why is that?)
- Approximation with random bases: pro et contra
- Principal manifolds for data visualization and dimension reduction. Reviews and original papers presented partially at the workshop `Principal manifolds for data cartography and dimension reduction', Leicester, UK, August 24--26, 2006.
- Oded Schramm 1961--2008
- Probabilistic lower bounds for approximation by shallow perceptron networks
- Title not available (Why is that?)
- Quasiorthogonal dimension of Euclidean spaces
- Stochastic separation theorems
- Title not available (Why is that?)
- One-trial correction of legacy AI systems and stochastic separation theorems
- Piece-wise quadratic approximations of arbitrary error functions for fast and robust machine learning
- High-dimensional brain: a tool for encoding and rapid learning of memories by single neurons
- Data complexity measured by principal graphs
Cited In (18)
- Fast construction of correcting ensembles for legacy artificial intelligence systems: algorithms and a case study
- Hilbert’s sixth problem: the endless road to rigour
- Revisiting `survival of the fittest' principle in global stochastic optimisation: incorporating anisotropic mutations
- Global optimisation in Hilbert spaces using the survival of the fittest algorithm
- General stochastic separation theorems with optimal bounds
- Coping with AI errors with provable guarantees
- Generalised Watson distribution on the hypersphere with applications to clustering
- The independent component analysis with the linear regression – predicting the energy costs of the public sector buildings in Croatia
- Ignorance is a bliss: Mathematical structure of many-box models
- On a posteriori error estimation using distances between numerical solutions and angles between truncation errors
- Blessing of dimensionality at the edge and geometry of few-shot learning
- Replica analysis of Bayesian data clustering
- Correction of AI systems by linear discriminants: probabilistic foundations
- Approximation of classifiers by deep perceptron networks
- High-dimensional brain: a tool for encoding and rapid learning of memories by single neurons
- Modelling biological evolution: developing novel approaches
- On a posteriori estimation of the approximation error norm for an ensemble of independent solutions
- Tensor train based isogeometric analysis for PDE approximation on parameter dependent geometries
This page was built for publication: Blessing of dimensionality: mathematical foundations of the statistical physics of data
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5154201)
