Theory of Classification: a Survey of Some Recent Advances

DOI10.1051/PS:2005018zbMATH Open1136.62355OpenAlexW2014902932WikidataQ58374465 ScholiaQ58374465MaRDI QIDQ3373749FDOQ3373749

Authors: Stéphane Boucheron, Olivier Bousquet, Gábor Lugosi 

Publication date: 9 March 2006

Published in: ESAIM: Probability and Statistics (Search for Journal in Brave)

Full work available at URL: http://www.numdam.org/item?id=PS_2005__9__323_0

Recommendations

• Learning pattern classification-a survey
• scientific article; zbMATH DE number 893887
• scientific article; zbMATH DE number 3923896
• A Bayesian Analysis of Recent Developments in Pattern Classification
• Pattern Recognition

zbMATH Keywords

model selectionempirical processesconcentration inequalitiesstatistical learning theoryPattern recognition

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Pattern recognition, speech recognition (68T10)

Cites Work

• A decision-theoretic generalization of on-line learning and an application to boosting
• The elements of statistical learning. Data mining, inference, and prediction
• Title not available (Why is that?)
• Weak convergence and empirical processes. With applications to statistics
• Title not available (Why is that?)
• Ideal spatial adaptation by wavelet shrinkage
• Some Comments on C P
• A new look at the statistical model identification
• Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
• Bootstrap methods: another look at the jackknife
• Support-vector networks
• Regularization networks and support vector machines
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• On nonparametric estimation of density level sets
• A note on margin-based loss functions in classification
• Boosting the margin: a new explanation for the effectiveness of voting methods
• Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
• Pattern classification.
• An introduction to support vector machines and other kernel-based learning methods.
• Title not available (Why is that?)
• Title not available (Why is that?)
• Probability Inequalities for Sums of Bounded Random Variables
• A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines
• Convergence of stochastic processes
• New concentration inequalities in product spaces
• Distribution-free inequalities for the deleted and holdout error estimates
• 10.1162/153244302760200704
• Some special Vapnik-Chervonenkis classes
• On the mathematical foundations of learning
• Title not available (Why is that?)
• Title not available (Why is that?)
• On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
• Risk bounds for model selection via penalization
• Arcing classifiers. (With discussion)
• A combinatorial problem; stability and order for models and theories in infinitary languages
• On the density of families of sets
• Title not available (Why is that?)
• Boosting With theL₂Loss
• Title not available (Why is that?)
• Sharper bounds for Gaussian and empirical processes
• Model selection in nonparametric regression
• Smooth discrimination analysis
• Model selection for regression on a fixed design
• A Bennett concentration inequality and its application to suprema of empirical processes
• About the constants in Talagrand's concentration inequalities for empirical processes.
• Statistical behavior and consistency of classification methods based on convex risk minimization.
• Measuring mass concentrations and estimating density contour clusters -- An excess mass approach
• Local Rademacher complexities and oracle inequalities in risk minimization. (2004 IMS Medallion Lecture). (With discussions and rejoinder)
• Local Rademacher complexities
• On the influence of the kernel on the consistency of support vector machines
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Uniform Central Limit Theorems
• Title not available (Why is that?)
• Title not available (Why is that?)
• Improving the sample complexity using global data
• Title not available (Why is that?)
• Minimum contrast estimators on sieves: Exponential bounds and rates of convergence
• Une inégalité de Bennett pour les maxima de processus empiriques. (A Bennet type inequality for maxima of empirical processes)
• Minimum complexity density estimation
• Title not available (Why is that?)
• Some applications of concentration inequalities to statistics
• Concentration inequalities and model selection. Ecole d'Eté de Probabilités de Saint-Flour XXXIII -- 2003.
• Empirical margin distributions and bounding the generalization error of combined classifiers
• Theoretical foundations of the potential function method in pattern recognition learning
• Learnability and the Vapnik-Chervonenkis dimension
• The Generic Chaining
• 10.1162/153244303321897690
• Enumeration of Seven-Argument Threshold Functions
• Convexity, Classification, and Risk Bounds
• Concentration inequalities using the entropy method
• Bounding \(\bar d\)-distance by informational divergence: A method to prove measure concentration
• Probability Inequalities for the Sum of Independent Random Variables
• A simple proof of the blowing-up lemma (Corresp.)
• Structural risk minimization over data-dependent hierarchies
• Title not available (Why is that?)
• Some limit theorems for empirical processes (with discussion)
• Exponential inequalities for sums of random vectors
• Boosting a weak learning algorithm by majority
• Concentration of measure and isoperimetric inequalities in product spaces
• Weighted sums of certain dependent random variables
• Advances in large-margin classifiers
• Title not available (Why is that?)
• Introduction to nonparametric estimation
• Large sample optimality of least squares cross-validation in density estimation
• Estimation of dependences based on empirical data. Transl. from the Russian by Samuel Kotz
• Central limit theorems for empirical measures
• Sphere packing numbers for subsets of the Boolean \(n\)-cube with bounded Vapnik-Chervonenkis dimension
• Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable bandwidth selectors
• Erratum to: ``A measure concentration inequality for contracting Markov chains
• Moment inequalities for functions of independent random variables
• Support vector machines are universally consistent
• Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilitiés de Saint-Flour XXXI -- 2001.
• Optimal aggregation of classifiers in statistical learning.
• Title not available (Why is that?)
• Title not available (Why is that?)
• Minimax nonparametric classification. II. Model selection for adaptation
• Minimax nonparametric classification .I. Rates of convergence
• Rademacher penalties and structural risk minimization
• The importance of convexity in learning with squared loss
• Neural Network Learning
• Logistic regression, AdaBoost and Bregman distances
• A general lower bound on the number of examples needed for learning
• The consistency of the BIC Markov order estimator.
• Rates of convergence for minimum contrast estimators
• Title not available (Why is that?)
• Consistency of Support Vector Machines and Other Regularized Kernel Classifiers
• Capacity of reproducing kernel spaces in learning theory
• Learning Theory
• On the trace of finite sets
• A new look at independence
• Statistical performance of support vector machines
• Probably Approximate Learning of Sets and Functions
• Majorizing measures: The generic chaining
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Uniform and universal Glivenko-Cantelli classes
• Estimating a regression function
• Title not available (Why is that?)
• Title not available (Why is that?)
• The densest hemisphere problem
• Concentration for Independent Permutations
• Title not available (Why is that?)
• Balls in \(\mathbb{R}^k\) do not cut all subsets of \(k+2\) points
• Title not available (Why is that?)
• 10.1162/1532443041827925
• An alternative point of view on Lepski's method
• PAC-Bayesian Generalisation Error Bounds for Gaussian Process Classification
• The Glivenko-Cantelli problem
• Risk bounds for statistical learning
• Square root penalty: Adaption to the margin in classification and in edge estimation
• Large-scale typicality of Markov sample paths and consistency of MDL order estimators
• Density and dimension
• On the Bayes-risk consistency of regularized boosting methods.
• Model selection and error estimation
• Universal Donsker classes and metric entropy
• Poincaré's inequalities and Talagrand's concentration phenomenon for the exponential distribution
• PAC-Bayesian stochastic model selection
• On Talagrand's deviation inequalities for product measures
• Title not available (Why is that?)
• A sharp concentration inequality with applications
• Concentration inequalities for set-indexed empirical processes
• Entropy and the combinatorial dimension
• Complexity regularization via localized random penalties
• Generalization bounds for averaged classifiers
• Distribution-free performance bounds for potential function rules
• Title not available (Why is that?)
• Scale-sensitive dimensions, uniform convergence, and learnability
• 10.1162/153244303321897681
• Concept learning using complexity regularization
• Population theory for boosting ensembles.
• Process consistency for AdaBoost.
• Decision theoretic generalizations of the PAC model for neural net and other learning applications
• Title not available (Why is that?)
• Hardness results for neural network approximation problems
• Bounding the Vapnik-Chervonenkis dimension of concept classes parameterized by real numbers
• Learning Theory
• 10.1162/1532443041424319
• A result of Vapnik with applications
• Title not available (Why is that?)
• Learning pattern classification-a survey
• Polynomial bounds for VC dimension of sigmoidal and general Pfaffian neural networks
• Information inequalities and concentration of measure
• Exponential Bounds for Large Deviations
• A new approach to least-squares estimation, with applications
• Title not available (Why is that?)
• Combinatorics of random processes and sections of convex bodies
• Necessary and Sufficient Conditions for the Uniform Convergence of Means to their Expectations
• 10.1162/153244302760200650
• The Glivenko-Cantelli problem, ten years later
• Improved bounds on the sample complexity of learning
• The probability problem of pattern recognition learning and the method of potential functions
• The method of potential functions for the problem of restoring the characteristic of a function converter from randomly observed points
• Potential function algorithms for pattern recognition learning machines
• Density estimation via exponential model selection
• Predicting \(\{ 0,1\}\)-functions on randomly drawn points
• Finiteness results for sigmoidal “neural” networks
• Neural networks with quadratic VC dimension
• Strongly consistent code-based identification and order estimation for constrained finite-state model classes
• Symmetrization approach to concentration inequalities for empirical processes.
• A note on Talagrand's concentration inequality
• Correction to bounds on conditional probabilities with applications
• Title not available (Why is that?)
• A graph-theoretic generalization of the Sauer-Shelah lemma
• 10.1162/153244303768966111
• Adaptive model selection using empirical complexities
• Concentration for locally acting permutations
• Strong minimax lower bounds for learning
• Some extensions of an inequality of Vapnik and Chervonenkis
• On the convexified Sauer-Shelah theorem
• Behavioral and prescriptive explanations of a reverse sunk cost effect
• Title not available (Why is that?)
• A finite sample distribution-free performance bound for local discrimination rules
• A Remark on the Szarek–Talagrand Theorem
• Automatic pattern recognition: a study of the probability of error
• Vapnik-Chervonenkis type conditions and uniform Donsker classes of functions
• Existence of submatrices with all possible columns
• On the infeasibility of training neural networks with small mean-squared error
• Empirical processes and applications: An overview. (With discussion)
• Title not available (Why is that?)
• 10.1162/153244303765208368
• Title not available (Why is that?)
• Combining PAC-Bayesian and generic chaining bounds
• On the importance of small coordinate projections

Cited In (only showing first 100 items - show all)

• Constructing processes with prescribed mixing coefficients
• Reducing mechanism design to algorithm design via machine learning
• Classification with reject option
• Testing piecewise functions
• Instability, complexity, and evolution
• Relative deviation learning bounds and generalization with unbounded loss functions
• Adaptive estimation of the optimal ROC curve and a bipartite ranking algorithm
• Learning noisy linear classifiers via adaptive and selective sampling
• Fast learning rates in statistical inference through aggregation
• Sampling and empirical risk minimization
• Robustness and generalization
• Efficiency of classification methods based on empirical risk minimization
• Supervised Learning by Support Vector Machines
• Convergence conditions for the observed mean method in stochastic programming
• A high-dimensional Wilks phenomenon
• Local nearest neighbour classification with applications to semi-supervised learning
• Risk bounds for CART classifiers under a margin condition
• Some properties of Gaussian reproducing kernel Hilbert spaces and their implications for function approximation and learning theory
• Statistical learning from biased training samples
• Fast learning rates for plug-in classifiers
• PAC-Bayesian high dimensional bipartite ranking
• Bayesian approach, theory of empirical risk minimization. Comparative analysis
• PAC-Bayesian bounds for randomized empirical risk minimizers
• Multi-kernel regularized classifiers
• Generalized mirror averaging and \(D\)-convex aggregation
• Robust \(k\)-means clustering for distributions with two moments
• Stability and minimax optimality of tangential Delaunay complexes for manifold reconstruction
• Cover-based combinatorial bounds on probability of overfitting
• Strong \(L^p\) convergence of wavelet deconvolution density estimators
• Upper bounds and aggregation in bipartite ranking
• On regularization algorithms in learning theory
• Ranking and empirical minimization of \(U\)-statistics
• Obtaining fast error rates in nonconvex situations
• Neural network approximation
• Approaches to supervised learning for pattern recognition
• Classification algorithms using adaptive partitioning
• On signal representations within the Bayes decision framework
• A partial overview of the theory of statistics with functional data
• Reference point logistic classification
• Empirical risk minimization for heavy-tailed losses
• Kullback-Leibler aggregation and misspecified generalized linear models
• Adaptive kernel methods using the balancing principle
• Pattern classification as interpolation in N dimensions
• Minimax fast rates for discriminant analysis with errors in variables
• Classification with minimax fast rates for classes of Bayes rules with sparse representation
• Plugin procedure in segmentation and application to hyperspectral image segmentation
• Optimal survey schemes for stochastic gradient descent with applications to \(M\)-estimation
• Simultaneous adaptation to the margin and to complexity in classification
• Permutational Rademacher Complexity
• A survey of cross-validation procedures for model selection
• Classification in general finite dimensional spaces with the \(k\)-nearest neighbor rule
• A statistical view of clustering performance through the theory of \(U\)-processes
• Agnostic active learning
• Optimal rates of aggregation in classification under low noise assumption
• Learning by mirror averaging
• Properties of convergence of a fuzzy set estimator of the density function
• Kernel methods in machine learning
• Adaptive partitioning schemes for bipartite ranking
• Statistical active learning algorithms for noise tolerance and differential privacy
• Variance-based regularization with convex objectives
• Optimal weighted nearest neighbour classifiers
• Overlaying classifiers: A practical approach to optimal scoring
• Group invariance, stability to deformations, and complexity of deep convolutional representations
• Model selection by bootstrap penalization for classification
• On the kernel rule for function classification
• Combinatorial bounds of overfitting for threshold classifiers
• Title not available (Why is that?)
• Title not available (Why is that?)
• Depth separations in neural networks: what is actually being separated?
• An empirical classification procedure for nonparametric mixture models
• Classification with many classes: challenges and pluses
• Concentration inequalities for two-sample rank processes with application to bipartite ranking
• Hold-out estimates of prediction models for Markov processes
• Measuring the capacity of sets of functions in the analysis of ERM
• For interpolating kernel machines, minimizing the norm of the ERM solution maximizes stability
• Nonparametric classification with missing data
• Title not available (Why is that?)
• Consistency of learning algorithms using Attouch-Wets convergence
• Sharpness estimation of combinatorial generalization ability bounds for threshold decision rules
• Maxisets for model selection
• Robust statistical learning with Lipschitz and convex loss functions
• Nearest neighbor empirical processes
• The coreness and H-index of random geometric graphs
• Guest editorial: Learning theory
• Cross-validation on extreme regions
• Stochastic difference-of-convex-functions algorithms for nonconvex programming
• Robust classification via MOM minimization
• How can we identify the sparsity structure pattern of high-dimensional data: an elementary statistical analysis to interpretable machine learning
• Optimal functional supervised classification with separation condition
• Title not available (Why is that?)
• SVRG meets AdaGrad: painless variance reduction
• Title not available (Why is that?)
• Bandwidth selection in kernel empirical risk minimization via the gradient
• Structure from randomness in halfspace learning with the zero-one loss
• Learning bounds for quantum circuits in the agnostic setting
• Learning sets with separating kernels
• Mathematical methods of randomized machine learning
• PACMAN: PAC-style bounds accounting for the mismatch between accuracy and negative log-loss
• When are epsilon-nets small?
• Statistical analysis of Mapper for stochastic and multivariate filters

Uses Software

• AdaBoost.MH

This page was built for publication: Theory of Classification: a Survey of Some Recent Advances

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3373749)