Boosting the margin: a new explanation for the effectiveness of voting methods

DOI10.1214/AOS/1024691352zbMath0929.62069OpenAlexW1975846642WikidataQ115720343 ScholiaQ115720343MaRDI QIDQ1807156

Wee Sun Lee, Robert E. Schapire, Yoav Freund, Bartlett, Peter L.

Publication date: 9 November 1999

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1214/aos/1024691352

zbMATH Keywords

neural networksMarkov chain Monte Carlodecision treescodingoutputbaggingboostingensemble methodserror-correcting

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Bagging predictors
• Bounds for the uniform deviation of empirical measures
• An optimal algorithm for multiplication in \(\mathbb{F}_{256}/\mathbb{F}_ 4\)
• A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training
• Rates of convex approximation in non-Hilbert spaces
• A decision-theoretic generalization of on-line learning and an application to boosting
• Arcing classifiers. (With discussion)
• Adaptive game playing using multiplicative weights
• Boosting a weak learning algorithm by majority
• Support-vector networks
• Improved boosting algorithms using confidence-rated predictions
• On the density of families of sets
• 10.1162/15324430152733133
• Universal approximation bounds for superpositions of a sigmoidal function
• Efficient agnostic learning of neural networks with bounded fan-in
• The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network
• The importance of convexity in learning with squared loss
• On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities

Related Items (only showing first 100 items - show all)

Bounding the generalization error of convex combinations of classifiers: Balancing the dimensionality and the margins. ⋮ Generalization error of combined classifiers. ⋮ Generalization bounds for averaged classifiers ⋮ On approximating weighted sums with exponentially many terms ⋮ Population theory for boosting ensembles. ⋮ Process consistency for AdaBoost. ⋮ On the Bayes-risk consistency of regularized boosting methods. ⋮ Statistical behavior and consistency of classification methods based on convex risk minimization. ⋮ Optimal aggregation of classifiers in statistical learning. ⋮ Large margin classification with indefinite similarities ⋮ Learning linear PCA with convex semi-definite programming ⋮ Chagas parasite detection in blood images using AdaBoost ⋮ A MapReduce-based distributed SVM ensemble for scalable image classification and annotation ⋮ Identifying the interacting positions of a protein using Boolean learning and support vector machines ⋮ Classification of gene-expression data: the manifold-based metric learning way ⋮ A precise high-dimensional asymptotic theory for boosting and minimum-\(\ell_1\)-norm interpolated classifiers ⋮ An algorithmic theory of learning: robust concepts and random projection ⋮ An analysis of diversity measures ⋮ Data-driven decomposition for multi-class classification ⋮ Quadratic boosting ⋮ Cost-sensitive boosting algorithms: do we really need them? ⋮ Tutorial series on brain-inspired computing. VI: Geometrical structure of boosting algorithm ⋮ Noise peeling methods to improve boosting algorithms ⋮ Multi-stage classifier design ⋮ \(L_{2}\) boosting in kernel regression ⋮ Pairwise fusion matrix for combining classifiers ⋮ Multi-label optimal margin distribution machine ⋮ Boosting random subspace method ⋮ Deep learning of support vector machines with class probability output networks ⋮ Preference disaggregation and statistical learning for multicriteria decision support: A review ⋮ On hybrid classification using model assisted posterior estimates ⋮ A time-series modeling method based on the boosting gradient-descent theory ⋮ Double-bagging: Combining classifiers by bootstrap aggregation ⋮ On robust classification using projection depth ⋮ A review of boosting methods for imbalanced data classification ⋮ Recursive aggregation of estimators by the mirror descent algorithm with averaging ⋮ Propositionalization and embeddings: two sides of the same coin ⋮ Empirical risk minimization is optimal for the convex aggregation problem ⋮ Risk bounds for CART classifiers under a margin condition ⋮ Further results on the margin explanation of boosting: new algorithm and experiments ⋮ Boosting algorithms: regularization, prediction and model fitting ⋮ Multiclass classification with potential function rules: margin distribution and generalization ⋮ Anytime classification for a pool of instances ⋮ Hierarchical linear support vector machine ⋮ On the equivalence of weak learnability and linear separability: new relaxations and efficient boosting algorithms ⋮ A noise-detection based AdaBoost algorithm for mislabeled data ⋮ Discussion of: ``Nonparametric regression using deep neural networks with ReLU activation function ⋮ Vote counting measures for ensemble classifiers. ⋮ Entropy and divergence associated with power function and the statistical application ⋮ The value of agreement a new boosting algorithm ⋮ Simultaneous adaptation to the margin and to complexity in classification ⋮ Optimal third root asymptotic bounds in the statistical estimation of thresholds ⋮ Optimal rates of aggregation in classification under low noise assumption ⋮ An empirical study of using Rotation Forest to improve regressors ⋮ Boosting conditional probability estimators ⋮ Optimal convergence rate of the universal estimation error ⋮ Analysis of boosting algorithms using the smooth margin function ⋮ From dynamic classifier selection to dynamic ensemble selection ⋮ Boosting and instability for regression trees ⋮ Maximum patterns in datasets ⋮ Using boosting to prune double-bagging ensembles ⋮ An efficient modified boosting method for solving classification problems ⋮ Regularization method for predicting an ordinal response using longitudinal high-dimensional genomic data ⋮ GA-Ensemble: a genetic algorithm for robust ensembles ⋮ Remembering Leo Breiman ⋮ Navigating random forests and related advances in algorithmic modeling ⋮ Supervised projection approach for boosting classifiers ⋮ Feature selection based on loss-margin of nearest neighbor classification ⋮ Computer science and decision theory ⋮ Random survival forests ⋮ Bayesian Weibull tree models for survival analysis of clinico-genomic data ⋮ Boosting GARCH and neural networks for the prediction of heteroskedastic time series ⋮ An algorithmic theory of learning: Robust concepts and random projection ⋮ Cost-sensitive learning and decision making revisited ⋮ An analysis on the relationship between uncertainty and misclassification rate of classifiers ⋮ Bandwidth choice for nonparametric classification ⋮ Deformation of log-likelihood loss function for multiclass boosting ⋮ BoostWofE: a new sequential weights of evidence model reducing the effect of conditional dependency ⋮ A novel margin-based measure for directed hill climbing ensemble pruning ⋮ Iterative feature selection in least square regression estimation ⋮ PAC-Bayesian bounds for randomized empirical risk minimizers ⋮ Arcing classifiers. (With discussion) ⋮ Parallelizing AdaBoost by weights dynamics ⋮ A stochastic approximation view of boosting ⋮ Multi-scale rois selection for classifying multi-spectral images ⋮ Negative correlation in incremental learning ⋮ A \(\mathbb R\)eal generalization of discrete AdaBoost ⋮ On generalization performance and non-convex optimization of extended \(\nu \)-support vector machine ⋮ Interpretable machine learning: fundamental principles and 10 grand challenges ⋮ Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors) ⋮ On weak base hypotheses and their implications for boosting regression and classification ⋮ AdaBoost and robust one-bit compressed sensing ⋮ On the perceptron's compression ⋮ A geometric approach to leveraging weak learners ⋮ A re-weighting strategy for improving margins ⋮ Random projections as regularizers: learning a linear discriminant from fewer observations than dimensions ⋮ Local discriminative distance metrics ensemble learning ⋮ Iterative Bayes ⋮ Top-down decision tree learning as information based boosting ⋮ Accurate tree-based missing data imputation and data fusion within the statistical learning paradigm

Uses Software

• UCI-ml
• AdaBoost.MH
• C4.5

This page was built for publication: Boosting the margin: a new explanation for the effectiveness of voting methods