Boosting the margin: a new explanation for the effectiveness of voting methods

DOI10.1214/aos/1024691352zbMath0929.62069OpenAlexW1975846642WikidataQ115720343 ScholiaQ115720343MaRDI QIDQ1807156

Wee Sun Lee, Robert E. Schapire, Yoav Freund, Bartlett, Peter L.

Publication date: 9 November 1999

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1214/aos/1024691352

zbMATH Keywords

neural networks Markov chain Monte Carlo decision trees coding output bagging boosting ensemble methods error-correcting

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Related Items

Bounding the generalization error of convex combinations of classifiers: Balancing the dimensionality and the margins., Generalization error of combined classifiers., Generalization bounds for averaged classifiers, On approximating weighted sums with exponentially many terms, Population theory for boosting ensembles., Process consistency for AdaBoost., On the Bayes-risk consistency of regularized boosting methods., Statistical behavior and consistency of classification methods based on convex risk minimization., Optimal aggregation of classifiers in statistical learning., Large margin classification with indefinite similarities, Learning linear PCA with convex semi-definite programming, Chagas parasite detection in blood images using AdaBoost, A MapReduce-based distributed SVM ensemble for scalable image classification and annotation, Identifying the interacting positions of a protein using Boolean learning and support vector machines, Classification of gene-expression data: the manifold-based metric learning way, A precise high-dimensional asymptotic theory for boosting and minimum-\(\ell_1\)-norm interpolated classifiers, An algorithmic theory of learning: robust concepts and random projection, An analysis of diversity measures, Data-driven decomposition for multi-class classification, Quadratic boosting, Cost-sensitive boosting algorithms: do we really need them?, Tutorial series on brain-inspired computing. VI: Geometrical structure of boosting algorithm, Noise peeling methods to improve boosting algorithms, Multi-stage classifier design, \(L_{2}\) boosting in kernel regression, Pairwise fusion matrix for combining classifiers, Multi-label optimal margin distribution machine, Boosting random subspace method, Deep learning of support vector machines with class probability output networks, Preference disaggregation and statistical learning for multicriteria decision support: A review, On hybrid classification using model assisted posterior estimates, A time-series modeling method based on the boosting gradient-descent theory, Double-bagging: Combining classifiers by bootstrap aggregation, On robust classification using projection depth, A review of boosting methods for imbalanced data classification, Recursive aggregation of estimators by the mirror descent algorithm with averaging, Propositionalization and embeddings: two sides of the same coin, Empirical risk minimization is optimal for the convex aggregation problem, Risk bounds for CART classifiers under a margin condition, Further results on the margin explanation of boosting: new algorithm and experiments, Boosting algorithms: regularization, prediction and model fitting, Multiclass classification with potential function rules: margin distribution and generalization, Anytime classification for a pool of instances, Hierarchical linear support vector machine, On the equivalence of weak learnability and linear separability: new relaxations and efficient boosting algorithms, A noise-detection based AdaBoost algorithm for mislabeled data, Discussion of: ``Nonparametric regression using deep neural networks with ReLU activation function, Vote counting measures for ensemble classifiers., Entropy and divergence associated with power function and the statistical application, The value of agreement a new boosting algorithm, Simultaneous adaptation to the margin and to complexity in classification, Optimal third root asymptotic bounds in the statistical estimation of thresholds, Optimal rates of aggregation in classification under low noise assumption, An empirical study of using Rotation Forest to improve regressors, Boosting conditional probability estimators, Optimal convergence rate of the universal estimation error, Analysis of boosting algorithms using the smooth margin function, From dynamic classifier selection to dynamic ensemble selection, Boosting and instability for regression trees, Maximum patterns in datasets, Using boosting to prune double-bagging ensembles, An efficient modified boosting method for solving classification problems, Regularization method for predicting an ordinal response using longitudinal high-dimensional genomic data, GA-Ensemble: a genetic algorithm for robust ensembles, Remembering Leo Breiman, Navigating random forests and related advances in algorithmic modeling, Supervised projection approach for boosting classifiers, Feature selection based on loss-margin of nearest neighbor classification, Computer science and decision theory, Random survival forests, Bayesian Weibull tree models for survival analysis of clinico-genomic data, Boosting GARCH and neural networks for the prediction of heteroskedastic time series, An algorithmic theory of learning: Robust concepts and random projection, Cost-sensitive learning and decision making revisited, An analysis on the relationship between uncertainty and misclassification rate of classifiers, Bandwidth choice for nonparametric classification, Deformation of log-likelihood loss function for multiclass boosting, BoostWofE: a new sequential weights of evidence model reducing the effect of conditional dependency, A novel margin-based measure for directed hill climbing ensemble pruning, Iterative feature selection in least square regression estimation, PAC-Bayesian bounds for randomized empirical risk minimizers, Arcing classifiers. (With discussion), Parallelizing AdaBoost by weights dynamics, A stochastic approximation view of boosting, Multi-scale rois selection for classifying multi-spectral images, Negative correlation in incremental learning, A \(\mathbb R\)eal generalization of discrete AdaBoost, On generalization performance and non-convex optimization of extended \(\nu \)-support vector machine, Interpretable machine learning: fundamental principles and 10 grand challenges, Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors), On weak base hypotheses and their implications for boosting regression and classification, AdaBoost and robust one-bit compressed sensing, On the perceptron's compression, A geometric approach to leveraging weak learners, A re-weighting strategy for improving margins, Random projections as regularizers: learning a linear discriminant from fewer observations than dimensions, Local discriminative distance metrics ensemble learning, Iterative Bayes, Top-down decision tree learning as information based boosting, Accurate tree-based missing data imputation and data fusion within the statistical learning paradigm, Deep learning: a statistical viewpoint, Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation, Measuring Impact of Diversity of Classifiers on the Accuracy of Evidential Ensemble Classifiers, An Integrated Fuzzy Cells-Classifier, Reduction from Cost-Sensitive Ordinal Ranking to Weighted Binary Classification, Boosting with Noisy Data: Some Views from Statistical Theory, Different Paradigms for Choosing Sequential Reweighting Algorithms, Properties of Bagged Nearest Neighbour Classifiers, Unnamed Item, A novel margin based algorithm for feature extraction, Ten More Years of Error Rate Research, Information Geometry of U-Boost and Bregman Divergence, Analysis of the generalization ability of a full decision tree, Discriminative Reranking for Natural Language Parsing, Parallel orthogonal deep neural network, Space-dependent turbulence model aggregation using machine learning, Large Scale Prediction with Decision Trees, Nested cross-validation with ensemble feature selection and classification model for high-dimensional biological data, Model and method for constructing a heterogeneous cluster ensemble, Weighted bagging: a modification of AdaBoost from the perspective of importance sampling, Structure from Randomness in Halfspace Learning with the Zero-One Loss, Attractor Networks for Shape Recognition, Boosting with missing predictors, Unnamed Item, Automated trading with boosting and expert weighting, Nonparametric Modeling of Neural Point Processes via Stochastic Gradient Boosting Regression, Aggregating classifiers with ordinal response structure, Selection of Binary Variables and Classification by Boosting, Robust Loss Functions for Boosting, An iterated classification rule based on auxiliary pseudo-predictors., Improving nonparametric regression methods by bagging and boosting., Prototype Classification: Insights from Machine Learning, Recent developments in bootstrap methodology, Comment, Theory of Classification: a Survey of Some Recent Advances, Theory and Algorithm for Learning with Dissimilarity Functions, Novel Aggregate Deletion/Substitution/Addition Learning Algorithms for Recursive Partitioning, Unnamed Item, Unnamed Item, Complexities of convex combinations and bounding the generalization error in classification, Boosting with early stopping: convergence and consistency, Superlinear Integrality Gaps for the Minimum Majority Problem

Uses Software

Cites Work