Boosting. Foundations and algorithms.
Suppose we have access to a learning algorithm that produces weak classifiers, i.e., classifiers that perform slightly better than random guessing, with high probability. Can we combine these classifiers with the help of data such that with high probability the resulting classifier performs arbitrarily well? This fundamental question in machine learning was first raised by M.~Kearns and G.~Valiant in 1994 when working in the probably approximately correct (PAC) learning framework. The authors of this book, who won the Gödel Prize for their work in 2003, gave a positive answer by inventing the first boosting algorithms around 1996. This book, which is the first solely focussed on this topic, provides a comprehensive introduction to boosting algorithms.NEWLINENEWLINEAfter a gentle introduction to machine learning in general and boosting in particular, the book is devided into 4 parts, where the first two parts contain the core material on boosting and the third and fourth part discuss more advanced material and extensions.NEWLINENEWLINEIn the first part, the basic statistical analysis is presented. To this end, the authors provide an introduction to uniform deviation bounds, VC-dimension, and compression bounds in Chapter 2. The subsequent chapter considers AdaBoosts training error, and the fourth chapter combines these results to establish generalization error bounds for AdaBoost. In addition, it shows the equivalence between weak and strong learnability. In Chapter 5, the authors present their preferred view on boosting as a margin maximization scheme.NEWLINENEWLINEThe second part contains three very nice chapters presenting rather different interpretations of boosting. The relationship of AdaBoost to game theory and online learning is explored in Chapter 6, while Chapter 7 shows that AdaBoost is greedily minimizing a particular empirical risk. In the third and last chapter of this part, the authors then interpret AdaBoost as an iterative projection algorithm and prove some of its convergence properties.NEWLINENEWLINEIn the third part of the book, extensions of AdaBoost to learning scenarios more coomplicated than binary classification are presented. The first such extension that uses confidence-rated weak learners is introduced in Chapter 9. Multiclass classification is considered in Chapter 10, and the final Chapter 11 introduces a boosting algorithm for the ranking problem.NEWLINENEWLINEIn the fourth and final part of the book some advanced material on boosting is presented. The highlight of this part is probably Chapter 12, which presents a complete proof of Adaboost's universal consistency. Chapter 13 revisits game theoretic aspects of boosting, and the last chapter introduces a continuous time modification of AdaBoost.NEWLINENEWLINEThis book is certainly much welcomed in the machine learning community as it presents the first comprehensive introduction to boosting and its theory. Its gentle style and the extensive number of interesting exercises at the end of each chapter makes it serve well as a textbook, in particular for advanced graduate courses in computer science, while the bibliographic notes nicely link to the existing literature on boosting. The mathematically oriented reader may however be warned that the book's style is not as rigorous as in mathematical textbooks, since the book obviously targets more towards computer scientists. Moreover, the author's preference towards the margin maximization view of boosting seems to rely more on philosophical reasoning than on compelling arguments. Nonetheless, the book is undoubtedly an excellent introduction to boosting.
- scientific article; zbMATH DE number 2086343
- Boosting algorithms: regularization, prediction and model fitting
- Stochastic boosting algorithms
- Stochastic boosting algorithms
- Some theory for generalized boosting algorithms
- The boosting approach to machine learning: an overview
- scientific article; zbMATH DE number 1931845
- scientific article; zbMATH DE number 1944955
- scientific article; zbMATH DE number 2089370
- A decision-theoretic generalization of on-line learning and an application to boosting
- Two-step sparse boosting for high-dimensional longitudinal data with varying coefficients
- An update on statistical boosting in biomedicine
- A robust AdaBoost.RT based ensemble extreme learning machine
- High-dimensional time series prediction using kernel-based koopman mode regression
- Gaussian-Gamma collaborative filtering: a hierarchical Bayesian model for recommender systems
- Population theory for boosting ensembles.
- Multiclass classification, information, divergence and surrogate risk
- Calibrating AdaBoost for phoneme classification
- Learning causal effect using machine learning with application to China's typhoon
- A likelihood-based boosting algorithm for factor analysis models with binary data
- Handling concept drift via model reuse
- Fast greedy \(\mathcal{C} \)-bound minimization with guarantees
- Grafting for combinatorial binary model using frequent itemset mining
- Feature extraction using conformal geometric algebra for AdaBoost algorithm based in-plane rotated face detection
- Ensemble quantile classifier
- Estimating a sharp convergence bound for randomized ensembles
- Boosting as a kernel-based method
- Ensembles of cost-diverse Bayesian neural learners for imbalanced binary classification
- scientific article; zbMATH DE number 1717315 (Why is no real title available?)
- Performance of empirical risk minimization in linear aggregation
- Labeled compression schemes for extremal classes
- Ensemble machine learning. Methods and applications
- scientific article; zbMATH DE number 2086343 (Why is no real title available?)
- Some theory for generalized boosting algorithms
- scientific article; zbMATH DE number 1804118 (Why is no real title available?)
- Component-wise AdaBoost algorithms for high-dimensional binary classification and class probability prediction
- Entropy Regularized LPBoost
- scientific article; zbMATH DE number 67633 (Why is no real title available?)
- Two oracle inequalities for regularized boosting classifiers
- scientific article; zbMATH DE number 1950578 (Why is no real title available?)
- Boosting algorithms: regularization, prediction and model fitting
- Comment on: Boosting algorithms: regularization, prediction and model fitting
- On parameterized complexity of the hitting set problem for axis-parallel squares intersecting a straight line
- Boosted KZ and LLL Algorithms
- Transport analysis of infinitely deep neural network
- Scheme of boosting in the problems of combinatorial optimization induced by the collective training algorithms
- scientific article; zbMATH DE number 1884340 (Why is no real title available?)
- Information Geometry of U-Boost and Bregman Divergence
- Pruning variable selection ensembles
- scientific article; zbMATH DE number 7625186 (Why is no real title available?)
- Toward Efficient Ensemble Learning with Structure Constraints: Convergent Algorithms and Applications
- Boosting Random Forests to Reduce Bias; One-Step Boosted Forest and Its Variance Estimate
- Optimization of tree ensembles
- A data-driven random subfeature ensemble learning algorithm for weather forecasting
- Multiclass boosting: margins, codewords, losses, and algorithms
- Explaining AdaBoost
- The boosting approach to machine learning: an overview
- A primal-dual convergence analysis of boosting
- A theory of multiclass boosting
- Learning Theory
- Multi-class \(\ell_2\)-Boost with the scoring coding
- On the existence of linear weak learners and applications to boosting
- New two-level ensemble method and its application to chemical compounds properties prediction
- Joint leaf-refinement and ensemble pruning through L₁ regularization
- Explainable subgradient tree boosting for prescriptive analytics in operations management
- The VC-dimension of a class of multiples of the primes and a connection to AdaBoost
- Synthetic learner: model-free inference on treatments over time
- New two-level machine learning method for evaluating the real characteristics of objects
- Adversarial bandits with knapsacks
- Boosting simple learners
- Reprint of: Some thoughts about transfer learning. What role for the source domain?
- The vanishing learning rate asymptotic for linear L^2-boosting
- Split Regularized Regression
- Estimating the algorithmic variance of randomized ensembles via the bootstrap
- Improving interpretability using surrogate models: an ecological case study
- Boosting causal additive models
- Smooth Nash equilibria: algorithms and complexity
- Bit-security preserving hardness amplification
- Multi-model subset selection
- Empirical investigations of boosting with pseudo-outcome imputation for missing responses
- Characterizing the implicit bias via a primal-dual analysis
- Online boosting with bandit feedback
- Sample compression for real-valued learners
- Boosting Prediction with Data Missing Not at Random
- A support vector machine-based ensemble algorithm for breast cancer diagnosis
- Soft-max boosting
- Tutorial series on brain-inspired computing. VI: Geometrical structure of boosting algorithm
- Committee polyhedral separability: complexity and polynomial approximation
- Deep learning of support vector machines with class probability output networks
This page was built for publication: Boosting. Foundations and algorithms.
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2893108)
