Comment on: Boosting algorithms: regularization, prediction and model fitting

DOI10.1214/07-STS242BzbMATH Open1246.62165arXiv0804.2757MaRDI QIDQ449783FDOQ449783

Authors: David Mease, Abraham J. Wyner, Andreas Buja 

Publication date: 1 September 2012

Published in: Statistical Science (Search for Journal in Brave)

Abstract: The authors are doing the readers of Statistical Science a true service with a well-written and up-to-date overview of boosting that originated with the seminal algorithms of Freund and Schapire. Equally, we are grateful for high-level software that will permit a larger readership to experiment with, or simply apply, boosting-inspired model fitting. The authors show us a world of methodology that illustrates how a fundamental innovation can penetrate every nook and cranny of statistical thinking and practice. They introduce the reader to one particular interpretation of boosting and then give a display of its potential with extensions from classification (where it all started) to least squares, exponential family models, survival analysis, to base-learners other than trees such as smoothing splines, to degrees of freedom and regularization, and to fascinating recent work in model selection. The uninitiated reader will find that the authors did a nice job of presenting a certain coherent and useful interpretation of boosting. The other reader, though, who has watched the business of boosting for a while, may have quibbles with the authors over details of the historic record and, more importantly, over their optimism about the current state of theoretical knowledge. In fact, as much as ``the statistical view has proven fruitful, it has also resulted in some ideas about why boosting works that may be misconceived, and in some recommendations that may be misguided. [arXiv:0804.2752]

Full work available at URL: https://arxiv.org/abs/0804.2757

Recommendations

• Boosting algorithms: regularization, prediction and model fitting
• Rejoinder: Boosting algorithms: regularization, prediction and model fitting
• Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
• Boosting. Foundations and algorithms.
• The boosting approach to machine learning: an overview

Mathematics Subject Classification ID

Estimation in survival analysis and censored data (62N02) Software, source code, etc. for problems pertaining to statistics (62-04) Generalized linear models (logistic models) (62J12)

Cites Work

• A decision-theoretic generalization of on-line learning and an application to boosting
• Greedy function approximation: A gradient boosting machine.
• Random forests
• Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
• Arcing classifiers. (With discussion)
• Boosting With theL₂Loss
• Stochastic gradient boosting.
• Improved boosting algorithms using confidence-rated predictions
• Population theory for boosting ensembles.
• Boosted classification trees and class probability/quantile estimation

Cited In (6)

• Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
• An update on statistical boosting in biomedicine
• Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation
• Rejoinder: Boosting algorithms: regularization, prediction and model fitting
• Feature selection filter for classification of power system operating states
• Boosting algorithms: regularization, prediction and model fitting

Uses Software

• AdaBoost.MH

This page was built for publication: Comment on: Boosting algorithms: regularization, prediction and model fitting

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q449783)