Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)

DOI10.1214/aos/1016218223zbMath1106.62323OpenAlexW2024046085WikidataQ93494458 ScholiaQ93494458MaRDI QIDQ1848780

Robert Tibshirani, Trevor Hastie, Jerome H. Friedman

Publication date: 14 November 2002

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://projecteuclid.org/euclid.aos/1016218223

zbMATH Keywords

classification tree machine learning nonparametric estimation stagewise fitting

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Related Items

Two-step sparse boosting for high-dimensional longitudinal data with varying coefficients, Learning ELM-tree from big data based on uncertainty reduction, Robust variable selection with exponential squared loss for the spatial autoregressive model, Cost-sensitive ensemble learning: a unifying framework, On the accuracy of cross-validation in the classification problem, Representation in the (artificial) immune system, Finding causative genes from high-dimensional data: an appraisal of statistical and machine learning approaches, Some relationships between fuzzy and random set-based classifiers and models, An empirical comparison of classification algorithms for mortgage default prediction: evidence from a distressed mortgage market, Logistic regression using covariates obtained by product-unit neural network models, Cost-sensitive boosting algorithms: do we really need them?, Tutorial series on brain-inspired computing. VI: Geometrical structure of boosting algorithm, Mean and quantile boosting for partially linear additive models, Improved nearest neighbor classifiers by weighting and selection of predictors, Logitboost autoregressive networks, Gradient boosting for high-dimensional prediction of rare events, Improved customer choice predictions using ensemble methods, \(L_{2}\) boosting in kernel regression, Self-improved gaps almost everywhere for the agnostic approximation of monomials, A Fisher consistent multiclass loss function with variable margin on positive examples, Small area estimation of the homeless in Los Angeles: an application of cost-sensitive stochastic gradient boosting, On hybrid classification using model assisted posterior estimates, Blasso for object categorization and retrieval: towards interpretable visual models, Complexity in the case against accuracy estimation, A simple extension of boosting for asymmetric mislabeled data, Further results on the margin explanation of boosting: new algorithm and experiments, Survival ensembles by the sum of pairwise differences with application to lung cancer microarray studies, Kullback-Leibler aggregation and misspecified generalized linear models, Boosting algorithms: regularization, prediction and model fitting, Comment on: Boosting algorithms: regularization, prediction and model fitting, Rejoinder: Boosting algorithms: regularization, prediction and model fitting, A boosting method for maximization of the area under the ROC curve, Representing and recognizing objects with massive local image patches, A boosting approach for supervised Mahalanobis distance metric learning, Functional gradient ascent for probit regression, A noise-detection based AdaBoost algorithm for mislabeled data, Fully corrective boosting with arbitrary loss and regularization, Vote counting measures for ensemble classifiers., On a method for constructing ensembles of regression models, Entropy and divergence associated with power function and the statistical application, Sparse regression and support recovery with \(\mathbb{L}_2\)-boosting algorithms, Boosting multi-features with prior knowledge for mini unmanned helicopter landmark detection, A fast genetic method for inducting descriptive fuzzy models., Statistical modeling: The two cultures. (With comments and a rejoinder)., A concrete statistical realization of Kleinberg's stochastic dicrimination for pattern recognition. I: Two-class classification, Does modeling lead to more accurate classification? A study of relative efficiency in linear classification, A sharp nonasymptotic bound and phase diagram of \(L_{1/2}\) regularization, Unsupervised weight parameter estimation method for ensemble learning, Cox process functional learning, Boosting conditional probability estimators, Counting and enumerating aggregate classifiers, Soft memberships for spectral clustering, with application to permeable language distinction, An extensive comparison of recent classification tools applied to microarray data, Boosting and instability for regression trees, Boosting additive models using component-wise P-splines, Using boosting to prune double-bagging ensembles, Additive prediction and boosting for functional data, Boosting nonlinear additive autoregressive time series, Taxonomy for characterizing ensemble methods in classification tasks: a review and annotated bibliography, Greedy function approximation: A gradient boosting machine., Non-crossing large-margin probability estimation and its application to robust SVM via pre\-condi\-tion\-ing, A cascade of boosted generative and discriminative classifiers for vehicle detection, Model-based boosting in R: a hands-on tutorial using the R package mboost, Remembering Leo Breiman, Remembrance of Leo Breiman, Node harvest, Quadratic Majorization for Nonconvex Loss with Applications to the Boosting Algorithm, Robust boosting with truncated loss functions, Navigating random forests and related advances in algorithmic modeling, Boosting GARCH and neural networks for the prediction of heteroskedastic time series, Nearly unbiased variable selection under minimax concave penalty, Cost-sensitive boosting for classification of imbalanced data, Functional dissipation microarrays for classification, Estimating the dimension of a model, New multicategory boosting algorithms based on multicategory Fisher-consistent losses, Tree-structured modelling of categorical predictors in generalized additive regression, A new hybrid classification algorithm for customer churn prediction based on logistic regression and decision trees, Boosting in Cox regression: a comparison between the likelihood-based and the model-based approaches with focus on the R-packages \textit{CoxBoost} and \textit{mboost}, Optimal prediction pools, Robust exponential squared loss-based variable selection for high-dimensional single-index varying-coefficient model, Obtaining linguistic fuzzy rule-based regression models from imprecise data with multiobjective genetic algorithms, Ensemble classification based on generalized additive models, Sketching information divergences, Boosted Bayesian network classifiers, Exact bootstrap \(k\)-nearest neighbor learners, Surrogate maximization/minimization algorithms and extensions, A dynamic model of expected bond returns: A functional gradient descent approach, A weight-adjusted voting algorithm for ensembles of classifiers, Iterative bias reduction: a comparative study, Soft-max boosting, Parallelizing AdaBoost by weights dynamics, Boosting ridge regression, A stochastic approximation view of boosting, A local boosting algorithm for solving classification problems, Logitboost with errors-in-variables, A \(\mathbb R\)eal generalization of discrete AdaBoost, On boosting kernel regression, Embedding ensemble tracking in a stochastic framework for robust object tracking, A geometric approach to leveraging weak learners, A conversation with Jerry Friedman, Interpreting uninterpretable predictors: kernel methods, Shtarkov solutions, and random forests, AdaBoost Semiparametric Model Averaging Prediction for Multiple Categories, A Statistical Approach to Crime Linkage, Tweedie gradient boosting for extremely unbalanced zero-inflated data, Using Differentiable Programming for Flexible Statistical Modeling, Generalized Additive Modeling with Implicit Variable Selection by Likelihood‐Based Boosting, Survival Regression with Accelerated Failure Time Model in XGBoost, Variable selection by ensembles for the Cox model, Toward Efficient Ensemble Learning with Structure Constraints: Convergent Algorithms and Applications, Unnamed Item, Evolution of the Viola-Jones Object Detection Method: A Survey, Robustifying AdaBoost by Adding the Naive Error Rate, Boosting with Noisy Data: Some Views from Statistical Theory, Different Paradigms for Choosing Sequential Reweighting Algorithms, Regularized Bayesian quantile regression, Unnamed Item, Information Geometry of U-Boost and Bregman Divergence, Machine learning based on extended generalized linear model applied in mixture experiments, A Unified Classification Model Based on Robust Optimization, Ensemble of fast learning stochastic gradient boosting, A General Alternating-Direction Implicit Framework with Gaussian Process Regression Parameter Prediction for Large Sparse Linear Systems, Estimation and Validation of Ratio-based Conditional Average Treatment Effects Using Observational Data, Robust penalized empirical likelihood in high dimensional longitudinal data analysis, Fuzzy OWL-Boost: learning fuzzy concept inclusions via real-valued boosting, Estimating propensity scores using neural networks and traditional methods: a comparative simulation study, Robust Boosting Algorithm Against Mislabeling in Multiclass Problems, Comparison of various machine learning algorithms for estimating generalized propensity score, Response versus gradient boosting trees, GLMs and neural networks under Tweedie loss and log-link, Multiclass Boosting Algorithms for Shrinkage Estimators of Class Probability, Weighted bagging: a modification of AdaBoost from the perspective of importance sampling, Robust estimation in partially linear regression models, Evolution of high-frequency systematic trading: a performance-driven gradient boosting model, Can a corporate network and news sentiment improve portfolio optimization using the Black–Litterman model?, Boosting Method for Local Learning in Statistical Pattern Recognition, Adaptive index models for marker-based risk stratification, Boosting method for nonlinear transformation models with censored survival data, Boosting with missing predictors, The synergy between PAV and AdaBoost, Model-based transductive learning of the kernel matrix, Statistical Learning With Time Series Dependence: An Application to Scoring Sleep in Mice, Unnamed Item, Automated trading with boosting and expert weighting, The synergy between PAV and AdaBoost, Embedding black-box regression techniques into hierarchical Bayesian models, Model-based transductive learning of the kernel matrix, Robust Variable Selection With Exponential Squared Loss, TESTS OF THE MARTINGALE DIFFERENCE HYPOTHESIS USING BOOSTING AND RBF NEURAL NETWORK APPROXIMATIONS, Nonparametric Modeling of Neural Point Processes via Stochastic Gradient Boosting Regression, Aggregating classifiers with ordinal response structure, Selection of Binary Variables and Classification by Boosting, Robust Loss Functions for Boosting, Logistic model trees, Stochastic gradient boosting., Looking for lumps: boosting and bagging for density estimation., Improving nonparametric regression methods by bagging and boosting., Generalized additive multi-mixture model for data mining., Delta Boosting Machine with Application to General Insurance, Logistic model trees, Listwise approaches based on feature ranking discovery, Stochastic boosting algorithms, New Bootstrap Applications in Supervised Learning, Boosted Regression Trees with Errors in Variables, Theory of Classification: a Survey of Some Recent Advances, Remembering Leo, Boosting in the Presence of Outliers: Adaptive Classification With Nonconvex Loss Functions, A Hybrid Approach of Boosting Against Noisy Data, Distributed Decision‐Tree Induction in Peer‐to‐Peer Systems, Prediction of transfers to tertiary care and hospital mortality: A gradient boosting decision tree approach, Composite large margin classifiers with latent subclasses for heterogeneous biomedical data, Skills in demand for ICT and statistical occupations: Evidence from web‐based job vacancies, Randomized Gradient Boosting Machine, Subject-specific Bradley–Terry–Luce models with implicit variable selection, Automatic model selection for high-dimensional survival analysis, Nonparametric multiple expectile regression via ER-Boost, Random gradient boosting for predicting conditional quantiles, New aspects of Bregman divergence in regression and classification with parametric and nonparametric estimation, Dimension reduction boosting, A Generic Path Algorithm for Regularized Statistical Estimation, A fast algorithm for the accelerated failure time model with high-dimensional time-to-event data, Variable Selection and Model Choice in Geoadditive Regression Models, Experiments with AdaBoost.RT, an Improved Boosting Scheme for Regression, Boosting in the presence of noise, Some Statistical Problems with High Dimensional Financial data, Confidence sets with expected sizes for Multiclass Classification, Detection of differential item functioning in Rasch models by boosting techniques, OR Practice–Data Analytics for Optimal Detection of Metastatic Prostate Cancer, SIMILARITY-BASED COMBINATION OF MULTIPLE CLUSTERINGS, Stochastic boosting algorithms, Large scale analysis of generalization error in learning using margin based classification methods, Unnamed Item, Appropriate machine learning techniques for credit scoring and bankruptcy prediction in banking and finance: A comparative study, Large dimensional analysis of general margin based classification methods, AN EMPIRICAL STUDY OF BOOSTED NEURAL NETWORK FOR PARTICLE CLASSIFICATION IN HIGH ENERGY COLLISIONS, Regularized Estimation in the Accelerated Failure Time Model with High-Dimensional Covariates, COST-SENSITIVE MULTI-CLASS ADABOOST FOR UNDERSTANDING DRIVING BEHAVIOR BASED ON TELEMATICS, Personal ID Image Normalization Using ISO/IEC 19794-5 Standards for Facial Recognition Improvement, General Sparse Boosting: Improving Feature Selection of L₂Boosting by Correlation-Based Penalty Family, Optimization by Gradient Boosting, On the Effect and Remedies of Shrinkage on Classification Probability Estimation, Some accelerated alternating proximal gradient algorithms for a class of nonconvex nonsmooth problems, Simplex-based Multinomial Logistic Regression with Diverging Numbers of Categories and Covariates, Fully corrective gradient boosting with squared hinge: fast learning rates and early stopping, A nonlinear kernel SVM Classifier via \(L_{0/1}\) soft-margin loss with classification performance, Fast convergence rates of deep neural networks for classification, Boosting Distributional Copula Regression, A hybrid generalized propensity score approach for observational studies, Gradient boosting with extreme-value theory for wildfire prediction, A systematic literature review on the use of machine learning in code clone research, A boosting first-hitting-time model for survival analysis in high-dimensional settings, Notes on the H-measure of classifier performance, Tests and classification methods in adaptive designs with applications, Does the default pecking order impact systemic risk? Evidence from Brazilian data, Infinitesimal gradient boosting, Merging components in linear Gaussian cluster-weighted models, Estimation and inference of treatment effects with \(L_2\)-boosting in high-dimensional settings, Data-driven state-of-charge prediction of a storage cell using ABC/GBRT, ABC/MLP and Lasso machine learning techniques, \(\alpha\)QBoost: an iteratively weighted adiabatic trained classifier, Robust estimation of heterogeneous treatment effects: an algorithm-based approach, STATISTICALLY VALIDATED LEAD-LAG NETWORKS AND INVENTORY PREDICTION IN THE FOREIGN EXCHANGE MARKET, Least angle regression. (With discussion), Population theory for boosting ensembles., Process consistency for AdaBoost., On the Bayes-risk consistency of regularized boosting methods., Statistical behavior and consistency of classification methods based on convex risk minimization., Nonparametric Decomposition of Time Series Data with Inputs, Multi-class boosting with asymmetric binary weak-learners, Logistic evolutionary product-unit neural networks: Innovation capacity of poor Guatemalan households, Boosting the partial least square algorithm for regression modelling, A precise high-dimensional asymptotic theory for boosting and minimum-\(\ell_1\)-norm interpolated classifiers, Density estimation with stagewise optimization of the empirical risk, Multi-class learning by smoothed boosting, Quadratic boosting, Regression trees for predicting mortality in patients with cardiovascular disease: What improvement is achieved by using ensemble-based methods?, Noise peeling methods to improve boosting algorithms, Combining biomarkers to optimize patient treatment recommendations, Probing for sparse and fast variable selection with model-based boosting, An update on statistical boosting in biomedicine, Probability estimation for multi-class classification using adaboost, Multilayer bootstrap networks, Multilogistic regression by means of evolutionary product-unit neural networks, Discriminative Reranking for Natural Language Parsing, Goal scoring, coherent loss and applications to machine learning, Quantitative convergence analysis of kernel based large-margin unified machines, Accelerated gradient boosting, Using LogitBoost classifier to predict protein structural classes, Modeling threshold interaction effects through the logistic classification trunk, Local uncertainty sampling for large-scale multiclass logistic regression, Forecasting with many predictors: is boosting a viable alternative?, Recursive aggregation of estimators by the mirror descent algorithm with averaging, Gradient boosting for distributional regression: faster tuning and improved variable selection via noncyclical updates, Pathway-based kernel boosting for the analysis of genome-wide association studies, Uncertainty and forecasts of U.S. recessions, Semiparametric regression during 2003--2007, Random classification noise defeats all convex potential boosters, An empirical comparison of learning algorithms for nonparametric scoring: the \textsc{TreeRank} algorithm and other methods, A probabilistic classifier ensemble weighting scheme based on cross-validated accuracy estimates, Boosting iterative stochastic ensemble method for nonlinear calibration of subsurface flow models, An incremental aggregated proximal ADMM for linearly constrained nonconvex optimization with application to sparse logistic regression problems, Covariate balancing propensity score by tailored loss functions, Bootstrap -- an exploration, Simultaneous adaptation to the margin and to complexity in classification, Application of “Aggregated Classifiers” in Survival Time Studies, Instance-dependent cost-sensitive learning for detecting transfer fraud, Optimal rates of aggregation in classification under low noise assumption, An empirical study of using Rotation Forest to improve regressors, A comparative study of the leading machine learning techniques and two new optimization algorithms, Analysis of boosting algorithms using the smooth margin function, Extending models via gradient boosting: an application to Mendelian models, Sample size determination for logistic regression, An efficient modified boosting method for solving classification problems, Boosting-based sequential output prediction, Calibrating AdaBoost for phoneme classification, A combination selection algorithm on forecasting, Multinomial logit models with implicit variable selection, GA-Ensemble: a genetic algorithm for robust ensembles, Prediction and classification in nonlinear data analysis: something old, something new, something borrowed, something blue, Big data analytics for seismic fracture identification using amplitude-based statistics, Robust estimation for the varying coefficient partially nonlinear models, Supervised projection approach for boosting classifiers, Automatic face detection in video sequences using local normalization and optimal adaptive correlation techniques, Boosting for high-dimensional linear models, Automatic gait classification patterns in spastic hemiplegia, Cost-sensitive learning and decision making revisited, Learning causal effect using machine learning with application to China's typhoon, Local fractal and multifractal features for volumic texture characterization, Discriminative deep belief networks for visual data classification, Bandwidth choice for nonparametric classification, Gender discriminating models from facial surface normals, Deformation of log-likelihood loss function for multiclass boosting, Statistical Monitoring of Nominal Logistic Profiles in Phase II, Robust estimation and empirical likelihood inference with exponential squared loss for panel data models, BoostWofE: a new sequential weights of evidence model reducing the effect of conditional dependency, Multicategory large margin classification methods: hinge losses vs. coherence functions, Isotonic boosting classification rules, Stochastic approximation: from statistical origin to big-data, multidisciplinary applications, An improved multiclass LogitBoost using adaptive-one-vs-one, Milp-hyperbox classification for structure-based drug design in the discovery of small molecule inhibitors of SIRTUIN6, Boosting high dimensional predictive regressions with time varying parameters, Designing a Boosted Classifier on Riemannian Manifolds, Data Reduction Using a Discrete Wavelet Transform in Discriminant Analysis of Very High Dimensionality Data, AN ASYMMETRIC ADAPTIVE CLASSIFICATION METHOD, Transformation boosting machines, SVM-boosting based on Markov resampling: theory and algorithm, Toward an explainable machine learning model for claim frequency: a use case in car insurance pricing with telematics data, On the interpretation of ensemble classifiers in terms of Bayes classifiers, Three Categories Customer Churn Prediction Based on the Adjusted Real Adaboost, Stratified Normalization LogitBoost for Two-Class Unbalanced Data Classification, A likelihood-based boosting algorithm for factor analysis models with binary data, An Extension of the Receiver Operating Characteristic Curve and AUC-Optimal Classification, Boosting as a kernel-based method, On weak base hypotheses and their implications for boosting regression and classification, Adaptive step-length selection in gradient boosting for Gaussian location and scale models, AdaBoost and robust one-bit compressed sensing, Boosting with early stopping: convergence and consistency, Machine learning for corporate default risk: multi-period prediction, frailty correlation, loan portfolios, and tail probabilities, A new accelerated proximal boosting machine with convergence rate \(O(1/t^2)\), Robust MAVE for single-index varying-coefficient models, Probability estimation with machine learning methods for dichotomous and multicategory outcome: Theory, Top-down decision tree learning as information based boosting

Uses Software

AdaBoost.MH

Cites Work