Boosting algorithms: regularization, prediction and model fitting
From MaRDI portal
Abstract: We present a statistical perspective on boosting. Special emphasis is given to estimating potentially complex parametric or nonparametric models, including generalized linear and additive models as well as regression models for survival analysis. Concepts of degrees of freedom and corresponding Akaike or Bayesian information criteria, particularly useful for regularization and variable selection in high-dimensional covariate spaces, are discussed as well. The practical aspects of boosting procedures for fitting statistical models are illustrated by means of the dedicated open-source software package mboost. This package implements functions which can be used for model fitting, prediction and variable selection. It is flexible, allowing for the implementation of new boosting algorithms optimizing user-specified loss functions.
Recommendations
- Comment: Boosting algorithms: regularization, prediction and model fitting
- Rejoinder: Boosting algorithms: regularization, prediction and model fitting
- Comment on: Boosting algorithms: regularization, prediction and model fitting
- Boosting methods for regression
- scientific article; zbMATH DE number 2086343
- Boosting. Foundations and algorithms.
- Stochastic boosting algorithms
- Stochastic boosting algorithms
- The boosting approach to machine learning: an overview
- Robust boosting for regression problems
Cites work
- 10.1162/1532443041424319
- 10.1162/153244304773936108
- A decision-theoretic generalization of on-line learning and an application to boosting
- A multivariate FGD technique to improve VaR computation in equity markets
- A new approach to variable selection in least squares problems
- AdaBoost is consistent
- Adaptive Lasso for sparse high-dimensional regression models
- Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
- Aggregating classifiers with ordinal response structure
- Arcing classifiers. (With discussion)
- Bagging predictors
- Better Subset Regression Using the Nonnegative Garrote
- Boosted classification trees and class probability/quantile estimation
- BoosTexter: A boosting-based system for text categorization
- Boosting algorithms: with an application to bootstrapping multivariate time series
- Boosting for high-dimensional linear models
- Boosting ridge regression
- Boosting the margin: a new explanation for the effectiveness of voting methods
- Boosting with early stopping: convergence and consistency
- Boosting With theL2Loss
- Convergence Rates of General Regularization Methods for Statistical Inverse Problems and Applications
- Convexity, Classification, and Risk Bounds
- Cryptographic limitations on learning Boolean formulae and finite automata
- ElemStatLearn
- Empirical margin distributions and bounding the generalization error of combined classifiers
- Generalized Additive Modeling with Implicit Variable Selection by Likelihood‐Based Boosting
- Generalized additive models
- Generalized monotonic regression based on B-splines with an application to air pollution data
- Greedy function approximation: A gradient boosting machine.
- High-dimensional graphs and variable selection with the Lasso
- scientific article; zbMATH DE number 5957408 (Why is no real title available?)
- scientific article; zbMATH DE number 5957506 (Why is no real title available?)
- scientific article; zbMATH DE number 3123424 (Why is no real title available?)
- scientific article; zbMATH DE number 47282 (Why is no real title available?)
- scientific article; zbMATH DE number 3637090 (Why is no real title available?)
- scientific article; zbMATH DE number 700016 (Why is no real title available?)
- scientific article; zbMATH DE number 1950578 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 5056247 (Why is no real title available?)
- Knot selection by boosting techniques
- Least angle regression. (With discussion)
- Looking for lumps: boosting and bagging for density estimation.
- Matching pursuits with time-frequency dictionaries
- Model Selection and the Principle of Minimum Description Length
- Multi-class AdaBoost
- On boosting kernel regression
- On early stopping in gradient descent learning
- On the Bayes-risk consistency of regularized boosting methods.
- Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
- Process consistency for AdaBoost.
- Random forests
- Smoothing Parameter Selection in Nonparametric Regression Using an Improved Akaike Information Criterion
- Soft margins for AdaBoost
- Sparse boosting
- Survival ensembles
- The Adaptive Lasso and Its Oracle Properties
- The boosting approach to machine learning: an overview
- The elements of statistical learning. Data mining, inference, and prediction
- Unified methods for censored longitudinal data and causality
- Weak greedy algorithms
Cited in
(only showing first 100 items - show all)- Boosting ridge regression
- Some challenges for statistics
- High-dimensional additive modeling
- Model-based boosting in R: a hands-on tutorial using the R package mboost
- Generalized additive models with flexible response functions
- Two-step sparse boosting for high-dimensional longitudinal data with varying coefficients
- Early stopping for statistical inverse problems via truncated SVD estimation
- Sequential double cross-validation for assessment of added predictive ability in high-dimensional omic applications
- Penalized likelihood and Bayesian function selection in regression models
- Variable selection in general multinomial logit models
- Boosting techniques for nonlinear time series models
- Probing for sparse and fast variable selection with model-based boosting
- An update on statistical boosting in biomedicine
- Boosting flexible functional regression models with a high number of functional historical effects
- Gradient boosting for distributional regression: faster tuning and improved variable selection via noncyclical updates
- Forecasting with many predictors: is boosting a viable alternative?
- Semiparametric regression during 2003--2007
- The reliability of classification of terminal nodes in GUIDE decision tree to predict the nonalcoholic fatty liver disease
- Determining cutoff point of ensemble trees based on sample size in predicting clinical dose with DNA microarray data
- Boosting high dimensional predictive regressions with time varying parameters
- Boosted nonparametric hazards with time-dependent covariates
- Asymptotic linear expansion of regularized M-estimators
- Adaptive step-length selection in gradient boosting for Gaussian location and scale models
- A review on instance ranking problems in statistical learning
- A precise high-dimensional asymptotic theory for boosting and minimum-\(\ell_1\)-norm interpolated classifiers
- Generalised joint regression for count data: a penalty extension for competitive settings
- Invariance, causality and robustness
- Ridge estimation for multinomial logit models with symmetric side constraints
- Variable selection and model choice in structured survival models
- Multinomial logit models with implicit variable selection
- Entropy-based estimation in classification problems
- Transformation boosting machines
- Inference for L₂-boosting
- Sparse kernel deep stacking networks
- Boosting as a kernel-based method
- A penalty approach to differential item functioning in Rasch models
- Accelerated gradient boosting
- Empirical risk minimization is optimal for the convex aggregation problem
- Boosting iterative stochastic ensemble method for nonlinear calibration of subsurface flow models
- A unified framework of constrained regression
- Wavelet-based gradient boosting
- Semiparametric regression and graphical models
- Boosting. Foundations and algorithms.
- Model-based boosting 2.0
- Boosting multi-state models
- Regression trees for predicting mortality in patients with cardiovascular disease: what improvement is achieved by using ensemble-based methods?
- Bayesian variable selection and estimation in semiparametric joint models of multivariate longitudinal and survival data
- Use of pretransformation to cope with extreme values in important candidate features
- Sparse boosting
- Three categories customer churn prediction based on the adjusted real AdaBoost
- Feature selection filter for classification of power system operating states
- An empirical comparison of classification algorithms for mortgage default prediction: evidence from a distressed mortgage market
- Rank-based estimation in the \(\ell_1\)-regularized partly linear model for censored outcomes with application to integrated analyses of clinical predictors and gene expression data
- Boosting with missing predictors
- Predicting the Whole Distribution with Methods for Depth Data Analysis Demonstrated on a Colorectal Cancer Treatment Study
- PBoostGA: pseudo-boosting genetic algorithm for variable ranking and selection
- Generalized Additive Models for Pair-Copula Constructions
- Mean and quantile boosting for partially linear additive models
- Improved nearest neighbor classifiers by weighting and selection of predictors
- Variable Selection and Model Choice in Geoadditive Regression Models
- A simple extension of boosting for asymmetric mislabeled data
- Geoadditive expectile regression
- scientific article; zbMATH DE number 1944955 (Why is no real title available?)
- Comment on: Boosting algorithms: regularization, prediction and model fitting
- Rejoinder: Boosting algorithms: regularization, prediction and model fitting
- Functional gradient ascent for probit regression
- Detection of differential item functioning in Rasch models by boosting techniques
- Delta Boosting Machine with Application to General Insurance
- CAM: causal additive models, high-dimensional order search and penalized regression
- Spike-and-slab priors for function selection in structured additive regression models
- Beyond mean regression
- The functional linear array model
- Subject-specific Bradley–Terry–Luce models with implicit variable selection
- Dimension reduction boosting
- Survival Regression with Accelerated Failure Time Model in XGBoost
- Toward Efficient Ensemble Learning with Structure Constraints: Convergent Algorithms and Applications
- A Tree-Based Semi-Varying Coefficient Model for the COM-Poisson Distribution
- Boosting beyond the mean -- extending component-wise gradient boosting algorithms to multiple dimensions
- Boosting functional response models for location, scale and shape with an application to bacterial competition
- Machine learning based on extended generalized linear model applied in mixture experiments
- Discriminant analyses of peanut allergy severity scores
- RandGA: injecting randomness into parallel genetic algorithm for variable selection
- Generalized additive models with unknown link function including variable selection
- A general framework for functional regression modelling
- Comparison and contrast of two general functional regression modelling frameworks
- Boosting for statistical modelling-A non-technical introduction
- Predicting matches in international football tournaments with random forests
- Functional martingale residual process for high-dimensional Cox regression with model averaging
- Boosting multiplicative model combination
- Automatic model selection for high-dimensional survival analysis
- Regularized proportional odds models
- Nonparametric multiple expectile regression via ER-Boost
- Random gradient boosting for predicting conditional quantiles
- EEBoost: a general method for prediction and variable selection based on estimating equations
- General sparse boosting: improving feature selection of \(L_{2}\) boosting by correlation-based penalty family
- Generalized Additive Modeling with Implicit Variable Selection by Likelihood‐Based Boosting
- Boosting algorithms: with an application to bootstrapping multivariate time series
- An overview of techniques for linking high-dimensional molecular data to time-to-event endpoints by risk prediction models
- Stochastic approximation boosting for incomplete data problems
- Optimization by Gradient Boosting
Describes a project that uses
Uses Software
This page was built for publication: Boosting algorithms: regularization, prediction and model fitting
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q449780)
