Adaptive step-length selection in gradient boosting for Gaussian location and scale models
From MaRDI portal
Publication:2095757
Abstract: Tuning of model-based boosting algorithms relies mainly on the number of iterations, while the step-length is fixed at a predefined value. For complex models with several predictors such as Generalized Additive Models for Location, Scale and Shape (GAMLSS), imbalanced updates of predictors, where some distribution parameters are updated more frequently than others, can be a problem that prevents some submodels to be appropriately fitted within a limited number of boosting iterations. We propose an approach using adaptive step-length (ASL) determination within a non-cyclical boosting algorithm for GAMLSS to prevent such imbalance. Moreover, for the important special case of the Gaussian distribution, we discuss properties of the ASL and derive a semi-analytical form of the ASL that avoids manual selection of the search interval and numerical optimization to find the optimal step-length, and consequently improves computational efficiency. We show competitive behavior of the proposed approaches compared to penalized maximum likelihood and boosting with a fixed step-length for GAMLSS models in two simulations and two applications, in particular for cases of large variance and/or more variables than observations. In addition, the idea of the ASL is also applicable to other models with more than one predictor like zero-inflated count model, and brings up insights into the choice of the reasonable defaults for the step-length in simpler special case of (Gaussian) additive models.
Recommendations
- Gradient boosting for distributional regression: faster tuning and improved variable selection via noncyclical updates
- Boosting beyond the mean -- extending component-wise gradient boosting algorithms to multiple dimensions
- Generalized Additive Modeling with Implicit Variable Selection by Likelihood‐Based Boosting
- Robust boosting for regression problems
- Stochastic gradient boosting.
Cites work
- Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
- Bayesian smoothing and regression for longitudinal, spatial and event history data.
- Boosting algorithms: regularization, prediction and model fitting
- Boosting for high-dimensional linear models
- Boosting with early stopping: convergence and consistency
- Boosting With theL2Loss
- Forward stagewise regression and the monotone lasso
- Generalized Additive Models for Location, Scale and Shape
- Gradient boosting for distributional regression: faster tuning and improved variable selection via noncyclical updates
- Greedy function approximation: A gradient boosting machine.
- High-dimensional inference: confidence intervals, \(p\)-values and R-software \texttt{hdi}
- scientific article; zbMATH DE number 1742537 (Why is no real title available?)
- Model-based boosting 2.0
- Model-based boosting in R: a hands-on tutorial using the R package mboost
- SELECTING AMONGST LARGE CLASSES OF MODELS
- Stability selection. With discussion and authors' reply
Cited in
(6)- Gradient boosting for distributional regression: faster tuning and improved variable selection via noncyclical updates
- Boosting Distributional Copula Regression
- Boosting multivariate structured additive distributional regression models
- Gradient boosting for Dirichlet regression models
- A balanced statistical boosting approach for GAMLSS via new step lengths
- Scalable Estimation for Structured Additive Distributional Regression
This page was built for publication: Adaptive step-length selection in gradient boosting for Gaussian location and scale models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2095757)
