Inference for L₂-boosting

DOI10.1007/S11222-019-09882-0MaRDI QIDQ2302490zbMATH OpenOpenAlexFDO

Authors David Rügamer, Sonja Greven

Publication date 26 February 2020

Published in Statistics and Computing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1805.01852

zbMATH Keywords

bootstrapfunctional gradient descent boostingpost-selection inferenceselective inferenceslow learner

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Nonparametric statistical resampling methods (62G09) Linear regression; mixed models (62J05) Applications of statistics to economics (62P20) Statistics on metric spaces (62R20)

Abstract: We propose a statistical inference framework for the component-wise functional gradient descent algorithm (CFGD) under normality assumption for model errors, also known as $L_2$-Boosting. The CFGD is one of the most versatile tools to analyze data, because it scales well to high-dimensional data sets, allows for a very flexible definition of additive regression models and incorporates inbuilt variable selection. Due to the variable selection, we build on recent proposals for post-selection inference. However, the iterative nature of component-wise boosting, which can repeatedly select the same component to update, necessitates adaptations and extensions to existing approaches. We propose tests and confidence intervals for linear, grouped and penalized additive model components selected by $L_2$-Boosting. Our concepts also transfer to slow-learning algorithms more generally, and to other selection techniques which restrict the response space to more complex sets than polyhedra. We apply our framework to an additive model for sales prices of residential apartments and investigate the properties of our concepts in simulation studies.

Recommendations

• Boosting With theL₂Loss
• Sparse boosting
• Dimension reduction boosting
• Boosting algorithms: regularization, prediction and model fitting
• Boosting for high-dimensional linear models

Cites work

• A new look at the statistical model identification
• An update on statistical boosting in biomedicine
• Asymptotics of selective inference
• Boosting algorithms: regularization, prediction and model fitting
• Boosting With theL₂Loss
• Exact post-selection inference, with application to the Lasso
• Flexible smoothing with B-splines and penalties. With comments and a rejoinder by the authors
• Greedy function approximation: A gradient boosting machine.
• High-dimensional variable selection
• Least angle regression. (With discussion)
• Model-based boosting 2.0
• On the length of post-model-selection confidence intervals conditional on polyhedral constraints
• Selective inference after likelihood- or test-based model selection in linear models
• The functional linear array model
• Uniform asymptotic inference and the bootstrap after model selection
• Valid post-selection inference
• Variable Selection with Error Control: Another Look at Stability Selection

Cited in

(9)

• Selective inference for additive and linear mixed models
• Comments on: ``Inference and computation with generalized additive models and their extensions
• Accelerated Componentwise Gradient Boosting Using Efficient Data Representation and Momentum-Based Optimization
• Estimation and inference of treatment effects with L₂-boosting in high-dimensional settings
• Advances in estimation and inference for historical functional linear models
• Bounded p values in parametric programming-based selective inference
• Additive density-on-scalar regression in Bayes Hilbert spaces with an application to gender economics
• Statistical inference for autoencoder-based anomaly detection after representation learning-based domain adaptation
• Statistical inference for sequential feature selection after domain adaptation

This page was built for publication: Inference for \(L_2\)-boosting

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2302490)