Boosting for high-dimensional linear models
From MaRDI portal
Publication:2497175
Abstract: We prove that boosting with the squared error loss, Boosting, is consistent for very high-dimensional linear models, where the number of predictor variables is allowed to grow essentially as fast as (exp(sample size)), assuming that the true underlying regression function is sparse in terms of the -norm of the regression coefficients. In the language of signal processing, this means consistency for de-noising using a strongly overcomplete dictionary if the underlying signal is sparse in terms of the -norm. We also propose here an -based method for tuning, namely for choosing the number of boosting iterations. This makes Boosting computationally attractive since it is not required to run the algorithm multiple times for cross-validation as commonly used so far. We demonstrate Boosting for simulated data, in particular where the predictor dimension is large in comparison to sample size, and for a difficult tumor-classification problem with gene expression microarray data.
Recommendations
Cites work
- Adaptive prediction and estimation in linear regression with infinitely many parameters.
- Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
- Arcing classifiers. (With discussion)
- Atomic Decomposition by Basis Pursuit
- Boosting with early stopping: convergence and consistency
- Boosting With theL2Loss
- Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data
- Finding predictive gene groups from microarray data
- Greedy function approximation: A gradient boosting machine.
- scientific article; zbMATH DE number 3637090 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 893887 (Why is no real title available?)
- Least angle regression. (With discussion)
- Matching pursuits with time-frequency dictionaries
- On the Bayes-risk consistency of regularized boosting methods.
- Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
- Process consistency for AdaBoost.
- Regularization and Variable Selection Via the Elastic Net
- Smoothing Parameter Selection in Nonparametric Regression Using an Improved Akaike Information Criterion
- Sparse boosting
- The boosting approach to machine learning: an overview
- Weak greedy algorithms
Cited in
(only showing first 100 items - show all)- Knot selection by boosting techniques
- Boosting ridge regression
- Robustified \(L_2\) boosting
- Simultaneous selection of variables and smoothing parameters in structured additive regression models
- On boosting kernel regression
- Model-based boosting in R: a hands-on tutorial using the R package mboost
- Two-step sparse boosting for high-dimensional longitudinal data with varying coefficients
- Using random subspace method for prediction and variable importance assessment in linear regression
- Boosting techniques for nonlinear time series models
- The expectation-maximization approach for Bayesian quantile regression
- Regression with stagewise minimization on risk function
- Deformation of log-likelihood loss function for multiclass boosting
- Asymptotic convergence of dimension reduction based boosting in classification
- Stochastic approximation: from statistical origin to big-data, multidisciplinary applications
- Boosting high dimensional predictive regressions with time varying parameters
- Asymptotic linear expansion of regularized M-estimators
- Scalar on network regression via boosting
- A forward-backward greedy approach for sparse multiscale learning
- A sequential feature selection procedure for high-dimensional Cox proportional hazards model
- Adaptive step-length selection in gradient boosting for Gaussian location and scale models
- AdaBoost and robust one-bit compressed sensing
- A review on instance ranking problems in statistical learning
- Post-model-selection inference in linear regression models: an integrated review
- A precise high-dimensional asymptotic theory for boosting and minimum-\(\ell_1\)-norm interpolated classifiers
- Model selection for high-dimensional linear regression with dependent observations
- A look at robustness and stability of \(\ell_1\)-versus \(\ell_0\)-regularization: discussion of papers by Bertsimas et al. and Hastie et al.
- A sequential approach to feature selection in high-dimensional additive models
- Multinomial logit models with implicit variable selection
- Regularization method for predicting an ordinal response using longitudinal high-dimensional genomic data
- On the differences between \(L_2\) boosting and the Lasso
- Inference for L₂-boosting
- Asymptotic properties of bridge estimators in sparse high-dimensional regression models
- Early stopping in \(L_{2}\)Boosting
- Boosting iterative stochastic ensemble method for nonlinear calibration of subsurface flow models
- Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities
- Instrumental variables estimation with many weak instruments using regularized JIVE
- Wavelet-based gradient boosting
- Greedy algorithms for prediction
- Reprint of: A forward-backward greedy approach for sparse multiscale learning
- Iterative selection using orthogonal regression techniques
- Boosting in structured additive models.
- Splines for financial volatility
- Tilting methods for assessing the influence of components in a classifier
- Bayesian variable selection and estimation in semiparametric joint models of multivariate longitudinal and survival data
- Sparse boosting
- Thresholding least-squares inference in high-dimensional regression models
- Econometric estimation with high-dimensional moment equalities
- scientific article; zbMATH DE number 5957419 (Why is no real title available?)
- Feature selection in finite mixture of sparse normal linear models in high-dimensional feature space
- Boosting with missing predictors
- A nonparametric estimator for the covariance function of functional data
- Variable Selection and Model Choice in Geoadditive Regression Models
- Statistical significance in high-dimensional linear models
- Characterizing \(L_{2}\)Boosting
- Boosting algorithms: regularization, prediction and model fitting
- On \(b\)-bit min-wise hashing for large-scale regression and classification with sparse data
- Detection of differential item functioning in Rasch models by boosting techniques
- Sparse regression and support recovery with \(\mathbb{L}_2\)-boosting algorithms
- Tests of the martingale difference hypothesis using boosting and RBF neural network approximations
- Subject-specific Bradley–Terry–Luce models with implicit variable selection
- Dimension reduction boosting
- Conditional sparse boosting for high-dimensional instrumental variable estimation
- A Tree-Based Semi-Varying Coefficient Model for the COM-Poisson Distribution
- Sparsity identification for high-dimensional partially linear model with measurement error
- Feature selection by canonical correlation search in high-dimensional multiresponse models with complex group structures
- General sparse boosting: improving feature selection of \(L_{2}\) boosting by correlation-based penalty family
- scientific article; zbMATH DE number 5056247 (Why is no real title available?)
- Optimization by Gradient Boosting
- Discussion on ``Two-stage procedures for high-dimensional data by Makoto Aoshima and Kazuyoshi Yata
- Stratified Cox models with time‐varying effects for national kidney transplant patients: A new blockwise steepest ascent method
- Greedy Variable Selection for High-Dimensional Cox Models
- Using predictability to improve matching of urban locations in Philadelphia
- Large Scale Prediction with Decision Trees
- Forward-selected panel data approach for program evaluation
- Estimation and inference of treatment effects with L₂-boosting in high-dimensional settings
- Gibbs Priors for Bayesian Nonparametric Variable Selection with Weak Learners
- Early stopping for L^2 -boosting in high-dimensional linear models
- Rank-based sequential feature selection for high-dimensional accelerated failure time models with main and interaction effects
- Robust statistical boosting with quantile-based adaptive loss functions
- Significance tests for boosted location and scale models with linear base-learners
- Variable selection in high dimensional linear regressions with parameter instability
- Gradient boosting for Dirichlet regression models
- A balanced statistical boosting approach for GAMLSS via new step lengths
- High-dimensional L₂-Boosting: rate of convergence
- Test and Measure for Partial Mean Dependence Based on Machine Learning Methods
- Gradient boosting for generalised additive mixed models
- Boosting for detection of gene-environment interactions
- Lassoed boosting and linear prediction in the equities market
- Loss-guided stability selection
- High-dimensional variable selection and prediction under competing risks with application to SEER-medicare linked data
- Sparse boosting for high-dimensional survival data with varying coefficients
- On the choice and influence of the number of boosting steps for high-dimensional linear Cox-models
- Robust gradient boosting for generalized additive models for location, scale and shape
- Model selection for unit-root time series with many predictors
- Gradient boosting-based discrete failure time model for selecting time-varying effects and interactions
- Boosting in Cox regression: a comparison between the likelihood-based and the model-based approaches with focus on the R-packages \textit{CoxBoost} and \textit{mboost}
- Variable selection in high-dimensional sparse multiresponse linear regression models
- High-dimensional variable selection
- Best subset selection, persistence in high-dimensional statistical learning and optimization under l₁ constraint
- Regularization in statistics
This page was built for publication: Boosting for high-dimensional linear models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2497175)
