Probing for sparse and fast variable selection with model-based boosting

DOI10.1155/2017/1421409zbMATH Open1397.92020DBLPjournals/cmmm/ThomasHMB17arXiv1702.04561OpenAlexW2588369015WikidataQ38613216 ScholiaQ38613216MaRDI QIDQ1664500FDOQ1664500

Janek Thomas, Andreas Mayr, Tobias Hepp, Bernd Bischl

Publication date: 27 August 2018

Published in: Computational \& Mathematical Methods in Medicine (Search for Journal in Brave)

Abstract: We present a new variable selection method based on model-based gradient boosting and randomly permuted variables. Model-based boosting is a tool to fit a statistical model while performing variable selection at the same time. A drawback of the fitting lies in the need of multiple model fits on slightly altered data (e.g. cross-validation or bootstrap) to find the optimal number of boosting iterations and prevent overfitting. In our proposed approach, we augment the data set with randomly permuted versions of the true variables, so called shadow variables, and stop the step-wise fitting as soon as such a variable would be added to the model. This allows variable selection in a single fit of the model without requiring further parameter tuning. We show that our probing approach can compete with state-of-the-art selection methods like stability selection in a high-dimensional classification benchmark and apply it on gene expression data for the estimation of riboflavin production of Bacillus subtilis.

Full work available at URL: https://arxiv.org/abs/1702.04561

Recommendations

• Sparse boosting
• General sparse boosting: improving feature selection of \(L_{2}\) boosting by correlation-based penalty family
• Component-wisely sparse boosting
• Selection of Binary Variables and Classification by Boosting
• Model-Free Variable Selection

zbMATH Keywords

cross-validationmodel-based boosting

Mathematics Subject Classification ID

Applications of statistics to biology and medical sciences; meta analysis (62P10) General biostatistics (92B15) Biochemistry, molecular biology (92C40)

Cites Work

• The elements of statistical learning. Data mining, inference, and prediction
• Least angle regression. (With discussion)
• Stability Selection
• 10.1162/153244303322753616
• 10.1162/153244303322753643
• Title not available (Why is that?)
• High-dimensional graphs and variable selection with the Lasso
• Regularization and Variable Selection Via the Elastic Net
• Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
• Boosting algorithms: regularization, prediction and model fitting
• Variable Selection with Error Control: Another Look at Stability Selection
• Generalized Additive Modeling with Implicit Variable Selection by Likelihood‐Based Boosting
• Controlling Variable Selection by the Addition of Pseudovariables
• Significance analysis of microarrays applied to the ionizing radiation response
• Title not available (Why is that?)
• The asymptotic theory of permutation statistics.

Cited In (8)

• Feature genes selection using supervised locally linear embedding and correlation coefficient for microarray classification
• An update on statistical boosting in biomedicine
• Boosting Distributional Copula Regression
• Prediction-based variable selection for component-wise gradient boosting
• Significance tests for boosted location and scale models with linear base-learners
• Corrigendum to: ``Probing for sparse and fast variable selection with model-based boosting
• Boosting multivariate structured additive distributional regression models
• Accelerated Componentwise Gradient Boosting Using Efficient Data Representation and Momentum-Based Optimization

Uses Software

• glmnet
• Boruta
• TIGRESS
• batchtools

This page was built for publication: Probing for sparse and fast variable selection with model-based boosting

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1664500)