Better subset regression
From MaRDI portal
Publication:5410312
DOI10.1093/BIOMET/AST041zbMATH Open1285.62076arXiv1212.0634OpenAlexW2964304472MaRDI QIDQ5410312FDOQ5410312
Authors: Shifeng Xiong 
Publication date: 16 April 2014
Published in: Biometrika (Search for Journal in Brave)
Abstract: To find efficient screening methods for high dimensional linear regression models, this paper studies the relationship between model fitting and screening performance. Under a sparsity assumption, we show that a subset that includes the true submodel always yields smaller residual sum of squares (i.e., has better model fitting) than all that do not in a general asymptotic setting. This indicates that, for screening important variables, we could follow a "better fitting, better screening" rule, i.e., pick a "better" subset that has better model fitting. To seek such a better subset, we consider the optimization problem associated with best subset regression. An EM algorithm, called orthogonalizing subset screening, and its accelerating version are proposed for searching for the best subset. Although the two algorithms cannot guarantee that a subset they yield is the best, their monotonicity property makes the subset have better model fitting than initial subsets generated by popular screening methods, and thus the subset can have better screening performance asymptotically. Simulation results show that our methods are very competitive in high dimensional variable screening even for finite sample sizes.
Full work available at URL: https://arxiv.org/abs/1212.0634
Recommendations
- Best subset selection, persistence in high-dimensional statistical learning and optimization under \(l_1\) constraint
- Resampling-based information criteria for best-subset regression
- A stepwise regression method and consistent model selection for highdimensional sparse linear models
- Bayesian selection of best subsets via hybrid search
- Subset selection for multiple linear regression via optimization
variable selectionEM algorithmcombinatorial optimizationsure screening propertyorthogonal designsbest subset regression
Cited In (8)
- Best subset selection, persistence in high-dimensional statistical learning and optimization under \(l_1\) constraint
- Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low
- Resampling-based information criteria for best-subset regression
- Complete subset regressions
- COMBSS: best subset selection via continuous optimization
- Some notes on concordance between optimization and statistics
- Linear screening for high-dimensional computer experiments
- A polynomial algorithm for best-subset selection problem
Uses Software
This page was built for publication: Better subset regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5410312)
