Model selection in regression under structural constraints
From MaRDI portal
Abstract: The paper considers model selection in regression under the additional structural constraints on admissible models where the number of potential predictors might be even larger than the available sample size. We develop a Bayesian formalism as a natural tool for generating a wide class of model selection criteria based on penalized least squares estimation with various complexity penalties associated with a prior on a model size. The resulting criteria are adaptive to structural constraints. We establish the upper bound for the quadratic risk of the resulting MAP estimator and the corresponding lower bound for the minimax risk over a set of admissible models of a given size. We then specify the class of priors (and, therefore, the class of complexity penalties) where for the "nearly-orthogonal" design the MAP estimator is asymptotically at least nearly-minimax (up to a log-factor) simultaneously over an entire range of sparse and dense setups. Moreover, when the numbers of admissible models are "small" (e.g., ordered variable selection) or, on the opposite, for the case of complete variable selection, the proposed estimator achieves the exact minimax rates.
Recommendations
Cites work
- scientific article; zbMATH DE number 4070082 (Why is no real title available?)
- scientific article; zbMATH DE number 1034042 (Why is no real title available?)
- scientific article; zbMATH DE number 3444596 (Why is no real title available?)
- A lasso for hierarchical interactions
- Adapting to unknown sparsity by controlling the false discovery rate
- Aggregation for Gaussian regression
- Bayesian constrained variable selection
- Bayesian variable selection with related predictors
- Estimating the dimension of a model
- Exponential screening and optimal rates of sparse estimation
- Gaussian model selection
- Ideal spatial adaptation by wavelet shrinkage
- Innovated higher criticism for detecting sparse signals in correlated noise
- MAP model selection in Gaussian regression
- Minimal penalties for Gaussian model selection
- Minimax Rates of Estimation for High-Dimensional Linear Regression Over $\ell_q$-Balls
- Note on large subsets of binary vectors with similar distances
- On optimality of Bayesian testimation in the normal means problem
- Oracle inequalities and optimal inference under group sparsity
- Some Comments on C P
- The risk inflation criterion for multiple regression
Cited in
(8)- Tabu search model selection in multiple regression analysis
- Model selection under order restriction
- A selection problem for a constrained linear regression model
- Selecting local models in multiple regression by maximizing power
- Consistent model selection in segmented line regression
- Factor-Adjusted Regularized Model Selection
- MAP model selection in Gaussian regression
- scientific article; zbMATH DE number 1051719 (Why is no real title available?)
This page was built for publication: Model selection in regression under structural constraints
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1951123)
