``Preconditioning for feature selection and regression in high-dimensional problems
From MaRDI portal
Publication:939656
Abstract: We consider regression problems where the number of predictors greatly exceeds the number of observations. We propose a method for variable selection that first estimates the regression function, yielding a "pre-conditioned" response variable. The primary method used for this initial regression is supervised principal components. Then we apply a standard procedure such as forward stepwise selection or the LASSO to the pre-conditioned response variable. In a number of simulated and real data examples, this two-step procedure outperforms forward stepwise selection or the usual LASSO (applied directly to the raw outcome). We also show that under a certain Gaussian latent variable model, application of the LASSO to the pre-conditioned response variable is consistent as the number of predictors and observations increases. Moreover, when the observational noise is rather large, the suggested procedure can give a more accurate estimate than LASSO. We illustrate our method on some real problems, including survival analysis with microarray data.
Recommendations
- Prediction by Supervised Principal Components
- Preselection in Lasso-type analysis for ultra-high dimensional genomic exploration
- Sure independence screening for ultrahigh dimensional feature space. With discussion and authors' reply
- High-dimensional variable selection
- Factor models and variable selection in high-dimensional regression analysis
Cites work
- scientific article; zbMATH DE number 5957408 (Why is no real title available?)
- scientific article; zbMATH DE number 3795247 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- Adaptive Model Selection
- Asymptotics for Lasso-type estimators.
- High-dimensional graphs and variable selection with the Lasso
- Least angle regression. (With discussion)
- Nonconcave penalized likelihood with a diverging number of parameters.
- Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ 1 minimization
- Prediction by Supervised Principal Components
- The Adaptive Lasso and Its Oracle Properties
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(30)- Non-crossing large-margin probability estimation and its application to robust SVM via pre\-condi\-tion\-ing
- Sure independence screening for ultrahigh dimensional feature space. With discussion and authors' reply
- Feature selection: from the past to the future
- Projective inference in high-dimensional problems: prediction and feature selection
- Preselection in Lasso-type analysis for ultra-high dimensional genomic exploration
- Canonical thresholding for nonsparse high-dimensional linear regression
- On data preconditioning for regularized loss minimization
- Noisy matrix completion: understanding statistical guarantees for convex relaxation via nonconvex optimization
- Estimation of a regression function corresponding to latent~variables
- Variable selection for kernel classification
- High-dimensional Cox regression analysis in genetic studies with censored survival outcomes
- A variable selection proposal for multiple linear regression analysis
- Robust high-dimensional factor models with applications to statistical machine learning
- Preconditioning the Lasso for sign consistency
- Augmented sparse reconstruction of protein signaling networks
- Optimal EMG placement for a robotic prosthesis controller with sequential, adaptive functional estimation (SAFE)
- Bayesian projection approaches to variable selection in generalized linear models
- Nonparametric estimation of a latent variable model
- Statistical and knowledge supported visualization of multivariate data
- Adaptive index models for marker-based risk stratification
- Tournament screening cum EBIC for feature selection with high-dimensional feature spaces
- High-dimensional model recovery from random sketched data by exploring intrinsic sparsity
- Compressed and Penalized Linear Regression
- Prediction by Supervised Principal Components
- Using reference models in variable selection
- Use of pretransformation to cope with extreme values in important candidate features
- Addressing the implementation challenge of risk prediction model due to missing risk factors: the submodel approximation approach
- scientific article; zbMATH DE number 7306913 (Why is no real title available?)
- Dimensionality reduction by feature clustering for regression problems
- Preconditioning for classical relationships: a note relating ridge regression and OLS \(p\)-values to preconditioned \textit{sparse} penalized regression
This page was built for publication: ``Preconditioning for feature selection and regression in high-dimensional problems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q939656)
