Abstract: Although the Lasso has been extensively studied, the relationship between its prediction performance and the correlations of the covariates is not fully understood. In this paper, we give new insights into this relationship in the context of multiple linear regression. We show, in particular, that the incorporation of a simple correlation measure into the tuning parameter can lead to a nearly optimal prediction performance of the Lasso even for highly correlated covariates. However, we also reveal that for moderately correlated covariates, the prediction performance of the Lasso can be mediocre irrespective of the choice of the tuning parameter. We finally show that our results also lead to near-optimal rates for the least-squares estimator with total variation penalty.
The prediction performance of the Lasso is studied for Gaussian multiple regression models with deterministic design. The term \textit{prediction performance} is understood as the magnitude of the risk measured by the prediction loss \(\frac{1}{n}\|X(\hat\beta-\beta)\|^2_2\), where \(n\) is the sample size, \(X\) is the design matrix, \(\hat\beta\) is the Lasso estimator, and \(\beta\) is the regression vector. In the paper under review, a simple measure of the correlations of the covariates is introduced. If this measure is incorporated in the choice of the tuning parameter, the Lasso prediction risk decays at a fast rate for a broad variety of settings including ones with strongly correlated covariates. However, for moderately correlated covariates, the prediction performance of the Lasso can be mediocre irrespective of the choice of the tuning parameter. The obtained results lead to near-optimal rates for the least-squares estimator with total variation penalty.
Recommendations
Cited in
(70)- Exact spike train inference via \(\ell_{0}\) optimization
- Stabilizing the Lasso against cross-validation variability
- On the prediction loss of the Lasso in the partially labeled setting
- On the total variation regularized estimator over a class of tree graphs
- Improving the prediction performance of the Lasso by subtracting the additive structural noises
- Oracle inequalities for high-dimensional prediction
- On the sensitivity of the Lasso to the number of predictor variables
- On the exponentially weighted aggregate with the Laplace prior
- Approximate \(\ell_0\)-penalized estimation of piecewise-constant signals on graphs
- Slope meets Lasso: improved oracle bounds and optimality
- Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy-Krause variation
- Finite impulse response models: a non-asymptotic analysis of the least squares estimator
- Adaptive estimation of multivariate piecewise polynomials and bounded variation functions by optimal decision trees
- Prediction bounds for higher order total variation regularized least squares
- Tuning parameter calibration for personalized prediction in medicine
- Tensor denoising with trend filtering
- Canonical thresholding for nonsparse high-dimensional linear regression
- Ridge regression revisited: debiasing, thresholding and bootstrap
- Augmented direct learning for conditional average treatment effect estimation with double robustness
- Adaptive risk bounds in univariate total variation denoising and trend filtering
- Removing the singularity of a penalty via thresholding function matching
- Estimating piecewise monotone signals
- Inference for high-dimensional instrumental variables regression
- ERM and RERM are optimal estimators for regression problems when malicious outliers corrupt the labels
- Prediction error bounds for linear regression with the TREX
- Prediction and estimation consistency of sparse multi-class penalized optimal scoring
- Sharp oracle inequalities for low-complexity priors
- Reconstruction of jointly sparse vectors via manifold optimization
- Tuning parameter calibration for _1-regularized logistic regression
- Localized Gaussian width of \(M\)-convex hulls with applications to Lasso and convex aggregation
- On Lasso refitting strategies
- Optimal two-step prediction in regression
- Orthogonal one step greedy procedure for heteroscedastic linear models
- On the robustness of the generalized fused Lasso to prior specifications
- Lasso–type and Heuristic Strategies in Model Selection and Forecasting
- Solution of linear ill-posed problems using overcomplete dictionaries
- On tight bounds for the Lasso
- The DFS fused Lasso: linear-time denoising over general graphs
- Adapting to unknown noise level in sparse deconvolution
- Corrected proof of the result of 'A prediction error property of the Lasso estimator and its generalization' by Huang (2003)
- Variable selection under multicollinearity using modified log penalty
- scientific article; zbMATH DE number 7626791 (Why is no real title available?)
- Penalized B-spline estimator for regression functions using total variation penalty
- Cross-validation with confidence
- Logistic regression with total variation regularization
- Adaptive rates for total variation image denoising
- Binarsity: a penalization for one-hot encoded features in linear supervised learning
- Strong Rules for Discarding Predictors in Lasso-Type Problems
- High-dimensional latent panel quantile regression with an application to asset pricing
- Group sparse recovery via group square-root elastic net and the iterative multivariate thresholding-based algorithm
- A Critical Review of LASSO and Its Derivatives for Variable Selection Under Dependence Among Covariates
- Ensemble Subset Regression (ENSURE): Efficient High-dimensional Prediction
- Statistical guarantees for regularized neural networks
- The Lasso with structured design and entropy of (absolute) convex hulls
- Sampling rates for ^1-synthesis
- Estimation of Linear Functionals in High-Dimensional Linear Models: From Sparsity to Nonsparsity
- Statistical guarantees for sparse deep learning
- Element-wise estimation error of generalized Fused Lasso
- Integrating additional knowledge into the estimation of graphical models
- Multivariate trend filtering for lattice data
- Empirical priors and posterior concentration in a piecewise polynomial sequence model
- Sample-Specific Learning of Lymphovascular Invasion with Heterogeneous Spatial Patterns
- A non-asymptotic analysis of the single component PLS regression
- Sample-specific cooperative learning integrating heterogeneous radiomics and pathomics data
- Solution of linear ill-posed problems using random dictionaries
- Denoising over networks with applications to partially observed epidemics
- Sharp optimality of sparse group LASSO and SLOPE under weaker assumptions
- Uncertainty quantification for iterative algorithms in linear models with application to early stopping
- A note on the minimax risk of sparse linear regression
- Frame-constrained total variation regularization for white noise regression
This page was built for publication: On the prediction performance of the Lasso
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q502891)
