Cross validation in Lasso and its acceleration
From MaRDI portal
(Redirected from Cross validation in Lasso and its acceleration)
Abstract: We investigate leave-one-out cross validation (CV) as a determinator of the weight of the penalty term in the least absolute shrinkage and selection operator (LASSO). First, on the basis of the message passing algorithm and a perturbative discussion assuming that the number of observations is sufficiently large, we provide simple formulas for approximately assessing two types of CV errors, which enable us to significantly reduce the necessary cost of computation. These formulas also provide a simple connection of the CV errors to the residual sums of squares between the reconstructed and the given measurements. Second, on the basis of this finding, we analytically evaluate the CV errors when the design matrix is given as a simple random matrix in the large size limit by using the replica method. Finally, these results are compared with those of numerical simulations on finite-size systems and are confirmed to be correct. We also apply the simple formulas of the first type of CV error to an actual dataset of the supernovae.
Recommendations
- On cross-validated Lasso in high dimensions
- Leave-one-out cross-validation is risk consistent for Lasso
- Accelerating cross-validation in multinomial logistic regression with \(\ell_1\)-regularization
- Cross validation in sparse linear regression with piecewise continuous nonconvex penalties and its acceleration
- Linear Model Selection by Cross-Validation
Cites work
- A CDMA multiuser detection algorithm on the basis of belief propagation
- Advanced mean field methods. Theory and practice
- Compressed sensing
- Decoding by Linear Programming
- Estimating the number of clusters in a data set via the gap statistic
- scientific article; zbMATH DE number 3860199 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 6438182 (Why is no real title available?)
- Least angle regression. (With discussion)
- Leave-one-out cross-validation is risk consistent for Lasso
- Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?
- Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing
- Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
Cited in
(13)- Stabilizing the Lasso against cross-validation variability
- Debiasing the Lasso: optimal sample size for Gaussian designs
- Evaluation of generalized degrees of freedom for sparse estimation by replica method
- A statistical mechanics approach to de-biasing and uncertainty estimation in Lasso for random measurements
- Accelerating cross-validation in multinomial logistic regression with \(\ell_1\)-regularization
- Cross validation in sparse linear regression with piecewise continuous nonconvex penalties and its acceleration
- Risk consistency of cross-validation with Lasso-type procedures
- Prediction errors for penalized regressions based on generalized approximate message passing
- Regularization methods for high-dimensional data as a tool for seafood traceability
- Accelerate the warm-up stage in the Lasso computation via a homotopic approach
- A Bayesian transfer sparse identification method for nonlinear ARX systems
- Equivalence of state equations from different methods in high-dimensional regression
- On cross-validated Lasso in high dimensions
This page was built for publication: Cross validation in Lasso and its acceleration
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3302665)
