Consistent parameter estimation for Lasso and approximate message passing
One of the regularization methods for linear models is LASSO. LASSO depends on a tuning parameter \(\lambda\); for the computation of the LASSO estimate one has to solve an optimization problem. Approximate message passing (AMP) is an iterative algorithm for finding the solution of this problem. In the present paper a linear model in which the input variables and the noise terms are i.i.d. zero mean Gaussian variables and investigate the choice of \(\lambda\) and the choice of a threshold parameter for AMP. In difference to well-known methods for choosing the regularization parameter for LASSO, as for example cross validation, the proposed approach is based on the asymptotic behavior of the estimates, that is, the behavior for \(n\to\infty\) and \(p\to\infty\) while \(n/p\to \delta\), where \(n\) is the sample size, \(p\) is the dimension of the underlying parameter vector and \(\delta\) is a fixed number. The authors define an asymptotically optimal threshold parameter for AMP and present a consistent estimator of this optimal parameter. The asymptotically optimal regularization parameter for LASSO is defined as minimizer of almost sure limit of the mean square error of the estimator. Using the connection between the estimates of the optimally-tuned AMP and the solution of LASSO a consistent estimator for the optimal regularization parameter for LASSO is derived. As by-product several interesting properties of the solution paths of LASSO and AMP are given, for example the quasi-convexity of the mean square error of the LASSO (as a function of \(\lambda\)) and the dependence of the size of the active set on the regularization parameter.
- Consistent parameter estimation for Lasso and approximate message passing
- Robust instance-optimal recovery of sparse signals at unknown noise levels
- Strong consistency of Lasso estimators
- The distribution of the Lasso: uniform control over sparse balls and adaptive parameter tuning
- A practical scheme and fast algorithm to tune the Lasso with optimality guarantees
- A significance test for the lasso
- Asymptotics for Lasso-type estimators.
- Atomic Decomposition by Basis Pursuit
- Consistent parameter estimation for Lasso and approximate message passing
- Counting faces of randomly projected polytopes when the projection radically lowers dimension
- Estimation of the mean of a multivariate normal distribution
- High-dimensional centrally symmetric polytopes with neighborliness proportional to dimension
- High-dimensional graphs and variable selection with the Lasso
- High-dimensional variable selection
- scientific article; zbMATH DE number 5957408 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- Lasso-type recovery of sparse representations for high-dimensional data
- Least angle regression. (With discussion)
- Leave-one-out cross-validation is risk consistent for Lasso
- Living on the edge: phase transitions in convex programs with random data
- Minimax Rates of Estimation for High-Dimensional Linear Regression Over \ell_q-Balls
- Neighborliness of randomly projected simplices in high dimensions
- Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing
- On Recovery of Sparse Signals Via $\ell _{1}$ Minimization
- On the ``degrees of freedom of the lasso
- On the conditions used to prove oracle results for the Lasso
- Shifting Inequality and Recovery of Sparse Signals
- Simultaneous analysis of Lasso and Dantzig selector
- Some sharp performance bounds for least squares regression with L₁ regularization
- Sparsity oracle inequalities for the Lasso
- Stable Recovery of Sparse Signals and an Oracle Inequality
- Stable signal recovery from incomplete and inaccurate measurements
- The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing
- The elements of statistical learning. Data mining, inference, and prediction
- The Lasso problem and uniqueness
- The LASSO Risk for Gaussian Matrices
- The Noise-Sensitivity Phase Transition in Compressed Sensing
- The sparsity and bias of the LASSO selection in high-dimensional linear regression
- Universality in polytope phase transitions and message passing algorithms
- Approximate message passing algorithms for rotationally invariant matrices
- Detangling robustness in high dimensions: composite versus model-averaged estimation
- Which bridge estimator is the best for variable selection?
- A practical scheme and fast algorithm to tune the Lasso with optimality guarantees
- Approximate Message Passing With Consistent Parameter Estimation and Applications to Sparse Learning
- scientific article; zbMATH DE number 7625169 (Why is no real title available?)
- A Unifying Tutorial on Approximate Message Passing
- Consistent parameter estimation for Lasso and approximate message passing
- Characterizing the SLOPE trade-off: a variational perspective and the Donoho-Tanner limit
- LASSO Reloaded: A Variational Analysis Perspective with Applications to Compressed Sensing
- Equivalence of state equations from different methods in high-dimensional regression
This page was built for publication: Consistent parameter estimation for Lasso and approximate message passing
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5916043)
