A study of error variance estimation in Lasso regression
From MaRDI portal
Abstract: Variance estimation in the linear model when is a difficult problem. Standard least squares estimation techniques do not apply. Several variance estimators have been proposed in the literature, all with accompanying asymptotic results proving consistency and asymptotic normality under a variety of assumptions. It is found, however, that most of these estimators suffer large biases in finite samples when true underlying signals become less sparse with larger per element signal strength. One estimator seems to be largely neglected in the literature: a residual sum of squares based estimator using Lasso coefficients with regularisation parameter selected adaptively (via cross-validation). In this paper, we review several variance estimators and perform a reasonably extensive simulation study in an attempt to compare their finite sample performance. It would seem from the results that variance estimators with adaptively chosen regularisation parameters perform admirably over a broad range of sparsity and signal strength settings. Finally, some intial theoretical analyses pertaining to these types of estimators are proposed and developed.
Recommendations
Cited in
(58)- Estimating the error variance in a high-dimensional linear model
- Stabilizing the Lasso against cross-validation variability
- Degrees of freedom for piecewise Lipschitz estimators
- High-dimensional simultaneous inference with the bootstrap
- High-dimensional inference: confidence intervals, \(p\)-values and R-software \texttt{hdi}
- Debiasing the Lasso: optimal sample size for Gaussian designs
- Greedy variance estimation for the LASSO
- Inference without compatibility: using exponential weighting for inference on a parameter of a linear model
- In defense of the indefensible: a very naïve approach to high-dimensional inference
- Confidence intervals for parameters in high-dimensional sparse vector autoregression
- Two-sample testing of high-dimensional linear regression coefficients via complementary sketching
- Sparse matrix linear models for structured high-throughput data
- Doubly debiased Lasso: high-dimensional inference under hidden confounding
- Projective inference in high-dimensional problems: prediction and feature selection
- Prediction error after model search
- Quasi-Bayesian estimation of large Gaussian graphical models
- Minimax posterior convergence rates and model selection consistency in high-dimensional DAG models based on sparse Cholesky factors
- A permutation approach for selecting the penalty parameter in penalized model selection
- Variance estimation in high-dimensional linear models
- Adapting to unknown noise level in sparse deconvolution
- Goodness-of-Fit Tests for High Dimensional Linear Models
- A high-dimensional focused information criterion
- Corrected proof of the result of 'A prediction error property of the Lasso estimator and its generalization' by Huang (2003)
- A study on tuning parameter selection for the high-dimensional lasso
- scientific article; zbMATH DE number 7626707 (Why is no real title available?)
- Bayesian jackknife empirical likelihood for the error variance in linear regression models
- Model Selection With Lasso-Zero: Adding Straw to the Haystack to Better Find Needles
- Projection-based Inference for High-dimensional Linear Models
- An algorithm for the multivariate group Lasso with covariance estimation
- Variational Bayes for High-Dimensional Linear Regression With Sparse Priors
- High-dimensional regression with unknown variance
- Variance estimation in high-dimensional linear regression via adaptive elastic-net
- A Critical Review of LASSO and Its Derivatives for Variable Selection Under Dependence Among Covariates
- A sequential modeling approach for predicting clinical outcomes with repeated measures
- Densely connected sub-Gaussian linear structural equation model learning via \(\ell_1\)- and \(\ell_2\)-regularized regressions
- Robust variable selection and estimation via adaptive elastic net S-estimators for linear regression
- Selective Inference for Hierarchical Clustering
- Asymptotic bias of the _2-regularized error variance estimator
- Feature-specific inference for penalized regression using local false discovery rates
- Detection of Hidden Additivity and Inference Under Model Uncertainty for Unreplicated Factorial Studies via Bayesian Model Selection and Averaging
- Sequential knockoffs for continuous and categorical predictors: with application to a large psoriatic arthritis clinical trial pool
- Bayesian variable selection methods for matched case-control studies
- A Decorrelating and Debiasing Approach to Simultaneous Inference for High-Dimensional Confounded Models
- Inference in models with omitted covariates: Cramér-type moderate deviations and applications to high-dimensional regression
- Estimation of Over-Parameterized Models from an Auto-Modeling Perspective
- Discovering the Network Granger Causality in Large Vector Autoregressive Models
- Post-transfer learning statistical inference in high-dimensional regression
- Heterogeneity-aware integrative regression for ancestry-specific association studies
- Estimation of the error structure in multivariate response linear regression models
- Higher Order Accurate Symmetric Bootstrap Confidence Intervals in High Dimensional Penalized Regression
- Cox reduction and confidence sets of models: a theoretical elucidation
- Sparse temporal disaggregation
- Variational Bayes for high-dimensional multi-source heterogeneous data with sparse priors
- Measurement error in Lasso: impact and likelihood bias correction
- Information Sharing for Robust and Stable Cross-Validation
- Efficient inference on high-dimensional linear models with missing outcomes
- Post-pretrained Lasso statistical inference
- Variable Selection Using a Smooth Information Criterion for Distributional Regression Models
This page was built for publication: A study of error variance estimation in Lasso regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3465093)
