Hypothesis Testing in High-Dimensional Regression Under the Gaussian Random Design Model: Asymptotic Theory

DOI10.1109/TIT.2014.2343629zbMATH Open1360.62074arXiv1301.4240OpenAlexW2963278901MaRDI QIDQ2986116FDOQ2986116

Authors: A. Javanmard, Andrea Montanari 

Publication date: 16 May 2017

Published in: IEEE Transactions on Information Theory (Search for Journal in Brave)

Abstract: We consider linear regression in the high-dimensional regime where the number of observations $n$ is smaller than the number of parameters $p$. A very successful approach in this setting uses $el{l}_1$-penalized least squares (a.k.a. the Lasso) to search for a subset of $s_0<n$ parameters that best explain the data, while setting the other parameters to zero. Considerable amount of work has been devoted to characterizing the estimation and model selection problems within this approach. In this paper we consider instead the fundamental, but far less understood, question of emph{statistical significance}. More precisely, we address the problem of computing p-values for single regression coefficients. On one hand, we develop a general upper bound on the minimax power of tests with a given significance level. On the other, we prove that this upper bound is (nearly) achievable through a practical procedure in the case of random design matrices with independent entries. Our approach is based on a debiasing of the Lasso estimator. The analysis builds on a rigorous characterization of the asymptotic distribution of the Lasso estimator and its debiased version. Our result holds for optimal sample size, i.e., when $n$ is at least on the order of $s_{0}log(p/s_0)$. We generalize our approach to random design matrices with i.i.d. Gaussian rows $x_{i}simN(0,Sigma)$. In this case we prove that a similar distributional characterization (termed `standard distributional limit') holds for $n$ much larger than $s_0(logp{)}^2$. Finally, we show that for optimal sample size, $n$ being at least of order $s_{0}log(p/s_0)$, the standard distributional limit for general Gaussian designs can be derived from the replica heuristics in statistical physics.

Full work available at URL: https://arxiv.org/abs/1301.4240

Mathematics Subject Classification ID

Parametric hypothesis testing (62F03) Ridge regression; shrinkage estimators (Lasso) (62J07) Minimax procedures in statistical decision theory (62C20) Information theory (general) (94A15)

Cited In (56)

• Large-Scale Two-Sample Comparison of Support Sets
• Online Debiasing for Adaptively Collected High-Dimensional Data With Applications to Time Series Analysis
• StarTrek: combinatorial variable selection with false discovery rate control
• One-step regularized estimator for high-dimensional regression models
• Enmsp: an elastic-net multi-step screening procedure for high-dimensional regression
• SLOPE-adaptive variable selection via convex optimization
• Generalized matrix decomposition regression: estimation and inference for two-way structured data
• Asymptotic normality of robust \(M\)-estimators with convex penalty
• LASSO risk and phase transition under dependence
• Discussion: ``A significance test for the lasso
• Discussion: ``A significance test for the lasso
• Discussion: ``A significance test for the lasso
• Discussion: ``A significance test for the lasso
• Asymptotic risk and phase transition of \(l_1\)-penalized robust estimator
• Ill-posed estimation in high-dimensional models with instrumental variables
• A significance test for the lasso
• On the asymptotic variance of the debiased Lasso
• Statistical Inference, Learning and Models in Big Data
• Uniformly valid post-regularization confidence regions for many functional parameters in z-estimation framework
• Debiasing the Lasso: optimal sample size for Gaussian designs
• Linear hypothesis testing in dense high-dimensional linear models
• Discussion: ``A significance test for the lasso
• Online rules for control of false discovery rate and false discovery exceedance
• Constructing confidence intervals for the signals in sparse phase retrieval
• Post-model-selection inference in linear regression models: an integrated review
• Efficient estimation of smooth functionals in Gaussian shift models
• Universality of regularized regression estimators in high dimensions
• Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation
• Statistical Inference for High-Dimensional Generalized Linear Models With Binary Outcomes
• Asymptotically efficient estimation of smooth functionals of covariance operators
• A unifying framework of high-dimensional sparse estimation with difference-of-convex (DC) regularizations
• Gene set priorization guided by regulatory networks with p-values through kernel mixed model
• The distribution of the Lasso: uniform control over sparse balls and adaptive parameter tuning
• Rejoinder: ``A significance test for the lasso
• In defense of the indefensible: a very naïve approach to high-dimensional inference
• Asymptotic normality and optimalities in estimation of large Gaussian graphical models
• Gaussian graphical model estimation with false discovery rate control
• On asymptotically optimal confidence regions and tests for high-dimensional models
• The benefit of group sparsity in group inference with de-biased scaled group Lasso
• Scalable inference for high-dimensional precision matrix
• Asymptotically honest confidence regions for high dimensional parameters by the desparsified conservative Lasso
• Inference for high-dimensional varying-coefficient quantile regression
• Debiasing convex regularized estimators and interval estimation in linear models
• Discussion: ``A significance test for the lasso
• Worst possible sub-directions in high-dimensional models
• Detangling robustness in high dimensions: composite versus model-averaged estimation
• Flexible and Interpretable Models for Survival Data
• Lasso-driven inference in time and space
• De-biasing the Lasso with degrees-of-freedom adjustment
• Global and Simultaneous Hypothesis Testing for High-Dimensional Logistic Regression Models
• Significance testing in non-sparse high-dimensional linear models
• Generalized M-estimators for high-dimensional Tobit I models
• Additive model selection
• The Lasso with general Gaussian designs with applications to hypothesis testing
• Semiparametric efficiency bounds for high-dimensional models
• Semi-analytic resampling in Lasso

This page was built for publication: Hypothesis Testing in High-Dimensional Regression Under the Gaussian Random Design Model: Asymptotic Theory

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2986116)