Accuracy assessment for high-dimensional linear regression

DOI10.1214/17-AOS1604MaRDI QIDQ2413610zbMATH OpenOpenAlexFDO

Authors Zi-Jian Guo, T. Tony Cai

Publication date 14 September 2018

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1603.03474

zbMATH Keywords

confidence intervalsparsityminimax lower boundadaptivityhigh-dimensional linear regressionminimaxityloss estimationaccuracy assessment

Mathematics Subject Classification ID

Linear regression; mixed models (62J05) Nonparametric tolerance and confidence regions (62G15) Minimax procedures in statistical decision theory (62C20) Image analysis in multivariate analysis (62H35)

Abstract: This paper considers point and interval estimation of the $el{l}_q$ loss of an estimator in high-dimensional linear regression with random design. We establish the minimax rate for estimating the $el{l}_q$ loss and the minimax expected length of confidence intervals for the $el{l}_q$ loss of rate-optimal estimators of the regression vector, including commonly used estimators such as Lasso, scaled Lasso, square-root Lasso and Dantzig Selector. Adaptivity of the confidence intervals for the $el{l}_q$ loss is also studied. Both the setting of known identity design covariance matrix and known noise level and the setting of unknown design covariance matrix and unknown noise level are studied. The results reveal interesting and significant differences between estimating the $el{l}_2$ loss and $el{l}_q$ loss with $1leq<2$ as well as between the two settings. New technical tools are developed to establish rate sharp lower bounds for the minimax estimation error and the expected length of minimax and adaptive confidence intervals for the $el{l}_q$ loss. A significant difference between loss estimation and the traditional parameter estimation is that for loss estimation the constraint is on the performance of the estimator of the regression vector, but the lower bounds are on the difficulty of estimating its $el{l}_q$ loss. The technical tools developed in this paper can also be of independent interest.

Recommendations

• Confidence intervals for high-dimensional linear regression: minimax rates and adaptivity
• Rate minimaxity of the Lasso and Dantzig selector for the \(l_{q}\) loss in \(l_{r}\) balls
• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• Rate optimal estimation and confidence intervals for high-dimensional regression with missing covariates
• Confidence intervals for low dimensional parameters in high dimensional linear models

Cites work

• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• A data-driven block thresholding approach to wavelet estimation
• Adapting to Unknown Smoothness via Wavelet Shrinkage
• Adaptive Confidence Bands for Nonparametric Regression Functions
• Adaptive confidence balls
• Adaptive nonparametric confidence sets
• An adaptation theory for nonparametric confidence intervals
• Confidence intervals for high-dimensional linear regression: minimax rates and adaptivity
• Confidence sets in sparse regression
• Detection boundary in sparse regression
• EigenPrism: inference for high dimensional signal-to-noise ratios
• Estimation of the mean of a multivariate normal distribution
• From Stein's unbiased risk estimates to the method of generalized cross- validation
• Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism
• Minimax Rates of Estimation for High-Dimensional Linear Regression Over $\ell_q$-Balls
• Minimax risks for sparse regressions: ultra-high dimensional phenomenons
• On adaptive inference and confidence bands
• On asymptotically optimal confidence regions and tests for high-dimensional models
• Rate minimaxity of the Lasso and Dantzig selector for the \(l_{q}\) loss in \(l_{r}\) balls
• SURE-tuned tapering estimation of large covariance matrices
• Scaled sparse linear regression
• Simultaneous analysis of Lasso and Dantzig selector
• Square-root lasso: pivotal recovery of sparse signals via conic programming
• Statistics for high-dimensional data. Methods, theory and applications.
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• The LASSO Risk for Gaussian Matrices
• The Noise-Sensitivity Phase Transition in Compressed Sensing

Cited in

(17)

• Mathematical foundations of machine learning. Abstracts from the workshop held March 21--27, 2021 (hybrid meeting)
• Asymptotic risk and phase transition of \(l_1\)-penalized robust estimator
• Debiasing the Lasso: optimal sample size for Gaussian designs
• Linear hypothesis testing in dense high-dimensional linear models
• Optimal sparsity testing in linear regression model
• Testability of high-dimensional linear models with nonsparse structures
• Sparse high-dimensional linear regression. Estimating squared error and a phase transition
• Statistical Inference for High-Dimensional Generalized Linear Models With Binary Outcomes
• The distribution of the Lasso: uniform control over sparse balls and adaptive parameter tuning
• Moderate-Dimensional Inferences on Quadratic Functionals in Ordinary Least Squares
• Optimal Estimation of Genetic Relatedness in High-Dimensional Linear Models
• Honest Confidence Sets for High-Dimensional Regression by Projection and Shrinkage
• Accuracy estimate of a linear regression model
• scientific article; zbMATH DE number 7626791 (Why is no real title available?)
• Comments on: ``High-dimensional simultaneous inference with the bootstrap
• Adaptive confidence sets for matrix completion
• Adaptive estimation of the sparsity in the Gaussian vector model

Describes a project that uses

Uses Software

• EigenPrism

This page was built for publication: Accuracy assessment for high-dimensional linear regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2413610)