Accuracy assessment for high-dimensional linear regression

DOI10.1214/17-AOS1604zbMATH Open1403.62131arXiv1603.03474OpenAlexW2963441330MaRDI QIDQ2413610FDOQ2413610

Zi-Jian Guo, T. Tony Cai

Publication date: 14 September 2018

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: This paper considers point and interval estimation of the $el{l}_q$ loss of an estimator in high-dimensional linear regression with random design. We establish the minimax rate for estimating the $el{l}_q$ loss and the minimax expected length of confidence intervals for the $el{l}_q$ loss of rate-optimal estimators of the regression vector, including commonly used estimators such as Lasso, scaled Lasso, square-root Lasso and Dantzig Selector. Adaptivity of the confidence intervals for the $el{l}_q$ loss is also studied. Both the setting of known identity design covariance matrix and known noise level and the setting of unknown design covariance matrix and unknown noise level are studied. The results reveal interesting and significant differences between estimating the $el{l}_2$ loss and $el{l}_q$ loss with $1leq<2$ as well as between the two settings. New technical tools are developed to establish rate sharp lower bounds for the minimax estimation error and the expected length of minimax and adaptive confidence intervals for the $el{l}_q$ loss. A significant difference between loss estimation and the traditional parameter estimation is that for loss estimation the constraint is on the performance of the estimator of the regression vector, but the lower bounds are on the difficulty of estimating its $el{l}_q$ loss. The technical tools developed in this paper can also be of independent interest.

Full work available at URL: https://arxiv.org/abs/1603.03474

Recommendations

• Confidence intervals for high-dimensional linear regression: minimax rates and adaptivity
• Rate minimaxity of the Lasso and Dantzig selector for the \(l_{q}\) loss in \(l_{r}\) balls
• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• Rate optimal estimation and confidence intervals for high-dimensional regression with missing covariates
• Confidence intervals for low dimensional parameters in high dimensional linear models

zbMATH Keywords

confidence intervalsparsityminimax lower boundadaptivityhigh-dimensional linear regressionminimaxityloss estimationaccuracy assessment

Mathematics Subject Classification ID

Linear regression; mixed models (62J05) Nonparametric tolerance and confidence regions (62G15) Minimax procedures in statistical decision theory (62C20) Image analysis in multivariate analysis (62H35)

Cites Work

• EigenPrism: Inference for High Dimensional Signal-to-Noise Ratios
• Title not available (Why is that?)
• Statistics for high-dimensional data. Methods, theory and applications.
• Estimation of the mean of a multivariate normal distribution
• Detection boundary in sparse regression
• Confidence intervals for high-dimensional linear regression: minimax rates and adaptivity
• Simultaneous analysis of Lasso and Dantzig selector
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• Square-root lasso: pivotal recovery of sparse signals via conic programming
• Adapting to Unknown Smoothness via Wavelet Shrinkage
• Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism
• On asymptotically optimal confidence regions and tests for high-dimensional models
• Confidence sets in sparse regression
• Scaled sparse linear regression
• Minimax Rates of Estimation for High-Dimensional Linear Regression Over $\ell_q$-Balls
• Rate minimaxity of the Lasso and Dantzig selector for the \(l_{q}\) loss in \(l_{r}\) balls
• Minimax risks for sparse regressions: ultra-high dimensional phenomenons
• A data-driven block thresholding approach to wavelet estimation
• The LASSO Risk for Gaussian Matrices
• An adaptation theory for nonparametric confidence intervals
• Adaptive nonparametric confidence sets
• Adaptive confidence balls
• Adaptive Confidence Bands for Nonparametric Regression Functions
• The Noise-Sensitivity Phase Transition in Compressed Sensing
• On adaptive inference and confidence bands
• From Stein's unbiased risk estimates to the method of generalized cross- validation
• SURE-tuned tapering estimation of large covariance matrices

Cited In (17)

• Mathematical foundations of machine learning. Abstracts from the workshop held March 21--27, 2021 (hybrid meeting)
• Asymptotic risk and phase transition of \(l_1\)-penalized robust estimator
• Debiasing the Lasso: optimal sample size for Gaussian designs
• Optimal sparsity testing in linear regression model
• Testability of high-dimensional linear models with nonsparse structures
• Sparse high-dimensional linear regression. Estimating squared error and a phase transition
• Statistical Inference for High-Dimensional Generalized Linear Models With Binary Outcomes
• Linear Hypothesis Testing in Dense High-Dimensional Linear Models
• The distribution of the Lasso: uniform control over sparse balls and adaptive parameter tuning
• Moderate-Dimensional Inferences on Quadratic Functionals in Ordinary Least Squares
• Optimal Estimation of Genetic Relatedness in High-Dimensional Linear Models
• Honest Confidence Sets for High-Dimensional Regression by Projection and Shrinkage
• Title not available (Why is that?)
• Accuracy estimate of a linear regression model
• Comments on: ``High-dimensional simultaneous inference with the bootstrap
• Adaptive confidence sets for matrix completion
• Adaptive estimation of the sparsity in the Gaussian vector model

Uses Software

• EigenPrism

This page was built for publication: Accuracy assessment for high-dimensional linear regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2413610)