The Price of Competition: Effect Size Heterogeneity Matters in High Dimensions
From MaRDI portal
Publication:5096992
DOI10.1109/TIT.2022.3166720zbMATH Open1505.62491arXiv2007.00566OpenAlexW3039028478WikidataQ114084315 ScholiaQ114084315MaRDI QIDQ5096992FDOQ5096992
Authors: Hua Wang, Yachong Yang, Weijie J. Su 
Publication date: 19 August 2022
Published in: IEEE Transactions on Information Theory (Search for Journal in Brave)
Abstract: In high-dimensional sparse regression, would increasing the signal-to-noise ratio while fixing the sparsity level always lead to better model selection? For high-dimensional sparse regression problems, surprisingly, in this paper we answer this question in the negative in the regime of linear sparsity for the Lasso method, relying on a new concept we term effect size heterogeneity. Roughly speaking, a regression coefficient vector has high effect size heterogeneity if its nonzero entries have significantly different magnitudes. From the viewpoint of this new measure, we prove that the false and true positive rates achieve the optimal trade-off uniformly along the Lasso path when this measure is maximal in a certain sense, and the worst trade-off is achieved when it is minimal in the sense that all nonzero effect sizes are roughly equal. Moreover, we demonstrate that the first false selection occurs much earlier when effect size heterogeneity is minimal than when it is maximal. The underlying cause of these two phenomena is, metaphorically speaking, the ``competition among variables with effect sizes of the same magnitude in entering the model. Taken together, our findings suggest that effect size heterogeneity shall serve as an important complementary measure to the sparsity of regression coefficients in the analysis of high-dimensional regression problems. Our proofs use techniques from approximate message passing theory as well as a novel technique for estimating the rank of the first false variable.
Full work available at URL: https://arxiv.org/abs/2007.00566
Linear regression; mixed models (62J05) Signal theory (characterization, reconstruction, filtering, etc.) (94A12)
Cited In (1)
This page was built for publication: The Price of Competition: Effect Size Heterogeneity Matters in High Dimensions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5096992)
