A power analysis for Model-X knockoffs with _p-regularized statistics

DOI10.1214/23-AOS2274arXiv2007.15346OpenAlexW4386035722MaRDI QIDQ6136579FDOQ6136579

Emmanuel J. Candès, Rina Foygel Barber, Weijie J. Su, Małgorzata Bogdan, Asaf Weinstein

Publication date: 31 August 2023

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: Variable selection properties of procedures utilizing penalized-likelihood estimates is a central topic in the study of high dimensional linear regression problems. Existing literature emphasizes the quality of ranking of the variables by such procedures as reflected in the receiver operating characteristic curve or in prediction performance. Specifically, recent works have harnessed modern theory of approximate message-passing (AMP) to obtain, in a particular setting, exact asymptotic predictions of the type I-type II error tradeoff for selection procedures that rely on $el{l}_p$-regularized estimators. In practice, effective ranking by itself is often not sufficient because some calibration for Type I error is required. In this work we study theoretically the power of selection procedures that similarly rank the features by the size of an $el{l}_p$-regularized estimator, but further use Model-X knockoffs to control the false discovery rate in the realistic situation where no prior information about the signal is available. In analyzing the power of the resulting procedure, we extend existing results in AMP theory to handle the pairing between original variables and their knockoffs. This is used to derive exact asymptotic predictions for power. We apply the general results to compare the power of the knockoffs versions of Lasso and thresholded-Lasso selection, and demonstrate that in the i.i.d. covariate setting under consideration, tuning by cross-validation on the augmented design matrix is nearly optimal. We further demonstrate how the techniques allow to analyze also the Type S error, and a corresponding notion of power, when selections are supplemented with a decision on the sign of the coefficient.

Full work available at URL: https://arxiv.org/abs/2007.15346

zbMATH Keywords

variable selectionhigh-dimensional regressionknockoffsfalse discovery rate (FDR)power analysis

Mathematics Subject Classification ID

Parametric hypothesis testing (62F03) Asymptotic properties of parametric estimators (62F12) Asymptotic properties of parametric tests (62F05) Statistical ranking and selection procedures (62F07)

Cites Work

• Type S error for classical and Bayesian single and multiple comparison procedures
• The Adaptive Lasso and Its Oracle Properties
• Statistics for high-dimensional data. Methods, theory and applications.
• High-dimensional graphs and variable selection with the Lasso
• Title not available (Why is that?)
• Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$-Constrained Quadratic Programming (Lasso)
• A knockoff filter for high-dimensional selective inference
• Adaptive linear step-up procedures that control the false discovery rate
• False discoveries occur early on the Lasso path
• Adaptive false discovery rate control under independence and dependence
• Panning for Gold: ‘Model-X’ Knockoffs for High Dimensional Controlled Variable Selection
• Controlling the false discovery rate via knockoffs
• The adaptive and the thresholded Lasso for potentially misspecified models (and a lower bound for the Lasso)
• SLOPE-adaptive variable selection via convex optimization
• Rate minimaxity of the Lasso and Dantzig selector for the \(l_{q}\) loss in \(l_{r}\) balls
• Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing
• The LASSO Risk for Gaussian Matrices
• The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing
• A necessary and sufficient condition for exact sparse recovery by \(\ell_1\) minimization
• Conclusions vs Decisions
• Overcoming the limitations of phase transition by higher order analysis of regularization techniques
• A high-dimensional power analysis of the conditional randomization test and knockoffs
• Which bridge estimator is the best for variable selection?
• On the sign recovery by least absolute shrinkage and selection operator, thresholded least absolute shrinkage and selection operator, and thresholded basis pursuit denoising
• A power analysis for Model-X knockoffs with \(\ell_p\)-regularized statistics

Cited In (3)

• A power analysis for Model-X knockoffs with \(\ell_p\)-regularized statistics
• Reconciling model-X and doubly robust approaches to conditional independence testing
• CoxKnockoff: controlled feature selection for the Cox model using knockoffs

   Recommendations

• The \(k\)th power expectile regression 👍 👎
• Likelihood-based inference for the power regression model 👍 👎
• A high-dimensional power analysis of the conditional randomization test and knockoffs 👍 👎
• A unified approach to power calculation and sample size determination for random regression models 👍 👎
• Improved kth power expectile regression with nonignorable dropouts 👍 👎
• The exact power function of an exact test of a regression model against multiple separate alternatives 👍 👎
• Explicit estimators of an unknown parameter in a power regression problem 👍 👎
• On improvement of statistical estimators in a power regression problem 👍 👎

This page was built for publication: A power analysis for Model-X knockoffs with \(\ell_p\)-regularized statistics

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6136579)