Adapting to unknown sparsity by controlling the false discovery rate

DOI10.1214/009053606000000074zbMath1092.62005arXivmath/0505374OpenAlexW2133264613WikidataQ56622266 ScholiaQ56622266MaRDI QIDQ2497176

Yoav Benjamini, David L. Donoho, Iain M. Johnstone, Felix P. Abramovich

Publication date: 3 August 2006

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/math/0505374

zbMATH Keywords

model selection thresholding multiple comparisons minimax estimation wavelet denoising smoothing parameter selection

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Minimax procedures in statistical decision theory (62C20) Statistics of extreme values; tail inference (62G32) Paired and multiple comparisons; multiple testing (62J15)

Related Items

Irrational Exuberance: Correcting Bias in Probability Estimates ⋮ Minimax optimal estimation of high-dimensional sparse covariance matrices with missing data ⋮ Group SLOPE – Adaptive Selection of Groups of Predictors ⋮ Characterizing the SLOPE trade-off: a variational perspective and the Donoho-Tanner limit ⋮ Optimal false discovery control of minimax estimators ⋮ Robust oracle estimation and uncertainty quantification for possibly sparse quantiles ⋮ Fast exact Bayesian inference for sparse signals in the normal sequence model ⋮ Controlling the Familywise Error Rate with Plug-in Estimator for the Proportion of True Null Hypotheses ⋮ Discovering the False Discovery Rate ⋮ Adaptive Sparse Estimation With Side Information ⋮ Block‐threshold‐adapted Estimators via a Maxiset Approach ⋮ Binary classification with pFDR‐pFNR losses ⋮ Operating Characteristics and Extensions of the False Discovery Rate Procedure ⋮ Model selection for estimating the non zero components of a Gaussian vector ⋮ Unnamed Item ⋮ Proportion of Non-Zero Normal Means: Universal Oracle Equivalences and Uniformly Consistent Estimators ⋮ Local comparison of empirical distributions via nonparametric regression ⋮ Regularization and the small-ball method II: complexity dependent error rates ⋮ Sparse Partial Least Squares Regression for Simultaneous Dimension Reduction and Variable Selection ⋮ Local Posterior Concentration Rate for Multilevel Sparse Sequences ⋮ Consistent variable selection for functional regression models ⋮ Higher criticism for detecting sparse heterogeneous mixtures. ⋮ A stochastic process approach to false discovery control. ⋮ Least angle regression. (With discussion) ⋮ Minimax estimation of linear functionals over nonconvex parameter spaces. ⋮ Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences ⋮ Distributed nonparametric function estimation: optimal rate of convergence and cost of adaptation ⋮ Minimal penalties for Gaussian model selection ⋮ SLOPE is adaptive to unknown sparsity and asymptotically minimax ⋮ GSDAR: a fast Newton algorithm for \(\ell_0\) regularized generalized linear models with statistical guarantee ⋮ FDR-controlling stepwise procedures and their false negatives rates ⋮ Asymptotic minimaxity of false discovery rate thresholding for sparse exponential data ⋮ Minimax and adaptive inference in nonparametric function estimation ⋮ Functional mixed effects wavelet estimation for spectra of replicated time series ⋮ A two-sample test for high-dimensional data with applications to gene-set testing ⋮ From local kernel to nonlocal multiple-model image denoising ⋮ Optimal rates of convergence for estimating the null density and proportion of nonnull effects in large-scale multiple testing ⋮ Nonlinear estimation over weak Besov spaces and minimax Bayes ⋮ Estimation of matrices with row sparsity ⋮ Simultaneous testing of multiple hypotheses using generalized \(p\)-values ⋮ General maximum likelihood empirical Bayes estimation of normal means ⋮ Strong consistency of the risk estimator in multiple hypothesis testing with the FDR threshold ⋮ On the optimality of sliced inverse regression in high dimensions ⋮ TFisher: a powerful truncation and weighting procedure for combining \(p\)-values ⋮ Thresholding procedure with priors based on Pareto distributions ⋮ A nonparametric empirical Bayes approach to adaptive minimax estimation ⋮ Bayesian linear regression with sparse priors ⋮ Feature selection when there are many influential features ⋮ Asymptotically optimal sequential FDR and pFDR control with (or without) prior information on the number of signals ⋮ Needles and straw in a haystack: posterior concentration for possibly sparse sequences ⋮ Comment: ``Microarrays, empirical Bayes and the two-groups model ⋮ Higher criticism thresholding: Optimal feature selection when useful features are rare and weak ⋮ Good, great, or lucky? Screening for firms with sustained superior performance using heavy-tailed priors ⋮ SLOPE-adaptive variable selection via convex optimization ⋮ A guided random walk through some high dimensional problems ⋮ Segmentation of the mean of heteroscedastic data via cross-validation ⋮ Model selection in regression under structural constraints ⋮ Some optimality properties of FDR controlling rules under sparsity ⋮ Asymptotic Bayes-optimality under sparsity of some multiple testing procedures ⋮ On the asymptotic properties of SLOPE ⋮ Data-driven wavelet-Fisz methodology for nonparametric function estimation ⋮ MAP model selection in Gaussian regression ⋮ Ideal denoising within a family of tree-structured wavelet estimators ⋮ Model selection and sharp asymptotic minimaxity ⋮ Consistent change-point detection with kernels ⋮ On the Benjamini-Hochberg method ⋮ UPS delivers optimal phase diagram in high-dimensional variable selection ⋮ Sparse PCA: optimal rates and adaptive estimation ⋮ On spike and slab empirical Bayes multiple testing ⋮ Sparse model selection under heterogeneous noise: exact penalisation and data-driven thresholding ⋮ Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector ⋮ Nonparametric significance testing and group variable selection ⋮ On the performance of FDR control: constraints and a partial solution ⋮ On control of the false discovery rate under no assumption of dependency ⋮ Generalized Augmentation to Control the False Discovery Exceedance in Multiple Testing ⋮ RANK: Large-Scale Inference With Graphical Nonlinear Knockoffs ⋮ A multiple testing approach to the regularisation of large sample correlation matrices ⋮ On optimality of Bayesian testimation in the normal means problem ⋮ Feature selection by higher criticism thresholding achieves the optimal phase diagram ⋮ Fast rate of convergence in high-dimensional linear discriminant analysis ⋮ Sparse semiparametric discriminant analysis ⋮ An adaptation theory for nonparametric confidence intervals ⋮ Adjusting Stepwisep-Values in Generalized Linear Models ⋮ Exponential screening and optimal rates of sparse estimation ⋮ An empirical Bayes mixture method for effect size and false discovery rate estimation ⋮ Innovated higher criticism for detecting sparse signals in correlated noise ⋮ On signal reconstruction in white noise using dictionaries ⋮ The false discovery rate: a variable selection perspective ⋮ Unsupervised empirical Bayesian multiple testing with external covariates ⋮ Adapting to unknown sparsity by controlling the false discovery rate ⋮ A review of modern multiple hypothesis testing, with particular attention to the false discovery proportion ⋮ Sharp variable selection of a sparse submatrix in a high-dimensional noisy matrix ⋮ Modified versions of the Bayesian information criterion for sparse generalized linear models ⋮ General empirical Bayes wavelet methods and exactly adaptive minimax estimation ⋮ On oracle inequalities related to data-driven hard thresholding ⋮ SPADES and mixture models ⋮ Spike and slab empirical Bayes sparse credible sets ⋮ Consistent variable selection in high dimensional regression via multiple testing ⋮ On optimality of the Benjamini-Hochberg procedure for the false discovery rate ⋮ Covariance regularization by thresholding ⋮ A universal procedure for aggregating estimators ⋮ False Discovery Rate Control Under General Dependence By Symmetrized Data Aggregation ⋮ Variable selection with Hamming loss ⋮ Optimal rates of convergence for sparse covariance matrix estimation ⋮ On false discovery rate thresholding for classification under sparsity ⋮ A simple forward selection procedure based on false discovery rate control ⋮ Control of the false discovery rate under dependence using the bootstrap and subsampling ⋮ Comparison of Bayesian and frequentist multiplicity correction for testing mutually exclusive hypotheses under data dependence ⋮ On the false discovery rate and an asymptotically optimal rejection curve ⋮ An adaptive step-down procedure with proven FDR control under independence ⋮ Gaussian model selection with an unknown variance ⋮ Rate of convergence of a risk estimator to the normal law in a multiple hypothesis testing problem using the FDR threshold ⋮ Reproducible feature selection in high-dimensional accelerated failure time models ⋮ Reproducible learning in large-scale graphical models ⋮ Some results on false discovery rate in stepwise multiple testing procedures. ⋮ Empirical Bayes selection of wavelet thresholds ⋮ Uncertainty quantification for robust variable selection and multiple testing ⋮ Rate of convergence of the risk estimate distribution to the normal law using FDR multiple hypothesis testing with inverting linear homogeneous operators ⋮ Thresholding rules for recovering a sparse signal from microarray experiments ⋮ Combining thresholding rules: a new way to improve the performance of wavelet estimators

Uses Software

EBayesThresh

Cites Work