Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses

DOI10.1214/009053605000000741zbMATH Open1091.62059arXivmath/0501289OpenAlexW2144766920MaRDI QIDQ142489FDOQ142489

Authors: Nicolai Meinshausen, John Rice, Nicolai Meinshausen, John Rice 

Publication date: 1 February 2006

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We consider the problem of estimating the number of false null hypotheses among a very large number of independently tested hypotheses, focusing on the situation in which the proportion of false null hypotheses is very small. We propose a family of methods for establishing lower $100(1-alpha)\mathrm{\%}$ confidence bounds for this proportion, based on the empirical distribution of the $p$-values of the tests. Methods in this family are then compared in terms of ability to consistently estimate the proportion by letting $alphao0$ as the number of hypothesis tests increases and the proportion decreases. This work is motivated by a signal detection problem that occurs in astronomy.

Full work available at URL: https://arxiv.org/abs/math/0501289

Recommendations

• Estimating the proportion of true null hypotheses in multiple testing problems
• Estimating the number of true null hypotheses in multiple hypothesis testing
• On efficient estimators of the proportion of true null hypotheses in a multiple testing setup
• Estimating the proportion of true null hypotheses under dependence
• A cross-validation based estimation of the proportion of true null hypotheses
• Estimating the proportion of true null hypotheses using the pattern of observed \(p\)-values
• Comparison of Methods for Estimating the Number of True Null Hypotheses in Multiplicity Testing
• Two new estimators for the proportion of true null hypotheses in multiple test
• Estimating false discovery proportion in multiple comparison under dependency
• Estimating false discovery proportion under arbitrary covariance dependence

zbMATH Keywords

hypothesis testingmultiple comparisonssparsity

Mathematics Subject Classification ID

Parametric hypothesis testing (62F03) Hypothesis testing in multivariate analysis (62H15) Paired and multiple comparisons; multiple testing (62J15) Applications of statistics to physics (62P35) Statistical astronomy (85A35)

Cites Work

• Title not available (Why is that?)
• Higher criticism for detecting sparse heterogeneous mixtures.
• A stochastic process approach to false discovery control.
• A Direct Approach to False Discovery Rates
• Statistical methods for detecting stellar occultations by Kuiper Belt objects: the Taiwanese--American occultation survey
• Title not available (Why is that?)
• Lower bounds for the number of false null hypotheses for multiple testing of associations under general dependence structures
• Title not available (Why is that?)
• The statistical theory of the strength of bundles of threads. I

Cited In (82)

• On the usage of randomized p-values in the Schweder-Spjøtvoll estimator
• An inverse Laplace transform oracle estimator for the normal means problem
• Fast calculation of p-values for one-sided Kolmogorov-Smirnov type statistics
• Weighted \(p\)-value procedures for controlling FDR of grouped hypotheses
• Null-free false discovery rate control using decoy permutations
• Uniformly consistently estimating the proportion of false null hypotheses via Lebesgue-Stieltjes integral equations
• Multiple testing approaches for hypotheses in integrative genomics
• Estimating the amount of sparsity in two-point mixture models
• An Efficient Morris Method-Based Framework for Simulation Factor Screening
• Efficient signal inclusion with genomic applications
• Model selection with mixed variables on the Lasso path
• Maximum likelihood estimation of the log-concave component in a semi-parametric mixture with a standard normal density
• Controlling false discovery proportion in identification of drug-related adverse events from multiple system organ classes
• Variable selection via adaptive false negative control in linear regression
• Estimating the proportion of signal variables under arbitrary covariance dependence
• Random thresholds for linear model selection
• Two-Component Mixture Model in the Presence of Covariates
• Discussion of ``Multiple testing for exploratory research by J. J. Goeman and A. Solari
• Bonferroni-type Plug-in Procedure Controlling Generalized Familywise Error Rate
• Controlling the Familywise Error Rate with Plug-in Estimator for the Proportion of True Null Hypotheses
• Goodness-of-fit tests based on sup-functionals of weighted empirical processes
• A review of modern multiple hypothesis testing, with particular attention to the false discovery proportion
• Control of the false discovery proportion for independently tested null hypotheses
• Hypothesis testing, power and sample size determination for between group comparisons in fMRI experiments
• Robust test for detecting a signal in a high dimensional sparse normal vector
• Empirical Bayes analysis of RNA sequencing experiments with auxiliary information
• Box-constrained monotone approximations to Lipschitz regularizations, with applications to robust testing
• Generalized Augmentation to Control the False Discovery Exceedance in Multiple Testing
• An empirical Bayes mixture method for effect size and false discovery rate estimation
• On the performance of FDR control: constraints and a partial solution
• Asymptotic Bayes-optimality under sparsity of some multiple testing procedures
• Simultaneous high-probability bounds on the false discovery proportion in structured, regression and online settings
• Detecting weak signals in high dimensions
• Controlling mixed directional false discovery rate in multidimensional decisions with applications to microarray studies
• Estimating the proportion of true null hypotheses in multiple testing problems
• Optimal rates of convergence for estimating the null density and proportion of nonnull effects in large-scale multiple testing
• Exploring the Information in p‐Values for the Analysis and Planning of Multiple‐Test Experiments
• A new perspective on robust \(M\)-estimation: finite sample theory and applications to dependence-adjusted multiple testing
• Efficient estimation of the number of false positives in high-throughput screening
• On efficient estimators of the proportion of true null hypotheses in a multiple testing setup
• GAP: a general framework for information pooling in two-sample sparse inference
• Goodness-of-fit tests via phi-divergences
• Estimation and confidence sets for sparse normal mixtures
• On false discovery control under dependence
• Properties of higher criticism under strong dependence
• Online rules for control of false discovery rate and false discovery exceedance
• On a generalized false discovery rate
• Higher criticism: \(p\)-values and criticism
• Multiple testing of composite null hypotheses in heteroscedastic models
• A nonparametric mixture approach to density and null proportion estimation in large‐scale multiple comparison problems
• Constructing confidence intervals for selected parameters
• Proportion of Non-Zero Normal Means: Universal Oracle Equivalences and Uniformly Consistent Estimators
• Theoretical Measures of Relative Performance of Classifiers for High Dimensional Data with Small Sample Sizes
• Adaptive testing on a regression function at a point
• On approximate validation of models: a Kolmogorov-Smirnov-based approach
• A guided random walk through some high dimensional problems
• Only closed testing procedures are admissible for controlling false discovery proportions
• A parametric model to estimate the proportion from true null using a distribution for \(p\)-values
• A powerful FDR control procedure for multiple hypotheses
• Adaptive sensing performance lower bounds for sparse signal detection and support estimation
• Mixed directional false discovery rate control in multiple pairwise comparisons using weighted \(p\)-values
• Stepup procedures controlling generalized FWER and generalized FDR
• Simultaneous critical values for \(t\)-tests in very high dimensions
• α-Investing: a Procedure for Sequential Control of Expected False Discoveries
• An inequality for correlations in unidimensional monotone latent variable models for binary variables
• A note on multiple testing for composite null hypotheses
• Straight to the Source: Detecting Aggregate Objects in Astronomical Images With Proper Error Control
• Innovated higher criticism for detecting sparse signals in correlated noise
• A conservative estimator for the proportion of false nulls based on Dvoretzky, Kiefer and Wolfowitz inequality
• Asymptotic optimality of the Westfall-Young permutation procedure for multiple testing under dependence
• False discovery rate control with multivariate \(p\)-values
• Some optimality properties of FDR controlling rules under sparsity
• HCTR
• Higher criticism for large-scale inference, especially for rare and weak effects
• High frequency asymptotics for wavelet-based tests for Gaussianity and isotropy on the torus
• Dynamic adaptive multiple tests with finite sample FDR control
• Adaptive estimation of the sparsity in the Gaussian vector model
• Farmtest: factor-adjusted robust multiple testing with approximate false discovery control
• On the false discovery rate and an asymptotically optimal rejection curve
• Empirical Bayes and Resampling Based Multiple Testing Procedure Controlling Tail Probability of the Proportion of False Positives.
• The sparse Poisson means model
• Estimating the number of true null hypotheses in multiple hypothesis testing

This page was built for publication: Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q142489)