Discussion of ``Multiple testing for exploratory research by J. J. Goeman and A. Solari

DOI10.1214/11-STS356AzbMATH Open1331.62373arXiv1208.3292MaRDI QIDQ5963768FDOQ5963768

Authors: R. Heller 

Publication date: 23 February 2016

Published in: Statistical Science (Search for Journal in Brave)

Abstract: Goeman and Solari [Statist. Sci. 26 (2011) 584-597, arXiv:1208.2841] have addressed the interesting topic of multiple testing for exploratory research, and provided us with nice suggestions for exploratory analysis. They defined properties that an inferential procedure should have for exploratory analysis: the procedure should be mild, flexible and post hoc. Their inferential procedure gives a lower bound on the number of false hypotheses among the selected hypotheses, and moreover whenever possible identifies elementary hypotheses that are false. The need to estimate a lower bound on the number of false hypotheses arises in various applications, and the partial conjunction approach was developed for this purpose in Biometrics 64 (2008) 1215-1222 (see also Philos. Trans. R. Soc. Lond. Ser. A 367 (2009) 4255-4271 for more details). For example, in a combined analysis of several studies that examine the same problem, it is of interest to give a lower bound on the number of studies in which the finding was reproduced. I will first address the relation between the method of Goeman and Solari and the partial conjunction approach. Then I will discuss possible extensions and address the issue of exploration in more general settings, where the local test may not be defined in advance or where the candidate hypotheses may not be known to begin with.

Full work available at URL: https://arxiv.org/abs/1208.3292

Recommendations

• Multiple testing for exploratory research
• Screening for Partial Conjunction Hypotheses
• A multiple testing protocol for exploratory data analysis and the local misclassification rate
• A note on multiple testing for composite null hypotheses
• Control of generalized error rates in multiple testing

Mathematics Subject Classification ID

Parametric hypothesis testing (62F03) Paired and multiple comparisons; multiple testing (62J15)

Cites Work

• Title not available (Why is that?)
• Screening for Partial Conjunction Hypotheses
• Selective inference in complex research
• A note on data-splitting for the evaluation of significance levels
• A systematic comparison of methods for combining \(p\)-values from independent tests
• Multiple testing for exploratory research

Cited In (3)

• Exploratory inference: localizing relevant effects with confidence
• Discussion of ``Multiple testing for exploratory research by J. J. Goeman and A. Solari
• Derandomizing Knockoffs

This page was built for publication: Discussion of ``Multiple testing for exploratory research by J. J. Goeman and A. Solari

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5963768)