A closer look at testing the ``no-treatment-effect hypothesis in a comparative experiment

DOI10.1214/15-STS513MaRDI QIDQ254464zbMATH OpenFDO

Authors Joseph B. Lang

Publication date 8 March 2016

Published in Statistical Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1509.03108, https://projecteuclid.org/euclid.ss/1439220717

zbMATH Keywords

Fisher's exact testpermutation testscausal effectscompletely randomized designfinite-population sampling theoryFisher vs. NeymanHorvitz-Thompson estimatornonmeasurable probability samplepotential variablesprocess-based inferencerandomization testsrandomization-based inferenceselection-based inference

Mathematics Subject Classification ID

Parametric hypothesis testing (62F03) Nonparametric hypothesis testing (62G10)

Abstract: Standard tests of the "no-treatment-effect" hypothesis for a comparative experiment include permutation tests, the Wilcoxon rank sum test, two-sample $t$ tests, and Fisher-type randomization tests. Practitioners are aware that these procedures test different no-effect hypotheses and are based on different modeling assumptions. However, this awareness is not always, or even usually, accompanied by a clear understanding or appreciation of these differences. Borrowing from the rich literatures on causality and finite-population sampling theory, this paper develops a modeling framework that affords answers to several important questions, including: exactly what hypothesis is being tested, what model assumptions are being made, and are there other, perhaps better, approaches to testing a no-effect hypothesis? The framework lends itself to clear descriptions of three main inference approaches: process-based, randomization-based, and selection-based. It also promotes careful consideration of model assumptions and targets of inference, and highlights the importance of randomization. Along the way, Fisher-type randomization tests are compared to permutation tests and a less well-known Neyman-type randomization test. A simulation study compares the operating characteristics of the Neyman-type randomization test to those of the other more familiar tests.

Recommendations

• Randomization-based tests for ``no treatment effects
• Randomization-based inference and the choice of randomization procedures
• On the Behavior of Randomization Tests Without a Group Invariance Assumption
• Randomization Tests for Weak Null Hypotheses in Randomized Experiments
• A paradox from randomization-based causal inference

Cites work

• scientific article; zbMATH DE number 1817585 (Why is no real title available?)
• scientific article; zbMATH DE number 3137858 (Why is no real title available?)
• scientific article; zbMATH DE number 3433389 (Why is no real title available?)
• scientific article; zbMATH DE number 3029212 (Why is no real title available?)
• scientific article; zbMATH DE number 3030169 (Why is no real title available?)
• scientific article; zbMATH DE number 3076948 (Why is no real title available?)
• A Generalization of Sampling Without Replacement From a Finite Universe
• A Unified Approach to Design of Experiments
• Causal Inference Using Potential Outcomes
• In Memoriam: Jerzy Neyman 1894–1981
• Model assisted survey sampling
• ON THE z-TEST IN RANDOMIZED BLOCKS AND LATIN SQUARES
• On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection
• Permutation methods: a basis for exact inference
• Statistics and Causal Inference
• THE INTERPRETATION OF THE EFFECTS OF NON-ADDITIVITY IN THE LATIN SQUARE
• THE SIGNIFICANCE OF THE DIFFERENCE BETWEEN TWO MEANS WHEN THE POPULATION VARIANCES ARE UNEQUAL
• The theory of the design of experiments
• Why randomize?

Cited in

(6)

• A paradox from randomization-based causal inference
• Channeling Fisher: randomization tests and the statistical insignificance of seemingly significant experimental results
• Randomization Tests for Weak Null Hypotheses in Randomized Experiments
• An apparent paradox explained
• Posterior predictive \(p\)-values with Fisher randomization tests in noncompliance settings: test statistics vs discrepancy measures
• Randomization-based tests for ``no treatment effects

This page was built for publication: A closer look at testing the ``no-treatment-effect hypothesis in a comparative experiment

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q254464)