Randomization Tests for Weak Null Hypotheses in Randomized Experiments
From MaRDI portal
Abstract: The Fisher randomization test (FRT) is appropriate for any test statistic, under a sharp null hypothesis that can recover all missing potential outcomes. However, it is often sought after to test a weak null hypothesis that the treatment does not affect the units on average. To use the FRT for a weak null hypothesis, we must address two issues. First, we need to impute the missing potential outcomes although the weak null hypothesis cannot determine all of them. Second, we need to choose a proper test statistic. For a general weak null hypothesis, we propose an approach to imputing missing potential outcomes under a compatible sharp null hypothesis. Building on this imputation scheme, we advocate a studentized statistic. The resulting FRT has multiple desirable features. First, it is model-free. Second, it is finite-sample exact under the sharp null hypothesis that we use to impute the potential outcomes. Third, it conservatively controls large-sample type I errors under the weak null hypothesis of interest. Therefore, our FRT is agnostic to the treatment effect heterogeneity. We establish a unified theory for general factorial experiments. We also extend it to stratified and clustered experiments.
Recommendations
- A closer look at testing the ``no-treatment-effect hypothesis in a comparative experiment
- Stratified exact tests for the weak causal null hypothesis in randomized trials with a binary outcome
- Randomization-based tests for ``no treatment effects
- Channeling Fisher: randomization tests and the statistical insignificance of seemingly significant experimental results
- A paradox from randomization-based causal inference
Cites work
- A paradox from randomization-based causal inference
- A randomization-based perspective on analysis of variance: a test statistic robust to treatment effect heterogeneity
- Agnostic notes on regression adjustments to experimental data: reexamining Freedman's critique
- An apparent paradox explained
- Asymptotic permutation tests in general factorial designs
- Box-Type Approximations in Nonparametric Factorial Designs
- Causal inference for statistics, social, and biomedical sciences. An introduction
- Causal inference from 2K factorial designs by using potential outcomes
- Causal Inference Using Potential Outcomes
- Comments on the Neyman-Fisher controversy and its consequences
- Conditional rank tests for the two-sample problem under random censorship
- Covariance adjustment in radomized experiments and observational studies
- Covariate adjustment in randomization-based causal inference for 2K factorial designs
- Effects attributable to treatment: Inference in experiments and observational studies with a discrete pivot
- Estimates of Location Based on Rank Tests
- Exact \(p\)-values for network interference
- Exact and asymptotically robust permutation tests
- Exact confidence intervals for nonconstant effects by inverting the signed rank test
- How do bootstrap and permutation tests work?
- scientific article; zbMATH DE number 3551712 (Why is no real title available?)
- scientific article; zbMATH DE number 3320839 (Why is no real title available?)
- scientific article; zbMATH DE number 3076948 (Why is no real title available?)
- scientific article; zbMATH DE number 3110608 (Why is no real title available?)
- Inference on means using the bootstrap
- Lasso adjustments of treatment effect estimates in randomized experiments
- MATS: inference for potentially singular and heteroscedastic MANOVA
- Multi-center clinical trials: randomization and ancillary statistics
- Multivariate and multiple permutation tests
- Observational studies.
- On mitigating the analytical limitations of finely stratified experiments
- On randomization-based and regression-based inferences for 2K factorial designs
- On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Translated from the Polish and edited by D. M. Dąbrowska and T. P. Speed
- On the Behavior of Randomization Tests Without a Group Invariance Assumption
- Ordered Hypotheses for Multiple Treatments: A Significance Test for Linear Ranks
- Parametric and nonparametric bootstrap methods for general MANOVA
- Permuting longitudinal data in spite of the dependencies
- Randomization Analysis of Experimental Data: The Fisher Randomization Test
- Randomization inference for treatment effect variation
- Randomization tests of causal effects under interference
- Randomization-based inference within principal strata
- Reduced Sensitivity to Hidden Bias at Upper Quantiles in Observational Studies with Dilated Treatment Effects
- Regression-assisted inference for the average treatment effect in paired experiments
- Sharp bounds on the variance in randomized experiments
- Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, I. Effect of Inequality of Variance in the One-Way Classification
- Studentized permutation tests for non-i.i.d. hypotheses and the generalized Behrens-Fisher problem
- Studentized sensitivity analysis for the sample average treatment effect in paired observational studies
- Testing nonparametric statistical functionals with applications to rank tests
- Tests for multivariate analysis of variance in high dimension under non-normality
- The behaviour of some significance tests under experimental randomization
- The Large-Sample Power of Tests Based on Permutations of Observations
- Theoretical comparison of bootstrap confidence intervals
- Using standard tools from finite population sampling to improve causal inference for complex experiments
Cited in
(28)- Verification of composite hypotheses from results of randomized experiments
- A paradox from randomization-based causal inference
- On the power of conditional independence testing under model-X
- Reconciling design-based and model-based causal inferences for split-plot experiments
- Evidence factors from multiple, possibly invalid, instrumental variables
- A closer look at testing the ``no-treatment-effect hypothesis in a comparative experiment
- Randomized empirical processes by algebraic groups, and tests for weak null hypotheses
- Single-case and small n experimental designs. A practical guide to randomization tests
- On the Behavior of Randomization Tests Without a Group Invariance Assumption
- Significance Levels of Randomization Trend Tests in the Event of Rare Occurrences
- Channeling Fisher: randomization tests and the statistical insignificance of seemingly significant experimental results
- Stratified exact tests for the weak causal null hypothesis in randomized trials with a binary outcome
- Randomization tests in recursive response-adaptive randomization procedures
- scientific article; zbMATH DE number 4186870 (Why is no real title available?)
- Hypothesis testing in adaptively sampled data: ART to maximize power beyond \textit{iid }sampling
- Pair-Switching Rerandomization
- Testing Biased Randomization Assumptions and Quantifying Imperfect Matching and Residual Confounding in Matched Observational Studies
- Rejective Sampling, Rerandomization, and Regression Adjustment in Survey Experiments
- Is Fisher inference inferior to Neyman inference for policy analysis?
- Berry-Esseen bounds for design-based causal inference with possibly diverging treatment levels and varying group sizes
- Sharp variance estimator and causal bootstrap in stratified randomized experiments
- Estimating causal effects under non-individualistic treatments due to network entanglement
- On the Term “Randomization Test”
- Exact bias correction for linear adjustment of randomized controlled trials
- Design-Based Uncertainty for Quasi-Experiments
- Fast Conservative Monte Carlo Confidence Sets
- Asymptotic validity and finite-sample properties of approximate randomization tests
- Title not available (Why is no real title available?)
This page was built for publication: Randomization Tests for Weak Null Hypotheses in Randomized Experiments
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5881970)
