Sequential implementation of Monte Carlo tests with uniformly bounded resampling risk

DOI10.1198/JASA.2009.TM08368zbMATH Open1205.65016arXivmath/0612488OpenAlexW3104812324WikidataQ29542701 ScholiaQ29542701MaRDI QIDQ3069883FDOQ3069883

Authors: Axel Gandy 

Publication date: 1 February 2011

Published in: Journal of the American Statistical Association (Search for Journal in Brave)

Abstract: This paper introduces an open-ended sequential algorithm for computing the p-value of a test using Monte Carlo simulation. It guarantees that the resampling risk, the probability of a different decision than the one based on the theoretical p-value, is uniformly bounded by an arbitrarily small constant. Previously suggested sequential or non-sequential algorithms, using a bounded sample size, do not have this property. Although the algorithm is open-ended, the expected number of steps is finite, except when the p-value is on the threshold between rejecting and not rejecting. The algorithm is suitable as standard for implementing tests that require (re-)sampling. It can also be used in other situations: to check whether a test is conservative, iteratively to implement double bootstrap tests, and to determine the sample size required for a certain power.

Full work available at URL: https://arxiv.org/abs/math/0612488

Recommendations

• A simple method for implementing Monte Carlo tests
• Bounding the resampling risk for sequential Monte Carlo implementation of hypothesis tests
• Designing Monte Carlo implementations of permutation or bootstrap hypothesis tests
• Power of the Sequential Monte Carlo Test
• MMCTest -- a safe algorithm for implementing multiple Monte Carlo tests

zbMATH Keywords

Monte Carlo testingsequential estimationsequential test\(p\)-valuesignificance test

Mathematics Subject Classification ID

Bootstrap, jackknife and other resampling methods (62F40) Monte Carlo methods (65C05)

Cited In (15)

• A statistical test for detecting discordance in rankings between \(k\) groups
• Bounding the resampling risk for sequential Monte Carlo implementation of hypothesis tests
• QuickMMCTest: quick multiple Monte Carlo testing
• Truncated sequential Monte Carlo test with exact power
• Designing Monte Carlo implementations of permutation or bootstrap hypothesis tests
• Model checks for Cox-type regression models based on optimally weighted martingale residuals
• A proportional hazards regression model with change-points in the baseline function
• Power of the Sequential Monte Carlo Test
• Frequentist-Bayesian Monte Carlo testing
• An algorithm to compute the power of Monte Carlo tests with guaranteed precision
• Implementing Monte Carlo tests with \(p\)-value buckets
• On the expected runtime of multiple testing algorithms with bounded error
• MMCTest -- a safe algorithm for implementing multiple Monte Carlo tests
• A simple method for implementing Monte Carlo tests
• Optimal allocation of Monte Carlo simulations to multiple hypothesis tests

Uses Software

• R
• simctest

This page was built for publication: Sequential implementation of Monte Carlo tests with uniformly bounded resampling risk

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3069883)