Conservative hypothesis tests and confidence intervals using importance sampling
From MaRDI portal
Abstract: Importance sampling is a common technique for Monte Carlo approximation, including Monte Carlo approximation of p-values. Here it is shown that a simple correction of the usual importance sampling p-values creates valid p-values, meaning that a hypothesis test created by rejecting the null when the p-value is <= alpha will also have a type I error rate <= alpha. This correction uses the importance weight of the original observation, which gives valuable diagnostic information under the null hypothesis. Using the corrected p-values can be crucial for multiple testing and also in problems where evaluating the accuracy of importance sampling approximations is difficult. Inverting the corrected p-values provides a useful way to create Monte Carlo confidence intervals that maintain the nominal significance level and use only a single Monte Carlo sample. Several applications are described, including accelerated multiple testing for a large neurophysiological dataset and exact conditional inference for a logistic regression model with nuisance parameters.
Recommendations
- Importance Sampling for Bootstrap Confidence Intervals
- Consistent estimation of the accuracy of importance sampling using regenerative simulation
- On improved estimation for importance sampling
- Computing highly accurate or exact \(P\)-values using importance sampling
- Importance Sampling Via the Estimated Sampler
Cited in
(14)- Testing the assumptions behind importance sampling
- Is the Simes improved Bonferroni procedure conservative?
- Accelerated spike resampling for accurate multiple testing controls
- Spatiotemporal conditional inference and hypothesis tests for neural ensemble spiking precision
- Testing Independence Under Biased Sampling
- Permutation tests using arbitrary permutation distributions
- Conformal prediction beyond exchangeability
- What is a Randomization Test?
- Confidence intervals for overall response rate difference in the sequential parallel comparison design
- Fair conformal prediction for incomplete covariate data
- Fast computation of exact confidence intervals for randomized experiments with binary outcomes
- Group-weighted conformal prediction
- Unifying different theories of conformal prediction
- Fast Conservative Monte Carlo Confidence Sets
This page was built for publication: Conservative hypothesis tests and confidence intervals using importance sampling
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3224215)
