Time-uniform, nonparametric, nonasymptotic confidence sequences
From MaRDI portal
Publication:2039804
Abstract: A confidence sequence is a sequence of confidence intervals that is uniformly valid over an unbounded time horizon. Our work develops confidence sequences whose widths go to zero, with nonasymptotic coverage guarantees under nonparametric conditions. We draw connections between the Cram'er-Chernoff method for exponential concentration, the law of the iterated logarithm (LIL), and the sequential probability ratio test -- our confidence sequences are time-uniform extensions of the first; provide tight, nonasymptotic characterizations of the second; and generalize the third to nonparametric settings, including sub-Gaussian and Bernstein conditions, self-normalized processes, and matrix martingales. We illustrate the generality of our proof techniques by deriving an empirical-Bernstein bound growing at a LIL rate, as well as a novel upper LIL for the maximum eigenvalue of a sum of random matrices. Finally, we apply our methods to covariance matrix estimation and to estimation of sample average treatment effect under the Neyman-Rubin potential outcomes model.
This paper is really a continuation of the authors' work [Probab. Surv. 17, 257--317 (2020; Zbl 1456.60054)] in the context of confidence sequences. A confidence sequence is defined as a sequence of confidence sets \(CI_t\), \(t=1,2,\dots\) which satisfy \(P(\forall t \geq 1:\theta_t \in CI_t)\geq 1-\alpha, \text{ where } \theta_t\) is a quantity of interest and \(1-\alpha\) is a preassigned confidence level. The authors develop methods to derive confidence sequences by using exponential concentration inequalities based on time-uniform Chernoff bounds. They draw connections between the Cramér-Chernoff method for exponential concentration, the law of the iterated logarithm (LIL) and the sequential probability ratio test. They explore confidence sequences as time-uniform extensions of the exponential concentration, provide nonasymptotic characterizations of the second, and generalize the sequential tests to nonparametric settings. Numerous illustrations of their methods are given such as an empirical-Bernstein bound growing at a LIL rate, and upper LIL for the maximum eigenvalue of a sum of random matrices. Some applications such as covariance matrix estimation, and estimation of sample average treatment effect under the Neyman-Rubin potential outcomes model are given.
Recommendations
- Catoni-style confidence sequences for heavy-tailed mean estimation
- scientific article; zbMATH DE number 3844859
- The asymptotic consistency and efficiency of fixed-size sequential confidence sets
- Sequential estimation of quantiles with applications to A/B testing and best-arm identification
- SPRT Fixed Length Confidence Intervals
- On the errors committed by sequences of estimator functionals
- An optimal purely sequential strategy with asymptotic second-order properties: Applications from statistical inference and data analysis
- Application of limit theorems for superposition of random functions to sequential estimation
- On Darling-Robbins type confidence sequences and sequential tests with power one for parameters of an autoregressive process
Cites work
- A maximized sequential probability ratio test for drug and vaccine safety surveillance
- A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations
- A nonlinear renewal theory with applications to sequential analysis II
- A nonlinear renewal theory with applications to sequential analysis. I
- A sequential clinical trial for testing p₁=p₂
- An introduction to matrix concentration inequalities
- Boundary crossing probabilities for sample sums and confidence sequences
- Boundary Crossing Probabilities for the Wiener Process and Sample Sums
- Causal inference for statistics, social, and biomedical sciences. An introduction
- Concentration inequalities and moment bounds for sample covariance operators
- Concentration inequalities for sums and martingales
- Concentration inequalities. A nonasymptotic theory of independence
- Concentration of measure inequalities in information theory, communications, and coding
- CONFIDENCE SEQUENCES FOR MEAN, VARIANCE, AND MEDIAN
- Context tree selection: a unifying view
- Discrete Sequential Boundaries for Clinical Trials
- Estimation Following Sequential Tests
- Exploration-exploitation tradeoff using variance estimates in multi-armed bandits
- Forcing a sequential experiment to be balanced
- Freedman's inequality for matrix martingales
- scientific article; zbMATH DE number 4149427 (Why is no real title available?)
- scientific article; zbMATH DE number 1246230 (Why is no real title available?)
- scientific article; zbMATH DE number 1348603 (Why is no real title available?)
- scientific article; zbMATH DE number 1057567 (Why is no real title available?)
- scientific article; zbMATH DE number 3335638 (Why is no real title available?)
- scientific article; zbMATH DE number 3034028 (Why is no real title available?)
- scientific article; zbMATH DE number 3045589 (Why is no real title available?)
- Incorporating scientific, ethical and economic considerations into the design of clinical trials in the pharmaceutical industry: a sequential approach
- ITERATED LOGARITHM INEQUALITIES
- Nonanticipating estimation applied to sequential analysis and changepoint detection
- On confidence sequences
- On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Translated from the Polish and edited by D. M. Dąbrowska and T. P. Speed
- On the complexity of best-arm identification in multi-armed bandit models
- PROBABILITY DISTRIBUTIONS RELATED TO THE LAW OF THE ITERATED LOGARITHM
- Probability Inequalities for Sums of Bounded Random Variables
- Pseudo-maximization and self-normalized processes
- Random vectors in the isotropic position
- Self-Normalized Processes
- Self-normalized processes: exponential inequalities, moment bounds and iterated logarithm laws.
- Sequential analysis. Tests and confidence intervals
- Sequential Tests of Statistical Hypotheses
- SOME FURTHER REMARKS ON INEQUALITIES FOR SAMPLE SUMS
- Statistical Methods Related to the Law of the Iterated Logarithm
- Test martingales, Bayes factors and \(p\)-values
- The expected sample size of some tests of power one
- The Hartman-Wintner Law of the Iterated Logarithm for Martingales
- Theory and applications of multivariate self-normalized processes
- Time-uniform Chernoff bounds via nonnegative supermartingales
- Weighted sums of certain dependent random variables
Cited in
(81)- Testing exchangeability: fork-convexity, supermartingales and e-processes
- Sequential estimation of quantiles with applications to A/B testing and best-arm identification
- On the bias, risk, and consistency of sample means in multi-armed bandits
- scientific article; zbMATH DE number 7626761 (Why is no real title available?)
- Universal inference
- Confidence sequences with composite likelihoods
- Likelihood, Replicability and Robbins' Confidence Sequences
- Trade-off Between Validity and Efficiency of Merging P-Values Under Arbitrary Dependence
- Exact anytime-valid confidence intervals for contingency tables and beyond
- Online multiple hypothesis testing
- Game-theoretic statistics and safe anytime-valid inference
- Catoni-style confidence sequences for heavy-tailed mean estimation
- A composite generalization of Ville's martingale theorem using e-processes
- Merging sequential e-values via martingales
- Online multiple testing with super-uniformity reward
- Testing with \(\mathrm{p}^*\)-values: between p-values, mid p-values, and e-values
- Sequential testing for elicitable functionals via supermartingales
- Evidential Calibration of Confidence Intervals
- Post-selection inference for e-value based confidence intervals
- Hoeffding and Bernstein inequalities for weighted sums of exchangeable random variables
- On the existence of powerful p-values and e-values for composite hypotheses
- Matrix Freedman inequality for sub-Weibull martingales
- Choosing alpha post hoc: the danger of multiple standard significance thresholds
- Randomized and exchangeable improvements of Markov's, Chebyshev's and Chernoff's inequalities
- Replicable bandits for digital health interventions
- Near-optimal inference in adaptive linear regression
- Contextual bandits with stage-wise constraints
- The extended Ville's inequality for nonintegrable nonnegative supermartingales
- Positive semidefinite matrix supermartingales
- Sequential Monte Carlo testing by betting
- Isotonic subgroup selection
- Online inference in high-dimensional regression with streaming clustered data
- Anytime-valid and asymptotically efficient inference driven by predictive recursion
- Testing the mean and variance by e-processes
- Authors' reply to the discussion of `safe testing'
- Zihao Wen and David L. Dowe's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Maozai Tian, Keming Yu and Jiangfeng Wang's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Andrej Srakar's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Judith ter Schure's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Christian P. Robert and Joshua Bon's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Stefano Rizzelli's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Luigi Pace and Alessandra Salvan's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Joris Mulder's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Alexander Ly's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Sander Greenland's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Neil Dey, Ryan Martin, and Jonathan P. Williams' contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Christine P. Chai's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Marco Cattaneo's contribution to the discussion of ``Safe testing by Grünwald, de Heide, and Koolen
- Joshua bon and christian P. Robert's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Vladimir Vovk's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Wenkai Xu's contribution to the discussion of `safe testing' by Grünwald, de Heide and Koolen
- Christian Hennig's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Glenn Shafer's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Thorsten Dickhaus's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Martin larsson, aaditya ramdas, and johannes Ruf's contribution to the discussion of `safe testing' by Grünwald, de heide, and koolen
- David R. Bickel's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Seconder of the vote of thanks to Grünwald, de Heide, and Koolen and contribution to the discussion of `safe testing'
- Proposer of the vote of thanks to Grünwald, de Heide, and Koolen and contribution to the discussion of `safe testing'
- Safe testing
- The numeraire e-variable and reverse information projection
- Semiparametric inference based on adaptively collected data
- Anytime-valid t -tests and confidence sequences for Gaussian means with unknown variance
- Dealing with unknown variances in best-arm identification
- Uncertainty quantification using martingales for misspecified Gaussian processes
- Leveraging initial hints for free in stochastic linear bandits
- Sparse recovery with multiple data streams: an adaptive sequential testing approach
- Time-uniform central limit theory and asymptotic confidence sequences
- Revisiting stochastic gradient descent for strongly convex objectives: tight uniform-in-time bounds
- Multiple testing with anytime-valid Monte Carlo p-values
- Nonasymptotic heavy-tailed mean estimation in smooth Banach spaces
- Basemath's test: group sequential testing without recurrent numerical integration
- Time-uniform self-normalized concentration for vector-valued processes
- Empirical Bernstein in smooth Banach spaces
- Generalized universal inference on risk minimizers
- Measuring evidence against exchangeability and group invariance with e-values
- Anytime validity is free: inducing sequential tests
- Combining evidence across filtrations
- Confidence sequences with informative, bounded-influence priors
- E-values: calibration, combination and applications
- Generic E-Variables for Exact Sequential k-Sample Tests that allow for Optional Stopping
- A Note on Online Change Point Detection
This page was built for publication: Time-uniform, nonparametric, nonasymptotic confidence sequences
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2039804)
