Almost sure convergence of bootstrapped means and \(U\)-statistics (Q1333117)

This paper is concerned with results like the Strong Law of Large Numbers (SLLN) and the Law of the Iterated Logarithm (LIL) for bootstrapped sample means and U-statistics of order 2. Consider a sequence of independent random variables \({\mathcal X}= \{X_ i\}\) with common distribution function. Let \({\mathcal X}_ n\) denote a sample of size \(n\) from \({\mathcal X}\) and let \(F_ n\) be its empirical distribution function. The bootstrap sample of size \(m\) from \(F_ n\) is denoted \({\mathcal X}^*_{nm}\), and define \(P^*(\cdot)= P(\cdot|{\mathcal X})\). If \(\overline {X}_ n\) and \(\overline{X}_{nm}\) denote the sample means of \({\mathcal X}_ n\) and \({\mathcal X}^*_{nm}\), this paper establishes conditions when the Bounded LIL, \[ P^* \Bigl( \limsup \varepsilon_ n^{-1} \sup_{m\geq k_ n} |\overline {X}^*_{nm}- \overline{X}_ n| \leq c \Bigr)=1 \quad P_{\mathcal X} \text{ -- a.s.} \] is true, i.e. for which sequences \(\varepsilon_ n\), \(k_ n\), and constants \(c\geq 0\). A sequence of examples nicely illustrates how increasingly stringent moment conditions lead to the Bounded LIL with smaller \(\varepsilon_ n\) and \(k_ n\). The case of bounded \(X_ i\) suggests that \(\log n\) is a natural lower bound for the \(k_ n\). The case of \(X_ i\) of finite variance leads to \(k_ n= n\log n\). In all cases the values of \(\varepsilon_ n\) are stated as random variables depending on \({\mathcal X}_ n\) through its sample variance. Under a resampling scheme where all samples are independent, the Appendix shows that when \(E X^ 2<\infty\) the \(\varepsilon_ n\) are equivalent to a non-random sequence, but when \(E X^ 2=\infty\) the correct normalizing sequence is random. A Marcinkiewicz-Zygmund-type SLLN is proven for the case where \(E X^ 2=\infty\) with non-random scaling. This extends results of \textit{K. B. Athreya}, \textit{M. Ghosh}, \textit{L. Y. Low} and \textit{P. K. Sen} [J. Stat. Plann. Inf. 9, 185-194 (1984; Zbl 0533.62043)]. Applications are presented to the almost sure convergence of bootstrapped sample moments, the central limit theorem for multiple bootstraps and the almost sure convergence of bootstrapped U-statistics. The proofs rely on Kolmogorov- type exponential inequalities on the tails of the bootstrapped statistics, properties of backward martingales and the Borel-Cantelli lemma.