Limit theorems for \(U\)-processes (Q688059)

Let \((S,{\mathcal S},P)\) be a probability space, \(\{X_ i\}\) an i.i.d. sequence of \(S\)-valued random variables with distribution \(P\) and \(F\) a class of measurable real functions on \(S^ m\), \(m \geq 2\). The paper gives a systematic study of the law of large numbers and the central limit theorem for \(U\)-processes of order \(m\), \[ U^ n_ m (f)=(n-m)!/n! \sum_{I_{mn}} f(X_{i_ 1},\dots,X_{i_ m}), \] indexed by kernels \(f \in F\), where \(I_{mn}=\{(i_ 1,\dots,i_ m):1 \leq i_ j \leq n\), \(i_ j \neq i_ k\) if \(j \neq k\}\). There are given necessary and sufficient conditions for LLN and sufficient conditions for CLT both for nondegenerate and degenerate \(U\)-processes. These conditions are in terms of random metric entropies. The results are relatively complete: the CLT and LLN for measurable VC subgraph classes \(F\) and for classes \(F\) satisfying bracketing conditions are corollaries of the general results. As applications there are considered some particular \(U\)-processes including Liu's simplicial depth process. The authors use various techniques as: a decoupling inequality, exponential inequalities including a new Bernstein type inequality for degenerate \(U\)-statistics, integrability of Gaussian and Rademacher chaos, etc.