Optimal rates for independence testing via U-statistic permutation tests

DOI10.1214/20-AOS2041zbMATH Open1504.62060arXiv2001.05513OpenAlexW3213451519MaRDI QIDQ130903FDOQ130903

Authors: Thomas B. Berrett, Ioannis Kontoyiannis, Richard J. Samworth, Thomas B. Berrett, Richard Samworth, Ioannis Kontoyiannis 

Publication date: 15 January 2020

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We study the problem of independence testing given independent and identically distributed pairs taking values in a $sigma$-finite, separable measure space. Defining a natural measure of dependence $D(f)$ as the squared $L^2$-distance between a joint density $f$ and the product of its marginals, we first show that there is no valid test of independence that is uniformly consistent against alternatives of the form $f:D(f)geqh{o}^2$. We therefore restrict attention to alternatives that impose additional Sobolev-type smoothness constraints, and define a permutation test based on a basis expansion and a $U$-statistic estimator of $D(f)$ that we prove is minimax optimal in terms of its separation rates in many instances. Finally, for the case of a Fourier basis on $[0,1{]}^2$, we provide an approximation to the power function that offers several additional insights. Our methodology is implemented in the R package USP.

Full work available at URL: https://arxiv.org/abs/2001.05513

Recommendations

• Minimax optimality of permutation tests
• Consistent nonparametric tests of independence
• Adaptive minimax test of independence
• Consistent distribution-free \(K\)-sample and independence tests for univariate random variables
• Nonparametric Independence Tests: Space Partitioning and Kernel Approaches

zbMATH Keywords

independence testingU-statistics\(U\)-statistic permutation (USP) testminimax separation ratespermutation testsStein's method

Mathematics Subject Classification ID

Nonparametric hypothesis testing (62G10) Asymptotic properties of nonparametric inference (62G20) Nonparametric statistical resampling methods (62G09) Measures of association (correlation, canonical correlation, etc.) (62H20) Central limit and other weak theorems (60F05) Minimax procedures in statistical decision theory (62C20)

Cites Work

• Algorithm AS 159: An Efficient Method of Generating Random R × C Tables with Given Row and Column Totals
• The Hardness of Conditional Independence Testing and the Generalised Covariance Measure
• Measuring and testing dependence by correlation of distances
• Equivalence of distance-based and RKHS-based statistics in hypothesis testing
• Approximation Theorems of Mathematical Statistics
• Bootstrap and randomization tests of some nonparametric hypotheses
• Detecting Novel Associations in Large Data Sets
• Testing Statistical Hypotheses
• A NEW MEASURE OF RANK CORRELATION
• A Non-Parametric Test of Independence
• Tests of independence among continuous random vectors based on Cramér-von Mises functionals of the empirical copula process
• Global power functions of goodness of fit tests.
• Exact and asymptotically robust permutation tests
• Algorithmic Learning Theory
• Nonparametric independence testing via mutual information
• Permutation tests in change point analysis
• Central limit theorem for integrated square error of multivariate nonparametric density estimators
• Asymptotically valid and exact permutation tests based on two-sample \(U\)-statistics
• Portmanteau Test of Independence for Functional Observations
• Finite-sample consistency of combination-based permutation tests with application to repeated measures designs
• Kernel-Based Tests for Joint Independence
• The Conditional Permutation Test for Independence While Controlling for Confounders
• Asymptotically minimax tests for nonparametric hypotheses concerning the distribution density
• Minimax testing of the hypothesis of independence for ellipsoids in \(l_ p\)
• On coupling constructions and rates in the CLT for dependent summands with applications to the antivoter model and weighted \(U\)-statistics
• Quantitative CLTs for symmetric \(U\)-statistics using contractions
• Efficient estimation of integral functionals of a density
• Asymptotic minimax testing of the independence hypothesis
• Concentration inequalities. A nonasymptotic theory of independence
• Consistent distribution-free \(K\)-sample and independence tests for univariate random variables
• A Century of Interplay Between Taylor Series, Fourier Series and Brownian Motion
• Quantitative de Jong theorems in any dimension
• Title not available (Why is that?)
• Elements of Copula Modeling with R
• Title not available (Why is that?)
• Distribution-Free Consistent Independence Tests via Center-Outward Ranks and Signs
• Bootstrap and permutation tests of independence for point processes
• A central limit theorem for generalized multilinear forms

Cited In (11)

• Adaptive test of independence based on HSIC measures
• Statistical Inferences for Complex Dependence of Multimodal Imaging Data
• Minimax optimality of permutation tests
• Title not available (Why is that?)
• The Binary Expansion Randomized Ensemble Test
• Exact detection thresholds and minimax optimality of Chatterjee's correlation coefficient
• A survey of some recent developments in measures of association
• USP
• Permutation Tests at Nonparametric Rates
• Consistency of permutation tests of independence using distance covariance, HSIC and dHSIC
• On universally consistent and fully distribution-free rank tests of vector independence

Uses Software

• R
• copula
• energy
• AS 159
• copula
• dHSIC
• GeneralisedCovarianceMeasure
• IndepTest

This page was built for publication: Optimal rates for independence testing via \(U\)-statistic permutation tests

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q130903)