Exact detection thresholds and minimax optimality of Chatterjee's correlation coefficient

DOI10.3150/23-BEJ1648MaRDI QIDQ6201868zbMATH OpenOpenAlexWikidataFDO

Authors Arnab Auddy, Nabarun Deb, Sagnik Nandy

Publication date 26 March 2024

Published in Bernoulli (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2104.15140, https://projecteuclid.org/journals/bernoulli/volume-30/issue-2/Exact-detection-thresholds-and-minimax-optimality-of-Chatterjees-correlation-coefficient/10.3150/23-BEJ1648.full

zbMATH Keywords

independence testinglocal powerKantorovic-Wasserstein distanceLe Cam's two-point methodStein's method for locally dependent structures

Mathematics Subject Classification ID

Statistics (62-XX) Information and communication theory, circuits (94-XX)

Abstract: Recently, Chatterjee (2021) introduced a new rank-based correlation coefficient which can be used to test for independence between two random variables. His test has already attracted much attention as it is distribution-free, consistent against all fixed alternatives, asymptotically normal under the null hypothesis of independence and computable in (near) linear time; thereby making it appropriate for large-scale applications. However, not much is known about the power properties of this test beyond consistency against fixed alternatives. In this paper, we bridge this gap by obtaining the asymptotic distribution of Chatterjee's correlation under any changing sequence of alternatives "converging" to the null hypothesis (of independence). We further obtain a general result that gives exact detection thresholds and limiting power for Chatterjee's test of independence under natural nonparametric alternatives "converging" to the null. As applications of this general result, we prove a non-standard $n^{-1/4}$ detection boundary for this test and compute explicitly the limiting local power on the detection boundary, for popularly studied alternatives in literature such as mixture models, rotation models and noisy nonparametric regression. Moreover our convergence results provide explicit finite sample bounds that depend on the "distance" between the null and the alternative. Our proof techniques rely on second order Poincar'{e} type inequalities and a non-asymptotic projection theorem.

Cites work

• scientific article; zbMATH DE number 3117929 (Why is no real title available?)
• scientific article; zbMATH DE number 3174889 (Why is no real title available?)
• scientific article; zbMATH DE number 1416649 (Why is no real title available?)
• A NEW MEASURE OF RANK CORRELATION
• A Non-Parametric Test of Independence
• A consistent test of independence based on a sign covariance related to Kendall's tau
• A copula-based non-parametric measure of regression dependence
• A new coefficient of correlation
• A new method of normal approximation
• A quadratic measure of deviation of two-dimensional density estimates and a test of independence
• A simple measure of conditional dependence
• A study of the power and robustness of a new test for independence against contiguous alternatives
• Asymptotic Efficiency of Nonparametric Tests
• Asymptotic Statistics
• Asymptotic efficiency of the Blest-type tests for independence
• Asymptotically minimax hypothesis testing for nonparametric alternatives. I
• Detection of sparse positive dependence
• Distribution Free Tests of Independence Based on the Sample Distribution Function
• Distribution-free tests of independence in high dimensions
• Efficient computation of the Bergsma-Dassios sign covariance
• Generalized R-squared for detecting dependence
• High-dimensional consistent independence testing with maxima of rank correlations
• Large-Sample Theory for the Bergsma-Dassios Sign Covariance
• Lectures on the nearest neighbor method
• Minimax Testing of Nonparametric Hypotheses on a Distribution Density in the $L_p$ Metrics
• Minimax optimality of permutation tests
• Multivariate Rank-Based Distribution-Free Nonparametric Testing Using Measure Transportation
• On a Measure of Dependence Between two Random Variables
• On a multivariate copula-based dependence measure and its estimation
• On boosting the power of Chatterjee’s rank correlation
• On measures of association and a related problem
• On the Power of Certain Tests for Independence in Bivariate Populations
• On the asymptotic null distribution of the symmetrized Chatterjee's correlation coefficient
• On the limiting Pitman efficiency of some rank tests of independence
• Optimal rates for independence testing via \(U\)-statistic permutation tests
• Pitman efficiency of independence tests based on weighted rank statistics
• Sensitivity analysis based on Cramér-von Mises distance
• Symmetric rank covariances: a generalized framework for nonparametric measures of dependence
• Testing for independence by the empirical characteristic function
• The asymptotic efficacies and relative efficiencies of various linear rank tests for independence
• The performance of some correlation coefficients for a general bivariate distribution

Cited in

(2)

• A conditional distribution function-based measure for independence and \(K\)-sample tests in multivariate data
• Asymptotic expected sensitivity function and its applications to measures of monotone association

This page was built for publication: Exact detection thresholds and minimax optimality of Chatterjee's correlation coefficient

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6201868)