Minimax optimal conditional independence testing

DOI10.1214/20-AOS2030zbMATH Open1480.62077OpenAlexW3203715578MaRDI QIDQ2054483FDOQ2054483

Authors: Matey Neykov, Sivaraman Balakrishnan, Larry Wasserman 

Publication date: 3 December 2021

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We consider the problem of conditional independence testing of $X$ and $Y$ given $Z$ where $X,Y$ and $Z$ are three real random variables and $Z$ is continuous. We focus on two main cases - when $X$ and $Y$ are both discrete, and when $X$ and $Y$ are both continuous. In view of recent results on conditional independence testing (Shah and Peters, 2018), one cannot hope to design non-trivial tests, which control the type I error for all absolutely continuous conditionally independent distributions, while still ensuring power against interesting alternatives. Consequently, we identify various, natural smoothness assumptions on the conditional distributions of $X,Y|Z=z$ as $z$ varies in the support of $Z$, and study the hardness of conditional independence testing under these smoothness assumptions. We derive matching lower and upper bounds on the critical radius of separation between the null and alternative hypotheses in the total variation metric. The tests we consider are easily implementable and rely on binning the support of the continuous variable $Z$. To complement these results, we provide a new proof of the hardness result of Shah and Peters.

Full work available at URL: https://arxiv.org/abs/2001.03039

Recommendations

• Testing conditional independence of discrete distributions
• Strongly consistent nonparametric tests of conditional independence
• The Hardness of Conditional Independence Testing and the Generalised Covariance Measure
• Testing conditional independence via integrating-up transform
• Testing conditional independence using maximal nonlinear conditional correlation

zbMATH Keywords

hypothesis testingminimax optimalityconditional independence

Mathematics Subject Classification ID

Nonparametric hypothesis testing (62G10) Minimax procedures in statistical decision theory (62C20)

Cites Work

• Title not available (Why is that?)
• A Class of Statistics with Asymptotically Normal Distribution
• A NONPARAMETRIC HELLINGER METRIC TEST FOR CONDITIONAL INDEPENDENCE
• A consistent characteristic function-based test for conditional independence
• A survey of exact inference for contingency tables. With comments and a rejoinder by the author
• An automatic inequality prover and instance optimal identity testing
• Approximation Theorems of Mathematical Statistics
• Causation, prediction, and search
• Characteristic function based testing for conditional independence: a nonparametric regression approach
• Consistent nonparametric tests of independence
• Distribution testing lower bounds via reductions from communication complexity
• Estimating the unseen, improved estimators for entropy and other properties
• Hypothesis testing for densities and high-dimensional multinomials: sharp local minimax rates
• Hypothesis testing for high-dimensional multinomials: a selective review
• Minimax nonparametric detection of signals in white Gaussian noise
• Non-asymptotic minimax rates of testing in signal detection
• Nonparametric goodness-of-fit testing under Gaussian models
• On a nonparametric notion of residual and its applications
• Probabilistic graphical models.
• Remember the curse of dimensionality: the case of goodness-of-fit testing in arbitrary dimension
• Strongly consistent nonparametric tests of conditional independence
• Testing conditional independence of discrete distributions
• Testing conditional independence using maximal nonlinear conditional correlation
• Testing conditional independence via Rosenblatt transforms
• Testing conditional independence via empirical likelihood
• Testing the conditional independence and monotonicity assumptions of item response theory
• The Hardness of Conditional Independence Testing and the Generalised Covariance Measure
• The conditional permutation test for independence while controlling for confounders

Cited In (14)

• Local permutation tests for conditional independence
• Minimax optimality of permutation tests
• The conditional permutation test for independence while controlling for confounders
• A double-robust test for high-dimensional gene coexpression networks conditioning on clinical information
• Reconciling model-X and doubly robust approaches to conditional independence testing
• Nonparametric conditional local independence testing
• Minimax optimal conditional density estimation under total variation smoothness
• On Azadkia-Chatterjee's conditional dependence coefficient
• A survey of some recent developments in measures of association
• Minimax-Optimal Hypothesis Testing With Estimation-Dependent Costs
• Adaptive minimax test of independence
• Testing conditional independence of discrete distributions
• Near-Optimal Learning of Tree-Structured Distributions by Chow and Liu
• Analysis of conditional randomisation and permutation schemes with application to conditional independence testing

Uses Software

• TETRAD
• GeneralisedCovarianceMeasure

This page was built for publication: Minimax optimal conditional independence testing

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2054483)