The asymptotic distribution and Berry-Esseen bound of a new test for independence in high dimension with an application to stochastic optimization

DOI10.1214/08-AAP527zbMATH Open1154.60021arXiv0901.2468OpenAlexW1963657942MaRDI QIDQ2378634FDOQ2378634

Authors: Weidong Liu, Zhengyan Lin, Q. M. Shao 

Publication date: 13 January 2009

Published in: The Annals of Applied Probability (Search for Journal in Brave)

Abstract: Let $mathbf{X}_1,\mathrm{...},mathbf{X}_n$ be a random sample from a $p$-dimensional population distribution. Assume that $c_1{n}^{alpha}leqpleq{c}_2{n}^{alpha}$ for some positive constants $c_1,c_2$ and $alpha$. In this paper we introduce a new statistic for testing independence of the $p$-variates of the population and prove that the limiting distribution is the extreme distribution of type I with a rate of convergence $O((logn{)}^{5/2}/sqrtn)$. This is much faster than $O(1/logn)$, a typical convergence rate for this type of extreme distribution. A simulation study and application to stochastic optimization are discussed.

Full work available at URL: https://arxiv.org/abs/0901.2468

Recommendations

• Applications and asymptotic power of marginal-free tests of stochastic vectorial independence
• Optimal Berry-Esseen bound for statistical estimations and its application to SPDE
• Berry-Esseen bounds for multivariate nonlinear statistics with applications to M-estimators and stochastic gradient descent algorithms
• Berry-Esseen bound for high dimensional asymptotic approximation of Wilks' lambda distribution
• Distribution-free tests of independence in high dimensions
• Large deviations and asymptotic efficiency of integral statistics for testing independence
• An asymptotic viewpoint on high-dimensional Bayesian testing
• An extremal problem with applications to the problem of testing multivariate independence
• Asymptotic minimax testing of the independence hypothesis
• Asymptotically independent U-statistics in high-dimensional testing

zbMATH Keywords

Berry-Esseen boundindependence teststochastic optimizationcorrelation matricesextreme distribution

Mathematics Subject Classification ID

Asymptotic properties of parametric tests (62F05) Central limit and other weak theorems (60F05)

Cites Work

• Extremes and related properties of random sequences and processes
• On the distribution of the largest eigenvalue in principal components analysis
• The asymptotic distributions of the largest entries of sample correlation matrices.
• Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ ¹ minimization
• Stable recovery of sparse overcomplete representations in the presence of noise
• Statistical challenges with high dimensionality: feature selection in knowledge discovery
• Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size
• Testing for complete independence in high dimensions
• Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
• For most large underdetermined systems of linear equations the minimal 𝓁₁‐norm solution is also the sparsest solution
• Two moments suffice for Poisson approximations: The Chen-Stein method
• Title not available (Why is that?)
• Uncertainty principles and ideal atomic decomposition
• On the rate of convergence of normal extremes
• Uncertainty Principles and Signal Recovery
• Title not available (Why is that?)
• Characterization of LIL behavior in Banach space
• Asymptotic distribution of the largest off-diagonal entry of correlation matrices

Cited In (42)

• Joint test for homogeneity of high-dimensional means and covariance matrices using maximum-type statistics
• Necessary and sufficient conditions for the asymptotic distribution of the largest entry of a sample correlation matrix
• A tuning-free efficient test for marginal linear effects in high-dimensional quantile regression
• Simultaneous covariance inference for multimodal integrative analysis
• Necessary and sufficient conditions for the asymptotic distributions of coherence of ultra-high dimensional random matrices
• High-dimensional sparse MANOVA
• Power enhancement for testing multi-factor asset pricing models via Fisher's method
• The entries of circular orthogonal ensembles
• Central limit theorems for classical likelihood ratio tests for high-dimensional normal distributions
• Testing independence in high dimensions with sums of rank correlations
• Test for bandedness of high-dimensional covariance matrices and bandwidth estimation
• Phase transition in limiting distributions of coherence of high-dimensional random matrices
• On high-dimensional tests for mutual independence based on Pearson's correlation coefficient
• A Cramér moderate deviation theorem for Hotelling's \(T^{2}\)-statistic with applications to global tests
• Max-sum tests for cross-sectional independence of high-dimensional panel data
• Testing independence with high-dimensional correlated samples
• A test for the identity of a high-dimensional correlation matrix based on the \(\ell_4\)-norm
• Asymptotically independent U-statistics in high-dimensional testing
• Asymptotic theory for maximum deviations of sample covariance matrix estimates
• Logarithmic law of large random correlation matrices
• The Berry-Esseen bound for \(\rho\)-mixing random variables and its applications in nonparametric regression model
• Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation
• A Darling-Erdős type result for stationary ellipsoids
• A test for the complete independence of high-dimensional random vectors
• Rank-based max-sum tests for mutual independence of high-dimensional random vectors
• Limiting distributions of largest entries of sample co-variance matrices from 1-dependent normal populations
• Extremes of weighted Brownian bridges in increasing dimension
• Two-Sample Covariance Matrix Testing and Support Recovery in High-Dimensional and Sparse Settings
• Asymptotic false discovery control of the Benjamini-Hochberg procedure for pairwise comparisons
• Global one-sample tests for high-dimensional covariance matrices
• Testing and support recovery of multiple high-dimensional covariance matrices with false discovery rate control
• The asymptotic distributions of the largest entries of sample correlation matrices under an \(\alpha\)-mixing assumption
• Asymptotic distribution of the maximum interpoint distance for high-dimensional data
• Estimating sparse precision matrix: optimal rates of convergence and adaptive estimation
• Limiting behavior of largest entry of random tensor constructed by high-dimensional data
• Point process convergence for the off-diagonal entries of sample covariance matrices
• Largest entries of sample correlation matrices from equi-correlated normal populations
• Limiting laws of coherence of random matrices with applications to testing covariance structure and construction of compressed sensing matrices
• Testing for practically significant dependencies in high dimensions via bootstrapping maxima of \(U\)-statistics
• Comparing large covariance matrices under weak conditions on the dependence structure and its application to gene clustering
• On Jiang's asymptotic distribution of the largest entry of a sample correlation matrix
• Asymptotic power of Rao's score test for independence in high dimensions

This page was built for publication: The asymptotic distribution and Berry-Esseen bound of a new test for independence in high dimension with an application to stochastic optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2378634)