Testing independence with high-dimensional correlated samples
From MaRDI portal
Abstract: Testing independence among a number of (ultra) high-dimensional random samples is a fundamental and challenging problem. By arranging identically distributed -dimensional random vectors into a data matrix, we investigate the problem of testing independence among columns under the matrix-variate normal modeling of data. We propose a computationally simple and tuning-free test statistic, characterize its limiting null distribution, analyze the statistical power and prove its minimax optimality. As an important by-product of the test statistic, a ratio-consistent estimator for the quadratic functional of a covariance matrix from correlated samples is developed. We further study the effect of correlation among samples to an important high-dimensional inference problem --- large-scale multiple testing of Pearson's correlation coefficients. Indeed, blindly using classical inference results based on the assumed independence of samples will lead to many false discoveries, which suggests the need for conducting independence testing before applying existing methods. To address the challenge arising from correlation among samples, we propose a "sandwich estimator" of Pearson's correlation coefficient by de-correlating the samples. Based on this approach, the resulting multiple testing procedure asymptotically controls the overall false discovery rate at the nominal level while maintaining good statistical power. Both simulated and real data experiments are carried out to demonstrate the advantages of the proposed methods.
Recommendations
- The distance correlation \(t\)-test of independence in high dimension
- Testing for independence of high-dimensional variables: V-coefficient based approach
- Test for high-dimensional correlation matrices
- On high-dimensional tests for mutual independence based on Pearson's correlation coefficient
- Testing independence in high dimensions with sums of rank correlations
Cites work
- scientific article; zbMATH DE number 48335 (Why is no real title available?)
- scientific article; zbMATH DE number 3578221 (Why is no real title available?)
- scientific article; zbMATH DE number 720689 (Why is no real title available?)
- scientific article; zbMATH DE number 1964693 (Why is no real title available?)
- scientific article; zbMATH DE number 1750182 (Why is no real title available?)
- scientific article; zbMATH DE number 889593 (Why is no real title available?)
- A constrained \(\ell _{1}\) minimization approach to sparse precision matrix estimation
- A statistical framework to infer functional gene relationships from biologically interrelated microarray experiments
- A two-sample test for high-dimensional data with applications to gene-set testing
- Adaptive thresholding for sparse covariance matrix estimation
- Are a set of microarrays independent of each other?
- Central limit theorems for classical likelihood ratio tests for high-dimensional normal distributions
- Corrections to LRT on large-dimensional covariance matrix by RMT
- Detecting column dependence when rows are correlated and estimating the strength of the row correlation
- Distribution-free tests of independence in high dimensions
- Estimating sparse precision matrix: optimal rates of convergence and adaptive estimation
- Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation
- Estimation of functionals of sparse covariance matrices
- Gaussian graphical model estimation with false discovery rate control
- Gemini: graph estimation with matrix variate normal instances
- Generalized thresholding of large covariance matrices
- Inference with transposable data: modelling the effects of row and column correlations
- Limiting laws of coherence of random matrices with applications to testing covariance structure and construction of compressed sensing matrices
- Model selection and estimation in the matrix normal graphical model
- On some test criteria for covariance matrix
- On the distribution of the largest eigenvalue in principal components analysis
- Phase transition and regularized bootstrap in large-scale \(t\)-tests with false discovery rate control
- Phase transition in limiting distributions of coherence of high-dimensional random matrices
- Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size
- Some matrix-variate distribution theory: Notational considerations and a Bayesian application
- Sparse estimation of a covariance matrix
- Sparsistency and rates of convergence in large covariance matrix estimation
- Testing for Pairwise Serial Independence Via the Empirical Distribution Function
- Testing for complete independence in high dimensions
- Testing independence among a large number of high-dimensional random vectors
- Testing independence with high-dimensional correlated samples
- The asymptotic distribution and Berry-Esseen bound of a new test for independence in high dimension with an application to stochastic optimization
- The asymptotic distributions of the largest entries of sample correlation matrices.
Cited in
(12)- A new test of independence for high-dimensional data
- Testing independence in high dimensions with sums of rank correlations
- Kronecker delta method for testing independence between two vectors in high-dimension
- Maximum pairwise Bayes factors for covariance structure testing
- Testing independence with high-dimensional correlated samples
- Tests of zero correlation using modified RV coefficient for high-dimensional vectors
- Test for high-dimensional correlation matrices
- Correlation tests for high-dimensional data using extended cross-data-matrix methodology
- Properties of linear spectral statistics of frequency-smoothed estimated spectral coherence matrix of high-dimensional Gaussian time series
- Likelihood ratio tests for many groups in high dimensions
- High dimensional cross-sectional dependence test under arbitrary serial correlation
- Asymptotic power of Rao's score test for independence in high dimensions
This page was built for publication: Testing independence with high-dimensional correlated samples
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1750290)
