Phase transition in limiting distributions of coherence of high-dimensional random matrices

DOI10.1016/J.JMVA.2011.11.008zbMATH Open1352.60006arXiv1102.2926OpenAlexW2019602541MaRDI QIDQ413738FDOQ413738

Authors: Tiefeng Jiang, T. Tony Cai 

Publication date: 7 May 2012

Published in: Journal of Multivariate Analysis (Search for Journal in Brave)

Abstract: The coherence of a random matrix, which is defined to be the largest magnitude of the Pearson correlation coefficients between the columns of the random matrix, is an important quantity for a wide range of applications including high-dimensional statistics and signal processing. Inspired by these applications, this paper studies the limiting laws of the coherence of $nimesp$ random matrices for a full range of the dimension $p$ with a special focus on the ultra high-dimensional setting. Assuming the columns of the random matrix are independent random vectors with a common spherical distribution, we give a complete characterization of the behavior of the limiting distributions of the coherence. More specifically, the limiting distributions of the coherence are derived separately for three regimes: $frac1nlogpo0$, $$, and $frac1nlogpoinfty$. The results show that the limiting behavior of the coherence differs significantly in different regimes and exhibits interesting phase transition phenomena as the dimension $p$ grows as a function of $n$. Applications to statistics and compressed sensing in the ultra high-dimensional setting are also discussed.

Full work available at URL: https://arxiv.org/abs/1102.2926

Recommendations

• Necessary and sufficient conditions for the asymptotic distributions of coherence of ultra-high dimensional random matrices
• Limiting laws of coherence of random matrices with applications to testing covariance structure and construction of compressed sensing matrices
• Concentration of measure and spectra of random matrices: applications to correlation matrices, elliptical distributions and beyond
• Random matrices and high-dimensional statistics: beyond covariance matrices
• The limiting spectral density of large dimensional sample covariance matrices

zbMATH Keywords

coherencephase transitionrandom matrixmaximumlimiting distributioncorrelation coefficientChen-Stein methodsample correlation matrix

Mathematics Subject Classification ID

Multivariate distribution of statistics (62H10) Estimation in multivariate analysis (62H12) Random matrices (probabilistic aspects) (60B20) Central limit and other weak theorems (60F05) Strong limit theorems (60F15)

Cites Work

• Simultaneous analysis of Lasso and Dantzig selector
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• Title not available (Why is that?)
• Sure Independence Screening for Ultrahigh Dimensional Feature Space
• The asymptotic distributions of the largest entries of sample correlation matrices.
• A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)
• Decoding by Linear Programming
• Stable recovery of sparse overcomplete representations in the presence of noise
• Compressed sensing
• Limiting laws of coherence of random matrices with applications to testing covariance structure and construction of compressed sensing matrices
• Two moments suffice for Poisson approximations: The Chen-Stein method
• On Sparse Representations in Arbitrary Redundant Bases
• Uncertainty principles and ideal atomic decomposition
• Title not available (Why is that?)
• The asymptotic distribution and Berry-Esseen bound of a new test for independence in high dimension with an application to stochastic optimization
• Near-ideal model selection by \(\ell _{1}\) minimization
• Large-Scale Correlation Screening
• On the distribution of the roots of certain symmetric matrices
• Shifting Inequality and Recovery of Sparse Signals
• Stable Recovery of Sparse Signals and an Oracle Inequality
• Some strong limit theorems for the largest entries of sample correlation matrices
• Asymptotic distribution of the largest off-diagonal entry of correlation matrices
• On Jiang's asymptotic distribution of the largest entry of a sample correlation matrix
• Necessary and sufficient conditions for the asymptotic distribution of the largest entry of a sample correlation matrix
• Robust tests for spherical symmetry
• Book review of: Daniel S. Alexander, A history of complex dynamics. From Schröder to Fatou and Julia.

Cited In (19)

• Necessary and sufficient conditions for the asymptotic distributions of coherence of ultra-high dimensional random matrices
• Approximation of rectangular beta-Laguerre ensembles and large deviations
• Central limit theorems for classical likelihood ratio tests for high-dimensional normal distributions
• Recent advances in directional statistics
• Spectral Properties of Rescaled Sample Correlation Matrix
• Testing independence with high-dimensional correlated samples
• Asymptotics of eigenstructure of sample correlation matrices for high-dimensional spiked models
• Phase transition in random tensors with multiple independent spikes
• Limiting distributions of largest entries of sample co-variance matrices from 1-dependent normal populations
• Penalized linear regression with high-dimensional pairwise screening
• Two-Sample Covariance Matrix Testing and Support Recovery in High-Dimensional and Sparse Settings
• Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing
• Asymptotic analysis for extreme eigenvalues of principal minors of random matrices
• The asymptotic distributions of the largest entries of sample correlation matrices under an \(\alpha\)-mixing assumption
• Quantification of model bias underlying the phenomenon of Einstein from Noise
• Point process convergence for the off-diagonal entries of sample covariance matrices
• High-dimensional tests for spherical location and spiked covariance
• Largest entries of sample correlation matrices from equi-correlated normal populations
• Testing for practically significant dependencies in high dimensions via bootstrapping maxima of \(U\)-statistics

This page was built for publication: Phase transition in limiting distributions of coherence of high-dimensional random matrices

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q413738)