Limiting spectral distribution of \(XX^{\prime }\) matrices (Q1958511)

Matrices of the form \(\frac{1}{n} X X^t\) are studied, where \(X\) is a \(p\times n\) matrix with real entries. In the classical theory of sample covariance matrices, where \(X\) consists of iid entries without further symmetries, the asymptotic behavior has been mostly studied in regime (a): \(p, n \to \infty,\;p/n \to y,\;0 < y < \infty\), where the empirical eigenvalue distribution of \(\frac{1}{n} X X^t\) converges to the Marčenko-Pastur law with parameter \(y\), but also in (b): \(p, n \to \infty,\;p/n \to 0\), where the empirical eigenvalue distribution of \(\sqrt{\frac{n}{p}}\left(\frac{1}{n} X X^t - I_p\right)\) converges to the semicircle law (see [\textit{Z. D. Bai} and \textit{Y. Q. Yin}, Ann. Probab. 16, No.~2, 863--875 (1988; Zbl 0648.60030)] for regime (b)). Now, the authors consider cases in which \(X\) exhibits additional symmetries, in particular, they choose \(X\) as a nonsymmetric Hankel, Toeplitz, circulant or reverse circulant matrix. Using the method of moments, they establish the existence of a limiting spectral distribution in both regimes for all four examples. In regime (a), these limits are new. In regime (b), one obtains the limit distribution that was found in [\textit{W. Bryc, A. Dembo} and \textit{T. Jiang}, Ann. Probab. 34, No.~1, 1--38 (2006; Zbl 1094.15009)] for symmetric Toeplitz matrices.