Estimating covariance and precision matrices along subspaces (Q2219236)

One of the most frequently discussed questions in multivariate analysis is the estimation of the covariance matrix \(\boldsymbol{\Sigma}\) or its inverse \(\boldsymbol{\Sigma}^{\dag}\) (i.e. corresponding precision matrix) of a random vector \(\mathbf{X}\). Let \(\hat{\boldsymbol{\Sigma}}\) be an estimator of \(\boldsymbol{\Sigma}\) given \(N\) independent copies of \(\mathbf{X}\) are available. The crucial question in this estimation is to quantify the minimum samples \(N\) ensuring that for a desired accuracy \(\epsilon>0\). Moreover, in practice the statisticians are often not directly interested in \(\boldsymbol{\Sigma}\) or \(\boldsymbol{\Sigma}^{\dag}\) themselves, but rather in how they, as (bi)-linear operators, act on specific vectors or matrices. In terms of concentration inequalities, this can be interpreted as developing bounds for \(\|\mathbf{A}(\hat{\boldsymbol{\Sigma}}-\boldsymbol{\Sigma})\mathbf{B}\|\) and \(\|\mathbf{A}(\hat{\boldsymbol{\Sigma}}^{\dag}-\boldsymbol{\Sigma}^{\dag})\mathbf{B}\|\), where \(\mathbf{A}\) and \(\mathbf{B}\) are a given pair of (rectangular) matrices. Bounds of this type in case of sub-Gaussian distributions are the principal subjects of this work. In Section 2, bounds for covariance and eigenspace estimation are presented which are sensitive to the distribution of a given random vector in the directions of interest. Section 3 is devoted to directional estimates of the precision matrix \(\boldsymbol{\Sigma}^{\dag}\) through the empirical precision matrix \(\hat{\boldsymbol{\Sigma}}^{\dag}\), analogously to results from Section 2. An application to single-index model estimation is provided, comprising both the theoretical analysis and numerical experiments can be found in Section 4. The analysis extends previous studies and shows how the accuracy of the estimator can be affected by the anisotropicity of \(\mathbf{X}\). A very long list of references concludes the paper.