Robust covariance estimation for distributed principal component analysis

DOI10.1007/S00184-021-00848-9MaRDI QIDQ2150893zbMATH OpenOpenAlexFDO

Authors Yanyan Li

Publication date 30 June 2022

Published in Metrika (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2010.06851

zbMATH Keywords

distributed estimationheavy-tailed dataoutliersprincipal component analysisrobust covariance estimation

Mathematics Subject Classification ID

Statistics of extreme values; tail inference (62G32) Nonparametric robustness (62G35) Estimation in multivariate analysis (62H12) Factor analysis and principal components; correspondence analysis (62H25) Analysis of variance and covariance (ANOVA) (62J10)

Abstract: Fan et al. [$mathitAnnals$ $mathitof$ $mathitStatistics$ $extbf47$(6) (2019) 3009-3031] constructed a distributed principal component analysis (PCA) algorithm to reduce the communication cost between multiple servers significantly. However, their algorithm's guarantee is only for sub-Gaussian data. Spurred by this deficiency, this paper enhances the effectiveness of their distributed PCA algorithm by utilizing robust covariance matrix estimators of Minsker [$mathitAnnals$ $mathitof$ $mathitStatistics$ $extbf46$(6A) (2018) 2871-2903] and Ke et al. [$mathitStatistical$ $mathitScience$ $extbf34$(3) (2019) 454-471] to tame heavy-tailed data. The theoretical results demonstrate that when the sampling distribution is symmetric innovation with the bounded fourth moment or asymmetric with the finite $6$-th moment, the statistical error rate of the final estimator produced by the robust algorithm is similar to that of sub-Gaussian tails. Extensive numerical trials support the theoretical analysis and indicate that our algorithm is robust to heavy-tailed data and outliers.

Recommendations

• Distributed estimation of principal eigenspaces
• Robust PCA for high-dimensional data
• Distributed Estimation for Principal Component Analysis: An Enlarged Eigenspace Analysis
• A covariance-free iterative algorithm for distributed principal component analysis on vertically partitioned data
• Communication-Efficient Distributed Eigenspace Estimation

Cites work

• A Distributed Framework for Dimensionality Reduction and Denoising
• A shrinkage principle for heavy-tailed data: high-dimensional robust low-rank matrix recovery
• A useful variant of the Davis-Kahan theorem for statisticians
• Asymptotic Theory for Principal Component Analysis
• ASYMPTOTIC THEORY FOR PRINCIPAL COMPONENT ANALYSIS: NON-NORMAL CASE1
• Asymptotics of empirical eigenstructure for high dimensional spiked covariance
• Challenging the empirical mean and empirical variance: a deviation study
• Communication-efficient distributed statistical inference
• Communication-efficient sparse regression
• Covariance regularization by thresholding
• Distributed estimation of principal eigenspaces
• ECA: High-Dimensional Elliptical Component Analysis in Non-Gaussian Distributions
• High dimensional covariance matrix estimation using a factor model
• Large covariance estimation through elliptical factor models
• Robust covariance estimation for approximate factor models
• Robust covariance estimation under \(L_4\)-\(L_2\) norm equivalence
• Robust Estimation of a Location Parameter
• Robust estimation of high-dimensional covariance and precision matrices
• Robust modifications of U-statistics and applications to covariance estimation problems
• Second order accurate distributed eigenvector computation for extremely large matrices
• Sub-Gaussian estimators of the mean of a random matrix with heavy-tailed entries
• User-friendly covariance estimation for heavy-tailed distributions

Cited in

(9)

• Heteroskedastic PCA: algorithm, optimality, and applications
• Distributed estimation of principal eigenspaces
• Parallel computation of high-dimensional robust correlation and covariance matrices
• Communication-efficient distributed covariance sketch, with application to distributed PCA
• Distributed Estimation for Principal Component Analysis: An Enlarged Eigenspace Analysis
• Robust distributed multicategory angle-based classification for massive data
• A covariance-free iterative algorithm for distributed principal component analysis on vertically partitioned data
• Distributed estimation of spiked eigenvalues in spiked population models
• Optimal subset selection for distributed local principal component analysis

Describes a project that uses

Uses Software

• UCI

This page was built for publication: Robust covariance estimation for distributed principal component analysis

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2150893)