Resampling Sensitivity of High-Dimensional PCA

MaRDI QIDQ6422043zbMATH OpenFDO

Authors Haoyu Wang

Publication date 29 December 2022

Abstract: The study of stability and sensitivity of statistical methods or algorithms with respect to their data is an important problem in machine learning and statistics. The performance of the algorithm under resampling of the data is a fundamental way to measure its stability and is closely related to generalization or privacy of the algorithm. In this paper, we study the resampling sensitivity for the principal component analysis (PCA). Given an $nimesp$ random matrix $mathbfX$, let $mathbf{X}^{[k]}$ be the matrix obtained from $mathbfX$ by resampling $k$ randomly chosen entries of $mathbfX$. Let $mathbfv$ and $mathbf{v}^{[k]}$ denote the principal components of $mathbfX$ and $mathbf{X}^{[k]}$. In the proportional growth regime $p/noxiin(0,1]$, we establish the sharp threshold for the sensitivity/stability transition of PCA. When $kgg{n}^{5/3}$, the principal components $mathbfv$ and $mathbf{v}^{[k]}$ are asymptotically orthogonal. On the other hand, when $kll{n}^{5/3}$, the principal components $mathbfv$ and $mathbf{v}^{[k]}$ are asymptotically colinear. In words, we show that PCA is sensitive to the input data in the sense that resampling even a negligible portion of the input may completely change the output.

This page was built for publication: Resampling Sensitivity of High-Dimensional PCA

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6422043)