On Robustness of Principal Component Regression
From MaRDI portal
Abstract: Principal component regression (PCR) is a simple, but powerful and ubiquitously utilized method. Its effectiveness is well established when the covariates exhibit low-rank structure. However, its ability to handle settings with noisy, missing, and mixed-valued, i.e., discrete and continuous, covariates is not understood and remains an important open challenge. As the main contribution of this work we establish the robustness of PCR, without any change, in this respect and provide meaningful finite-sample analysis. To do so, we establish that PCR is equivalent to performing linear regression after pre-processing the covariate matrix via hard singular value thresholding (HSVT). As a result, in the context of counterfactual analysis using observational data, we show PCR is equivalent to the recently proposed robust variant of the synthetic control method, known as robust synthetic control (RSC). As an immediate consequence, we obtain finite-sample analysis of the RSC estimator that was previously absent. As an important contribution to the synthetic controls literature, we establish that an (approximate) linear synthetic control exists in the setting of a generalized factor model or latent variable model; traditionally in the literature, the existence of a synthetic control needs to be assumed to exist as an axiom. We further discuss a surprising implication of the robustness property of PCR with respect to noise, i.e., PCR can learn a good predictive model even if the covariates are tactfully transformed to preserve differential privacy. Finally, this work advances the state-of-the-art analysis for HSVT by establishing stronger guarantees with respect to the -norm rather than the Frobenius norm as is commonly done in the matrix estimation literature, which may be of interest in its own right.
Recommendations
Cites work
- 10.1162/153244303321897690
- CoCoLasso for high-dimensional error-in-variables regression
- Estimation of average treatment effects with panel data: asymptotic theory and implementation
- Exponential screening and optimal rates of sparse estimation
- High-dimensional regression with noisy and missing data: provable guarantees with nonconvexity
- High-dimensional statistics. A non-asymptotic viewpoint
- Improved matrix uncertainty selector
- Linear and conic programming estimators in high dimensional errors-in-variables models
- Linear regression for panel with unknown number of factors as interactive fixed effects
- Matrix estimation by universal singular value thresholding
- Panel data approach vs synthetic control method
- Probabilistic Principal Component Analysis
- Restricted eigenvalue properties for correlated Gaussian designs
- Robust synthetic control
- Sparse recovery under matrix uncertainty
- Sparsity and incoherence in compressive sampling
- Why Are Big Data Matrices Approximately Low Rank?
Cited in
(8)- Stability of robust and non-robust principal components analysis
- Prediction Intervals for Synthetic Control Methods
- Ridge Regression Under Dense Factor Augmented Models
- Confidence intervals for high-dimensional accelerated failure time models under measurement errors
- On model identification and out-of-sample prediction of PCR with applications to synthetic controls
- Spectrum-aware debiasing: a modern inference framework with applications to principal components regression
- Multivariate Bayesian dynamic modeling for causal prediction
- Robust tests for the common principal components model
This page was built for publication: On Robustness of Principal Component Regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5881959)
