Envelopes and principal component regression
From MaRDI portal
(Redirected from Publication:6184884)
Abstract: Envelope methods offer targeted dimension reduction for various models. The overarching goal is to improve efficiency in multivariate parameter estimation by projecting the data onto a lower-dimensional subspace known as the envelope. Envelope approaches have advantages in analyzing data with highly correlated variables, but their iterative Grassmannian optimization algorithms do not scale very well with ultra high-dimensional data. While the connections between envelopes and partial least squares in multivariate linear regression have promoted recent progress in high-dimensional studies of envelopes, we propose a more straightforward way of envelope modeling from a novel principal components regression perspective. The proposed procedure, Non-Iterative Envelope Component Estimation (NIECE), has excellent computational advantages over the iterative Grassmannian optimization alternatives in high dimensions. We develop a unified NIECE theory that bridges the gap between envelope methods and principal components in regression. The new theoretical insights also shed light on the envelope subspace estimation error as a function of eigenvalue gaps of two symmetric positive definite matrices used in envelope modeling. We apply the new theory and algorithm to several envelope models, including response and predictor reduction in multivariate linear models, logistic regression, and Cox proportional hazard model. Simulations and illustrative data analysis show the potential for NIECE to improve standard methods in linear and generalized linear models significantly.
Cites work
- A Bayesian approach for envelope models
- A feasible method for optimization with orthogonality constraints
- A note on fast envelope estimation
- A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis
- A Review of Envelope Models
- A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
- A useful variant of the Davis-Kahan theorem for statisticians
- An introduction to envelopes. Dimension reduction for efficient estimation in multivariate statistics
- Coordinate-independent sparse sufficient dimension reduction and variable selection
- Envelope models for parsimonious and efficient multivariate linear regression
- Envelope quantile regression
- Envelope-based sparse partial least squares
- Envelopes and Partial Least Squares Regression
- Envelopes and reduced-rank regression
- Fast envelope algorithms
- Foundations for envelope models and methods
- High-dimensional analysis of semidefinite relaxations for sparse principal components
- scientific article; zbMATH DE number 4174160 (Why is no real title available?)
- scientific article; zbMATH DE number 41813 (Why is no real title available?)
- scientific article; zbMATH DE number 47995 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 5223994 (Why is no real title available?)
- scientific article; zbMATH DE number 3340880 (Why is no real title available?)
- Inner envelopes: efficient estimation in multivariate linear regression
- Matrix Variate Regressions and Envelope Models
- Minimax sparse principal subspace estimation in high dimensions
- Model-free envelope dimension selection
- Oracle inequalities for the lasso in the Cox model
- Partial common principal component subspaces
- Partial envelopes for efficient estimation in multivariate linear regression
- Partial least squares Cox regression for genome-wide data
- Partial least squares prediction in high-dimensional regression
- Partial likelihood
- Prediction by Supervised Principal Components
- Principal component analysis.
- Principal component analysis: a review and recent developments
- Principal envelope model
- Probabilistic Principal Component Analysis
- Reduced rank stochastic regression with a sparse singular value decomposition
- Reducing subspace models for large‐scale covariance regression
- Regularization for Cox's proportional hazards model with NP-dimensionality
- Ridge Regression: Biased Estimation for Nonorthogonal Problems
- Scaled envelopes: scale-invariant and efficient estimation in multivariate linear regression
- Selecting the number of components in principal component analysis using cross-validation approximations
- Shared subspace models for multi-group covariance estimation
- Sparse envelope model: efficient estimation and response variable selection in multivariate linear regression
- Sparse Partial Least Squares Regression for Simultaneous Dimension Reduction and Variable Selection
- Sparse PCA: optimal rates and adaptive estimation
- Sparse principal component analysis and iterative thresholding
- Sparse principal component analysis via regularized low rank matrix approximation
- Sparse reduced-rank regression for simultaneous dimension reduction and variable selection
- Supervised singular value decomposition and its asymptotic properties
- The Geometry of Algorithms with Orthogonality Constraints
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(4)
This page was built for publication: Envelopes and principal component regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6184884)
