Coordinate-independent sparse sufficient dimension reduction and variable selection

DOI10.1214/10-AOS826zbMATH Open1204.62107arXiv1211.3215MaRDI QIDQ620565FDOQ620565

Authors: Xin Chen, Changliang Zou, R. Dennis Cook 

Publication date: 19 January 2011

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: Sufficient dimension reduction (SDR) in regression, which reduces the dimension by replacing original predictors with a minimal set of their linear combinations without loss of information, is very helpful when the number of predictors is large. The standard SDR methods suffer because the estimated linear combinations usually consist of all original predictors, making it difficult to interpret. In this paper, we propose a unified method - coordinate-independent sparse estimation (CISE) - that can simultaneously achieve sparse sufficient dimension reduction and screen out irrelevant and redundant variables efficiently. CISE is subspace oriented in the sense that it incorporates a coordinate-independent penalty term with a broad series of model-based and model-free SDR approaches. This results in a Grassmann manifold optimization problem and a fast algorithm is suggested. Under mild conditions, based on manifold theories and techniques, it can be shown that CISE would perform asymptotically as well as if the true irrelevant predictors were known, which is referred to as the oracle property. Simulation studies and a real-data example demonstrate the effectiveness and efficiency of the proposed approach.

Full work available at URL: https://arxiv.org/abs/1211.3215

Recommendations

• Sparse sufficient dimension reduction
• Sparse sufficient dimension reduction using optimal scoring
• Sparse minimum discrepancy approach to sufficient dimension reduction with simultaneous variable selection in ultrahigh dimension
• On the consistency of coordinate-independent sparse estimation with BIC
• Estimating a sparse reduction for general regression in high dimensions

zbMATH Keywords

variable selectioncentral subspacesufficient dimension reductionoracle propertyGrassmann manifoldsCISE

Mathematics Subject Classification ID

Multivariate analysis (62H99) Estimation in multivariate analysis (62H12) Ridge regression; shrinkage estimators (Lasso) (62J07) Eigenvalues, singular values, and eigenvectors (15A18)

Cites Work

• The Adaptive Lasso and Its Oracle Properties
• Ideal spatial adaptation by wavelet shrinkage
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Testing predictor contributions in sufficient dimension reduction.
• Sliced Inverse Regression for Dimension Reduction
• A note on shrinkage sliced inverse regression
• Model Selection and Estimation in Regression with Grouped Variables
• The Geometry of Algorithms with Orthogonality Constraints
• On consistency and sparsity for principal components analysis in high dimensions
• Tuning parameter selectors for the smoothly clipped absolute deviation method
• Sufficient dimension reduction in regressions with categorical predictors
• Contour regression: a general approach to dimension reduction
• On Directional Regression for Dimension Reduction
• Title not available (Why is that?)
• On the Interpretation of Regression Plots
• Fisher lecture: Dimension reduction in regression
• Likelihood-based sufficient dimension reduction
• Model-Free Variable Selection
• Principal fitted components for dimension reduction in regression
• Smooth nonlinear optimization of \(\mathbb R^n\)
• Dimension reduction for conditional mean in regression
• On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another Application of Stein's Lemma
• Dimension Reduction for the Conditionalkth Moment in Regression
• Sparse sufficient dimension reduction
• Asymptotics for sliced average variance estimation
• Title not available (Why is that?)
• Principal Hessian Directions Revisited
• Regression Model Selection—A Residual Likelihood Approach
• On minimization on Stiefel manifolds
• Dimension reduction based on constrained canonical correlation and variable filtering
• Optimization algorithms exploiting unitary constraints
• Title not available (Why is that?)
• Title not available (Why is that?)

Cited In (78)

• Dimension reduction regressions with measurement errors subject to additive distortion
• Estimation and variable selection in partial linear single index models with error-prone linear covariates
• MM algorithms for distance covariance based sufficient dimension reduction and sufficient variable selection
• Model averaging assisted sufficient dimension reduction
• Model-free variable selection for conditional mean in regression
• On expectile-assisted inverse regression estimation for sufficient dimension reduction
• Simultaneous estimation for semi-parametric multi-index models
• Nonlinear surface regression with dimension reduction method
• Ranking the importance of variables in nonlinear system identification
• Semiparametric model average prediction in panel data analysis
• Sufficient dimension reduction using Hilbert-Schmidt independence criterion
• A selective overview of sparse sufficient dimension reduction
• Sufficient dimension folding via tensor inverse regression
• Dimension reduction based on conditional multiple index density function
• A sequential test for variable selection in high dimensional complex data
• Sparse sufficient dimension reduction using optimal scoring
• The effect of data contamination in sliced inverse regression and finite sample breakdown point
• Dimension reduction via adaptive slicing
• Supervised dimension reduction for ordinal predictors
• Sufficient dimension reduction via random-partitions for the large-\(p\)-small-\(n\) problem
• Variable selection and estimation for semi-parametric multiple-index models
• Covariate information matrix for sufficient dimension reduction
• Feature filter for estimating central mean subspace and its sparse solution
• Sparse minimum discrepancy approach to sufficient dimension reduction with simultaneous variable selection in ultrahigh dimension
• The maximum separation subspace in sufficient dimension reduction with categorical response
• Bayesian model averaging sliced inverse regression
• Learning sparse gradients for variable selection and dimension reduction
• Sparse SIR: optimal rates and adaptive estimation
• On post dimension reduction statistical inference
• Estimating a sparse reduction for general regression in high dimensions
• QUADRO: a supervised dimension reduction method via Rayleigh quotient optimization
• Sufficient dimension reduction for censored predictors
• On the consistency of coordinate-independent sparse estimation with BIC
• Sparse sufficient dimension reduction for Markov blanket discovery
• Sufficient dimension reduction in multivariate regressions with categorical predictors
• Robust sufficient dimension reduction via ball covariance
• Penalized Orthogonal Iteration for Sparse Estimation of Generalized Eigenvalue Problem
• On the single-index model estimate of the conditional density function: consistency and implementation
• Dimension-wise sparse low-rank approximation of a matrix with application to variable selection in high-dimensional integrative analyzes of association
• A dimension reduction based approach for estimation and variable selection in partially linear single-index models with high-dimensional covariates
• An ensemble of inverse moment estimators for sufficient dimension reduction
• Some aspects of response variable selection and estimation in multivariate linear regression
• Envelope-based sparse partial least squares
• A model-free variable selection method for reducing the number of redundant variables
• Trace pursuit variable selection for multi-population data
• On the conditional distributions of low-dimensional projections from high-dimensional data
• Partial linear single index models with distortion measurement errors
• Sparsity considerations for dependent variables
• Dimension reduction for block-missing data based on sparse sliced inverse regression
• Concordance-based estimation approaches for the optimal sufficient dimension reduction score
• On Partial Sufficient Dimension Reduction With Applications to Partially Linear Multi-Index Models
• A distribution-based Lasso for a general single-index model
• Sharp variable selection of a sparse submatrix in a high-dimensional noisy matrix
• Fourier transform sparse inverse regression estimators for sufficient variable selection
• A Review on Sliced Inverse Regression, Sufficient Dimension Reduction, and Applications
• Sparse sufficient dimension reduction
• Minimax sparse principal subspace estimation in high dimensions
• Variable selection using data splitting and projection for principal fitted component models in high dimension
• Subspace Estimation with Automatic Dimension and Variable Selection in Sufficient Dimension Reduction
• [HDDA] sparse subspace constrained partial least squares
• Sparse dimension reduction based on energy and ball statistics
• Testing the Linear Mean and Constant Variance Conditions in Sufficient Dimension Reduction
• Response variable selection in multivariate linear regression
• Variable-dependent partial dimension reduction
• Envelopes and principal component regression
• A Review of Envelope Models
• High-Dimensional Elliptical Sliced Inverse Regression in Non-Gaussian Distributions
• High-dimensional local polynomial regression with variable selection and dimension reduction
• New forest-based approaches for sufficient dimension reduction
• Slicing-free inverse regression in high-dimensional sufficient dimension reduction
• Exact penalty function for \(\ell_{2,1}\) norm minimization over the Stiefel manifold
• Higher-order sliced inverse regressions
• Matching Using Sufficient Dimension Reduction for Causal Inference
• Pruning a sufficient dimension reduction with a \(p\)-value guided hard-thresholding
• A penalty-free infeasible approach for a class of nonsmooth optimization problems over the Stiefel manifold
• High-dimensional sparse single-index regression via Hilbert-Schmidt independence criterion
• First-order algorithms for a class of fractional optimization problems
• Efficient Sparse Estimate of Sufficient Dimension Reduction in High Dimension

This page was built for publication: Coordinate-independent sparse sufficient dimension reduction and variable selection

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q620565)