Variable selection for general index models via sliced inverse regression

DOI10.1214/14-AOS1233zbMATH Open1305.62234arXiv1304.4056MaRDI QIDQ480962FDOQ480962

Authors: Bo Jiang, Jun S. Liu 

Publication date: 12 December 2014

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: Variable selection, also known as feature selection in machine learning, plays an important role in modeling high dimensional data and is key to data-driven scientific discoveries. We consider here the problem of detecting influential variables under the general index model, in which the response is dependent of predictors through an unknown function of one or more linear combinations of them. Instead of building a predictive model of the response given combinations of predictors, we model the conditional distribution of predictors given the response. This inverse modeling perspective motivates us to propose a stepwise procedure based on likelihood-ratio tests, which is effective and computationally efficient in identifying important variables without specifying a parametric relationship between predictors and the response. For example, the proposed procedure is able to detect variables with pairwise, three-way or even higher-order interactions among $p$ predictors with a computational time of $O(p)$ instead of $O(p^k)$ (with $k$ being the highest order of interactions). Its excellent empirical performance in comparison with existing methods is demonstrated through simulation studies as well as real data examples. Consistency of the variable selection procedure when both the number of predictors and the sample size go to infinity is established.

Full work available at URL: https://arxiv.org/abs/1304.4056

Recommendations

• Correlation pursuit: forward stepwise variable selection for index models
• Model-Free Variable Selection
• On marginal sliced inverse regression for ultrahigh dimensional model-free feature selection
• Variable selection for single-index varying-coefficient model
• Forward selection and estimation in high dimensional single index models

zbMATH Keywords

variable selectioninteractionssliced inverse regressionsure independence screeninginverse models

Mathematics Subject Classification ID

Factor analysis and principal components; correspondence analysis (62H25) General nonlinear regression (62J02) Applications of statistics to biology and medical sciences; meta analysis (62P10)

Cites Work

• A lasso for hierarchical interactions
• The Adaptive Lasso and Its Oracle Properties
• Least angle regression. (With discussion)
• Pathwise coordinate optimization
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Title not available (Why is that?)
• Testing predictor contributions in sufficient dimension reduction.
• Sliced Inverse Regression for Dimension Reduction
• Title not available (Why is that?)
• Feature screening via distance correlation learning
• Fisher lecture: Dimension reduction in regression
• Model-Free Variable Selection
• Standardization and the group lasso penalty
• Variable selection for general index models via sliced inverse regression
• Sparse sufficient dimension reduction
• The sliced inverse regression algorithm as a maximum likelihood procedure
• Selection of Subsets of Regression Variables
• Variable selection and updating in model-based discriminant analysis for high dimensional data with food authenticity applications

Cited In (24)

• On consistency and sparsity for sliced inverse regression in high dimensions
• High-Dimensional Interaction Detection With False Sign Rate Control
• Robust Variable and Interaction Selection for Logistic Regression and General Index Models
• Title not available (Why is that?)
• Title not available (Why is that?)
• Estimating a sparse reduction for general regression in high dimensions
• A Generalized Levene's Scale Test for Variance Heterogeneity in the Presence of Sample Correlation and Group Uncertainty
• Estimation and inference in generalized additive coefficient models for nonlinear interactions with high-dimensional covariates
• Quantile Correlation-based Variable Selection
• On dual model-free variable selection with two groups of variables
• Model Selection for High-Dimensional Quadratic Regression via Regularization
• Misspecified nonconvex statistical optimization for sparse phase retrieval
• Data-guided Treatment Recommendation with Feature Scores
• Penalized Interaction Estimation for Ultrahigh Dimensional Quadratic Regression
• Sliced-Inverse-Regression--Aided Rotated Compressive Sensing Method for Uncertainty Quantification
• Innovated interaction screening for high-dimensional nonlinear classification
• Trace pursuit variable selection for multi-population data
• Estimation of Optimal Individualized Treatment Rules Using a Covariate-Specific Treatment Effect Curve With High-Dimensional Covariates
• Dimension reduction for block-missing data based on sparse sliced inverse regression
• RaSE: A Variable Screening Framework via Random Subspace Ensembles
• Projections of a general binary model on a logistic regression
• Sparse Sliced Inverse Regression Via Lasso
• Title not available (Why is that?)
• Variable selection for general index models via sliced inverse regression

Uses Software

• hierNet
• RSIR

This page was built for publication: Variable selection for general index models via sliced inverse regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q480962)