Robust Variable and Interaction Selection for Logistic Regression and General Index Models

DOI10.1080/01621459.2017.1401541zbMATH Open1478.62170arXiv1611.08649OpenAlexW2706069792WikidataQ99629429 ScholiaQ99629429MaRDI QIDQ5229910FDOQ5229910

Authors: Yang Li, Jun S. Liu 

Publication date: 19 August 2019

Published in: Journal of the American Statistical Association (Search for Journal in Brave)

Abstract: We propose Stepwise cOnditional likelihood variable selection for Discriminant Analysis (SODA) to detect both main and quadratic interaction effects in logistic regression and quadratic discriminant analysis (QDA) models. In the forward stage, SODA adds in important predictors evaluated based on their overall contributions, whereas in the backward stage SODA removes unimportant terms so as to optimize the extended Bayesian Information Criterion (EBIC). Compared with existing methods on QDA variable selections, SODA can deal with high-dimensional data with the number of predictors much larger than the sample size and does not require the joint normality assumption on predictors, leading to much enhanced robustness. We further extend SODA to conduct variable selection and model fitting for multiple index models. Compared with existing variable selection methods based on the Sliced Inverse Regression (SIR) (Li 1991), SODA requires neither the linearity nor the constant variance condition and is much more robust. Our theoretical analyses establish the variable-selection consistency of SODA under high-dimensional settings, and our simulation studies as well as real-data applications demonstrate superior performances of SODA in dealing with non-Gaussian design matrices in both classification problems and multiple index models.

Full work available at URL: https://arxiv.org/abs/1611.08649

Recommendations

• Variable selection for general index models via sliced inverse regression
• Robust variable selection in the logistic regression model
• Robust direction identification and variable selection in high dimensional general single-index models
• scientific article; zbMATH DE number 6468204
• Model-Free Variable Selection

zbMATH Keywords

classificationsemiparametrichigh-dimensionalquadratic discriminant analysisstepwise selectionforward screening

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Generalized linear models (logistic models) (62J12) Robustness and adaptive procedures (parametric inference) (62F35)

Cites Work

• Regularized linear discriminant analysis and its application in microarrays
• Penalized classification using Fisher's linear discriminant
• A lasso for hierarchical interactions
• Estimating the dimension of a model
• Extended Bayesian information criteria for model selection with large model spaces
• Title not available (Why is that?)
• Random forests
• Variable selection in model-based discriminant analysis
• On consistency and sparsity for sliced inverse regression in high dimensions
• Title not available (Why is that?)
• Sliced Inverse Regression for Dimension Reduction
• Feature screening via distance correlation learning
• Regularization and Variable Selection Via the Elastic Net
• High-dimensional classification using features annealed independence rules
• Extended BIC for small-\(n\)-large-\(P\) sparse GLM
• A direct approach to sparse discriminant analysis in ultra-high dimensions
• A direct estimation approach to sparse linear discriminant analysis
• Title not available (Why is that?)
• Sparse linear discriminant analysis by thresholding for high dimensional data
• High-dimensional variable selection
• High-dimensional Ising model selection using \(\ell _{1}\)-regularized logistic regression
• Fisher lecture: Dimension reduction in regression
• A Model Selection Approach for the Identification of Quantitative Trait Loci in Experimental Crosses
• Model-Free Variable Selection
• Large-scale inference. Empirical Bayes methods for estimation, testing, and prediction
• CODA: high dimensional copula discriminant analysis
• Variable selection for general index models via sliced inverse regression
• Sparse sufficient dimension reduction
• Correlation Pursuit: Forward Stepwise Variable Selection for Index Models
• Optimal classification in sparse Gaussian graphic model
• Forward regression for ultra-high dimensional variable screening
• On BIC's selection consistency for discriminant analysis
• On model selection consistency of the elastic net when \(p \gg n\)
• Empirical Bayes estimates for large-scale prediction problems
• The sliced inverse regression algorithm as a maximum likelihood procedure
• Variable selection and updating in model-based discriminant analysis for high dimensional data with food authenticity applications
• Innovated interaction screening for high-dimensional nonlinear classification
• A Permutation Approach to Testing Interactions for Binary Response by Comparing Correlations Between Classes

Cited In (12)

• The Kendall interaction filter for variable interaction screening in high dimensional classification problems
• Robust variable selection with application to quality of life research
• Structure learning via unstructured kernel-based M-estimation
• An efficient model-free approach to interaction screening for high dimensional data
• Interaction screening for high-dimensional heterogeneous data via robust hybrid metrics
• BOLT-SSI: A Statistical Approach to Screening Interaction Effects for Ultra-High Dimensional Data
• Sparse Learning and Structure Identification for Ultrahigh-Dimensional Image-on-Scalar Regression
• Interaction screening via Kendall's rank correlation for imbalanced multi-class classification
• Title not available (Why is that?)
• RaSE: A Variable Screening Framework via Random Subspace Ensembles
• Unified model-free interaction screening via CV-entropy filter
• Robust variable selection in the logistic regression model

Uses Software

• rda
• hierNet
• penalizedLDA

This page was built for publication: Robust Variable and Interaction Selection for Logistic Regression and General Index Models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5229910)