High-Dimensional Classification by Sparse Logistic Regression

DOI10.1109/TIT.2018.2884963zbMATH Open1432.62177arXiv1706.08344OpenAlexW2963548292MaRDI QIDQ5223986FDOQ5223986

Authors: Vadim Grinshtein, Felix P. Abramovich 

Publication date: 19 July 2019

Published in: IEEE Transactions on Information Theory (Search for Journal in Brave)

Abstract: We consider high-dimensional binary classification by sparse logistic regression. We propose a model/feature selection procedure based on penalized maximum likelihood with a complexity penalty on the model size and derive the non-asymptotic bounds for the resulting misclassification excess risk. The bounds can be reduced under the additional low-noise condition. The proposed complexity penalty is remarkably related to the VC-dimension of a set of sparse linear classifiers. Implementation of any complexity penalty-based criterion, however, requires a combinatorial search over all possible models. To find a model selection procedure computationally feasible for high-dimensional data, we extend the Slope estimator for logistic regression and show that under an additional weighted restricted eigenvalue condition it is rate-optimal in the minimax sense.

Full work available at URL: https://arxiv.org/abs/1706.08344

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Generalized linear models (logistic models) (62J12)

Cited In (13)

• Large Scale Prediction with Decision Trees
• Optimal linear discriminators for the discrete choice model in growing dimensions
• On robust learning in the canonical change point problem under heavy tailed errors in finite and growing dimensions
• General feasibility bounds for sample average approximation via Vapnik-Chervonenkis dimension
• On the asymptotic properties of SLOPE
• High dimensional binary classification under label shift: phase transition and regularization
• A model of double descent for high-dimensional binary linear classification
• Classification of sparse high-dimensional vectors
• A two-stage sparse logistic regression for optimal gene selection in high-dimensional microarray data classification
• High-dimensional sparse classification using exponential weighting with empirical hinge loss
• Penalized principal logistic regression for sparse sufficient dimension reduction
• Global and Simultaneous Hypothesis Testing for High-Dimensional Logistic Regression Models
• Classification in high dimension with an Alternative Feature Augmentation via Nonparametrics and Selection (AFANS)

This page was built for publication: High-Dimensional Classification by Sparse Logistic Regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5223986)