High-Dimensional Classification by Sparse Logistic Regression

DOI10.1109/TIT.2018.2884963MaRDI QIDQ5223986zbMATH OpenOpenAlexFDO

Authors Vadim Grinshtein, Felix P. Abramovich

Publication date 19 July 2019

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1706.08344

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Generalized linear models (logistic models) (62J12)

Abstract: We consider high-dimensional binary classification by sparse logistic regression. We propose a model/feature selection procedure based on penalized maximum likelihood with a complexity penalty on the model size and derive the non-asymptotic bounds for the resulting misclassification excess risk. The bounds can be reduced under the additional low-noise condition. The proposed complexity penalty is remarkably related to the VC-dimension of a set of sparse linear classifiers. Implementation of any complexity penalty-based criterion, however, requires a combinatorial search over all possible models. To find a model selection procedure computationally feasible for high-dimensional data, we extend the Slope estimator for logistic regression and show that under an additional weighted restricted eigenvalue condition it is rate-optimal in the minimax sense.

Cited in

(13)

• Large Scale Prediction with Decision Trees
• A two-stage sparse logistic regression for optimal gene selection in high-dimensional microarray data classification
• A model of double descent for high-dimensional binary linear classification
• Classification in high dimension with an Alternative Feature Augmentation via Nonparametrics and Selection (AFANS)
• Penalized principal logistic regression for sparse sufficient dimension reduction
• On robust learning in the canonical change point problem under heavy tailed errors in finite and growing dimensions
• Classification of sparse high-dimensional vectors
• High dimensional binary classification under label shift: phase transition and regularization
• General feasibility bounds for sample average approximation via Vapnik-Chervonenkis dimension
• Optimal linear discriminators for the discrete choice model in growing dimensions
• On the asymptotic properties of SLOPE
• Global and Simultaneous Hypothesis Testing for High-Dimensional Logistic Regression Models
• High-dimensional sparse classification using exponential weighting with empirical hinge loss

This page was built for publication: High-Dimensional Classification by Sparse Logistic Regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5223986)