Optimal Feature Selection in High-Dimensional Discriminant Analysis

DOI10.1109/TIT.2014.2381241MaRDI QIDQ2978789zbMATH OpenOpenAlexWikidataFDO

Authors Mladen Kolar, Han Liu

Publication date 28 April 2017

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1306.6557

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Ridge regression; shrinkage estimators (Lasso) (62J07) Information theory (general) (94A15)

Abstract: We consider the high-dimensional discriminant analysis problem. For this problem, different methods have been proposed and justified by establishing exact convergence rates for the classification risk, as well as the l2 convergence results to the discriminative rule. However, sharp theoretical analysis for the variable selection performance of these procedures have not been established, even though model interpretation is of fundamental importance in scientific data analysis. This paper bridges the gap by providing sharp sufficient conditions for consistent variable selection using the sparse discriminant analysis (Mai et al., 2012). Through careful analysis, we establish rates of convergence that are significantly faster than the best known results and admit an optimal scaling of the sample size n, dimensionality p, and sparsity level s in the high-dimensional setting. Sufficient conditions are complemented by the necessary information theoretic limits on the variable selection problem in the context of high-dimensional discriminant analysis. Exploiting a numerical equivalence result, our method also establish the optimal results for the ROAD estimator (Fan et al., 2012) and the sparse optimal scaling estimator (Clemmensen et al., 2011). Furthermore, we analyze an exhaustive search procedure, whose performance serves as a benchmark, and show that it is variable selection consistent under weaker conditions. Extensive simulations demonstrating the sharpness of the bounds are also provided.

Cited in

(17)

• Tournament screening cum EBIC for feature selection with high-dimensional feature spaces
• Classification for high-throughput data with an optimal subset of principal components
• Optimal feature selection for sparse linear discriminant analysis and its applications in gene expression data
• Kick-one-out-based variable selection method for Euclidean distance-based classifier in high-dimensional settings
• Feature selection for high-dimensional data
• Prediction and estimation consistency of sparse multi-class penalized optimal scoring
• Toward a Fundamental Theory of Optimal Feature Selection: Part I
• scientific article; zbMATH DE number 1179950 (Why is no real title available?)
• Variable selection of linear programming discriminant estimator
• High-Dimensional Feature Selection by Feature-Wise Kernelized Lasso
• Nonparametric discriminant analysis with network structures in predictor
• Sparse quadratic classification rules via linear dimension reduction
• Optimal discriminant analysis in high-dimensional latent factor models
• A LINEAR-PROGRAMMING PORTFOLIO OPTIMIZER TO MEAN–VARIANCE OPTIMIZATION
• Asset splitting algorithm for ultrahigh dimensional portfolio selection and its theoretical property
• Optimal Linear Discriminant Analysis for High-Dimensional Functional Data
• Optimal variable selection in multi-group sparse discriminant analysis

This page was built for publication: Optimal Feature Selection in High-Dimensional Discriminant Analysis

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2978789)