On the dimension effect of regularized linear discriminant analysis

DOI10.1214/18-EJS1469zbMATH Open1404.62070arXiv1710.03136OpenAlexW2762471879WikidataQ129249731 ScholiaQ129249731MaRDI QIDQ1786573FDOQ1786573

Authors: Cheng Wang, Binyan Jiang 

Publication date: 24 September 2018

Published in: Electronic Journal of Statistics (Search for Journal in Brave)

Abstract: This paper studies the dimension effect of the linear discriminant analysis (LDA) and the regularized linear discriminant analysis (RLDA) classifiers for large dimensional data where the observation dimension $p$ is of the same order as the sample size $n$. More specifically, built on properties of the Wishart distribution and recent results in random matrix theory, we derive explicit expressions for the asymptotic misclassification errors of LDA and RLDA respectively, from which we gain insights of how dimension affects the performance of classification and in what sense. Motivated by these results, we propose adjusted classifiers by correcting the bias brought by the unequal sample sizes. The bias-corrected LDA and RLDA classifiers are shown to have smaller misclassification rates than LDA and RLDA respectively. Several interesting examples are discussed in detail and the theoretical results on dimension effect are illustrated via extensive simulation studies.

Full work available at URL: https://arxiv.org/abs/1710.03136

Recommendations

• A modified linear discriminant analysis for high-dimensional data
• Asymptotic properties of the EPMC for modified linear discriminant analysis when sample size and dimension are both large
• Computation of regularized linear discriminant analysis
• Discriminant analysis in small and large dimensions
• The Dantzig discriminant analysis with high dimensional data

zbMATH Keywords

linear discriminant analysisrandom matrix theoryWishart distributionasymptotic misclassification errorsdimension effectregularized linear discriminant analysis

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Asymptotic distribution theory in statistics (62E20) Random matrices (probabilistic aspects) (60B20)

Cites Work

• Regularized linear discriminant analysis and its application in microarrays
• Title not available (Why is that?)
• Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data
• Spectral analysis of large dimensional random matrices
• Some theory for Fisher's linear discriminant function, `naive Bayes', and some alternatives when there are many more variables than observations
• A direct approach to sparse discriminant analysis in ultra-high dimensions
• A direct estimation approach to sparse linear discriminant analysis
• Statistical significance in high-dimensional linear models
• Sparse linear discriminant analysis by thresholding for high dimensional data
• DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES
• High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: risk underestimation
• A constrained \(\ell _{1}\) minimization approach to sparse precision matrix estimation
• Title not available (Why is that?)
• Spectrum estimation for large dimensional covariance matrices using random matrix theory
• Classifier technology and the illusion of progress
• Title not available (Why is that?)
• Two-Sample Test of High Dimensional Means Under Dependence
• A Road to Classification in High Dimensional Space: The Regularized Optimal Affine Discriminant
• Eigenvectors of some large sample covariance matrix ensembles
• Asymptotic expansion of the misclassification probabilities of D- and A- criteria for discrimination from two high dimensional populations using the theory of large dimensional random matrices
• A regularized Hotelling's \(T^{2}\) test for pathway analysis in proteomic studies
• High Dimensional Ordinary Least Squares Projection for Screening Variables
• Central limit theorem for Hotelling's \(T^{2}\) statistic under large dimension
• On asymptotics of eigenvectors of large sample covariance matrix
• Shrinkage estimation of large dimensional precision matrix using random matrix theory
• Estimation of the precision matrix of a singular Wishart distribution and its application in high-dimensional data
• Asymptotic probabilities of misclassification of two discriminant functions in cases of high dimensional data
• On two simple and effective procedures for high dimensional classification of general populations
• Enhancement of the applicability of Markowitz's portfolio optimization by utilizing random matrix theory
• Central limit theorems for classical likelihood ratio tests for high-dimensional normal distributions
• Integration with respect to the Haar measure on unitary, orthogonal and symplectic group
• General moments of the inverse real Wishart distribution and orthogonal Weingarten functions
• Bias-corrected diagonal discriminant rules for high-dimensional classification
• On the mean and variance of the generalized inverse of a singular Wishart matrix
• Scale adjustments for classifiers in high-dimensional, low sample size settings
• A distance-based, misclassification rate adjusted classifier for multiclass, high-dimensional data
• Asymptotic properties of eigenmatrices of a large sample covariance matrix
• A Closer Look at Two Alternative Methods of Statistical Discrimination
• Generalized Consistent Error Estimator of Linear Discriminant Analysis
• High-dimensional asymptotics of prediction: ridge regression and classification

Cited In (9)

• Title not available (Why is that?)
• Bias-corrected diagonal discriminant rules for high-dimensional classification
• The effect of intrinsic dimension on the Bayes-error of projected quadratic discriminant classification
• High dimensional discriminant rules with shrinkage estimators of the covariance matrix and mean vector
• Do unbalanced data have a negative effect on LDA?
• High dimensional binary classification under label shift: phase transition and regularization
• Dimension Reduction with Linear Discriminant Functions Based on an Odds Ratio Parameterization
• Generalized two-dimensional linear discriminant analysis with regularization
• Asymptotic properties of the EPMC for modified linear discriminant analysis when sample size and dimension are both large

Uses Software

• rda

This page was built for publication: On the dimension effect of regularized linear discriminant analysis

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1786573)