High Dimensional Quadratic Discriminant Analysis: Optimality and Phase Transitions

arXiv2108.10802MaRDI QIDQ6375941FDOQ6375941

Authors: Wanjie Wang, Jingjing Wu, Zhigang Yao

Publication date: 24 August 2021

Abstract: Consider a two-class classification problem where we observe samples

(X_{i}, Y_{i})

for i = 1, ..., n,

X_{i} i n R^{p}

and

Y_{i}

in {0, 1}. Given

Y_{i} = k

,

X_{i}

is assumed to follow a multivariate normal distribution with mean

m u_{k} i n R^{k}

and covariance matrix

S i g m a_{k}

, k=0,1. Supposing a new sample X from the same mixture is observed, our goal is to estimate its class label Y. Such a high-dimensional classification problem has been studied thoroughly when Sigma_0 = Sigma_1. However, the discussions over the case

S i g m a_{0} e q S i g m a_{1}

are much less over the years. This paper presents the quadratic discriminant analysis (QDA) for the weak signals (QDAw) algorithm, and the QDA with feature selection (QDAfs) algorithm. QDAfs applies Partial Correlation Screening to estimate

h a t {O m e g a}_{0}

and

h a t {O m e g a}_{1}

, and then applies a hard-thresholding on the diagonals of

h a t {O m e g a}_{0} - h a t O m e g a_{1}

. QDAfs further includes the linear term

d^{T} X

, where d is achieved by a hard-thresholding on

h a t {O m e g a}_{1} h a t {m u}_{1} - h a t {O m e g a}_{0} h a t {m u}_{0}

. We further propose the rare and weak model to model the signals in

O m e g a_{0} - O m e g a_{1}

and

m u_{0} - m u_{1}

. Based on the signal weakness and sparsity in

m u_{0} - m u_{1}

, we propose two ways to estimate labels: 1) QDAw for weak but dense signals; 2) QDAfs for relatively strong but sparse signals. We figure out the classification boundary on the 4-dim parameter space: 1) Region of possibility, where either QDAw or QDAfs will achieve a mis-classification error rate of 0; 2) Region of impossibility, where all classifiers will have a constant error rate. The numerical results from real datasets support our theories and demonstrate the necessity and superiority of using QDA over LDA for classification.

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Asymptotic properties of parametric tests (62F05)

This page was built for publication: High Dimensional Quadratic Discriminant Analysis: Optimality and Phase Transitions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6375941)