Dimension reduction for predictive discrimination

DOI10.1016/0167-9473(86)90005-8MaRDI QIDQ1081252zbMATH OpenOpenAlexWikidataFDO

Authors Dean M. Young, Virgil R. Marco, P. L. Odell

Publication date 1986

Published in Computational Statistics and Data Analysis (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/0167-9473(86)90005-8

zbMATH Keywords

Bayesian posterior densitydimension-reduction methodlinear feature selectionMonte Carlo simulationmultivariate normalp-dimensional Student-type densitypattern recognitionpredictive discriminationprobability of correct classificationtraining data

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30)

Review text

Let the discrimination procedure for the classes \(\Pi_ i\) be based on the density functions \(P(x| \Pi_ i,\theta)\). Two alternative approaches are possible when the parameters \(\theta\) are unknown. The estimative approach preserves the form of the function but replaces \(\theta\) with some estimate \({\hat \theta}\). The predictive approach replaces \(P(x| \Pi_ i,\theta)\) by \[ P^*(x| \Pi_ i,z_ i)=\int_{\theta}P(x| \Pi_ i,\theta)P(\theta | z_ i)d\theta, \] where \(P(\theta | z_ i)\) is a Bayesian posterior density function and \(z_ i\) are the sets of training data. When \(P(x| \Pi_ i,\theta)\) is p-dimensional multivariate normal, \(P^*(x| \Pi_ i,z_ i)\) is a p-dimensional Student-type density function, \(X\sim St_ p(n_ i,\bar z_ i,S_ i)\) where \(S_ i\) is the usual sample covariance matrix. If B is a \(q\times p\) \((q<p)\) matrix of rank q, then \(Y=BX\sim St_ q(n,B\bar z,BSB')\). A method for constructing the linear dimension-reduction matrix is derived, and the procedure is proposed to indicate the lowest dimension that preserves the original expected probability of correct classification for each given case. This method avoids the inversion of high-dimensional matrices and allows to reduce the necessary size of samples. A Monte Carlo simulation is performed with three multivariate normal populations of dimension six. The predictive discrimination method in conjunction with the proposed dimension-reduction method yields results superior to the usual linear discrimination procedure as the last assumes equal population covariance matrices.

Recommendations

• Dimensionality reduction in quadratic discriminant analysis
• Component reduction in linear discriminant analysis
• Quadratic discrimination: Some results on optimal low-dimensional representation
• The Discrimination Subspace Model
• Dimensionality reduction approach to multivariate prediction

Cites work

• A Closer Look at Two Alternative Methods of Statistical Discrimination
• A model for dimension reduction in pattern recognition using continuous data
• Distance preserving linear feature selection
• Feature combinations and the bhattacharyya criterion
• Feature combinations and the divergence criterion
• Feature selection for best mean square approximation of class densities
• scientific article; zbMATH DE number 3656939 (Why is no real title available?)
• scientific article; zbMATH DE number 3787821 (Why is no real title available?)
• scientific article; zbMATH DE number 3513162 (Why is no real title available?)
• scientific article; zbMATH DE number 3205746 (Why is no real title available?)
• scientific article; zbMATH DE number 3244356 (Why is no real title available?)
• scientific article; zbMATH DE number 3288321 (Why is no real title available?)
• Linear dimension reduction and Bayes classification
• Linear dimension reduction and Bayes classification with unknown population parameters
• Linear feature selection with applications
• On minimizing the probability of misclassification for linear feature selection

Cited in

(16)

• Statistical classification and purposeful projection of observations into a space of dimensionality no more than the number of classes
• Quadratic discrimination: Some results on optimal low-dimensional representation
• Dimensionality reduction in quadratic discriminant analysis
• An entropic approach to dimensionality reduction on discrete processes
• Direct minimization of error rates in multivariate classification
• Local dimensionality reduction
• Probability-enhanced sufficient dimension reduction for binary classification
• Sufficient dimension reduction through informative predictor subspace
• scientific article; zbMATH DE number 3881706 (Why is no real title available?)
• Optimal linear feature selection for a general class of statistical pattern recognition models
• Reduced Rank Discrimination
• The Discrimination Subspace Model
• Component reduction in linear discriminant analysis
• scientific article; zbMATH DE number 1843024 (Why is no real title available?)
• On estimating the dimensionality in discriminant analysis
• A formulation and comparison of two linear feature selection techniques applicable to statistical classification

This page was built for publication: Dimension reduction for predictive discrimination

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1081252)