Genetic algorithm for feature selection for parallel classifiers (Q685521)

Parallel classification is a pattern recognition paradigm which resembles the process of group decision making. The scheme consists of \(r\) first- level decision makers (operating on different subsets of features) and a second-level ``aggregator'' of their classification decisions. The complication of the structure aims rather at a higher classification accuracy than at facilitating the computation process. It can be formally proven that the voting scheme of independent classifiers outperforms the best one of them. Let X be the set of \(n\) features describing the objects. The features selection task for conventional (one level) classifier consists in choosing the best subset of \(X\) in terms of a certain classification criterion. The main difficulty in feature selection is that the searching surface is usually multimodal, and the criterion can hardly be described analytically. This fact precludes the application of the classical searching tools based on criterion derivatives. The problem of feature selection becomes even more cumbersome in the parallel classification scheme because a set of subsets \(S=\{S_ 1,...,S_ r\}\), \(S_ i\subseteq X\), is to be chosen instead of a single subset. Possessing a variety of assets, genetic algorithms (GA) have demonstrated good performance in different optimization fields. On the contrast to many trivial optimization techniques, genetic algorithms can operate on discontinuous, noisy, multidimensional and multimodal surfaces. The feature selection task fits easily GA frameworks because a subset of \(X\) can be encoded as an \(n\)-size chromosome containing `1' at the \(i\)-th position if the respective feature is in the subset, and `0', otherwise. A ``fit'' criterion \(J(S_ k)\) for evaluating the chromosomes is defined which in this case may be a measure of discrimination power or classification accuracy of the respective subset. We consider a voting two-level scheme with three first-level classifiers obtaining different feature sets as inputs and performing the same classification rule. A modified criterion is suggested with a variant of the so called `elitist' model of combination. The criterion evaluates the accuracy of the two-level classifier for all combinations of three chromosomes from the current population set. The best triple along with the winners from the rest of the population and offsprings survive for the renewed population set. Three medical data sets were used to test the proposed feature selection paradigm: data for duodenal ulcer of 77 patients, data for hyaline membrane disease for 78 preterm newborn infants, and data for hypoxic resistance of 200 healthy male pilots examined in hypobaric chamber. A comparison with a plain random search and genetic algorithm which uses only classification accuracy as a criterion was performed. The results showed that the GA with the modified criterion surpasses the other two strategies for the purposes of feature selection for parallel classifiers.