Combining clustering of variables and feature selection using random forests

DOI10.1080/03610918.2018.1563145MaRDI QIDQ5083993zbMATH OpenOpenAlexFDO

Authors Marie Chavent, Robin Genuer, Jérôme Saracco

Publication date 21 June 2022

Published in Communications in Statistics. Simulation and Computation (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1608.06740

zbMATH Keywords

random forestsvariable selectionsupervised classificationclustering of variables

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: Standard approaches to tackle high-dimensional supervised classification problem often include variable selection and dimension reduction procedures. The novel methodology proposed in this paper combines clustering of variables and feature selection. More precisely, hierarchical clustering of variables procedure allows to build groups of correlated variables in order to reduce the redundancy of information and summarizes each group by a synthetic numerical variable. Originality is that the groups of variables (and the number of groups) are unknown a priori. Moreover the clustering approach used can deal with both numerical and categorical variables (i.e. mixed dataset). Among all the possible partitions resulting from dendrogram cuts, the most relevant synthetic variables (i.e. groups of variables) are selected with a variable selection procedure using random forests. Numerical performances of the proposed approach are compared with direct applications of random forests and variable selection using random forests on the original p variables. Improvements obtained with the proposed methodology are illustrated on two simulated mixed datasets (cases n>p and n<p, where n is the sample size) and on a real proteomic dataset. Via the selection of groups of variables (based on the synthetic variables), interpretability of the results becomes easier.

Recommendations

• Supervised clustering of variables
• scientific article; zbMATH DE number 5251644
• Variable selection for clustering and classification
• Variable Selection for Model-Based High-Dimensional Clustering and Its Application to Microarray Data
• A new variable selection approach using random forests

Cites work

• scientific article; zbMATH DE number 3860199 (Why is no real title available?)
• scientific article; zbMATH DE number 1348600 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• 10.1162/153244303322753616
• Bayesian Variable Selection in Clustering High-Dimensional Data
• ClustGeo: an R package for hierarchical clustering with spatial constraints
• Do we need hundreds of classifiers to solve real world classification problems?
• Finding Groups in Data
• Model Selection and Estimation in Regression with Grouped Variables
• Orthogonal rotation in PCAMIX
• Principal component analysis.
• Random forests
• Regularization and Variable Selection Via the Elastic Net
• Selection bias in gene extraction on the basis of microarray gene-expression data
• Simple structure in component analysis techniques for mixtures of qualitative and quantitative variables
• Sparse Partial Least Squares Regression for Simultaneous Dimension Reduction and Variable Selection
• Variable Selection for Clustering with Gaussian Mixture Models
• Variable Selection for Model-Based Clustering

Cited in

(2)

• Interval selection: a case-study-based approach
• Dendrograms, minimum spanning trees and feature selection

Describes a project that uses

Uses Software

• R
• glasso
• randomForest
• PCAmixdata
• mixOmics
• ClustOfVar
• clusfind
• VSURF

This page was built for publication: Combining clustering of variables and feature selection using random forests

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5083993)