Random-projection ensemble classification. (With discussion).
From MaRDI portal
Publication:5364897
DOI10.1111/RSSB.12228zbMATH Open1373.62301arXiv1504.04595OpenAlexW2105234758MaRDI QIDQ5364897FDOQ5364897
Authors: Timothy I. Cannings, Richard Samworth 
Publication date: 29 September 2017
Published in: Journal of the Royal Statistical Society Series B: Statistical Methodology (Search for Journal in Brave)
Abstract: We introduce a very general method for high-dimensional classification, based on careful combination of the results of applying an arbitrary base classifier to random projections of the feature vectors into a lower-dimensional space. In one special case that we study in detail, the random projections are divided into disjoint groups, and within each group we select the projection yielding the smallest estimate of the test error. Our random projection ensemble classifier then aggregates the results of applying the base classifier on the selected projections, with a data-driven voting threshold to determine the final assignment. Our theoretical results elucidate the effect on performance of increasing the number of projections. Moreover, under a boundary condition implied by the sufficient dimension reduction assumption, we show that the test excess risk of the random projection ensemble classifier can be controlled by terms that do not depend on the original data dimension and a term that becomes negligible as the number of projections increases. The classifier is also compared empirically with several other popular high-dimensional classifiers via an extensive simulation study, which reveals its excellent finite-sample performance.
Full work available at URL: https://arxiv.org/abs/1504.04595
Recommendations
- Random projections as regularizers: learning a linear discriminant from fewer observations than dimensions
- A weighted multiple classifier framework based on random projection
- Randomized nonlinear projections uncover high-dimensional structure
- Stability of random-projection based classifiers. The Bayes error perspective
- Random Projection Ensemble Classification with High-Dimensional Time Series
Classification and discrimination; cluster analysis (statistical aspects) (62H30) Applications of statistics to biology and medical sciences; meta analysis (62P10)
Cited In (47)
- A weighted multiple classifier framework based on random projection
- Pattern recognition via projection-based \(k\)NN rules
- Classification by ensembles from random partitions of high-dimensional data
- Random projections as regularizers: learning a linear discriminant from fewer observations than dimensions
- Asymptotics, finite-sample comparisons and applications for two-sample tests with functional data
- Ensemble of a subset of \(k\)NN classifiers
- On principal components regression, random projections, and column subsampling
- A multivariate adaptive stochastic search method for dimensionality reduction in classification
- Reduced rank regression with matrix projections for high-dimensional multivariate linear regression model
- Estimating a sharp convergence bound for randomized ensembles
- Fuzzy \(c\)-means and cluster ensemble with random projection for big data clustering
- Random embedding machines for pattern recognition
- Advance of the sufficient dimension reduction
- Title not available (Why is that?)
- An Outer-Product-of-Gradient Approach to Dimension Reduction and its Application to Classification in High Dimensional Space
- Regularizing axis-aligned ensembles via data rotations that favor simpler learners
- Robust ranking by ensembling of diverse models and assessment metrics
- High dimensional classification for spatially dependent data with application to neuroimaging
- Comments on: ``A random forest guided tour
- Covariance matrix testing in high dimension using random projections
- Statistical embedding: beyond principal components
- Comprehensive study of variational Bayes classification for dense deep neural networks
- On the fusion of threshold classifiers for categorization and dimensionality reduction
- Multiclass sparse discriminant analysis incorporating graphical structure among predictors
- Ensemble quantile classifier
- Ensembles of least squares classifiers with randomized kernels
- Ensemble Subset Regression (ENSURE): Efficient High-dimensional Prediction
- Title not available (Why is that?)
- Ensemble of optimal trees, random forest and random projection ensemble classification
- Random Projection Ensemble Classification with High-Dimensional Time Series
- Random forest kernel for high-dimension low sample size classification
- Quadratic discriminant analysis by projection
- Stability of random-projection based classifiers. The Bayes error perspective
- Ensemble LDA via the modified Cholesky decomposition
- A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems
- Targeted random projection for prediction from high-dimensional features
- Stabilizing variable selection and regression
- Bioinspired Random Projections for Robust, Sparse Classification
- Predictive performances of implicitly and explicitly robust classifiers on high dimensional data
- Measuring the algorithmic convergence of randomized ensembles: the regression setting
- RaSE: A Variable Screening Framework via Random Subspace Ensembles
- Learning from high dimensional data based on weighted feature importance in decision tree ensembles
- Estimating the algorithmic variance of randomized ensembles via the bootstrap
- High-dimensional clustering via random projections
- Variable selection using axis-aligned random projections for partial least-squares regression
- Title not available (Why is that?)
- Metric Learning via Cross-Validation
This page was built for publication: Random-projection ensemble classification. (With discussion).
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5364897)
