On the asymptotics of random forests
From MaRDI portal
Abstract: The last decade has witnessed a growing interest in random forest models which are recognized to exhibit good practical performance, especially in high-dimensional settings. On the theoretical side, however, their predictive power remains largely unexplained, thereby creating a gap between theory and practice. The aim of this paper is twofold. Firstly, we provide theoretical guarantees to link finite forests used in practice (with a finite number M of trees) to their asymptotic counterparts. Using empirical process theory, we prove a uniform central limit theorem for a large class of random forest estimates, which holds in particular for Breiman's original forests. Secondly, we show that infinite forest consistency implies finite forest consistency and thus, we state the consistency of several infinite forests. In particular, we prove that q quantile forests---close in spirit to Breiman's forests but easier to study---are able to combine inconsistent trees to obtain a final consistent prediction, thus highlighting the benefits of random forests compared to single trees.
Recommendations
Cites work
- A distribution-free theory of nonparametric regression
- An empirical comparison of ensemble methods based on classification trees
- Analysis of a random forests model
- Bagging predictors
- Concentration inequalities. A nonasymptotic theory of independence
- Consistency of random forests
- Consistency of random forests and other averaging classifiers
- Consistency of random survival forests
- Consistent nonparametric regression. Discussion
- Contributions in infinite-dimensional statistics and related topics. Selected papers from the 3rd international workshop on functional and operatorial statistics (IWFOS'2014), Stresa, Italy, June 19--21, 2014
- Extremely randomized trees
- Functional data analysis
- scientific article; zbMATH DE number 6378123 (Why is no real title available?)
- scientific article; zbMATH DE number 3860199 (Why is no real title available?)
- scientific article; zbMATH DE number 893887 (Why is no real title available?)
- Inference for functional data with applications
- Nonparametric functional data analysis. Theory and practice.
- Quantile regression forests
- Random forests
- Random survival forests
- Ranking forests
- Weak convergence and empirical processes. With applications to statistics
Cited in
(50)- Coalescent random forests
- Bootstrap bias corrections for ensemble methods
- Towards convergence rate analysis of random forests for classification
- Models under which random forests perform badly; consequences for applications
- Random forest estimation of conditional distribution functions and conditional quantiles
- Asymptotic properties of high-dimensional random forests
- Rates of convergence for random forests via generalized U-statistics
- Modeling of time series using random forests: theoretical developments
- Minimax optimal rates for Mondrian trees and forests
- On PAC-Bayesian bounds for random forests
- Consistent estimation of residual variance with random forest out-of-bag errors
- On the limiting distribution of the metric dimension for random forests
- Consistency of random forests
- A random forest guided tour
- Smoothing and adaptation of shifted Pólya tree ensembles
- An introduction to recent advances in high/infinite dimensional statistics
- Quantifying uncertainty in random forests via confidence intervals and hypothesis tests
- scientific article; zbMATH DE number 6378123 (Why is no real title available?)
- A Study of Strength and Correlation in Random Forests
- Quantile regression forests
- Consistency of random forests and other averaging classifiers
- Variance reduction in purely random forests
- Limit Distributions of the Height of a Random Forest
- Nonunitarizable Representations and Random Forests
- scientific article; zbMATH DE number 1537528 (Why is no real title available?)
- Critical random forests
- Tuning parameters in random forests
- Impact of subsampling and tree depth on random forests
- Measuring the algorithmic convergence of randomized ensembles: the regression setting
- A Random Forest Approach for Bounded Outcome Variables
- Predictive Distribution Modeling Using Transformation Forests
- Random forests for time-dependent processes
- Analysis of a random forests model
- Comments on: ``A random forest guided tour
- A version of the random directed forest and its convergence to the Brownian web
- Model-Assisted Estimation Through Random Forests in Finite Population Sampling
- Improved convergence rates for some kernel random forest algorithms
- Medoid splits for efficient random forests in metric spaces
- Estimating the algorithmic variance of randomized ensembles via the bootstrap
- Generalized random forests
- Benchmarking sparse variable selection methods for genomic data analyses
- On regression-adjusted imputation estimators of average treatment effects
- A model-free multivariate non-recursive feature elimination for feature selection on high-dimensional complex multiple response data
- Stone's theorem for distributional regression in Wasserstein distance
- Statistical inference in the presence of imputed survey data through regression trees and random forests
- Ensemble learning for portfolio valuation and risk management
- Survival tree averaging by functional martingale-based residuals
- Infinite random forests for imbalanced classification tasks
- Mathematical optimization in classification and regression trees
- Consistency of random survival forests
This page was built for publication: On the asymptotics of random forests
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q268730)
