Impact of subsampling and tree depth on random forests

DOI10.1051/PS/2018008MaRDI QIDQ4615432zbMATH OpenOpenAlexFDO

Authors Roxane Duroux, Erwan Scornet

Publication date 28 January 2019

Published in ESAIM: Probability and Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1603.04261

zbMATH Keywords

random forestsrandomizationdecision treessubsamplingparameter tuningtree depth

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Asymptotic properties of nonparametric inference (62G20) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: Random forests are ensemble learning methods introduced by Breiman (2001) that operate by averaging several decision trees built on a randomly selected subspace of the data set. Despite their widespread use in practice, the respective roles of the different mechanisms at work in Breiman's forests are not yet fully understood, neither is the tuning of the corresponding parameters. In this paper, we study the influence of two parameters , namely the subsampling rate and the tree depth, on Breiman's forests performance. More precisely, we show that fully developed sub-sampled forests and pruned (without subsampling) forests have similar performances, as long as respective parameters are well chosen. Moreover , experiments show that a proper tuning of subsampling or pruning lead in most cases to an improvement of Breiman's original forests errors.

Recommendations

• The effect of splitting on random forests
• scientific article; zbMATH DE number 1984549
• Random forests and selected samples
• On the asymptotics of random forests
• scientific article; zbMATH DE number 1294094
• Machine Learning: ECML 2004

Cites work

• scientific article; zbMATH DE number 3860199 (Why is no real title available?)
• scientific article; zbMATH DE number 893887 (Why is no real title available?)
• Analysis of a random forests model
• Cellular tree classifiers
• Comments on: ``A random forest guided tour
• Consistency of random forests
• Consistency of random forests and other averaging classifiers
• Consistency of random survival forests
• Do we need hundreds of classifiers to solve real world classification problems?
• Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
• Handbook of computational statistics. Concepts and methods.
• High-dimensional additive modeling
• On the asymptotics of random forests
• Optimal global rates of convergence for nonparametric regression
• Optimal rates of convergence for nonparametric estimators
• Quantifying uncertainty in random forests via confidence intervals and hypothesis tests
• Random forests
• Super Learner
• Variance reduction in purely random forests

Cited in

(8)

• Tuning parameters in random forests
• Large Scale Prediction with Decision Trees
• Comments on: ``A random forest guided tour
• Variance reduction in purely random forests
• To tune or not to tune the number of trees in random forest
• Search for the smallest random forest
• scientific article; zbMATH DE number 7307469 (Why is no real title available?)
• Models under which random forests perform badly; consequences for applications

Describes a project that uses

Uses Software

• GeneSrF
• SuperLearner

This page was built for publication: Impact of subsampling and tree depth on random forests

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4615432)