Impact of subsampling and tree depth on random forests

DOI10.1051/PS/2018008zbMATH Open1409.62072arXiv1603.04261OpenAlexW2794159684MaRDI QIDQ4615432FDOQ4615432

Authors: Roxane Duroux, Erwan Scornet 

Publication date: 28 January 2019

Published in: ESAIM: Probability and Statistics (Search for Journal in Brave)

Abstract: Random forests are ensemble learning methods introduced by Breiman (2001) that operate by averaging several decision trees built on a randomly selected subspace of the data set. Despite their widespread use in practice, the respective roles of the different mechanisms at work in Breiman's forests are not yet fully understood, neither is the tuning of the corresponding parameters. In this paper, we study the influence of two parameters , namely the subsampling rate and the tree depth, on Breiman's forests performance. More precisely, we show that fully developed sub-sampled forests and pruned (without subsampling) forests have similar performances, as long as respective parameters are well chosen. Moreover , experiments show that a proper tuning of subsampling or pruning lead in most cases to an improvement of Breiman's original forests errors.

Full work available at URL: https://arxiv.org/abs/1603.04261

Recommendations

• The effect of splitting on random forests
• scientific article; zbMATH DE number 1984549
• Random forests and selected samples
• On the asymptotics of random forests
• scientific article; zbMATH DE number 1294094
• Machine Learning: ECML 2004

zbMATH Keywords

random forestsrandomizationdecision treessubsamplingparameter tuningtree depth

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Asymptotic properties of nonparametric inference (62G20) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Cites Work

• High-dimensional additive modeling
• Consistency of random forests
• Consistency of random forests and other averaging classifiers
• Title not available (Why is that?)
• Random forests
• Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
• Optimal rates of convergence for nonparametric estimators
• Quantifying uncertainty in random forests via confidence intervals and hypothesis tests
• Optimal global rates of convergence for nonparametric regression
• Title not available (Why is that?)
• Super Learner
• Do we need hundreds of classifiers to solve real world classification problems?
• On the asymptotics of random forests
• Analysis of a random forests model
• Consistency of random survival forests
• Cellular tree classifiers
• Handbook of computational statistics. Concepts and methods.
• Comments on: ``A random forest guided tour
• Variance reduction in purely random forests

Cited In (8)

• Tuning parameters in random forests
• Large Scale Prediction with Decision Trees
• Comments on: ``A random forest guided tour
• Variance reduction in purely random forests
• To tune or not to tune the number of trees in random forest
• Search for the smallest random forest
• Title not available (Why is that?)
• Models under which random forests perform badly; consequences for applications

Uses Software

• SuperLearner
• GeneSrF

This page was built for publication: Impact of subsampling and tree depth on random forests

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4615432)