Forest Garrote

DOI10.1214/09-EJS434MaRDI QIDQ1952025zbMATH OpenOpenAlexFDO

Authors Nicolai Meinshausen

Publication date 27 May 2013

Published in Electronic Journal of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0906.3590, https://projecteuclid.org/euclid.ejs/1259944247

zbMATH Keywords

nonnegative Garroterandom forestssparsitytree ensembles

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08)

Abstract: Variable selection for high-dimensional linear models has received a lot of attention lately, mostly in the context of l1-regularization. Part of the attraction is the variable selection effect: parsimonious models are obtained, which are very suitable for interpretation. In terms of predictive power, however, these regularized linear models are often slightly inferior to machine learning procedures like tree ensembles. Tree ensembles, on the other hand, lack usually a formal way of variable selection and are difficult to visualize. A Garrote-style convex penalty for trees ensembles, in particular Random Forests, is proposed. The penalty selects functional groups of nodes in the trees. These could be as simple as monotone functions of individual predictor variables. This yields a parsimonious function fit, which lends itself easily to visualization and interpretation. The predictive power is maintained at least at the same level as the original tree ensemble. A key feature of the method is that, once a tree ensemble is fitted, no further tuning parameter needs to be selected. The empirical performance is demonstrated on a wide array of datasets.

Recommendations

• Trees garrote for regression analysis
• GARROTE TREES AS TREE STRUCTURED REGRESSION ANALYSIS
• Analysis of a random forests model
• Random forests
• Variable importance in binary regression trees and forests

Cites work

• A note on the Lasso and related procedures in model selection
• Atomic decomposition by basis pursuit
• Boosting With theL₂Loss
• High-dimensional graphs and variable selection with the Lasso
• scientific article; zbMATH DE number 3860199 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• Least angle regression. (With discussion)
• Model Selection and Estimation in Regression with Grouped Variables
• Multivariate adaptive regression splines
• On the Non-Negative Garrotte Estimator
• Predictive learning via rule ensembles
• Random forests
• Sparse CCA using a lasso with positivity constraints
• The Adaptive Lasso and Its Oracle Properties
• The composite absolute penalties family for grouped and hierarchical variable selection
• The elements of statistical learning. Data mining, inference, and prediction
• The Group Lasso for Logistic Regression

Cited in

(6)

• A random forest guided tour
• Tree-based censored regression with applications in insurance
• GARROTE TREES AS TREE STRUCTURED REGRESSION ANALYSIS
• Node harvest
• Trees garrote for regression analysis
• Paired bondage in trees

Describes a project that uses

Uses Software

• UCI
• PDCO

This page was built for publication: Forest Garrote

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1952025)