Bayesian additive regression trees with model trees

DOI10.1007/S11222-021-09997-3zbMATH Open1475.62055arXiv2006.07493OpenAlexW3135504614MaRDI QIDQ2058722FDOQ2058722

Authors: Estevão B. Prado, Rafael A. Moral, Andrew C. Parnell 

Publication date: 9 December 2021

Published in: Statistics and Computing (Search for Journal in Brave)

Abstract: Bayesian Additive Regression Trees (BART) is a tree-based machine learning method that has been successfully applied to regression and classification problems. BART assumes regularisation priors on a set of trees that work as weak learners and is very flexible for predicting in the presence of non-linearity and high-order interactions. In this paper, we introduce an extension of BART, called Model Trees BART (MOTR-BART), that considers piecewise linear functions at node levels instead of piecewise constants. In MOTR-BART, rather than having a unique value at node level for the prediction, a linear predictor is estimated considering the covariates that have been used as the split variables in the corresponding tree. In our approach, local linearities are captured more efficiently and fewer trees are required to achieve equal or better performance than BART. Via simulation studies and real data applications, we compare MOTR-BART to its main competitors. R code for MOTR-BART implementation is available at https://github.com/ebprado/MOTR-BART.

Full work available at URL: https://arxiv.org/abs/2006.07493

Recommendations

• Bayesian additive regression trees using Bayesian model averaging
• BART: Bayesian additive regression trees
• Tree models of Bayesian regression: a case study
• Variable Selection and Interaction Detection with Bayesian Additive Regression Trees
• Inference in Bayesian additive vector autoregressive tree models
• Bayesian treed models
• Log-linear Bayesian additive regression trees for multinomial logistic and count regression models
• TAIL FORECASTING WITH MULTIVARIATE BAYESIAN ADDITIVE REGRESSION TREES
• Prediction with missing data via Bayesian additive regression trees
• A semiparametric modeling approach using Bayesian additive regression trees with an application to evaluate heterogeneous treatment effects

zbMATH Keywords

machine learninglinear modelsBayesian nonparametric regressionBayesian trees

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Nonparametric regression and quantile regression (62G08) Learning and adaptive systems in artificial intelligence (68T05)

Cites Work

• BART: Bayesian additive regression trees
• Greedy function approximation: A gradient boosting machine.
• Bayesian additive regression trees using Bayesian model averaging
• BART with targeted smoothing: an analysis of patient-specific stillbirth risk
• Generalized random forests
• Title not available (Why is that?)
• Random forests
• The horseshoe estimator for sparse signals
• Bayesian Analysis of Binary and Polychotomous Response Data
• The Bayesian additive classification tree applied to credit risk modelling
• Bayesian regression tree models for causal inference: regularization, confounding, and heterogeneous effects (with discussion)
• Logistic model trees
• Posterior concentration for Bayesian regression trees and forests
• Semiparametric mixed-scale models using shared Bayesian forests
• Bayesian regression trees for high-dimensional prediction and variable selection
• Bayesian regression tree ensembles that adapt to smoothness and sparsity
• A Bayesian credible subgroups approach to identifying patient subgroups with positive treatment effects

Cited In (15)

• BART with targeted smoothing: an analysis of patient-specific stillbirth risk
• A semiparametric modeling approach using Bayesian additive regression trees with an application to evaluate heterogeneous treatment effects
• Machine Learning vs. Survival Analysis Models: a study on right censored heart failure data
• Comment: Contributions of model features to BART causal inference performance using ACIC 2016 competition data
• Bayesian regression trees for high-dimensional prediction and variable selection
• Bayesian additive regression trees for genotype by environment interaction models
• GP-BART: a novel Bayesian additive regression trees approach using Gaussian processes
• BART: Bayesian additive regression trees
• Model Mixing Using Bayesian Additive Regression Trees
• Log-linear Bayesian additive regression trees for multinomial logistic and count regression models
• Treed Avalanche Forecasting: Mitigating Avalanche Danger Utilizing Bayesian Additive Regression Trees
• The Bayesian additive classification tree applied to credit risk modelling
• Stochastic Tree Ensembles for Regularized Nonlinear Regression
• Bayesian additive regression trees using Bayesian model averaging
• Prediction with missing data via Bayesian additive regression trees

Uses Software

• R
• GitHub

This page was built for publication: Bayesian additive regression trees with model trees

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2058722)