BART: Bayesian additive regression trees

DOI10.1214/09-AOAS285zbMATH Open1189.62066arXiv0806.3286OpenAlexW3099006712MaRDI QIDQ65651FDOQ65651

Authors: Hugh A. Chipman, Edward I. George, Robert E. McCulloch, Hugh Chipman, Edward I. George, Robert E. McCulloch 

Publication date: 1 March 2010

Published in: The Annals of Applied Statistics (Search for Journal in Brave)

Abstract: We develop a Bayesian "sum-of-trees" model where each tree is constrained by a regularization prior to be a weak learner, and fitting and inference are accomplished via an iterative Bayesian backfitting MCMC algorithm that generates samples from a posterior. Effectively, BART is a nonparametric Bayesian regression approach which uses dimensionally adaptive random basis elements. Motivated by ensemble methods in general, and boosting algorithms in particular, BART is defined by a statistical model: a prior and a likelihood. This approach enables full posterior inference including point and interval estimates of the unknown regression function as well as the marginal effects of potential predictors. By keeping track of predictor inclusion frequencies, BART can also be used for model-free variable selection. BART's many features are illustrated with a bake-off against competing methods on 42 different data sets, with a simulation experiment and on a drug discovery classification problem.

Full work available at URL: https://arxiv.org/abs/0806.3286

Recommendations

• Bayesian additive regression trees with model trees
• Log-linear Bayesian additive regression trees for multinomial logistic and count regression models
• Bayesian regression trees for high-dimensional prediction and variable selection
• Variable selection for BART: an application to gene regulation
• Bayesian treed models

zbMATH Keywords

variable selectionBayesian backfittingboostingCARTclassificationensembleMCMCnonparametric regressionprobit modelrandom basisregularizatiosum-of-trees modelweak learner

Mathematics Subject Classification ID

Bayesian inference (62F15) Nonparametric regression and quantile regression (62G08) Classification and discrimination; cluster analysis (statistical aspects) (62H30)

Cites Work

• Statistical Analysis of Financial Data in S-Plus
• A decision-theoretic generalization of on-line learning and an application to boosting
• Greedy function approximation: A gradient boosting machine.
• Least angle regression. (With discussion)
• Bagging predictors
• Reversible jump Markov chain Monte Carlo computation and Bayesian model determination
• Multivariate adaptive regression splines
• Alternative models for stock price dynamics.
• Bayesian backfitting. (With comments and a rejoinder).
• A Bayesian CART algorithm
• Bayesian Inference on Network Traffic Using Link Count Data
• Title not available (Why is that?)
• Bayesian Analysis of Binary and Polychotomous Response Data
• An Efficient Method of Estimating Seemingly Unrelated Regressions and Tests for Aggregation Bias
• A spatially-adjusted Bayesian additive regression tree model to merge two datasets
• The Bayesian additive classification tree applied to credit risk modelling

Cited In (only showing first 100 items - show all)

• Learning algorithms to evaluate forensic glass evidence
• Estimating Individual Treatment Effect in Observational Data Using Random Forest Methods
• Robust discrete choice models with \(t\)-distributed kernel errors
• Nowcasting in a pandemic using non-parametric mixed frequency VARs
• Adaptive Bayesian Sum of Trees Model for Covariate-Dependent Spectral Analysis
• Heteroscedastic BART via Multiplicative Regression Trees
• Minimax optimal rates for Mondrian trees and forests
• Comments on: ``A random forest guided tour
• Variable prioritization in nonlinear black box methods: a genetic association case study
• Deep learning for ranking response surfaces with applications to optimal stopping problems
• Monotonic effects of characteristics on returns
• An adaptive sampling scheme guided by BART -- with an application to predict processor performance
• Log-Linear Bayesian Additive Regression Trees for Multinomial Logistic and Count Regression Models
• Tree models of Bayesian regression: a case study
• Variable selection via a multi-stage strategy
• Nonparametric Bayesian analysis of the compound Poisson prior for support boundary recovery
• Model-guided adaptive sampling for Bayesian model selection
• Inflection points in community-level homeless rates
• Bayesian computation: a summary of the current state, and samples backwards and forwards
• Title not available (Why is that?)
• TAIL FORECASTING WITH MULTIVARIATE BAYESIAN ADDITIVE REGRESSION TREES
• Logitboost autoregressive networks
• Flexible Sensitivity Analysis for Observational Studies Without Observable Implications
• Robust Bayesian inference for big data: combining sensor-based records with traditional survey data
• Efficient Bayesian multivariate surface regression
• Automated versus do-it-yourself methods for causal inference: lessons learned from a data analysis competition
• A Bayesian non-parametric modeling to estimate student response to ICT investment
• A Bayesian regression tree approach to identify the effect of nanoparticles' properties on toxicity profiles
• Random Forest Prediction Intervals
• Spline-Based Emulators for Radiative Shock Experiments With Measurement Error
• Optional Pólya trees: posterior rates and uncertainty quantification
• Spatial CART classification trees
• Maximum relevance minimum common redundancy feature selection for nonlinear data
• Heterogeneous causal effects with imperfect compliance: a Bayesian machine learning approach
• Bayesian Regression Trees for High-Dimensional Prediction and Variable Selection
• Bayesian additive regression trees with model trees
• Tree based credible set estimation
• Inference in Bayesian additive vector autoregressive tree models
• Toward an explainable machine learning model for claim frequency: a use case in car insurance pricing with telematics data
• Bayesian Neural Networks for Selection of Drug Sensitive Genes
• Augmented direct learning for conditional average treatment effect estimation with double robustness
• Neural random forests
• Data driven conditional optimal transport
• Uncertainty quantification for honest regression trees
• Uncertainty quantification for Bayesian CART
• Data-driven modelling of the Reynolds stress tensor using random forests with invariance
• A decision-theoretic approach for model interpretability in Bayesian framework
• Bayesian data synthesis and the utility-risk trade-off for mixed epidemiological data
• Estimating the health effects of environmental mixtures using Bayesian semiparametric regression and sparsity inducing priors
• Stochastic Tree Ensembles for Regularized Nonlinear Regression
• A Generalized Estimating Equation Approach to Multivariate Adaptive Regression Splines
• Spike-and-slab priors for function selection in structured additive regression models
• BART with targeted smoothing: an analysis of patient-specific stillbirth risk
• A semiparametric modeling approach using Bayesian additive regression trees with an application to evaluate heterogeneous treatment effects
• Variance prior forms for high-dimensional Bayesian variable selection
• Subgroup causal effect identification and estimation via matching tree
• Estimating population average causal effects in the presence of non-overlap: the effect of natural gas compressor station exposure on cancer mortality
• Targeted smooth Bayesian causal forests: an analysis of heterogeneous treatment effects for simultaneous vs. interval medical abortion regimens over gestation
• Bayesian synthesis: combining subjective analyses, with an application to ozone data
• Smoothing and adaptation of shifted Pólya tree ensembles
• Stochastic Tree Search for Estimating Optimal Dynamic Treatment Regimes
• Analyzing stochastic computer models: a review with opportunities
• Entity resolution with empirically motivated priors
• The dependent Dirichlet process and related models
• Modeling threshold interaction effects through the logistic classification trunk
• Preserving data utility via BART
• Bayesian nonparametric regression with varying residual density
• Bayesian Deep Net GLM and GLMM
• Bayesian regression tree models for causal inference: regularization, confounding, and heterogeneous effects (with discussion)
• Adaptive-modal Bayesian nonparametric regression
• Dynamic treatment regimes with interference
• Sequential design for computer experiments with a flexible Bayesian additive model
• Simultaneous dimension reduction and variable selection in modeling high dimensional data
• Minimax-optimal nonparametric regression in high dimensions
• Bayesian multiple response kernel regression model for high dimensional data and its practical applications in near infrared spectroscopy
• Posterior concentration for Bayesian regression trees and forests
• Cluster‐Specific Variable Selection for Product Partition Models
• Scalable Bayesian nonparametric regression via a Plackett-Luce model for conditional ranks
• Measuring the Algorithmic Convergence of Randomized Ensembles: The Regression Setting
• Multiple imputation for sharing precise geographies in public use data
• Rule-based Bayesian regression
• Stochastic boosting algorithms
• Stochastic boosting algorithms
• Model Interpretation Through Lower-Dimensional Posterior Summarization
• Title not available (Why is that?)
• A Semiparametric Bayesian Approach to Dropout in Longitudinal Studies With Auxiliary Covariates
• Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
• BayesTree
• Estimating treatment effect heterogeneity in randomized program evaluation
• Prediction with missing data via Bayesian Additive Regression Trees
• dbarts
• All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
• Scalable Bayesian Nonparametric Clustering and Classification
• FSR methods for second-order regression models
• Variable selection for BART: an application to gene regulation
• High-dimensional Bayesian inference in nonparametric additive models
• cjbart
• Simultaneous transformation and rounding (STAR) models for integer-valued data
• stan4bart
• Generalized random forests

Uses Software

• R
• MASS (R)
• ROCR
• LIBSVM
• e1071
• gbm
• AdaBoost.MH
• BartPy

This page was built for publication: BART: Bayesian additive regression trees

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q65651)