BART: Bayesian additive regression trees

DOI10.1214/09-AOAS285zbMATH Open1189.62066arXiv0806.3286OpenAlexW3099006712MaRDI QIDQ65651FDOQ65651

Authors: Hugh A. Chipman, Edward I. George, Robert E. McCulloch, Hugh Chipman, Edward I. George, Robert E. McCulloch 

Publication date: 1 March 2010

Published in: The Annals of Applied Statistics (Search for Journal in Brave)

Abstract: We develop a Bayesian "sum-of-trees" model where each tree is constrained by a regularization prior to be a weak learner, and fitting and inference are accomplished via an iterative Bayesian backfitting MCMC algorithm that generates samples from a posterior. Effectively, BART is a nonparametric Bayesian regression approach which uses dimensionally adaptive random basis elements. Motivated by ensemble methods in general, and boosting algorithms in particular, BART is defined by a statistical model: a prior and a likelihood. This approach enables full posterior inference including point and interval estimates of the unknown regression function as well as the marginal effects of potential predictors. By keeping track of predictor inclusion frequencies, BART can also be used for model-free variable selection. BART's many features are illustrated with a bake-off against competing methods on 42 different data sets, with a simulation experiment and on a drug discovery classification problem.

Full work available at URL: https://arxiv.org/abs/0806.3286

Recommendations

• Bayesian additive regression trees with model trees
• Log-linear Bayesian additive regression trees for multinomial logistic and count regression models
• Bayesian regression trees for high-dimensional prediction and variable selection
• Variable selection for BART: an application to gene regulation
• Bayesian treed models

zbMATH Keywords

variable selectionBayesian backfittingboostingCARTclassificationensembleMCMCnonparametric regressionprobit modelrandom basisregularizatiosum-of-trees modelweak learner

Mathematics Subject Classification ID

Bayesian inference (62F15) Nonparametric regression and quantile regression (62G08) Classification and discrimination; cluster analysis (statistical aspects) (62H30)

Cites Work

• Statistical Analysis of Financial Data in S-Plus
• A decision-theoretic generalization of on-line learning and an application to boosting
• Greedy function approximation: A gradient boosting machine.
• Least angle regression. (With discussion)
• Bagging predictors
• Reversible jump Markov chain Monte Carlo computation and Bayesian model determination
• Multivariate adaptive regression splines
• Alternative models for stock price dynamics.
• Bayesian backfitting. (With comments and a rejoinder).
• A Bayesian CART algorithm
• Bayesian Inference on Network Traffic Using Link Count Data
• Title not available (Why is that?)
• Bayesian Analysis of Binary and Polychotomous Response Data
• An Efficient Method of Estimating Seemingly Unrelated Regressions and Tests for Aggregation Bias
• A spatially-adjusted Bayesian additive regression tree model to merge two datasets
• The Bayesian additive classification tree applied to credit risk modelling

Cited In (only showing first 100 items - show all)

• Learning algorithms to evaluate forensic glass evidence
• Estimating Individual Treatment Effect in Observational Data Using Random Forest Methods
• Robust discrete choice models with \(t\)-distributed kernel errors
• Nowcasting in a pandemic using non-parametric mixed frequency VARs
• Adaptive Bayesian Sum of Trees Model for Covariate-Dependent Spectral Analysis
• Heteroscedastic BART via Multiplicative Regression Trees
• Minimax optimal rates for Mondrian trees and forests
• Comments on: ``A random forest guided tour
• Variable prioritization in nonlinear black box methods: a genetic association case study
• Deep learning for ranking response surfaces with applications to optimal stopping problems
• Monotonic effects of characteristics on returns
• An adaptive sampling scheme guided by BART -- with an application to predict processor performance
• Log-Linear Bayesian Additive Regression Trees for Multinomial Logistic and Count Regression Models
• Tree models of Bayesian regression: a case study
• Variable selection via a multi-stage strategy
• Nonparametric Bayesian analysis of the compound Poisson prior for support boundary recovery
• Model-guided adaptive sampling for Bayesian model selection
• Inflection points in community-level homeless rates
• Bayesian computation: a summary of the current state, and samples backwards and forwards
• Title not available (Why is that?)
• TAIL FORECASTING WITH MULTIVARIATE BAYESIAN ADDITIVE REGRESSION TREES
• Logitboost autoregressive networks
• Flexible Sensitivity Analysis for Observational Studies Without Observable Implications
• Robust Bayesian inference for big data: combining sensor-based records with traditional survey data
• Efficient Bayesian multivariate surface regression
• Automated versus do-it-yourself methods for causal inference: lessons learned from a data analysis competition
• A Bayesian non-parametric modeling to estimate student response to ICT investment
• A Bayesian regression tree approach to identify the effect of nanoparticles' properties on toxicity profiles
• Random Forest Prediction Intervals
• Spline-Based Emulators for Radiative Shock Experiments With Measurement Error
• Optional Pólya trees: posterior rates and uncertainty quantification
• Spatial CART classification trees
• Maximum relevance minimum common redundancy feature selection for nonlinear data
• Heterogeneous causal effects with imperfect compliance: a Bayesian machine learning approach
• Bayesian Regression Trees for High-Dimensional Prediction and Variable Selection
• Bayesian additive regression trees with model trees
• Tree based credible set estimation
• Inference in Bayesian additive vector autoregressive tree models
• Toward an explainable machine learning model for claim frequency: a use case in car insurance pricing with telematics data
• Bayesian Neural Networks for Selection of Drug Sensitive Genes
• Augmented direct learning for conditional average treatment effect estimation with double robustness
• Neural random forests
• Data driven conditional optimal transport
• Uncertainty quantification for honest regression trees
• Uncertainty quantification for Bayesian CART
• Data-driven modelling of the Reynolds stress tensor using random forests with invariance
• A decision-theoretic approach for model interpretability in Bayesian framework
• Bayesian data synthesis and the utility-risk trade-off for mixed epidemiological data
• Estimating the health effects of environmental mixtures using Bayesian semiparametric regression and sparsity inducing priors
• Stochastic Tree Ensembles for Regularized Nonlinear Regression
• Influential Observations in Bayesian Regression Tree Models
• Bayesian nonparametric adjustment of confounding
• Fair and Efficient Allocation of Scarce Resources Based on Predicted Outcomes: Implications for Homeless Service Delivery
• A penalized complexity prior for deep Bayesian transfer learning with application to materials informatics
• A simple approach for local and global variable importance in nonlinear regression models
• Balancing vs modeling approaches to weighting in practice
• Type I Tobit Bayesian additive regression trees for censored outcome regression
• Heterogeneous treatment effect-based random forest: HTERF
• Variable selection using Bayesian additive regression trees
• An evolutionary estimation procedure for generalized semilinear regression trees
• Optimization on Manifolds via Graph Gaussian Processes
• Bayesian CART models for insurance claims frequency
• A Mass-Shifting Phenomenon of Truncated Multivariate Normal Priors
• Conformal Sensitivity Analysis for Individual Treatment Effects
• Heterogeneous Distributed Lag Models to Estimate Personalized Effects of Maternal Exposures to Air Pollution
• Improved inference for doubly robust estimators of heterogeneous treatment effects
• Nonparametric failure time: time-to-event machine learning with heteroskedastic Bayesian additive regression trees and low information omnibus Dirichlet process mixtures
• Prior and posterior checking of implicit causal assumptions
• Hierarchical Bayesian bootstrap for heterogeneous treatment effect estimation
• Evaluation of the health impacts of the 1990 Clean Air Act Amendments using causal inference and machine learning
• Comparing emulation methods for a high-resolution storm surge model
• BART-based inference for Poisson processes
• A Bayesian nonparametric model for zero‐inflated outcomes: Prediction, clustering, and causal estimation
• A shared latent process model to correct for preferential sampling in disease surveillance systems
• Understanding the effect of contextual factors and decision making on team performance in Twenty20 cricket: an interpretable machine learning approach
• Bayesian additive regression trees for genotype by environment interaction models
• Gibbs Priors for Bayesian Nonparametric Variable Selection with Weak Learners
• Bayesian Nonparametric Common Atoms Regression for Generating Synthetic Controls in Clinical Trials
• Bayesian tree-based heterogeneous mediation analysis with a time-to-event outcome
• Bayesian projection pursuit regression
• GP-BART: a novel Bayesian additive regression trees approach using Gaussian processes
• A semiparametric Bayesian approach to population finding with time‐to‐event and toxicity data in a randomized clinical trial
• Bayesian neural tree models for nonparametric regression
• The taxicab sampler: MCMC for discrete spaces with application to tree models
• Adaptive Conditional Distribution Estimation with Bayesian Decision Tree Ensembles
• Bayesian survival tree ensembles with submodel shrinkage
• mBART: multidimensional monotone BART
• Dynamic treatment regimes using Bayesian additive regression trees for censored outcomes
• Variable Selection Via Thompson Sampling
• Bayesian Conjugacy in Probit, Tobit, Multinomial Probit and Extensions: A Review and New Results
• Learning from high dimensional data based on weighted feature importance in decision tree ensembles
• Multinomial probit Bayesian additive regression trees
• Bayesian quickest detection of credit card fraud
• Joint Bayesian analysis of multiple response-types using the hierarchical generalized transformation model
• Robust and accurate regression-based techniques for period inference in real-time systems
• Causal effect random forest of interaction trees for learning individualized treatment regimes with multiple treatments in observational studies
• A localized ensemble of approximate Gaussian processes for fast sequential emulation
• Ordered probit Bayesian additive regression trees for ordinal data
• Nonparametric machine learning for precision medicine with longitudinal clinical trials and Bayesian additive regression trees with mixed models
• Using BART to Perform Pareto Optimization and Quantify its Uncertainties

Uses Software

• R
• MASS (R)
• ROCR
• LIBSVM
• e1071
• gbm
• AdaBoost.MH
• BartPy

This page was built for publication: BART: Bayesian additive regression trees

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q65651)