BART: Bayesian additive regression trees

DOI10.1214/09-AOAS285zbMATH Open1189.62066arXiv0806.3286OpenAlexW3099006712MaRDI QIDQ65651FDOQ65651

Authors: Hugh A. Chipman, Edward I. George, Robert E. McCulloch, Hugh Chipman, Edward I. George, Robert E. McCulloch 

Publication date: 1 March 2010

Published in: The Annals of Applied Statistics (Search for Journal in Brave)

Abstract: We develop a Bayesian "sum-of-trees" model where each tree is constrained by a regularization prior to be a weak learner, and fitting and inference are accomplished via an iterative Bayesian backfitting MCMC algorithm that generates samples from a posterior. Effectively, BART is a nonparametric Bayesian regression approach which uses dimensionally adaptive random basis elements. Motivated by ensemble methods in general, and boosting algorithms in particular, BART is defined by a statistical model: a prior and a likelihood. This approach enables full posterior inference including point and interval estimates of the unknown regression function as well as the marginal effects of potential predictors. By keeping track of predictor inclusion frequencies, BART can also be used for model-free variable selection. BART's many features are illustrated with a bake-off against competing methods on 42 different data sets, with a simulation experiment and on a drug discovery classification problem.

Full work available at URL: https://arxiv.org/abs/0806.3286

Recommendations

• Bayesian additive regression trees with model trees
• Log-linear Bayesian additive regression trees for multinomial logistic and count regression models
• Bayesian regression trees for high-dimensional prediction and variable selection
• Variable selection for BART: an application to gene regulation
• Bayesian treed models

zbMATH Keywords

variable selectionBayesian backfittingboostingCARTclassificationensembleMCMCnonparametric regressionprobit modelrandom basisregularizatiosum-of-trees modelweak learner

Mathematics Subject Classification ID

Bayesian inference (62F15) Nonparametric regression and quantile regression (62G08) Classification and discrimination; cluster analysis (statistical aspects) (62H30)

Cites Work

• Statistical Analysis of Financial Data in S-Plus
• A decision-theoretic generalization of on-line learning and an application to boosting
• Greedy function approximation: A gradient boosting machine.
• Least angle regression. (With discussion)
• Bagging predictors
• Reversible jump Markov chain Monte Carlo computation and Bayesian model determination
• Multivariate adaptive regression splines
• Alternative models for stock price dynamics.
• Bayesian backfitting. (With comments and a rejoinder).
• A Bayesian CART algorithm
• Bayesian Inference on Network Traffic Using Link Count Data
• Title not available (Why is that?)
• Bayesian Analysis of Binary and Polychotomous Response Data
• An Efficient Method of Estimating Seemingly Unrelated Regressions and Tests for Aggregation Bias
• A spatially-adjusted Bayesian additive regression tree model to merge two datasets
• The Bayesian additive classification tree applied to credit risk modelling

Cited In (only showing first 100 items - show all)

• Influential Observations in Bayesian Regression Tree Models
• Bayesian nonparametric adjustment of confounding
• Fair and Efficient Allocation of Scarce Resources Based on Predicted Outcomes: Implications for Homeless Service Delivery
• A penalized complexity prior for deep Bayesian transfer learning with application to materials informatics
• A simple approach for local and global variable importance in nonlinear regression models
• Balancing vs modeling approaches to weighting in practice
• Type I Tobit Bayesian additive regression trees for censored outcome regression
• Heterogeneous treatment effect-based random forest: HTERF
• Variable selection using Bayesian additive regression trees
• An evolutionary estimation procedure for generalized semilinear regression trees
• Optimization on Manifolds via Graph Gaussian Processes
• Bayesian CART models for insurance claims frequency
• A Mass-Shifting Phenomenon of Truncated Multivariate Normal Priors
• Conformal Sensitivity Analysis for Individual Treatment Effects
• Heterogeneous Distributed Lag Models to Estimate Personalized Effects of Maternal Exposures to Air Pollution
• Improved inference for doubly robust estimators of heterogeneous treatment effects
• Nonparametric failure time: time-to-event machine learning with heteroskedastic Bayesian additive regression trees and low information omnibus Dirichlet process mixtures
• Prior and posterior checking of implicit causal assumptions
• Hierarchical Bayesian bootstrap for heterogeneous treatment effect estimation
• Evaluation of the health impacts of the 1990 Clean Air Act Amendments using causal inference and machine learning
• Comparing emulation methods for a high-resolution storm surge model
• BART-based inference for Poisson processes
• A Bayesian nonparametric model for zero‐inflated outcomes: Prediction, clustering, and causal estimation
• A shared latent process model to correct for preferential sampling in disease surveillance systems
• Understanding the effect of contextual factors and decision making on team performance in Twenty20 cricket: an interpretable machine learning approach
• Bayesian additive regression trees for genotype by environment interaction models
• Gibbs Priors for Bayesian Nonparametric Variable Selection with Weak Learners
• Bayesian Nonparametric Common Atoms Regression for Generating Synthetic Controls in Clinical Trials
• Bayesian tree-based heterogeneous mediation analysis with a time-to-event outcome
• Bayesian projection pursuit regression
• GP-BART: a novel Bayesian additive regression trees approach using Gaussian processes
• A semiparametric Bayesian approach to population finding with time‐to‐event and toxicity data in a randomized clinical trial
• Bayesian neural tree models for nonparametric regression
• The taxicab sampler: MCMC for discrete spaces with application to tree models
• Adaptive Conditional Distribution Estimation with Bayesian Decision Tree Ensembles
• Bayesian survival tree ensembles with submodel shrinkage
• mBART: multidimensional monotone BART
• Dynamic treatment regimes using Bayesian additive regression trees for censored outcomes
• Variable Selection Via Thompson Sampling
• Bayesian Conjugacy in Probit, Tobit, Multinomial Probit and Extensions: A Review and New Results
• Learning from high dimensional data based on weighted feature importance in decision tree ensembles
• Multinomial probit Bayesian additive regression trees
• Bayesian quickest detection of credit card fraud
• Joint Bayesian analysis of multiple response-types using the hierarchical generalized transformation model
• Robust and accurate regression-based techniques for period inference in real-time systems
• Causal effect random forest of interaction trees for learning individualized treatment regimes with multiple treatments in observational studies
• A localized ensemble of approximate Gaussian processes for fast sequential emulation
• Ordered probit Bayesian additive regression trees for ordinal data
• Nonparametric machine learning for precision medicine with longitudinal clinical trials and Bayesian additive regression trees with mixed models
• Using BART to Perform Pareto Optimization and Quantify its Uncertainties
• A Generalized Estimating Equation Approach to Multivariate Adaptive Regression Splines
• BART with targeted smoothing: an analysis of patient-specific stillbirth risk
• A semiparametric modeling approach using Bayesian additive regression trees with an application to evaluate heterogeneous treatment effects
• Variance prior forms for high-dimensional Bayesian variable selection
• Subgroup causal effect identification and estimation via matching tree
• Estimating population average causal effects in the presence of non-overlap: the effect of natural gas compressor station exposure on cancer mortality
• Targeted smooth Bayesian causal forests: an analysis of heterogeneous treatment effects for simultaneous vs. interval medical abortion regimens over gestation
• Bayesian synthesis: combining subjective analyses, with an application to ozone data
• Smoothing and adaptation of shifted Pólya tree ensembles
• Stochastic Tree Search for Estimating Optimal Dynamic Treatment Regimes
• Analyzing stochastic computer models: a review with opportunities
• Entity resolution with empirically motivated priors
• The dependent Dirichlet process and related models
• Modeling threshold interaction effects through the logistic classification trunk
• Preserving data utility via BART
• Bayesian nonparametric regression with varying residual density
• Bayesian Deep Net GLM and GLMM
• Bayesian regression tree models for causal inference: regularization, confounding, and heterogeneous effects (with discussion)
• Adaptive-modal Bayesian nonparametric regression
• Dynamic treatment regimes with interference
• Sequential design for computer experiments with a flexible Bayesian additive model
• Simultaneous dimension reduction and variable selection in modeling high dimensional data
• Minimax-optimal nonparametric regression in high dimensions
• Bayesian multiple response kernel regression model for high dimensional data and its practical applications in near infrared spectroscopy
• Posterior concentration for Bayesian regression trees and forests
• Cluster‐Specific Variable Selection for Product Partition Models
• Scalable Bayesian nonparametric regression via a Plackett-Luce model for conditional ranks
• Measuring the Algorithmic Convergence of Randomized Ensembles: The Regression Setting
• Multiple imputation for sharing precise geographies in public use data
• Rule-based Bayesian regression
• Stochastic boosting algorithms
• Stochastic boosting algorithms
• Model Interpretation Through Lower-Dimensional Posterior Summarization
• Title not available (Why is that?)
• A Semiparametric Bayesian Approach to Dropout in Longitudinal Studies With Auxiliary Covariates
• Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
• BayesTree
• Estimating treatment effect heterogeneity in randomized program evaluation
• Prediction with missing data via Bayesian Additive Regression Trees
• dbarts
• All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
• Spike-and-Slab Priors for Function Selection in Structured Additive Regression Models
• Scalable Bayesian Nonparametric Clustering and Classification
• FSR methods for second-order regression models
• Variable selection for BART: an application to gene regulation
• High-dimensional Bayesian inference in nonparametric additive models
• cjbart
• Simultaneous transformation and rounding (STAR) models for integer-valued data
• stan4bart
• Generalized random forests

Uses Software

• R
• MASS (R)
• ROCR
• LIBSVM
• e1071
• gbm
• AdaBoost.MH
• BartPy

This page was built for publication: BART: Bayesian additive regression trees

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q65651)