Bayesian additive regression trees using Bayesian model averaging

DOI10.1007/S11222-017-9767-1MaRDI QIDQ1704023zbMATH OpenOpenAlexWikidataFDO

Authors Belinda Hernández, Adrian E. Raftery, Stephen R. Pennington, Andrew C. Parnell

Publication date 8 March 2018

Published in Statistics and Computing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1507.00181, http://europepmc.org/articles/pmc6238959

zbMATH Keywords

Bayesian additive regression treesBayesian model averagingbiomarker selectionrandom forestsmall \(n\) large \(p\)

Mathematics Subject Classification ID

Bayesian inference (62F15) Applications of statistics to biology and medical sciences; meta analysis (62P10) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: Bayesian Additive Regression Trees (BART) is a statistical sum of trees model. It can be considered a Bayesian version of machine learning tree ensemble methods where the individual trees are the base learners. However for data sets where the number of variables $p$ is large (e.g. $p>5,000$) the algorithm can become prohibitively expensive, computationally. Another method which is popular for high dimensional data is random forests, a machine learning algorithm which grows trees using a greedy search for the best split points. However, as it is not a statistical model, it cannot produce probabilistic estimates or predictions. We propose an alternative algorithm for BART called BART-BMA, which uses Bayesian Model Averaging and a greedy search algorithm to produce a model which is much more efficient than BART for datasets with large $p$. BART-BMA incorporates elements of both BART and random forests to offer a model-based algorithm which can deal with high-dimensional data. We have found that BART-BMA can be run in a reasonable time on a standard laptop for the "small $n$ large $p$" scenario which is common in many areas of bioinformatics. We showcase this method using simulated data and data from two real proteomic experiments; one to distinguish between patients with cardiovascular disease and controls and another to classify agressive from non-agressive prostate cancer. We compare our results to their main competitors. Open source code written in R and Rcpp to run BART-BMA can be found at: https://github.com/BelindaHernandez/BART-BMA.git

Recommendations

• BART: Bayesian additive regression trees
• Bayesian additive regression trees with model trees
• Bayesian regression trees for high-dimensional prediction and variable selection
• Bayesian regression tree ensembles that adapt to smoothness and sparsity
• Variable selection for BART: an application to gene regulation

Cites work

• Algorithm runtime prediction: methods \& evaluation
• Bagging predictors
• BART: Bayesian additive regression trees
• Bayesian Analysis of Binary and Polychotomous Response Data
• Efficient Metropolis-Hastings proposal mechanisms for Bayesian regression tree models
• Empirical characterization of random forest variable importance measures
• Estimating the dimension of a model
• Estimation of a noisy discrete-time step function: Bayes and empirical Bayes approaches
• Extremely randomized trees
• Fitting multiple change-point models to data
• Greedy function approximation: A gradient boosting machine.
• scientific article; zbMATH DE number 6378123 (Why is no real title available?)
• scientific article; zbMATH DE number 3860199 (Why is no real title available?)
• Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window
• Multivariate adaptive regression splines
• Multivariate statistics. High dimensional and large-sample approximations.
• Optimal detection of changepoints with a linear computational cost
• Prediction with missing data via Bayesian additive regression trees
• Problems in the Analysis of Survey Data, and a Proposal
• Quantile regression forests
• Random forests
• Regression conformal prediction with random forests
• Stacked regressions
• Statistics for high-dimensional data. Methods, theory and applications.
• The huge Package for High-dimensional Undirected Graph Estimation in R
• Variable selection for BART: an application to gene regulation

Cited in

(16)

• Bayesian additive regression trees with model trees
• Inference in Bayesian additive vector autoregressive tree models
• BART with targeted smoothing: an analysis of patient-specific stillbirth risk
• A decision-theoretic approach for model interpretability in Bayesian framework
• Targeted smooth Bayesian causal forests: an analysis of heterogeneous treatment effects for simultaneous vs. interval medical abortion regimens over gestation
• BART-BMA
• Treed Avalanche Forecasting: Mitigating Avalanche Danger Utilizing Bayesian Additive Regression Trees
• Do German economic research institutes publish efficient growth and inflation forecasts? A Bayesian analysis
• GP-BART: a novel Bayesian additive regression trees approach using Gaussian processes
• TAIL FORECASTING WITH MULTIVARIATE BAYESIAN ADDITIVE REGRESSION TREES
• Black Box Variational Bayesian Model Averaging
• Bayesian additive regression trees for genotype by environment interaction models
• Borrowing from supplemental sources to estimate causal effects from a primary data source
• Accounting for shared covariates in semiparametric Bayesian additive regression trees
• Adaptive use of co-data through empirical Bayes for Bayesian additive regression trees
• Learning texture-property relationships for brittle porous materials: a Bayesian approach with graph-theoretical features

Describes a project that uses

Uses Software

• R
• huge
• BartPy
• BayesTree
• changepoint
• GeneSrF
• Rcpp
• randomForest
• bartMachine
• dbarts
• conformal
• GitHub

This page was built for publication: Bayesian additive regression trees using Bayesian model averaging

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1704023)