Efficient Metropolis-Hastings proposal mechanisms for Bayesian regression tree models

DOI10.1214/16-BA999zbMATH Open1357.62178arXiv1312.1895OpenAlexW2293510337MaRDI QIDQ516518FDOQ516518

Authors: Matthew T. Pratola 

Publication date: 14 March 2017

Published in: Bayesian Analysis (Search for Journal in Brave)

Abstract: Bayesian regression trees are flexible non-parametric models that are well suited to many modern statistical regression problems. Many such tree models have been proposed, from the simple single- tree model to more complex tree ensembles. Their non-parametric formulation allows for effective and efficient modeling of datasets exhibiting complex non-linear relationships between the model pre- dictors and observations. However, the mixing behavior of the Markov Chain Monte Carlo (MCMC) sampler is sometimes poor. This is because the proposals in the sampler are typically local alterations of the tree structure, such as the birth/death of leaf nodes, which does not allow for efficient traversal of the model space. This poor mixing can lead to inferential problems, such as under-representing uncertainty. In this paper, we develop novel proposal mechanisms for efficient sampling. The first is a rule perturbation proposal while the second we call tree rotation. The perturbation proposal can be seen as an efficient variation of the change proposal found in existing literature. The novel tree rotation proposal is simple to implement as it only requires local changes to the regression tree structure, yet it efficiently traverses disparate regions of the model space along contours of equal probability. When combined with the classical birth/death proposal, the resulting MCMC sampler exhibits good acceptance rates and properly represents model uncertainty in the posterior samples. We implement this sampling algorithm in the Bayesian Additive Regression Tree (BART) model and demonstrate its effectiveness on a prediction problem from computer experiments and a test function where structural tree variability is needed to fully explore the posterior.

Full work available at URL: https://arxiv.org/abs/1312.1895

Recommendations

• Tree models of Bayesian regression: a case study
• Bayesian regression trees for high-dimensional prediction and variable selection
• Posterior concentration for Bayesian regression trees and forests
• BART: Bayesian additive regression trees
• A new go-to sampler for Bayesian probit regression

zbMATH Keywords

Markov chain Monte Carlocomputer experimentscredible intervaluncertainty quantificationproposal distributioncoverage probability

Mathematics Subject Classification ID

Bayesian inference (62F15) Nonparametric regression and quantile regression (62G08) Monte Carlo methods (65C05)

Cited In (14)

• Influential Observations in Bayesian Regression Tree Models
• Heteroscedastic BART via Multiplicative Regression Trees
• Bayesian regression trees for high-dimensional prediction and variable selection
• Smoothing and adaptation of shifted Pólya tree ensembles
• Bayesian CART models for insurance claims frequency
• Tree models of Bayesian regression: a case study
• Nonparametric failure time: time-to-event machine learning with heteroskedastic Bayesian additive regression trees and low information omnibus Dirichlet process mixtures
• BART-based inference for Poisson processes
• Stochastic tree search for estimating optimal dynamic treatment regimes
• Title not available (Why is that?)
• The taxicab sampler: MCMC for discrete spaces with application to tree models
• Model Mixing Using Bayesian Additive Regression Trees
• Log-linear Bayesian additive regression trees for multinomial logistic and count regression models
• Bayesian additive regression trees using Bayesian model averaging

This page was built for publication: Efficient Metropolis-Hastings proposal mechanisms for Bayesian regression tree models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q516518)