Concentration of posterior model probabilities and normalized L₀ criteria

DOI10.1214/21-BA1262MaRDI QIDQ6202921zbMATH OpenOpenAlexFDO

Authors David Rossell

Publication date 27 February 2024

Published in Bayesian Analysis (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1806.04071, https://projecteuclid.org/journals/bayesian-analysis/volume-17/issue-2/Concentration-of-Posterior-Model-Probabilities-and-Normalized-L0-Criteria/10.1214/21-BA1262.full

zbMATH Keywords

\({L_0}\) penaltyBayes factorsconsistencyhigh-dimensional inferencemodel misspecificationmodel selectionuncertainty quantification

Mathematics Subject Classification ID

Statistics (62-XX)

Abstract: We study frequentist properties of Bayesian and $L_0$ model selection, with a focus on (potentially non-linear) high-dimensional regression. We propose a construction to study how posterior probabilities and normalized $L_0$ criteria concentrate on the (Kullback-Leibler) optimal model and other subsets of the model space. When such concentration occurs, one also bounds the frequentist probabilities of selecting the correct model, type I and type II errors. These results hold generally, and help validate the use of posterior probabilities and $L_0$ criteria to control frequentist error probabilities associated to model selection and hypothesis tests. Regarding regression, we help understand the effect of the sparsity imposed by the prior or the $L_0$ penalty, and of problem characteristics such as the sample size, signal-to-noise, dimension and true sparsity. A particular finding is that one may use less sparse formulations than would be asymptotically optimal, but still attain consistency and often also significantly better finite-sample performance. We also prove new results related to misspecifying the mean or covariance structures, and give tighter rates for certain non-local priors than currently available.

Cites work

• A general framework for Bayes structured linear models
• Bayes and empirical Bayes: do they merge?
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• Bayesian block-diagonal variable selection and model averaging
• Bayesian linear regression with sparse priors
• Bayesian model selection in high-dimensional settings
• Bayesian sparse linear regression with unknown symmetric error
• Bayesian variable selection with shrinking and diffusing priors
• Benchmark priors for Bayesian model averaging.
• Empirical Bayes posterior concentration in sparse high-dimensional linear models
• Estimating the dimension of a model
• High-dimensional statistics. A non-asymptotic viewpoint
• Information-Theoretic Limits on Sparsity Recovery in the High-Dimensional and Noisy Setting
• On the computational complexity of high-dimensional Bayesian variable selection
• On the use of non-local prior densities in Bayesian hypothesis tests
• Optimal predictive model selection.
• Optimal Sample Size for Multiple Testing
• Posterior concentration for Bayesian regression trees and forests
• Scalable Bayesian variable selection using nonlocal prior densities in ultrahigh-dimensional settings
• The risk inflation criterion for multiple regression

Cited in

(4)

• Semiparametric local variable selection under misspecification
• A metropolized adaptive subspace algorithm for high-dimensional Bayesian variable selection
• Bayesian inference on hierarchical nonlocal priors in generalized linear models
• Inference for multiple treatment effects using confounder importance learning

This page was built for publication: Concentration of posterior model probabilities and normalized \({L_0}\) criteria

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6202921)