Variational inference for Bayesian bridge regression

DOI10.1007/S11222-023-10317-0MaRDI QIDQ6089223zbMATH OpenFDO

Authors Carlos Tadeu Pagani Zanini, Helio S. Migon, Ronaldo Dias

Publication date 17 November 2023

Published in Statistics and Computing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2205.09515

zbMATH Keywords

Bayesian inferencebridge penalizationsplinesvariational inference

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Bayesian inference (62F15) Ridge regression; shrinkage estimators (Lasso) (62J07)

Abstract: We study the implementation of Automatic Differentiation Variational inference (ADVI) for Bayesian inference on regression models with bridge penalization. The bridge approach uses $el{l}_{alpha}$ norm, with $alphain(0,+infty)$ to define a penalization on large values of the regression coefficients, which includes the Lasso ($alpha=1$) and ridge $(alpha=2)$ penalizations as special cases. Full Bayesian inference seamlessly provides joint uncertainty estimates for all model parameters. Although MCMC aproaches are available for bridge regression, it can be slow for large dataset, specially in high dimensions. The ADVI implementation allows the use of small batches of data at each iteration (due to stochastic gradient based algorithms), therefore speeding up computational time in comparison with MCMC. We illustrate the approach on non-parametric regression models with B-splines, although the method works seamlessly for other choices of basis functions. A simulation study shows the main properties of the proposed method.

Recommendations

• Bayesian bridge regression
• Sparse Online Variational Bayesian Regression
• Scalable variational inference for Bayesian variable selection in regression, and its accuracy in genetic association studies
• Bridge regression: adaptivity and group selection
• Variational inference for heteroscedastic semiparametric regression

Cites work

• A Statistical View of Some Chemometrics Regression Tools
• A variational Bayes approach to a semiparametric regression using Gaussian process priors
• Adaptive subgradient methods for online learning and stochastic optimization
• Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (with discussion)
• Automatic Bayesian Curve Fitting
• Automatic differentiation variational inference
• Bayesian adaptive Lasso
• Bayesian backfitting. (With comments and a rejoinder).
• Bayesian bridge regression
• Bayesian forecasting and dynamic models.
• Efficient Bayesian multivariate surface regression
• Flexible smoothing with B-splines and penalties. With comments and a rejoinder by the authors
• Flexible smoothing with P-splines: a unified approach
• Generalized additive models
• scientific article; zbMATH DE number 1799343 (Why is no real title available?)
• scientific article; zbMATH DE number 700016 (Why is no real title available?)
• scientific article; zbMATH DE number 6982909 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• scientific article; zbMATH DE number 6438182 (Why is no real title available?)
• scientific article; zbMATH DE number 6734253 (Why is no real title available?)
• Penalized regression, standard errors, and Bayesian Lassos
• Regularization and Variable Selection Via the Elastic Net
• Ridge Regression: Applications to Nonorthogonal Problems
• Ridge Regression: Biased Estimation for Nonorthogonal Problems
• Simultaneous probability statements for Bayesian P-splines
• The Bayesian Bridge
• The Bayesian elastic net
• The Bayesian Lasso
• Variational inference for count response semiparametric regression
• Variational inference for heteroscedastic semiparametric regression

Cited in

(2)

• Bayesian bridge regression
• SURE-tuned bridge regression

This page was built for publication: Variational inference for Bayesian bridge regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6089223)