Neuronized Priors for Bayesian Sparse Linear Regression
From MaRDI portal
Abstract: Although Bayesian variable selection methods have been intensively studied, their routine use in practice has not caught up with their non-Bayesian counterparts such as Lasso, likely due to difficulties in both computations and flexibilities of prior choices. To ease these challenges, we propose the neuronized priors to unify and extend some popular shrinkage priors, such as Laplace, Cauchy, horseshoe, and spike-and-slab priors. A neuronized prior can be written as the product of a Gaussian weight variable and a scale variable transformed from Gaussian via an activation function. Compared with classic spike-and-slab priors, the neuronized priors achieve the same explicit variable selection without employing any latent indicator variables, which results in both more efficient and flexible posterior sampling and more effective posterior modal estimation. Theoretically, we provide specific conditions on the neuronized formulation to achieve the optimal posterior contraction rate, and show that a broadly applicable MCMC algorithm achieves an exponentially fast convergence rate under the neuronized formulation. We also examine various simulated and real data examples and demonstrate that using the neuronization representation is computationally more or comparably efficient than its standard counterpart in all well-known cases. An R package NPrior is provided in the CRAN for using neuronized priors in Bayesian linear regression.
Recommendations
- Bayesian linear regression with sparse priors
- Bayesian variable selection with shrinking and diffusing priors
- Fast Bayesian variable selection for high dimensional linear models: marginal solo spike and slab priors
- Sparsity information and regularization in the horseshoe and other shrinkage priors
- Variable selection using shrinkage priors
Cites work
- A majorization-minimization approach to variable selection using spike and slab priors
- Adaptive Lasso for sparse high-dimensional regression models
- Asymptotic optimality of one-group shrinkage priors in sparse high-dimensional problems
- Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
- Bayesian estimation of sparse signals with a continuous spike-and-slab prior
- Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables
- Bayesian linear regression with sparse priors
- Bayesian model selection in high-dimensional settings
- Bayesian Variable Selection in Linear Regression
- Bayesian variable selection regression for genome-wide association studies and other large-scale problems
- Bayesian variable selection with shrinking and diffusing priors
- Choice of hierarchical priors: Admissibility in estimation of normal means
- Component-wise Markov chain Monte Carlo: uniform and geometric ergodicity under mixing and composition
- Conditions for posterior contraction in the sparse normal means problem
- Consistent model selection criteria on high dimensions
- Convergence of Slice Sampler Markov Chains
- Decoupling shrinkage and selection in Bayesian linear models: a posterior summary perspective
- Dirichlet-Laplace priors for optimal shrinkage
- EMVS: the EM approach to Bayesian variable selection
- Estimating the dimension of a model
- Exploration, normalization, and summaries of high density oligonucleotide array probe level data
- Extended Bayesian information criteria for model selection with large model spaces
- Fixed-Width Output Analysis for Markov Chain Monte Carlo
- General state space Markov chains and MCMC algorithms
- Generalised Gibbs sampler and multigrid Monte Carlo for Bayesian computation
- Generalized double Pareto shrinkage
- Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms
- Geometric ergodicity of Metropolis algorithms
- scientific article; zbMATH DE number 4070082 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 3388498 (Why is no real title available?)
- Implementing MCMC: estimating with confidence
- Kernel estimators of asymptotic variance for adaptive Markov chain Monte Carlo
- Lasso, fractional norm and structured sparse estimation using a Hadamard product parametrization
- Learning representations by back-propagating errors
- Lookahead and piloting strategies for variable selection
- Nearly optimal Bayesian shrinkage for high-dimensional regression
- Needles and straw in a haystack: posterior concentration for possibly sparse sequences
- Nonparametric independence screening in sparse ultra-high-dimensional additive models
- On Bayesian model and variable selection using MCMC
- On the geometric ergodicity of two-variable Gibbs samplers
- On the use of non-local prior densities in Bayesian hypothesis tests
- Optimization by simulated annealing
- Parameter Expansion for Data Augmentation
- Posterior model probabilities via path‐based pairwise priors
- Rates of convergence of the Hastings and Metropolis algorithms
- Reversible jump Markov chain Monte Carlo computation and Bayesian model determination
- Scalable Bayesian variable selection using nonlocal prior densities in ultrahigh-dimensional settings
- Shotgun Stochastic Search for “Largep” Regression
- Skinny Gibbs: a consistent and scalable Gibbs sampler for model selection
- Smoothly clipped absolute deviation on high dimensions
- Statistics for high-dimensional data. Methods, theory and applications.
- The Bayesian Lasso
- The Collapsed Gibbs Sampler in Bayesian Computations with Applications to a Gene Regulation Problem
- The horseshoe estimator for sparse signals
- The horseshoe estimator: posterior concentration around nearly black vectors
- The Multiple-Try Method and Local Optimization in Metropolis Sampling
- The spike-and-slab LASSO
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
- Variance prior forms for high-dimensional Bayesian variable selection
Cited in
(11)- Bayesian Bootstrap Spike-and-Slab LASSO
- scientific article; zbMATH DE number 6378124 (Why is no real title available?)
- Combining neural networks for function approximation under conditions of sparse data: the biased regression approach
- Model selection in Bayesian neural networks via horseshoe priors
- Horseshoe Regularisation for Machine Learning in Complex and Deep Models1
- On posterior consistency of Bayesian factor models in high dimensions
- Genome-wide search algorithms for identifying dynamic gene co-expression via Bayesian variable selection
- Bayesian Inference Using the Proximal Mapping: Uncertainty Quantification Under Varying Dimensionality
- Controlled Discovery and Localization of Signals via Bayesian Linear Programming
- Provably Efficient Posterior Sampling for Sparse Linear Regression via Measure Decomposition
- Variational inference for sparse Poisson regression
This page was built for publication: Neuronized Priors for Bayesian Sparse Linear Regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6110693)
