Learning from a lot: empirical Bayes for high-dimensional model-based prediction
From MaRDI portal
Publication:4629271
DOI10.1111/SJOS.12335zbMATH Open1417.62018arXiv1709.04192OpenAlexW2805488672WikidataQ90102019 ScholiaQ90102019MaRDI QIDQ4629271FDOQ4629271
Authors: Mark A. van de Wiel, Dennis E. Te Beest, Magnus M. Münch 
Publication date: 21 March 2019
Published in: Scandinavian Journal of Statistics (Search for Journal in Brave)
Abstract: Empirical Bayes is a versatile approach to `learn from a lot' in two ways: first, from a large number of variables and second, from a potentially large amount of prior information, e.g. stored in public repositories. We review applications of a variety of empirical Bayes methods to several well-known model-based prediction methods including penalized regression, linear discriminant analysis, and Bayesian models with sparse or dense priors. We discuss `formal' empirical Bayes methods which maximize the marginal likelihood, but also more informal approaches based on other data summaries. We contrast empirical Bayes to cross-validation and full Bayes, and discuss hybrid approaches. To study the relation between the quality of an empirical Bayes estimator and , the number of variables, we consider a simple empirical Bayes estimator in a linear model setting. We argue that empirical Bayes is particularly useful when the prior contains multiple parameters which model a priori information on variables, termed `co-data'. In particular, we present two novel examples that allow for co-data. First, a Bayesian spike-and-slab setting that facilitates inclusion of multiple co-data sources and types; second, a hybrid empirical Bayes-full Bayes ridge regression approach for estimation of the posterior predictive interval.
Full work available at URL: https://arxiv.org/abs/1709.04192
Recommendations
- Empirical priors for prediction in sparse high-dimensional linear regression
- Empirical Bayes estimates for large-scale prediction problems
- Empirical Bayes posterior concentration in sparse high-dimensional linear models
- Empirical Bayes regression analysis with many regressors but fewer observations
- Efficient Empirical Bayes Variable Selection and Estimation in Linear Models
Linear regression; mixed models (62J05) Empirical decision procedures; empirical Bayes procedures (62C12)
Cites Work
- IPF-LASSO: integrative \(L_1\)-penalized regression with penalty factors for prediction based on multi-omics data
- EMVS: The EM Approach to Bayesian Variable Selection
- Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (with discussion)
- The Bayesian Lasso
- Regularization and Variable Selection Via the Elastic Net
- Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables
- A review of Bayesian variable selection methods: what, how and which
- Some theory for Fisher's linear discriminant function, `naive Bayes', and some alternatives when there are many more variables than observations
- Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
- Sampling-Based Approaches to Calculating Marginal Densities
- The Group Lasso for Logistic Regression
- Title not available (Why is that?)
- Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
- The Bayesian elastic net
- Spike and slab variable selection: frequentist and Bayesian strategies
- Calibration and empirical Bayes variable selection
- Maximizing Generalized Linear Mixed Model Likelihoods With an Automated Monte Carlo EM Algorithm
- Scalable variational inference for Bayesian variable selection in regression, and its accuracy in genetic association studies
- Bayes and empirical Bayes methods for data analysis.
- Weighted lasso with data integration
- On high-dimensional misspecified mixed model analysis in genome-wide association study
- Large-scale inference. Empirical Bayes methods for estimation, testing, and prediction
- Parametric Empirical Bayes Inference: Theory and Applications
- Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator
- Data Analysis Using Stein's Estimator and its Generalizations
- Needles and straw in a haystack: posterior concentration for possibly sparse sequences
- Ridge regression:some simulations
- Empirical Bayes Gibbs sampling
- Incorporating biological information into linear models: a Bayesian approach to the selection of pathways and genes
- Ridge Estimators in Logistic Regression
- Coupling a stochastic approximation version of EM with an MCMC procedure
- Empirical Bayes prediction intervals in a normal regression model: higher order asymptotics.
- Empirical Bayes Estimates for Large-Scale Prediction Problems
- High-dimensional classification via nonparametric empirical Bayes and maximum likelihood inference
- Marginal maximum likelihood estimation methods for the tuning parameters of ridge, power ridge, and generalized ridge regression
- Empirical Bayesian Estimators for a Poisson Process Propagated in Time
- Laplace Approximation in High-Dimensional Bayesian Regression
- Empirical and fully Bayesian approaches for random effects models in microarray data analysis
- The role of empirical Bayes methodology as a leading principle in modern medical statistics
- Gene network reconstruction using global-local shrinkage priors
Cited In (6)
- Flexible co-data learning for high-dimensional prediction
- Scalable multiple network inference with the joint graphical horseshoe
- A review of uncertainty quantification for density estimation
- Drug sensitivity prediction with normal inverse Gaussian shrinkage informed by external data
- Fast marginal likelihood estimation of penalties for group-adaptive elastic net
- A global-local approach for detecting hotspots in multiple-response regression
Uses Software
This page was built for publication: Learning from a lot: empirical Bayes for high-dimensional model-based prediction
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4629271)
