Tractable Bayesian Variable Selection: Beyond Normality

DOI10.1080/01621459.2017.1371025zbMATH Open1409.62136arXiv1609.01708OpenAlexW2516618917WikidataQ92582472 ScholiaQ92582472MaRDI QIDQ3121566FDOQ3121566

F. J. Rubio, David Rossell

Publication date: 20 March 2019

Published in: Journal of the American Statistical Association (Search for Journal in Brave)

Abstract: Bayesian variable selection often assumes normality, but the effects of model misspecification are not sufficiently understood. There are sound reasons behind this assumption, particularly for large $p$: ease of interpretation, analytical and computational convenience. More flexible frameworks exist, including semi- or non-parametric models, often at the cost of some tractability. We propose a simple extension of the Normal model that allows for skewness and thicker-than-normal tails but preserves tractability. It leads to easy interpretation and a log-concave likelihood that facilitates optimization and integration. We characterize asymptotically parameter estimation and Bayes factor rates, in particular studying the effects of model misspecification. Under suitable conditions misspecified Bayes factors are consistent and induce sparsity at the same asymptotic rates than under the correct model. However, the rates to detect signal are altered by an exponential factor, often resulting in a loss of sensitivity. These deficiencies can be ameliorated by inferring the error distribution from the data, a simple strategy that can improve inference substantially. Our work focuses on the likelihood and can thus be combined with any likelihood penalty or prior, but here we focus on non-local priors to induce extra sparsity and ameliorate finite-sample effects caused by misspecification. Our results highlight the practical importance of focusing on the likelihood rather than solely on the prior, when it comes to Bayesian variable selection. The methodology is available in R package `mombf'.

Full work available at URL: https://arxiv.org/abs/1609.01708

zbMATH Keywords

variable selectionmodel misspecificationrobust regressionBayes factorstwo-piece errors

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Linear regression; mixed models (62J05) Bayesian problems; characterization of Bayes procedures (62C10) Robustness and adaptive procedures (parametric inference) (62F35)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Sparse least trimmed squares regression for analyzing high-dimensional large data sets
• Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
• Scalable Bayesian Variable Selection Using Nonlocal Prior Densities in Ultrahigh-dimensional Settings
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Robust regression: Asymptotics, conjectures and Monte Carlo
• The epsilon-skew-normal distribution for analyzing near-normal data
• The two-piece normal, binormal, or double Gaussian distribution: its origin and rediscoveries
• Nonparametric Bayes Conditional Distribution Modeling With Variable Selection
• On Bayesian Modeling of Fat Tails and Skewness
• Bayes Variable Selection in Semiparametric Linear Models
• Statistical inference for a general class of asymmetric distributions
• Optimal predictive model selection.
• Adaptive robust variable selection
• Bayesian robust transformation and variable selection: A unified approach
• Asymptotic Normality and Consistency of the Least Squares Estimators for Families of Linear Regressions
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• Robust regression through the Huber's criterion and adaptive lasso penalty
• Statistical consistency and asymptotic normality for high-dimensional robust \(M\)-estimators
• An Algorithm for Least-Squares Estimation of Nonlinear Parameters
• Bayesian Model Selection in High-Dimensional Settings
• A method for the solution of certain non-linear problems in least squares
• Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it
• Methods for the two-piece normal distribution
• Criteria for Bayesian model choice with application to variable selection
• Weighted LAD-LASSO method for robust parameter estimation and variable selection in regression
• On the use of Non-Local Prior Densities in Bayesian Hypothesis Tests
• High-dimensional heteroscedastic regression with an application to eQTL data analysis
• Newton’s Method with a Model Trust Region Modification
• Bayesian variable selection in quantile regression
• Inference in two-piece location-scale models with Jeffreys priors
• Learning without concentration for general loss functions
• Weighted Wilcoxon‐Type Smoothly Clipped Absolute Deviation Method
• Asymptotics for L₁‐estimators of regression parameters under heteroscedasticityY
• Tests of Linear Hypotheses and l"1 Estimation
• On numerical aspects of Bayesian model selection in high and ultrahigh-dimensional settings
• On Fixed-Width Confidence Bounds for Regression Parameters
• Robust nonnegative garrote variable selection in linear regression
• Natural (Non‐)Informative Priors for Skew‐symmetric Distributions
• Flexible objective Bayesian linear regression with applications in survival analysis

Cited In (14)

• Scalable Importance Tempering and Bayesian Variable Selection
• A loss-based prior for variable selection in linear regression methods
• Variations of power-expected-posterior priors in normal regression models
• Bayesian variable selection for non‐Gaussian responses: a marginally calibrated copula approach
• Bayesian high-dimensional semi-parametric inference beyond sub-Gaussian errors
• Bayesian variable selection and estimation in quantile regression using a quantile-specific prior
• Letter to the Editor: ‘On Quantile‐based Asymmetric Family of Distributions: Properties and Inference’
• mombf
• A novel variational Bayesian method for variable selection in logistic regression models
• Bayesian effect selection in structured additive distributional regression models
• Selecting the normal population with the best regression value -- a Bayesian approach
• Additive Bayesian variable selection under censoring and misspecification
• Adaptive random neighbourhood informed Markov chain Monte Carlo for high-dimensional Bayesian variable selection
• Bayesian bandwidth test and selection for high-dimensional banded precision matrices

Uses Software

• R

This page was built for publication: Tractable Bayesian Variable Selection: Beyond Normality

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3121566)