SURE-tuned bridge regression

DOI10.1007/S11222-023-10350-ZMaRDI QIDQ6190648zbMATH OpenOpenAlexFDO

Authors Jorge Loría, Anindya Bhadra

Publication date 6 February 2024

Published in Statistics and Computing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2212.02709

zbMATH Keywords

bridge regressioncross validationMonte Carlo estimationStein's unbiased risk estimate

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Ridge regression; shrinkage estimators (Lasso) (62J07)

Abstract: Consider the $el{l}_{alpha}$ regularized linear regression, also termed Bridge regression. For $alphain(0,1)$, Bridge regression enjoys several statistical properties of interest such as sparsity and near-unbiasedness of the estimates (Fan and Li, 2001). However, the main difficulty lies in the non-convex nature of the penalty for these values of $alpha$, which makes an optimization procedure challenging and usually it is only possible to find a local optimum. To address this issue, Polson et al. 2013 took a sampling based fully Bayesian approach to this problem, using the correspondence between the Bridge penalty and a power exponential prior on the regression coefficients. However, their sampling procedure relies on Markov chain Monte Carlo (MCMC) techniques, which are inherently sequential and not scalable to large problem dimensions. Cross validation approaches are similarly computation-intensive. To this end, our contribution is a novel non-iterative method to fit a Bridge regression model. The main contribution lies in an explicit formula for Stein's unbiased risk estimate for the out of sample prediction risk of Bridge regression, which can then be optimized to select the desired tuning parameters, allowing us to completely bypass MCMC as well as computation-intensive cross validation approaches. Our procedure yields results in about 1/8th to 1/10th of the computational time compared to iterative schemes, without any appreciable loss in statistical performance. An R implementation is publicly available online at: https://github.com/loriaJ/Sure-tuned_BridgeRegression.

Recommendations

• Variational inference for Bayesian bridge regression
• Bayesian bridge regression
• Selection of tuning parameters in bridge regression models via Bayesian information criterion
• Bridge regression: adaptivity and group selection
• Robust and sparse bridge regression

Cites work

• $l_{q}$ Sparsity Penalized Linear Regression With Cyclic Descent
• Bayesian bridge regression
• Data augmentation for non-Gaussian regression models using variance-mean mixtures
• Degrees of freedom in lasso problems
• Estimation of the mean of a multivariate normal distribution
• Financial Modelling with Jump Processes
• scientific article; zbMATH DE number 3426675 (Why is no real title available?)
• scientific article; zbMATH DE number 3122730 (Why is no real title available?)
• scientific article; zbMATH DE number 509150 (Why is no real title available?)
• scientific article; zbMATH DE number 614990 (Why is no real title available?)
• Inference with normal-gamma prior distributions in regression problems
• Local shrinkage rules, Lévy processes and regularized regression
• Multivariate Exponential Power Distributions as Mixtures of Normal Distributions with Bayesian Applications
• On scale mixtures of normal distributions
• On the ``degrees of freedom of the lasso
• Prediction risk for the horseshoe regression
• Random variate generation for exponentially and polynomially tilted stable distributions
• SparseNet: coordinate descent with nonconvex penalties
• SURE Estimates for a Heteroscedastic Hierarchical Model
• The Bayesian Bridge
• The Estimation of Prediction Error
• Tweedie’s Formula and Selection Bias
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

This page was built for publication: SURE-tuned bridge regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6190648)