Semiparametric regression using variational approximations
From MaRDI portal
Publication:5208080
DOI10.1080/01621459.2018.1518235zbMATH Open1428.62156arXiv1810.01949OpenAlexW3104810320WikidataQ129280276 ScholiaQ129280276MaRDI QIDQ5208080FDOQ5208080
Authors:
Publication date: 15 January 2020
Published in: Journal of the American Statistical Association (Search for Journal in Brave)
Abstract: Semiparametric regression offers a flexible framework for modeling non-linear relationships between a response and covariates. A prime example are generalized additive models where splines (say) are used to approximate non-linear functional components in conjunction with a quadratic penalty to control for overfitting. Estimation and inference are then generally performed based on the penalized likelihood, or under a mixed model framework. The penalized likelihood framework is fast but potentially unstable, and choosing the smoothing parameters needs to be done externally using cross-validation, for instance. The mixed model framework tends to be more stable and offers a natural way for choosing the smoothing parameters, but for non-normal responses involves an intractable integral. In this article, we introduce a new framework for semiparametric regression based on variational approximations. The approach possesses the stability and natural inference tools of the mixed model framework, while achieving computation times comparable to using penalized likelihood. Focusing on generalized additive models, we derive fully tractable variational likelihoods for some common response types. We present several features of the variational approximation framework for inference, including a variational information matrix for inference on parametric components, and a closed-form update for estimating the smoothing parameter. We demonstrate the consistency of the variational approximation estimates, and an asymptotic normality result for the parametric component of the model. Simulation studies show the variational approximation framework performs similarly to and sometimes better than currently available software for fitting generalized additive models.
Full work available at URL: https://arxiv.org/abs/1810.01949
Recommendations
- A variational Bayes approach to a semiparametric regression using Gaussian process priors
- Variational inference for count response semiparametric regression
- Variational inference for heteroscedastic semiparametric regression
- Semiparametric regression and graphical models
- Semiparametric mixture of additive regression models
Cites Work
- Semiparametric Regression
- Generalized additive models. An introduction with R.
- Simultaneous confidence bands for penalized~spline~estimators
- Flexible smoothing with \(B\)-splines and penalties. With comments and a rejoinder by the authors
- Variational inference for count response semiparametric regression
- Title not available (Why is that?)
- Fast Stable Restricted Maximum Likelihood and Marginal Likelihood Estimation of Semiparametric Generalized Linear Models
- Strictly Proper Scoring Rules, Prediction, and Estimation
- Penalized Spline Estimation for Varying-Coefficient Models
- Inference in Generalized Additive Mixed Models by Using Smoothing Splines
- Penalized Spline Estimation for Partially Linear Single-Index Models
- Approximate Inference in Generalized Linear Mixed Models
- Generalized additive models
- Explaining variational approximations
- Title not available (Why is that?)
- Generalized Additive Models for Location, Scale and Shape
- Smoothing parameter selection for a class of semiparametric linear models
- A correlated topic model of science
- Nonconcave penalized likelihood with a diverging number of parameters.
- On the use of bootstrap with variational inference: theory, interpretation, and a two-sample test example
- Some asymptotic results on generalized penalized spline smoothing
- Consistent variable selection in additive models
- A note on smoothing parameter selection for penalized spline smoothing
- Asymptotic normality and valid inference for Gaussian variational approximation
- On the asymptotics of penalized spline smoothing
- A variational Bayes approach to variable selection
- Hierarchical selection of fixed and random effects in generalized linear mixed models
Cited In (16)
- Fast, Approximate Maximum Likelihood Estimation of Log-Gaussian Cox Processes
- Variational inference: uncertainty quantification in additive models
- A penalized least product relative error loss function based on wavelet decomposition for non-parametric multiplicative additive models
- Generalized functional linear model with a point process predictor
- Sufficient dimension reduction for clustered data via finite mixture modelling
- Variational approximation for heteroscedastic linear models and matching pursuit algorithms
- Ultra-Fast Approximate Inference Using Variational Functional Mixed Models
- A variational Bayes approach to a semiparametric regression using Gaussian process priors
- Variational inference for count response semiparametric regression
- Fast and universal estimation of latent variable models using extended variational approximations
- On Variance Estimation in Semiparametric Regression Models
- Variational inference for heteroscedastic semiparametric regression
- Laplace approximations for fast Bayesian inference in generalized additive models based on P-splines
- On the estimation of variance parameters in non-standard generalised linear mixed models: application to penalised smoothing
- A new algorithm for fitting semi-parametric variance regression models
- Semiparametric regression and graphical models
Uses Software
This page was built for publication: Semiparametric regression using variational approximations
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5208080)
