Semiparametric regression using variational approximations
From MaRDI portal
Publication:5208080
Abstract: Semiparametric regression offers a flexible framework for modeling non-linear relationships between a response and covariates. A prime example are generalized additive models where splines (say) are used to approximate non-linear functional components in conjunction with a quadratic penalty to control for overfitting. Estimation and inference are then generally performed based on the penalized likelihood, or under a mixed model framework. The penalized likelihood framework is fast but potentially unstable, and choosing the smoothing parameters needs to be done externally using cross-validation, for instance. The mixed model framework tends to be more stable and offers a natural way for choosing the smoothing parameters, but for non-normal responses involves an intractable integral. In this article, we introduce a new framework for semiparametric regression based on variational approximations. The approach possesses the stability and natural inference tools of the mixed model framework, while achieving computation times comparable to using penalized likelihood. Focusing on generalized additive models, we derive fully tractable variational likelihoods for some common response types. We present several features of the variational approximation framework for inference, including a variational information matrix for inference on parametric components, and a closed-form update for estimating the smoothing parameter. We demonstrate the consistency of the variational approximation estimates, and an asymptotic normality result for the parametric component of the model. Simulation studies show the variational approximation framework performs similarly to and sometimes better than currently available software for fitting generalized additive models.
Recommendations
- A variational Bayes approach to a semiparametric regression using Gaussian process priors
- Variational inference for count response semiparametric regression
- Variational inference for heteroscedastic semiparametric regression
- Semiparametric regression and graphical models
- Semiparametric mixture of additive regression models
Cites work
- scientific article; zbMATH DE number 3768770 (Why is no real title available?)
- scientific article; zbMATH DE number 3567782 (Why is no real title available?)
- A correlated topic model of science
- A note on smoothing parameter selection for penalized spline smoothing
- A variational Bayes approach to variable selection
- Approximate Inference in Generalized Linear Mixed Models
- Asymptotic normality and valid inference for Gaussian variational approximation
- Consistent variable selection in additive models
- Explaining variational approximations
- Fast Stable Restricted Maximum Likelihood and Marginal Likelihood Estimation of Semiparametric Generalized Linear Models
- Flexible smoothing with \(B\)-splines and penalties. With comments and a rejoinder by the authors
- Generalized Additive Models for Location, Scale and Shape
- Generalized additive models
- Generalized additive models. An introduction with R.
- Hierarchical selection of fixed and random effects in generalized linear mixed models
- Inference in Generalized Additive Mixed Models by Using Smoothing Splines
- Nonconcave penalized likelihood with a diverging number of parameters.
- On the asymptotics of penalized spline smoothing
- On the use of bootstrap with variational inference: theory, interpretation, and a two-sample test example
- Penalized Spline Estimation for Partially Linear Single-Index Models
- Penalized Spline Estimation for Varying-Coefficient Models
- Semiparametric Regression
- Simultaneous confidence bands for penalized~spline~estimators
- Smoothing parameter selection for a class of semiparametric linear models
- Some asymptotic results on generalized penalized spline smoothing
- Strictly Proper Scoring Rules, Prediction, and Estimation
- Variational inference for count response semiparametric regression
Cited in
(16)- Variational inference: uncertainty quantification in additive models
- On the estimation of variance parameters in non-standard generalised linear mixed models: application to penalised smoothing
- Ultra-Fast Approximate Inference Using Variational Functional Mixed Models
- A new algorithm for fitting semi-parametric variance regression models
- Semiparametric regression and graphical models
- Fast, Approximate Maximum Likelihood Estimation of Log-Gaussian Cox Processes
- Generalized functional linear model with a point process predictor
- A penalized least product relative error loss function based on wavelet decomposition for non-parametric multiplicative additive models
- Sufficient dimension reduction for clustered data via finite mixture modelling
- Variational approximation for heteroscedastic linear models and matching pursuit algorithms
- Variational inference for count response semiparametric regression
- Laplace approximations for fast Bayesian inference in generalized additive models based on P-splines
- On Variance Estimation in Semiparametric Regression Models
- A variational Bayes approach to a semiparametric regression using Gaussian process priors
- Variational inference for heteroscedastic semiparametric regression
- Fast and universal estimation of latent variable models using extended variational approximations
This page was built for publication: Semiparametric regression using variational approximations
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5208080)
