Online inference with multi-modal likelihood functions

DOI10.1214/21-AOS2076MaRDI QIDQ2073695zbMATH OpenOpenAlexFDO

Authors Mathieu Gerber, Kari Heine

Publication date 7 February 2022

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1809.11108

zbMATH Keywords

multi-modal likelihood functionsonline inferencescalable inference

Mathematics Subject Classification ID

Point estimation (62F10) Asymptotic properties of parametric estimators (62F12) Sequential estimation (62L12) Online algorithms; streaming algorithms (68W27) Nonconvex programming, global optimization (90C26)

Abstract: Let $(Y_t{)}_{tgeq1}$ be a sequence of i.i.d. observations and $f_{h}eta,hetainmathbb{R}^d$ be a parametric model. We introduce a new online algorithm for computing a sequence $(hat{heta}_t{)}_{tgeq1}$ which is shown to converge almost surely to $ext{argmax}_{hetainmathbb{R}^d}mathbbE[log{f}_{h}eta(Y_1)]$ at rate $mathcalO(log(t{)}^{(1+varepsilon)/2}{t}^{-1/2})$, with $varepsilon>0$ a user specified parameter. This convergence result is obtained under standard conditions on the statistical model and, most notably, we allow the mapping $hetamapstomathbbE[log{f}_{h}eta(Y_1)]$ to be multi-modal. However, the computational cost to process each observation grows exponentially with the dimension of $heta$, which makes the proposed approach applicable to low or moderate dimensional problems only. We also derive a version of the estimator $hat{heta}_t$ which is well suited to Student-t linear regression models. The corresponding estimator of the regression coefficients is robust to the presence of outliers, as shown by experiments on simulated and real data, and thus, as a by-product of this work, we obtain a new online and adaptive robust estimation method for linear regression models.

Recommendations

• On-line expectation-maximization algorithm for latent data models
• Online expectation maximization based algorithms for inference in hidden Markov models
• Online inference in high-dimensional generalized linear models with streaming data
• Online model selection based on the variational Bayes
• On-line maximum likelihood prediction with respect to general loss functions
• Online k-MLE for mixture modeling with exponential families
• Online Multitask Learning
• scientific article; zbMATH DE number 1569116
• Online but accurate inference for latent variable models with local Gibbs sampling
• Online Bayesian inference and learning of Gaussian-process state-space models

Cites work

• Asymptotic and finite-sample properties of estimators based on stochastic gradients
• Asymptotic Statistics
• Convergence and convergence rate of stochastic gradient search in the case of multiple and non-isolated extrema
• scientific article; zbMATH DE number 5797591 (Why is no real title available?)
• Multivariate Student-t regression models: Pitfalls and inference
• Objective Bayesian analysis for the Student-t linear regression
• Scalable estimation strategies based on stochastic approximations: classical results and new insights
• The Bernstein-von Mises theorem under misspecification

Cited in

(1)

• Online optimisation for dynamic electrical impedance tomography

This page was built for publication: Online inference with multi-modal likelihood functions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2073695)