Dynamics of coordinate ascent variational inference: A case study in 2D Ising models

DOI10.3390/E22111263arXiv2007.06715WikidataQ104108740 ScholiaQ104108740MaRDI QIDQ6345017FDOQ6345017

Authors: Sean Plummer, Debdeep Pati, Anirban Bhattacharya 

Publication date: 13 July 2020

Abstract: Variational algorithms have gained prominence over the past two decades as a scalable computational environment for Bayesian inference. In this article, we explore tools from the dynamical systems literature to study convergence of coordinate ascent algorithms for mean field variational inference. Focusing on the Ising model defined on two nodes, we fully characterize the dynamics of the sequential coordinate ascent algorithm and its parallel version. We observe that in the regime where the objective function is convex, both the algorithms are stable and exhibit convergence to the unique fixed point. Our analyses reveal interesting {em discordances} between these two versions of the algorithm in the region when the objective function is non-convex. In fact, the parallel version exhibits a periodic oscillatory behavior which is absent in the sequential version. Drawing intuition from the Markov chain Monte Carlo literature, we {em empirically} show that a parameter expansion of the Ising model, popularly called as the Edward--Sokal coupling, leads to an enlargement of the regime of convergence to the global optima.

This page was built for publication: Dynamics of coordinate ascent variational inference: A case study in 2D Ising models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6345017)