A Latent Variational Framework for Stochastic Optimization

Publication date 5 May 2019

Calculus of variations and optimal control; optimization (49-XX) Probability theory and stochastic processes (60-XX) Numerical solutions to stochastic differential and integral equations (65C30)

Abstract: This paper provides a unifying theoretical framework for stochastic optimization algorithms by means of a latent stochastic variational problem. Using techniques from stochastic control, the solution to the variational problem is shown to be equivalent to that of a Forward Backward Stochastic Differential Equation (FBSDE). By solving these equations, we recover a variety of existing adaptive stochastic gradient descent methods. This framework establishes a direct connection between stochastic optimization algorithms and a secondary Bayesian inference problem on gradients, where a prior measure on noisy gradient observations determines the resulting algorithm.

This page was built for publication: A Latent Variational Framework for Stochastic Optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6318237)