Black-box optimization using geodesics in statistical manifolds

DOI10.3390/E17010304zbMATH Open1338.94029arXiv1309.7168OpenAlexW2061414435MaRDI QIDQ296505FDOQ296505

Authors: Jérémy Bensadon 

Publication date: 15 June 2016

Published in: Entropy (Search for Journal in Brave)

Abstract: Information geometric optimization (IGO) is a general framework for stochastic optimization problems aiming at limiting the influence of arbitrary parametrization choices. The initial problem is transformed into the optimization of a smooth function on a Riemannian manifold, defining a parametrization-invariant first order differential equation. However, in practice, it is necessary to discretize time, and then, parametrization invariance holds only at first order in the step size. We define the Geodesic IGO update (GIGO), which uses the Riemannian manifold structure to obtain an update entirely independent from the parametrization of the manifold. We test it with classical objective functions. Thanks to Noether's theorem from classical mechanics, we find an efficient way to write a first order differential equation satisfied by the geodesics of the statistical manifold of Gaussian distributions, and thus to compute the corresponding GIGO update. We then compare GIGO, pure rank-$mu$ CMA-ES cite{CMATuto} and xNES cite{xNES} (two previous algorithms that can be recovered by the IGO framework), and show that while the GIGO and xNES updates coincide when the mean is fixed, they are different in general, contrary to previous intuition. We then define a new algorithm (Blockwise GIGO) that recovers the xNES update from abstract principles.

Full work available at URL: https://arxiv.org/abs/1309.7168

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Information theory (general) (94A15)

Cites Work

• Learning probability distributions in continuous evolutionary algorithms -- a comparative review
• Title not available (Why is that?)
• Combinatorial optimization with information geometry: the Newton method
• Title not available (Why is that?)

Cited In (3)

• Warped Riemannian metrics for location-scale models
• Koszul information geometry and Souriau geometric temperature/capacity of Lie group thermodynamics
• Statistical manifolds with almost quaternionic structures and quaternionic Kähler-like statistical submersions

This page was built for publication: Black-box optimization using geodesics in statistical manifolds

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q296505)