Explicit solution of relative entropy weighted control

DOI10.1016/J.SYSCONLE.2014.08.001zbMATH Open1297.93181arXiv1205.6946OpenAlexW2951021288MaRDI QIDQ465547FDOQ465547

Authors: Hilbert J. Kappen, Joris Bierkens 

Publication date: 23 October 2014

Published in: Systems \& Control Letters (Search for Journal in Brave)

Abstract: We consider the minimization over probability measures of the expected value of a random variable, regularized by relative entropy with respect to a given probability distribution. In the general setting we provide a complete characterization of the situations in which a finite optimal value exists and the situations in which a minimizing probability distribution exists. Specializing to the case where the underlying probability distribution is Wiener measure, we characterize finite relative entropy changes of measure in terms of square integrability of the corresponding change of drift. For the optimal change of measure for the relative entropy weighted optimization, an expression involving the Malliavin derivative of the cost random variable is derived. The theory is illustrated by its application to several examples, including the case where the cost variable is the maximum of a standard Brownian motion over a finite time horizon. For this example we obtain an exact optimal drift, as well as an approximation of the optimal drift through a Monte-Carlo algorithm.

Full work available at URL: https://arxiv.org/abs/1205.6946

Recommendations

• scientific article; zbMATH DE number 799798
• On Minimization and Maximization of Entropy in Various Disciplines
• scientific article; zbMATH DE number 6987429
• Optimal control of measures
• About the analogy between optimal transport and minimal entropy

zbMATH Keywords

Malliavin calculusBrownian motionrelative entropystochastic optimal controlMonte-Carlo samplingpath integral controlItô calculus

Mathematics Subject Classification ID

Brownian motion (60J65) Optimal stochastic control (93E20)

Cites Work

• The Malliavin Calculus and Related Topics
• A variational representation for certain functionals of Brownian motion
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Optimal Control of the Running Max
• Title not available (Why is that?)
• Malliavin calculus method for asymptotic expansion of dual control problems
• Title not available (Why is that?)
• Title not available (Why is that?)
• A generalized clark representation formula, with application to optimal portfolios
• Title not available (Why is that?)
• Title not available (Why is that?)

Cited In (9)

• Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space
• The most probable transition paths of stochastic dynamical systems: a sufficient and necessary characterisation
• Information projection on Banach spaces with applications to state independent KL-weighted optimal control
• Linear PDEs and eigenvalue problems corresponding to ergodic stochastic optimization problems on compact manifolds
• Title not available (Why is that?)
• Minimal Kullback–Leibler Divergence for Constrained Lévy–Itô Processes
• Variational and optimal control representations of conditioned and driven processes
• Robust decentralized control of coupled systems via risk sensitive control of decoupled or simple models with measure change
• Nonlinear stochastic receding horizon control: stability, robustness and Monte Carlo methods for control approximation

This page was built for publication: Explicit solution of relative entropy weighted control

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q465547)