Information Relaxation and Dual Formulation of Controlled Markov Diffusions

DOI10.1109/TAC.2015.2418672MaRDI QIDQ2982547zbMATH OpenOpenAlexFDO

Authors Fan Ye, Enlu Zhou

Publication date 16 May 2017

Published in IEEE Transactions on Automatic Control (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1303.2388

Mathematics Subject Classification ID

Optimality conditions for problems involving randomness (49K45) Dynamic programming in optimal control and differential games (49L20) Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20)

Abstract: Information relaxation and duality in Markov decision processes have been studied recently by several researchers with the goal to derive dual bounds on the value function. In this paper we extend this dual formulation to controlled Markov diffusions: in a similar way we relax the constraint that the decision should be made based on the current information and impose penalty to punish the access to the information in advance. We establish the weak duality, strong duality and complementary slackness results in a parallel way as those in Markov decision processes. We explore the structure of the optimal penalties and expose the connection between Markov decision processes and controlled Markov diffusions. We demonstrate the use of the dual representation for controlled Markov diffusions in a classic dynamic portfolio choice problem. We evaluate the lower bounds on the expected utility by Monte Carlo simulation under a sub-optimal policy, and we propose a new class of penalties to derive upper bounds with little extra computation. The small gaps between the lower bounds and upper bounds indicate that the available policy is near optimal as well as the effectiveness of our proposed penalty in the dual method.

Recommendations

• Information relaxations and duality in stochastic dynamic programs
• Information Relaxations, Duality, and Convex Stochastic Dynamic Programs
• The variational principle for optimal control of diffusions with partial information
• The Maximum Principle for Optimal Control of Diffusions with Partial Information
• scientific article; zbMATH DE number 3972777
• An information based approach to stochastic control problems
• scientific article; zbMATH DE number 29647
• Approximations to stochastic dynamic programs via information relaxation duality
• Partial Information Linear Quadratic Control for Jump Diffusions
• Convex Duality Approach to the Optimal Control of Diffusions

Cited in

(4)

• Solving optimal stopping problems under model uncertainty via empirical dual optimisation
• Solution to the variation problem for information path functional of a controlled random process
• Information Relaxations, Duality, and Convex Stochastic Dynamic Programs
• Fast estimation of true bounds on Bermudan option prices under jump-diffusion processes

This page was built for publication: Information Relaxation and Dual Formulation of Controlled Markov Diffusions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2982547)