Bellman equation and viscosity solutions for mean-field stochastic control problem
From MaRDI portal
Publication:3177924
DOI10.1051/COCV/2017019zbMATH Open1396.93134arXiv1512.07866OpenAlexW2962818191MaRDI QIDQ3177924FDOQ3177924
Authors: Huyên Pham, Xiaoli Wei 
Publication date: 2 August 2018
Published in: ESAIM: Control, Optimisation and Calculus of Variations (Search for Journal in Brave)
Abstract: We consider the stochastic optimal control problem of McKean-Vlasov stochastic differential equation where the coefficients may depend upon the joint law of the state and control. By using feedback controls, we reformulate the problem into a deterministic control problem with only the marginal distribution of the process as controlled state variable, and prove that dynamic programming principle holds in its general form. Then, by relying on the notion of differentiability with respect to pro-bability measures recently introduced by P.L. Lions in [32], and a special It{^o} formula for flows of probability measures, we derive the (dynamic programming) Bellman equation for mean-field stochastic control problem, and prove a veri-fication theorem in our McKean-Vlasov framework. We give explicit solutions to the Bellman equation for the linear quadratic mean-field control problem, with applications to the mean-variance portfolio selection and a systemic risk model. We also consider a notion of lifted visc-sity solutions for the Bellman equation, and show the viscosity property and uniqueness of the value function to the McKean-Vlasov control problem. Finally, we consider the case of McKean-Vlasov control problem with open-loop controls and discuss the associated dynamic programming equation that we compare with the case of closed-loop controls.
Full work available at URL: https://arxiv.org/abs/1512.07866
Recommendations
- Ergodic control of McKean-Vlasov SDEs and associated Bellman equation
- Dynamic Programming for Optimal Control of Stochastic McKean--Vlasov Dynamics
- Optimal control of path-dependent McKean-Vlasov SDEs in infinite-dimension
- Dynamic programming for mean-field type control
- scientific article; zbMATH DE number 5016968
Stochastic partial differential equations (aspects of stochastic analysis) (60H15) Interacting random processes; statistical mechanics type models; percolation theory (60K35) Dynamic programming (90C39) Vlasov equations (35Q83) Dynamic programming in optimal control and differential games (49L20) Viscosity solutions to Hamilton-Jacobi equations in optimal control and differential games (49L25) Control/observation systems governed by partial differential equations (93C20) Optimal stochastic control (93E20)
Cites Work
- Optimal Transport
- Mean field games
- Continuous-time mean-variance portfolio selection: a stochastic LQ framework
- Viscosity solutions of fully nonlinear second-order equations and optimal stochastic control in infinite dimensions. I: The case of bounded stochastic evolutions
- Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle
- Controlled Markov processes and viscosity solutions
- Linear-quadratic optimal control problems for mean-field stochastic differential equations
- Linear-quadratic mean field games
- Title not available (Why is that?)
- Control of McKean-Vlasov dynamics versus mean field games
- Title not available (Why is that?)
- Title not available (Why is that?)
- A general stochastic maximum principle for SDEs of mean-field type
- Viscosity solutions of fully nonlinear second-order equations and optimal stochastic control in infinite dimensions. III: Uniqueness of viscosity solutions for general second-order equations
- Title not available (Why is that?)
- A maximum principle for SDEs of mean-field type
- Mean-field stochastic differential equations and associated PDEs
- The master equation for large population equilibriums
- Forward-backward stochastic differential equations and controlled McKean-Vlasov dynamics
- The master equation in mean field theory
- Mean field games and systemic risk
- Nonlinear SDEs driven by L\'evy processes and related PDEs
- Stochastic Optimal Control in Infinite Dimension
- A comparison principle for Hamilton-Jacobi equations related to controlled gradient flows in infinite dimensions
- Title not available (Why is that?)
- Hamilton-Jacobi equations in the Wasserstein space
- On time-inconsistent stochastic control in continuous time
- Metric viscosity solutions of Hamilton-Jacobi equations depending on local slopes
- On the interpretation of the master equation
- Continuous time mean-variance portfolio optimization through the mean field approach
- Dynamic programming for mean-field type control
- Randomized dynamic programming principle and Feynman-Kac representation for optimal control of McKean-Vlasov dynamics
Cited In (70)
- A mean-field optimal control formulation of deep learning
- Controlled Markov chains with non-exponential discounting and distribution-dependent costs
- Title not available (Why is that?)
- Mean–field moral hazard for optimal energy demand response management
- McKean-Vlasov optimal control: the dynamic programming principle
- Mean-field limit for a class of stochastic ergodic control problems
- Controllability Gramian and Kalman rank condition for mean-field control systems
- Optimality conditions in variational form for non-linear constrained stochastic control problems
- Rate of convergence for particle approximation of PDEs in Wasserstein space
- Linear quadratic stochastic optimal control problems with operator coefficients: open-loop solutions
- A stability property in mean field type differential games
- A probabilistic approach to classical solutions of the master equation for large population equilibria
- Optimal control for stochastic partial differential equations and viscosity solutions of Bellman equations
- A varying terminal time mean-variance model
- Finite Dimensional Approximations of Hamilton–Jacobi–Bellman Equations for Stochastic Particle Systems with Common Noise
- Title not available (Why is that?)
- Viscosity solutions to parabolic master equations and McKean-Vlasov SDEs with closed-loop controls
- Extended mean field control problems: stochastic maximum principle and transport perspective
- Finite dimensional approximations of Hamilton-Jacobi-Bellman equations in spaces of probability measures
- Schauder estimates for a class of potential mean field games of controls
- Density of subalgebras of Lipschitz functions in metric Sobolev spaces and applications to Wasserstein Sobolev spaces
- Closed-Loop Equilibrium for Time-Inconsistent McKean--Vlasov Controlled Problem
- Nonexponential Sanov and Schilder theorems on Wiener space: BSDEs, Schrödinger problems and control
- Mean-field linear-quadratic stochastic differential games in an infinite horizon
- Extended Mckean-Vlasov optimal stochastic control applied to smart grid management,
- Finite state \(N\)-agent and mean field control problems
- Optimal control of path-dependent McKean-Vlasov SDEs in infinite-dimension
- Ergodic control of McKean-Vlasov SDEs and associated Bellman equation
- Title not available (Why is that?)
- Dynamic Programming Equation for the Mean Field Optimal Stopping Problem
- Discrete-time mean-field stochastic control with partial observations
- Mean field verification theorem
- A closed-loop saddle point for zero-sum linear-quadratic stochastic differential games with mean-field type
- Extended mean-field control problem with partial observation
- On Bellman's equations for mean and variance control of a Markov diffusion
- Near optimality of stochastic control for singularly perturbed McKean-Vlasov systems
- Krasovskii-Subbotin approach to mean field type differential games
- Lattice approximations of the first-order mean field type differential games
- Viscosity solutions for controlled McKean-Vlasov jump-diffusions
- Hamilton-Jacobi equations for controlled gradient flows: the comparison principle
- Viscosity Solutions for Obstacle Problems on Wasserstein Space
- A strong comparison result for the bellman equation arising in stochastic exit time control problems and its applications
- Characterization of stochastic equilibrium controls by the Malliavin calculus
- Mean-field linear-quadratic stochastic differential games
- Itô's formula for flows of measures on semimartingales
- McKean–Vlasov Optimal Control: Limit Theory and Equivalence Between Different Formulations
- Stochastic control with delayed information and related nonlinear master equation
- Viability theorem for deterministic mean field type control systems
- Mean-field optimal control as Gamma-limit of finite agent controls
- Title not available (Why is that?)
- Dynamic Programming for Optimal Control of Stochastic McKean--Vlasov Dynamics
- Mean-field type FBSDEs in a domination-monotonicity framework and LQ multi-level Stackelberg games
- An extended McKean-Vlasov dynamic programming approach to robust equilibrium controls under ambiguous covariance matrix
- Controlled measure-valued martingales: a viscosity solution approach
- Exact controllability for mean-field type linear game-based control systems
- Well-posedness for Hamilton-Jacobi equations on the Wasserstein space on graphs
- Importance sampling for the empirical measure of weakly interacting diffusions
- Master Bellman equation in the Wasserstein space: Uniqueness of viscosity solutions
- A fast iterative PDE-based algorithm for feedback controls of nonsmooth mean-field control problems
- A level-set approach to the control of state-constrained McKean-Vlasov equations: application to renewable energy storage and portfolio selection
- Classical solution of path-dependent mean-field semilinear PDEs
- Fully-coupled mean-field FBSDE and maximum principle for related optimal control problem
- Stochastic recursive optimal control of McKean-Vlasov type: a viscosity solution approach
- Control on Hilbert spaces and application to some mean field type control problems
- Viscosity Solutions for McKean–Vlasov Control on a Torus
- Viscosity solutions for mean field optimal switching with a two-time-scale Markov chain
- A Global Optimality Principle for Fully Coupled Mean-field Control Systems
- Kolmogorov equations on the space of probability measures associated to the nonlinear filtering equation: the viscosity approach
- Wellposedness of second order master equations for mean field games with nonsmooth data
- Hypergraphon mean field games
This page was built for publication: Bellman equation and viscosity solutions for mean-field stochastic control problem
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3177924)
