Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games. II: The finite horizon case

DOI10.1214/21-AAP1715zbMATH Open1505.65243arXiv1908.01613MaRDI QIDQ2108885FDOQ2108885

Authors: Mathieu Laurière, René Carmona 

Publication date: 20 December 2022

Published in: The Annals of Applied Probability (Search for Journal in Brave)

Abstract: We propose two numerical methods for the optimal control of McKean-Vlasov dynamics in finite time horizon. Both methods are based on the introduction of a suitable loss function defined over the parameters of a neural network. This allows the use of machine learning tools, and efficient implementations of stochastic gradient descent in order to perform the optimization. In the first method, the loss function stems directly from the optimal control problem. The second method tackles a generic forward-backward stochastic differential equation system (FBSDE) of McKean-Vlasov type, and relies on suitable reformulation as a mean field control problem. To provide a guarantee on how our numerical schemes approximate the solution of the original mean field control problem, we introduce a new optimization problem, directly amenable to numerical computation, and for which we rigorously provide an error rate. Several numerical examples are provided. Both methods can easily be applied to certain problems with common noise, which is not the case with the existing technology. Furthermore, although the first approach is designed for mean field control problems, the second is more general and can also be applied to the FBSDE arising in the theory of mean field games.

Full work available at URL: https://arxiv.org/abs/1908.01613

Recommendations

• Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games. I: The ergodic case
• CEMRACS 2017: numerical probabilistic approach to MFG
• Numerical resolution of McKean-Vlasov FBSDEs using neural networks
• Numerical methods for mean-field type optimal control problems
• A class of finite-dimensional numerically solvable McKean-Vlasov control problems

zbMATH Keywords

machine learningmean field gamesnumerical approximationforward-backward SDEmean field controlMcKean-Vlasov

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Neural networks for/in biological studies, artificial life and related topics (92B20) Stochastic ordinary differential equations (aspects of stochastic analysis) (60H10) Vlasov equations (35Q83) PDEs in connection with control and optimization (35Q93) PDEs with randomness, stochastic partial differential equations (35R60) Numerical solutions to stochastic differential and integral equations (65C30) Finite difference methods for initial value and initial-boundary value problems involving PDEs (65M06) Stability and convergence of numerical methods for initial value and initial-boundary value problems involving PDEs (65M12) Error bounds for initial value and initial-boundary value problems involving PDEs (65M15) Optimal stochastic control (93E20) Mean field games and control (49N80) Mean field games (aspects of game theory) (91A16) PDEs in connection with mean field game theory (35Q89)

Cites Work

• Mean field games
• On the system of partial differential equations arising in mean field type control
• Title not available (Why is that?)
• Mean field games and mean field type control theory
• Mean field games: numerical methods
• A fully discrete semi-Lagrangian scheme for a first order mean field game problem
• A stochastic particle method for the McKean-Vlasov and the Burgers equation
• Long time average of first order mean field games and weak KAM theory
• Contract theory in continuous-time models
• Mean field games: numerical methods for the planning problem
• Degree of approximation by neural and translation networks with a single hidden layer
• Mean field games and systemic risk
• Sensitivity Analysis Using Itô--Malliavin Calculus and Martingales, and Application to Stochastic Optimal Control
• Numerical methods for mean-field type optimal control problems
• The Master Equation and the Convergence Problem in Mean Field Games
• Two numerical approaches to stationary mean-field games
• Mean field type control with congestion. II: An augmented Lagrangian method
• Mean Field Games for Modeling Crowd Motion
• Probabilistic theory of mean field games with applications I. Mean field FBSDEs, control, and games
• Dynamic programming approach to principal-agent problems
• Numerical method for FBSDEs of McKean-Vlasov type
• A probabilistic approach to classical solutions of the master equation for large population equilibria
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Probabilistic theory of mean field games with applications II. Mean field games with common noise and master equations
• Dynamic programming for mean-field type control
• Optimization methods for large-scale machine learning
• Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games. I: The ergodic case
• Proximal methods for stationary mean field games with local couplings
• $L^2$-Multivariate Approximation Theory
• On the implementation of a primal-dual algorithm for second order time-dependent mean field games with local couplings
• An ergodic problem for Mean Field Games: qualitative properties and numerical simulations
• Convergence of the deep BSDE method for coupled FBSDEs
• CEMRACS 2017: numerical probabilistic approach to MFG
• A class of finite-dimensional numerically solvable McKean-Vlasov control problems

Cited In (24)

• Model-free mean-field reinforcement learning: mean-field MDP and mean-field Q-learning
• Importance sampling for the empirical measure of weakly interacting diffusions
• Convergence of the Backward Deep BSDE Method with Applications to Optimal Stopping Problems
• Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games. I: The ergodic case
• A fast iterative PDE-based algorithm for feedback controls of nonsmooth mean-field control problems
• A level-set approach to the control of state-constrained McKean-Vlasov equations: application to renewable energy storage and portfolio selection
• Connecting GANs, mean-field games, and optimal transport
• A Lagrange–Galerkin Scheme for First Order Mean Field Game Systems
• Learning High-Dimensional McKean–Vlasov Forward-Backward Stochastic Differential Equations with General Distribution Dependence
• Recent developments in machine learning methods for stochastic control and games
• Unified reinforcement Q-learning for mean field game and control problems
• Sequential propagation of chaos for mean-field BSDE systems
• Nonstandard stochastic control with nonlinear Feynman-Kac costs
• Numerical resolution of McKean-Vlasov FBSDEs using neural networks
• On numerical approximations of fractional and nonlocal mean field games
• A class of finite-dimensional numerically solvable McKean-Vlasov control problems
• A posteriori estimate for a class of mean-field forward-backward stochastic differential equations
• Learning a functional control for high-frequency finance
• Numerical methods for backward stochastic differential equations: a survey
• Deep neural network solution for finite state mean field game with error estimation
• Approximation of optimal feedback controls for stochastic reaction-diffusion equations
• Relative wealth concerns with partial information and heterogeneous priors
• Computational mean-field games on manifolds
• Performance of a Markovian neural network versus dynamic programming on a fishing control problem

This page was built for publication: Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games. II: The finite horizon case

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2108885)