State constrained stochastic optimal control for continuous and hybrid dynamical systems using DFBSDE

DOI10.1016/J.AUTOMATICA.2023.111146MaRDI QIDQ6165319zbMATH OpenFDO

Authors Bolun Dai, Prashanth Krishnamurthy, Andrew Papanicolaou, Farshad Khorrami

Publication date 31 July 2023

Published in Automatica (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2305.06499

zbMATH Keywords

forward and backward stochastic differential equationsoptimal controlstochastic control

Mathematics Subject Classification ID

Hamilton-Jacobi equations in optimal control and differential games (49L12) Applications of stochastic analysis (to PDEs, etc.) (60H30) Nonlinear systems in control theory (93C10) Control/observation systems governed by functional relations other than differential equations (such as hybrid and switching systems) (93C30) Optimal stochastic control (93E20)

Abstract: We develop a computationally efficient learning-based forward-backward stochastic differential equations (FBSDE) controller for both continuous and hybrid dynamical (HD) systems subject to stochastic noise and state constraints. Solutions to stochastic optimal control (SOC) problems satisfy the Hamilton-Jacobi-Bellman (HJB) equation. Using current FBSDE-based solutions, the optimal control can be obtained from the HJB equations using deep neural networks (e.g., long short-term memory (LSTM) networks). To ensure the learned controller respects the constraint boundaries, we enforce the state constraints using a soft penalty function. In addition to previous works, we adapt the deep FBSDE (DFBSDE) control framework to handle HD systems consisting of continuous dynamics and a deterministic discrete state change. We demonstrate our proposed algorithm in simulation on a continuous nonlinear system (cart-pole) and a hybrid nonlinear system (five-link biped).

Recommendations

• Stochastic optimal control of state constrained systems
• Convergence of a Robust Deep FBSDE Method for Stochastic Control
• Stochastic optimal control via forward and backward stochastic differential equations and importance sampling
• Highly accurate numerical schemes for stochastic optimal control via FBSDEs
• Stochastic \(L^1\)-optimal control via forward and backward sampling

Cites work

• An introduction to trajectory optimization: how to do your own direct collocation
• Efficient numerical methods for nonlinear MPC and moving horizon estimation
• scientific article; zbMATH DE number 1965513 (Why is no real title available?)
• Models, feedback control, and open problems of 3D bipedal robotic walking
• Navigating over 3D environments while minimizing cumulative exposure to hazardous fields
• Optimal control for stochastic linear quadratic singular system using neural networks
• Optimal control for stochastic nonlinear singular system using neural networks
• Stochastic MPC with offline uncertainty sampling
• Stochastic optimal control via Bellman's principle.

Cited in

(2)

• Event-triggered non-fragile guaranteed cost output feedback control for switched systems under DoS attacks
• Exact controllability of discrete-time stochastic system with multiplicative noise and control constraint

This page was built for publication: State constrained stochastic optimal control for continuous and hybrid dynamical systems using DFBSDE

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6165319)