Approximating optimal feedback controllers of finite horizon control problems using hierarchical tensor formats
From MaRDI portal
Publication:5084512
Abstract: Controlling systems of ordinary differential equations (ODEs) is ubiquitous in science and engineering. For finding an optimal feedback controller, the value function and associated fundamental equations such as the Bellman equation and the Hamilton-Jacobi-Bellman (HJB) equation are essential. The numerical treatment of these equations poses formidable challenges due to their non-linearity and their (possibly) high-dimensionality. In this paper we consider a finite horizon control system with associated Bellman equation. After a time-discretization, we obtain a sequence of short time horizon problems which we call local optimal control problems. For solving the local optimal control problems we apply two different methods, one being the well-known policy iteration, where a fixed-point iteration is required for every time step. The other algorithm borrows ideas from Model Predictive Control (MPC), by solving the local optimal control problem via open-loop control methods on a short time horizon, allowing us to replace the fixed-point iteration by an adjoint method. For high-dimensional systems we apply low rank hierarchical tensor product approximation/tree-based tensor formats, in particular tensor trains (TT tensors) and multi-polynomials, together with high-dimensional quadrature, e.g. Monte-Carlo. We prove a linear error propagation with respect to the time discretization and give numerical evidence by controlling a diffusion equation with unstable reaction term and an Allen-Kahn equation.
Recommendations
- Tensor decomposition methods for high-dimensional Hamilton-Jacobi-Bellman equations
- Approximate finite-horizon optimal control without PDEs
- Approximative policy iteration for exit time feedback control problems driven by stochastic differential equations using tensor train format
- Approximation of optimal feedback control: a dynamic programming approach
- Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs
Cites work
- A continuous analogue of the tensor-train decomposition
- A new scheme for the tensor representation
- A numerical approach to the infinite horizon problem of deterministic control theory
- A regression-based Monte Carlo method to solve backward stochastic differential equations
- A splitting algorithm for Hamilton-Jacobi-Bellman equations
- Actor-critic method for high dimensional static Hamilton-Jacobi-Bellman partial differential equations based on neural networks
- Adaptive deep learning for high-dimensional Hamilton-Jacobi-Bellman equations
- Adaptive stochastic Galerkin FEM for lognormal coefficients in hierarchical tensor representations
- Algorithms for PDE-constrained optimization
- An efficient policy iteration algorithm for dynamic programming equations
- An HJB-POD approach for the control of nonlinear PDEs on a tree structure
- Application of variational iteration method for Hamilton-Jacobi-Bellman equations
- Approximative policy iteration for exit time feedback control problems driven by stochastic differential equations using tensor train format
- Breaking the Curse of Dimensionality, Or How to Use SVD in Many Dimensions
- Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design
- Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
- Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations
- Feedback control methodologies for nonlinear systems
- scientific article; zbMATH DE number 5968745 (Why is no real title available?)
- scientific article; zbMATH DE number 3126094 (Why is no real title available?)
- scientific article; zbMATH DE number 3148886 (Why is no real title available?)
- scientific article; zbMATH DE number 3167340 (Why is no real title available?)
- Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations
- Model predictive control: Theory and practice - a survey
- Nonlinear model predictive control. Theory and algorithms
- Numerical tensor calculus
- On manifolds of tensors of fixed TT-rank
- On some recent aspects of stochastic control and their applications
- On the Bellman equation for some unbounded control problems
- On the Convergence of Policy Iteration in Stationary Dynamic Programming
- On the mathematical foundations of learning
- Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations
- Optimal control: novel directions and applications
- Optimal feedback law recovery by gradient-augmented sparse polynomial regression
- Optimality, stability, and convergence in nonlinear control
- Overcoming the curse of dimensionality for some Hamilton-Jacobi partial differential equations via neural network architectures
- Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
- Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs
- Probabilistic max-plus schemes for solving Hamilton-Jacobi-Bellman equations
- Semi-Lagrangian approximation schemes for linear and Hamilton-Jacobi equations
- Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space
- Stable als approximation in the TT-format for rank-adaptive tensor completion
- Support Vector Machines
- Tensor decomposition methods for high-dimensional Hamilton-Jacobi-Bellman equations
- Tensor networks and hierarchical tensors for the solution of high-dimensional partial differential equations
- Tensor products of Sobolev-Besov spaces and applications to approximation from the hyperbolic cross
- Tensor spaces and hierarchical tensor representations
- Tensor-train decomposition
- The alternating linear scheme for tensor optimization in the tensor train format
- The Max-Plus Finite Element Method for Solving Deterministic Optimal Control Problems: Basic Properties and Convergence Analysis
- TT-cross approximation for multidimensional arrays
Cited in
(21)- Approximation of optimal control problems for the Navier-Stokes equation via multilinear HJB-POD
- High order Bellman equations and weakly chained diagonally dominant tensors
- Approximative policy iteration for exit time feedback control problems driven by stochastic differential equations using tensor train format
- Low-rank tensor methods for partial differential equations
- Data-Driven Tensor Train Gradient Cross Approximation for Hamilton–Jacobi–Bellman Equations
- Optimal polynomial feedback laws for finite horizon control problems
- A tree structure approach to reachability analysis
- Approximating the stationary Bellman equation by hierarchical tensor products
- A multilinear HJB-POD method for the optimal control of PDEs on a tree structure
- Consistent smooth approximation of feedback laws for infinite horizon control problems with non-smooth value functions
- A multilevel fast marching method for the minimum time problem
- Sample complexity bounds for the local convergence of least squares approximation
- Numerical realization of the Mortensen observer via a Hessian-augmented polynomial approximation of the value function
- Approximation of optimal feedback controls for stochastic reaction-diffusion equations
- Reverse diffusion sampling with tensor train approximations of Hamilton-Jacobi-Bellman equations
- Discrete time dynamic programming using tensor trains
- Control of high-dimensional collective dynamics by deep neural feedback laws and kinetic modelling
- A comparison study of supervised learning techniques for the approximation of high dimensional functions and feedback control
- Some thoughts on compositional tensor networks
- A stochastic algorithm for deterministic multistage optimization problems
- The state-dependent Riccati equation in nonlinear optimal control: analysis and numerical approximation
This page was built for publication: Approximating optimal feedback controllers of finite horizon control problems using hierarchical tensor formats
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5084512)
