Approximating optimal feedback controllers of finite horizon control problems using hierarchical tensor formats
From MaRDI portal
Publication:5084512
Abstract: Controlling systems of ordinary differential equations (ODEs) is ubiquitous in science and engineering. For finding an optimal feedback controller, the value function and associated fundamental equations such as the Bellman equation and the Hamilton-Jacobi-Bellman (HJB) equation are essential. The numerical treatment of these equations poses formidable challenges due to their non-linearity and their (possibly) high-dimensionality. In this paper we consider a finite horizon control system with associated Bellman equation. After a time-discretization, we obtain a sequence of short time horizon problems which we call local optimal control problems. For solving the local optimal control problems we apply two different methods, one being the well-known policy iteration, where a fixed-point iteration is required for every time step. The other algorithm borrows ideas from Model Predictive Control (MPC), by solving the local optimal control problem via open-loop control methods on a short time horizon, allowing us to replace the fixed-point iteration by an adjoint method. For high-dimensional systems we apply low rank hierarchical tensor product approximation/tree-based tensor formats, in particular tensor trains (TT tensors) and multi-polynomials, together with high-dimensional quadrature, e.g. Monte-Carlo. We prove a linear error propagation with respect to the time discretization and give numerical evidence by controlling a diffusion equation with unstable reaction term and an Allen-Kahn equation.
Recommendations
- Tensor decomposition methods for high-dimensional Hamilton-Jacobi-Bellman equations
- Approximate finite-horizon optimal control without PDEs
- Approximative policy iteration for exit time feedback control problems driven by stochastic differential equations using tensor train format
- Approximation of optimal feedback control: a dynamic programming approach
- Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs
Cites work
- scientific article; zbMATH DE number 5968745 (Why is no real title available?)
- scientific article; zbMATH DE number 3126094 (Why is no real title available?)
- scientific article; zbMATH DE number 3148886 (Why is no real title available?)
- scientific article; zbMATH DE number 3167340 (Why is no real title available?)
- scientific article; zbMATH DE number 7370565 (Why is no real title available?)
- A continuous analogue of the tensor-train decomposition
- A new scheme for the tensor representation
- A numerical approach to the infinite horizon problem of deterministic control theory
- A regression-based Monte Carlo method to solve backward stochastic differential equations
- A splitting algorithm for Hamilton-Jacobi-Bellman equations
- Actor-critic method for high dimensional static Hamilton-Jacobi-Bellman partial differential equations based on neural networks
- Adaptive deep learning for high-dimensional Hamilton-Jacobi-Bellman equations
- Adaptive stochastic Galerkin FEM for lognormal coefficients in hierarchical tensor representations
- Algorithms for PDE-constrained optimization
- An HJB-POD approach for the control of nonlinear PDEs on a tree structure
- An efficient policy iteration algorithm for dynamic programming equations
- Application of variational iteration method for Hamilton-Jacobi-Bellman equations
- Approximative policy iteration for exit time feedback control problems driven by stochastic differential equations using tensor train format
- Breaking the Curse of Dimensionality, Or How to Use SVD in Many Dimensions
- Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design
- Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
- Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations
- Feedback control methodologies for nonlinear systems
- Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations
- Model predictive control: Theory and practice - a survey
- Nonlinear model predictive control. Theory and algorithms
- Numerical tensor calculus
- On manifolds of tensors of fixed TT-rank
- On some recent aspects of stochastic control and their applications
- On the Bellman equation for some unbounded control problems
- On the Convergence of Policy Iteration in Stationary Dynamic Programming
- On the mathematical foundations of learning
- Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations
- Optimal control: novel directions and applications
- Optimality, stability, and convergence in nonlinear control
- Overcoming the curse of dimensionality for some Hamilton-Jacobi partial differential equations via neural network architectures
- Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
- Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs
- Probabilistic max-plus schemes for solving Hamilton-Jacobi-Bellman equations
- Semi-Lagrangian approximation schemes for linear and Hamilton-Jacobi equations
- Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space
- Stable als approximation in the TT-format for rank-adaptive tensor completion
- Support Vector Machines
- TT-cross approximation for multidimensional arrays
- Tensor decomposition methods for high-dimensional Hamilton-Jacobi-Bellman equations
- Tensor networks and hierarchical tensors for the solution of high-dimensional partial differential equations
- Tensor products of Sobolev-Besov spaces and applications to approximation from the hyperbolic cross
- Tensor spaces and hierarchical tensor representations
- Tensor-train decomposition
- The Max-Plus Finite Element Method for Solving Deterministic Optimal Control Problems: Basic Properties and Convergence Analysis
- The alternating linear scheme for tensor optimization in the tensor train format
Cited in
(14)- High order Bellman equations and weakly chained diagonally dominant tensors
- A tree structure approach to reachability analysis
- Sample complexity bounds for the local convergence of least squares approximation
- Approximating the stationary Bellman equation by hierarchical tensor products
- Numerical realization of the Mortensen observer via a Hessian-augmented polynomial approximation of the value function
- Data-Driven Tensor Train Gradient Cross Approximation for Hamilton–Jacobi–Bellman Equations
- A multilevel fast marching method for the minimum time problem
- Approximative policy iteration for exit time feedback control problems driven by stochastic differential equations using tensor train format
- Approximation of optimal control problems for the Navier-Stokes equation via multilinear HJB-POD
- Low-rank tensor methods for partial differential equations
- A multilinear HJB-POD method for the optimal control of PDEs on a tree structure
- Consistent smooth approximation of feedback laws for infinite horizon control problems with non-smooth value functions
- Optimal polynomial feedback laws for finite horizon control problems
- Approximation of optimal feedback controls for stochastic reaction-diffusion equations
This page was built for publication: Approximating optimal feedback controllers of finite horizon control problems using hierarchical tensor formats
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5084512)
