Approximating optimal feedback controllers of finite horizon control problems using hierarchical tensor formats
From MaRDI portal
Publication:5084512
DOI10.1137/21M1412190zbMATH Open1491.49021arXiv2104.06108OpenAlexW3156706052WikidataQ114074054 ScholiaQ114074054MaRDI QIDQ5084512FDOQ5084512
Authors: Mathias Oster, Leon Sallandt, Reinhold Schneider 
Publication date: 24 June 2022
Published in: SIAM Journal on Scientific Computing (Search for Journal in Brave)
Abstract: Controlling systems of ordinary differential equations (ODEs) is ubiquitous in science and engineering. For finding an optimal feedback controller, the value function and associated fundamental equations such as the Bellman equation and the Hamilton-Jacobi-Bellman (HJB) equation are essential. The numerical treatment of these equations poses formidable challenges due to their non-linearity and their (possibly) high-dimensionality. In this paper we consider a finite horizon control system with associated Bellman equation. After a time-discretization, we obtain a sequence of short time horizon problems which we call local optimal control problems. For solving the local optimal control problems we apply two different methods, one being the well-known policy iteration, where a fixed-point iteration is required for every time step. The other algorithm borrows ideas from Model Predictive Control (MPC), by solving the local optimal control problem via open-loop control methods on a short time horizon, allowing us to replace the fixed-point iteration by an adjoint method. For high-dimensional systems we apply low rank hierarchical tensor product approximation/tree-based tensor formats, in particular tensor trains (TT tensors) and multi-polynomials, together with high-dimensional quadrature, e.g. Monte-Carlo. We prove a linear error propagation with respect to the time discretization and give numerical evidence by controlling a diffusion equation with unstable reaction term and an Allen-Kahn equation.
Full work available at URL: https://arxiv.org/abs/2104.06108
Recommendations
- Tensor decomposition methods for high-dimensional Hamilton-Jacobi-Bellman equations
- Approximate finite-horizon optimal control without PDEs
- Approximative policy iteration for exit time feedback control problems driven by stochastic differential equations using tensor train format
- Approximation of optimal feedback control: a dynamic programming approach
- Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs
feedback controlmodel predictive controlPontryagin maximum principleBellman equationpolicy iterationtensor train
Multilinear algebra, tensor calculus (15A69) Optimality conditions for problems involving ordinary differential equations (49K15) Optimal feedback synthesis (49N35) PDE constrained optimization (numerical aspects) (49M41)
Cites Work
- Model predictive control: Theory and practice - a survey
- Title not available (Why is that?)
- Title not available (Why is that?)
- Support Vector Machines
- Tensor products of Sobolev-Besov spaces and applications to approximation from the hyperbolic cross
- On the mathematical foundations of learning
- Tensor-train decomposition
- TT-cross approximation for multidimensional arrays
- Breaking the Curse of Dimensionality, Or How to Use SVD in Many Dimensions
- On the Bellman equation for some unbounded control problems
- A regression-based Monte Carlo method to solve backward stochastic differential equations
- Title not available (Why is that?)
- Semi-Lagrangian Approximation Schemes for Linear and Hamilton—Jacobi Equations
- Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations
- A numerical approach to the infinite horizon problem of deterministic control theory
- A splitting algorithm for Hamilton-Jacobi-Bellman equations
- Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations
- Numerical tensor calculus
- On some recent aspects of stochastic control and their applications
- Title not available (Why is that?)
- Feedback control methodologies for nonlinear systems
- An Efficient Policy Iteration Algorithm for Dynamic Programming Equations
- A new scheme for the tensor representation
- Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design
- The alternating linear scheme for tensor optimization in the tensor train format
- Algorithms for PDE-constrained optimization
- Application of variational iteration method for Hamilton-Jacobi-Bellman equations
- The Max-Plus Finite Element Method for Solving Deterministic Optimal Control Problems: Basic Properties and Convergence Analysis
- On the Convergence of Policy Iteration in Stationary Dynamic Programming
- On manifolds of tensors of fixed TT-rank
- Optimality, stability, and convergence in nonlinear control
- Tensor networks and hierarchical tensors for the solution of high-dimensional partial differential equations
- Tensor Spaces and Hierarchical Tensor Representations
- A continuous analogue of the tensor-train decomposition
- Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
- Nonlinear model predictive control. Theory and algorithms
- Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations
- Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
- Stable als approximation in the TT-format for rank-adaptive tensor completion
- Adaptive stochastic Galerkin FEM for lognormal coefficients in hierarchical tensor representations
- Polynomial Approximation of High-Dimensional Hamilton--Jacobi--Bellman Equations and Applications to Feedback Control of Semilinear Parabolic PDEs
- Overcoming the curse of dimensionality for some Hamilton-Jacobi partial differential equations via neural network architectures
- Adaptive Deep Learning for High-Dimensional Hamilton--Jacobi--Bellman Equations
- Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networks
- Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space
- Probabilistic max-plus schemes for solving Hamilton-Jacobi-Bellman equations
- An HJB-POD approach for the control of nonlinear PDEs on a tree structure
- Tensor Decomposition Methods for High-dimensional Hamilton--Jacobi--Bellman Equations
- Optimal control: novel directions and applications
- Title not available (Why is that?)
- Approximative Policy Iteration for Exit Time Feedback Control Problems Driven by Stochastic Differential Equations using Tensor Train Format
Cited In (11)
- Low-rank tensor methods for partial differential equations
- Optimal polynomial feedback laws for finite horizon control problems
- Data-Driven Tensor Train Gradient Cross Approximation for Hamilton–Jacobi–Bellman Equations
- A tree structure approach to reachability analysis
- A multilinear HJB-POD method for the optimal control of PDEs on a tree structure
- Consistent smooth approximation of feedback laws for infinite horizon control problems with non-smooth value functions
- A multilevel fast marching method for the minimum time problem
- Sample complexity bounds for the local convergence of least squares approximation
- Numerical realization of the Mortensen observer via a Hessian-augmented polynomial approximation of the value function
- Approximation of optimal feedback controls for stochastic reaction-diffusion equations
- Approximation of optimal control problems for the Navier-Stokes equation via multilinear HJB-POD
Uses Software
This page was built for publication: Approximating optimal feedback controllers of finite horizon control problems using hierarchical tensor formats
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5084512)
