Stability-constrained Markov decision processes using MPC
From MaRDI portal
Abstract: In this paper, we consider solving discounted Markov Decision Processes (MDPs) under the constraint that the resulting policy is stabilizing. In practice MDPs are solved based on some form of policy approximation. We will leverage recent results proposing to use Model Predictive Control (MPC) as a structured policy in the context of Reinforcement Learning to make it possible to introduce stability requirements directly inside the MPC-based policy. This will restrict the solution of the MDP to stabilizing policies by construction. The stability theory for MPC is most mature for the undiscounted MPC case. Hence, we will first show in this paper that stable discounted MDPs can be reformulated as undiscounted ones. This observation will entail that the MPC-based policy with stability requirements will produce the optimal policy for the discounted MDP if it is stable, and the best stabilizing policy otherwise.
Recommendations
- Learning algorithms for discounted MDPs with constraints
- Learning for MPC with stability \& safety guarantees
- Economic MPC of Markov decision processes: dissipativity in undiscounted infinite-horizon optimal control
- Dynamic programming in constrained Markov decision processes
- Finite linear programming approximations of constrained discounted Markov decision processes
Cites work
- scientific article; zbMATH DE number 1321699 (Why is no real title available?)
- scientific article; zbMATH DE number 700091 (Why is no real title available?)
- A Lyapunov Function for Economic Optimizing Model Predictive Control
- A new dissipativity condition for asymptotic stability of discounted economic MPC
- Data-Driven Economic NMPC Using Reinforcement Learning
- Discrete Dynamic Programming
- Dynamic programming and optimal control. Vol. 1.
- Economic MPC of Markov decision processes: dissipativity in undiscounted infinite-horizon optimal control
- Markov Chains and Stochastic Stability
- Nonlinear model predictive control. Theory and algorithms.
- Probabilistic Model Predictive Safety Certification for Learning-Based Control
- Provably safe and robust learning-based model predictive control
- Reinforcement learning. An introduction
- Robust model predictive control of constrained linear systems with bounded disturbances
- Safe Reinforcement Learning Using Robust MPC
- Stabilization of strictly dissipative discrete time systems with discounted optimal control
- Systems with persistent disturbances: Predictive control with restricted constraints
Cited in
(8)- Economic MPC of Markov decision processes: dissipativity in undiscounted infinite-horizon optimal control
- Learning for MPC with stability \& safety guarantees
- Cost-aware defense for parallel server systems against reliability and security failures
- Markov decision processes methods for static stability of DES
- Stability and singular perturbations in constrained Markov decision problems
- Simplified design of practically stable MPC schemes
- Constrained model predictive control for networked jump systems under denial-of-service attacks and time delays
- Dissipativity in infinite horizon optimal control and dynamic programming
This page was built for publication: Stability-constrained Markov decision processes using MPC
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2158972)
