Reduced-dimensional reinforcement learning control using singular perturbation approximations
From MaRDI portal
Abstract: We present a set of model-free, reduced-dimensional reinforcement learning (RL) based optimal control designs for linear time-invariant singularly perturbed (SP) systems. We first present a state-feedback and output-feedback based RL control design for a generic SP system with unknown state and input matrices. We take advantage of the underlying time-scale separation property of the plant to learn a linear quadratic regulator (LQR) for only its slow dynamics, thereby saving a significant amount of learning time compared to the conventional full-dimensional RL controller. We analyze the sub-optimality of the design using SP approximation theorems and provide sufficient conditions for closed-loop stability. Thereafter, we extend both designs to clustered multi-agent consensus networks, where the SP property reflects through clustering. We develop both centralized and cluster-wise block-decentralized RL controllers for such networks, in reduced dimensions. We demonstrate the details of the implementation of these controllers using simulations of relevant numerical examples and compare them with conventional RL designs to show the computational benefits of our approach.
Recommendations
- Model-based reinforcement learning for approximate optimal regulation
- Time-suboptimal control design of singularly perturbed systems by reduced order feedback design
- Reinforcement learning for adaptive optimal control of continuous-time linear periodic systems
- Reduced Optimal Control of Nonlinear Singularly Perturbed Systems
- A reinforcement learning-based scheme for direct adaptive optimal control of linear stochastic systems
Cites work
- A decomposition of near-optimum regulators for systems with slow and fast modes
- Adaptive optimal control for continuous-time linear systems based on policy iteration
- Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics
- Control strategies for decision makers using different models of the same system
- Nonlinear systems.
- Q-learning for continuous-time linear systems: A model-free infinite horizon optimal control approach
- Robust adaptive dynamic programming
- Singular perturbations and order reduction in control theory - an overview
- Time scale modeling of sparse dynamic networks
Cited in
(9)- Incremental reinforcement learning and optimal output regulation under unmeasurable disturbances
- Model-based and model-free designs for an extended continuous-time LQR with exogenous inputs
- Robust H∞ control for fractional order singular systems 0 < α < 1 with uncertainty
- Suboptimal reduced control of unknown nonlinear singularly perturbed systems via reinforcement learning
- Data-model-hybrid-driven near-optimal operational control of two-time-scale industrial systems with unknown operational model
- Linear quadratic tracking control of unknown systems: a two-phase reinforcement learning method
- Suboptimal control for nonlinear slow-fast coupled systems using reinforcement learning and Takagi-Sugeno fuzzy methods
- A novel actor-critic-identifier architecture for nonlinear multiagent systems with gradient descent method
- Optimal control for continuous-time Markov jump singularly perturbed systems: a hybrid reinforcement learning scheme
This page was built for publication: Reduced-dimensional reinforcement learning control using singular perturbation approximations
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2664206)
