Model-based reinforcement learning for approximate optimal control with temporal logic specifications

DOI10.1145/3447928.3456639MaRDI QIDQ6201592zbMATH OpenWikidataFDO

Authors M. Cohen, Calin Belta

Publication date 21 February 2024

Published in Proceedings of the 24th International Conference on Hybrid Systems: Computation and Control (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2101.07156

zbMATH Keywords

hybrid systemsoptimal controlsafe reinforcement learning

Mathematics Subject Classification ID

Formal languages and automata (68Q45) Specification and verification (program logics, model checking, etc.) (68Q60) Control/observation systems governed by functional relations other than differential equations (such as hybrid and switching systems) (93C30)

Abstract: In this paper we study the problem of synthesizing optimal control policies for uncertain continuous-time nonlinear systems from syntactically co-safe linear temporal logic (scLTL) formulas. We formulate this problem as a sequence of reach-avoid optimal control sub-problems. We show that the resulting hybrid optimal control policy guarantees the satisfaction of a given scLTL formula by constructing a barrier certificate. Since solving each optimal control problem may be computationally intractable, we take a learning-based approach to approximately solve this sequence of optimal control problems online without requiring full knowledge of the system dynamics. Using Lyapunov-based tools, we develop sufficient conditions under which our approximate solution maintains correctness. Finally, we demonstrate the efficacy of the developed method with a numerical example.

Cites work

• scientific article; zbMATH DE number 1982198 (Why is no real title available?)
• scientific article; zbMATH DE number 5585443 (Why is no real title available?)
• A Fully Automated Framework for Control of Linear Systems from Temporal Logic Specifications
• A time-dependent Hamilton-Jacobi formulation of reachable sets for continuous dynamic games
• Barrier function based model predictive control
• Control Barrier Function Based Quadratic Programs for Safety Critical Systems
• Controllers for reachability specifications for hybrid systems
• Data-based reinforcement learning approximate optimal control for an uncertain nonlinear system with control effectiveness faults
• Distributed Coordination Control for Multi-Robot Networks Using Lyapunov-Like Barrier Functions
• Efficient model-based reinforcement learning for approximate online optimal control
• Formal Guarantees in Data-Driven Model Identification and Control Synthesis
• Formal Synthesis of Control Strategies for Positive Monotone Systems
• Formal Techniques, Modelling and Analysis of Timed and Fault-Tolerant Systems
• Formal methods for discrete-time dynamical systems
• Hybrid dynamical systems. Modeling, stability, and robustness
• Integral concurrent learning: adaptive control with parameter convergence using finite excitation
• LTL receding horizon control for finite deterministic systems
• Linear Time Logic Control of Discrete-Time Linear Systems
• Model checking of safety properties
• Model-based reinforcement learning for approximate optimal regulation
• Nonlinear control of engineering systems. A Lyapunov-based approach.
• Nonlinear systems.
• Reactive synthesis from signal temporal logic specifications
• Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers
• Reinforcement learning for optimal feedback control. A Lyapunov-based approach
• Sampling-based approximate optimal control under temporal logic constraints
• Satisfaction of Linear Temporal Logic Specifications Through Recurrence Tools for Hybrid Systems
• Verification and Control of Hybrid Systems

Cited in

(1)

• Data-driven abstraction-based control synthesis

This page was built for publication: Model-based reinforcement learning for approximate optimal control with temporal logic specifications

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6201592)