Model-based reinforcement learning for approximate optimal control with temporal logic specifications

DOI10.1145/3447928.3456639arXiv2101.07156WikidataQ130844063 ScholiaQ130844063MaRDI QIDQ6201592FDOQ6201592

Authors: M. Cohen, Calin Belta 

Publication date: 21 February 2024

Published in: Proceedings of the 24th International Conference on Hybrid Systems: Computation and Control (Search for Journal in Brave)

Abstract: In this paper we study the problem of synthesizing optimal control policies for uncertain continuous-time nonlinear systems from syntactically co-safe linear temporal logic (scLTL) formulas. We formulate this problem as a sequence of reach-avoid optimal control sub-problems. We show that the resulting hybrid optimal control policy guarantees the satisfaction of a given scLTL formula by constructing a barrier certificate. Since solving each optimal control problem may be computationally intractable, we take a learning-based approach to approximately solve this sequence of optimal control problems online without requiring full knowledge of the system dynamics. Using Lyapunov-based tools, we develop sufficient conditions under which our approximate solution maintains correctness. Finally, we demonstrate the efficacy of the developed method with a numerical example.

Full work available at URL: https://arxiv.org/abs/2101.07156

zbMATH Keywords

hybrid systemsoptimal controlsafe reinforcement learning

Mathematics Subject Classification ID

Formal languages and automata (68Q45) Specification and verification (program logics, model checking, etc.) (68Q60) Control/observation systems governed by functional relations other than differential equations (such as hybrid and switching systems) (93C30)

Cites Work

• Nonlinear systems.
• Hybrid dynamical systems. Modeling, stability, and robustness
• A time-dependent Hamilton-Jacobi formulation of reachable sets for continuous dynamic games
• Verification and Control of Hybrid Systems
• Title not available (Why is that?)
• Formal Techniques, Modelling and Analysis of Timed and Fault-Tolerant Systems
• Nonlinear control of engineering systems. A Lyapunov-based approach.
• A Fully Automated Framework for Control of Linear Systems from Temporal Logic Specifications
• Controllers for reachability specifications for hybrid systems
• Linear Time Logic Control of Discrete-Time Linear Systems
• Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers
• Efficient model-based reinforcement learning for approximate online optimal control
• Model-based reinforcement learning for approximate optimal regulation
• Title not available (Why is that?)
• Model checking of safety properties
• Barrier function based model predictive control
• Distributed Coordination Control for Multi-Robot Networks Using Lyapunov-Like Barrier Functions
• Control Barrier Function Based Quadratic Programs for Safety Critical Systems
• Reactive synthesis from signal temporal logic specifications
• Formal methods for discrete-time dynamical systems
• Reinforcement learning for optimal feedback control. A Lyapunov-based approach
• LTL receding horizon control for finite deterministic systems
• Data-based reinforcement learning approximate optimal control for an uncertain nonlinear system with control effectiveness faults
• Integral concurrent learning: adaptive control with parameter convergence using finite excitation
• Formal Guarantees in Data-Driven Model Identification and Control Synthesis
• Formal Synthesis of Control Strategies for Positive Monotone Systems
• Sampling-based approximate optimal control under temporal logic constraints
• Satisfaction of Linear Temporal Logic Specifications Through Recurrence Tools for Hybrid Systems

Cited In (1)

• Data-driven abstraction-based control synthesis

This page was built for publication: Model-based reinforcement learning for approximate optimal control with temporal logic specifications

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6201592)