Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem

DOI10.1016/j.automatica.2010.02.018zbMath1191.49038OpenAlexW1983523797MaRDI QIDQ980921

Frank L. Lewis, Kyriakos G. Vamvoudakis

Publication date: 8 July 2010

Published in: Automatica (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1016/j.automatica.2010.02.018

zbMATH Keywords

adaptive controlLQRpersistence of excitationadaptive criticssynchronous policy iteration

Mathematics Subject Classification ID

Related Items (only showing first 100 items - show all)

Robust differential game guidance laws design for uncertain interceptor-target engagement via adaptive dynamic programming ⋮ A new iterative algorithm for solvingH_∞control problem of continuous-time Markovian jumping linear systems based on online implementation ⋮ A new approximate dynamic programming algorithm based on an actor–critic framework for optimal control of alkali–surfactant–polymer flooding ⋮ Adaptive critic designs for decentralised robust control of nonlinear interconnected systems via event-triggering mechanism ⋮ A data-based neural policy learning strategy towards robust tracking control design for uncertain dynamic systems ⋮ Simultaneous identification and optimal tracking control of unknown continuous-time systems with actuator constraints ⋮ Approximately adaptive neural cooperative control for nonlinear multiagent systems with performance guarantee ⋮ Learning feedback Nash strategies for nonlinear port-Hamiltonian systems ⋮ Adaptive critic optimization to decentralized event‐triggered control of continuous‐time nonlinear interconnected systems ⋮ Adaptive dynamic programming for decentralized neuro‐control of nonlinear systems subject to mismatched interconnections ⋮ Self‐learning‐based optimal tracking control of an unmanned surface vehicle with pose and velocity constraints ⋮ Learning‐based super‐twisting sliding‐mode control for space circumnavigation mission with suboptimal reaching under input constraints ⋮ Modified general policy iteration based adaptive dynamic programming for unknown discrete‐time linear systems ⋮ Suboptimal reduced control of unknown nonlinear singularly perturbed systems via reinforcement learning ⋮ Event-triggered adaptive dynamic programming for decentralized tracking control of input constrained unknown nonlinear interconnected systems ⋮ Optimal event‐triggered control for C‐T system with asymmetric constraints based on dual heuristic dynamic programing structure ⋮ Critic‐only online adaptive learning based decentralized control schemes for nonlinear large‐scale systems ⋮ Learning‐based T‐sHDP() for optimal control of a class of nonlinear discrete‐time systems ⋮ Model‐free incremental adaptive dynamic programming based approximate robust optimal regulation ⋮ Optimal tracking control of mechatronic servo system using integral reinforcement learning ⋮ Robust tracking control of quadrotor via on‐policy adaptive dynamic programming ⋮ Hamiltonian‐driven adaptive dynamic programming for mixed H₂/H_∞ performance using sum‐of‐squares ⋮ Output‐feedback Q‐learning for discrete‐time linear H^∞ tracking control: A Stackelberg game approach ⋮ Distributed optimal event‐triggered cooperative control for nonlinear multi‐missile guidance systems with partially unknown dynamics ⋮ Nonlinear control using human behavior learning ⋮ Event‐triggered optimal tracking control of multiplayer unknown nonlinear systems via adaptive critic designs ⋮ Reinforcement learning-based optimised control for a class of second-order nonlinear dynamic systems ⋮ A novel actor-critic-identifier architecture for nonlinear multiagent systems with gradient descent method ⋮ Optimal control of unknown nonlinear system under event‐triggered mechanism and identifier‐critic‐actor architecture ⋮ Data‐assisted control: A framework development by exploiting NASA Generic Transport platform ⋮ Safe adaptive learning algorithm with neural network implementation for H∞ control of nonlinear safety‐critical system ⋮ Finite‐horizon optimal trajectory control of near space hypersonic vehicle with multi‐constraints ⋮ Command filter and dynamic surface control technology based adaptive optimal control of uncertain nonlinear systems in strict‐feedback form ⋮ Adaptive neural optimal control via command filter for nonlinear multi‐agent systems including time‐varying output constraints ⋮ Reinforcement learning‐based robust optimal output regulation for constrained nonlinear systems with static and dynamic uncertainties ⋮ Adaptive optimal backstepping control for strict-feedback nonlinear systems with time-varying partial output constraints ⋮ Observer‐based event triggered ADP approach for input‐constrained nonlinear systems with disturbances ⋮ Finite‐time sub‐optimal control design for control affine nonlinear systems ⋮ A brief survey on nonlinear control using adaptive dynamic programming under engineering-oriented complexities ⋮ Adaptive dynamic surface-based differential games for a class of pure-feedback nonlinear systems with output constraints ⋮ Output‐feedback robust control of systems with uncertain dynamics via data‐driven policy learning ⋮ Robust tracking control with reinforcement learning for nonlinear‐constrained systems ⋮ Optimized adaptive event‐triggered tracking control for multi‐agent systems with full‐state constraints ⋮ Performance‐guaranteed containment control for pure‐feedback multi agent systems via reinforcement learning algorithm ⋮ Adaptive dynamic programming‐based adaptive‐gain sliding mode tracking control for fixed‐wing unmanned aerial vehicle with disturbances ⋮ Improved off‐policy reinforcement learning algorithm for robust control of unmodeled nonlinear system with asymmetric state constraints ⋮ Reinforcement learning‐based optimal trajectory tracking control of surface vessels under input saturations ⋮ Adaptive optimal formation control for unmanned surface vehicles with guaranteed performance using actor‐critic learning architecture ⋮ Trajectory tracking control for underactuated autonomous vehicles via adaptive dynamic programming ⋮ Optimized tracking control based on reinforcement learning for a class of high-order unknown nonlinear dynamic systems ⋮ Distributed fault‐tolerant control for over‐actuated multi‐agent systems with uncertain perturbations using control allocation ⋮ Event‐triggered neural experience replay learning for nonzero‐sum tracking games of unknown continuous‐time nonlinear systems ⋮ Reinforcement learning‐based tracking control for a quadrotor unmanned aerial vehicle under external disturbances ⋮ Connectivity‐preserving consensus: An adaptive event‐triggered strategy ⋮ Integral sliding mode‐based event‐triggered nearly optimal tracking control for uncertain nonlinear systems ⋮ Solution of the linear quadratic regulator problem of black box linear systems using reinforcement learning ⋮ Reinforcement learning based optimal synchronization control for multi-agent systems with input constraints using vanishing viscosity method ⋮ Efficient Multi-objective Reinforcement Learning via Multiple-gradient Descent with Iteratively Discovered Weight-Vector Sets ⋮ Unnamed Item ⋮ Modified value-function-approximation for synchronous policy iteration with single-critic configuration for nonlinear optimal control ⋮ Online solution of nonlinear two‐player zero‐sum games using synchronous policy iteration ⋮ Online H∞ control for completely unknown nonlinear systems via an identifier–critic-based ADP structure ⋮ Online concurrent reinforcement learning algorithm to solve two‐player zero‐sum games for partially unknown nonlinear continuous‐time systems ⋮ A set‐based model‐free reinforcement learning design technique for nonlinear systems ⋮ Online optimal tracking control of continuous-time linear systems with unknown dynamics by using adaptive dynamic programming ⋮ Decentralised zero-sum differential game for a class of large-scale interconnected systems via adaptive dynamic programming ⋮ Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints ⋮ Finite-horizon optimal control for continuous-time uncertain nonlinear systems using reinforcement learning ⋮ Continuous-time reinforcement learning for robust control under worst-case uncertainty ⋮ Backstepping-based adaptive dynamic programming for missile-target guidance systems with state and input constraints ⋮ On integral generalized policy iteration for continuous-time linear quadratic regulations ⋮ Heuristic dynamic programming-based learning control for discrete-time disturbed multi-agent systems ⋮ Neural network-based adaptive decentralized learning control for interconnected systems with input constraints ⋮ Dynamic event-triggered distributed guaranteed cost FTC scheme for nonlinear interconnected systems via ADP approach ⋮ Nonzero-sum differential games of continuous-time nonlinear systems with uniformly ultimately \(\varepsilon\)-bounded by adaptive dynamic programming ⋮ Approximate guaranteed cost fault-tolerant control of unknown nonlinear systems with time-varying actuator faults ⋮ Closed-form solution to finite-horizon suboptimal control of nonlinear systems ⋮ Efficient model-based reinforcement learning for approximate online optimal control ⋮ Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality ⋮ Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming ⋮ Discrete-time dynamic graphical games: model-free reinforcement learning solution ⋮ Disturbance observer-based robust missile autopilot design with full-state constraints via adaptive dynamic programming ⋮ Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics ⋮ A neural-network-based online optimal control approach for nonlinear robust decentralized stabilization ⋮ Reinforcement learning solution for HJB equation arising in constrained optimal control problem ⋮ Simplified optimized control using reinforcement learning algorithm for a class of stochastic nonlinear systems ⋮ Online adaptive algorithm for optimal control with integral reinforcement learning ⋮ Event-triggered distributed self-learning robust tracking control for uncertain nonlinear interconnected systems ⋮ Data‐Driven Adaptive Critic Approach for Nonlinear Optimal Control via Least Squares Support Vector Machine ⋮ A novel adaptive dynamic programming based on tracking error for nonlinear discrete-time systems ⋮ Optimal control of port-Hamiltonian systems: a continuous-time learning approach ⋮ Novel iterative neural dynamic programming for data-based approximate optimal control design ⋮ Self-triggering adaptive optimal control for nonlinear systems based on encoding mechanism ⋮ Suboptimal control for nonlinear systems with disturbance via integral sliding mode control and policy iteration ⋮ Neural network robust tracking control with adaptive critic framework for uncertain nonlinear systems ⋮ Self-learning robust optimal control for continuous-time nonlinear systems with mismatched disturbances ⋮ Neural robust stabilization via event-triggering mechanism and adaptive learning technique ⋮ Adaptive critic designs for optimal control of uncertain nonlinear systems with unmatched interconnections ⋮ Online barrier-actor-critic learning for \(H_\infty\) control with full-state constraints and input saturation ⋮ Distributed zero-sum differential game for multi-agent systems in strict-feedback form with input saturation and output constraint

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Adaptive optimal control for continuous-time linear systems based on policy iteration
• Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation
• Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach
• Adaptive Control Tutorial
• L/sub 2/-gain analysis of nonlinear systems and nonlinear state-feedback H/sub infinity / control
• Adaptive Control Design and Analysis
• Notes on uniform approximation of time-varying systems on finite time intervals

This page was built for publication: Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem