Adaptive optimal control for continuous-time linear systems based on policy iteration

DOI10.1016/j.automatica.2008.08.017zbMath1158.93354OpenAlexW2024303516MaRDI QIDQ1012874

Frank L. Lewis, Octavian Pastravanu, Draguna Vrabie, Murad Abu-Khalaf

Publication date: 23 April 2009

Published in: Automatica (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1016/j.automatica.2008.08.017

zbMATH Keywords

adaptive controlLQRpolicy iterationsadaptive critics

Mathematics Subject Classification ID

Adaptive control/observation systems (93C40) Linear systems in control theory (93C05)

Related Items (94)

Homotopic policy iteration-based learning design for unknown linear continuous-time systems ⋮ Online identifier–actor–critic algorithm for optimal control of nonlinear systems ⋮ Optimal tracking control based on reinforcement learning value iteration algorithm for time-delayed nonlinear systems with external disturbances and input constraints ⋮ A review of stochastic algorithms with continuous value function approximation and some new approximate policy iteration algorithms for multidimensional continuous applications ⋮ Adaptive dynamic programming for online solution of a zero-sum differential game ⋮ Data-driven optimal switching and control of switched systems ⋮ Heuristic dynamic programming-based learning control for discrete-time disturbed multi-agent systems ⋮ Output-feedback adaptive optimal control of interconnected systems based on robust adaptive dynamic programming ⋮ A new iterative algorithm for solvingH_∞control problem of continuous-time Markovian jumping linear systems based on online implementation ⋮ Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design ⋮ Approximate guaranteed cost fault-tolerant control of unknown nonlinear systems with time-varying actuator faults ⋮ Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality ⋮ Discrete-time dynamic graphical games: model-free reinforcement learning solution ⋮ Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics ⋮ Adaptive dynamic programming for discrete-time linear quadratic regulation based on multirate generalised policy iteration ⋮ Reinforcement learning solution for HJB equation arising in constrained optimal control problem ⋮ Online adaptive algorithm for optimal control with integral reinforcement learning ⋮ Reduced-dimensional reinforcement learning control using singular perturbation approximations ⋮ Data-driven model reduction by moment matching for linear and nonlinear systems ⋮ Approximately adaptive neural cooperative control for nonlinear multiagent systems with performance guarantee ⋮ Robust adaptive dynamic programming for linear and nonlinear systems: an overview ⋮ Reinforcement learning‐based adaptive optimal tracking algorithm for Markov jump systems with partial unknown dynamics ⋮ Off‐policy integral reinforcement learning‐based optimal tracking control for a class of nonzero‐sum game systems with unknown dynamics ⋮ A New Integral Critic Learning for Optimal Tracking Control with Applications to Boiler‐Turbine Systems ⋮ Dynamic event-triggered controller design for nonlinear systems: reinforcement learning strategy ⋮ Heterogeneous multi-player imitation learning ⋮ Framework for learning and control in the classical and quantum domains ⋮ Online inverse reinforcement learning for nonlinear systems with adversarial attacks ⋮ A new self-learning optimal control laws for a class of discrete-time nonlinear systems based on ESN architecture ⋮ Partial model‐free sliding mode control design for a class of disturbed systems via computational learning algorithm ⋮ Learning‐based T‐sHDP() for optimal control of a class of nonlinear discrete‐time systems ⋮ Optimal robust formation control for heterogeneous multi‐agent systems based on reinforcement learning ⋮ Adaptive dynamic optimal control for triggered networked switched systems under dual‐ended denial‐of‐service attacks ⋮ Optimal tracking control of mechatronic servo system using integral reinforcement learning ⋮ Integral reinforcement learning‐based approximate minimum time‐energy path planning in an unknown environment ⋮ Adaptive optimization algorithm for nonlinear Markov jump systems with partial unknown dynamics ⋮ Output‐feedback Q‐learning for discrete‐time linear H^∞ tracking control: A Stackelberg game approach ⋮ Inverse reinforcement learning for identification of linear-quadratic zero-sum differential games ⋮ An adaptive dynamic programming method for observer‐based sliding mode control of connected vehicles subject to deception attacks ⋮ Event-triggered-based online integral reinforcement learning for optimal control of unknown constrained nonlinear systems ⋮ Off-policy reinforcement learning for tracking control of discrete-time Markov jump linear systems with completely unknown dynamics ⋮ Safe adaptive learning algorithm with neural network implementation for H∞ control of nonlinear safety‐critical system ⋮ An adaptive dynamic programming-based algorithm for infinite-horizon linear quadratic stochastic optimal control problems ⋮ Adaptive neural optimal control via command filter for nonlinear multi‐agent systems including time‐varying output constraints ⋮ Observer‐based event triggered ADP approach for input‐constrained nonlinear systems with disturbances ⋮ Reinforcement learning algorithms with function approximation: recent advances and applications ⋮ Improved off‐policy reinforcement learning algorithm for robust control of unmodeled nonlinear system with asymmetric state constraints ⋮ Trajectory tracking control for underactuated autonomous vehicles via adaptive dynamic programming ⋮ Adaptive output feedback reinforcement learning control for continuous time switched stochastic nonlinear systems with unknown control coefficients and full-state constraints ⋮ Solution of the linear quadratic regulator problem of black box linear systems using reinforcement learning ⋮ Multi-player non-zero-sum games: online adaptive learning solution of coupled Hamilton-Jacobi equations ⋮ New insight into the simultaneous policy update algorithms related to \(H_\infty\) state feedback control ⋮ Online policy iterative-based \(H_\infty\) optimization algorithm for a class of nonlinear systems ⋮ Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning ⋮ Cooperative guidance for simultaneous attack: a fully distributed, adaptive, and optimal approach ⋮ Optimal fault-tolerant control against descriptor time-varying systems with nonlinear input ⋮ Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems ⋮ Reinforcement \(Q\)-learning for optimal tracking control of linear discrete-time systems with unknown dynamics ⋮ Organization learning oriented approach with application to discrete flight control ⋮ Adaptive dynamic programming and optimal control of nonlinear nonaffine systems ⋮ Optimal and robust control of a class of nonlinear systems using dynamically re-optimised single network adaptive critic design ⋮ Adaptive fault-tolerant output regulation of linear systems with unknown dynamics and actuator faults ⋮ Model-based and model-free designs for an extended continuous-time LQR with exogenous inputs ⋮ Integral reinforcement learning-based guaranteed cost control for unknown nonlinear systems subject to input constraints and uncertainties ⋮ Q-learning for continuous-time linear systems: A model-free infinite horizon optimal control approach ⋮ Online reinforcement learning multiplayer non-zero sum games of continuous-time Markov jump linear systems ⋮ Integral \(Q\)-learning and explorized policy iteration for adaptive optimal control of continuous-time linear systems ⋮ Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem ⋮ Finite-Horizonε-Optimal Tracking Control of Discrete-Time Linear Systems Using Iterative Approximate Dynamic Programming ⋮ Online adaptive optimal control for continuous-time nonlinear systems with completely unknown dynamics ⋮ Convergence of the standard RLS method andUDU^Tfactorisation of covariance matrix for solving the algebraic Riccati equation of the DLQR via heuristic approximate dynamic programming ⋮ Some recent advances in learning and adaptation for uncertain feedback control systems ⋮ Adaptive neural network-based optimal control of nonlinear continuous-time systems in strict-feedback form ⋮ Adaptive dynamic programming as a theory of sensorimotor control ⋮ Online adaptive policy iteration based fault-tolerant control algorithm for continuous-time nonlinear tracking systems with actuator failures ⋮ Optimized control for human-multi-robot collaborative manipulation via multi-player Q-learning ⋮ Multi-step heuristic dynamic programming for optimal control of nonlinear discrete-time systems ⋮ Output regulation of heterogeneous linear multi-agent systems with differential graphical game ⋮ Adaptive control of diffusion processes with a discounted reward criterion ⋮ Output‐feedback H_∞ quadratic tracking control of linear systems using reinforcement learning ⋮ Intelligent Human–Robot Interaction Systems Using Reinforcement Learning and Neural Networks ⋮ Model-FreeH_∞Control Design for Unknown Continuous-Time Linear System Using Adaptive Dynamic Programming ⋮ Bias-policy iteration based adaptive dynamic programming for unknown continuous-time linear systems ⋮ Design and Comparison Base Analysis of Adaptive Estimator for Completely Unknown Linear Systems in the Presence of OE Noise and Constant Input Time Delay ⋮ Data-driven adaptive optimal control of linear uncertain systems with unknown jumping dynamics ⋮ Online optimal tracking control of continuous-time linear systems with unknown dynamics by using adaptive dynamic programming ⋮ Online optimal and adaptive integral tracking control for varying discrete‐time systems using reinforcement learning ⋮ Decentralised adaptive synchronisation of a class of discrete-time and nonlinearly parametrised coupled multi-agent systems ⋮ Reinforcement learning for distributed control and multi-player games ⋮ Optimal adaptive control of partially uncertain linear continuous-time systems with state delay ⋮ Computational intelligence in uncertainty quantification for learning control and differential games ⋮ Multi-agent discrete-time graphical games and reinforcement learning solutions ⋮ Adaptive dynamic programming-based decentralised control for large-scale nonlinear systems subject to mismatched interconnections with unknown time-delay ⋮ Multi-objective optimal control for a class of nonlinear time-delay systems via adaptive dynamic programming

Cites Work

• Unnamed Item
• Unnamed Item
• Model-free \(Q\)-learning designs for linear discrete-time zero-sum games with application to \(H^\infty\) control
• Solving the algebraic Riccati equation with the matrix sign function
• Optimal design of adaptive tracking controllers for nonlinear systems
• Stabilization of nonlinear uncertain systems
• Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach
• Adaptive Control Tutorial
• A Schur method for solving algebraic Riccati equations
• Accelerated convergence of the matrix sign function method of solving Lyapunov, Riccati and other matrix equations
• A Numerical Algorithm for Optimal Feedback Gains in High Dimensional Linear Quadratic Regulator Problems
• Some new algorithms for recursive estimation in constant linear systems
• Kronecker products and matrix calculus in system theory
• Analysis and modificaton of Newton’s method for algebraic Riccati equations
• Policy Iterations on the Hamilton–Jacobi–Isaacs Equation for $H_{\infty}$ State Feedback Control With Input Saturation
• Matrix Quadratic Solutions
• Robust nonlinear control design. State-space and Lyapunov techniques

This page was built for publication: Adaptive optimal control for continuous-time linear systems based on policy iteration