Model-free \(Q\)-learning designs for linear discrete-time zero-sum games with application to \(H^\infty\) control

DOI10.1016/J.AUTOMATICA.2006.09.019zbMath1137.93321OpenAlexW2005437559MaRDI QIDQ875484

Murad Abu-Khalaf, Asma Al-Tamimi, Frank L. Lewis

Publication date: 13 April 2007

Published in: Automatica (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1016/j.automatica.2006.09.019

zbMATH Keywords

adaptive controlzero-sum gamesapproximate dynamic programming\(Q\)-functionpolicy iterations\(H_{\infty }\) optimal controladaptive critics\(Q\)-learning

Mathematics Subject Classification ID

Computational learning theory (68Q32) Learning and adaptive systems in artificial intelligence (68T05) Dynamic programming in optimal control and differential games (49L20) Discrete-time games (91A50) Adaptive control/observation systems (93C40) (H^infty)-control (93B36)

Related Items (87)

Online identifier–actor–critic algorithm for optimal control of nonlinear systems ⋮ A review of stochastic algorithms with continuous value function approximation and some new approximate policy iteration algorithms for multidimensional continuous applications ⋮ Output feedback Q-learning for discrete-time linear zero-sum games with application to the \(H_\infty\) control ⋮ Three bounded proofs for nonlinear multi‐input multi‐output approximate dynamic programming based on the <scp>L</scp>yapunov stability theory ⋮ An integrated data-driven Markov parameters sequence identification and adaptive dynamic programming method to design fault-tolerant optimal tracking control for completely unknown model systems ⋮ Event-triggered adaptive dynamic programming for discrete-time multi-player games ⋮ Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics ⋮ Model-free \(H_{\infty }\) control design for unknown linear discrete-time systems via Q-learning with LMI ⋮ On the effect of probing noise in optimal control LQR via Q-learning using adaptive filtering algorithms ⋮ H_∞ optimal control of unknown linear systems by adaptive dynamic programming with applications to time‐delay systems ⋮ Modified general policy iteration based adaptive dynamic programming for unknown discrete‐time linear systems ⋮ Nonlinear discrete time optimal control based on Fuzzy Models ⋮ optimal control for semi‐Markov jump linear systems via TP‐free temporal difference () learning ⋮ Improved model-free H∞ control for batch processes via off-policy 2D game Q-learning ⋮ A new self-learning optimal control laws for a class of discrete-time nonlinear systems based on ESN architecture ⋮ Optimal event‐triggered control for C‐T system with asymmetric constraints based on dual heuristic dynamic programing structure ⋮ optimal control of unknown continuous time linear periodic systems by adaptive dynamic programming with applications to magnetic attitude control ⋮ Complete stability analysis of a heuristic approximate dynamic programming control design ⋮ Output‐feedback Q‐learning for discrete‐time linear H^∞ tracking control: A Stackelberg game approach ⋮ Optimal game theoretic solution of the pursuit‐evasion intercept problem using on‐policy reinforcement learning ⋮ Model-free finite-horizon optimal control of discrete-time two-player zero-sum games ⋮ Optimal output tracking control of linear discrete-time systems with unknown dynamics by adaptive dynamic programming and output feedback ⋮ Minimax Q-learning control for linear systems using the Wasserstein metric ⋮ Undiscounted reinforcement learning for infinite-time optimal output tracking and disturbance rejection of discrete-time LTI systems with unknown dynamics ⋮ Off‐policy model‐based end‐to‐end safe reinforcement learning ⋮ Specified convergence rate guaranteed output tracking of discrete-time systems via reinforcement learning ⋮ Unnamed Item ⋮ An adaptive dynamic programming-based algorithm for infinite-horizon linear quadratic stochastic optimal control problems ⋮ Model-based reinforcement learning for approximate optimal regulation ⋮ Robust H_∞ tracking of linear <scp>discrete‐time</scp> systems using <scp>Q‐learning</scp> ⋮ Non-zero sum Nash Q-learning for unknown deterministic continuous-time linear systems ⋮ Data-based \(\mathcal{L}_2\) gain optimal control for discrete-time system with unknown dynamics ⋮ An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games ⋮ Model-free policy iteration approach to NCE-based strategy design for linear quadratic Gaussian games ⋮ Reinforcement learning algorithms with function approximation: recent advances and applications ⋮ Data‐driven control for networked systems with multiple packet dropouts ⋮ Robust optimal control of the multi‐input systems with unknown disturbance based on adaptive integral reinforcement learning Q‐function ⋮ Output feedback tracking control of a class of continuous-time nonlinear systems via adaptive dynamic programming approach ⋮ Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses ⋮ From model-based control to data-driven control: survey, classification and perspective ⋮ Designing of robust adaptive passivity-based controller based on reinforcement learning for nonlinear port-Hamiltonian model with disturbance ⋮ Reinforcement \(Q\)-learning for optimal tracking control of linear discrete-time systems with unknown dynamics ⋮ A Q-learning predictive control scheme with guaranteed stability ⋮ Data-driven policy iteration algorithm for continuous-time stochastic linear-quadratic optimal control problems ⋮ Finite-horizon optimal control of discrete-time linear systems with completely unknown dynamics using Q-learning ⋮ Q-learning for continuous-time linear systems: A model-free infinite horizon optimal control approach ⋮ Optimal distributed synchronization control for continuous-time heterogeneous multi-agent differential graphical games ⋮ Data-driven approximate value iteration with optimality error bound analysis ⋮ \(\mathrm{H}_\infty\) control of linear discrete-time systems: off-policy reinforcement learning ⋮ Integral \(Q\)-learning and explorized policy iteration for adaptive optimal control of continuous-time linear systems ⋮ Stochastic Optimal Design for Unknown Linear Discrete‐Time System Zero‐Sum Games in Input‐Output form Under Communication Constraints ⋮ \( \mathbb{Q} \)-learning algorithm in solving consensusability problem of discrete-time multi-agent systems ⋮ Data driven secure control for cyber-physical systems under hybrid attacks: a Stackelberg game approach ⋮ A reinforcement learning-based scheme for direct adaptive optimal control of linear stochastic systems ⋮ Stability and monotone convergence of generalised policy iteration for discrete-time linear quadratic regulations ⋮ Convergence of the standard RLS method andUDU^Tfactorisation of covariance matrix for solving the algebraic Riccati equation of the DLQR via heuristic approximate dynamic programming ⋮ Multiperson zero-sum differential games for a class of uncertain nonlinear systems ⋮ Nearly data-based optimal control for linear discrete model-free systems with delays via reinforcement learning ⋮ Value iteration for LQR control of unknown stochastic-parameter linear systems ⋮ Reinforcement learning and non-zero-sum game output regulation for multi-player linear uncertain systems ⋮ Disturbance compensation based model-free adaptive tracking control for nonlinear systems with unknown disturbance ⋮ Infinite time linear quadratic Stackelberg game problem for unknown stochastic discrete-time systems via adaptive dynamic programming approach ⋮ Finite-frequency output feedback MPC for Markov jump systems ⋮ Finite-horizon Q-learning for discrete-time zero-sum games with application to \(H_{\infty}\) control ⋮ Reinforcement learning for inverse linear-quadratic dynamic non-cooperative games ⋮ \(\mathcal{L}_2\) gain tracking control of linear completely unknown discrete-time networked control systems with dropout ⋮ Input-output data based tracking control under DoS attacks ⋮ Robust \(Q\)-learning algorithm for Markov decision processes under Wasserstein uncertainty ⋮ Successive over relaxation for model-free LQR control of discrete-time Markov jump systems ⋮ Mean field LQG social optimization: a reinforcement learning approach ⋮ Adaptive optimal control for continuous-time linear systems based on policy iteration ⋮ Model-FreeH_∞Control Design for Unknown Continuous-Time Linear System Using Adaptive Dynamic Programming ⋮ Bias-policy iteration based adaptive dynamic programming for unknown continuous-time linear systems ⋮ Off-policy Q-learning: solving Nash equilibrium of multi-player games with network-induced delay and unmeasured state ⋮ Design and Comparison Base Analysis of Adaptive Estimator for Completely Unknown Linear Systems in the Presence of OE Noise and Constant Input Time Delay ⋮ Linear-quadratic zero-sum mean-field type games: optimality conditions and policy optimization ⋮ Q-learning solution for optimal consensus control of discrete-time multiagent systems using reinforcement learning ⋮ Adaptive optimal output regulation of linear discrete-time systems based on event-triggered output-feedback ⋮ Adaptive optimal tracking controls of unknown multi-input systems based on nonzero-sum game theory ⋮ Approximate-optimal control algorithm for constrained zero-sum differential games through event-triggering mechanism ⋮ Data-driven control of hydraulic servo actuator based on adaptive dynamic programming ⋮ Mixed density methods for approximate dynamic programming ⋮ Dissipativity-based verification for autonomous systems in adversarial environments ⋮ Multi-agent reinforcement learning: a selective overview of theories and algorithms ⋮ Computational intelligence in uncertainty quantification for learning control and differential games ⋮ Finite-horizon optimal control for continuous-time uncertain nonlinear systems using reinforcement learning ⋮ Multi-objective optimal control for a class of nonlinear time-delay systems via adaptive dynamic programming

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Values and strategies for infinite time linear quadratic games
• On values and strategies for infinite-time linear quadratic games
• Kronecker products and matrix calculus in system theory
• The discrete-time Riccati equation related to the H/sub ∞/ control problem
• H/sub ∞/-control of discrete-time nonlinear systems
• \(H^ \infty\)-optimal control and related minimax design problems. A dynamic game approach.

This page was built for publication: Model-free \(Q\)-learning designs for linear discrete-time zero-sum games with application to \(H^\infty\) control