Convergence of stochastic approximation via martingale and converse Lyapunov methods

DOI10.1007/s00498-023-00342-9zbMath1518.93148arXiv2205.01303MaRDI QIDQ6097904

Mathukumalli Vidyasagar

Publication date: 7 June 2023

Published in: MCSS. Mathematics of Control, Signals, and Systems (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2205.01303

zbMATH Keywords

stochastic approximationglobal asymptotic stabilitymartingale methodsconverse Lyapunov theory

Mathematics Subject Classification ID

Generalizations of martingales (60G48) Asymptotic stability in control theory (93D20) Stochastic stability in control theory (93E15)

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Stochastic approximation methods for constrained and unconstrained systems
• Strong convergence of a stochastic approximation algorithm
• New converse Lyapunov theorems and related results on exponential stability
• Two models for analyzing the dynamics of adaptation algorithms
• A stability criterion for two timescale stochastic approximation schemes
• Applications of a Kushner and Clark lemma to general classes of stochastic algorithms
• Bounds for Solutions of Ordinary Differential Equations
• Analysis of recursive stochastic algorithms
• An analysis of temporal-difference learning with function approximation
• Nonlinear Systems Analysis
• The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning
• A Stochastic Approximation Method
• Stochastic approximation