Asynchronous stochastic approximation with differential inclusions
From MaRDI portal
Abstract: The asymptotic pseudo-trajectory approach to stochastic approximation of Benaim, Hofbauer and Sorin is extended for asynchronous stochastic approximations with a set-valued mean field. The asynchronicity of the process is incorporated into the mean field to produce convergence results which remain similar to those of an equivalent synchronous process. In addition, this allows many of the restrictive assumptions previously associated with asynchronous stochastic approximation to be removed. The framework is extended for a coupled asynchronous stochastic approximation process with set-valued mean fields. Two-timescales arguments are used here in a similar manner to the original work in this area by Borkar. The applicability of this approach is demonstrated through learning in a Markov decision process.
Recommendations
- Asynchronous Stochastic Approximations
- Stochastic recursive inclusion in two timescales with an application to the Lagrangian dual problem
- The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning
- The Borkar-Meyn theorem for asynchronous stochastic approximations
- Asymptotic behavior of asynchronous stochastic approximation
- Asynchronous stochastic approximation and Q-learning
- Stochastic recursive inclusions with non-additive iterate-dependent Markov noise
- Event-driven stochastic approximation
Cites work
- A Dynamical System Approach to Stochastic Approximations
- Actor-Critic--Type Learning Algorithms for Markov Decision Processes
- Analysis of recursive stochastic algorithms
- Asymptotic Properties of Distributed and Communicating Stochastic Approximation Algorithms
- Asynchronous stochastic approximation and Q-learning
- Asynchronous Stochastic Approximations
- Convergence results for single-step on-policy reinforcement-learning algorithms
- Convergent multiple-timescales reinforcement learning algorithms in normal form games
- scientific article; zbMATH DE number 47179 (Why is no real title available?)
- scientific article; zbMATH DE number 1354815 (Why is no real title available?)
- Mixed equilibria and dynamical systems arising from fictitious play in perturbed games
- On the Theory of Dynamic Programming
- OnActor-Critic Algorithms
- Stabilization of stochastic approximation by step size adaptation
- Stochastic approximation algorithms for parallel and distributed processing
- Stochastic approximation methods for constrained and unconstrained systems
- Stochastic approximation with `controlled Markov' noise
- Stochastic approximation with two time scales
- Stochastic approximation. A dynamical systems viewpoint.
- Stochastic Approximations and Differential Inclusions
- Stochastic Approximations and Differential Inclusions, Part II: Applications
- Stochastic approximations for finite-state Markov chains
- Viability theory
Cited in
(17)- Learning in games with continuous action sets and unknown payoff functions
- Q-learning for Markov decision processes with a satisfiability criterion
- Reference points and learning
- An asynchronous stochastic approximation theorem and some applications
- Stochastic recursive inclusion in two timescales with an application to the Lagrangian dual problem
- Stochastic Recursive Inclusions in Two Timescales with Nonadditive Iterate-Dependent Markov Noise
- Penalty-regulated dynamics and robust learning procedures in games
- Asynchronous Stochastic Approximations
- Robustness properties in fictitious-play-type algorithms
- Asymptotic agreement and convergence of asynchronous stochastic algorithms
- Stochastic recursive inclusions with non-additive iterate-dependent Markov noise
- Stochastic Approximations and Differential Inclusions, Part II: Applications
- The Borkar-Meyn theorem for asynchronous stochastic approximations
- Stochastic approximation with discontinuous dynamics, differential inclusions, and applications
- Independent learning in stochastic games
- Algorithmic collusion and a Folk theorem from learning with bounded rationality
- Logit-Q dynamics for efficient learning in stochastic teams
This page was built for publication: Asynchronous stochastic approximation with differential inclusions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5168859)
