Asynchronous Stochastic Approximations
The asymptotic behavior of a distributed, asynchronous stochastic approximation scheme is analyzed in terms of a limiting nonautonomous differential equation. The relation between the latter and the relative values of suitably rescaled relative frequencies of updates of different components is underscored. The model of asynchronism postulates a set-valued random process that marks the indices to be updated at each iteration. This clumping of indices into sets can be an artifice as long as causal relationships are not violated; thus the set-up is very general, indeed. We impose on this process a natural condition that requires all indices to be updated comparably often in a precise sense. In addition, we allow random, possibly nonstationary and unbounded delays that are required to satisfy a mild conditional moment condition.The analysis depends on proving that the algorithm asymptotically tracks a nonautonomous ODE, in contrast to the traditional autonomous ODE limit. In particular, it gives a handle on situations when the latter may not be feasible. The ODE in question differs from the traditional one in that each component of the driving vector field is now weighted by a time-varying nonnegative scalar. These scalars add to 1 and may be interpreted as relative frequencies of updates of different components after suitable time-scaling. This brings out the desired relationships between updated schemes for different components and paves the way for analyzing situations where they are not desirable.
- A stochastic gradient type algorithm for closed-loop problems
- A note on the effect of asynchronous sampling on estimation accuracy
- Reinforcement learning for long-run average cost.
- Convergence analysis of contrastive divergence algorithm based on gradient method with errors
- Approachability in Stackelberg stochastic games with vector costs
- Q-learning for Markov decision processes with a satisfiability criterion
- Distributed time synchronization for networks with random delays and measurement noise
- Whittle index based Q-learning for restless bandits with average reward
- Fully asynchronous stochastic coordinate descent: a tight lower bound on the parallelism achieving linear speedup
- Random asynchronous iterations in distributed coordination algorithms
- Event-driven stochastic approximation
- Reinforcement learning based algorithms for average cost Markov decision processes
- Nonlinear gossip
- Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design
- Stochastic fictitious play with continuous action sets
- An online actor-critic algorithm with function approximation for constrained Markov decision processes
- Iterative learning control for large scale nonlinear systems with observation noise
- scientific article; zbMATH DE number 1788242 (Why is no real title available?)
- Asymptotic agreement and convergence of asynchronous stochastic algorithms
- Partially asynchronous co-state prediction algorithm
- Asymptotics of Reinforcement Learning with Neural Networks
- Asynchronous stochastic approximation with differential inclusions
- The Borkar-Meyn theorem for asynchronous stochastic approximations
- Stochastic approximation algorithms: overview and recent trends.
- A sensitivity formula for risk-sensitive cost and the actor-critic algorithm
- Convergence rates for stochastic approximation: biased noise with unbounded variance, and applications
- Current fluctuations of a self-interacting diffusion on a ring
- Analysis of multiscale reinforcement Q-learning algorithms for mean field control games
- Stochastic approximation and reinforcement learning: the interface and a little beyond
- Distributed asynchronous consensus-based algorithm for blind calibration of sensor networks with autonomous gain correction
- Stochastic approximation in infinite dimensions
- Charge-based control of DiffServ-like queues
- Recent advances in stochastic approximation with applications to optimization and fixed point problems
- Asynchronous stochastic approximation with applications to average-reward reinforcement learning
- Revisiting stochastic approximation and stochastic gradient descent
- Asymptotic behavior of asynchronous stochastic approximation
This page was built for publication: Asynchronous Stochastic Approximations
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4388937)
