Stabilization of stochastic approximation by step size adaptation

DOI10.1016/J.SYSCONLE.2012.02.005zbMATH Open1250.93128arXiv1007.4689OpenAlexW2080900138MaRDI QIDQ450652FDOQ450652

Sameer Kamal

Publication date: 14 September 2012

Published in: Systems \& Control Letters (Search for Journal in Brave)

Abstract: A scheme for stabilizing stochastic approximation iterates by adaptively scaling the step sizes is proposed and analyzed. This scheme leads to the same limiting differential equation as the original scheme and therefore has the same limiting behavior, while avoiding the difficulties associated with projection schemes. The proof technique requires only that the limiting o.d.e. descend a certain Lyapunov function outside an arbitrarily large bounded set.

Full work available at URL: https://arxiv.org/abs/1007.4689

zbMATH Keywords

stochastic approximationalmost sure boundednesslimiting ordinary differential equationstep size adaptation

Mathematics Subject Classification ID

Stochastic ordinary differential equations (aspects of stochastic analysis) (60H10) Stochastic stability in control theory (93E15)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Stochastic approximation methods for constrained and unconstrained systems
• A Stochastic Approximation Method
• Stability of Stochastic Approximation under Verifiable Conditions
• Stochastic approximation and its applications
• Two models for analyzing the dynamics of adaptation algorithms
• Analysis of recursive stochastic algorithms
• OnActor-Critic Algorithms
• Asynchronous stochastic approximation and Q-learning
• Stochastic Approximation for Nonexpansive Maps: Application to Q-Learning Algorithms
• A Stochastic Approximation Algorithm with Varying Bounds
• The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning
• A Scaled Stochastic Approximation Algorithm

Cited In (9)

• Rates of convergence of adaptive step-size of stochastic approximation algorithms
• Optimal step sizes in semi-stochastic approximation procedures. I
• Optimal step sizes in semi-stochastic approximation procedures. II
• Convergence of Markovian stochastic approximation with discontinuous dynamics
• Asynchronous stochastic approximation with differential inclusions
• MINISYMPOSIUM 3
• Analysis of practical step size selection in stochastic approximation algorithms
• Markovian stochastic approximation with expanding projections
• Stability of annealing schemes and related processes

This page was built for publication: Stabilization of stochastic approximation by step size adaptation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q450652)