Stabilization of stochastic approximation by step size adaptation

DOI10.1016/J.SYSCONLE.2012.02.005MaRDI QIDQ450652zbMATH OpenOpenAlexFDO

Authors Sameer Kamal

Publication date 14 September 2012

Published in Systems \& Control Letters (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1007.4689

zbMATH Keywords

stochastic approximationalmost sure boundednesslimiting ordinary differential equationstep size adaptation

Mathematics Subject Classification ID

Stochastic ordinary differential equations (aspects of stochastic analysis) (60H10) Stochastic stability in control theory (93E15)

Abstract: A scheme for stabilizing stochastic approximation iterates by adaptively scaling the step sizes is proposed and analyzed. This scheme leads to the same limiting differential equation as the original scheme and therefore has the same limiting behavior, while avoiding the difficulties associated with projection schemes. The proof technique requires only that the limiting o.d.e. descend a certain Lyapunov function outside an arbitrarily large bounded set.

Recommendations

• Adaptive stochastic approximation algorithm
• scientific article; zbMATH DE number 4029264
• Rates of convergence of adaptive step-size of stochastic approximation algorithms
• Stability of Stochastic Approximation under Verifiable Conditions
• Stochastic approximation with adaptive step sizes for optimization in noisy environment and its application in regression models

Cites work

• scientific article; zbMATH DE number 5348356 (Why is no real title available?)
• scientific article; zbMATH DE number 48727 (Why is no real title available?)
• scientific article; zbMATH DE number 976356 (Why is no real title available?)
• scientific article; zbMATH DE number 1972910 (Why is no real title available?)
• scientific article; zbMATH DE number 825585 (Why is no real title available?)
• scientific article; zbMATH DE number 1405930 (Why is no real title available?)
• A Scaled Stochastic Approximation Algorithm
• A Stochastic Approximation Algorithm with Varying Bounds
• A Stochastic Approximation Method
• Analysis of recursive stochastic algorithms
• Asynchronous stochastic approximation and Q-learning
• OnActor-Critic Algorithms
• Stability of Stochastic Approximation under Verifiable Conditions
• Stochastic Approximation for Nonexpansive Maps: Application to Q-Learning Algorithms
• Stochastic approximation and its applications
• Stochastic approximation methods for constrained and unconstrained systems
• The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning
• Two models for analyzing the dynamics of adaptation algorithms

Cited in

(12)

• Analysis of practical step size selection in stochastic approximation algorithms
• Stability of annealing schemes and related processes
• Optimal step sizes in semi-stochastic approximation procedures. I
• Optimal step sizes in semi-stochastic approximation procedures. II
• Adaptive stochastic approximation algorithm
• Markovian stochastic approximation with expanding projections
• MINISYMPOSIUM 3
• Asynchronous stochastic approximation with differential inclusions
• Convergence of Markovian stochastic approximation with discontinuous dynamics
• Stability of Stochastic Approximation under Verifiable Conditions
• Rates of convergence of adaptive step-size of stochastic approximation algorithms
• Adaptive stepsize selection for tracking in a regime-switching environment

This page was built for publication: Stabilization of stochastic approximation by step size adaptation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q450652)