Stochastic Recursive Inclusions in Two Timescales with Nonadditive Iterate-Dependent Markov Noise

DOI10.1287/moor.2019.1037zbMath1460.62136arXiv1611.05961OpenAlexW3044123424MaRDI QIDQ3387930

Vinayaka G. Yaji, Shalabh Bhatnagar

Publication date: 8 January 2021

Published in: Mathematics of Operations Research (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1611.05961

zbMATH Keywords

set-valued mapsdifferential inclusionsstochastic approximationconstrained convex optimizationiterate-dependent Markov noise

Mathematics Subject Classification ID

Variational inequalities (49J40) Stochastic programming (90C15) Stochastic approximation (62L20) Processes in random environments (60K37) Numerical methods for variational inequalities and related problems (65K15)

Related Items

Constant step stochastic approximations involving differential inclusions: stability, long-run convergence and applications, Full Gradient DQN Reinforcement Learning: A Provably Convergent Scheme

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Subgradient methods for saddle-point problems
• Convergence and robustness of the Robbins-Monro algorithm truncated at randomly varying bounds
• Stochastic approximation with two time scales
• The inflation of attractors and their discretization: The autonomous case
• On dynamical properties of general dynamical systems and differential inclusions
• Simultaneous perturbation Newton algorithms for simulation optimization
• Stochastic approximation with `controlled Markov' noise
• Convergence of Markovian Stochastic Approximation with Discontinuous Dynamics
• Stochastic recursive inclusion in two timescales with an application to the Lagrangian dual problem
• Applications of a Kushner and Clark lemma to general classes of stochastic algorithms
• Numerical solution of saddle point problems
• Multivariate stochastic approximation using a simultaneous perturbation gradient approximation
• A Dynamical System Approach to Stochastic Approximations
• The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning
• Stochastic Approximation Approaches to the Stochastic Variational Inequality Problem
• Stochastic recursive inclusions with non-additive iterate-dependent Markov noise
• Analysis of Stochastic Approximation Schemes With Set-Valued Maps in the Absence of a Stability Guarantee and Their Stabilization
• Asynchronous stochastic approximation with differential inclusions
• Two Time-Scale Stochastic Approximation with Controlled Markov Noise and Off-Policy Temporal-Difference Learning
• Stability of Stochastic Approximation under Verifiable Conditions
• Stochastic Approximations and Differential Inclusions
• Regularized Iterative Stochastic Approximation Methods for Stochastic Variational Inequality Problems
• A Generalization of the Borkar-Meyn Theorem for Stochastic Recursive Inclusions
• Envelope Theorems for Arbitrary Choice Sets
• Extragradient Method with Variance Reduction for Stochastic Variational Inequalities
• A Stochastic Approximation Method