Stochastic Recursive Inclusions in Two Timescales with Nonadditive Iterate-Dependent Markov Noise

DOI10.1287/moor.2019.1037zbMath1460.62136arXiv1611.05961MaRDI QIDQ3387930

Vinayaka G. Yaji, Shalabh Bhatnagar

Publication date: 8 January 2021

Published in: Mathematics of Operations Research (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1611.05961

zbMATH Keywords

set-valued maps; differential inclusions; stochastic approximation; constrained convex optimization; iterate-dependent Markov noise

Mathematics Subject Classification ID

49J40: Variational inequalities

90C15: Stochastic programming

62L20: Stochastic approximation

60K37: Processes in random environments

65K15: Numerical methods for variational inequalities and related problems

Related Items

Constant step stochastic approximations involving differential inclusions: stability, long-run convergence and applications, Full Gradient DQN Reinforcement Learning: A Provably Convergent Scheme

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Subgradient methods for saddle-point problems
• Convergence and robustness of the Robbins-Monro algorithm truncated at randomly varying bounds
• Stochastic approximation with two time scales
• The inflation of attractors and their discretization: The autonomous case
• On dynamical properties of general dynamical systems and differential inclusions
• Simultaneous perturbation Newton algorithms for simulation optimization
• Stochastic approximation with `controlled Markov' noise
• Convergence of Markovian Stochastic Approximation with Discontinuous Dynamics
• Stochastic recursive inclusion in two timescales with an application to the Lagrangian dual problem
• Applications of a Kushner and Clark lemma to general classes of stochastic algorithms
• Numerical solution of saddle point problems
• Multivariate stochastic approximation using a simultaneous perturbation gradient approximation
• A Dynamical System Approach to Stochastic Approximations
• The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning
• Stochastic Approximation Approaches to the Stochastic Variational Inequality Problem
• Stochastic recursive inclusions with non-additive iterate-dependent Markov noise
• Analysis of Stochastic Approximation Schemes With Set-Valued Maps in the Absence of a Stability Guarantee and Their Stabilization
• Asynchronous stochastic approximation with differential inclusions
• Two Time-Scale Stochastic Approximation with Controlled Markov Noise and Off-Policy Temporal-Difference Learning
• Stability of Stochastic Approximation under Verifiable Conditions
• Stochastic Approximations and Differential Inclusions
• Regularized Iterative Stochastic Approximation Methods for Stochastic Variational Inequality Problems
• A Generalization of the Borkar-Meyn Theorem for Stochastic Recursive Inclusions
• Envelope Theorems for Arbitrary Choice Sets
• Extragradient Method with Variance Reduction for Stochastic Variational Inequalities
• A Stochastic Approximation Method