A stochastic contraction mapping theorem

DOI10.1016/J.SYSCONLE.2023.105482MaRDI QIDQ6161346zbMATH OpenWikidataFDO

Authors Anthony Almudevar

Publication date 27 June 2023

Published in Systems \& Control Letters (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2207.00618

Mathematics Subject Classification ID

Optimal stochastic control (93E20) Least squares and related methods for stochastic control systems (93E24) Stochastic learning and adaptive control (93E35)

Abstract: In this paper we define contractive and nonexpansive properties for adapted stochastic processes $X_1,X_2,ldots$ which can be used to deduce limiting properties. In general, nonexpansive processes possess finite limits while contractive processes converge to zero $a.e.$ Extensions to multivariate processes are given. These properties may be used to model a number of important processes, including stochastic approximation and least-squares estimation of controlled linear models, with convergence properties derivable from a single theory. The approach has the advantage of not in general requiring analytical regularity properties such as continuity and differentiability.

Recommendations

• A concentration bound for contractive stochastic approximation
• Stochastic Approximation for Nonexpansive Maps: Application to Q-Learning Algorithms
• Contraction mappings underlying undiscounted Markov decision problems. II
• scientific article; zbMATH DE number 3870440
• On the almost sure convergence of a general stochastic approximation procedure

Cites work

• \({\mathcal Q}\)-learning
• A Stochastic Approximation Method
• Approximate Fixed Point Iteration with an Application to Infinite Horizon Markov Decision Processes
• Approximate iterative algorithms
• Approximation Methods which Converge with Probability one
• Asymptotic Properties of Multiperiod Control Rules in the Linear Regression Model
• Asynchronous stochastic approximation and Q-learning
• scientific article; zbMATH DE number 5957196 (Why is no real title available?)
• scientific article; zbMATH DE number 3538599 (Why is no real title available?)
• scientific article; zbMATH DE number 1321699 (Why is no real title available?)
• scientific article; zbMATH DE number 1972910 (Why is no real title available?)
• scientific article; zbMATH DE number 3449561 (Why is no real title available?)
• Optimal Approximation Schedules for a Class of Iterative Algorithms, With an Application to Multigrid Value Iteration
• Probability
• Robust Stochastic Approximation Approach to Stochastic Programming
• Stochastic approximation
• Strong consistency of least squares estimates in dynamic models
• Strong consistency of least squares estimates in multiple regression II
• Strong consistency of least squares estimates in normal linear regression
• Strong consistency of least squares estimators in linear regression models
• Strong convergence of a stochastic approximation algorithm
• Weak and strong consistency of the least squares estimators in regression models

Cited in

(4)

• A concentration bound for contractive stochastic approximation
• scientific article; zbMATH DE number 4067986 (Why is no real title available?)
• Stochastic Approximation for Nonexpansive Maps: Application to Q-Learning Algorithms
• Stochastic Fixed-Point Iterations for Nonexpansive Maps: Convergence and Error Bounds

This page was built for publication: A stochastic contraction mapping theorem

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6161346)