A learning algorithm for discrete-time stochastic control

DOI10.1017/S0269964800142081MaRDI QIDQ2711577zbMATH OpenOpenAlexFDO

Authors Vivek Borkar

Publication date 2 February 2004

Published in Probability in the Engineering and Informational Sciences (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1017/s0269964800142081

zbMATH Keywords

\(Q\)-learning algorithmalmost sure convergencecompact action spacecompact state spacediscrete-time stochastic controllearningnonlinear controlsimulation based algorithm

Mathematics Subject Classification ID

Nonlinear systems in control theory (93C10) Discrete-time control/observation systems (93C55) Stochastic learning and adaptive control (93E35)

Review text

A simulation based algorithm for learning ``good policies for a discrete-time stochastic control process with unknown transition law is treated with the state and action spaces both being compact subsets of Euclidean spaces. Under suitable conditions almost sure convergence is proved. The paper is in the spirit of \textit{W. L. Baker} (PhD. Thesis, Harvard University 1997), but it analyzes the full nonlinear case and is in the tradition of the ordinary differential equation approach.

Recommendations

• On a discrete-time stochastic learning control algorithm
• Learning optimal control in deterministic systems
• A discrete-time stochastic learning control algorithm
• A Q-Learning Algorithm for Discrete-Time Linear-Quadratic Control with Random Parameters of Unknown Distribution: Convergence and Stabilization
• scientific article; zbMATH DE number 4106720

Cited in

(8)

• A near-optimal polynomial time algorithm for learning in certain classes of stochastic games
• A simple strategy of behavior in a stationary medium with exponential guaranteed convergence rate
• On learning discontinuous human control strategies
• A discrete-time stochastic learning control algorithm
• On a discrete-time stochastic learning control algorithm
• A controller design algorithm with learning property
• A Q-Learning Algorithm for Discrete-Time Linear-Quadratic Control with Random Parameters of Unknown Distribution: Convergence and Stabilization
• Stochastic approximation algorithms: overview and recent trends.

This page was built for publication: A learning algorithm for discrete-time stochastic control

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2711577)