Convergence of the relative value iteration for the ergodic control problem of nondegenerate diffusions under near-monotone costs

DOI10.1137/130912918MaRDI QIDQ5494866zbMATH OpenOpenAlexWikidataFDO

Authors Vivek Borkar, K. Suresh Kumar, Ari Arapostathis

Publication date 30 July 2014

Published in SIAM Journal on Control and Optimization (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1303.0618

zbMATH Keywords

controlled diffusionsergodic controlHamilton-Jacobi-Bellman equationparabolic Cauchy problemrelative value iteration

Mathematics Subject Classification ID

Stochastic partial differential equations (aspects of stochastic analysis) (60H15) Continuous-time Markov processes on general state spaces (60J25) Diffusion processes (60J60) Markov and semi-Markov decision processes (90C40) Stochastic stability in control theory (93E15) Optimal stochastic control (93E20)

Abstract: We study the relative value iteration for the ergodic control problem under a near-monotone running cost structure for a nondegenerate diffusion controlled through its drift. This algorithm takes the form of a quasilinear parabolic Cauchy initial value problem in $R{R}^d$. We show that this Cauchy problem stabilizes, or in other words, that the solution of the quasilinear parabolic equation converges for every bounded initial condition in $C{c}^2(R{R}^d)$ to the solution of the Hamilton--Jacobi--Bellman (HJB) equation associated with the ergodic control problem.

Recommendations

• A relative value iteration algorithm for nondegenerate controlled diffusions
• A correction to ``A relative value iteration algorithm for nondegenerate controlled diffusions
• On Iteration Improvement for Averaged Expected Cost Control for One-Dimensional Ergodic Diffusions
• Ergodic control problem for one-dimensional diffusions with near-monotone cost
• On the global convergence of relative value iteration for infinite-horizon risk-sensitive control of diffusions

Cited in

(11)

• On the global convergence of relative value iteration for infinite-horizon risk-sensitive control of diffusions
• A relative value iteration algorithm for nondegenerate controlled diffusions
• On the policy iteration algorithm for nondegenerate controlled diffusions under the ergodic criterion
• Ergodic problems for viscous Hamilton-Jacobi equations with inward drift
• On the relative value iteration with a risk-sensitive criterion
• On uniform exponential ergodicity of Markovian multiclass many-server queues in the Halfin-Whitt regime
• On solutions of mean field games with ergodic cost
• Open Problem—Convergence and Asymptotic Optimality of the Relative Value Iteration in Ergodic Control
• On Iteration Improvement for Averaged Expected Cost Control for One-Dimensional Ergodic Diffusions
• A correction to ``A relative value iteration algorithm for nondegenerate controlled diffusions
• Asymptotic behavior of the generalized principal eigenvalue for Hamilton-Jacobi-Bellman equations with asymptotically radial data

This page was built for publication: Convergence of the relative value iteration for the ergodic control problem of nondegenerate diffusions under near-monotone costs

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5494866)