Long-term average cost control problems for continuous time Markov processes: A survey

DOI10.1007/BF00046603MaRDI QIDQ788690zbMATH OpenFDO

Publication date 1983

Published in Acta Applicandae Mathematicae (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1007/BF00046603

continuous control, stopping problem impulse control long-term average cost undiscounted stochastic control Markov processes

Mathematics Subject Classification ID

Variational inequalities (49J40) Existence of optimal solutions to problems involving randomness (49J55) Optimality conditions for problems involving randomness (49K45) Dynamic programming in optimal control and differential games (49L20) Stopping times; optimal stopping problems; gambling theory (60G40) Continuous-time Markov processes on general state spaces (60J25) Optimal stopping in statistics (62L15) Dynamic programming (90C39) Optimal stochastic control (93E20)

Review text

This paper is a survey of long term average cost control problems, namely the cost is defined by \[ J(v)=\underline{\lim}_{T\to \infty}\frac{1}{T}E\int^{T}_{0}f(y(s,v),v(s))ds, \] where \(y(\cdot,v)\) is the response for control v. The object is to minimize J(v). The following simple example illustrate the kind of phenomena treated in the paper. Example: Let V be the totality of Lipschitz continuous functions. For \(v\in V\), \(y(t,v)\) is the solution of SDE, \[ dy_ x(t)=v(y_ x(t))dt+\sqrt{2}dw(t),\quad y_ x(0)=x(\in R^ 1). \] The discounted cost is given by \(J_ x^{\alpha}(v)=E\int^{\infty}_{0}e^{-\alpha t}(y_ x(t)^ 2+v^ 2(y_ x(t)))dt\). The optimal cost function \(u_{\alpha}(x)=\inf_{v\in V}J_ x^{\alpha}(v)\) satisfies the Hamilton-Jacobi-Bellman (H-J-B) equation \(u_{\alpha}(x)-\inf_{v\in R^ 1}(vu'_{\alpha}(x)+v^ 2)+\alpha u_{\alpha}=x^ 2\). Actually \(u_{\alpha}(x)=(\sqrt{\alpha^ 2+4}-\alpha)(\frac{x^ 2}{2}+\frac{1}{\alpha})\). Hence, \(\lim_{\alpha \to 0}\alpha u_{\alpha}(x)=2\) and \(W(x)=\lim_{\alpha \to 0}(u_{\alpha}(x)- u_{\alpha}(0))=x^ 2\) satisfies \[ -W(x)-\inf_{v\in R^ 1}(vW'(x)+v^ 2)+\lambda =x^ 2. \] Moreover, putting \(V_ 0=\{v\in V;\lim_{T\to \infty}\frac{1}{T}EW(y_ x(T,v))=0\}\), \[ \lambda =\inf_{v\in V_ 0}(\lim_{T\to \infty}\frac{1}{T}E\int^{T}_{0}y^ 2_ x(t)+v^ 2(y_ x(t))dt) \] and an optimal control \(\hat v(x)=-x\) is a minimum selection of \((vW'(x)+v^ 2)\). In many cases, the problem relates to a solution \((\lambda,W)\) of the H-J-B equation \(\inf_{v\in K}(A^ vW+f(x,v))- \lambda =0,\) and using the discounted problem a solution is obtained. The author treats three problems: continuous control, stopping problem and impulse control, and discusses some open problems.

Recommendations

Cites work

Cited in

(39)

This page was built for publication: Long-term average cost control problems for continuous time Markov processes: A survey

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q788690)