System stability and optimal control (Q1286601): Difference between revisions

A random walk process with two absorbing boundaries \(x=0\) and \(x=n\) is considered. Let \(X_t\), \(t=1,2,\dots\), be the jumps of the random walk and let \(S_t= x+ \sum_{i=1}^t X_i\) where \(x\) is an initial state. The main object of investigation is the following stopping time \(\eta_{x,n}= \min \{t>0: S_t= 0\) or \(S_t=n\}\). It is supposed that for some positive functions \(p\), \(q\), \(r\), defined on the state space of \(\{S_t\}\), conditional probabilities satisfy \[ \begin{aligned} P(X_{t+1}&= 1\mid S_t= y)= p(y),\\ P(X_{t+1}&= -1\mid S_t= y)= q(y),\\ P(X_{t+1}&= 0\mid S_t= y)= r(y). \end{aligned} \] The process can be controlled. A control strategy with \(\ell\) switching levels \(\eta_i\), \(i=1, 2,\dots, \ell\), \(0= \eta_0< \eta_1<\cdots< \eta_\ell< \eta_{\ell+1}= n\), means that \(p(y)= p_t\), \(q(y)= q_t\), \(r(y)= r_t\) for \(y\in [\eta_t, \eta_{t+1})\) and moreover for some \(k,m\), \(1\leq k\leq m\leq\ell\); \(p_g> q_t\), \(t=0,1,\dots, k-1\); \(p_t= q_t\), \(t= k, k+1,\dots, m-1\); \(p_t< q_t\), \(t= m+1,\dots, \ell\). The asymptotic behaviour (as \(n\to \infty\)) of the normalized stopping time \(\eta_{x,n}/ E\eta_{x,n}\) corresponding to the control strategy is studied. For an objective function some optimal control problems are solved.