Suboptimal regulators for discrete-time jump linear systems with time-multiplied performance index (Q1810923)

Given the linear control system \[ x(t+1)=A(r(t))x(t) + B(r(t))u(t), \bigl(x(0)=x_0 \in {\mathbb{R}}^n;\;t=0,1,2,\dots,)\tag{1} \] with control \(u(t) \in {\mathbb{R}}^p\), the Markov chain \(r(t)\) taking values in \(S=\{1,2,\dots,N\}\) and being governed by the transition matrix \(((p_{ij}))\) and \(A\) and \(B\) matrix-valued functions on \(S\), the authors discuss the task to find a gain matrix \(K(r(t))\) that is a set of matrices \(K_1,\dots,K_N\) such that (i) the closed-loop system determined by the feedback control \(u(t)=-K(r(t))x(t)\) is stoschastically stable in the sense that \({\mathbf E}\sum_{t=1}^{\infty} | x(t)| ^2 < \infty\), and (ii) the time-multiplied performance index defined as \[ J(x_0,r_0) = {\mathbf E} \{ \sum_{t=1}^{\infty} t^k [x^T(t)Q(r(t))x(t) + u^T(t)R(r(t))u(t)] \}\;(Q \geq 0,\;R> 0\text{ given)}\tag{2} \] is minimized (where the minimum is taken over all controls \(u\)). Due to the fact that for a linear system of the form \(x(t+1)=C(r(t))x(t)\), \(C\) a matrix-valued function on \(S\), stochastic stability is equivalent to exponential mean square stability (i.e., existence of numbers \(\alpha >0\), \(0< \beta <1\), such that for all initial conditions and all \(t = 0,1,2,\dots\) \({\mathbf E}| x(t)| ^2 \leq \alpha | x_0| \beta^{-t}\)) as well as to the solvability of the Lyapunov equations \(P_i - C_i^T (\sum_{j=1}^N p_{ij} P_j) C_i = Q_i\) (\(Q_i>0\) given, \(i \in S\)) by positive definite matrices \(P_1,\dots,P_N\), under stochastic stability the time-multiplied index (2) is defined and finite for all \(k\) and can be represented with the help of the solutions \(P_i\). Hereby it is also seen that one cannot expect a closed-form solution for the minimum or optimum of (2). The authors propose an algorithm for suboptimal solution and illustrate the procedure by an example. The question of its convergence remains open.