Minimax Q-learning control for linear systems using the Wasserstein metric (Q2681392)

Consider an optimal regulator problem with a discrete-time, time-invariant linear stochastic state equation. The probability distribution of a random variable \(\xi\) in the additive noise term is unknown, but samples of the random variable are available. The parameter matrices in the linear state equation are only partly known. The control input \(u(t)\) depends on the control history \(h_t\) up to time \(t\). The objective is the expectation of a cost function being quadratic in the control and state vectors and still depends on the distance between the empirical distribution and a possible distribution of \(\xi\). The uncertainty with respect to the distribution of \(\xi\) is incorporated by treating the optimal regulator problem as a minimax problem -- maximizing with respect to the disturbances and minimizing with respect to the controls. The resulting discounted infinite stage stochastic decision process is then approximately solved by using the Bellman equation. A numerical example is given.