Optimal control of an Ornstein-Uhlenbeck process (Q1090304)

A one-dimensional linear time-invariant control system is considered described by a first order differential state equation with additive white Gaussian disturbance of zero mean and known covariance rate. The control objective is to minimize the expected value of the cost function which is a linear combination of time and control energy. The terminal time is the moment of first entry into a prescribed set. Two terminal state sets are considered. The first one is the complement of the prescribed interval symmetric with respect to zero. In this case the weight of the time in the cost functional is positive. Thus the control problem is to increase the absolute value of the state variable above the given level as soon as possible taking however the control energy into account. In the second problem the terminal set contains all states less than a given value and the weight of the time in the performance index is negative. Thus the objective is to keep the state above the fixed value as long as possible minimizing the control energy at the same time. Unfortunately, the author does not specify what type of information is available, but he seems to assume that the state is measured perfectly. The dynamic programming equation is applied and the optimal control law is found in the form of a nonlinear function of the state. The solution, although interesting from a mathematical point of view, is too complicated to fix attention of a control engineer. Only for very large and very small states the control law is linear and independent of the bounds imposed on the terminal state. The control strategy simplifies also in the deterministic case which is considered in the paper by assuming zero covariance rate of the noise. It is almost linear independent of the barrier and leads to an unstable equation of the closed loop system. It is proved that the weight of the terminal time in the cost function must not be too negative, because it leads to infinite cost.