Nonconvexities in a stochastic control problem with learning (Q1175372)

The paper studies a class of discrete-time scalar linear-quadratic- Gaussian stochastic control problems where the coefficient terms in the state equation are unknown parameters with prescribed first and second moments. Such problems are known to be difficult in stochastic control (as far as the derivation of optimal controllers go), because of the dual (and sometimes triple) role of control. An optimal controller generally involves active learning, which brings in a tradeoff between probing and the cost of control activity. In the context of the specific example considered in this paper, the author shows that the probing component of the loss function has nonconvexities (which is one of the sources of difficulty), arising for primarily two reasons: (1) Failure of the precision matrix of the parameters to increase monotonically with the control variable; (2) Changes in the path of future state variables induced by modifying the certainty-equivalent control. The paper ends with some simulation studies on an example previously studied by \textit{E. C. McRae} [Ann. Econ. Social Meas. 1, 43-47 (1972)].