Stochastic control and nonequilibrium thermodynamical systems (Q1115004)

Let (\(\Omega\),\(\underset {=} F,P)\) be a probability space carrying a standard n-dimensional Wiener process \(W=(W_ t\); \(0\leq t\leq T)\) and \(X_ 0\), \(Y_ 0\) of dimension m and n, respectively, independent of W. We consider the stochastic processes \(X=(X(t)\); \(0\leq t\leq T)\) and \(Y=(Y_ t\); \(0\leq t\leq T)\) defined as unique strong solutions of: \[ dX_ t=f(X_ t,Y_ t,t)dt,\quad X(0)=X_ 0;\quad dY_ t=b_+(X_ t,Y_ t,t)dt+\sigma (t)dW_ t,\quad Y(0)=Y_ 0, \] (with smooth assumptions on f, \(b_+\), \(\sigma)\). We assume that the probability density p(x,y,t) is a positive smooth solution of the Fokker-Planck equation \[ \partial p/\partial t=-\nabla_ x(fp)-\nabla_ y(b_+p)+2^{-1}a\nabla_ yp,\quad p(x,y,0)=p_ 0(x,y) \] and satisfies \(\lim_{(\| x\| +\| y\|)\to \infty}p(x,y,t)=0\). Let us define \[ dW^-_ t=dW_ t+\sigma (t)\nabla_ yLog p(X_ t,Y_ t,t) dt. \] The process \(W^-_ t\) is a (backward) standard Wiener process adapted to the decreasing family \(F_ t(X,Y)=\sigma (X_{\tau},Y_{\tau}\); \(t\leq \tau \leq T)\). We consider the following control problem. Let \({\mathcal U}_ 0\) be the class of feedback controls u such that: \[ dX_ t=f(X_ t,Y_ t,t)dt,\quad X(t)=x;\quad dY_ t=u(X_ t,Y_ t,t)dt+\sigma (t)dW^-(t),\quad Y(t)=y, \] has a weak solution. We seek to minimize over \(u\in {\mathcal U}_ 0\) the cost functional \[ J(x,y,t,u)=E\{| \int^{t}_{0}L(X(s),Y(s),u(s),s)ds+S_ 0(X(0),Y(0))|,\quad X(t)=x,\quad Y(t)=y\}, \] where \(L(x,y,u,s)=(2\alpha)^{-1}\| b_+- u\|^ 2+\nabla_ xf+\nabla_ y-b_+\) and \(S_ 0(x,y)=-Log p_ 0(x,y).\) The author proves that \(S(x,y,t)=-Log p(x,y,t)\) is the value function of the stochastic optimal control problem. Moreover, an optimal feedback control is given by \[ u^*(x,y,t)=b_+(x,y,t)-a(t)\nabla_ yLog p(x,y,t)\quad (a(t)=\sigma (t)\sigma^*(t)). \] For non-equilibrium thermodynamical systems this provides a Hamilton-Jacobi-like theory, where the action is the entropy function. For the Ornstein-Uhlenbeck model of physical Brownian motion, the principle is related to a pathwise Newton law. For this latter model other pathwise results are also derived (in particular the stochastic Helmholtz free energy is shown to be a backward submartingale w.r.t. the natural filtration).