Stochastic control and nonequilibrium thermodynamical systems (Q1115004)

From MaRDI portal
scientific article
Language Label Description Also known as
English
Stochastic control and nonequilibrium thermodynamical systems
scientific article

    Statements

    Stochastic control and nonequilibrium thermodynamical systems (English)
    0 references
    0 references
    1989
    0 references
    Let (\(\Omega\),\(\underset {=} F,P)\) be a probability space carrying a standard n-dimensional Wiener process \(W=(W_ t\); \(0\leq t\leq T)\) and \(X_ 0\), \(Y_ 0\) of dimension m and n, respectively, independent of W. We consider the stochastic processes \(X=(X(t)\); \(0\leq t\leq T)\) and \(Y=(Y_ t\); \(0\leq t\leq T)\) defined as unique strong solutions of: \[ dX_ t=f(X_ t,Y_ t,t)dt,\quad X(0)=X_ 0;\quad dY_ t=b_+(X_ t,Y_ t,t)dt+\sigma (t)dW_ t,\quad Y(0)=Y_ 0, \] (with smooth assumptions on f, \(b_+\), \(\sigma)\). We assume that the probability density p(x,y,t) is a positive smooth solution of the Fokker-Planck equation \[ \partial p/\partial t=-\nabla_ x(fp)-\nabla_ y(b_+p)+2^{-1}a\nabla_ yp,\quad p(x,y,0)=p_ 0(x,y) \] and satisfies \(\lim_{(\| x\| +\| y\|)\to \infty}p(x,y,t)=0\). Let us define \[ dW^-_ t=dW_ t+\sigma (t)\nabla_ yLog p(X_ t,Y_ t,t) dt. \] The process \(W^-_ t\) is a (backward) standard Wiener process adapted to the decreasing family \(F_ t(X,Y)=\sigma (X_{\tau},Y_{\tau}\); \(t\leq \tau \leq T)\). We consider the following control problem. Let \({\mathcal U}_ 0\) be the class of feedback controls u such that: \[ dX_ t=f(X_ t,Y_ t,t)dt,\quad X(t)=x;\quad dY_ t=u(X_ t,Y_ t,t)dt+\sigma (t)dW^-(t),\quad Y(t)=y, \] has a weak solution. We seek to minimize over \(u\in {\mathcal U}_ 0\) the cost functional \[ J(x,y,t,u)=E\{| \int^{t}_{0}L(X(s),Y(s),u(s),s)ds+S_ 0(X(0),Y(0))|,\quad X(t)=x,\quad Y(t)=y\}, \] where \(L(x,y,u,s)=(2\alpha)^{-1}\| b_+- u\|^ 2+\nabla_ xf+\nabla_ y-b_+\) and \(S_ 0(x,y)=-Log p_ 0(x,y).\) The author proves that \(S(x,y,t)=-Log p(x,y,t)\) is the value function of the stochastic optimal control problem. Moreover, an optimal feedback control is given by \[ u^*(x,y,t)=b_+(x,y,t)-a(t)\nabla_ yLog p(x,y,t)\quad (a(t)=\sigma (t)\sigma^*(t)). \] For non-equilibrium thermodynamical systems this provides a Hamilton-Jacobi-like theory, where the action is the entropy function. For the Ornstein-Uhlenbeck model of physical Brownian motion, the principle is related to a pathwise Newton law. For this latter model other pathwise results are also derived (in particular the stochastic Helmholtz free energy is shown to be a backward submartingale w.r.t. the natural filtration).
    0 references
    Hamilton-Jacobi theory
    0 references
    nonequilibrium thermodynamical system
    0 references
    unique strong solutions
    0 references
    Fokker-Planck equation
    0 references

    Identifiers

    0 references
    0 references
    0 references
    0 references
    0 references
    0 references