Stochastic impulse control of nonsmooth dynamics with partial observation and execution delay: application to an environmental restoration problem

DOI10.1002/OCA.2723MaRDI QIDQ5159840zbMATH OpenOpenAlexWikidataFDO

Authors Hidekazu Yoshioka, Yuta Yaegashi

Publication date 28 October 2021

Published in Optimal Control Applications \& Methods (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2006.16034

zbMATH Keywords

Fokker-Planck equationHamilton-Jacobi-Bellman equationnonsmooth dynamicsenvironmental restorationstochastic impulse control with random observations and execution delay

Mathematics Subject Classification ID

Fokker-Planck equations (35Q84) Control/observation systems governed by partial differential equations (93C20) Optimal stochastic control (93E20) Impulsive control/observation systems (93C27)

Abstract: Non-smooth dynamics driven by stochastic disturbance arise in a wide variety of engineering problems. Impulsive interventions are often employed to control stochastic systems; however, the modeling and analysis subject to execution delay have been less explored. In addition, continuously receiving information of the dynamics is not always possible. In this paper, with an application to an environmental restoration problem, a continuous-time stochastic impulse control problem subject to execution delay under discrete and random observations is newly formulated and analyzed. The dynamics have a non-smooth coefficient modulated by a Markov chain, and eventually attain an undesirable state like a depletion due to the non-smoothness. The goal of the control problem is to find the most cost-efficient policy that can prevent the dynamics from attaining the undesirable state. We demonstrate that finding the optimal policy reduces to solving a non-standard system of degenerate elliptic equations, the optimality equation, which is rigorously and analytically verified in a simplified case. The associated Fokker-Planck equation for the controlled dynamics is derived and solved explicitly as well. The model is finally applied to numerical computation of a recent river environmental restoration problem. The optimality and Fokker-Planck equations are successfully computed, and the optimal policy and the probability density functions are numerically obtained. The impacts of execution delay are discussed to deeper analyze the model.

Recommendations

• A hybrid stochastic river environmental restoration modeling with discrete and costly observations
• Optimal stochastic impulse control with random coefficients and execution delay
• Infinite horizon stochastic impulse control with delay and random coefficients
• Analysis and computation of an optimality equation arising in an impulse control problem with discrete and costly observations
• Optimal implementation of environmental improvement policy with implementation costs

Cited in

(6)

• Time‐average stochastic control based on a singular local Lévy model for environmental project planning under habit formation
• Analysis and computation of an optimality equation arising in an impulse control problem with discrete and costly observations
• A non-local Fokker-Planck equation with application to probabilistic evaluation of sediment replenishment projects
• Modeling and computation of cost-constrained adaptive environmental management with discrete observation and intervention
• Optimal sizing of the sediment replenishment capacity based on robust ergodic control of subordinator-driven dynamics
• Hamilton-Jacobi-Bellman-Isaacs equation for rational inattention in the long-run management of river environments under uncertainty

This page was built for publication: Stochastic impulse control of nonsmooth dynamics with partial observation and execution delay: application to an environmental restoration problem

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5159840)