Optimal consumption and exploration: A case study in piecewise-deterministic Markov modelling (Q1808218)

The optimal decision problem for the consumption and exploration of resources are studied in the framework of Piecewise-Deterministic Markov Process (PDP) optimal control theory [cf. \textit{M. H. A. Davis}, Markov models and optimization, Monographs on Statistics and Applied Probability 49, Chapman-Hall (1993; Zbl 0780.60002), in particular, Chap. 4]. It is assumed that the inter-arrival random times \((\{T_i\})\) of the new resources are exponentially distributed, that the amount of resource found at each \(T_i\) is of a known distribution and that the exploration costs are paid for from the value of current reserves. Under some general but mild assumptions, this problem is transformed into a deterministic control problem, the value function of the PDP optimal control is solved as the unique viscosity solution of the associated Hamilton-Jacobi-Bellman (HJB) equation. Simulation results by an iterative method for solving the value function using the HJB equation are well presented under guaranteed convergence.