The stochastic reach-avoid problem and set characterization for diffusions

DOI10.1016/J.AUTOMATICA.2016.03.016zbMATH Open1339.93125arXiv1202.4375OpenAlexW1891592265MaRDI QIDQ290823FDOQ290823

Authors: Peyman Mohajerin Esfahani, Debasish Chatterjee, John Lygeros 

Publication date: 3 June 2016

Published in: Automatica (Search for Journal in Brave)

Abstract: In this article we approach a class of stochastic reachability problems with state constraints from an optimal control perspective. Preceding approaches to solving these reachability problems are either confined to the deterministic setting or address almost-sure stochastic requirements. In contrast, we propose a methodology to tackle problems with less stringent requirements than almost sure. To this end, we first establish a connection between two distinct stochastic reach-avoid problems and three classes of stochastic optimal control problems involving discontinuous payoff functions. Subsequently, we focus on solutions of one of the classes of stochastic optimal control problems---the exit-time problem, which solves both the two reach-avoid problems mentioned above. We then derive a weak version of a dynamic programming principle (DPP) for the corresponding value function; in this direction our contribution compared to the existing literature is to develop techniques that admit discontinuous payoff functions. Moreover, based on our DPP, we provide an alternative characterization of the value function as a solution of a partial differential equation in the sense of discontinuous viscosity solutions, along with boundary conditions both in Dirichlet and viscosity senses. Theoretical justifications are also discussed to pave the way for deployment of off-the-shelf PDE solvers for numerical computations. Finally, we validate the performance of the proposed framework on the stochastic Zermelo navigation problem.

Full work available at URL: https://arxiv.org/abs/1202.4375

Recommendations

• Probabilistic Reach-Avoid Computation for Partially Degenerate Stochastic Processes
• scientific article; zbMATH DE number 2134070
• A stochastic reach-avoid problem with random obstacles
• scientific article; zbMATH DE number 4197829
• Diffusive limit approximation of pure-jump optimal stochastic control problems
• On diffusions that cannot escape from a convex set
• Sets of accessibility for controlled stochastic differential equations
• scientific article; zbMATH DE number 4183323
• A stochastic approximation method for reachability computations
• Setwise convergence of solution measures of stochastic differential equations

zbMATH Keywords

dynamic programmingpartial differential equationsreachabilitystochastic controldiscontinuous viscosity solutions

Mathematics Subject Classification ID

Dynamic programming in optimal control and differential games (49L20) Viscosity solutions to Hamilton-Jacobi equations in optimal control and differential games (49L25) Attainable sets, reachability (93B03) Control/observation systems governed by partial differential equations (93C20) Optimal stochastic control (93E20)

Cites Work

• Hybrid Systems: Computation and Control
• Foundations of Modern Probability
• On reachability and minimum cost optimal control
• Controlled Markov processes and viscosity solutions
• Weak dynamic programming principle for viscosity solutions
• Stochastic target problems with controlled loss
• Title not available (Why is that?)
• Title not available (Why is that?)
• User’s guide to viscosity solutions of second order partial differential equations
• Existence of stochastic control under state constraints
• Title not available (Why is that?)
• Dynamic programming for stochastic target problems and geometric flows
• Optimal stochastic control, stochastic target problems, and backward SDE.
• Title not available (Why is that?)
• Title not available (Why is that?)
• Controlled diffusion processes
• Controlled diffusion processes. Translated by A. B. Aries
• Diffusions and Elliptic Operators
• On the Existence of Pure and Mixed Strategy Nash Equilibria in Discontinuous Games
• Title not available (Why is that?)
• Optimal times for constrained nonlinear control problems without local controllability
• Stochastic invariance for differential inclusions
• A geometric characterization of viable sets for controlled degenerate diffusions
• Invariance of stochastic control systems with deterministic arguments
• Stochastic viability for compact sets in terms of the distance function
• The stochastic reach-avoid problem and set characterization for diffusions
• Title not available (Why is that?)
• The viability theorem for stochastic differential inclusions²
• Stochastic Target Problems, Dynamic Programming, and Viscosity Solutions
• Title not available (Why is that?)
• A Differential Game with Two Players and One Target
• Maximizing the probability of attaining a target prior to extinction

Cited In (10)

• Recursively feasible stochastic model predictive control using indirect feedback
• The stochastic reach-avoid problem and set characterization for diffusions
• Applications of stochastic modeling in air traffic management: methods, challenges and opportunities for solving air traffic problems under uncertainty
• A neural network-based policy iteration algorithm with global \(H^2\)-superlinear convergence for stochastic games on domains
• Time-extremal navigation in arbitrary winds on conformally flat Riemannian manifolds
• Automated verification and synthesis of stochastic hybrid systems: a survey
• Randers metrics based on deformations by gradient winds
• Hybrid controller synthesis for nonlinear systems subject to reach-avoid constraints
• A stochastic reach-avoid problem with random obstacles
• From infinite to finite programs: explicit error bounds with applications to approximate dynamic programming

Uses Software

• ToolboxLS

This page was built for publication: The stochastic reach-avoid problem and set characterization for diffusions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q290823)