Optimal control of infinite-dimensional piecewise deterministic Markov processes and application to the control of neuronal dynamics via optogenetics

DOI10.3934/NHM.2017019zbMATH Open1376.93118arXiv1607.05574OpenAlexW2962802695MaRDI QIDQ1680943FDOQ1680943

Authors: Vincent Renault, Michèle Thieullen, Emmanuel Trélat 

Publication date: 17 November 2017

Published in: Networks and Heterogeneous Media (Search for Journal in Brave)

Abstract: In this paper we define an infinite-dimensional controlled piecewise deterministic Markov process (PDMP) and we study an optimal control problem with finite time horizon and unbounded cost. This process is a coupling between a continuous time Markov Chain and a set of semilinear parabolic partial differential equations, both processes depending on the control. We apply dynamic programming to the embedded Markov decision process to obtain existence of optimal relaxed controls and we give some sufficient conditions ensuring the existence of an optimal ordinary control. This study, which constitutes an extension of controlled PDMPs to infinite dimension, is motivated by the control that provides Optogenetics on neuron models such as the Hodgkin-Huxley model. We define an infinite-dimensional controlled Hodgkin-Huxley model as an infinite-dimensional controlled piecewise deterministic Markov process and apply the previous results to prove the existence of optimal ordinary controls for a tracking problem.

Full work available at URL: https://arxiv.org/abs/1607.05574

Recommendations

• Optimal control of infinite-dimensional piecewise deterministic Markov processes: a BSDE approach. Application to the control of an excitable cell membrane
• Publication:3032183
• Optimal impulsive control of piecewise deterministic Markov processes
• Optimal control of piecewise deterministic Markov processes
• Optimal control of piecewise deterministic markov process

zbMATH Keywords

dynamic programmingoptimal controlsemilinear parabolic equationsMarkov decision processespiecewise deterministic Markov processesoptogenetics

Mathematics Subject Classification ID

Dynamic programming (90C39) Neural biology (92C20) Continuous-time Markov processes on general state spaces (60J25) Semilinear parabolic equations (35K58) Dynamic programming in optimal control and differential games (49L20) Kinetics in biochemical problems (pharmacokinetics, enzyme kinetics, etc.) (92C45) Optimal stochastic control (93E20)

Cites Work

• Title not available (Why is that?)
• Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales
• Title not available (Why is that?)
• Title not available (Why is that?)
• Optimal control of piecewise deterministic markov process
• One-Parameter Semigroups for Linear Evolution Equations
• Title not available (Why is that?)
• Convergence of stochastic gene networks to hybrid piecewise deterministic processes
• Relaxed variational problems
• Stochastic optimal control. The discrete time case
• Stability and Ergodicity of Piecewise Deterministic Markov Processes
• Title not available (Why is that?)
• Title not available (Why is that?)
• An exact stochastic hybrid model of excitable membranes including spatio-temporal evolution
• The emergence of the deterministic Hodgkin-Huxley equations as a limit from the underlying stochastic ion-channel mechanism
• Quantization
• Markov decision processes with applications to finance.
• Averaging for a Fully Coupled Piecewise-Deterministic Markov Process in Infinite Dimensions
• Numerical Methods for Simulation and Optimization of Piecewise Deterministic Markov Processes
• A Markovian analysis of additive-increase multiplicative-decrease algorithms
• Limit theorems for infinite-dimensional piecewise deterministic Markov processes. Applications to stochastic excitable membrane models
• Algebraic invariance conditions in the study of approximate (null-)controllability of Markov switch processes
• A general denseness result for relaxed control theory
• Optimal control of systems governed by a class of nonlinear evolution equations in a reflexive Banach space
• Properties of Relaxed Trajectories for a Class of Nonlinear Evolution Equations on a Banach Space
• Necessary conditions for minimum in relaxed variational problems
• Title not available (Why is that?)
• Properties of the Relaxed Trajectories of Evolution Equations and Optimal Control
• MDP algorithms for portfolio optimization problems in pure jump markets
• Piecewise deterministic Markov control processes with feedback controls and unbounded costs
• On Reducing a Jump Controllable Markov Model to a Model with Discrete Time
• Singular perturbation for the discounted continuous control of piecewise deterministic Markov processes
• SPECTRAL OPERATORS IN A DIRECT SUM OF HILBERT SPACES
• Markov decision processes with their applications
• Minimal time spiking in various ChR2-controlled neuron models
• Title not available (Why is that?)
• Numerical methods for the exit time of a piecewise-deterministic Markov process
• Properties of Relaxed Trajectories of Evolution Equations and Optimal Control
• Optimal stopping with continuous control of piecewise deterministic Markov processes

Cited In (5)

• Optimal control of infinite-dimensional piecewise deterministic Markov processes: a BSDE approach. Application to the control of an excitable cell membrane
• Stochastic Optimal Control as a Theory of Brain-Machine Interface Operation
• Nonparametric estimation of jump rates for a specific class of piecewise deterministic Markov processes
• Stochastic filtering and optimal control of pure jump Markov processes with noise-free partial observation
• Asymptotic analysis and optimal control of an integro-differential system modelling healthy and cancer cells exposed to chemotherapy

This page was built for publication: Optimal control of infinite-dimensional piecewise deterministic Markov processes and application to the control of neuronal dynamics via optogenetics

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1680943)