Optimal control of molecular dynamics using Markov state models
type: Item
instance of: scholarly article
title: Optimal control of molecular dynamics using Markov state models
zbMATH Open document ID: 1262.49036
author: Christof Schütte
author: Stefanie Winkelmann
author: Carsten Hartmann
published in: Mathematical Programming. Series A. Series B
publication date: 2012-11-02T00:00:00Z
review text: In the article the authors ``propose an approach to solving the Hamilton-Jacobi-Bellman (HJB) equation for optimal control of stochastic molecular dynamics (MD) in high dimensions. The main idea is to first approximate the dominant modes of the molecular dynamics transfer operator by a low-dimensional, so-called Markov state model (MSM), and then solve the HJB for the MSM rather then the full MD. The type of optimal control problems that [they] consider here, and which appear relevant in molecular dynamics applications, belong to the class of ergodic stochastic control problems'' (from authors' introduction). From the abstract: ``A numerical scheme for solving high dimensional stochastic control problems [as described above] on an infinite time horizon \dots is outlined. The scheme rests on the interpretation of the corresponding HJB equation as a nonlinear eigenvalue problem that, using a logarithmic transformation, can be recast as a linear eigenvalue problem, for which the principal eigenvalue and its eigenfunction are sought.'' The authors give a proof of concept that the linear eigenvalue problem ``can be computed efficiently by approximating the underlying stochastic process with a coarse-grained Markov state model for the dominant metastable sets'' (from the abstract). A good feature of the proposed scheme is that the MSM can be sampled from MD simulation data using the uncontrolled dynamics. In conclusion the method is illustrated with ``two numerical examples, one of which involves the task of maximizing the population of \(\alpha\)-helices in an ensemble of small biomolecules (alanine dipeptide)'' (from the abstract).
Mathematics Subject Classification ID: 49M25
Mathematics Subject Classification ID: 74A25
Mathematics Subject Classification ID: 60J28
zbMATH DE Number: 6101842
zbMATH Keywords: optimal control
zbMATH Keywords: molecular dynamics
zbMATH Keywords: Markov state models
zbMATH Keywords: Hamilton-Jacobi-Bellman equation
zbMATH Keywords: ergodic control
MaRDI profile type: MaRDI publication profile
full work available at URL: s10107-012-0547-6
OpenAlex ID: W2137522587
cites work: Controlled Markov processes and viscosity solutions
cites work: Q2703816
cites work: Q3355178
cites work: Error bounds for monotone approximation schemes for parabolic Hamilton-Jacobi-Bellman equations
cites work: An efficient algorithm for Hamilton-Jacobi equations in high dimension
cites work: Risk-Sensitive Control on an Infinite Time Horizon
cites work: Asymptotic for the principal eigenvalue and eigenfunction of a nearly first-order operator with large potential
cites work: Q4225410
cites work: Constructing the equilibrium ensemble of folding pathways from short off-equilibrium simulations
cites work: Identification of almost invariant aggregates in reversible nearly uncoupled Markov chains
cites work: A direct approach to conformational dynamics based on hybrid Monte Carlo
cites work: Q4664503
cites work: Metastability and low lying spectra in reversible Markov chains
cites work: On the Approximation Quality of Markov State Models
cites work: Ergodic control of multidimensional diffusions. II: Adaptive control
cites work: An ergodic control problem arising from the principal eigenfunction of an elliptic operator
cites work: Phase transitions and metastability in Markovian and molecular systems
cites work: Asymptotic analysis of nonlinear stochastic risk-sensitive control and differential games
cites work: A minimum principle for the principal eigenvalue for second-order linear elliptic equations with natural boundary conditions
cites work: Risk-sensitivity, large deviations and stochastic control
cites work: Asymptotic evaluation of certain markov process expectations for large time, I
cites work: Q3681702
cites work: Fluctuation theorem for stochastic dynamics
cites work: A Gallavotti-Cohen-type symmetry in the large deviation functional for stochastic dynamics
cites work: Continuous-time stochastic control and optimization with financial applications
cites work: Long-term average cost control problems for continuous time Markov processes: A survey
DOI: 10.1007/S10107-012-0547-6
description: scientific article; zbMATH DE number 6101842