Forward sensitivity analysis for contracting stochastic systems

DOI10.1017/APR.2018.6MaRDI QIDQ5214994zbMATH OpenOpenAlexWikidataFDO

Authors Thomas Flynn

Publication date 5 February 2020

Published in Advances in Applied Probability (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1610.09456

zbMATH Keywords

contractionderivative estimationiterated function systemMarkov chain

Mathematics Subject Classification ID

Discrete-time Markov processes on general state spaces (60J05) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Monte Carlo methods (65C05) Markov and semi-Markov decision processes (90C40)

Abstract: In this work we investigate gradient estimation for a class of contracting stochastic systems on a continuous state space. We find conditions on the one-step transitions, namely differentiability and contraction in a Wasserstein distance, that guarantee differentiability of stationary costs. Then we show how to estimate the derivatives, deriving an estimator that can be seen as a generalization of the forward sensitivity analysis method used in deterministic systems. We apply the results to examples, including a neural network model.

Recommendations

• scientific article; zbMATH DE number 4038959
• Generalized estimates for performance sensitivities of stochastic systems
• A Monte Carlo Method for Sensitivity Analysis and Parametric Optimization of Nonlinear Stochastic Systems: The Ergodic Case
• Convergence Rates of Finite-Difference Sensitivity Estimates for Stochastic Systems
• Convergence of parameter sensitivity estimates in a stochastic experiment

Cites work

• A course in metric geometry
• A Differential Lyapunov Framework for Contraction Analysis
• A survey of average contractive iterated function systems
• Characterization and sufficient conditions for normed ergodicity of Markov chains
• Contraction theory on Riemannian manifolds
• Curvature, concentration and error estimates for Markov chain Monte Carlo
• Dynamics and architecture for neural computation
• Estimation of the derivative of a stationary measure with respect to a control parameter
• Evaluating Derivatives
• Gradient estimates for the performance of Markov chains and discrete event processes
• scientific article; zbMATH DE number 1005357 (Why is no real title available?)
• Learning representations by back-propagating errors
• Locally contractive iterated function systems
• Measure-Valued Differentiation for Stationary Markov Chains
• On contraction analysis for non-linear systems
• Optimal Transport
• Spectral gaps in Wasserstein distances and the 2D stochastic Navier-Stokes equations
• Taylor series expansions for stationary Markov chains
• Timescale separation in recurrent neural networks

Cited in

(3)

• A simultaneous perturbation weak derivative estimator for stochastic neural networks
• Asymptotic behavior for Markovian iterated function systems
• Sensitivities in models with backward dynamics

This page was built for publication: Forward sensitivity analysis for contracting stochastic systems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5214994)