Likelihood ratio gradient estimation for steady-state parameters

DOI10.1287/STSY.2018.0023MaRDI QIDQ5113892zbMATH OpenOpenAlexWikidataFDO

Authors Mariana Olvera-Cravioto, Peter W. Glynn

Publication date 18 June 2020

Published in Stochastic Systems (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/1707.02659

zbMATH Keywords

simulationlikelihood ratiogradient estimation

Mathematics Subject Classification ID

Monte Carlo methods (65C05) Functional limit theorems; invariance principles (60F17) Discrete-time Markov processes on general state spaces (60J05)

Abstract: We consider a discrete-time Markov chain $$ on a general state-space $sfX$, whose transition probabilities are parameterized by a real-valued vector $$. Under the assumption that $$ is geometrically ergodic with corresponding stationary distribution $$, we are interested in estimating the gradient $$ of the steady-state expectation ${\displaystyle }$alpha(�oldsymbol{ heta}) = pi( �oldsymbol{ heta}) f.${\displaystyle }$ To this end, we first give sufficient conditions for the differentiability of $$ and for the calculation of its gradient via a sequence of finite horizon expectations. We then propose two different likelihood ratio estimators and analyze their limiting behavior.

Recommendations

• Likelihood ratio gradient estimation for stochastic recursions
• scientific article; zbMATH DE number 744084
• Gradient estimates for the performance of Markov chains and discrete event processes
• Performance gradient estimation for the very large finite Markov chains
• scientific article; zbMATH DE number 4038959

Cites work

• scientific article; zbMATH DE number 3951715 (Why is no real title available?)
• scientific article; zbMATH DE number 50675 (Why is no real title available?)
• A Liapounov bound for solutions of the Poisson equation
• A Unified View of the IPA, SF, and LR Gradient Estimation Techniques
• Efficiency of Multivariate Control Variates in Monte Carlo Simulation
• Handbooks in operations research and management science: Simulation
• Likelihood ratio gradient estimation for stochastic recursions
• Markov chains and stochastic stability
• Measure-Valued Differentiation for Stationary Markov Chains
• Measure-valued differentiation for the cycle cost performance in the G/G/1 queue in the presence of heavy-tailed distributions
• Note: On the Interchange of Derivative and Expectation for Likelihood Ratio Derivative Estimators
• Proofs of the martingale FCLT
• Weak limit theorems for stochastic integrals and stochastic differential equations

Cited in

(11)

• Convergence rates for a class of estimators based on Stein's method
• Convergence of the likelihood ratio method for linear response of non-equilibrium stationary states
• Perturbation analysis via coupling
• Likelihood ratio gradient estimation for Meixner distribution and Lévy processes
• scientific article; zbMATH DE number 4038959 (Why is no real title available?)
• Steady-state sensitivity analysis of continuous time Markov chains
• Variance comparison between infinitesimal perturbation analysis and likelihood ratio estimators to stochastic gradient
• Likelihood ratio gradient estimation for stochastic recursions
• Gradient of the log-likelihood ratio for infinite-dimensional stochastic systems
• Gradient estimates for the performance of Markov chains and discrete event processes
• Measure-Valued Differentiation for Stationary Markov Chains

This page was built for publication: Likelihood ratio gradient estimation for steady-state parameters

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5113892)