Likelihood ratio gradient estimation for steady-state parameters

DOI10.1287/STSY.2018.0023zbMATH Open1435.60051arXiv1707.02659OpenAlexW2963160669WikidataQ127733366 ScholiaQ127733366MaRDI QIDQ5113892FDOQ5113892

Authors: Mariana Olvera-Cravioto, Peter W. Glynn 

Publication date: 18 June 2020

Published in: Stochastic Systems (Search for Journal in Brave)

Abstract: We consider a discrete-time Markov chain $$ on a general state-space $sfX$, whose transition probabilities are parameterized by a real-valued vector $$. Under the assumption that $$ is geometrically ergodic with corresponding stationary distribution $$, we are interested in estimating the gradient $$ of the steady-state expectation ${\displaystyle }$alpha(�oldsymbol{ heta}) = pi( �oldsymbol{ heta}) f.${\displaystyle }$ To this end, we first give sufficient conditions for the differentiability of $$ and for the calculation of its gradient via a sequence of finite horizon expectations. We then propose two different likelihood ratio estimators and analyze their limiting behavior.

Full work available at URL: https://arxiv.org/abs/1707.02659

Recommendations

• Likelihood ratio gradient estimation for stochastic recursions
• scientific article; zbMATH DE number 744084
• Gradient estimates for the performance of Markov chains and discrete event processes
• Performance gradient estimation for the very large finite Markov chains
• scientific article; zbMATH DE number 4038959

zbMATH Keywords

simulationlikelihood ratiogradient estimation

Mathematics Subject Classification ID

Monte Carlo methods (65C05) Functional limit theorems; invariance principles (60F17) Discrete-time Markov processes on general state spaces (60J05)

Cites Work

• Markov chains and stochastic stability
• Title not available (Why is that?)
• Weak limit theorems for stochastic integrals and stochastic differential equations
• A Liapounov bound for solutions of the Poisson equation
• A Unified View of the IPA, SF, and LR Gradient Estimation Techniques
• Note: On the Interchange of Derivative and Expectation for Likelihood Ratio Derivative Estimators
• Proofs of the martingale FCLT
• Handbooks in operations research and management science: Simulation
• Likelihood ratio gradient estimation for stochastic recursions
• Measure-valued differentiation for the cycle cost performance in the G/G/1 queue in the presence of heavy-tailed distributions
• Measure-Valued Differentiation for Stationary Markov Chains
• Title not available (Why is that?)
• Efficiency of Multivariate Control Variates in Monte Carlo Simulation

Cited In (9)

• Convergence of the likelihood ratio method for linear response of non-equilibrium stationary states
• Perturbation analysis via coupling
• Likelihood ratio gradient estimation for Meixner distribution and Lévy processes
• Title not available (Why is that?)
• Variance comparison between infinitesimal perturbation analysis and likelihood ratio estimators to stochastic gradient
• Likelihood ratio gradient estimation for stochastic recursions
• Gradient of the log-likelihood ratio for infinite-dimensional stochastic systems
• Measure-Valued Differentiation for Stationary Markov Chains
• Gradient estimates for the performance of Markov chains and discrete event processes

This page was built for publication: Likelihood ratio gradient estimation for steady-state parameters

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5113892)