Perturbation realization, potentials, and sensitivity analysis of Markov processes

DOI10.1109/9.633827MaRDI QIDQ4368697zbMATHOpenAlexFDO

Authors Cao, Xiren, Hanfu Chen

Publication date 29 June 1998

Published in IEEE Transactions on Automatic Control (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1109/9.633827

zbMATH Keywords

Poisson equationLyapunov equationperturbation analysisperformance potentialsperformance sensitivitiesdifferent initial statessample path of a Markov process

Mathematics Subject Classification ID

Stochastic systems and control (93E99) Sensitivity (robustness) (93B35) Sample path properties (60G17) Control/observation systems governed by functional relations other than differential equations (such as hybrid and switching systems) (93C30) Perturbations in control/observation systems (93C73)

Recommendations

• Single sample path-based sensitivity analysis of Markov processes using uniformization
• Perturbation analysis of inhomogeneous finite Markov chains
• scientific article; zbMATH DE number 744081
• scientific article; zbMATH DE number 1594525

Cited in

(46)

• Geometric ergodicity in a weighted Sobolev space
• Parameterized Markov decision process and its application to service rate control
• Single sample path-based optimization of Markov chains
• Importance and sensitivity analysis in dynamic reliability
• Sensitivity analysis of continuous time Bayesian network reliability models
• Event-based optimization approach for solving stochastic decision problems with probabilistic constraint
• Potentials based optimization with embedded Markov chain for stochastic constrained system
• Single sample path-based sensitivity analysis of Markov processes using uniformization
• Perturbation analysis for denumerable Markov chains with application to queueing models
• A unified approach to Markov decision problems and performance sensitivity analysis
• Finding optimal memoryless policies of POMDPs under the expected average reward criterion
• Perturbation analysis of Markov modulated fluid models
• Sensitivity-based nested partitions for solving finite-horizon Markov decision processes
• ASYMPTOTIC VARIANCE OF PASSAGE TIME ESTIMATORS IN MARKOV CHAINS
• Variance minimization of parameterized Markov decision processes
• The control of a two-level Markov decision process by time aggregation
• On-line policy gradient estimation with multi-step sampling
• Observable augmented systems for sensitivity analysis of Markov and semi-Markov processes
• Power and delay optimisation in multi-hop wireless networks
• Natural actor-critic algorithms
• Service rate control of closed Jackson networks from game theoretic perspective
• Efficient sensitivity analysis for parametric robust Markov chains
• Temporal difference-based policy iteration for optimal control of stochastic systems
• Policy iteration for customer-average performance optimization of closed queueing systems
• Robust queueing theory: an initial study using imprecise probabilities
• From differential to difference importance measures for Markov reliability models
• A sensitivity formula for risk-sensitive cost and the actor-critic algorithm
• Modeling and optimization of M/G/1-type queueing networks: an efficient sensitivity analysis approach
• Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces
• A unified approach to Markov decision problems and performance sensitivity analysis with discounted and average criteria: multichain cases
• Average sample-path optimality for continuous-time Markov decision processes in Polish spaces
• Coupling based estimation approaches for the average reward performance potential in Markov chains
• General solution of the Poisson equation for quasi-birth-and-death processes
• Policy gradient approach of event-based optimization and its online implementation
• Bias optimality and strong \(n\) \((n= -1,0)\) discount optimality for Markov decision processes
• Error bounds of optimization algorithms for semi-Markov decision processes
• Performance optimization of semi-Markov decision processes with discounted-cost criteria
• A unified approach to time-aggregated Markov decision processes
• Basic ideas for event-based optimization of Markov systems
• Optimization of Markov decision processes under the variance criterion
• Sample-path optimality and variance-maximization for Markov decision processes
• Performance optimization of queueing systems with perturbation realization
• An overview for Markov decision processes in queues and networks
• A tutorial on event-based optimization -- a new optimization framework
• A Jackson network model and threshold policy for joint optimization of energy and delay in multi-hop wireless networks
• Strong \(n\)-discount and finite-horizon optimality for continuous-time Markov decision processes

This page was built for publication: Perturbation realization, potentials, and sensitivity analysis of Markov processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4368697)