Analysis of inhomogeneous Markov reward models (Q1434438): Difference between revisions
From MaRDI portal
Latest revision as of 18:34, 6 June 2024
scientific article
Language | Label | Description | Also known as |
---|---|---|---|
English | Analysis of inhomogeneous Markov reward models |
scientific article |
Statements
Analysis of inhomogeneous Markov reward models (English)
0 references
4 August 2004
0 references
Let \(\{Z(t),\, t\geq0\}\) be an inhomogeneous continuous-time Markov chain (CTMC) on state space \(S=\{1,2,\dots,N\}\) with initial probability vector \(\pi\) and generator \({\mathbb Q}(t,w)\) depending both on time \(t\) and the level of accumulated reward \(w\) so that \({\mathbb Q}(t,w)\) is irreducible for \(t,w\geq0\). Whenever the CTMC stays in state \(i\) at time \(t\) and the level of accumulated reward is \(w\), the reward is accumulated at rate \(r_ i(t,w)\in[0,\infty)\). The amount of accumulated reward \(B(t)\) at time \(t\) is defined by \(dB(t)\over {dt}=r_ {Z(t)}(t,w)\), \(B(0)=0\). The authors provide forward partial differential equations describing the distributions of \(B(t)\) and of the completion time \(C(w):=\min\{t:B(t)\geq w\}\). On the basis of these descriptions, a set of ordinary differential equations describing the moments of these reward measures is obtained. This description allows the authors to give an effective numerical analysis of rather large inhomogeneous Markov reward models with rate and impulse reward accumulation. A numerical example is given to demonstrate an application of inhomogeneous Markov reward models in practice.
0 references
inhomogeneous Markov reward models
0 references
ordinary and partial differential equations
0 references
moments of reward measures
0 references
inhomogeneous continuous-time Markov chain
0 references
numerical example
0 references
0 references
0 references
0 references