Monotonicity in a Markov Decision Process
Consider the Markov decision process with state space \(I=\{0,...,n\}\) and closed action set \(A\subseteq [0,\infty]\), having optimality equation \[ V_ i=\min_{a}\{a+c+\int^{i}_{j=0}p(i,j,a)V_ j\},\quad 1\leq i\leq n,\quad V_ 0=0, \] in which the transition probabilities are given by \[ p(i,j,a)=\left( \begin{matrix} i\\ j\end{matrix} \right)p^{aj}(1-p^ a)^{i-j}, \] 0\(<p<1\) and \(c>0\). The model can be interpreted in terms of collective sequential testing of n subjects. When \(A=[0,\infty]\) it is shown that \(V_ i\) is increasing and concave in i, and that the optimal actions \(a_ i\) are strictly increasing in i. However, when A is the set of positive integers, whilst the same properties hold for \(V_ i\), monotonicity of \(a_ i\) is more difficult to establish: a restriction on the values of the parameters p and c is given as a sufficient condition.
- Some Monotonicity Results for Partially Observed Markov Decision Processes
- Monotonic properties for a sequential decision problem with partial maintenance on a Markov process
- Monotone optimal policies in discounted Markov decision processes with transition probabilities independent of the current state: existence and approximation
- scientific article; zbMATH DE number 1420959
- Sensitivity of constrained Markov decision processes
- Monotone Case for an Extended Process
- Structural Properties of Stochastic Dynamic Programs
- How Does the Value Function of a Markov Decision Process Depend on the Transition Probabilities?
- Monotonic properties for a sequential decision problem with partial maintenance on a Markov process
- Monotonicity of minimizers in optimization problems with applications to Markov control processes
This page was built for publication: Monotonicity in a Markov Decision Process
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3790966)