Multiplicative Markov Decision Chains (Q3320154)
From MaRDI portal
!
This is the item page for this Wikibase entity, intended for internal use and editing purposes. Please use the normal view instead:
scientific article; zbMATH DE number 3850856
| Language | Label | Description | Also known as |
|---|---|---|---|
| default for all languages | No label defined |
||
| English | Multiplicative Markov Decision Chains |
scientific article; zbMATH DE number 3850856 |
Statements
Multiplicative Markov Decision Chains (English)
0 references
1984
0 references
multiplicative Markov decision chains
0 references
constructive proof
0 references
existence of optimal policies
0 references
nonnilpotent transition matrix
0 references
policy improvement algorithm
0 references
0.8512110710144043
0 references
0.830146074295044
0 references
0.8204348087310791
0 references
This paper considers multiplicative Markov decision chains in which the transition matrices are nonnegative, but not necessarily substochastic, and the associated reward gained by the end of period \(N+1\) is the product of the one-period transition matrices in periods 1,...,N times a positive terminal reward. Previous treatments of this model restricted attention to stationary policies and assumed that all transition matrices are irreducible and aperiodic. They also used a ''first term'' optimality criterion, namely maximizing the spectral radius of the associated transition matrix. In this paper a constructive proof is given for the existence of optimal policies among all polices under new cumulative average optimality criteria which are more sensitive than the maximization of the spectral radius. The algorithm for finding an optimal policy, first searches for a stationary policy with a nonnilpotent transition matrix, provided such a rule exists. Otherwise, the method still finds an optimal policy; though in this case the set of optimal policies usually does not contain a stationary policy! If a stationary policy with a nonnilpotent transition matrix exists, then a policy improvement algorithm which finds a stationary optimal policy is followed.
0 references