Remarks on the existence of solutions to the average cost optimality equation in Markov decision processes (Q1814435): Difference between revisions
From MaRDI portal
Set OpenAlex properties. |
Created claim: Wikidata QID (P12): Q60167570, #quickstatements; #temporary_batch_1712190744730 |
||
Property / Wikidata QID | |||
Property / Wikidata QID: Q60167570 / rank | |||
Normal rank |
Revision as of 03:02, 4 April 2024
scientific article
Language | Label | Description | Also known as |
---|---|---|---|
English | Remarks on the existence of solutions to the average cost optimality equation in Markov decision processes |
scientific article |
Statements
Remarks on the existence of solutions to the average cost optimality equation in Markov decision processes (English)
0 references
25 June 1992
0 references
Considering Markov decision processes with Borel state and action spaces, the paper deals with the `` average cost optimality equation'' and presents necessary conditions for the existence of a bounded solution to this equation. Roughly spoken, the long-run expected average cost incurred by policy \(\mu\) is given by \[ J(x,\mu):=\limsup_{n\to\infty}{1\over n+1} E^ \mu_ x\left[ \sum_{t=0}^ n c(X_ t,U_ t)\right] \] for state \(X\), control \(U\), cost function \(c\) and initial state \(x\). The interest in finding conditions that guarantee solutions to the average cost optimality equation derives from a result showing that a bounded solution of that equation leads to, e.g., optimal stationary policies for the decision process. The objective of the paper is to exhibit some necessary conditions that complement known sufficient conditions. The authors stress the fact that from their results it can be appreciated more clearly how restrictive it is to require bounded solutions to the average cost optimality equation, which in turn motivates further studies dealing with unbounded solutions.
0 references
Markov decision processes
0 references
average cost optimality equation
0 references