Adaptive control of constrained Markov chains (Q3985342)
From MaRDI portal
!
This is the item page for this Wikibase entity, intended for internal use and editing purposes. Please use the normal view instead:
scientific article; zbMATH DE number 30294
| Language | Label | Description | Also known as |
|---|---|---|---|
| default for all languages | No label defined |
||
| English | Adaptive control of constrained Markov chains |
scientific article; zbMATH DE number 30294 |
Statements
Adaptive control of constrained Markov chains (English)
0 references
27 June 1992
0 references
adaptive control of finite-state Markov chains
0 references
competing queues
0 references
countable state space
0 references
0.9129951596260072
0 references
0.8669871687889099
0 references
0.845093846321106
0 references
0.8430277705192566
0 references
We consider the adaptive control of finite-state Markov chains, where the optimal performance is characterized through the minimization of a long- run average cost functional, subject to constraints on several other such functionals. In contrast with the unconstrained problem, applying the control which is optimal for the current parameter estimates (``certainty equivalence'' control) may not yield optimal performance. This problem is related to the fact that a feasible set of a linear program is not, in general, continuous in the parameters. Under mild structural and feasibility conditions, we exhibit two explicit adaptive control policies for the case where the transition probabilities are unknown, which are optimal under the constrained optimization criterion. These policies rely on a powerful estimation scheme of ``probing'', which provides consistent estimators for the transition probabilities. This scheme is of independent interest, as it provides strong consistency under a large number of adaptive schemes, and is independent of any ``identifiability'' conditions. As an application, we derive an optimal adaptive policy for a system of \(K\) competing queues with countable state space, and for which the constrained criteria arise naturally in the context of communication networks.
0 references