Adaptive control of constrained Markov chains (Q3985342)

From MaRDI portal

!

This is the item page for this Wikibase entity, intended for internal use and editing purposes. Please use the normal view instead:

scientific article; zbMATH DE number 30294
Language Label Description Also known as
default for all languages
No label defined
    English
    Adaptive control of constrained Markov chains
    scientific article; zbMATH DE number 30294

      Statements

      Adaptive control of constrained Markov chains (English)
      0 references
      0 references
      0 references
      27 June 1992
      0 references
      adaptive control of finite-state Markov chains
      0 references
      competing queues
      0 references
      countable state space
      0 references
      We consider the adaptive control of finite-state Markov chains, where the optimal performance is characterized through the minimization of a long- run average cost functional, subject to constraints on several other such functionals. In contrast with the unconstrained problem, applying the control which is optimal for the current parameter estimates (``certainty equivalence'' control) may not yield optimal performance. This problem is related to the fact that a feasible set of a linear program is not, in general, continuous in the parameters. Under mild structural and feasibility conditions, we exhibit two explicit adaptive control policies for the case where the transition probabilities are unknown, which are optimal under the constrained optimization criterion. These policies rely on a powerful estimation scheme of ``probing'', which provides consistent estimators for the transition probabilities. This scheme is of independent interest, as it provides strong consistency under a large number of adaptive schemes, and is independent of any ``identifiability'' conditions. As an application, we derive an optimal adaptive policy for a system of \(K\) competing queues with countable state space, and for which the constrained criteria arise naturally in the context of communication networks.
      0 references

      Identifiers

      0 references
      0 references
      0 references
      0 references
      0 references
      0 references
      0 references
      0 references