Multiaction learning automata possessing ergodicity of the mean
type: Item
instance of: scholarly article
title: Multiaction learning automata possessing ergodicity of the mean
zbMATH Open document ID: 0581.68047
DOI: 10.1016/0020-0255(85)90049-0
author: M. A. L. Thathachar
author: B. John Oommen
published in: Information Sciences
publication date: 1985-01-01T00:00:00Z
review text: Multiaction learning automata which update their action probabilities on the basis of the responses they get from an environment are considered in this paper. The automata update the probabilities according to whether the environment responds with a reward or a penalty. Learning automata are said to possess ergodicity of the mean if the mean action probability is the state probability (or unconditional probability) of an ergodic Markov chain. In an earlier paper [IEEE Trans. Syst. Man. Cybern. SMC-13, 1143-1148 (1983; Zbl 0537.68052)] we considered the problem of a two- action learning automaton being ergodic in the mean (EM). The family of such automata was characterized completely by proving the necessary and sufficient conditions for automata to be EM. In this paper, we generalize our earlier results [loc. cit.] and obtain necessary and sufficient conditions for the multiaction learning automaton to be EM. These conditions involve two families of probability updating functions. It is shown that for the automaton to be EM the two families must be linearly dependent. The vector defining the linear dependence is the only vector parameter which controls the rate of convergence of the automaton. Further, the technique for reducing the variance of the limiting distribution is discussed. Just as in the two-action case, it is shown that the set of absolutely expedient schemes and the set of schemes which possess ergodicity of the mean are mutually disjoint.
Mathematics Subject Classification ID: 68T05
zbMATH DE Number: 3930355
zbMATH Keywords: mean action probability
zbMATH Keywords: ergodic Markov chain
zbMATH Keywords: probability updating functions
zbMATH Keywords: linear dependence
Wikidata QID: Q60328382
MaRDI profile type: MaRDI publication profile
full work available at URL: 0020-0255(85)90049-0
OpenAlex ID: W1967960734
cites work: Automaton theory and modelling of biological systems. Translated by Scitran (Scientific Translation Service)
cites work: Q5643915
cites work: On the behavior of stochastic automata with a variable structure
cites work: Learning Automata - A Survey
cites work: Q4098444
cites work: Absolutely Expedient Learning Algorithms for Stochastic Automata
cites work: Markov processes and learning models
cites work: Some convergence theorems for stochastic learning models with distance diminishing operators
cites work: Learning automata processing ergodicity of the mean: The two-action case
cites work: Q3926044
cites work: Use of learning automata for the control of random search
cites work: Gradient approach for recursive estimation and control in finite Markov chains
cites work: Multi-automaton games: a rationale for expedient collective behavior
Recommended article: Learning automata processing ergodicity of the mean: The two-action case
Recommended article: Absorbing and Ergodic Discretized Two-Action Learning Automata
Recommended article: Generalized learning automata for multi-agent reinforcement learning
Recommended article: Q3838835
Recommended article: Learning automata and stochastic optimization
Recommended article: Continuous action set learning automata for stochastic optimization
Recommended article: An estimator algorithm for learning automata with changing number of actions
Recommended article: Q3677539
Recommended article: Learning automata with continuous input and changing number of actions
description: scientific article; zbMATH DE number 3930355