A hierarchical system of learning automata that can learn the globally optimal path (Q1096403)

A class of hierarchical systems of learning automata is considered where the system gets responses from its environment at each level of the hierarchy. Hierarchical systems are classified into three types, depending on the nature of the actions constituting the optimal path. If each action is also optimum for the respective level in the hierarchy, then the system is type I. If each action is also optimum for the respective automaton, then the system is type II. Any general hierarchy is type III. It is argued that none of the known reinforcement schemes of learning automata can learn the optimum path in the type III hierarchy. An algorithm for learning the globally optimal path suitable for the most general type of hierarchical problems is presented, and its convergence behavior is examined. The algorithm requires information transfer from the lower levels to the higher levels. The paper cites the connection between the hierarchical model studied and the finite Markov decision processes. No practical application examples are given.