No label defined (Q4490812)

From MaRDI portal

!

This is the item page for this Wikibase entity, intended for internal use and editing purposes. Please use the normal view instead:

scientific article; zbMATH DE number 1479845
Language Label Description Also known as
default for all languages
No label defined
    English
    No label defined
    scientific article; zbMATH DE number 1479845

      Statements

      0 references
      23 July 2000
      0 references
      sonar processing technique
      0 references
      hybrid architecture
      0 references
      Dyna
      0 references
      Book contains: Chapter 1 introduction. Chapter 2 describes in detail the proposed sonar processing technique. Chapter 3 through 5 are devoted to reinforcement learning and the last chapter summarizes the thesis. Readers familiar with reinforcement learning and want to go directly to the learning part of the thesis should skip chapter 3. Chapter 2 starts off with the description of the experimental testbed used to verify our sonar processing method. To address the inconsistency between the input dimension and the task dimension, a hashing method is presented. Hashing abstracts away the input space by partitioning, in a natural way, the sensors mounted on the robot into few and relevant regions. Following this, a linear recursive Kalman filter is introduced. The filter estimates a region depth by propagating an assumed conditional probability density function. The remaining sections present open and closed loop experiments where this processing technique is tested and compared with other existing methods. The chapter also includes material on fuzzy controller that is implemented in the closed loop experiment. Chapter 3 begins with a simple hypothetical experiment that is used to define the vocabulary and elements of a reinforcement learning paradigm. Then, it discusses the different optimality criteria by which learning algorithms can be judged. Known problems associated with the measure of convergence and the tradeoff between exploitation and exploration are also raised. Afterwards, delayed reinforcement where actions are chosen to optimize future reinforcements is dealt with. In subsequent sections, we review existing techniques, model based and model free approaches of handling delayed reinforcement learning problems. The chapter closes by presenting a hybrid architecture called Dyna that integrates planning, learning and action. Chapter 4 first presents the essence of bias-variance dilemma by decomposing the estimation error into two components. Following, bias in the context of learning is formally defined and some useful forms of biases that have been dominant in the past are discussed. Since it is difficult how to bias a general problem towards important aspects, a particular problem domain that eases the study of amount and quality of bias is first chosen. Further, as a way of introducing prior knowledge into the problem, the notion of belief matrix is introduced. Then, a variety of biases each altering the belief matrix differently are derived. While some of the biases are derived by considering only the environment, others are derived directly from the task. A performance index that measures the effectiveness of a bias is defined. Based on this index, the biases are assessed, and new results are reported. Before closing the chapter, the advantage of continual learning over learning from scratch is also presented. Chapter 5 reports an implemented biased reinforcement learning experiment. The most pressing issue in the chapter is the integration of learned knowledge with existing knowledge. The learner is supplied with two kinds of knowledge domain knowledge that provides symbolic knowledge to the learner on key world features and basic reflex rules that enables the robot to be operational right from the beginning. The learning arhcitecture is a two layered neural network. The first layer is a growing radial basis function network (section 15) that is used to construct state space adaptively, and the second layer is a stochastic layer that performs the reinforcement learning algorithm. The reinforcement learning method is structured around estimating the long term value function from which actions are chosen. The kind of application presented is to learn to reach a goal position following a minimum cost path or trajectory. Chapter 6 summarizes the results of the thesis and ends with the author's final closing thoughts.
      0 references

      Identifiers