Learning to win by reading manuals in a Monte-Carlo framework

DOI10.1613/JAIR.3484zbMATH Open1237.68138arXiv1401.5390OpenAlexW3121854931WikidataQ129489497 ScholiaQ129489497MaRDI QIDQ2887091FDOQ2887091

Authors: S. R. K. Branavan, David Silver, R. Barzilay 

Publication date: 16 May 2012

Published in: The Journal of Artificial Intelligence Research (JAIR) (Search for Journal in Brave)

Abstract: Domain knowledge is crucial for effective performance in autonomous control systems. Typically, human effort is required to encode this knowledge into a control algorithm. In this paper, we present an approach to language grounding which automatically interprets text in the context of a complex control application, such as a game, and uses domain knowledge extracted from the text to improve control performance. Both text analysis and control strategies are learned jointly using only a feedback signal inherent to the application. To effectively leverage textual information, our method automatically extracts the text segment most relevant to the current game state, and labels it with a task-centric predicate structure. This labeled text is then used to bias an action selection policy for the game, guiding it towards promising regions of the action space. We encode our model for text analysis and game playing in a multi-layer neural network, representing linguistic decisions via latent variables in the hidden layers, and game action quality via the output layer. Operating within the Monte-Carlo Search framework, we estimate model parameters using feedback from simulated games. We apply our approach to the complex strategy game Civilization II using the official game manual as the text guide. Our results show that a linguistically-informed game-playing agent significantly outperforms its language-unaware counterpart, yielding a 34% absolute improvement and winning over 65% of games when playing against the built-in AI of Civilization.

Full work available at URL: https://arxiv.org/abs/1401.5390

Recommendations

• Reinforcement learning and its application to the game of Go
• Knowledge acquisition for adaptive game AI
• A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
• scientific article; zbMATH DE number 1931836
• Grounding language for transfer in deep reinforcement learning

zbMATH Keywords

text analysisstrategic gamesMonte Carlo searchcontrol strategiesdomain knowledgelanguage groundingCivilization

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Noncooperative games (91A10) Application models in control theory (93C95) Rationality and learning in game theory (91A26)

Cited In (1)

• Learning from natural instructions

This page was built for publication: Learning to win by reading manuals in a Monte-Carlo framework

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2887091)