Numerical analysis of a reinforcement learning model with the dynamic aspiration level in the iterated prisoner's dilemma
From MaRDI portal
Publication:2263458
DOI10.1016/J.JTBI.2011.03.005zbMATH Open1307.91029arXiv1012.0121OpenAlexW2085118870WikidataQ48148607 ScholiaQ48148607MaRDI QIDQ2263458FDOQ2263458
Authors: Naoki Masuda, Mitsuhiro Nakamura 
Publication date: 18 March 2015
Published in: Journal of Theoretical Biology (Search for Journal in Brave)
Abstract: Humans and other animals can adapt their social behavior in response to environmental cues including the feedback obtained through experience. Nevertheless, the effects of the experience-based learning of players in evolution and maintenance of cooperation in social dilemma games remain relatively unclear. Some previous literature showed that mutual cooperation of learning players is difficult or requires a sophisticated learning model. In the context of the iterated Prisoner's Dilemma, we numerically examine the performance of a reinforcement learning model. Our model modifies those of Karandikar et al. (1998), Posch et al. (1999), and Macy and Flache (2002) in which players satisfice if the obtained payoff is larger than a dynamic threshold. We show that players obeying the modified learning mutually cooperate with high probability if the dynamics of threshold is not too fast and the association between the reinforcement signal and the action in the next round is sufficiently strong. The learning players also perform efficiently against the reactive strategy. In evolutionary dynamics, they can invade a population of players adopting simpler but competitive strategies. Our version of the reinforcement learning model does not complicate the previous model and is sufficiently simple yet flexible. It may serve to explore the relationships between learning and evolution in social dilemma situations.
Full work available at URL: https://arxiv.org/abs/1012.0121
Recommendations
- Evolution of cooperation facilitated by reinforcement learning with adaptive aspiration levels
- A theoretical analysis of temporal difference learning in the iterated prisoner's dilemma game
- Interaction dynamics of two reinforcement learners
- Learning dynamics in social dilemmas
- Convergence of aspirations and (partial) cooperation in the prisoner's dilemma
Cites Work
- Title not available (Why is that?)
- Title not available (Why is that?)
- The calculus of selfishness.
- Evolutionary dynamics of cooperation
- Evolving aspirations and cooperation
- Learning dynamics in social dilemmas
- The evolution of stochastic strategies in the prisoner's dilemma
- Game-dynamical aspects of the prisoner's dilemma
- Satisficing leads to cooperation in mutual interests games
- Stochastic strategies in the prisoner's dilemma
- Automata, repeated games and noise
- Learning aspiration in repeated games
- Convergence of aspirations and (partial) cooperation in the prisoner's dilemma
- Aspiration adaptation in the ultimatum minigame.
- Transient and asymptotic dynamics of reinforcement learning in games
- A theoretical analysis of temporal difference learning in the iterated prisoner's dilemma game
- Dynamics of internal models in game players
- Satisfying and optimality in \(2\times 2\) common interest games
Cited In (17)
- Empty nodes affect conditional cooperation under reinforcement learning
- Heterogeneous aspiration resolves social dilemma in structured populations
- Interaction dynamics of two reinforcement learners
- The influence of experienced guider on cooperative behavior in the prisoner's dilemma game
- Reinforcement learning explains various conditional cooperation
- Title not available (Why is that?)
- The average abundance function with mutation of the multi-player snowdrift evolutionary game model
- A theoretical analysis of temporal difference learning in the iterated prisoner's dilemma game
- Evolution of cooperation facilitated by reinforcement learning with adaptive aspiration levels
- Experiments on individual strategy updating in iterated snowdrift game under random rematching
- Asymmetrical expectations of future interaction and cooperation in the iterated prisoner's dilemma game
- Aspiration-based full cooperation in finite systems of players
- Memory-two strategies forming symmetric mutual reinforcement learning equilibrium in repeated prisoners' dilemma game
- Cooperative behavior under the influence of multiple experienced guiders in prisoner's dilemma game
- Promoting effect of adaptive interaction based on random neighbors to cooperation in the spatial prisoner's dilemma game
- Learning dynamics in social dilemmas
- Learning dynamics and norm psychology supports human cooperation in a large-scale prisoner's dilemma on networks
This page was built for publication: Numerical analysis of a reinforcement learning model with the dynamic aspiration level in the iterated prisoner's dilemma
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2263458)
