Effective memory-one state representations for reinforcement learning in spatial social dilemma games

Huizhen Zhang, Ran Zhang, Juan Li and Zhen Wang

Chaos, Solitons & Fractals, 2026, vol. 210, issue P2

Abstract: Reinforcement learning offers a new methodological perspective for the study of the evolution of cooperation. The key lies in state representation, which determines how agents encode environmental information and make decisions accordingly. Existing research has largely focused on whether a specific state representation can promote cooperative behavior, while systematic comparisons of which types of state representations are more conducive to the emergence and maintenance of cooperation remain limited. To address this, the paper proposes a unified memory-one state representation framework to systematically compare different state representations and identify those that more effectively sustain high levels of cooperation. The results show that both the amount of local environmental information contained in the state representation and the way it is organized significantly affect the levels of cooperation. Among the representation methods examined, the one defined by the number of cooperators among an agent and its neighbors, which captures complete local information, proves to be the most effective. However, not all representation methods that include complete local information yield the same effect. Representation methods with overly coarse or overly fine granularity can impair the effective identification of local cooperative structures, leading to the spread of defection. These findings remain robust in different reinforcement learning algorithms, network structures, and game models. This study deepens our understanding of state representation mechanisms and provides guidance for state design in cooperative evolution research using reinforcement learning.

Keywords: Reinforcement learning; Spatial games; State representation; Cooperation (search for similar items in EconPapers)
Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0960077926008301
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:chsofr:v:210:y:2026:i:p2:s0960077926008301

DOI: 10.1016/j.chaos.2026.118689

Access Statistics for this article

Chaos, Solitons & Fractals is currently edited by Stefano Boccaletti and Stelios Bekiros

More articles in Chaos, Solitons & Fractals from Elsevier
Bibliographic data for series maintained by Thayer, Thomas R. ().