Cooperation dynamics driven by reinforcement learning with interactive diversity in structured populations

Tianbo An, Huizhen Zhang, Zhanshuo Zhang, Guanghui Liu, Jiayu Li, Liangyu Chen and Zhen Wang

Chaos, Solitons & Fractals, 2025, vol. 201, issue P2

Abstract: In reality, individuals tend to make different decisions based on differences in relationships and behaviors with their neighbors. Based on this observation, the paper explores the evolution of cooperative behavior when agents develop separated actions for each neighbor by the reinforcement learning approach. Through simulation experiments, it is shown that our model improves the cooperative level compared to results that only consider the agent’s own behavior. This is because agents tend to adopt cooperative strategies toward their neighbors while avoiding exploitation, thus promoting the steady expansion of cooperation. Notably, we find that agents do not always choose the action with the highest expected rewards. Therefore, we classify the behavior strategies of the agents into 16 types, corresponding to all possible combinations of actions selected in different states. We observe that agents adopting a specific behavior strategy tend to dominate the evolutionary process: when they choose to cooperate, they switch to defection in the next round regardless of the opponent’s action; conversely, when they defect, they switch to cooperation in the next round, again independent of the opponent’s behavior. These agents are typically distributed among others with different strategy types, playing a bridging and buffering role. By facilitating the expansion of neighboring agents, they contribute to the spread of cooperative behavior and ultimately enhance the overall level of cooperation in the population. Similar phenomena are also observed under initial specific distributions (e.g., ALLC, ALLD). Next, the hyperparameters of reinforcement learning are analyzed, and the results show that cooperation is easier to maintain and expand when agents make decisions based on past experiences and fully consider potential future rewards. We also compare this model with a control model that adopted the assumption of interactive homogeneity, and further examine the impact of different network structures on the cooperative evolution. Finally, we introduce the memory mechanism of agents as an extended analysis of the model.

Keywords: Evolutionary games; Interactive diversity; Reinforcement learning; Cooperation (search for similar items in EconPapers)
Date: 2025
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0960077925013219
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:chsofr:v:201:y:2025:i:p2:s0960077925013219

DOI: 10.1016/j.chaos.2025.117308

Access Statistics for this article

Chaos, Solitons & Fractals is currently edited by Stefano Boccaletti and Stelios Bekiros

More articles in Chaos, Solitons & Fractals from Elsevier
Bibliographic data for series maintained by Thayer, Thomas R. ().