Q-learning facilitates norm emergence in metanorm game model with topological structures

Wei Zhang, Dongkai Zhao, Xing Jin, Huizhen Zhang, Tianbo An, Guanghai Cui and Zhen Wang

Chaos, Solitons & Fractals, 2025, vol. 195, issue C

Abstract: Axelrod’s model and its subsequent studies have become a valuable framework for fostering cooperation norms among self-interested agents. Within this framework, the concepts of “boldness” and “vengefulness” are specifically employed to characterize agents’ behaviors in terms of cooperation and punishment (including metapunishment). Describing behavior solely through the parameters B and V may be overly simplistic and lacks generalizability, making it difficult to apply to other scenarios. Moreover, privacy concerns and the difficulty of evaluating complex states in real-world scenarios limit agents’ access to detailed payoff information from their neighbors. To address these questions, our paper employs self-regarding Q-learning, a well-established method for examining the dynamics of strategy updates and agents’ learning processes, to investigate whether metanorms can naturally emerge through players’ strategy selection. Through extensive experiments, we observe cooperative norms’ successful emergence driven by agents’ strategy selection variations. Over 90% of agents choose to cooperate on average. In subsequent analyses, we explore the underlying reasons for the emergence of cooperative norms from perspectives of changes in Q-values, punishment and metapunishment frequencies. Additionally, we examine the impact of topological structures on players’ strategy selection and assess the emergence of norms across different temptation levels, population sizes, and regulatory intensity levels to validate the model’s sensitivity.

Keywords: Social dilemma; Punishment mechanism; Metanorm game; Self-regarding Q-learning; Metapunishment; Norm emergence (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0960077925003108
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:chsofr:v:195:y:2025:i:c:s0960077925003108

DOI: 10.1016/j.chaos.2025.116297

Access Statistics for this article

Chaos, Solitons & Fractals is currently edited by Stefano Boccaletti and Stelios Bekiros

More articles in Chaos, Solitons & Fractals from Elsevier
Bibliographic data for series maintained by Thayer, Thomas R. ().