An adaptive exploration mechanism for Q-learning in spatial public goods games

Shaofei Shen, Xuejun Zhang, Aobo Xu and Taisen Duan

Chaos, Solitons & Fractals, 2024, vol. 189, issue P1

Abstract: The Q-learning algorithm has been widely applied to investigate the emergence of cooperation in social dilemmas. Despite ϵ -greedy being the most common exploration strategy in Q-learning, mechanisms for adjusting exploration as the game environment changes have not been thoroughly researched. To stay close to reality, this paper proposes an environment-adaptive exploration-based Q-Learning algorithm. We applied the registration concept from image processing to characterize agents’ sensitivity to changes in their surrounding environment to obtain local stimulation. Additionally, we calculated the advantage differences between the agent and the global environment to acquire global stimulation. Simulation results on the public goods game show that the level of cooperation increases and the fraction of exploration consequently decreases when the agents focus more on the local environment. However, the impact of the basic exploration rate on the level of cooperation is not uniform: when the enhancement factor is low, an increase in the exploration rate promotes cooperation, while when the enhancement factor is high, increasing the exploration rate reduces the level of cooperation. The basic exploration rate directly affects the fraction of exploration. Therefore, increasing the basic exploration rate can stably increase the fraction of exploration of the agents. Similarly, the effect of the memory strength parameter λ on the level of cooperation is positively correlated, and increasing the value of λ increases the level of cooperation across the board. These evolutionary dynamics could enrich the understanding of cooperation in complex systems.

Keywords: Spatial public goods game; Reinforcement learning; Exploration behavior; Registration (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0960077924012578
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:chsofr:v:189:y:2024:i:p1:s0960077924012578

DOI: 10.1016/j.chaos.2024.115705

Access Statistics for this article

Chaos, Solitons & Fractals is currently edited by Stefano Boccaletti and Stelios Bekiros

More articles in Chaos, Solitons & Fractals from Elsevier
Bibliographic data for series maintained by Thayer, Thomas R. ().