 An improved Q-learning algorithm integrating past and future rewardsQiu Lou, Dun Han and Mei Sun Chaos, Solitons & Fractals, 2026, vol. 208, issue P1 Abstract: Achieving stable and efficient learning remains a core challenge in reinforcement learning. However, traditional Q-learning methods often suffer from instability and slow convergence due to an unbalanced focus on either immediate or future rewards, while neglecting the role of past rewards. To address this, we propose an improved Q-learning algorithm that systematically integrates both past and future rewards through a balancing factor. Specifically, we design three reward weighting strategies: static discount, double exponential discount, and weighted moving average, to process past rewards. Theoretical analysis proves the convergence of the algorithm and reveals that the weighted moving average method minimizes reward variance. In a series of simulation experiments, we find that the weighted moving average approach significantly outperforms other methods in terms of convergence speed, cumulative reward, and learning stability. This study provides a new perspective on reward utilization in reinforcement learning through the balancing factor and adaptive weighting mechanisms. Keywords: Q-learning; Past rewards; Future rewards; Stability; Convergence (search for similar items in EconPapers) Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: https://EconPapers.repec.org/RePEc:eee:chsofr:v:208:y:2026:i:p1:s0960077926002687 DOI: 10.1016/j.chaos.2026.118127 Access Statistics for this article Chaos, Solitons & Fractals is currently edited by Stefano Boccaletti and Stelios Bekiros More articles in Chaos, Solitons & Fractals from Elsevier |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
School of Business at Örebro University.