Robust Reinforcement Learning under Diffusion Models for Data with Jumps

Chenyang Jiang, Donggyu Kim (), Alejandra Quintos and Yazhen Wang
Additional contact information
Donggyu Kim: Department of Economics, University of California Riverside

No 202512, Working Papers from University of California at Riverside, Department of Economics

Abstract: Reinforcement Learning (RL) has proven effective in solving complex decision-making tasks across various domains, but challenges remain in continuous-time settings, particularly when state dynamics are governed by stochastic differential equations (SDEs) with jump components. In this paper, we address this challenge by introducing the Mean-Square Bipower Variation Error (MSBVE) algorithm, which enhances robustness and convergence in scenarios involving significant stochastic noise and jumps. We first revisit the Mean-Square TD Error (MSTDE) algorithm, commonly used in continuous-time RL, and highlight its limitations in handling jumps in state dynamics. The proposed MSBVE algorithm minimizes the mean-square quadratic variation error, offering improved performance over MSTDE in environments characterized by SDEs with jumps. Simulations and formal proofs demonstrate that the MSBVE algorithm reliably estimates the value function in complex settings, surpassing MSTDE's performance when faced with jump processes. These findings underscore the importance of alternative error metrics to improve the resilience and effectiveness of RL algorithms in continuous-time frameworks.

Pages: 45 Pages
Date: 2025-10
References: Add references at CitEc
Citations:

Downloads: (external link)
https://economics.ucr.edu/repec/ucr/wpaper/202512.pdf First version, 2025 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ucr:wpaper:202512

Access Statistics for this paper

More papers in Working Papers from University of California at Riverside, Department of Economics Contact information at EDIRC.
Bibliographic data for series maintained by Kelvin Mac ().