Gradient-Based Reinforcement Learning for Dynamic Quantile

Lukas Janasek ()
Additional contact information
Lukas Janasek: Institute of Economic Studies, Charles University, Prague, Czech Republic

No 2025/12, Working Papers IES from Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies

Abstract: This paper develops a novel gradient-based reinforcement learning algorithm for solving dynamic quantile models with uncertainty. Unlike traditional approaches that rely on expected utility maximization, we focus on agents who evaluate outcomes based on specific quantiles of the utility distribution, capturing intratemporal risk attitudes via a quantile level ? ? (0, 1). We formulate a recursive quantile value function associated with time consistent dynamic quantile preferences in Markov decision process. At each period, the agent aims to maximize the quantile of a distribution composed of instantaneous utility combined with the discounted future value, conditioned on the current state. Next, we adapt the Actor-Critic framework to learn ?-quantile of the distribution and policy maximizing the ?-quantile. We demonstrate the accuracy and robustness of the proposed algorithm using an quantile intertemporal consumption model with known analytical solutions. The results confirm the effectiveness of our algorithm in capturing optimal quantile-based behavior and stability of the algorithm.

Keywords: Dynamic programming; Quantile preferences; Reinforcement learning (search for similar items in EconPapers)
JEL-codes: C61 C63 (search for similar items in EconPapers)
Pages: 33 pages
Date: 2025-07, Revised 2025-07
New Economics Papers: this item is included in nep-big, nep-cmp, nep-dge and nep-upt
References: Add references at CitEc
Citations:

Downloads: (external link)
https://ies.fsv.cuni.cz/en/gradient-based-reinforc ... ing-dynamic-quantile (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:fau:wpaper:wp2025_12

Access Statistics for this paper

More papers in Working Papers IES from Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies Contact information at EDIRC.
Bibliographic data for series maintained by Natalie Svarcova ().