Reinforcement Learning from Experience Feedback: Application to Economic Policy

Tohid Atashbar

No 2024/114, IMF Working Papers from International Monetary Fund

Abstract: Learning from the past is critical for shaping the future, especially when it comes to economic policymaking. Building upon the current methods in the application of Reinforcement Learning (RL) to the large language models (LLMs), this paper introduces Reinforcement Learning from Experience Feedback (RLXF), a procedure that tunes LLMs based on lessons from past experiences. RLXF integrates historical experiences into LLM training in two key ways - by training reward models on historical data, and by using that knowledge to fine-tune the LLMs. As a case study, we applied RLXF to tune an LLM using the IMF's MONA database to generate historically-grounded policy suggestions. The results demonstrate RLXF's potential to equip generative AI with a nuanced perspective informed by previous experiences. Overall, it seems RLXF could enable more informed applications of LLMs for economic policy, but this approach is not without the potential risks and limitations of relying heavily on historical data, as it may perpetuate biases and outdated assumptions.

Keywords: LLMs; GAI; RLHF; RLAIF; RLXF; experience feedback; RL model; applying RLXF; Reinforcement Learning from AI feedback; Reinforcement Learning method; Artificial intelligence; Economic classification (search for similar items in EconPapers)
Pages: 23
Date: 2024-06-07
New Economics Papers: this item is included in nep-ain, nep-big and nep-cmp
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.imf.org/external/pubs/cat/longres.aspx?sk=549977 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:imf:imfwpa:2024/114

Ordering information: This working paper can be ordered from
http://www.imf.org/external/pubs/pubs/ord_info.htm

Access Statistics for this paper

More papers in IMF Working Papers from International Monetary Fund International Monetary Fund, Washington, DC USA. Contact information at EDIRC.
Bibliographic data for series maintained by Akshay Modi ().