Enhancing IMF Economics Training: AI-Powered Analysis of Qualitative Learner Feedback

András Komáromi, Xiaomin Wu, Ran Pan, Yang Liu, Pablo Cisneros, Anchal Manocha and Hiba El Oirghi

No 2024/166, IMF Working Papers from International Monetary Fund

Abstract: The International Monetary Fund (IMF) has expanded its online learning program, offering over 100 Massive Open Online Courses (MOOCs) to support economic and financial policymaking worldwide. This paper explores the application of Artificial Intelligence (AI), specifically Large Language Models (LLMs), to analyze qualitative feedback from participants in these courses. By fine-tuning a pre-trained LLM on expert-annotated text data, we develop models that efficiently classify open-ended survey responses with accuracy comparable to human coders. The models’ robust performance across multiple languages, including English, French, and Spanish, demonstrates its versatility. Key insights from the analysis include a preference for shorter, modular content, with variations across genders, and the significant impact of language barriers on learning outcomes. These and other findings from unstructured learner feedback inform the continuous improvement of the IMF's online courses, aligning with its capacity development goals to enhance economic and financial expertise globally.

Keywords: IMF Economics training; measuring training impact; learner feedback; application of artificial intelligence; learner satisfaction; Artificial intelligence; Machine learning; Information systems and platforms; Automation; Sub-Saharan Africa; Western Hemisphere; Middle East and Central Asia; Europe; Asia and Pacific (search for similar items in EconPapers)
Pages: 37
Date: 2024-08-02
New Economics Papers: this item is included in nep-ain, nep-big, nep-cmp and nep-inv
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.imf.org/external/pubs/cat/longres.aspx?sk=552926 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:imf:imfwpa:2024/166

Ordering information: This working paper can be ordered from
http://www.imf.org/external/pubs/pubs/ord_info.htm

Access Statistics for this paper

More papers in IMF Working Papers from International Monetary Fund International Monetary Fund, Washington, DC USA. Contact information at EDIRC.
Bibliographic data for series maintained by Akshay Modi ().