Overview of FinMMEval 2026 Task 3: Live Financial Decision-Making Agents

Zhuohan Xie (), Lingfei Qian, Georgi Georgiev, Dimitar Dimitrov, Yuyang Dai, Rania Elbadry, Vanshikaa Jani, Xueqing Peng, Fan Zhang, Jimin Huang, Jiahui Geng, Yankai Chen, Ye Yuan, Haolun Wu, Yuxia Wang, Ivan Koychev, Veselin Stoyanov, Mingzi Song, Yu Chen, Xue Liu and Preslav Nakov ()
Additional contact information
Zhuohan Xie: MBZUAI - Mohamed bin Zayed University of Artificial Intelligence
Lingfei Qian: The Fin AI
Georgi Georgiev: Faculty of Mathematics and Informatics [Sofia] - Sofia University Saint Kliment Ohridski - Софийски университет „Свети Климент Охридски
Dimitar Dimitrov: Faculty of Mathematics and Informatics [Sofia] - Sofia University Saint Kliment Ohridski - Софийски университет „Свети Климент Охридски
Yuyang Dai: INSAIT - Institute for Computer Science, Artificial Intelligence and Technology
Rania Elbadry: MBZUAI - Mohamed bin Zayed University of Artificial Intelligence
Vanshikaa Jani: University of Arizona
Xueqing Peng: The Fin AI
Fan Zhang: UTokyo - The University of Tokyo
Jimin Huang: The Fin AI
Jiahui Geng: LIU - Linköpings Universitet = Linköping University
Yankai Chen: MBZUAI - Mohamed bin Zayed University of Artificial Intelligence, McGill University = Université McGill [Montréal, Canada]
Ye Yuan: McGill University = Université McGill [Montréal, Canada], Mila - Institut québécois d’intelligence artificielle = Quebec Artificial Intelligence Institute
Haolun Wu: McGill University = Université McGill [Montréal, Canada], Mila - Institut québécois d’intelligence artificielle = Quebec Artificial Intelligence Institute
Yuxia Wang: INSAIT - Institute for Computer Science, Artificial Intelligence and Technology
Ivan Koychev: Faculty of Mathematics and Informatics [Sofia] - Sofia University Saint Kliment Ohridski - Софийски университет „Свети Климент Охридски
Veselin Stoyanov: MBZUAI - Mohamed bin Zayed University of Artificial Intelligence
Mingzi Song: Meiji Gakuin University
Yu Chen: UTokyo - The University of Tokyo
Xue Liu: MBZUAI - Mohamed bin Zayed University of Artificial Intelligence, McGill University = Université McGill [Montréal, Canada], Mila - Institut québécois d’intelligence artificielle = Quebec Artificial Intelligence Institute
Preslav Nakov: MBZUAI - Mohamed bin Zayed University of Artificial Intelligence

Post-Print from HAL

Abstract: FinMMEval 2026 Task 3 evaluates financial decision-making agents in a live endpoint-based setting. Participating teams deployed public HTTP endpoints that received daily market payloads and returned BUY, HOLD, or SELL decisions for BTC and TSLA. Unlike static question-answering benchmarks, this task evaluates trading decisions produced by complete agent services under repeated daily calls; endpoint reachability and response validity are deployment constraints of the protocol. For each accepted endpoint, the official ranking averages the BTC and TSLA cumulative returns, each computed from a long/flat/short portfolio simulation with an initial balance of 100,000. Within the official scoring window, simulated returns differed sharply by asset: Infraglyph led the cross-asset mean CR, Fin-Analyst achieved the highest TSLA return, and NLP-DE achieved the highest BTC return. This divergence shows why a multi-asset evaluation should report both aggregate and asset-level results.

Keywords: cumulative return; endpoint evaluation; trading agents; live evaluation; financial agents; FinMMEval; FinMMEval financial agents live evaluation trading agents endpoint evaluation cumulative return (search for similar items in EconPapers)
Date: 2026-09-21
Note: View the original document on HAL open archive server: https://hal.science/hal-05726648v1
References: Add references at CitEc
Citations:

Published in CLEF 2026 Working Notes, CLEF Initiative, Sep 2026, Jena, Germany

Downloads: (external link)
https://hal.science/hal-05726648v1/document (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hal:journl:hal-05726648

Access Statistics for this paper

More papers in Post-Print from HAL
Bibliographic data for series maintained by CCSD ().