Following the Crowd: Literature Support and the Capabilities of Autonomous Research Agents

Michele Zampa ()
Additional contact information
Michele Zampa: Geneva Graduate Institute

No 14-2026, IHEID Working Papers from Economics Section, The Graduate Institute of International Studies

Abstract: Machine-learning models often perform poorly when asked to generalize beyond the support of the training distribution. This paper asks whether the same limitation shapes the research capabilities of autonomous large language model (LLM) agents: do they perform better when generating papers that follow research paradigms already well represented in the literature? I study this question using evidence from the Autonomous Policy Evaluation (APE) project, an open platform developed by the Social Catalyst Lab at the University of Zurich in which LLM agents generate empirical economic policy papers and compete in a tournament-style evaluation against human-written benchmarks. I construct a measure of literature support by locating each paper abstract in the semantic space of economics using a comprehensive corpus of English-language economics abstracts from OpenAlex. This measure captures whether a paper lies in a crowded or sparse region of the discipline’s existing research landscape. I then test whether literature support predicts tournament performance. I find that literature support shows a statistically significant positive association with performance for AI-generated papers, but not for human-written papers. Because outcomes are assigned by an LLM judge, this relationship could partly reflect evaluation bias toward more familiar topics. However, the absence of a comparable pattern among human papers suggests that the result is not purely judge-side. The evidence is more consistent with a production-side interpretation: autonomous research agents perform better when operating in areas that are more densely represented in the existing literature and, plausibly, in model training data. The findings shed some light on both the promise and the limits of agentic LLM systems as producers of scientific research.

Keywords: LLM agents; AI-generated research; economics of science; semantic embeddings; scientific novelty (search for similar items in EconPapers)
JEL-codes: A14 B41 D83 O33 (search for similar items in EconPapers)
Pages: 15 pages
Date: 2026-05-18
References: Add references at CitEc
Citations:

Downloads: (external link)
http://repec.graduateinstitute.ch/pdfs/Working_papers/HEIDWP14-2026.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gii:giihei:heidwp14-2026

Access Statistics for this paper

More papers in IHEID Working Papers from Economics Section, The Graduate Institute of International Studies Contact information at EDIRC.
Bibliographic data for series maintained by Dorina Dobre ().