An Auditable AI Agent Loop for Empirical Economics: A Case Study in Forecast Combination

Minchul Shin

No 26-41, Working Papers from Federal Reserve Bank of Philadelphia

Abstract: AI coding agents, general-purpose assistants that write and execute code, make empirical specification search fast and cheap, but they also widen hidden researcher degrees of freedom. This paper adapts an open-source agent-loop architecture to an empirical economics workflow and adds a post-search holdout evaluation. In a forecast-combination illustration, independent agent searches find methods that improve on benchmarks from the original study. Logged search and holdout evaluation together make adaptive specification search more transparent and help distinguish robust improvements from sample-specific discoveries.

Keywords: Agent loops; Autoresearch; Specification search; Researcher degrees of freedom; Responsible AI; Forecast combination (search for similar items in EconPapers)
JEL-codes: C18 C52 C53 (search for similar items in EconPapers)
Pages: 36
Date: 2026-08-27
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.philadelphiafed.org/-/media/FRBP/Asset ... ers/2026/wp26-41.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:fip:fedpwp:103699

Ordering information: This working paper can be ordered from

DOI: 10.21799/frbp.wp.2026.41

Access Statistics for this paper

More papers in Working Papers from Federal Reserve Bank of Philadelphia Contact information at EDIRC.
Bibliographic data for series maintained by Beth Paul ().