(Human) Attention Is (Still) All You Need: Human Oversight Makes Ai-Assisted Social Science

Chen Zhu, Xiaolu Wang and Weilong Zhang

Cambridge Working Papers in Economics from Faculty of Economics, University of Cambridge

Abstract: Large language models (LLMs) are increasingly delegated tasks once reserved for trained researchers: generating hypotheses, choosing specifications, drafting conclusions. Whether this delegation produces trustworthy science is not solely a technical question about model capability; it also depends on how cognitive labour is structured between humans and machines. We study this problem by organising an AI-assisted research workflow around behavioural-science principles, pre-commitment, decision sequencing, accountability, and attention allocation, rather than by directly measuring human psychology at the gates. We propose that reliability in AI-assisted research is a property of decision architecture: the placement, sequencing, and binding force of the choices that humans and AI components are each permitted to make. In a pre-specified 2 × 4 factorial experiment (N = 280 complete research runs across four datasets), an unconstrained multi-agent baseline produced critical failures in 72% of runs; the same underlying model and the same agent decomposition, with identical prompts on the reasoning agents shared by both arms, failed in 16% once organised by three architectural commitments (LLMs restricted to reasoning, data and estimation executed deterministically, three human decision gates; Fisher’s exact p

Date: 2026-06-12
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.econ.cam.ac.uk/sites/default/files/pub ... pe-pdfs/cwpe2643.pdf

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cam:camdae:2643

Access Statistics for this paper

More papers in Cambridge Working Papers in Economics from Faculty of Economics, University of Cambridge
Bibliographic data for series maintained by Jake Dyer ().