Training Novices to Think, or Giving Them LLMs? Evidence from an RCT

Hemath Asirvatham, Chiara Betti, Rachel Brown, Arnaldo Camuffo, Aaron Chatterji, Chiara Fumagalli, Alfonso Gambardella, Myriam Mariani, Abhinav Pandey, Steve Ramos, Giovanni Salvucci and Vladimir Simic

No 21882, CEPR Discussion Papers from Centre for Economic Policy Research

Abstract: Our ability to solve problems is changing with LLMs, but we do not know how LLMs combine with cognitive skills. We study the implications of this combination for a set of processes and outcomes: coherent logic (if A then B), falsification (counterfactual thinking), mechanisms (understanding why), evaluated quality of outcomes, distance from expert solutions, and number and diversity of ideas. A pre-registered 2 x 2 randomized controlled trial assigns 1,053 first-year undergraduate business school students to four conditions: training in causal reasoning, access to ChatGPT, none, or both. Exposure to both treatments is a net benefit because, for each outcome, at least one of the two treatments improves it and neither harms it. Our study is unique in the literature in its scale, in combining an LLM treatment with a treatment targeting a relevant cognitive skill, and in the breadth of processes and outcomes it covers.

JEL-codes: J01 J08 M31 (search for similar items in EconPapers)
Date: 2026-08
References: Add references at CitEc
Citations:

Downloads: (external link)
https://cepr.org/publications/DP21882 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cpr:ceprdp:21882

Ordering information: This working paper can be ordered from
https://cepr.org/publications/DP21882

Access Statistics for this paper

More papers in CEPR Discussion Papers from Centre for Economic Policy Research 33 Great Sutton Street, London EC1V 0DX, UK.
Bibliographic data for series maintained by CEPR ().