 Training Novices to Think, or Giving Them LLMs? Evidence from an RCTHemath Asirvatham, Chiara Betti, Rachel Brown, Arnaldo Camuffo, Aaron Chatterji, Chiara Fumagalli, Alfonso Gambardella, Myriam Mariani, Abhinav Pandey, Steve Ramos, Giovanni Salvucci and Vladimir Simic No 21882, CEPR Discussion Papers from Centre for Economic Policy Research Abstract: Our ability to solve problems is changing with LLMs, but we do not know how LLMs combine with cognitive skills. We study the implications of this combination for a set of processes and outcomes: coherent logic (if A then B), falsification (counterfactual thinking), mechanisms (understanding why), evaluated quality of outcomes, distance from expert solutions, and number and diversity of ideas. A pre-registered 2 x 2 randomized controlled trial assigns 1,053 first-year undergraduate business school students to four conditions: training in causal reasoning, access to ChatGPT, none, or both. Exposure to both treatments is a net benefit because, for each outcome, at least one of the two treatments improves it and neither harms it. Our study is unique in the literature in its scale, in combining an LLM treatment with a treatment targeting a relevant cognitive skill, and in the breadth of processes and outcomes it covers. JEL-codes: J01 J08 M31 (search for similar items in EconPapers) Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: https://EconPapers.repec.org/RePEc:cpr:ceprdp:21882 Ordering information: This working paper can be ordered from Access Statistics for this paper More papers in CEPR Discussion Papers from Centre for Economic Policy Research 33 Great Sutton Street, London EC1V 0DX, UK. |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
School of Business at Örebro University.