When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content

Shakked Dabran-Zivan, Inbal Klein-Avraham and Ayelet Baram-Tsabari

PLOS ONE, 2026, vol. 21, issue 3, 1-22

Abstract: Generative artificial intelligence (GenAI) blurs the boundaries between expert and non-expert sources, as it increasingly distributes and creates scientific content. This study examines how individuals adapt evaluation strategies, including content and source evaluation, and corroboration, when using GenAI versus a search engine. Based on performance tasks in which participants evaluated science-related socio-scientific dilemmas and follow-up interviews with 30 adult participants from diverse educational backgrounds, findings reveal that users employed these strategies on both platforms but adapted them in distinct ways. We identified two evaluation strategies that emerged as analytical constructs from the qualitative data. First, to corroborate output, participants frequently used a strategy we titled ‘representation evaluation,’ assessing whether GenAI accurately summarized its sources rather than verifying source agreement independently. Second, participants also applied ‘meta source evaluation,’ relying on their familiarity with sources provided by GenAI instead of directly evaluating the sources themselves. Although all participants engaged in dialogue with the chat, they did not leverage the bot’s dialogue capabilities to assess credibility, and many relied on a “machine heuristic”, assuming GenAI’s inherent correctness, reflecting a well-documented over-trust in automated systems. This research underscores the importance of developing and assessing critical evaluation skills for navigating AI-generated scientific information. Specifically, it extends existing models of online information evaluation to contexts mediated by artificial intelligence.

Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0345300 (text/html)
https://journals.plos.org/plosone/article/file?id= ... 45300&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pone00:0345300

DOI: 10.1371/journal.pone.0345300

Access Statistics for this article

More articles in PLOS ONE from Public Library of Science
Bibliographic data for series maintained by plosone ().