All Models Are Wrong, but Can They Be Useful? Lessons from COVID-19 Agent-Based Models: A Systematic Review

Emma Von Hoene (), Sara Von Hoene (), Szandra Péter (), Ethan Hopson (), Emily Csizmadia (), Faith Fenyk (), Kai Barner (), Timothy Leslie (), Hamdi Kavak (), Andreas Züfle (), Amira Roess () and Taylor Anderson ()
Additional contact information
Emma Von Hoene: https://science.gmu.edu/directory/taylor-anderson
Timothy Leslie: https://science.gmu.edu/directory/timothy-leslie
Hamdi Kavak: https://hamdikavak.com/
Andreas Züfle: https://spatial.cs.emory.edu/pages/member/andreas-zufle.html
Amira Roess: https://publichealth.gmu.edu/profiles/aroess
Taylor Anderson: https://science.gmu.edu/directory/taylor-anderson

Journal of Artificial Societies and Social Simulation, 2026, vol. 29, issue 1, 7

Abstract: The COVID-19 pandemic prompted a surge in computational models to simulate disease dynamics and guide interventions. Agent-based models (ABMs) are well-suited to capture population and environmental heterogeneity, but their rapid deployment raised questions about utility for health policy. We systematically reviewed 536 COVID-19 ABM studies published from January 2020 to December 2023, retrieved from Web of Science, PubMed, and Wiley on January 30, 2024. Studies were included if they used ABMs to simulate COVID-19 transmission, where reviews were excluded. Studies were assessed against nine criteria of model usefulness, including transparency and re-use, interdisciplinary collaboration and stakeholder engagement, and evaluation practices. Publications peaked in late 2021 and were concentrated in a few countries. Most models explored behavioral or policy interventions (n = 294, 54.85%) rather than real-time forecasting (n = 9, 1.68%). While most described model assumptions (n = 491, 91.60%), fewer disclosed limitations (n = 349, 65.11%), shared code (n = 219, 40.86%), or built on existing models (n = 195, 36.38%). Standardized reporting protocols (n = 36, 6.72%) and stakeholder engagement were rare (13.62%, n = 73). Only 2.24% (n = 12) described a comprehensive validation framework, though uncertainty was often quantified (n = 407, 75.93%). Over time, reporting of stakeholder engagement and evaluation increased. Studies that claimed policy relevance (n = 354, 66.05%) more often included some evaluation (n = 283, 79.94% vs. n = 125, 68.68%) and stakeholder engagement (n = 61, 17.23% vs. n = 12, 6.59%), though they were less likely to re-use models or share code. Limitations of this review include underrepresentation of non-English studies, subjective data extraction, variability in study quality, and limited generalizability. Overall, COVID-19 ABMs advanced quickly, but lacked transparency, accessibility, and participatory engagement. Stronger standards are needed for ABMs to serve as reliable decision-support tools in future public health crises.

Keywords: COVID-19; Agent-Based Models; Infectious Disease Modeling; Systematic Literature Review (search for similar items in EconPapers)
Date: 2026-01-31
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.jasss.org/29/1/7/7.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:jas:jasssj:2025-147-3

Access Statistics for this article

More articles in Journal of Artificial Societies and Social Simulation from Journal of Artificial Societies and Social Simulation
Bibliographic data for series maintained by Francesco Renzini ().