What can simulation test beds teach us about social science? Results of the ground truth program

Asmeret Naugle (), Daniel Krofcheck, Christina Warrender, Kiran Lakkaraju, Laura Swiler, Stephen Verzi, Ben Emery, Jaimie Murdock, Michael Bernard and Vicente Romero
Additional contact information
Asmeret Naugle: Sandia National Laboratories Albuquerque
Daniel Krofcheck: Sandia National Laboratories Albuquerque
Christina Warrender: Sandia National Laboratories Albuquerque
Kiran Lakkaraju: Sandia National Laboratories Albuquerque
Laura Swiler: Sandia National Laboratories Albuquerque
Stephen Verzi: Sandia National Laboratories Albuquerque
Ben Emery: Sandia National Laboratories Albuquerque
Jaimie Murdock: Sandia National Laboratories Albuquerque
Michael Bernard: Sandia National Laboratories Albuquerque
Vicente Romero: Sandia National Laboratories Albuquerque

Computational and Mathematical Organization Theory, 2023, vol. 29, issue 1, No 9, 242-263

Abstract: Abstract The ground truth program used simulations as test beds for social science research methods. The simulations had known ground truth and were capable of producing large amounts of data. This allowed research teams to run experiments and ask questions of these simulations similar to social scientists studying real-world systems, and enabled robust evaluation of their causal inference, prediction, and prescription capabilities. We tested three hypotheses about research effectiveness using data from the ground truth program, specifically looking at the influence of complexity, causal understanding, and data collection on performance. We found some evidence that system complexity and causal understanding influenced research performance, but no evidence that data availability contributed. The ground truth program may be the first robust coupling of simulation test beds with an experimental framework capable of teasing out factors that determine the success of social science research.

Keywords: Social science; Simulation test beds; Complexity; Causal structure; Data efficiency; Metascience (search for similar items in EconPapers)
Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10588-021-09349-6 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:comaot:v:29:y:2023:i:1:d:10.1007_s10588-021-09349-6

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10588

DOI: 10.1007/s10588-021-09349-6

Access Statistics for this article

Computational and Mathematical Organization Theory is currently edited by Terrill Frantz and Kathleen Carley

More articles in Computational and Mathematical Organization Theory from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().