Rethinking Repeatability in Observational Social Science

Jonathan Ben-Menachem, Ari Galper and Nic Fishman

No 6ehrw_v1, SocArXiv from Center for Open Science

Abstract: Sociology has remained relatively insulated from debates about the ‘replication crisis.’ Heeding calls to consider replication more deeply, we introduce a distinction between two types of research reforms that have emerged in the wake of the crisis: specification-restricting reforms and specification-expanding reforms. Specification-restricting reforms—the more popular of the two—aim to increase the repeatability of research findings by controlling false positives. We show how these reforms’ internal logic breaks down outside of randomized experiments; in observational contexts, they risk enshrining fragile or misspecified models. We further argue that the premise of these reforms is flawed. Replication rates cannot be reduced to the purported prevalence of false positive findings. In their place, we propose a replication framework centered on specification-expanding reforms, stronger incentives for confirmatory research, and meta-analysis. This approach equips sociology to assess the repeatability of findings and build a more cumulative discipline.

Date: 2025-12-05
New Economics Papers: this item is included in nep-exp and nep-hpe
References: Add references at CitEc
Citations:

Downloads: (external link)
https://osf.io/download/6931f03793776124ca0d8bae/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:socarx:6ehrw_v1

DOI: 10.31219/osf.io/6ehrw_v1

Access Statistics for this paper

More papers in SocArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().