Stability of Experimental Results: Forecasts and Evidence

Stefano DellaVigna and Devin Pope

American Economic Journal: Microeconomics, 2022, vol. 14, issue 3, 889-925

Abstract: How robust are experimental results to changes in design? And can researchers anticipate which changes matter most? We consider a real-effort task with multiple behavioral treatments and examine the stability along six dimensions: (i) pure replication, (ii) demographics, (iii) geography and culture, (iv) the task, (v) the output measure, and (vi) the presence of a consent form. We find near-perfect replication of the experimental results and full stability of the results across demographics, significantly higher than a group of experts expected. The results differ instead across task and output change, mostly because the task change adds noise to the findings.

JEL-codes: C90 D82 D91 (search for similar items in EconPapers)
Date: 2022
References: Add references at CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://www.aeaweb.org/doi/10.1257/mic.20200129 (application/pdf)
https://doi.org/10.3886/E135221V1 (text/html)
https://www.aeaweb.org/doi/10.1257/mic.20200129.appx (application/pdf)
https://www.aeaweb.org/doi/10.1257/mic.20200129.ds (application/zip)
Access to full text is restricted to AEA members and institutional subscribers.

Related works:
Working Paper: Stability of Experimental Results: Forecasts and Evidence (2019)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:aea:aejmic:v:14:y:2022:i:3:p:889-925

Ordering information: This journal article can be ordered from
https://www.aeaweb.org/journals/subscriptions

DOI: 10.1257/mic.20200129

Access Statistics for this article

American Economic Journal: Microeconomics is currently edited by Johannes Hörner

More articles in American Economic Journal: Microeconomics from American Economic Association Contact information at EDIRC.
Bibliographic data for series maintained by Michael P. Albert ().