A Sharp and Robust Test for Selective Reporting

Stefan Faridani

Papers from arXiv.org

Abstract: This paper proposes a test that is consistent against every detectable form of selective reporting and remains interpretable even when the t-scores are not exactly normal. The test statistic is the distance between the smoothed empirical t-curve and the set of all t-curves that would be possible in the absence of any selective reporting. This novel projection test can only be evaded in large meta-samples by selective reporting that also evades all other valid tests of restrictions on the t-curve. A second benefit of the projection test is that under the null we can interpret the projection residual as noise plus bias incurred from approximating the t-score's exact distribution with the normal. Applying the test to the Brodeur et al. (2020) meta-data, we find that the t-curves for RCTs, IVs, and DIDs are more distorted than could arise by chance. But an Edgeworth Expansion reveals that these distortions are small enough to be plausibly explained by the only approximate normality of the individual t-scores. The detection of selective reporting in this meta-sample is therefore more fragile than previously known.

Date: 2025-06
References: Add references at CitEc
Citations:

Downloads: (external link)
http://arxiv.org/pdf/2506.20035 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2506.20035

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().