Is Inference Conditional on Not Rejecting a Pre-test Less Reliable than Unconditional Inference?

Cl\'ement de Chaisemartin and Xavier D'Haultf{\oe}uille

Papers from arXiv.org

Abstract: Researchers often report an estimate and confidence interval only if a specification test does not reject - for example, only if a pre-trend test supports a difference-in-differences design. Does this make the reported confidence interval less reliable? We show that when the identifying assumptions hold, the answer is no: conditional on passing the pre-test, the confidence interval may over-cover but cannot under-cover. When the assumptions fail, pre-testing can reduce coverage, but under an additional condition it still improves coverage relative to not pre-testing. A calibration to 12 differences-in-differences examples suggests modest effects of pre-testing on coverage.

Date: 2024-07, Revised 2026-10
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://arxiv.org/pdf/2407.03725 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2407.03725

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().