Reevaluating Causal Estimation Methods with Data from a Product Release

Justin Young and Eleanor Wiske Dillon

Papers from arXiv.org

Abstract: Machine learning methods estimate flexible relationships between confounders, treatments and outcomes, making unconfoundedness assumptions in causal analysis more palatable. How successful are these approaches in recovering ground truth baselines? We analyze new data covering an experimental rollout of a software product feature for one group of users and a voluntary opt-in to the same feature for another similar group. Recovering causal effects from nonrandomized data is feasible--but only with careful modeling choices. We offer best practices for more credible treatment effect estimation in modern, high-dimensional datasets and evaluate which tests provide meaningful signals about the reliability of estimated effects.

Date: 2026-01, Revised 2026-10
New Economics Papers: this item is included in nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://arxiv.org/pdf/2601.11845 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2601.11845

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().