 Identifying Prediction Mistakes in Observational Data*Ashesh Rambachan The Quarterly Journal of Economics, 2024, vol. 139, issue 3, 1665-1711 Abstract: Decision makers, such as doctors, judges, and managers, make consequential choices based on predictions of unknown outcomes. Do these decision makers make systematic prediction mistakes based on the available information? If so, in what ways are their predictions systematically biased? In this article, I characterize conditions under which systematic prediction mistakes can be identified in empirical settings such as hiring, medical diagnosis, and pretrial release. I derive a statistical test for whether the decision maker makes systematic prediction mistakes under these assumptions and provide methods for estimating the ways the decision maker’s predictions are systematically biased. I analyze the pretrial release decisions of judges in New York City, estimating that at least 20% of judges make systematic prediction mistakes about misconduct risk given defendant characteristics. Motivated by this analysis, I estimate the effects of replacing judges with algorithmic decision rules and find that replacing judges with algorithms where systematic prediction mistakes occur dominates the status quo. Date: 2024 Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: https://EconPapers.repec.org/RePEc:oup:qjecon:v:139:y:2024:i:3:p:1665-1711. Ordering information: This journal article can be ordered from Access Statistics for this article The Quarterly Journal of Economics is currently edited by Robert J. Barro, Lawrence F. Katz, Nathan Nunn, Andrei Shleifer and Stefanie Stantcheva More articles in The Quarterly Journal of Economics from President and Fellows of Harvard College |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
School of Business at Örebro University.