Shining a Light on Forensic Black-Box Studies

Kori Khan and Alicia L. Carriquiry

Statistics and Public Policy, 2023, vol. 10, issue 1, 2216748

Abstract: Forensic science plays a critical role in the United States criminal legal system. For decades, many feature-based fields of forensic science, such as firearm and toolmark identification, developed outside the scientific community’s purview. The results of these studies are widely relied on by judges nationwide. However, this reliance is misplaced. Black-box studies to date suffer from inappropriate sampling methods and high rates of missingness. Current black-box studies ignore both problems in arriving at the error rate estimates presented to courts. We explore the impact of each type of limitation using available data from black-box studies and court materials. We show that black-box studies rely on unrepresentative samples of examiners. Using a case study of a popular ballistics study, we find evidence that these nonrepresentative samples may commit fewer errors than the wider population from which they came. We also find evidence that the missingness in black-box studies is non-ignorable. Using data from a recent latent print study, we show that ignoring this missingness likely results in systematic underestimates of error rates. Finally, we offer concrete steps to overcome these limitations. Supplementary materials for this article areavailable online.

Date: 2023
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/2330443X.2023.2216748 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:usppxx:v:10:y:2023:i:1:p:2216748

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/uspp20

DOI: 10.1080/2330443X.2023.2216748

Access Statistics for this article

Statistics and Public Policy is currently edited by Eric Sampson

More articles in Statistics and Public Policy from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().