A Re-Analysis of Repeatability and Reproducibility in the Ames-USDOE-FBI Study

Alan H. Dorfman and Richard Valliant

Statistics and Public Policy, 2022, vol. 9, issue 1, 175-184

Abstract: Forensic firearms identification, the determination by a trained firearms examiner as to whether or not bullets or cartridges came from a common weapon, has long been a mainstay in the criminal courts. Reliability of forensic firearms identification has been challenged in the general scientific community, and, in response, several studies have been carried out aimed at showing that firearms examination is accurate, that is, has low error rates. Less studied has been the question of consistency, of whether two examinations of the same bullets or cartridge cases come to the same conclusion, carried out by an examiner on separate occasions—intrarater reliability or repeatability—or by two examiners—interrater reliability or reproducibility.One important study, described in a 2020 Report by the Ames Laboratory-USDOE to the Federal Bureau of Investigation, went beyond considerations of accuracy to investigate firearms examination repeatability and reproducibility. The Report’s conclusions were paradoxical. The observed agreement of examiners with themselves or with other examiners appears mediocre. However, the study concluded repeatability and reproducibility are satisfactory, on grounds that the observed agreement exceeds a quantity called the expected agreement. We find that appropriately employing expected agreement as it was intended does not suggest satisfactory repeatability and reproducibility, but the opposite.

Date: 2022
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/2330443X.2022.2120137 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:usppxx:v:9:y:2022:i:1:p:175-184

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/uspp20

DOI: 10.1080/2330443X.2022.2120137

Access Statistics for this article

Statistics and Public Policy is currently edited by Eric Sampson

More articles in Statistics and Public Policy from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().