Accuracy and reproducibility of bullet comparison decisions by forensic examiners

Abstract

Few previous studies have assessed the accuracy and reproducibility of bullet comparison decisions by firearms examiners, and none have evaluated accuracy of examiners’ decisions when comparing damaged bullets, comparisons of questioned bullets, or the effects on decision rates of using jacketed hollow-point vs. full metal jacket bullets. In this study, 49 practicing forensic firearms examiners conducted 3156 comparisons of bullets, including bullets ranging in quality, bullets from different types of ammunition, and bullets fired from various makes/models of firearms. The study evaluated two scenarios commonly used in casework: questioned-questioned (QQ) comparisons of two bullets from unknown sources, and known-questioned (KQ) comparisons in which a bullet from an unknown source is compared to three known exemplars from a single firearm. Key findings: after controlling for other factors, QQ vs. KQ comparisons had relatively limited effects on decision rates; rates of inconclusive responses were inversely related to bullet quality; bullets fired from polygonally-rifled pistols resulted in more inconclusive or unsuitable responses than conventional rifling; on nonmated comparison sets, the rate of (true) exclusions was particularly high when comparing different caliber bullets, and was higher on comparisons of different makes/models of firearms vs. the same model of firearm; comparisons in which different types of ammunition were fired from the same firearm had a high rate of erroneous exclusions; decision rates differed notably by firearm model; decision rates varied notably among the participants. Because the measured rates vary dramatically due to these various factors, we recommend against using overall decision rates to summarize the results of this study.

Significance statement

Comparisons of bullets by forensic firearms examiners are a critical part of the criminal justice system, seeking to determine whether recovered bullets from crime scenes can be attributed to specific firearms. This paper reports the results of a research study designed to assess the accuracy and reliability of forensic bullet comparison decisions, which is important to assess scientific validity for admissibility in court. Few studies have been conducted assessing the accuracy of forensic bullet comparison decisions, and no previous studies have evaluated the accuracy and reproducibility of decisions made by practicing forensic firearms examiners when comparing bullets of varying quality or bullets of different types, or when making comparisons of questioned bullets.