Skip to content

Treatment of inconclusives in the AFTE range of conclusions

Journal: Law, Probability and Risk
Published: 2021
Primary Author: Heike Hofmann
Secondary Authors: Susan Vanderplas, Alicia Carriquiry
Research Area: Firearms and Toolmarks

In the past decade, and in response to the recommendations set forth by the National Research Council Committee on Identifying the Needs of the Forensic Sciences Community (2009), scientists have conducted several black-box studies that attempt to estimate the error rates of firearm examiners. Most of these studies have resulted in vanishingly small error rates, and at least one of them (D. P. Baldwin, S. J. Bajic, M. Morris, and D. Zamzow. A Study of False-Positive and False-Negative Error Rates in Cartridge Case Comparisons. Technical report, Ames Lab IA, Performing, Fort Belvoir, VA, April 2014.) was cited by the President’s Council of Advisors in Science and Technology (PCAST) during the Obama administration, as an example of a well-designed experiment. What has received little attention, however, is the actual calculation of error rates and in particular, the effect of inconclusive findings on those error estimates. The treatment of inconclusives in the assessment of errors has far-reaching implications in the legal system. Here, we revisit several black-box studies in the area of firearms examination, investigating their treatment of inconclusive results. It is clear that there are stark differences in the rate of inconclusive results in regions with different norms for training and reporting conclusions. More surprisingly, the rate of inconclusive decisions for materials from different sources is notably higher than the rate of inconclusive decisions for same-source materials in some regions. To mitigate the effects of this difference we propose a unifying approach to the calculation of error rates that is directly applicable in forensic laboratories and in legal settings.

Related Resources

Using Machine Learning Methods to Predict Similarity of Striations on Bullet Lands

Using Machine Learning Methods to Predict Similarity of Striations on Bullet Lands

Recent advances in microscopy have made it possible to collect 3D topographic data, enabling virtual comparisons based on the collected 3D data next to traditional comparison microscopy. Automatic matching algorithms…
CSAFE 2021 Field Update

CSAFE 2021 Field Update

The 2021 Field Update was held June 14, 2021, and served as the closing to the first year of CSAFE 2.0. CSAFE brought together researchers, forensic science partners and interested…
Treatment of Inconclusive Results in Error Rates of Firearm Studies

Treatment of Inconclusive Results in Error Rates of Firearm Studies

This CSAFE webinar was held on February 10, 2021. Presenters: Heike Hofmann Professor and Kingland Faculty Fellow, Iowa State University Susan VanderPlas Research Assistant Professor, University of Nebraska, Lincoln Alicia…
CSAFE 2020 All Hands Meeting

CSAFE 2020 All Hands Meeting

The 2020 All Hands Meeting was held May 12 and 13, 2020 and served as the closing to the last 5 years of CSAFE research and focused on kicking off…