Skip to content

Treatment of Inconclusive Results in Error Rates of Firearm Studies

Type: Webinar
Research Area: Firearms and Toolmarks

This CSAFE webinar was held on February 10, 2021.

Presenters:

Heike Hofmann
Professor and Kingland Faculty Fellow, Iowa State University

Susan VanderPlas
Research Assistant Professor, University of Nebraska, Lincoln

Alicia Carriquiry
Distinguished Professor and President’s Chair in Statistics, CSAFE Director

Presentation Description:

In the past decade, and in response to the recommendations set forth by the National Research Council Committee on Identifying the Needs of the Forensic Sciences Community [2009], scientists have conducted several black-box studies that attempt to estimate the error rates of firearm examiners. Most of these studies have resulted in vanishingly small error rates, and at least one of them [Baldwin et al., 2014] was cited by the President’s Council of Advisors in Science and Technology (PCAST) during the Obama administration, as an example of a well-designed experiment. What has received little attention, however, is the actual calculation of error rates, and in particular, the effect of inconclusive findings on those error estimates. The treatment of inconclusive results in the assessment of errors has far-reaching implications in the legal system. We will revisit several black-box studies in the area of firearms examination, investigating their treatment of inconclusive results. It is clear that there are stark differences in the rate of inconclusive results in regions with different norms for training and reporting conclusions. More surprisingly, the rate of inconclusive decisions for materials from different sources is notably higher than the rate of inconclusive decisions for same-source materials in some regions. To mitigate the effects of this difference we propose a unifying approach to the calculation of error rates that is directly applicable in forensic laboratories and legal settings.

Associated Reading:

Webinar Q&A: Treatment of Inconclusive Results in Error Rates of Firearms Studies

Insights: Treatment of Inconclusives in the AFTE Range of Conclusions

Related Resources

Surveying Practicing Firearm Examiners

Surveying Practicing Firearm Examiners

A sample (n = 79) of practicing firearm and toolmark examiners was queried about casework as well as their views about the potential role that statistics might play in future…
Blind testing in firearms: Preliminary results from a blind quality control program

Blind testing in firearms: Preliminary results from a blind quality control program

Open proficiency tests meet accreditation requirements and measure examiner competence but may not represent actual casework. In December 2015, the Houston Forensic Science Center began a blind quality control program…
Ruger LCP Study: a two-pronged approach

Ruger LCP Study: a two-pronged approach

This study started as an attempt to take a closer look at the rifling observed in the 380 Auto Ruger LCP pistol
Treatment of inconclusives in the AFTE range of conclusions

Treatment of inconclusives in the AFTE range of conclusions

In the past decade, and in response to the recommendations set forth by the National Research Council Committee on Identifying the Needs of the Forensic Sciences Community (2009), scientists have…