Skip to content

Blind Testing in Firearms Examination: Preliminary Results, Benefits, Limitations, and Future Directions

Primary Author: Maddisen Neuman
Type: Webinar

This CSAFE webinar was held on June 23, 2022.

Maddisen Neuman
Quality / Research Associate, Houston Forensic Science Center


Presentation Description:
Open proficiency tests created by external vendors are used to meet accreditation requirements and demonstrate examiner competence; however, the tests are limited in the ability to monitor the entire quality management system from evidence submission to reporting of results. As such, many scholars have considered blind proficiency testing to be a more precise testing an individual’s accuracy (1).

In December 2015, the Houston Forensic Science Center (HFSC) implemented a blind quality control (blind QC) program in firearms examination. The intent of the blind QC program is to supplement open proficiency tests by creating mock cases that are submitted to the Firearms section as routine casework, with the expectation that the examiners are unaware that they are receiving test cases. Blind QC cases are created by Firearms section management and submitted by members of the Quality Division. Firearms section management evaluates the mock evidence prior to submission to determine the ground truth and expected results. Once the case is complete, the Firearms section management reviews the results to determine satisfactory completion.

This presentation will outline the results of 51 blind QC cases that were reported between December 2015 and June 2021 resulting in 570 sufficiency and comparison conclusions. No eliminations were reported for true matching pairs, and no identifications were reported for true non-matching pairs; thus, all results were considered satisfactory. In 40.3% (n=225) of the comparisons, the ground truth result was either elimination or identification, but an inconclusive conclusion was reached. Variables such as the examiners assigned to the case, training program, examiner experience level, intended complexity of the case, and evidence type were examined for their contributions to the inconclusive determinations. They type of evidence compared was determined to be the main factor contributing to inconclusive results; specifically, bullets were the main contributors to inconclusive results at a rate of 61.8% (n=168) compared to cartridge cases at a rate of 21.5% (n=57).

The webinar will conclude with a discussion on the rate of inconclusive conclusions, benefits and challenges of the blind QC program, and future directions for blind testing research in firearms examination. While a blind testing program can be challenging to implement, HFSC’s blind QC program demonstrates that the quality management system and procedures used by the Firearms section can obtain accurate and reliable results and provides examiners added confidence in court. Additionally, the blind QC program can be tailored to target specific research questions and provide opportunities for collaboration with other laboratories and researchers.

1. Committee on Identifying the Needs of the Forensic Sciences Community, National Research Council. Strengthening forensic science in the United States: A path forward. Washington, DC, 2009.

Webinar Recording:

Read more about the study in this post

Related Resources

The q–q Boxplot

The q–q Boxplot

Boxplots have become an extremely popular display of distribution summaries for collections of data, especially when we need to visualize summaries for several collections simultaneously. The whiskers in the boxplot…
Forensic Science in Legal Education

Forensic Science in Legal Education

In criminal cases, forensic science reports and expert testimony play an increasingly important role in adjudication. More states now follow a federal reliability standard, following Daubert v. Merrell Dow Pharmaceuticals…
The Contribution of Forensic and Expert Evidence to DNA Exoneration Cases: An Interim Report

The Contribution of Forensic and Expert Evidence to DNA Exoneration Cases: An Interim Report

This report is from Simon A. Cole, Vanessa Meterko, Sarah Chu, Glinda Cooper, Jessica Weinstock Paredes, Maurice Possley, and Ken Otterbourg (2022), The Contribution of Forensic and Expert Evidence to…
Likelihood ratios for categorical count data with applications in digital forensics

Likelihood ratios for categorical count data with applications in digital forensics

We consider the forensic context in which the goal is to assess whether two sets of observed data came from the same source or from different sources. In particular, we…