This CSAFE webinar was held on June 23, 2022.
Presenters:
Maddisen Neuman
Quality / Research Associate, Houston Forensic Science Center
Presentation Description:
Open proficiency tests created by external vendors are used to meet accreditation requirements and demonstrate examiner competence; however, the tests are limited in the ability to monitor the entire quality management system from evidence submission to reporting of results. As such, many scholars have considered blind proficiency testing to be a more precise testing an individual’s accuracy (1).
In December 2015, the Houston Forensic Science Center (HFSC) implemented a blind quality control (blind QC) program in firearms examination. The intent of the blind QC program is to supplement open proficiency tests by creating mock cases that are submitted to the Firearms section as routine casework, with the expectation that the examiners are unaware that they are receiving test cases. Blind QC cases are created by Firearms section management and submitted by members of the Quality Division. Firearms section management evaluates the mock evidence prior to submission to determine the ground truth and expected results. Once the case is complete, the Firearms section management reviews the results to determine satisfactory completion.
This presentation will outline the results of 51 blind QC cases that were reported between December 2015 and June 2021 resulting in 570 sufficiency and comparison conclusions. No eliminations were reported for true matching pairs, and no identifications were reported for true non-matching pairs; thus, all results were considered satisfactory. In 40.3% (n=225) of the comparisons, the ground truth result was either elimination or identification, but an inconclusive conclusion was reached. Variables such as the examiners assigned to the case, training program, examiner experience level, intended complexity of the case, and evidence type were examined for their contributions to the inconclusive determinations. They type of evidence compared was determined to be the main factor contributing to inconclusive results; specifically, bullets were the main contributors to inconclusive results at a rate of 61.8% (n=168) compared to cartridge cases at a rate of 21.5% (n=57).
The webinar will conclude with a discussion on the rate of inconclusive conclusions, benefits and challenges of the blind QC program, and future directions for blind testing research in firearms examination. While a blind testing program can be challenging to implement, HFSC’s blind QC program demonstrates that the quality management system and procedures used by the Firearms section can obtain accurate and reliable results and provides examiners added confidence in court. Additionally, the blind QC program can be tailored to target specific research questions and provide opportunities for collaboration with other laboratories and researchers.
1. Committee on Identifying the Needs of the Forensic Sciences Community, National Research Council. Strengthening forensic science in the United States: A path forward. Washington, DC, 2009.
Webinar Recording:
