INSIGHT

Implementing Blind Proficiency Testing in Forensic Laboratories:

Motivation, Obstacles, and Recommendations

OVERVIEW

Accredited forensic laboratories are required to conduct proficiency testing –– but most rely solely on declared proficiency tests. A 2014 study showed that only 10% of forensic labs in the United States performed blind proficiency testing, whereas blind tests are standard in other fields including medical and drug testing laboratories. Researchers wanted to identify the barriers to widespread blind proficiency testing and generate solutions to removing these obstacles. After reviewing the existing research, they realized they must convene a meeting of experts to establish an understanding of the challenges to implementation.

Lead Researchers

Robin Mejia
Maria Cuellar
Jeff Salyards

Journal

Forensic Science International: Synergy

Publication Date

September 2020

Publication Number

IN 110 LP

Participants

CSAFE met with laboratory directors and quality managers from seven forensic laboratory systems in the eastern US and the Houston Forensic Science Center. Two of the quality managers represented the Association of Forensic Quality Assurance Managers (AFQAM). In addition, several professors, graduate students and researchers from three universities attended the meeting.

APPROACH AND METHODOLOGY

1

Compare blind proficiency testing to declared testing then have participants discuss the potential advantages of establishing blind testing as standard.

2

Facilitate and document a discussion of the logistical and cultural barriers labs might face when adopting blind testing. Use this to create a list of challenges.

3

Collect and analyze suggested steps labs can take to overcome the list of challenges to implementing blind proficiency testing.

Challenges and Solutions

Challenge	Proposed Solution
Realistic test case creation can be complex.	Quality managers develop the expertise to create test cases; laboratories create a shared evidence bank.
The development of realistic submission materials may be difficult.	The QA staff must develop the knowledge locally to ensure the test evidence conforms with a jurisdiction’s typical cases.
Cost may be prohibitively expensive.	Multiple laboratories can share resources and make joint purchases; external test providers could develop materials to lower the cost.
Test must be submitted to the lab by an outside LEA.	Choosing which law enforcement agency (LEA) to work with should be decided locally based on the relationship between lab management and the LEA.
Not all LIMS are equipped to easily flag and track test cases.	Labs can choose to either use a Laboratory Information Management System (LIMS) with this functionality or develop an in-house system to flag test cases.
Labs must ensure results are not released as real cases.	The QA team will need to work with individuals in other units of the lab to prevent accidental releases. It may also be useful to have contacts in the submitting LEA or local District Attorney’s office.
Proficiency tests could impact metrics, so labs need to decide whether to include them.	These decisions must be made on a lab-by-lab basis; a consortium of labs or organizations such as AFQAM can aid in standardization.
Blind testing challenges the cultural myth of 100% accuracy.	Senior lab management must champion blind testing and show that adding it as a tool will demonstrate both the quality of examiners and help labs discover and remedy errors.