Representatives from the CSAFE research team recently organized the workshop “Blinding in Forensic Proficiency Testing and Casework” in November 2018 in Pittsburgh, Pennsylvania. The meeting hosted by Allegheny County Medical Examiner’s drew attendees from the three primary forensic laboratory systems in Pennsylvania as well as from several other states. CSAFE researchers from Carnegie Mellon University and the University of Virginia also attended the event.
During the event CSAFE researchers provided expertise in experimental design, statistics and psychology as they relate to blinding in forensic laboratories. Our team discussed how academic researchers can support laboratories that have already implemented blind testing, those who are in the process of implementation and labs simply interested in learning more. A workshop report is now available for review.
Proficiency Testing Versus Blind Testing- Learning the Difference
The meeting began with introductions by CSAFE co-director Dr. William F. Eddy and Chief Medical Examiner Karl Williams. Workshop co-organizer, assistant professor at University of Pennsylvania and CSAFE researcher Dr. Maria Cuellar then explained the difference between proficiency testing and blind testing.
“Blind testing means submitting known samples as if it were regular casework to test the procedures and error rates of examiners,” she said.
Proficiency testing can help laboratories find mistakes, but examiners know they are being tested. This means that they will possibly behave differently than they do in everyday casework.
Why Blind Testing Matters to Forensic Laboratories
CSAFE Senior Advisory Board member and Principal Analyst at Compass Scientific Consulting Dr. Jeff Salyards highlighted the importance of blind proficiency testing for forensic laboratories.
“There is no better way for laboratory leaders to learn about the performance of the laboratory than a true blind proficiency test,” Salyards said. “In addition to the obvious information about scientific accuracy, these tests reveal many other aspects about how a laboratory is functioning from workflow to customer service.”
Workshop co-organizer, CSAFE researcher and Manager of the Statistics and Human Rights Center at Carnegie Mellon University Dr. Robin Mejia states that implementing blind testing can improve a lab’s ability to assess quality issues.
Mejia said, “Blind proficiency testing could also provide preliminary data on examiner performance in situations designed to mimic case work.” She also notes that the field should continue to pursue error rate studies.
Addressing Challenges and Increased Collaborations
Dr. Peter Stout, President and CEO of the Houston Forensic Science Center and valued CSAFE partner led a panel discussion on implementing blind proficiency testing in forensic laboratories. The key takeaway according to Cuellar was that there is widespread interest from forensic laboratories in implementing blind testing as a complement to regular proficiency testing.
“In workshop presentations and discussions, participants described the obstacles they’d encountered in implementing blind testing and how they’d overcome those obstacles,” Mejia said.
Workshop attendees launched discussions about how laboratories interested in implementing blind testing could collaborate with each other and researchers.
“The connections made between representatives from forensic laboratories and academia will allow both sides to make well-informed decisions and plans for implementing blind testing,” Cuellar said.
The workshop concluded with a discussion on best practices, research needs and practical steps to take moving forward.
Moving Proficiency Testing Forward
“The future is bright as more and more laboratory leaders see value of blind proficiency testing. Hopefully, this workshop will grow into a consortium of blind proficiency early-adopters,” Salyards said.
“This was an incredibly productive workshop and we look forward to continuing the discussion,” Mejia commented.
Discover CSAFE’s Role in Additional Workshops
To learn more about CSAFE workshops with forensic science applications, visit our news archives for details on workshops in digital evidence, statistical training for forensic scientists, forensic databases, and error rates.
Interested in collaborating with CSAFE to host a training for your team of forensic science professionals? Visit our website to request assistance with an event.
