Latent Print Quality in Blind Proficiency Testing: Using Quality Metrics to Examine Laboratory Performance
June 2, 2021 10:09 am
Lead Researchers
Brett O. Gardner
Maddisen Neuman
Sharon Kelley
Journal
Forensic Science International
Publication Date
May 7, 2021
Publication Number
IN 122 LP
Overview
Organizations and leaders continuously call for blind proficiency testing in modern forensic labs because it more accurately simulates routine examiner casework. In response, researchers funded by CSAFE worked with the Houston Forensic Science Center to assess the results of their blind quality control program and then applied quality metrics to the test materials to see if the quality of the prints impacted their conclusions.
Lead Researchers
Goals
1
Examine the results of blind proficiency testing within a fingerprint examination unit of a crime laboratory.
2
Use available quality metrics software to measure the quality of the submitted prints.
2
See if there is an association between fingerprint quality and examiners’ conclusions.
The Studies
The Quality Division at the Houston Forensic Science Center inserted 376 prints into 144 blind test cases over a two-year period. In these cases, examiners determined if the prints were of sufficient quality to search in their Automated Fingerprint Identification System (AFIS). After searching AFIS for corresponding prints, they concluded whether or not the test prints were similar enough to make a Primary AFIS Association (PAA). Then, the Blind Quality Control (BQC) team judged the examiners’ accuracy.
Meanwhile, Gardner et al. entered the same test prints into LQMetrics –– a commonly used software tool for fingerprint examiners that rates the quality of a print image on a scale of 0–100. The team scored print images with a quality score greater than 65 as “Good,” 45–65 as “Bad,” and lower than 45 as “Ugly.”
The Study Results
- Over 92% of latent prints were of sufficient quality to enter into AFIS and only 1.6% of latent prints were of no comparative value.
- Examiners committed zero false positive errors and generated only two false negatives in instances where AFIS returned the correct candidate.
- However, examiners judged that 41% of the test prints had no match, despite the source being in AFIS. These false negatives were largely from prints whose quality was “Bad” or “Ugly” based on LQMetrics scoring.
- “Good” prints were more than twice as likely to result in correct conclusions than “Ugly” ones, while “Ugly” prints were 370% more likely to result in incorrect conclusions than “Good” ones.
Examiner Conclusions Among Good, Bad and Ugly Latent Prints
Print quality (as categorized by Good, Bad and Ugly) was significantly associated with examiner conclusions and ultimate accuracy. Note: N = 373 prints. There were 133 Good prints, 114 Bad prints and 126 Ugly prints
Focus on the future
The correct source for prints submitted to AFIS appeared in the top ten results only 41.7% of the time, lower than an estimated 53.4% of the time based on the quality of such prints. This highlights the potential for blind proficiency testing to gauge the accuracy of the entire system –– including AFIS.
This study only included prints that had the potential to be submitted to AFIS, dismissing images not labeled as latent prints. Future studies should include a full range of images to more closely reflect real-world casework.