A recent SciPod episode summarized the findings from a Center for Statistics and Applications in Forensic Evidence (CSAFE) publication on the reliability of firearm examinations. Hear the episode at https://www.scipod.global/dr-susan-vanderplas-investigating-the-reliability-of-firearm-examinations/.
The paper, “Treatment of Inconclusives in the AFTE Range of Conclusions,” was published in Law, Probability and Risk and written by Heike Hofmann, professor of statistics and professor-in-charge of the Data Science Program at Iowa State University; Susan Vanderplas, assistant professor of statistics at the University of Nebraska-Lincoln and corresponding author of the study; and Alicia Carriquiry, CSAFE director and a Distinguished Professor and President’s Chair in statistics at Iowa State.
In their study, Hofmann, Vanderplas and Carriquiry revisited previous black box studies that estimated the error rates of firearm examiners, focusing on their treatment of inconclusive results.
The SciPod episode explained that the treatment of inconclusive results when calculating firearm examiner error rates could significantly impact a study’s outcome. “If an inconclusive result is considered to be a decision that conveys with certainty the absence of conclusive data, then several inconclusive results would not translate into higher error rates. Contrarily, if they are perceived as incorrect results, then a high number of them would increase error rates.”
According to the podcast, “Overall, the researchers found that there were stark differences in the rates of inconclusive results in different studies and in the ways these results were treated in analyses. They also found that many estimations were unreliable or impossible to calculate, as they were based on problematic experimental procedures or poorly designed experiments.”
Hofmann, Vanderplas and Carriquiry concluded that many existing error rate estimates for firearm examinations are unreliable and emphasized the need for new studies that are based on large, representative samples of examiners who are asked to make several sets of single-comparison predictions.
The episode ended by discussing the researchers’ recommendations that would make studies in this area more reliable.
SciPod provides audio or video summaries of published scientific papers, allowing researchers to share their research with a wider audience. All episodes are free to watch or listen to and include downloadable transcripts and links to the original papers. More information about SciPod can be found at https://www.scipod.global.
Download the journal article and read insights from this study at https://forensicstats.org/blog/2020/10/20/insights-treatment-of-inconclusives-in-the-afte-range-of-conclusions/.
Hofmann, Vanderplas and Carriquiry discussed this study during a CSAFE-hosted webinar. Watch it at https://forensicstats.org/blog/portfolio/treatment-of-inconclusive-results-in-error-rates-of-firearm-studies/.
