Scientist in lab coat and mask examines evidence at desk, writing notes, with "Investigating the Reliability of Firearm Examinations" text.

Posted August 26, 2022


A recent SciPod episode summarized the findings from a Center for Statistics and Applications in Forensic Evidence (CSAFE) publication on the reliability of firearm examinations. Hear the episode at https://www.scipod.global/dr-susan-vanderplas-investigating-the-reliability-of-firearm-examinations/.
The paper, “Treatment of Inconclusives in the AFTE Range of Conclusions,” was published in Law, Probability and Risk and written by Heike Hofmann, professor of statistics and professor-in-charge of the Data Science Program at Iowa State University; Susan Vanderplas, assistant professor of statistics at the University of Nebraska-Lincoln and corresponding author of the study; and Alicia Carriquiry, CSAFE director and a Distinguished Professor and President’s Chair in statistics at Iowa State.

In their study, Hofmann, Vanderplas and Carriquiry revisited previous black box studies that estimated the error rates of firearm examiners, focusing on their treatment of inconclusive results.

The SciPod episode explained that the treatment of inconclusive results when calculating firearm examiner error rates could significantly impact a study’s outcome. “If an inconclusive result is considered to be a decision that conveys with certainty the absence of conclusive data, then several inconclusive results would not translate into higher error rates. Contrarily, if they are perceived as incorrect results, then a high number of them would increase error rates.”

According to the podcast, “Overall, the researchers found that there were stark differences in the rates of inconclusive results in different studies and in the ways these results were treated in analyses. They also found that many estimations were unreliable or impossible to calculate, as they were based on problematic experimental procedures or poorly designed experiments.”

Hofmann, Vanderplas and Carriquiry concluded that many existing error rate estimates for firearm examinations are unreliable and emphasized the need for new studies that are based on large, representative samples of examiners who are asked to make several sets of single-comparison predictions.

The episode ended by discussing the researchers’ recommendations that would make studies in this area more reliable.

SciPod provides audio or video summaries of published scientific papers, allowing researchers to share their research with a wider audience. All episodes are free to watch or listen to and include downloadable transcripts and links to the original papers. More information about SciPod can be found at https://www.scipod.global.

Download the journal article and read insights from this study at https://forensicstats.org/blog/2020/10/20/insights-treatment-of-inconclusives-in-the-afte-range-of-conclusions/.

Hofmann, Vanderplas and Carriquiry discussed this study during a CSAFE-hosted webinar. Watch it at https://forensicstats.org/blog/portfolio/treatment-of-inconclusive-results-in-error-rates-of-firearm-studies/.

From the Blog

Journal article titled "Challenges in Modeling...Images as Forensic Evidence" by Kafadar and Carriquiry.

Two Journals for Sharing Your Forensic Science and Statistics Research

If you’re looking for a journal to showcase your latest research findings, consider exploring these …

NIST sign outside, surrounded by flowers and trees, on a sunny day.

NIST Report Highlights Key Challenges in Forensic Science and Provides Strategies for Addressing Them

In September, the National Institute of Standards and Technology (NIST) released a new report titled …

Recent Posts

ICFIS 2026 logo features a clock tower and the Forensic Inference and Statistics name.

Keynote Speakers Announced for ICFIS 2026

August 10, 2026

The Scientific Committee for the 12th International Conference on Forensic Inference and Statistics is pleased to announce the keynote speakers …

2026 Wisconsin NAFSB Conference logo on a dark blue background.

CSAFE Researchers will be Panel Speakers at NAFSB 2026 Conference

August 7, 2026

The annual National Association of Forensic Science Boards (NAFSB) Conference will be Oct. 13-16 at the Milwaukee Crime Lab Bureau …

ICFIS 2026 logo features a clock tower and the Forensic Inference and Statistics name.

ICFIS 2026 Moves Online, Abstract Deadline Extended to Aug. 17

July 27, 2026

The 12th International Conference on Forensic Inference and Statistics (ICFIS) will now be an online-only event, allowing participants from around the world …