Scientific research is driven by our ability to use methods, procedures, and materials from previous studies and further research by adding to it. As the need for computationally-intensive methods to analyze large amounts of data grows, the criteria needed to achieve reproducibility, specifically computational reproducibility, have become more sophisticated. In general, prosaic descriptions of algorithms are not detailed or precise enough to ensure complete reproducibility of a method. Results may be sensitive to conditions not commonly specified in written-word descriptions such as implicit parameter settings or the programming language used. To achieve true computational reproducibility, it is necessary to provide all intermediate data and code used to produce published results. In this paper, we consider a class of algorithms developed to perform firearm evidence identification on cartridge case evidence known as the Congruent Matching Cells (CMC) methods. To date, these algorithms have been published as textual descriptions only. We introduce the first open-source implementation of the Congruent Matching Cells methods in the R package cmcR. We have structured the cmcR package as a set of sequential, modularized functions intended to ease the process of parameter experimentation. We use cmcR and a novel variance ratio statistic to explore the CMC methodology and demonstrate how to fill in the gaps when provided with computationally ambiguous descriptions of algorithms.
A Study in Reproducibility: The Congruent Matching Cells Algorithm and cmcR Package
Journal: The R Journal
Published: 2023
Primary Author: Joseph Zemmels
Secondary Authors: Susan Vanderplas, Heike Hofmann
Type: Publication
Research Area: Firearms and Toolmarks
Related Resources
Forensic Toolmark Comparisons
Forensic practitioners determine whether two marks were generated by the same tool by observing the 2D images of the marks using a comparison microscope and deciding whether the “surface contours…
Effect of Subclass Characteristics on Congruent Matching Cells (CMC) Algorithm
This presentation is a continuation of subclass characteristics present on consecutively manufactured breech face inserts. The objective is to assess manufacturing methods similar to those used by firearm manufacturers. It…
Diagnostic Tools for Automatic Cartridge Case Comparisons
The following was presented at the Association of Firearm and Tool Mark Examiners (AFTE) 2023, Austin, Texas, May 21-26, 2023. Copyright 2023, The Authors. Posted with permission of CSAFE.
CSAFE Project Update & ASCLD FRC Collaboration
This presentation highlighted CSAFE’s collaboration with the ASCLD FRC Collaboration Hub.