CSAFE researchers have made great strides in developing statistical and scientific foundations for assessing and matching firearms and toolmarks. Firearm and toolmark examination, like many other pattern analysis disciplines, is currently grounded in the subjective visual comparison of evidence relying on examiner training. Automatic matching algorithms, like those CSAFE is developing, allow an objective assessment of whether a set of pattern evidence comes from the same source or different sources and, unlike subjective assessments, also provides more quantitative detail about the degree of similarity between the items which are compared.
Iowa State University
West Virginia University
University of Pennsylvania
Additional Team Members
Susan Vanderplas email@example.com
Robin Mejia firstname.lastname@example.org
Curtis Mosher email@example.com
Alan Zeng (NIST) firstname.lastname@example.org
Robert Thompson (NIST) email@example.com
- (1) Developing methods to acquire and analyze 360° scans, allowing us to apply the matching algorithm to firearms with polygonal rifling
- (2) Exploring factors affecting the matching algorithm’s performance. It is well known that different types of firearms mark differently. An objective assessment allows us to quantify this difference. Leveraging ongoing collaborations with crime labs, we will assess and quantify the quality of marks made by different firearms on different types of ammunition. We will also use the same techniques to assess the quality of marks in a single scan to identify areas which are suitable for further analysis.
- (3) Transitioning the algorithm from research to practice. The matching algorithms have been developed to augment and support examiner assessment of evidence; they are not designed to replace the need for examiner expertise. We anticipate working with crime labs to explore avenues for funding high-resolution microscopy capabilities in forensic labs, and provide training to examiners in the use of the algorithm in casework while we fine-tune the algorithm’s performance in a challenging, high-stakes environment.
The aim of this research is to determine the accuracy and reproducibility of firearms examiners in identifying subclass characteristics and separating subclass from individual characteristics on firearm evidence. This study will show the ability of examiners to identify subclass characteristics that should be avoided when performing a comparison. The consistency of examiners in identifying subclass characteristics will also be determined. At the conclusion of this study, double-casts or digital files may be requested for teaching and training purposes.
The aim of this research proposal is to determine the accuracy and reproducibility of firearms examiners in identifying subclass characteristics and separating subclass from individual characteristics on firearm evidence. A firearm with the capability of a changeable breech face will be used throughout this study. This allows for many breech faces to be manufactured utilizing the different manufacturing techniques typically used by firearm manufacturers (e.g., broaching, milling and turning). Along with the manufacturing techniques being used, several finishing methods will be used to complete the manufacturing process. Test fires will be collected to examine the reproduction of potential subclass characteristics onto fired cartridge cases. A Sensofar® S neox 3D optical profiler will be used to collect high-quality topographic scans of both the breech face inserts and the cartridge cases. These scans will be compared using the NIST congruent matching cells (CMC) algorithm through an ongoing collaboration to analyze the effects of subclass characteristics on objective comparison algorithms. Replications of the collected cartridge cases and the breech face inserts will be produced using a double-casting procedure. This allows examiners who participate in this study to examine the same surfaces with the same features as other examiners. This study will show the ability of examiners to identify subclass characteristics that should be avoided when performing a comparison. The consistency of examiners in identifying subclass characteristics will also be determined. All physical cartridge cases and manufactured breech faces will be retained along with their digital surface scans. At the conclusion of this study, double-casts or digital files may be requested for teaching and training purposes.
Forensic laboratories will be contacted to request permission to test fire firearms from their reference collections to be included in a database as well as to acquire a wide variety of firearms to test for subclass characteristics. Four calibers will be the main focus of this study, including the common pistol calibers of 9 mm Luger, .40 S&W, and .45 ACP, and the rifle caliber .223 Remington/5.56 mm x 45 mm due to the popularity of the AR-15 semi-automatic rifle and its clones. Samples will be collected from the reference collections of five forensic laboratories. After collection, the researchers begin the acquisition of the cartridge cases, running the NIST congruent matching cell (CMC) algorithm, saving crops of 3D topography images and uploading images to the NIST database. Based on our predictions, by the end of this study, 15 cartridge cases from approximately 3,000 firearms will be included in the database. This database will be made publicly available through NIST. It can be used for research and evidence interpretation. This database would allow for error rate studies of firearm examiner performance and the assessment of automated comparison systems. It would also allow for the assessment similarity metrics, such as likelihood ratios, for firearm evidence. Additionally, it will provide a relevant background database for forensic laboratories when they adopt 3D microscopy as a primary comparison tool.
Similar to firearms, the determination of whether a suspect tool (e.g., screwdriver, wire cutter) is the source of a crime scene toolmark is currently a subjective decision. CSAFE is undertaking the development of a database of toolmark images (2D and 3D) and will use the database to carry out a black-box study in collaboration with partners. This will allow us to estimate the error rate of toolmark analysts. The initial database will focus on one type of tool while varying other features.
Forensic toolmark examiners need a statistical method to determine how likely it is that two toolmarks were created by the same tool, as opposed to by different tools, that is, a statistical method for analyzing toolmarks. (1) To develop such a method, it is first necessary to create a high-quality database of toolmarks. If approved, in this project we will create a database of toolmarks by using screwdrivers — one of the simplest tools that is commonly used in crimes and that can serve as an initial step in statistical models of toolmark examination. The database will have a factorial design with 2D and 3D images of marks made by different tools, at different angles, replicated several times, as inspired by the NIST NBIDE database (this allows for evaluating within- and between-tool variation). The database will be made available to the public for further research. (2) After the database is completed, we will develop an algorithm to determine a score-based likelihood ratio, which compares the likelihood that the toolmark was created by the same tool and the likelihood that it was not. This algorithm will build upon the work in firearms by CSAFE in its first five years. (3) Finally, we will validate the algorithm by testing its external validity. We will acquire toolmarks created by forensic laboratories for which the analysts know the right answer and we do not, so we can test the performance of our algorithm. The CSAFE firearms algorithms have been validated in this way successfully in the past. Finally, the creation of an algorithm for toolmark analysis, developed by using the work previously done by CSAFE in firearms, will help us learn how to expand the algorithm approach to other forensic disciplines.
Page 1 of 2
Published: 2023 | By: Joseph Zemmels
Scientific research is driven by our ability to use methods, procedures, and materials from previous studies and further research by adding to it. As the need for computationally-intensive methods to analyze large amounts of data grows, the criteria needed to…
Published: 2022 | By: Jeff Salyards
This presentation highlighted CSAFE's collaboration with the ASCLD FRC Collaboration Hub.
Published: 2023 | By: Brandon L. Garrett
Firearms violence results in hundreds of thousands of criminal investigations each year. To try to identify a culprit, firearms examiners seek to link fired shell casings or bullets from crime scene evidence to a particular firearm. The underlying assumption is…
Published: 2023 | By: Maddisen Neuman
This workshop provides a summary of the Houston Forensic Science Center's blind quality control program, focusing on blind quality control and blind verification in the Firearms section. The presenters will share their experiences- successes and failures- to best inform other…
Published: 2022 | By: Wangqian Ju
In this paper, we introduce the R package cmpsR, an open-source implementation of the Congruent Matching Profile Segments (CMPS) method developed at the National Institute of Standards and Technology (NIST) for objective comparison of striated tool marks. The functionality of…
Published: 2022 | By: Joseph Zemmels
We introduce novel sub-procedures in the pre-processing, comparison and similarity feature steps. The sub-procedures result in an improved error rate compared to our implementation of the CMC method while also requiring fewer parameters. The fitted Decision tree model is easily…
Published: 2022 | By: Veronica Franklin
The following is a presentation given at Association for Firearms and Toolmarks (AFTE) (2022).
Published: 2022 | By: Maria Cuellar
Forensic handheld toolmark examiners currently compare toolmarks (e.g. scratch marks on wire found as part of an explosive device, or on a door frame after someone broke into a house) by observing 2D images and subjectively deciding whether two marks…
Published: 2021 | By: Veronica Franklin
A webinar presented during the Forensic Technology Center of Excellence (FTCoE0 Firearm and Toolmark webinar series on the progress of our CSAFE project. This study assesses the production of subclass characteristics during the manufacturing of breech faces by three different…
Published: 2022 | By: Nicholas Scurich
A sample (n = 79) of practicing firearm and toolmark examiners was queried about casework as well as their views about the potential role that statistics might play in future firearm examinations and expert witness testimony. Principal findings include: The…
Published: 2022 | By: Maddisen Neuman
Open proficiency tests meet accreditation requirements and measure examiner competence but may not represent actual casework. In December 2015, the Houston Forensic Science Center began a blind quality control program in firearms examination. Mock cases are created to mimic routine…
Published: 2021 | By: Melissa Nally
This study started as an attempt to take a closer look at the rifling observed in the 380 Auto Ruger LCP pistol
Published: 2021 | By: Heike Hofmann
In the past decade, and in response to the recommendations set forth by the National Research Council Committee on Identifying the Needs of the Forensic Sciences Community (2009), scientists have conducted several black-box studies that attempt to estimate the error…
Published: 2020 | By: Heike Hofmann
Recent advances in microscopy have made it possible to collect 3D topographic data, enabling virtual comparisons based on the collected 3D data next to traditional comparison microscopy. Automatic matching algorithms have been introduced for various scenarios, such as matching cartridge…
The 2021 Field Update was held June 14, 2021, and served as the closing to the first year of CSAFE 2.0. CSAFE brought together researchers, forensic science partners and interested community members to highlight the organization’s achievements, identify areas for…
This CSAFE webinar was held on February 10, 2021. Presenters: Heike Hofmann Professor and Kingland Faculty Fellow, Iowa State University Susan VanderPlas Research Assistant Professor, University of Nebraska, Lincoln Alicia Carriquiry Distinguished Professor and President's Chair in Statistics, CSAFE Director…
The 2020 All Hands Meeting was held May 12 and 13, 2020 and served as the closing to the last 5 years of CSAFE research and focused on kicking off new initiatives for the next phase of the center, CSAFE…
Published: 2020 | By: Susan VanderPlas
Recent advances in microscopy have made it possible to collect 3D topographic data, enabling more precise virtual comparisons based on the collected 3D data as a supplement to traditional comparison microscopy and 2D photography. Automatic comparison algorithms have been introduced…
Published: 2019 | By: Kiegan Rice
Land engraved areas (LEAs) provide evidence to address the same source–different source problem in forensic firearms examination. Collecting 3D images of bullet LEAs requires capturing portions of the neighboring groove engraved areas (GEAs). Analyzing LEA and GEA data separately is…
CSAFE is a NIST Center of Excellence in Forensic Science. A large portion of CSAFE’s research portfolio is on what is known as pattern evidence, which encompasses any evidence that can be represented as an image. Examples of pattern evidence…
Page 1 of 2
Want to collaborate with CSAFE on a project. Contact us to share your idea.