Skip to content

Firearms and Toolmarks Analysis

Overarching GOALS

CSAFE researchers have made great strides in developing statistical and scientific foundations for assessing and matching firearms and toolmarks. Firearm and toolmark examination, like many other pattern analysis disciplines, is currently grounded in the subjective visual comparison of evidence relying on examiner training. Automatic matching algorithms, like those CSAFE is developing, allow an objective assessment of whether a set of pattern evidence comes from the same source or different sources and, unlike subjective assessments, also provides more quantitative detail about the degree of similarity between the items which are compared. 

Heiki Hofmann

Heike Hofmann

Professor and Kingland Faculty Fellow

Iowa State University

Morris,-Keith

Keith Morris

Professor, Ming Hsieh Distinguished Teaching Professor of Forensic and Investigative Science, Co-Director of CSAFE

West Virginia University

Maria Cuellar

Maria Cuellar

Assistant Professor

University of Pennsylvania

Additional Team Members

Susan Vanderplas susan.vanderplas@unl.edu

Robin Mejia  rmejia@andrew.cmu.edu

Curtis Mosher cmosher@iastate.edu

Alan Zeng (NIST) alan.zheng@nist.gov

Robert Thompson (NIST) robert.m.thompson@nist.gov

focus Areas

In the last five years, CSAFE has made great strides in assessing striae similarity of bullet striations fired from traditionally rifled barrels using automatic matching algorithms which incorporate both mathematical correlations and features used by examiners. We will build on this existing foundation by expanding the scope of this matching algorithm and widen its applicability along several different dimensions. Firearm and toolmark examination, like many other pattern analysis disciplines, is currently grounded in subjective visual comparison of evidence relying on examiner training. Automatic matching algorithms allow an objective assessment of whether a set of pattern evidence comes from the same source or different sources and, unlike subjective assessments, also provides more quantitative detail about the degree of similarity between the items which are compared. In the last five years, we have made great strides in assessing striae similarity of bullet striations fired from traditionally rifled barrels using automatic matching algorithms which incorporate both mathematical correlations and features used by examiners. We will build on this existing foundation by expanding the scope of this matching algorithm and widen its applicability along several different dimensions by:
  • (1) Developing methods to acquire and analyze 360° scans, allowing us to apply the matching algorithm to firearms with polygonal rifling
  • (2) Exploring factors affecting the matching algorithm’s performance. It is well known that different types of firearms mark differently. An objective assessment allows us to quantify this difference. Leveraging ongoing collaborations with crime labs, we will assess and quantify the quality of marks made by different firearms on different types of ammunition. We will also use the same techniques to assess the quality of marks in a single scan to identify areas which are suitable for further analysis.
  • (3) Transitioning the algorithm from research to practice. The matching algorithms have been developed to augment and support examiner assessment of evidence; they are not designed to replace the need for examiner expertise. We anticipate working with crime labs to explore avenues for funding high-resolution microscopy capabilities in forensic labs, and provide training to examiners in the use of the algorithm in casework while we fine-tune the algorithm’s performance in a challenging, high-stakes environment.

The aim of this research is to determine the accuracy and reproducibility of firearms examiners in identifying subclass characteristics and separating subclass from individual characteristics on firearm evidence. This study will show the ability of examiners to identify subclass characteristics that should be avoided when performing a comparison. The consistency of examiners in identifying subclass characteristics will also be determined. At the conclusion of this study, double-casts or digital files may be requested for teaching and training purposes.

The aim of this research proposal is to determine the accuracy and reproducibility of firearms examiners in identifying subclass characteristics and separating subclass from individual characteristics on firearm evidence. A firearm with the capability of a changeable breech face will be used throughout this study. This allows for many breech faces to be manufactured utilizing the different manufacturing techniques typically used by firearm manufacturers (e.g., broaching, milling and turning). Along with the manufacturing techniques being used, several finishing methods will be used to complete the manufacturing process. Test fires will be collected to examine the reproduction of potential subclass characteristics onto fired cartridge cases. A Sensofar® S neox 3D optical profiler will be used to collect high-quality topographic scans of both the breech face inserts and the cartridge cases. These scans will be compared using the NIST congruent matching cells (CMC) algorithm through an ongoing collaboration to analyze the effects of subclass characteristics on objective comparison algorithms. Replications of the collected cartridge cases and the breech face inserts will be produced using a double-casting procedure. This allows examiners who participate in this study to examine the same surfaces with the same features as other examiners. This study will show the ability of examiners to identify subclass characteristics that should be avoided when performing a comparison. The consistency of examiners in identifying subclass characteristics will also be determined. All physical cartridge cases and manufactured breech faces will be retained along with their digital surface scans. At the conclusion of this study, double-casts or digital files may be requested for teaching and training purposes. 

Forensic laboratories will be contacted to request permission to test fire firearms from their reference collections to be included in a database as well as to acquire a wide variety of firearms to test for subclass characteristics. Four calibers will be the main focus of this study, including the common pistol calibers of 9 mm Luger, .40 S&W, and .45 ACP, and the rifle caliber .223 Remington/5.56 mm x 45 mm due to the popularity of the AR-15 semi-automatic rifle and its clones. Samples will be collected from the reference collections of five forensic laboratories. After collection, the researchers begin the acquisition of the cartridge cases, running the NIST congruent matching cell (CMC) algorithm, saving crops of 3D topography images and uploading images to the NIST database. Based on our predictions, by the end of this study, 15 cartridge cases from approximately 3,000 firearms will be included in the database. This database will be made publicly available through NIST. It can be used for research and evidence interpretation. This database would allow for error rate studies of firearm examiner performance and the assessment of automated comparison systems. It would also allow for the assessment similarity metrics, such as likelihood ratios, for firearm evidence. Additionally, it will provide a relevant background database for forensic laboratories when they adopt 3D microscopy as a primary comparison tool.

Similar to firearms, the determination of whether a suspect tool (e.g., screwdriver, wire cutter) is the source of a crime scene toolmark is currently a subjective decision. CSAFE is undertaking the development of a database of toolmark images (2D and 3D) and will use the database to carry out a black-box study in collaboration with partners. This will allow us to estimate the error rate of toolmark analysts. The initial database will focus on one type of tool while varying other features.

Forensic toolmark examiners need a statistical method to determine how likely it is that two toolmarks were created by the same tool, as opposed to by different tools, that is, a statistical method for analyzing toolmarks. (1) To develop such a method, it is first necessary to create a high-quality database of toolmarks. If approved, in this project we will create a database of toolmarks by using screwdrivers — one of the simplest tools that is commonly used in crimes and that can serve as an initial step in statistical models of toolmark examination. The database will have a factorial design with 2D and 3D images of marks made by different tools, at different angles, replicated several times, as inspired by the NIST NBIDE database (this allows for evaluating within- and between-tool variation). The database will be made available to the public for further research. (2) After the database is completed, we will develop an algorithm to determine a score-based likelihood ratio, which compares the likelihood that the toolmark was created by the same tool and the likelihood that it was not. This algorithm will build upon the work in firearms by CSAFE in its first five years. (3) Finally, we will validate the algorithm by testing its external validity. We will acquire toolmarks created by forensic laboratories for which the analysts know the right answer and we do not, so we can test the performance of our algorithm. The CSAFE firearms algorithms have been validated in this way successfully in the past. Finally, the creation of an algorithm for toolmark analysis, developed by using the work previously done by CSAFE in firearms, will help us learn how to expand the algorithm approach to other forensic disciplines.  

Knowledge Transfer

  • Type

Found 13 Results
Page 1 of 1

CSAFE 2020 All Hands Meeting

Type: Research Area(s): ,,,,,,,,

The 2020 All Hands Meeting was held May 12 and 13, 2020 and served as the closing to the last 5 years of CSAFE research and focused on kicking off new initiatives for the next phase of the center, CSAFE…


Comparison of three similarity scores for bullet LEA matching

Type: Research Area(s):

Published: 2020 | By: Susan VanderPlas

Recent advances in microscopy have made it possible to collect 3D topographic data, enabling more precise virtual comparisons based on the collected 3D data as a supplement to traditional comparison microscopy and 2D photography. Automatic comparison algorithms have been introduced…

View on Digital Repository


A Robust Approach to Automatically Locating Grooves in 3D Bullet Land Scans

Type: Research Area(s):

Published: 2019 | By: Kiegan Rice

Land engraved areas (LEAs) provide evidence to address the same source–different source problem in forensic firearms examination. Collecting 3D images of bullet LEAs requires capturing portions of the neighboring groove engraved areas (GEAs). Analyzing LEA and GEA data separately is…

View on Digital Repository


Pattern Evidence Research in CSAFE-An Update

Type: Research Area(s): ,,,,

CSAFE is a NIST Center of Excellence in Forensic Science. A large portion of CSAFE’s research portfolio is on what is known as pattern evidence, which encompasses any evidence that can be represented as an image. Examples of pattern evidence…


Reproducibility of Automated Bullet Matching Scores Using High-Resolution 3D LEA Scans

Type: Research Area(s):

Published: 2019 | By: Kiegan Rice

Development of automated bullet matching algorithms based on 3D scans of land engraved areas (LEAs) has become a prominent area of research in recent years. However, automated methods rely heavily on the data gathered in 3D scans. The high-resolution 3D…

View on Digital Repository


Adapting the Chumbley Score to Match Striae on Land Engraved Areas (LEAs) of Bullets

Type: Research Area(s):

Published: 2019 | By: Ganesh Krishnan

The same‐source problem remains a major challenge in forensic toolmark and firearm examination. Here, we investigate the applicability of the Chumbley method (J Forensic Sci, 2018, 63, 849; J Forensic Sci, 2010, 55, 953) (10,12), developed for screwdriver markings, for…

View on Digital Repository


Automatic Matching of Scans from Hamby Sets: Friends, False Friends, and Clones

Type: Research Area(s):

Published: 2019 | By: Heike Hofmann

Over the past year, we have had the opportunity to scan several Hamby Sets from different Forensic Labs. Scans from another two Hamby sets are available from the National Ballistics and Toolmarks Research Database. We were also given two clone…

View on Digital Repository


An Experiment to Explore Persistence of Bullet Striations

Type: Research Area(s):

Published: 2019 | By: Heike Hofmann

After attending this presentation, attendees will better understand study design, 3D imaging, and persistence of bullet striations. Attendees will also learn about new technologies for comparing bullets using 3D images of land impressions.

View on Digital Repository


Comparing Cartridge Breechface Marks: 2D versus 3D

Type: Research Area(s):

This CSAFE Center Wide Webinar was presented by Xiao Hui Tai from Carnegie Mellon University on January 31, 2018. Description: We will describe work that we have done on an automated method to compare breechface marks on 2D images of cartridge…


Algorithmic approaches to match degraded land impressions

Type: Research Area(s):

Published: 2017 | By: Eric Hare

Bullet matching is a process used to determine whether two bullets may have been fired from the same gun barrel. Historically, this has been a manual process performed by trained forensic examiners. Recent work, however, has shown that it is…

View on Digital Repository


A Fully Automatic Method for Comparing Cartridge Case Images

Type: Research Area(s):

Published: 2017 | By: Xiao Hui Tai

When a gun is fired, it leaves marks on cartridge cases that are thought to be unique to the gun. In current practice, firearms examiners inspect cartridge cases for “sufficient agreement,” in which case they conclude that they come from…

View on Digital Repository


Statistical and Algorithmic Approaches to Matching Bullets

Type: Research Area(s):

This CSAFE Center Wide Meeting Webinar was presented by Eric Hare from Iowa State University on April 14, 2017. Description: CSAFE researchers at Iowa State University will discuss the advances they have made towards providing firearms examiners with an objective,…


Automatic Matching of Bullet Land Impressions

Type: Research Area(s):

Published: 2017 | By: Eric Hare

In 2009, the National Academy of Sciences published a report questioning the scientific validity of many forensic methods including firearm examination. Firearm examination is a forensic tool used to help the court determine whether two bullets were fired from the…

View on Digital Repository


Page 1 of 1

COMMUNITY CALL-TO-ACTION

Want to collaborate with CSAFE on a project. Contact us to share your idea.

Do you have 44.03 seconds?

44.3 Seconds. That is the average amount of time it takes for a visitor to provide site feedback.
Test it yourself by taking the survey.


    A scientist/researcherA member of the forensic science communityA journalist/publicationA studentOther. Please indicate.


    Learn more about CSAFE overall.Discover research CSAFE is undertaking.Explore collaboration opportunities.Find tools and education opportunities.Other. Please indicate.


    YesNo