Firearms and Toolmarks Analysis

• (1) Developing methods to acquire and analyze 360° scans, allowing us to apply the matching algorithm to firearms with polygonal rifling
• (2) Exploring factors affecting the matching algorithm’s performance. It is well known that different types of firearms mark differently. An objective assessment allows us to quantify this difference. Leveraging ongoing collaborations with crime labs, we will assess and quantify the quality of marks made by different firearms on different types of ammunition. We will also use the same techniques to assess the quality of marks in a single scan to identify areas which are suitable for further analysis.
• (3) Transitioning the algorithm from research to practice. The matching algorithms have been developed to augment and support examiner assessment of evidence; they are not designed to replace the need for examiner expertise. We anticipate working with crime labs to explore avenues for funding high-resolution microscopy capabilities in forensic labs, and provide training to examiners in the use of the algorithm in casework while we fine-tune the algorithm’s performance in a challenging, high-stakes environment.

The aim of this research is to determine the accuracy and reproducibility of firearms examiners in identifying subclass characteristics and separating subclass from individual characteristics on firearm evidence. This study will show the ability of examiners to identify subclass characteristics that should be avoided when performing a comparison. The consistency of examiners in identifying subclass characteristics will also be determined. At the conclusion of this study, double-casts or digital files may be requested for teaching and training purposes.

The aim of this research proposal is to determine the accuracy and reproducibility of firearms examiners in identifying subclass characteristics and separating subclass from individual characteristics on firearm evidence. A firearm with the capability of a changeable breech face will be used throughout this study. This allows for many breech faces to be manufactured utilizing the different manufacturing techniques typically used by firearm manufacturers (e.g., broaching, milling and turning). Along with the manufacturing techniques being used, several finishing methods will be used to complete the manufacturing process. Test fires will be collected to examine the reproduction of potential subclass characteristics onto fired cartridge cases. A Sensofar® S neox 3D optical profiler will be used to collect high-quality topographic scans of both the breech face inserts and the cartridge cases. These scans will be compared using the NIST congruent matching cells (CMC) algorithm through an ongoing collaboration to analyze the effects of subclass characteristics on objective comparison algorithms. Replications of the collected cartridge cases and the breech face inserts will be produced using a double-casting procedure. This allows examiners who participate in this study to examine the same surfaces with the same features as other examiners. This study will show the ability of examiners to identify subclass characteristics that should be avoided when performing a comparison. The consistency of examiners in identifying subclass characteristics will also be determined. All physical cartridge cases and manufactured breech faces will be retained along with their digital surface scans. At the conclusion of this study, double-casts or digital files may be requested for teaching and training purposes. 

Forensic laboratories will be contacted to request permission to test fire firearms from their reference collections to be included in a database as well as to acquire a wide variety of firearms to test for subclass characteristics. Four calibers will be the main focus of this study, including the common pistol calibers of 9 mm Luger, .40 S&W, and .45 ACP, and the rifle caliber .223 Remington/5.56 mm x 45 mm due to the popularity of the AR-15 semi-automatic rifle and its clones. Samples will be collected from the reference collections of five forensic laboratories. After collection, the researchers begin the acquisition of the cartridge cases, running the NIST congruent matching cell (CMC) algorithm, saving crops of 3D topography images and uploading images to the NIST database. Based on our predictions, by the end of this study, 15 cartridge cases from approximately 3,000 firearms will be included in the database. This database will be made publicly available through NIST. It can be used for research and evidence interpretation. This database would allow for error rate studies of firearm examiner performance and the assessment of automated comparison systems. It would also allow for the assessment similarity metrics, such as likelihood ratios, for firearm evidence. Additionally, it will provide a relevant background database for forensic laboratories when they adopt 3D microscopy as a primary comparison tool.

Similar to firearms, the determination of whether a suspect tool (e.g., screwdriver, wire cutter) is the source of a crime scene toolmark is currently a subjective decision. CSAFE is undertaking the development of a database of toolmark images (2D and 3D) and will use the database to carry out a black-box study in collaboration with partners. This will allow us to estimate the error rate of toolmark analysts. The initial database will focus on one type of tool while varying other features.

Forensic toolmark examiners need a statistical method to determine how likely it is that two toolmarks were created by the same tool, as opposed to by different tools, that is, a statistical method for analyzing toolmarks. (1) To develop such a method, it is first necessary to create a high-quality database of toolmarks. If approved, in this project we will create a database of toolmarks by using screwdrivers — one of the simplest tools that is commonly used in crimes and that can serve as an initial step in statistical models of toolmark examination. The database will have a factorial design with 2D and 3D images of marks made by different tools, at different angles, replicated several times, as inspired by the NIST NBIDE database (this allows for evaluating within- and between-tool variation). The database will be made available to the public for further research. (2) After the database is completed, we will develop an algorithm to determine a score-based likelihood ratio, which compares the likelihood that the toolmark was created by the same tool and the likelihood that it was not. This algorithm will build upon the work in firearms by CSAFE in its first five years. (3) Finally, we will validate the algorithm by testing its external validity. We will acquire toolmarks created by forensic laboratories for which the analysts know the right answer and we do not, so we can test the performance of our algorithm. The CSAFE firearms algorithms have been validated in this way successfully in the past. Finally, the creation of an algorithm for toolmark analysis, developed by using the work previously done by CSAFE in firearms, will help us learn how to expand the algorithm approach to other forensic disciplines.