INSIGHTS
Comparison of three similarity scores for bullet LEA matching
OVERVIEW
As technology advances in the forensic sciences, it is important to evaluate the performance of recent innovations. Researchers funded by CSAFE judged the efficacy of different scoring methods for comparing land engraved areas (LEAs) found on bullets.
Lead Researchers
Susan Vanderplas
Melissa Nally
Tylor Klep
Christina Cadevall
Heike Hofmann
Journal
Forensic Science International
Publication Date
March 2020
Publication Number
IN 105 FT
THE GOALS
Evaluate the performance of scoring measures at a land-to-land level, using random forest scoring, cross correlation and consecutive matching striae (CMS).
Consider the efficacy of these scoring measures on a bullet-to-bullet level.
The Study
- Data was taken from three separate studies, each using similar firearms from the same manufacturer, Ruger, to compare land engraved areas (LEAs), areas on a bullet marked by a gun barrel’s lands –– the sections in between the grooves on the barrel’s rifling.
- Examiners processed the LEA data through a matching algorithm and scored it using these three methods:
1
Random Forest (RF):
A form of machine-learning that utilizes a series of decision trees to reach a single result.
2
Cross-Correlation (CC):
A measure of similarity between two series of data.
3
Consecutive Matching Striae (CMS):
Identifying the similarities between the peaks and valleys of LEAs.
Results
The Equal Error rate of each scoring method across multiple studies
- On a bullet-to-bullet level, the Random Forest and Cross-Correlation scoring methods made no errors.
- On a land-to-land level, the RF and CC methods outperformed the CMS method.
- When comparing equal error rates, the CMS method had an error rate of over 20%, while both the RF and CC methods’ error rates were roughly 5%. The RF method performed slightly better.
FOCUS ON THE FUTURE
The random forest algorithm struggled to identify damage to bullets that obscured LEAs caused by deficiencies in the gun barrel such as pitting from gunpowder or “tank rash” from expended bullets.
- In future studies, examiners could pair the RF algorithm with another algorithm to assess the quality of the data and determine which portions can be used for comparison.
All the studies used firearms from Ruger, a manufacturer picked because their firearms mark very well on bullets. Future studies can assess the performance of these scoring methods on firearms from different manufacturers with differing quality marks.
