Skip to content

A finely tuned deep transfer learning algorithm to compare outsole images

Journal: Statistical Analysis and Data Mining
Published: 2023
Primary Author: Moonsoo Jang
Secondary Authors: Soyoung Park, Alicia Carriquiry
Research Area: Footwear

In forensic practice, evaluating shoeprint evidence is challenging because the differences between images of two different outsoles can be subtle. In this paper, we propose a deep transfer learning-based matching algorithm called the Shoe-MS algorithm that quantifies the similarity between two outsole images. The Shoe-MS algorithm consists of a Siamese neural network for two input images followed by a transfer learning component to extract features from outsole impression images. The added layers are finely tuned using images of shoe soles. To test the performance of the method we propose, we use a study dataset that is both realistic and challenging. The pairs of images for which we know ground truth include (1) close non-matches and (2) mock-crime scene pairs. The Shoe-MS algorithm performedwell in terms of prediction accuracy andwas able to determine the source of pairs of outsole images, even when comparisonswere challenging. When using a score-based likelihood ratio, the algorithm made the correct decision with high probability in a test of the hypothesis that images had a common source. An important advantage of the proposed approach is that pairs of images can be compared without alignment. In initial tests, Shoe-MS exhibited better-discriminating power than existing methods.

Related Resources

Computational Shoeprint Analysis for Forensic Science

Computational Shoeprint Analysis for Forensic Science

Shoeprints are a common type of evidence found at crime scenes and are regularly used in forensic investigations. However, their utility is limited by the lack of reference footwear databases…
Challenges in Modeling, Interpreting, and Drawing Conclusions from Images as Forensic Evidence

Challenges in Modeling, Interpreting, and Drawing Conclusions from Images as Forensic Evidence

When a crime is committed, law enforcement directs crime scene experts to obtain evidence that may be pertinent to identifying the perpetrator(s). Much of this evidence comes in the form…
Aligning Shoeprint Images that have nonlinear distortion effects

Aligning Shoeprint Images that have nonlinear distortion effects

Shoeprints are aligned before assessing similarity, and automatic alignment algorithms can handle differences in translation, rotation [1], and scale. But shoeprints recorded at a crime scene may be partials photographed…
Graph-Theoretic Techniques for Forensic Image Comparisons

Graph-Theoretic Techniques for Forensic Image Comparisons

This presentation is from the 76th Annual Conference of the American Academy of Forensic Sciences (AAFS), Denver, Colorado, February 19-24, 2024.