Skip to content

Assessing Footwear Examiner Performance

Type: Webinar
Research Area: Footwear

This CSAFE webinar was held on May 13, 2021.

Presenter:

Corey Katz
Graduate Researcher, University of California, Irvine

Presentation Description:

There has recently been increased attention on the reliability and validity of methods used in the analysis of forensic evidence. Black-box studies, studies in which questioned-known evidence pairs for which the underlying truth is known are analyzed by forensic examiners, have emerged as a powerful tool to learn about intra-examiner reliability, inter-examiner reliability, and validity. Results of such studies can be complemented by additional studies that focus on performance of examiners on different aspects of the examination. This webinar focused on the development of a statistical approach to assessing the performance of examiners in identifying randomly acquired characteristics on footwear impressions.

In this webinar, the examination process of footwear was discussed. The main objective of examiners is to compare a footwear impression found at a crime scene (sometimes known as the questioned impression) with an impression from one or more suspect (known) shoes. One critical part of the examination process, and the focus of this webinar, was the identification and examination of randomly acquired characteristics (RACs), markings that indicate scratches or holes that have formed on the bottom of shoe soles as the shoe is worn. Identifying RACs takes a great deal of training and, in many cases, is a difficult and time-consuming process. Footwear examiners record the location, type (holes, scratches, etc.), and other key attributes of each RAC. The current standards for measuring performance are proficiency tests. A limitation of these tests is that they focus almost exclusively on correctly matching a questioned impression to a putative source. This limitation, and the key role played by RACs, motivates this work on developing a new method of performance evaluation that focuses on the identification of RACs on a footwear impression.

The method described in this presentation is a first step in estimating the performance of examiners in identifying RACs examinations. It is based on a probability model that assumes there are repeated annotations of the same impression either by a single examiner or by multiple examiners. Importantly, the approach does not require that there be an a prior definition of ground truth. The probability model builds on an approach developed in the context of brain imaging. It has been further extended into a fully Bayesian hierarchical model. Data from a pilot study conducted by the Division of Identification and Forensic Science of the Israel Police (DIFS) allows for the estimation of performance because multiple student trainees identified RACs on the same footwear impression and each examiner examined multiple shoes. Using these data, performance parameters were defined as the ability of the examiner to determine whether or not RACs appear on the impression. The new statistical method was introduced and explored in detail during this webinar.

Related Resources

CSAFE 2021 Field Update

CSAFE 2021 Field Update

The 2021 Field Update was held June 14, 2021, and served as the closing to the first year of CSAFE 2.0. CSAFE brought together researchers, forensic science partners and interested…
Open Forensic Science in R

Open Forensic Science in R

This book is for anyone looking to do forensic science analysis in a data-driven and open way. Whether you are a student, teacher, or scientist, this book is for you. We take…
Quantifying the similarity of 2D images using edge pixels: An application to the forensic comparison of footwear impressions

Quantifying the similarity of 2D images using edge pixels: An application to the forensic comparison of footwear impressions

We propose a novel method to quantify the similarity between an impression (Q) from an unknown source and a test impression (K) from a known source. Using the property of…
Quantifying the similarity of 2D images using edge pixels: An application to the forensic comparison of footwear impressions

Quantifying the similarity of 2D images using edge pixels: An application to the forensic comparison of footwear impressions

We propose a novel method to quantify the similarity between an impression (Q) from an unknown source and a test impression (K) from a known source. Using the property of…