This CSAFE webinar was held on May 13, 2021.
Presenter:
Corey Katz
Graduate Researcher, University of California, Irvine
Presentation Description:
There has recently been increased attention on the reliability and validity of methods used in the analysis of forensic evidence. Black-box studies, studies in which questioned-known evidence pairs for which the underlying truth is known are analyzed by forensic examiners, have emerged as a powerful tool to learn about intra-examiner reliability, inter-examiner reliability, and validity. Results of such studies can be complemented by additional studies that focus on performance of examiners on different aspects of the examination. This webinar focused on the development of a statistical approach to assessing the performance of examiners in identifying randomly acquired characteristics on footwear impressions.
In this webinar, the examination process of footwear was discussed. The main objective of examiners is to compare a footwear impression found at a crime scene (sometimes known as the questioned impression) with an impression from one or more suspect (known) shoes. One critical part of the examination process, and the focus of this webinar, was the identification and examination of randomly acquired characteristics (RACs), markings that indicate scratches or holes that have formed on the bottom of shoe soles as the shoe is worn. Identifying RACs takes a great deal of training and, in many cases, is a difficult and time-consuming process. Footwear examiners record the location, type (holes, scratches, etc.), and other key attributes of each RAC. The current standards for measuring performance are proficiency tests. A limitation of these tests is that they focus almost exclusively on correctly matching a questioned impression to a putative source. This limitation, and the key role played by RACs, motivates this work on developing a new method of performance evaluation that focuses on the identification of RACs on a footwear impression.
The method described in this presentation is a first step in estimating the performance of examiners in identifying RACs examinations. It is based on a probability model that assumes there are repeated annotations of the same impression either by a single examiner or by multiple examiners. Importantly, the approach does not require that there be an a prior definition of ground truth. The probability model builds on an approach developed in the context of brain imaging. It has been further extended into a fully Bayesian hierarchical model. Data from a pilot study conducted by the Division of Identification and Forensic Science of the Israel Police (DIFS) allows for the estimation of performance because multiple student trainees identified RACs on the same footwear impression and each examiner examined multiple shoes. Using these data, performance parameters were defined as the ability of the examiner to determine whether or not RACs appear on the impression. The new statistical method was introduced and explored in detail during this webinar.
