Quantifying Bayes Factors for Forensic Handwriting Evidence

Questioned Document Examiners (QDEs) are tasked with analyzing handwriting evidence to make source (or writership) determinations. The Center for Statistics and Applications of Forensic Evidence (CSAFE) has previously developed computational methods to automatically extract quantifiable handwriting features and statistical methods to analyze handwriting evidence to aid QDEs.1-3 The method developed by Crawford et. al uses a K-means clustering algorithm and Bayesian hierarchical model to perform closed-set writer identification.2 This means a questioned document is assigned to its most likely writer from a set of known writers but does not allow for the possibility of the questioned document to be written by someone not included in the set. Another method developed by Johnson and Ommen utilized machine learning techniques and score-based likelihood ratios (SLRs).3 SLRs have been criticized for a variety of shortcomings, including a lack of coherence and ability to incorporate the rarity of the features. Our goal is to develop a method that supports feature-based open-set writer identification while avoiding these issues. We implement an approach to quantify the value of forensic handwriting evidence using Bayes factors and Markov chain Monte Carlo (MCMC) computational techniques like those described in Collins and Ommen.4 There are two paths to consider depending on the forensic question: the common source and the specific source identification problems. We demonstrate the approach for each identification problem using documents from the CSAFE Handwriting database, which consists of documents of various lengths from over 240 writers: the London Letter is the longest, followed by an excerpt chosen from the book The Wonderful Wizard of Oz, and the phrase “The early bird may get the worm, but the second mouse gets the cheese” is the shortest.5 Handwriting features are extracted using the “handwriter” system, clustered using K-means, and subsequently used to quantify the Bayes factor. The performance of the methods is assessed using cross-validation and rates of misleading evidence (among other measures).