Studying Reproducibility and Repeatability for Pattern Evidence Comparisons

The forensics community has recently conducted black-box studies to establish the accuracy and reliability of subjective forensic examination decisions. We have developed a statistical framework to assess the reliability of such subjective decisions that may be on different scales, for example, binary (match/ non-match) or ordered categories. Our methodology also enables inference for interactions between examiners and forensic samples. We also extend our approach to identify groups of examiners that might share decision making abilities or thresholds for decisions for the purpose of understanding reliability within these subgroups.