Reliability for Binary and Ordinal Data in Forensics

Forensics studies of feature-based comparison decisions typically focus on the accuracy and reliability the decisions. Decisions can be reports in the form of binary conclusions (value / no value) or in the form of ordered categories. In general there is limited covariate information available about either the examiners or the forensic samples being assessed. We propose a methodology to identify groups of examiners that might share decision making abilities or thresholds for decisions. Identifying clusters of examiners can enable us to assess reliability of decisions within these clusters which may be different from the reliability that is evaluated through marginalized data.