Skip to content

Accounting for individual differences among decision-makers with applications to the evaluation of forensic evidence

Published: 2019
Primary Author: Amanda Luby
Research Area: Latent Print

Forensic science often involves the comparison of crime-scene evidence to a known-source sample to determine if the evidence arose from the same source as the reference sample. Common examples include determining if a fingerprint or DNA was left by a suspect, or if a bullet was fired from a specific gun. Even as forensic measurement and analysis tools become increasingly accurate and objective, final source decisions are often left to individual examiners’ interpretation of the evidence (President’s Council of Advisors on Science and Technology, 2016). The current approach to characterizing uncertainty in forensic decision-making has largely centered around the calculation of error rates, which is problematic when different examiners respond to different sets of items, as their error rates are not directly comparable. Furthermore, forensic analyses often consist of a series of steps. While some steps may be straightforward and relatively objective, substantial variation may exist in more subjective decisions. The goal of this dissertation is to adapt and implement statistical models for human decisionmaking for the forensic science domain. Item Response Theory (IRT), a class of statistical methods used prominently in psychometrics and educational testing, is one approach that accounts for differences among decision-makers and additionally accounts for varying difficulty among decision-making tasks. By casting forensic decision-making tasks in the IRT framework, well-developed statistical methods, theory, and tools become available. However, substantial differences exist between forensic decision-making tasks and standard IRT applications such as educational testing. I focus on three developments in IRT for forensic settings: (1) modeling sequential responses explicitly, (2) determining expected answers from responses when an answer key does not exist, and (3) incorporating self-reported assessments of performance into the model. While this dissertation focuses on fingerprint analysis, specifically the FBI Black Box study (Ulery et al., 2011), methods are broadly applicable to other forensic domains in which subjective decisionmaking plays a role, such as bullet comparisons, DNA mixture interpretation, and handwriting analysis.

Related Resources

Implementation of a Blind Quality Control Program in a Forensic Laboratory

Implementation of a Blind Quality Control Program in a Forensic Laboratory

A blind quality control (QC) program was successfully developed and implemented in the Toxicology, Seized Drugs, Firearms, Latent Prints (Processing and Comparison), Forensic Biology, and Multimedia (Digital and Audio/Video) sections…
CSAFE 2020 All Hands Meeting

CSAFE 2020 All Hands Meeting

The 2020 All Hands Meeting was held May 12 and 13, 2020 and served as the closing to the last 5 years of CSAFE research and focused on kicking off…
How do latent print examiners perceive proficiency testing? An analysis of examiner perceptions, performance, and print quality

How do latent print examiners perceive proficiency testing? An analysis of examiner perceptions, performance, and print quality

Proficiency testing has the potential to serve several important purposes for crime laboratories and forensic science disciplines. Scholars and other stakeholders, however, have criticized standard proficiency testing procedures since their…
Crime Lab Proficiency Testing and Quality Management

Crime Lab Proficiency Testing and Quality Management

In the wake of recent reports documenting the vulnerability of forensic science methodologies to human error (e.g., NAS, 2009; PCAST, 2016), the field has sometimes pointed to proficiency testing as…
Do you have 44.03 seconds?

44.3 Seconds. That is the average amount of time it takes for a visitor to provide site feedback.
Test it yourself by taking the survey.


A scientist/researcherA member of the forensic science communityA journalist/publicationA studentOther. Please indicate.


Learn more about CSAFE overall.Discover research CSAFE is undertaking.Explore collaboration opportunities.Find tools and education opportunities.Other. Please indicate.


YesNo