Probabilistic Reporting in Criminal Cases in the United States: A Baseline Study
October 10, 2020 4:04 pm
Lead Researchers
Simon A. Cole
Matt Barno
Journal
Science & Justice
Publication Date
September 2020
Publication Number
IN 112 IMPL
Overview
Forensic examiners are frequently asked to give reports and testimonies in court and there have been calls for them to report their findings probabilistically. Terms like match, consistent with or identical are categorical in nature, not statistical –– they do not communicate the value of the evidence in terms of probability. While there is robust debate over how forensic scientists should report, less attention is paid to how they do report.
Key Research Questions
1
To what extent are forensic reports in these disciplines consistent with published standards?
2
To what extent are forensic reports in these disciplines probabilistic, and, if so, how is probability expressed?
Approach & Methodology
Data Set
572 transcripts and reports from Westlaw, consultants’ files and proficiency tests using a heterogeneous, opportunistic data collection approach.
WHAT
Researchers reviewed reports across four pattern disciplines:
- Friction Ridge Prints
- Firearms & Toolmarks
- Questioned Documents
- Shoeprints
HOW
Using disciplinary standards as a framework, researchers determined the type of report being reviewed and if it used standard terminology. Then, they coded each report both for whether or not it was probabilistic and for the type of language used, such as “same source,” “identified” and “consistent.”
Key Takeaways For Practitioners
Across all four disciplines, the prevailing standards for reporting were categorical in nature. The majority of reports analyzed adhered to the reporting standards for their discipline –– but discussion of probability was extremely rare and, even in those cases, frequently used to dismiss the use of probability itself.
Focus on the future
To increase the probabilistic reporting of forensics results:
Incorporate probabilistic reporting into disciplinary standards.
Educate practitioners, lawyers, and judges on the reasons for, and importance, of probabilistic reporting.
Demand that experts quantify their uncertainty when testifying in court.