One of the most troubling trends in criminal investigations is the growing use of “black box” technology, in which law enforcement rely on artificial intelligence (AI) models or algorithms that are either too complex for people to understand or they simply conceal how it functions. In criminal cases, black box systems have proliferated in forensic areas such as DNA mixture interpretation, facial recognition, and recidivism risk assessments. The champions and critics of AI argue, mistakenly, that we face a catch 22: While black box AI is not understandable by people, they assume that it produces more accurate forensic evidence. In this Article, we question this assertion, which has so powerfully affected judges, policymakers, and academics. We describe a mature body of computer science research showing how “glass box” AI—designed to be interpretable—can be more accurate than black box alternatives. Indeed, black box AI performs predictably worse in settings like the criminal system. Debunking the black box performance myth has implications for forensic evidence, constitutional criminal procedure rights, and legislative policy. Absent some compelling—or even credible—government interest in keeping AI as a black box, and given the constitutional rights and public safety interests at stake, we argue that a substantial burden rests on the government to justify black box AI in criminal cases. We conclude by calling for judicial rulings and legislation to safeguard a right to interpretable forensic AI.
Interpretable algorithmic forensics

Journal: Proceedings of the National Academy of Sciences
Published: 2023
Primary Author: Brandon Garrett
Secondary Authors: Cynthia Rudin
Type: Publication
Related Resources
Score-based Likelihood Ratios Using Stylometric Text Embeddings
We consider the problem setting in which we have two sets of texts in digital form and would like to quantify our beliefs that the two sets of texts were…
How do Labs Ensure Quality? A Nationwide Review of SOPs for Latent Print Examination
This presentation is from the 108th International Association for Identification (IAI) Annual Educational Conference, Reno, Nevada, August 11-17, 2024. Posted with permission of CSAFE.
Statistics and its Applications in Forensic Science and the Criminal Justice System
This presentation is from the 2024 Joint Statistical Meetings (JSM), Portland, Oregon, August 3-8, 2024.
Algorithmic matching of striated tool marks
Automatic matching algorithms for assessing the similarity between striation marks have been investigated for bullet lands and some tool marks, such as screwdrivers. We are interested in the investigation of…