Skip to content

Interpretable algorithmic forensics

Journal: Proceedings of the National Academy of Sciences
Published: 2023
Primary Author: Brandon Garrett
Secondary Authors: Cynthia Rudin

One of the most troubling trends in criminal investigations is the growing use of “black box” technology, in which law enforcement rely on artificial intelligence (AI) models or algorithms that are either too complex for people to understand or they simply conceal how it functions. In criminal cases, black box systems have proliferated in forensic areas such as DNA mixture interpretation, facial recognition, and recidivism risk assessments. The champions and critics of AI argue, mistakenly, that we face a catch 22: While black box AI is not understandable by people, they assume that it produces more accurate forensic evidence. In this Article, we question this assertion, which has so powerfully affected judges, policymakers, and academics. We describe a mature body of computer science research showing how “glass box” AI—designed to be interpretable—can be more accurate than black box alternatives. Indeed, black box AI performs predictably worse in settings like the criminal system. Debunking the black box performance myth has implications for forensic evidence, constitutional criminal procedure rights, and legislative policy. Absent some compelling—or even credible—government interest in keeping AI as a black box, and given the constitutional rights and public safety interests at stake, we argue that a substantial burden rests on the government to justify black box AI in criminal cases. We conclude by calling for judicial rulings and legislation to safeguard a right to interpretable forensic AI.

Related Resources

Is it a True Match? Top k correlations in a database search

Is it a True Match? Top k correlations in a database search

This presentation is from the 76th Annual Conference of the American Academy of Forensic Sciences (AAFS), Denver, Colorado, February 19-24, 2024.
Graph-Theoretic Techniques for Forensic Image Comparisons

Graph-Theoretic Techniques for Forensic Image Comparisons

This presentation is from the 76th Annual Conference of the American Academy of Forensic Sciences (AAFS), Denver, Colorado, February 19-24, 2024.
Presumption of Innocence, Probable Cause, and Prior Probability—Bayes Meets Due Process

Presumption of Innocence, Probable Cause, and Prior Probability—Bayes Meets Due Process

This presentation is from the 76th Annual Conference of the American Academy of Forensic Sciences (AAFS), Denver, Colorado, February 19-24, 2024.
Demonstrative Evidence and the Use of Algorithms in Jury Trials

Demonstrative Evidence and the Use of Algorithms in Jury Trials

We investigate how the use of bullet comparison algorithms and demonstrative evidence may affect juror perceptions of reliability, credibility, and understanding of expert witnesses and presented evidence. The use of…