Green gear surrounded by two green circular arrows on a light gray background.

Journal: Proceedings of the National Academy of Sciences

Published: 2023

Primary Author: Brandon Garrett

Secondary Authors: Cynthia Rudin

One of the most troubling trends in criminal investigations is the growing use of “black box” technology, in which law enforcement rely on artificial intelligence (AI) models or algorithms that are either too complex for people to understand or they simply conceal how it functions. In criminal cases, black box systems have proliferated in forensic areas such as DNA mixture interpretation, facial recognition, and recidivism risk assessments. The champions and critics of AI argue, mistakenly, that we face a catch 22: While black box AI is not understandable by people, they assume that it produces more accurate forensic evidence. In this Article, we question this assertion, which has so powerfully affected judges, policymakers, and academics. We describe a mature body of computer science research showing how “glass box” AI—designed to be interpretable—can be more accurate than black box alternatives. Indeed, black box AI performs predictably worse in settings like the criminal system. Debunking the black box performance myth has implications for forensic evidence, constitutional criminal procedure rights, and legislative policy. Absent some compelling—or even credible—government interest in keeping AI as a black box, and given the constitutional rights and public safety interests at stake, we argue that a substantial burden rests on the government to justify black box AI in criminal cases. We conclude by calling for judicial rulings and legislation to safeguard a right to interpretable forensic AI.


Related Resources

A thick gray wavy line forming an abstract, looping shape on a light gray background.

An Introduction to the Forensic Handwriting Analysis Software handwriter

July 18, 2025

Blue shoeprint with a tread pattern on a white background.

Forensic Footwear: A Retrospective of the Development of the MANTIS Shoe Scanning System

July 10, 2025

There currently are no shoe-scanning devices developed in the United States that can operate in a real-world, variable-weather environment in …

A green fingerprint icon on a light gray background.

Examiner consistency in perceptions of fingerprint minutia rarity

July 10, 2025

Friction ridge examiners (FREs) identify distinctive features (minutiae) in fingerprints and consider how rare these observed minutiae are in their …