Skip to content

The Proficiency of Experts

Journal: University of Pennsylvania Law Review
Published: 2017
Primary Author: Brandon Garrett
Secondary Authors: G. Mitchell

Expert evidence plays a crucial role in civil and criminal litigation. Changes in the rules concerning expert admissibility, following the Supreme Court’s Daubert ruling, strengthened judicial review of the reliability and the validity of an expert’s methods. Judges and scholars, however, have neglected the threshold question for expert evidence: whether a person should be qualified as an expert in the first place. Judges traditionally focus on credentials or experience when qualifying experts without regard to whether those criteria are good proxies for true expertise. We argue that credentials and experience are often poor proxies for proficiency. Qualification of an expert presumes that the witness can perform in a particular domain with a proficiency that non-experts cannot achieve, yet many experts cannot provide empirical evidence that they do in fact perform at high levels of proficiency. To demonstrate the importance of proficiency data, we collect and analyze two decades of proficiency testing of latent fingerprint examiners. In this important domain, we found surprisingly high rates of false positive identifications for the period 1995 to 2016. These data would qualify the claims of many fingerprint examiners regarding their near infallibility, but unfortunately, judges do not seek out such information. We survey the federal and state case law and show how judges typically accept expert credentials as a proxy for proficiency in lieu of direct proof of proficiency. Indeed, judges often reject parties’ attempts to obtain and introduce at trial empirical data on an expert’s actual proficiency. We argue that any expert who purports to give falsifiable opinions can be subjected to proficiency testing and that proficiency testing is the only objective means of assessing the accuracy and reliability of experts who rely on subjective judgments to formulate their opinions (so-called “black-box experts”). Judges should use proficiency data to make expert qualification decisions when the data is available, should demand proof of proficiency before qualifying black-box experts, and should admit at trial proficiency data for any qualified expert. We seek to revitalize the standard for qualifying experts: expertise should equal proficiency.

Related Resources

Reply to Response to Vacuous standards – Subversion of the OSAC standards-development process

Reply to Response to Vacuous standards – Subversion of the OSAC standards-development process

This Letter to the Editor is a reply to Mohammed et al. (2021) https://doi.org/10.1016/j.fsisyn.2021.100145, which in turn is a response to Morrison et al. (2020) “Vacuous standards – subversion of…
Unpacking the Sources of Error in Forensic Evidence

Unpacking the Sources of Error in Forensic Evidence

An overview of the Autopsy of a Crime Lab book and the ways in which error can occur in forensic evidence
Autopsy of a Crime Lab: Addressing the Sources of Error in Forensics

Autopsy of a Crime Lab: Addressing the Sources of Error in Forensics

Keynote presentation:  Autopsy of a Crime Lab book and overview of IMPL 1 Project Area