Should the Likelihood Ratio Be Used In the Courtroom? NIST Researchers Weigh In

When investigating and communicating the value of evidence, the forensic science community makes every effort to explain results to juries in objective and understandable ways.

The analysis of evidence involves the consideration of multiple hypotheses.  Let us take DNA evidence for example – What is the probability of observing a genotypic match if the biological sample was left by the suspect?  How likely is a match if someone else left the sample?

When quantifying the results of DNA analysis, forensic science experts often use the likelihood ratio. This technique enables experts to communicate the weight of the evidence with a single number.

Forensic DNA analysis rests on a firm foundation of biology, and the public often understands principles such as genetic inheritance and unique combinations of genetic markers. Thus, confidence in the power of DNA evidence to identify an individual is strong.

Forensic scientists have long since used the likelihood ratio effectively in cases where DNA evidence is prominent. We know how biology works and can successfully use statistics to quantify the strength of the evidence.

But what about other types of evidence?  Can the likelihood help us to quantify the value of evidence such as fingerprints or bullet striations or the chemical composition of glass? That is still up for debate.

National Institute of Standards and Technology (NIST) statisticians Steve Lund and Hari Iyer recently discussed the value of applying the likelihood ratio in the courtroom.  A review of their conclusions is available in a new article published in the NIST Journal of Research.

Evidence analysis and the likelihood ratio applied to DNA evidence is often cut and dry.  However, we can’t say the same for evidence such as bullets or fingerprints.  Why is that?  It is because the fundamental scientific underpinnings of DNA evidence are absent almost everywhere else. We do not have a physical or statistical model for how striations on bullets arise, and we don’t know what determines specific patterns in fingerprints.

The question then becomes-is it still appropriate to use the same statistical tool in areas such as pattern evidence when our level of understanding is not the same? According to Lund and Iyer, the answer is maybe not.  Or rather, not yet.

Because in those cases, the likelihood ratio may rest on assumptions that cannot be verified in practice, a degree of subjectivity is inevitable and it may happen that two experts arrive at different values of the likelihood ratio.  Thus, say Lund and Iyer, when a decision involves human judgment, caution is key.

Without established models on which to base conclusions in evidence beyond DNA, jurors may find it difficult to interpret the results of forensic analysis. Jurors need more information that just the value of a likelihood ratio, Lund and Iyer say.

Lund and Iyer advocate for complete transparency when explaining the value of evidence to members of a jury.  In this light, it is imperative that jurors understand every aspect of the analysis of a specific evidence, and of the methods that were used to draw conclusions.  Forensic scientists must also communicate their level of uncertainty in their results. Armed with additional context, jurors are then better prepared to make informed decisions.

As the groundwork for the statistical treatment of evidence is being laid, Lund and Iyer issue a challenge to the forensic science community.  When communicating the value of evidence in the courtroom, they call for research that does not focus exclusively on the likelihood ratio approach.  More work needs to be done, but they are confident that it is possible to develop more than one framework for the analysis of evidence other than DNA.

Review this NIST news article for more information.  Visit the NIST website to discover more forensic science research.