Forensic Evidence and Forensic Examinations – The Basics

This post is based on the article “Statistical Issues in Forensic Science” by CSAFE Co-Director and Professor of Statistics Hal Stern of Univeristy of California, Irive. Review the full article in the Annual Review of Statistics and Its Application.

When it comes to solving crimes, forensic examiners play a key role in the investigation process and trial proceedings. Their responsibilities vary widely depending on the type of crime and available evidence.

For example, examiners may be called on to identify time of death in a homicide case, reconstruct the events of crime from blood spatter patterns, or take a close look at physical evidence such as broken glass or fingerprints found at the scene. Their analysis of evidence provides crucial details that help in the search for the culprit. The remainder of the discussion here is focused on the role of forensic examiners in attempting to identify if a suspect is linked to particular item of evidence in the case.

It is important to understand that the goal in examining a single piece of evidence is actually not to determine the guilt or innocence of the suspect. Rather, it is an effort to identify if the evidence is associated with the suspect, e.g., if the suspect is a possible source of a fingerprint found at a crime scene. Any number of legitimate reasons could lead to finding evidence from a suspect at the crime scene, such as a visit to the scene at a different time.

Exploring the Different Types of Evidence

  • DNA – DNA found in biological samples recovered at the crime scene is among today’s most powerful forms of forensic evidence. The forensic examiner’s task when analyzing this type of evidence is to determine if the DNA profile of a suspect matches the DNA profile found in the crime scene sample. They then must assess the significance of this agreement.
  • Trace Evidence – Refers to evidence types characterized as a fragment or sample of a larger object that is left behind at the time of the crime. This type of evidence could include glass fragments from a broken window, hairs from an individual, or fibers from clothing or carpet. The challenge here is to determine if a sample of trace evidence from the crime scene matches another sample obtained from a suspect (or perhaps from an object in the suspect’s possession).
  • Pattern evidence – Evidence left at the crime scene that is the result of an impression left by a person or object is known as pattern evidence. The forensic examiner must attempt to determine if the impression found at the crime scene matches the pattern of an analogous sample obtained from the suspect or from an object known to belong to the suspect. Types of pattern evidence include fingerprints, shoeprints, documents/handwriting, toolmarks and firearm impressions.
  • Digital evidence – Digital technology is playing a larger and larger role in criminal and civil investigations. Digital evidence can refer to any information obtained from a device implicated in an investigation. Examples include images or messages found on a smartphone belonging to a suspect. Digital evidence can be challenging to analyze due to the wide variety of different evidence types that may be found on digital devices.

Recent Events Have Raised Concerns About Current Methods

Forensic examiners summarize their evidence analysis in reports that play a crucial role in investigations and trials. Recent events have called into question the scientific and statistical foundations of evidence analysis and interpretation.

This includes the release of two reports, the 2009 National Research Council (NRC) report and a 2016 report of the President’s Council of Advisors on Science and Technology (PCAST), in which committees comprised of scientists from a number of disciplines questioned whether there was sufficient quantitative data supporting the statements made by forensic examiners in a number of disciplines.

Beyond these reports, an additional source of concern is a number of cases in which forensic science errors have been identified. A famous example is the very public case involving American lawyer Brandon Mayfield. Fingerprint examiners from the Federal Bureau of Investigation (FBI) mistakenly identified Mayfield as the source of a latent fingerprint found at the scene of a 2004 train bombing in Spain.

The Innocence Project, a nonprofit legal organization founded in 1992, details countless other situations where unreliable or improper forensic science led to wrongful convictions. The organization has been instrumental in freeing more than 300 wrongfully convicted individuals through the beginning of 2018 and improper forensic science is identified as a contributor in roughly half of these cases.

Why We Need Statistics In Forensic Evidence Analysis

Both the 2009 NRC report and the 2016 PCAST report emphasized the need for additional study of forensic science methods. It is clear that statistical methods have a key role to play in strengthening the scientific foundations of forensic examinations. Statistics is the science concerned with designing studies and experiments, analyzing and interpreting the results, and summarizing the information obtained. As such, it can contribute to studies aimed at determining the accuracy of the conclusions drawn by forensic examiners, addressing cognitive biases, examining the influence of irrelevant information on analysis, evaluating modifications to case processing procedures and more.

This post is based on the article “Statistical Issues in Forensic Science” by CSAFE Co-Director Hal Stern of UCI published in Annual Review of Statistics and Its Application.












Remembering Daubert: A Recap of the CSAFE Symposium on Forensics, Statistics and Law at UVA

This is an invited blog post from Brandon Garrett, CSAFE researcher and White Burkett Miller Professor of Law and Public Affairs and Justice Thurgood Marshall Distinguished Professor of Law at University of Virginia School of Law. Guest blog posts do not necessarily reflect the views of CSAFE.  CSAFE is highlighting this topic due to our team’s commitment to advancing forensic science through enhanced connections between resesarch, statistics and the law.  

Twenty-five years ago, the U.S. Supreme Court ruled in Daubert v. Merrell Dow Pharmaceuticals Inc., that federal judges must conduct a scientific gatekeeping inquiry before admitting expert evidence.  A March 26, 2018 conference at the University of Virginia School of Law explored the challenges and the changes in the law, research and in the practice of forensic science in the years since Daubert was decided. The Virginia Journal of Criminal Law will publish as a symposium issue contributions by experts in forensics, statistics and the law from the event.  The conference was made possible by the Center for Statistics and Applications in Forensic Evidence.

The conference was live streamed and videotaped, with videos available on the UVA School of Law YouTube channel.

Available Sessions:
Forensics, Statistics and Law
Statistics, Research and Forensics
Judging Forensics
Statistics in the Crime Lab
Bringing Statistics into the Courtroom

Opening Remarks Highlights

The conference began with remarks describing how forensic science has changed in the twenty-five years since Daubert was decided and how researchers are beginning to address those shortcomings.  First, Professor Karen Kafadar (UVA) described the importance of statistics to forensic science.  Second, Sue Ballou, Program Manager at NIST, and incoming President of the American Academy of Forensic Science (AAFS), described her career as a forensic analyst and how her work came to engage more over time with questions of statistics and scientific research.  Ballou described, in remarks to be published in the symposium volume, how the CSAFE collaboration and the work of NIST researchers have done more to connect science with forensics. Third, Peter Neufeld described his career, including examples from cases in which he used scientific experts before he co-founded the Innocence Project and began to use DNA testing to free innocent convicts.

Examining How Statistics Applies to Forensic Science

Following those introductory remarks, the first panel discussed the role of statistics in forensic science.  Professor Alicia Carriquiry (Iowa State), whose remarks are to be published in this volume, described importance of statistics in forensics. Hari Iyer (NIST) addressed a different question: assuming there is quantified information about forensics, how are those statistics to be presented in court?  Iyer described recent work with Steven Lund, both at NIST, arguing that there are real concerns with the proposed use of likelihood ratios to express forensic conclusions, including because of the subjectivity of the decisions that are incorporated into such expressions.  Karen Kafadar (UVA), whose remarks are to be published in the volume, described not only the importance of bringing statistical rigor to forensic sciences, but also training on statistics and educational efforts to encourage future statisticians to examine practical and pressing problems in forensics.

Judge Jed Rakoff of the U.S. District Court for the Southern District of New York delivered the keynote address, which is to be published in this volume, asking and addressing why it is that judges have not acted as more forceful gatekeepers in the area of forensic science.

Linda Jackson, Director of the Virginia Department of Forensic Services described how the lab has long made all of its operating procedures available online and works closely with researchers.  Peter Stout, President of the Houston Forensic Science Center, whose remarks are to be published in the volume, described a substantial program to conduct blind proficiency testing in forensic disciplines.  Sharon Kelley (UVA) described case processing data of fingerprint examiner at HFSC, including data on how disagreements between examiners were resolved.  Henry Swofford, of the Defense Forensic Science Center described, in remarks published in this volume, a new program, called FRStat, developed in order to provide quantitative conclusions regarding fingerprint testing.

Investigating the Role of Statistics in the Courtroom

The final panel discussed the role of statistics in the courtroom.  Chancellor and Dean David Faigman (U.C. Hastings) described the need to attend to the connection between general research and individual evidence in criminal cases, as part of a larger question regarding how to associate the general to the individual.  David Kaye (Penn State Law) described several areas in which statistical inferences can be misstated or misleading.  A.J. Kramer, of the Federal Defender, provided a criminal defense perspective, explaining how judges have been almost entirely indifferent to challenges to unreliable forensic evidence, which has in turn discouraged defense lawyers from even raising challenges.  Professor Bobbie Spellman (UVA) argued that to explain forensic evidence to jurors, the goal should not be to train jurors or expect them to be amateur statisticians, but rather to provide them with the information they need to reach statistically sound results.

We could not be more grateful to each of the presenters and contributors to this remarkable Symposium.  We hope that the publications and the remarks themselves are of interest.  Twenty-five years after Daubert more remains to be done to connect research, statistics and the law that governs the forensic science evidence that has become an increasingly integral part of our system of criminal justice.