The Innocence Project: 30 Years of Advocating for Justice Reform

Innocence Project

By Samantha Springer, a research assistant at the Center for Statistics and Applications in Forensic Evidence (CSAFE)

 

As the Innocence Project enters its 30th year, Christina Swarns, executive director of the Innocence Project, reflects in an open letter on the challenges and opportunities that lie ahead.

In the letter, In the Vanguard of Justice Reform: The Road Ahead, Swarns takes inventory of what the Innocence Project has learned throughout its three decades in the field of forensic science and reminds all of us in the discipline what the path forward will look like.

Although the Innocence Project primarily works with DNA evidence, the organization plans to extend its advocation to some cases dealing with non-DNA evidence, including research areas CSAFE specializes in. Their foundational pillars of Restoring Freedom, Transforming Systems and Advancing the Movement have relevance for not only CSAFE but for all parties interacting with the criminal justice system.

Expanding Knowledge

One of the Innocence Project’s goals is to create a comprehensive literacy program to educate all players in the criminal process on the science of forensic evidence, including judges, attorneys and jurors. The program will help legal professionals understand the foundations of scientific evidence with the goal of reducing the rate of wrongful convictions based on the misapplication of forensic science. CSAFE is currently working with the Innocence Project on this program.

Swarns writes, “We will launch an ambitious scientific literacy program to educate system actors — from public defenders, to prosecutors, to judges — about science and the limits of forensic evidence. Because too many attorneys have too little grasp of the foundations of scientific evidence, we believe that, with this program, we can and will reduce the rate of wrongful convictions based on the misapplication of forensic science. This program — which we are undertaking with leaders in the field like the Center for Statistics and Applications in Forensic Evidence (CSAFE) — will help legal professionals understand the basics of the evidence in the cases they handle.” 

The Innocence Project collaborates with CSAFE to increase and improve forensic science literacy. One of the results of this collaboration was contributing to a special issue of Significance Magazine, published in 2019 and dedicated solely to articles regarding forensic science and statistics.

Overturning Wrongful Convictions

Based on research done by the National Registry of Exonerations, 52 percent of the wrongful conviction cases handled by the Innocence Project have been due in part to misapplications of forensic evidence. Some examples of errors made include the use of unreliable evidence, misleading expert testimony and the submission of forensic sciences that have been discredited. Following the recommendations of multiple reports, including the National Academy of Sciences’ 2009 report Strengthening Forensic Science in the United States: A Path Forward, the Innocence Project will continue working with lawmakers to create legislation that allows a retrial on the basis of discredited science.

One of the problems underlying the use of unreliable forensic evidence in a criminal trial is the insufficient validation of the scientific methods being presented. To increase the validity, and therefore the quality, of analysis methods being used, more research must be done. CSAFE is one of the organizations dedicated to such reform by conducting research on promising yet under-analyzed types of forensic evidence, such as footwear impression analysis. As well as working to bolster the validity of existing forensic evidence, CSAFE also researches new avenues in an evidence type when a previous method shows itself to be unreliable, such as the former use of comparative bullet lead analysis. Current research by CSAFE now looks into firearm analysis through toolmark comparison in both bullets and cartridge cases. Through these large and well-constructed studies, organizations like CSAFE further the potential of new forensic evidence analysis and bolsters the public opinion of forensic science.

Learn more about CSAFE’s key research areas in probability and statistics for pattern and digital evidence, cross-cutting issues and training and education at https://forensicstats.org/our-research/.

GAO Releases a Second Report on Forensic Science Algorithms

From GAO Report 21-435
GAO-21-435 — Forensic Technology: Algorithms Strengthen Forensic Analysis, but Several Factors Can Affect Outcomes
GAO-21-435 — Forensic Technology: Algorithms Strengthen Forensic Analysis, but Several Factors Can Affect Outcomes

In July, the U.S. Government Accountability Office (GAO) released the report, Forensic Technology: Algorithms Strengthen Forensic Analysis, but Several Factors Can Affect Outcomes.

This is the second report in a two-part series of technology assessments responding to a request to examine the use of forensic algorithms in law enforcement. The first report, Forensic Technology: Algorithms Used in Federal Law Enforcement (GAO-20-479SP), described forensic algorithms used by federal law enforcement agencies and how they work.

In this report, GAO conducted an in-depth analysis of three types of algorithms used by federal law enforcement agencies and selected state and local law enforcement agencies: latent print, facial recognition and probabilistic genotyping. The report discusses

  1. the key performance metrics for assessing latent print, facial recognition and probabilistic genotyping algorithms;
  2. the strengths of these algorithms compared to related forensic methods;
  3. the key challenges affecting the use of these algorithms and the associated social and ethical implications; and
  4. options policymakers could consider to address these challenges.

GAO developed three policy options that could help address challenges related to law enforcement use of forensic algorithms. The policy options identify possible actions by policymakers, which may include Congress, other elected officials, federal agencies, state and local governments and industry.

In conducting this assessment, GAO interviewed federal officials, select non-federal law enforcement agencies and crime laboratories, algorithm vendors, academic researchers and nonprofit groups. It also convened an interdisciplinary meeting of 16 experts with assistance from the National Academies of Sciences, Engineering, and Medicine; and reviewed relevant literature. CSAFE co-director Karen Kafadar, professor and chair of statistics at the University of Virginia, participated in the meeting, as well as Will Guthrie, a CSAFE Research and Technology Transfer Advisory Board member. Guthrie is chief of the Statistical Engineering Division at the National Institute of Standards and Technology.

Read More:

Learn More:

CSAFE researchers are developing open-source software tools, allowing for peer-reviewed, transparent software for forensic scientists and researchers to apply to forensic evidence analysis. These automatic matching algorithms provide objective and reproducible scores as a foundation for a fair judicial process. Learn more about CSAFE’s open-source software tools.

A NEW NAME AND A NEW HORIZON: THE WILSON CENTER FOR SCIENCE AND JUSTICE

In recognition of a $5 million dollar grant from the Wilson Foundation, CSAFE partner institution Duke University renamed its Center for Science and Justice the Wilson Center for Science and Justice. Funds from the grant will expand the center’s work in three key areas: accuracy of evidence in criminal cases, the role of equity in criminal outcomes, and the mental and behavioral health treatment needs of people in the justice system. 

CSAFE’s own Co-Director, Brandon Garrett, the L. Neil Williams Jr. Professor of Law and Faculty Director of the center, aims for Duke Law School to build a long-term presence as a national leader in reforming the US criminal justice system. The Center, originally launched in September 2019, plans to encourage and instruct students and faculty on how to bolster legal and scientific data-driven research.  

“Bringing law and science together to prevent injustice has been my life’s work, since I was a young lawyer, and now, in my work with the new Wilson Center for Science and Justice at Duke,” said Professor Garrett. 

The Wilson Center champions the integration of students, faculty, and staff across multiple disciplines, gathering their expertise in law, public policy, medicine, the arts, and sciences, to create a unique lens for criminal justice research. The philosophy of interdisciplinary collaboration driving impactful research, policy, and teaching is what forms the foundation of our educational initiatives at CSAFE.

Strengthening research in the prevention of wrongful convictions, the examination of how jurors evaluate forensic evidence, and the accuracy of evidence in court are primary objectives of the Wilson Center — all of which cannot be achieved without the practice of scientifically valid methods. At its core, CSAFE is committed to the application of verified statistical and scientific techniques to ensure the accuracy of forensic analysis and interpretation and to uphold the fair administration of justice.

With more than just a strong alignment of legal and scientific ideology, CSAFE’s partnership with the Wilson Center for Science and Justice will strengthen the field of forensic science and remedy structural inequalities ingrained in the justice system.

Click here to learn more about CSAFE’s commitment to statistical and scientific data-driven research. 

Texas Forensic Science Commission Advises Implementation of OSAC Registry Standards for Crime Laboratories

In a unanimous October 2019 decision, the Texas Forensic Science Commission recommended that all crime laboratories accredited to perform forensic analysis in the State of Texas voluntarily adopt the Organization of Scientific Area Committees for Forensic Science (OSAC) standards for forensic science. The Commission is the first regulatory body in the United States to recommend the implementation of these standards.

OSAC standards found on the OSAC Registry describe best practices, explain scientific protocols and define minimum requirements for the field. Each standard aims to ensure the reliability and reproducibility of forensic analysis results.

CSAFE partner Houston Forensic Science Center has already announced the laboratory will adopt these standards. CEO and President Peter Stout states that his team continuously seeks to improve the services provided to the community, and adopting these standards is the next step in the process.

NIST created OSAC in 2014 in partnership with the Department of Justice. The organization is comprised of roughly 560 members with expertise in 25 forensic disciplines, in addition to general expertise in scientific research, measurement science, statistics, law, and policy. At this time, 12 standards are available on the OSAC Registry, with more than 200 in development.

Where to Find Answers to Your Forensic Science Questions

Do you have questions about fingerprint identification methods, applications of criminal behavior, or another forensic science-related inquiry? Head over to the International Association for Identification research library to find over 2,000 resources to uncover the answers. Compiled by the world’s oldest and largest criminal identification organization, here you will find archives, manuscripts, books periodicals and more, all related to scientific criminal investigation. Readers can even dive back into the 19th century to learn what the field of forensic science criminal investigation was like in its infancy, leading up to today’s advancements.

Find out more about IAI’s mission to educate, critique and publish methods, techniques and research in the physical forensic science disciplines on their website. Be sure to also mark your calendars for their Educational Conference on August 9-15, 2020.

Another great resource is the NIST compilation of influential and seminal papers relevant to the quantification of the weight of evidence. Find it on their website.

OSAC Public Update and Open Discussion Recordings Now Available

OSAC Logo

Did you miss The Organization of Scientific Area Committees for Forensic Science (OSAC)  Public Update & Open Discussion meeting? Videos and presentations are now available from the June 5, 2019 meeting held at NIST.

Featured presentations include updates from the five Chairs of the Scientific Area Committees (SAC), three Resource Committees, the Statistics Task Group, and the Forensic Science Standards Board (FSSB). Review the recording to learn about the latest activities for each OSAC unit, including new standards under development, identified research gaps, challenges being addressed, and priorities for the upcoming year. Audience questions and feedback are also available for viewing. Slides from the individual presenters can be found under the agenda section on the meeting website.

The OSAC works to strengthen the nation’s use of forensic science by facilitating the development of scientifically sound forensic science standards and by promoting the adoption of those standards by the forensic science community.

JustScience Lab: Bringing Empirical Research to Criminal Justice

Impactful research, policy, and teaching are the foundation of a new laboratory led by CSAFE Co-Director and leader in criminal justice outcomes from Duke University, Brandon Garrett.

Focusing on projects with a direct impact in North Carolina, the JustScience team seeks to engage with state and local government and community stakeholders to translate research into effective and practical policy. Garrett explains how JustScience Lab’s evidence-based work is making a difference in the field.

How does the JustScience Lab build mutually beneficial relationships with judicial and government officials?

The JustScience Lab sees research as a community service. We did not begin our work with a list of projects that we wanted to complete as researchers. Instead, we spent months reaching out to a wide range of community groups, policy groups, lawyers, lawmakers, and the courts, to listen and to ask how we could help.

We want to do work that fulfills needs and answer questions that the community found important.  Every one of our projects has been driven, not by us, but by people in the community that urgently sought answers to pressing questions in criminal justice.

Why is it important that researchers don’t work in isolation, but instead partner with individuals working in criminal justice?

Lawyers can help to identify questions that matter to the legal system and solutions that are feasible in the legal system. Conversely, lawyers are often unfamiliar with scientific methods or the types of solutions that scientific research can bring. That is one of the strengths of collaborations like CSAFE – bringing scientists, lawyers, and practitioners together to solve complex justice system problems.

Why does criminal justice reform need an evidence-based approach driven by data?

Any scientific work must follow the data, but it is often a real challenge to collect data in the criminal justice system because the courts often do not have the funding. As researchers, we have to be willing to get our hands dirty and assist the government in collecting the kind of data that the public could benefit from.

There are some public problems, where data may not make a difference because people have strong normative views and their choices are not evidence-based. For several decades, crime policy in the U.S. has been based on a retributive mindset, grounded in a moral view. It was not primarily about the effectiveness of crime policy. Today, attitudes have changed and that makes data and analysis much more relevant.

In what ways does JustScience Lab research connect to CSAFE efforts in forensic science?

We have a common project of communication at the intersection of law and science – bringing data to criminal justice. In building connections with prosecutors, police, and courts, the work that we are doing in JustScience on criminal justice more broadly, can help us engage with those same stakeholders in CSAFE, as we focus on improving forensic science.

As CSAFE identifies new methods that can benefit the field, we will also have to work to educate legal actors and explain how to use these new tools. Collaborative groups like the JustScience Lab can help with those efforts as well.

How can research groups like CSAFE and JustScience Lab translate these new research-based methods into practice?

There is a need for support for research applied in real-world settings. Not only does research have to illuminate the precise nature of a problem, but there needs to be work done to educate practitioners and develop solutions that can be practically implemented in a cost-efficient way.

Patience is also important because the legal system can operate through the legislative process and through judicial precedent, both of which can move quite slowly. That said, in many areas like wrongful convictions, change has happened far more rapidly than many might have thought two decades ago.

The connection between science and criminal justice has never been more vital, and organizations like CSAFE and JustScience Lab are using a collaborative approach to produce important research in key forensic disciplines.

To learn more about the projects JustScience Lab is working on, review this Duke Law News Story or visit the JustScience Lab website.

Remembering Daubert: A Recap of the CSAFE Symposium on Forensics, Statistics and Law at UVA

This is an invited blog post from Brandon Garrett, CSAFE researcher and White Burkett Miller Professor of Law and Public Affairs and Justice Thurgood Marshall Distinguished Professor of Law at University of Virginia School of Law. Guest blog posts do not necessarily reflect the views of CSAFE.  CSAFE is highlighting this topic due to our team’s commitment to advancing forensic science through enhanced connections between resesarch, statistics and the law.  

Twenty-five years ago, the U.S. Supreme Court ruled in Daubert v. Merrell Dow Pharmaceuticals Inc., that federal judges must conduct a scientific gatekeeping inquiry before admitting expert evidence.  A March 26, 2018 conference at the University of Virginia School of Law explored the challenges and the changes in the law, research and in the practice of forensic science in the years since Daubert was decided. The Virginia Journal of Criminal Law will publish as a symposium issue contributions by experts in forensics, statistics and the law from the event.  The conference was made possible by the Center for Statistics and Applications in Forensic Evidence.

The conference was live streamed and videotaped, with videos available on the UVA School of Law YouTube channel.

Available Sessions:
Forensics, Statistics and Law
Statistics, Research and Forensics
Judging Forensics
Statistics in the Crime Lab
Bringing Statistics into the Courtroom

Opening Remarks Highlights

The conference began with remarks describing how forensic science has changed in the twenty-five years since Daubert was decided and how researchers are beginning to address those shortcomings.  First, Professor Karen Kafadar (UVA) described the importance of statistics to forensic science.  Second, Sue Ballou, Program Manager at NIST, and incoming President of the American Academy of Forensic Science (AAFS), described her career as a forensic analyst and how her work came to engage more over time with questions of statistics and scientific research.  Ballou described, in remarks to be published in the symposium volume, how the CSAFE collaboration and the work of NIST researchers have done more to connect science with forensics. Third, Peter Neufeld described his career, including examples from cases in which he used scientific experts before he co-founded the Innocence Project and began to use DNA testing to free innocent convicts.

Examining How Statistics Applies to Forensic Science

Following those introductory remarks, the first panel discussed the role of statistics in forensic science.  Professor Alicia Carriquiry (Iowa State), whose remarks are to be published in this volume, described importance of statistics in forensics. Hari Iyer (NIST) addressed a different question: assuming there is quantified information about forensics, how are those statistics to be presented in court?  Iyer described recent work with Steven Lund, both at NIST, arguing that there are real concerns with the proposed use of likelihood ratios to express forensic conclusions, including because of the subjectivity of the decisions that are incorporated into such expressions.  Karen Kafadar (UVA), whose remarks are to be published in the volume, described not only the importance of bringing statistical rigor to forensic sciences, but also training on statistics and educational efforts to encourage future statisticians to examine practical and pressing problems in forensics.

Judge Jed Rakoff of the U.S. District Court for the Southern District of New York delivered the keynote address, which is to be published in this volume, asking and addressing why it is that judges have not acted as more forceful gatekeepers in the area of forensic science.

Linda Jackson, Director of the Virginia Department of Forensic Services described how the lab has long made all of its operating procedures available online and works closely with researchers.  Peter Stout, President of the Houston Forensic Science Center, whose remarks are to be published in the volume, described a substantial program to conduct blind proficiency testing in forensic disciplines.  Sharon Kelley (UVA) described case processing data of fingerprint examiner at HFSC, including data on how disagreements between examiners were resolved.  Henry Swofford, of the Defense Forensic Science Center described, in remarks published in this volume, a new program, called FRStat, developed in order to provide quantitative conclusions regarding fingerprint testing.

Investigating the Role of Statistics in the Courtroom

The final panel discussed the role of statistics in the courtroom.  Chancellor and Dean David Faigman (U.C. Hastings) described the need to attend to the connection between general research and individual evidence in criminal cases, as part of a larger question regarding how to associate the general to the individual.  David Kaye (Penn State Law) described several areas in which statistical inferences can be misstated or misleading.  A.J. Kramer, of the Federal Defender, provided a criminal defense perspective, explaining how judges have been almost entirely indifferent to challenges to unreliable forensic evidence, which has in turn discouraged defense lawyers from even raising challenges.  Professor Bobbie Spellman (UVA) argued that to explain forensic evidence to jurors, the goal should not be to train jurors or expect them to be amateur statisticians, but rather to provide them with the information they need to reach statistically sound results.

We could not be more grateful to each of the presenters and contributors to this remarkable Symposium.  We hope that the publications and the remarks themselves are of interest.  Twenty-five years after Daubert more remains to be done to connect research, statistics and the law that governs the forensic science evidence that has become an increasingly integral part of our system of criminal justice.

 

 

Misuse of Statistics in the Courtroom: The Sally Clark Case

The Center for Statistics and Applications in Forensic Evidence Director and Distinguished Professor of Statistics Dr. Alicia Carriquiry walks us through a landmark case that got statistics wrong.

When forensic practitioners, lawyers and other expert witnesses use statistics appropriately, juries have a tool that can help them make a more informed decision about the guilt or innocence of a suspect.  However, misrepresenting statistics in the courtroom can come at a high cost.

The Sally Clark Case is an infamous criminal case from the United Kingdom that illustrates how the use of inaccurate statistics in forensic science can lead to grave injustices.

Who was Sally Clark?

In December 1996, Sally Clark was home alone with her apparently healthy baby Christopher, aged 2 and half months. Sometime during the evening, Sally found her baby unresponsive and called an ambulance. Resuscitation attempts failed and the baby was pronounced dead. A post-mortem report suggested death of natural causes and a possible lower tract respiratory infection.

Less than two years later in January of 1998, Sally’s second baby Harry died at age two months under almost identical circumstances. The post-mortem report described signs of recent bleeding at the back of the eyes and in the spinal cord. Harry was an apparently healthy baby who was carefully observed by the UK’s Care of Next Infants program, which provides resources to families who have previously experienced the sudden death of an infant.

Alerted by the same pathologist who had performed the autopsy on Christopher, the police arrested Sally and her husband on murder charges. Officials quickly dropped the charges against Steve Clark, but accused Sally of smothering her two children.

Aside from the two dead infants, there was no other evidence to suggest murder. Sally’s friends and family praised her character and her mothering. Much of the evidence presented at trial for the prosecution came from medical experts, including renowned pediatrician Sir Roy Meadows.

The statistical argument

Using epidemiological data from the UK, Sir Meadows argued that two unexplained infant deaths (SIDS deaths) are extremely rare in a family such as the Clarks.

In the 1990s, the incidence of SIDS in the UK was approximately 1 in 8,500 in middle-class  families with no known risk factors. Sir Meadows further argued that the two deaths could be considered to be independent events and that the probability of observing two SIDS deaths in the Clark family could be computed as 1/8,500 squared, or 1 in 75 million.

The question is, can these two deaths really be treated as independent? Two SIDS deaths occurred in the same home under the supervision of the same two parents. How likely is it that those are unrelated events?

 Jury confusion: The Prosecutor’s Fallacy

It’s not hard to see that these statistics could easily be misunderstood and are likely prejudicial. Many people believe that the chance of a rare event happening is the same as the chance of a suspect’s innocence, an error known as the Prosecutor’s Fallacy. In reality, these two probabilities are not equal: saying that there is a 1 in 73 million chance that the babies died of SIDS is not the same as saying that there is a 1 in 73 million chance that the mother did not kill them.

 Understanding the Correct Statistical Applications

In Sally’s case, we have two extremely rare events to examine. Either the two babies died of SIDS, or the two babies were murdered. The probabilities of both events are difficult, if not impossible, to compute precisely, and both are likely to be extremely small. Luckily, we do not need precise estimates for those probabilities. Instead, we just need a reasonable estimate of their ratio.

Using UK crime statistics and the exact same reasoning that Sir Meadows used in his testimony against Sally, we find that the probability that two infants will be murdered in the same household is just 1 in 2 billion.

When considering the likelihood of two SIDS deaths and the likelihood of two murders, the odds of Sally’s guilt are very small, about 4%. It is in fact much more likely (about 95% more likely) that the babies died of SIDS than that they were murdered by their mother.

 What happened to Sally?

In 2003 Sally’s conviction was overturned on the basis of additional medical evidence of a possible bacterial infection in Harry. Sir Meadows used the same statistical arguments in other similar cases and was found guilty of professional misconduct in 2005, though the verdict was later overturned.

Sally Clark died on March 15, 2007, from a suspected heart attack thought to be a result of alcohol abuse. She was 42 and spent four years in prison because neither the judge nor the jurors could correctly interpret the significance of the numbers put forth by the expert witnesses.

Lessons we can learn

The Sally Clark case urges caution when using statistics in the courtroom. All parties must remember that there are always two (or more) sides to the story. Forensic scientists, jurors, lawyers and judges need to consider the probability of the evidence under the two competing hypotheses: the suspect is guilty or the suspect is not guilty. The ratio of these two probabilities is the likelihood ratio, and it can be approximated, if not explicitly calculated, in many cases. High values of the likelihood ratio would tend to support the hypothesis of guilt, while low values would tend to support the hypothesis of not guilty.

Armed with an accurate understanding of this key statistical concept, courts can prevent wrongful convictions such as Clark’s through the correct interpretation of evidence.