Open Source Software: Applications in Forensics and The Courtroom

This is an invited blog post from Richard Torres, an attorney at the Legal Aid Society in New York City. Guest blog posts do not necessarily reflect the views of CSAFE. CSAFE is highlighting this topic due our team’s commitment to the development of open source software and its role in increasing the fair administration of justice. 

There is no national oversight body to protect our courts from bad forensic science. The scientific community is the only gatekeeper between questionable forensic practices and the courts, but scientists are not able to examine what they cannot see.

Forensic science software responsible for analyzing evidence is often built with closed source code. This prevents the public, scientists and ultimately, the legal system from being able to review the methods (or code) used to derive scientific conclusions – conclusions that are of utmost importance when someone’s freedom, and in some cases life, is on the line.

Software tools with open source, publically available code allow the scientific and legal communities to conduct a fair review of the processes and steps used to evaluate evidence. The case study below demonstrates the need for greater use of and access to open source software within the criminal justice system.

The Necessity of Open Source Code: A Case Study 

The New York City Office of the Chief Medical Examiner (OCME) was one of the first crime laboratories to implement probabilistic genotyping.  Probabilistic genotyping is a recent approach for interpreting complex DNA mixtures, many otherwise uninterpretable, and assigning them a statistical weight.  It seems that most forensic labs are heading in this direction.

OCME was early in the probabilistic genotyping game when they created a program called the Forensic Statistical Tool (FST).  The OCME, with the backing of the five New York City District Attorneys, fought hard to keep the source code for FST behind a tightly closed door even though it was created using tax payers’ dollars.

FST was brought online around 2011.  For over six years, OCME lab analysts would testify under oath as to FST results but were unable to explain how FST got its answer.  Most judges and jurors did not appear particularly concerned – people trust computers.

My office, The Legal Aid Society, repeatedly sought access to the FST source code in cases where our clients were facing years, even decades in prison.  We need to know if faulty forensic science is being used to imprison our clients.  Unfortunately, in state court, judges simply would not force OCME to share their code.

A lawyer in our office, Clinton Hughes, decided to build an open source program using the same exact calculations as FST called reQBT.  He recruited college interns with math and computer science backgrounds to assist with the finer details – we are lawyers, not scientists.  He ran reQBT on FST’s validation study data and, in many cases, reQBT got the same answers.  Clint’s team kept reviewing reQBT’s code and could not figure out why reQBT was getting different answers in some cases.  We had growing concerns over whether the problem was with reQBT.  Was FST’s computer code implementing OCME’s biological models reliably?

In late 2016, as FST was being replaced with a newer program called STRmix, Chris Flood and Sylvie Levine from the New York Federal Defenders’ Office were able to convince  Federal Judge Valerie Caproni to order OCME to provide the defense with the FST source code.  The Federal Defenders hired Nathan Adams of Forensic Bioinformatics to perform a code review – this was the first time a defense expert reviewed the FST source code.  Our suspicion that FST was not properly implementing OCME’s models was confirmed.  Adams found that FST performed its calculations differently than what OCME said and it affected FST’s results.  There was a catch.  Judge Caproni signed a court order preventing Adams from disclosing the specifics of the code problems.  FST remained mostly in the dark.

OCME did not immediately respond to Adams alleging in federal court that FST was performing different calculations from what OCME claimed.  There were no letters to defense attorneys stating that there may be a problem with FST.  It appears that the journals that published the FST studies were not notified either.  Were state prosecutors notified?

We renewed our requests to state court judges to order the source code given what Adams found.  Ultimately, OCME employees conceded that there was a code change that affected FST’s calculations – possibly a different code change from the one found by Adams.  Yet, there was no full scale validation study to establish how well the calculation changes worked.

The Legal Aid Society and Federal Defenders lodged a complaint to the New York State Inspector General requesting an investigation into the changes to FST’s code alongside other concerns about OCME.  FST was used in over a thousand cases.  Will they need to be reopened?  Are people in prison based on faulty forensics?

National press recently reported on our complaint to the Inspector General.  At that point, OCME did an about face as to open source. They now say they will share the FST source code.

Applying Lessons Learned to All Forensic Science Disciplines

This problem is not unique to DNA software.  Access to open source software is critical to the analysis of pattern and digital evidence.  Eliminating bias and error in forensic science is only possible through the total transparency open source code allows. Forensic scientists and researchers responsible for developing these tools must be committed to making their methods publicly available.  Lawyers and judges need this to be able to ensure the reliability of the adversarial trial process to better ensure that innocent people are not sitting in jail.

More information about the mission of CSAFE can be found on our homepage.  Learn more the impact the Legal Aid Society is making on the criminal justice system on their website.

Stronger Together: How Collaborative Research Is Paving the Way for Groundbreaking Innovation in Forensics

Scientific collaborative research is far from a new idea. Research papers rarely have one author. National funding organizations acknowledge the benefits of scientific collaboration. Today’s technologic advancements make working across institutions, across departments, between disciplines easier than ever.

Yet, as the Center for Statistics and Applications in Forensic Evidence, many are led to believe — and easily so — that we are a team composed solely of statisticians. While we do work with numerous statisticians, our research team comprises over 60 distinguished scientists and practitioners across departments and disciplines.

We collaborate with lawyers to analyze and research forensic testimony and evaluation of forensic evidence. We work with computer engineers, some of whom are undertaking mobile app forensic analysis. We have psychologists and behavior analysts on our team who investigate human factors at crime laboratories. We have mechanical, electrical and computer engineers some of whom are undertaking research in fluid dynamics in bloodstain pattern recognition.

How Collaborative Research Across Scientific Fields Expands Our Capabilities

The goals of CSAFE as a whole are to develop statistical foundations to help forensic scientists analyze and interpret evidence with consistent objectivity, and reduce human factors. We strive to help educate the community of forensic practitioners and other key stakeholders on how to communicate these complex results and implications in clear, honest ways. We work to translate groundbreaking research into practical applications for real-world forensic science investigations.

CSAFE’s key research areas require much time and effort, multi-disciplinary knowledge and a large network of people with varying capabilities and capacities.

Collaborative research also allows us to:

  • Share resources
  • Expand our scope of research
  • Lend more credibility to projects
  • Tackle the complex issues facing the forensic community

By incorporating a diverse group of scientists across fields of study, CSAFE has been able to undertake 29 current projects with others on queue for the future.

“The types of problems CSAFE aims to solve and the research in which we are engaged absolutely requires that we collaborate with scientists and practitioners,” said CSAFE Director Dr. Alicia Carriquiry.

Welcoming Collaborative Research Across All Fields of Science

CSAFE’s collaborative philosophy is simple and inclusive: By bringing together some of the most accomplished scientists and practitioners across scientific disciplines and from around the world, we are better equipped to build strong scientific foundations to apply to real-world forensic science investigations.

We welcome scientists from any field to contact us to discuss how you and your team could help undertake projects in our research areas.

Get to Know the Different Types of Forensic Scientists

At the Center for Statistics and Applications in Forensic Evidence (CSAFE), we often discuss the community of forensic scientists who drive our efforts to increase the scientific foundation of forensic science. Some may wonder, though, who are these forensic scientists? What do they do?

Unlike modern-day stereotypes of the forensic science field, forensic scientists cannot solve crimes in an hour’s time with absolute certainty. Forensic scientists of all types must work together to collect, analyze and interpret digital and pattern evidence, with the goal to bring the correct person to justice in a criminal investigation.

While “collecting,” “analyzing” and “interpreting” are easy words to say, they are far from simple in practice. Forensic scientists must sift through crime scenes for evidence deemed relevant to the crime. They analyze every inch of that evidence, look at it from all angles, and apply scientific principals and tests to understand that evidence’s significance and culpability. Even then, the results do not manifest in a black and white answer. Rarely does evidence provide 100 percent certainty — and nor can forensic scientists claim that their work is 100 percent error- and bias-free. That is where CSAFE’s statistical models support practitioners in the challenge to increase accuracy.

In the process of analyzing evidence, forensic scientists from different focus areas use their unique set of skills to help bring the correct person to justice. The information below provides an inside look at several common forensic scientist professions. Although the duties of each specialty differ and may seem unrelated, each plays a vital role in the discovery of the facts related to a crime.

Criminalist or Forensic Scientist: This forensic scientist works to reconstruct crime scenes in order to collect evidence. They analyze the results with the goal of determining the circumstances that led up to the crime and potential motives of the suspect. Their work requires a broad range of knowledge in scientific principles and effective deductive reasoning skills.

Blood Splatter Analyst: Experts assess the diameter and shape of bloodstains, which reflect the origin and trajectory of external blood flow. This professional works to help provide important clues as to the victim’s movements at the time of an assault, and movement of the person’s body afterwards.

Firearms Examiner: Professionals in this field analyze guns and weapons that may have been used to commit crimes. This professional may fire test shots from a certain weapon in order to compare bullet fragments or shell casings. They also may present ballistic evidence and their findings in court as an expert witness.

Digital Forensic Investigator: This is a type of forensic scientist who focuses on cyber crimes and works to discover evidence from computers, mobile devices, cloud computing, and electronic storage. This professional works within IT systems to extract data that can provide insights into the elements of a crime.

Forensic Anthropologist: Specialists study human bones and skeletal structures to determine age, gender, race and other identifying characteristics of a victim of a crime.

Pathologist: As a trained medical doctor, this person performs autopsies to assist with postmortem identification and to determine the cause of death of a victim of a crime. This professional may visit the scene of a crime to help investigators determine circumstances leading up to the fatality.

Questioned Document Examiner: This is a forensic scientist who analyzes handwriting and signatures for authenticity. They may also restore, decipher and assess missing pieces and pages of documents when needed.

Toxicologist: Combining the disciplines of toxicology, analytical chemistry, pharmacology and clinical chemistry, this scientist analyzes bodily fluids, hair, and nails for chemical substances to aid in investigations related to poisoning and drug use.

Forensic Psychologist: This specialty focuses on profiling persons involved in a crime through interviewing suspects, victims, and witnesses to help determine motives and mental states. They focus on issues such as competence to stand trial, testify or make medical decisions. They also assess mental illness as it relates to criminal responsibility,

Professor of Forensic Science: Working to to educate the next generation of forensic scientists, this professional typically has real world experience in the field, and helps students build knowledge of forensic science guided by scientific principles. Professors may teach undergraduate students, or work with graduate students pursuing a Master’s or Ph.D. degree. They may also lead a research group and study different areas of forensics.

Collectively, these scientists pool their efforts to use statistical foundations to reduce speculations in the analyzation of evidence and to limit human factors that influence decision making. Forensic practitioners work toward a common goal of communicating findings in clear, effective ways to the broader forensic and judicial communities. The mission of CSAFE is to align itself with these same goals and to build on the skills each team member brings to advance the field of forensics. To collaborate with CSAFE and help us accomplish these goals, contact us here.