Category: Forensic Science

The Weight of Evidence Question: Resources to Help Guide the Discussion

When it comes to a piece of evidence, a jury wants to know just how likely it is that a crime scene bullet or shoeprint matches a given suspect. An analysis may almost certainly be correct, but how does an examiner account for the remote possibility that it wasn’t the same gun?

Quantifying the weight of evidence is no simple task, and integrating room for error and uncertainty has historically proven difficult in forensic science.

NIST takes a closer look at how strategies have evolved and shared the newest research in a publicly available collection of influential and seminal papers. The curated content is a valuable resource for anyone interested in learning more about the critical role quantifying evidence plays in the pursuit of justice.

The best way to approach the weight of evidence is often highly debated amongst the forensic science community and researchers. A lively discussion on limitations and implementation of statistical methods for quantifying the weight of evidence can be found by reviewing the 2016 NIST Technical Colloquium: Quantifying the Weight of Forensic Evidence session.

One approach to quantifying evidence, the likelihood ratio, is examined in detail by CSAFE team members and NIST personnel in a 2018 CSAFE webinar. View the panel discussion in the CSAFE education center.

Interested in training on this issue by our expert statisticians? Please contact our team.

NIST Explores New Strategies to Implement Innovation in Crime Laboratories

Research papers abound in forensic science journals each year, and yet often these innovative ideas never find their way into the crime laboratory. What exactly are the barriers, and how can the forensic science community successfully translate research into practice?

At the June 2019, Forensic Science Research Innovation to Implementation Symposium (RI2I), NIST leaders, forensic scientists, researchers and technology transfer experts joined together with laboratory professionals, business leaders and other stakeholders to brainstorm potential solutions.

If you missed the symposium, you can still tune in to hear the different perspectives from plenary sessions. Learn more about potential strategies to pave the way from fundamental research to innovation implementation on the NIST YouTube channel. Day One and Day Two sessions are both available for viewing.

Learn more about NIST forensic science research on their website, and discover how NIST and CSAFE partner to improve forensic science.

Toolmark vs. Firearm Analysis: Do the Same Methods Apply?

Just how closely related are methods for forensic toolmark and firearm examination? Though seemingly quite similar, analysis techniques for one may not be transferrable. CSAFE researchers take a closer look at this question in a new 2019 paper published in the Journal of Forensic Science.

In “Adapting the Chumbley Score to Match Striae on Land Engraved Areas (LEAs) of Bullets,” investigators examine if a statistical method developed for screwdriver markings successfully identifies striations on bullets.

While exploring Chumbley method performance on bullet analysis as opposed to screwdrivers, CSAFE researchers compare error rates. Results show that when judged against toolmarks, error rates increase.

Researchers discovered that parameter changes strongly impact the performance of the test. Next up, the CSAFE team plans to increase the test’s power with bullet-to-bullet comparisons.

Review the full article to learn more about this study. If you’d like a quick snapshot, view the CSAFE poster from AFTE 2018. For more information on CSAFE firearms and toolmark projects, visit our research page.

Influencing Forensic Science for a Decade: Remembering the 2009 NAS Report

10 years ago on February 18, 2009, The National Academies of Science unveiled a monumental report, shedding light on a critical need to reform the forensic sciences. “Strengthening Forensic Science in the United States: A Path Forward” acknowledged the dedication and expertise of professionals serving the forensic science community. However, it highlighted the limitations they face due to lack of adequate resources, sound policies and national support.

Report Brings Several Key Challenges to Light

Over the past several decades, forensic science has played a valuable role in criminal investigation convictions as well as exonerations. The 2009 report recognized progress, yet cautioned giving excessive weight to evidence and testimony derived from imperfect analysis methods. The NAS report outlined key contributing factors to faulty forensic science. Examples are as follows:

Absences of standardization in operational procedures
Lack of uniformity in certification of forensic practitioners or accreditation of crime laboratories
Unevenness in techniques, methodologies, reliability, error rates, research and more across forensic science disciplines
Variations in reliability of expert interpretation of evidence
Lack of rigor to consistently and with high degree of certainty demonstrate a connection between evidence an specific individual or source
Lack of research on established limits and measures of performance to address the impact of variability and bias

NAS Report Increases National Awareness of Forensic Science

The 2009 report launched a national discussion about the necessity of forensic science reform. From increased funding for forensic science research to re-examination of forensic techniques, the report has played a crucial role in courtrooms, legislation and executive actions over the past decade.

In his 2017 Harvard Law Review President Barack Obama stated the “2009 report from the National Academy of Sciences brought to light many of the challenges that the forensic sciences face in reliability and validity.” He, along with the FBI, recognized the need for quality research and technology development in forensic science as the government seeks to reform the criminal justice system.

Examples of governmental initiatives resulting from the goals of the NAS report include the launch of the National Commission on Forensic Science, the release of the President’s Council of Advisors on Science and Technology (PCAST) report, as well as the initiation of CSAFE, a NIST forensic science center of excellence created to address scientific concerns.

Initiation of CSAFE Addresses Strategic Recommendations of 2009 Report

The 2009 NAS report proposed several initiatives moving forward to increase the scientific foundations of forensic evidence. Recommendations included widespread adoption of uniform and enforceable best practices and better training for forensic science professionals. CSAFE has consistently responded to the call-to-action.

“The creation of the Forensic Science Center of Excellence marks a significant milestone in the national effort to establish more rigorous, science-based standards and practices for forensic evidence analysis,” said Willie May, NIST Director Under Secretary of Commerce for Standards and Technology.

CSAFE researchers are committed to long-term projects that revolutionize how forensic scientists and other stakeholders interpret and understand forensic evidence. By undertaking more than 30 active projects in the last five years, CSAFE has made substantial progress — and yet in a field as complex and vital as forensic science, many more foundational and complicated questions remain to be answered.

Looking Ahead: The 2009 NAS Report’s Lasting Impact

Today, the 2009 NAS report initiates collaboration amongst the scientific, forensic, criminal justice and judicial communities to find new solutions. CSAFE Director Dr. Alicia Carriquiry reflects on the report’s impact.

“Among all reports published by the National Research Council, the 2009 report on strengthening the forensic sciences must be among the most impactful. Much remains to be done, but it is undeniable that in the past decade, concepts such as uncertainty, the probability of a coincidental match, reliability and reproducibility of forensic methods and several others have found their way into the conversation about forensic science and its application. This is a good first step toward the goals set forth in the NRC report, as is the fact that collaboration among forensic scientists and the broader scientific community has also become more commonplace.”

Collaboration is at the heart of the CSAFE mission. With a culture strongly rooted in science, robust academic research community, and ample partnerships with forensic science colleagues, the CSAFE team of experts in statistics and forensic science have paved the way for advancements that directly address the concerns raised in the 2009 report. In the years to come, CSAFE, alongside our partners in the forensic and judicial communities aim to provide tangible tools forensic science experts can directly apply to their work.

Probative Value of Forensic Science Conclusions Should be Based on Empirical Data, Not Subjective Impressions

*This blog post was authored by the Innocence Project staff and originally appeared on the Innocence Project website. The piece is republished with the permission of the organization.*

The top statistical society in the United States issued guidelines for the statistically sound expression of the probative value of forensic evidence. The 2009 National Academy of Sciences report on Strengthening Forensic Science in the United States: The Path Forward and the 2016 President’s Council of Advisors on Science and Technology report on Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods have encouraged the use of empirical data and quantitative analyses as the basis of probabilistic statements that link a piece of evidence from a crime scene to a potential source.

The misapplication of forensic science is a contributing factor in 45% of the 362 wrongful convictions overturned by DNA evidence. Among the problems that account for the misapplication of forensic evidence is misleading forensic testimony that overstates or exaggerates the significance of similarities between evidence from a crime scene and evidence from an individual, or oversimplifies the data. In 2015, the FBI and U.S. Department of Justice formally acknowledged that FBI examiners gave flawed testimony regarding the probative value of microscopic hair comparison evidence in over 95% of the first 268 trials in which the evidence was used to ascertain the guilt of a defendant.

“Past errors provide us with opportunities to work with the forensic science community to improve the process and strengthen the use of forensic evidence,” said ASA president Karen Kafadar. “This statement is just one step in that direction, but we hope it will be an important step.”

The American Statistical Association (ASA) guidelines build upon a document guided by the late and esteemed statistician, Dr. Stephen Fienberg, at the National Commission on Forensic Science, but was not passed before the Department of Justice declined to renew its charter. In the absence of databases and empirical evidence, the interpretation of forensic evidence has been based on an examiner’s training and experience. The ASA statement explains why personal impressions are not sufficient for communicating scientific forensic conclusions to the investigators and fact finders who must make critical decisions based on that information.

“The ASA statement being adopted by the entire organization is extremely important because the ASA has an extraordinary reputation and a very large membership,” said Peter Neufeld, co-founder of the Innocence Project, the nonprofit legal organization that exonerates the wrongly convicted through DNA testing and reforms the criminal justice system to prevent future injustice.

“We have plenty of first-hand encounters where forensic examiners relied on their subjective impressions and personal experience instead of statistical data to explain the value of the evidence,” added Neufeld. “As a consequence, there were serious miscarriages of justice, where factually innocent people lost decades of their lives and some of them were sentenced to death. Hopefully policymakers, judges and people who really care about the strength of evidence used to make decisions about life and liberty will take notice and do everything they can to apply rigorous scientific and statistical principles in the future.”

Prominent members of the ASA also serve as lead investigators for the Center for Statistics and Applications in Forensic Evidence (CSAFE), a center of excellence sponsored by the National Institute of Standards and Technology. This research center “works to build a statistically sound and scientifically solid foundation for the analysis and interpretation of forensic evidence to grow competence in the forensic science and legal communities,” providing the foundations that would support valid and reliable interpretation of forensic evidence in the criminal justice system.

To read the ASA guidelines, click here.

To learn more about CSAFE, click here.

NIST Supports Accurate Forensic Science Measurements with Updated Standard Reference Materials

What role do measurements play in analyzing evidence? It’s actually pretty significant, but accuracy is key. When forensic laboratories use tools such as instruments, microscopes or DNA profiling kits it’s important to have physical standards.

The National Institute of Standards and Technology develops standard reference materials (SRMS) to help labs calibrate their analytical instruments. In September 2018, NIST released updated versions of two SRMS.

The Standard Bullet

Every bullet fired from a gun leaves unique markings. If a bullet is recovered at a crime scene, investigators can test-fire a suspect’s weapon to compare these bullet signatures.

A standard bullet is not real, but it looks like a typical bullet you’d see fired from a gun. Each standard bullet contains those markings called striations, simulating the impressions left by a gun.

Firearms examiners are able to use the standard bullet with advanced microscope technology to measure the striations on the NIST standard bullet. This helps test the instrument’s calibration and analysts can compare results to NIST data. A new manufacturing method has led to a more cost effective standard bullet, allowing access to more laboratories.

In a news release, the NIST the physical scientist who led the project stated, “This is one way to catch problems quickly. That way you can diagnose and fix the problem before doing additional casework.”

CSAFE not only owns a standard bullet, but our team’s firearm and ballistic research is leading to new technologies for bullet matching and cartridge case comparison. Find CSAFE tools for analyzing bullet lands or comparing cartridge case images on our resource page. CSAFE Director Dr. Alicia Carriquiry also recently contributed to an article highlighting the need for consistent standards in the firearms examination field.

Human DNA Quantitation

NIST also released a new SRM for Human DNA analysis. While the CSAFE team does not research human DNA, we partner with organizations such as the Innocence Project who work with this type of evidence.

If investigators discover blood or other biological evidence at a crime scene, analysts can extract DNA to create a profile to compare suspects. Accuracy relies on an analyst knowing how much DNA is in the extract prior to processing. NIST’s new standard for creating suspect DNA profiles assists with calibration for this type of measurement.

Learn more about other types of SRMs manufactured by NIST to ensure reliability of scientific evidence. Find additional information on NIST work and its partnership with CSAFE in our January 2018 blog post.

Forensics, Justice, and the Case for Science-Based Decision Making

*The information below is an excerpt from CSAFE researcher and University of California, Irvine’s Professor of Criminology, Law and Society Dr. Simon Cole’s recent blog post “Forensics, Justice, and the Case for Science-Based Decision Making” originally published November 14, 2018 on the Union of Concerned Scientists website. Review the full blog post.

Forensic science—and the language forensic scientists use to talk about their findings–has real-world impacts, sometimes life-or-death impacts, for real people. If the criminal justice system is going to really serve the cause of justice, it needs to be informed by the best available science.

In recent years, some progress has been made toward recognizing the inherently probabilistic nature of all scientific evidence and seeking ways of communicating those probabilities to lay audiences. The ULTRs signal that the DOJ is not yet ready to join that effort. This is unfortunate, given the DOJ’s power and influence.

Scientists don’t need to know anything about forensic science to understand that categorical statements of certainty are not plausible. Any scientist can help by letting the DOJ know that their statements are not scientifically credible and that the opinions of individual scientists and scientific institutions should be taken seriously by the nation’s most important purveyor of justice.

Overstating the certainty of forensic evidence has been implicated in many miscarriages of justice. And it is scientifically wrong. The people who are the ultimate consumers of forensic evidence deserve better.

Does the Age Old Technique of Fingerprinting Need an Update? A Closer Look at this Forensic Science Tool

Just because a technique has been used for over a 100 years, does that make it scientifically sound or reliable? The forensic science community is taking a closer look at this question in regards to fingerprinting, a long used tool in criminal investigations to help convict or rule out suspects.

News website Vox.com recently released a new video “How Reliable is Fingerprint Evidence,” featuring a forensic science expert that walks viewers through how fingerprinting is actually done, and digs a little deeper into the concerns surrounding its use.

CSAFE researcher and Professor of Criminology, Law and Society from University of California, Irvine Simon Cole also weighs in. “(Fingerprinting) is a useful tool that obviously has value, but I think it’s problematic to overstate its value especially in a criminal justice context.”

The video highlights the famous case of Brandon Mayfield and the 2004 Madrid train bombing where a false positive fingerprint identification had major consequences. Fingerprinting requires that investigators visually compare prints for similarity and differences. The Mayfield case clearly shows that due to the reliance on human judgment, error and cognitive bias can affect accuracy.

“I would suggest learning from history and realizing none of these things are going to be error free. So there’s going to be mistakes and errors and screw-ups but also evidence is inherently probabilistic,” Cole recommends.

In the search for more reliable technology, Cole and the team at CSAFE are working to use statistics to address key forensic science challenges in an effort to reduce human error and prevent the conviction of innocent people. Learn more about our research on the CSAFE website. To partner with CSAFE in developing increased scientific foundations for forensic evidence analysis, contact us to discuss collaboration opportunities.

The Evolution of Blood-Stain Pattern Analysis Verdicts in the Courtroom

Blood-stain pattern analysts are often called upon to help determine events surrounding a crime, and today their testimony is accepted in courts across the country. A recent New York Times Magazine article provides a window into the history of blood-stain pattern analysis and a closer look at how the discipline took hold in the American justice system.

Looking back to 1954, we can see that modern bloodstain-pattern analysis began with a small group of scientists and forensic investigators testifying as experts in a new technique. As the years progress, this group begins to train many more police officers, investigators and crime-lab technicians, who go on to testify in court as well.

However, in recent years, the training of these experts and the validity of the field itself have been called into question by various scientists and legal scholars.

Defendants began to appeal the legitimacy of the experts’ testimony, and their cases started to make their way to state appeals courts. Over time, the New York Times articles shows that once one court ruled that bloodstain testimony was admissible, others often followed suit, basing the decision on predecessor’s rulings. The timeline shows that judges have relied more on their own or the testifying experts’ assessment when examining the reliability and accuracy of blood stain pattern analysis, not objective proof.

The article highlights the groundbreaking 2009 National Academy of Sciences study that found major deficiencies in the United States forensic science field. The reports states, “The uncertainties of bloodstain-pattern analysis are enormous,” and highlights that analysts’ opinions are more subjective than scientific.

The Center for Statistics and Applications in Forensic Evidence was created in response to the 2009 NAS report, with a mission of building a stronger scientific foundation for the analysis and interpretation of forensic evidence. Our experts are studying a variety of forms of pattern and digital evidence, to include blood-stain analysis, and are working to develop new tools to improve accuracy. Learn how CSAFE is using statistical modeling software to gain a better understanding of how blood behaves, and more about a new blood spatter database designed to test analysis methods in our news section.

Human Versus Machine: NIST Study Investigates AI’s Role in Forensic Face Recognition

The debate of human versus machine continues to increase with rapidly changing technology. A new study by the National Institute of Standards and Technology confronts this conflict head-on with an investigation of accuracy in forensic face examination.

A photo from a security camera can play a critical role in criminal cases, determining the fate of the suspect. Trained forensic face examiners provide courtroom testimony, informing a jury whether that still image actually represents the accused.

NIST researchers and three university partners have combined the disciplines of forensic science, psychology and computer vision to determine just how good facial recognition experts really are. For the first time, researchers reveal the science behind the work of forensic face examiners, and raise the question of artificial intelligence’s role.

Researchers at NIST and organizations like CSAFE have revealed that algorithm performance has steadily increased over the past few years, and has valuable applications to forensic investigations. When comparing algorithms to human experts, the NIST study showed the highest accuracy in facial recognition was achieved with a collaboration of both the strengths of a human and a machine.

As a NIST Center of Excellence, CSAFE uses similar algorithmic approaches in other forensic disciplines to develop new tools to come alongside the human experts. Explore how our team is working to improve evidence analysis accuracy using algorithms in the comparison of cartridge cases and bullets.