Author: scarraher

OSAC Public Update and Open Discussion Recordings Now Available

Did you miss The Organization of Scientific Area Committees for Forensic Science (OSAC) Public Update & Open Discussion meeting? Videos and presentations are now available from the June 5, 2019 meeting held at NIST.

Featured presentations include updates from the five Chairs of the Scientific Area Committees (SAC), three Resource Committees, the Statistics Task Group, and the Forensic Science Standards Board (FSSB). Review the recording to learn about the latest activities for each OSAC unit, including new standards under development, identified research gaps, challenges being addressed, and priorities for the upcoming year. Audience questions and feedback are also available for viewing. Slides from the individual presenters can be found under the agenda section on the meeting website.

The OSAC works to strengthen the nation’s use of forensic science by facilitating the development of scientifically sound forensic science standards and by promoting the adoption of those standards by the forensic science community.

JustScience Lab: Bringing Empirical Research to Criminal Justice

Impactful research, policy, and teaching are the foundation of a new laboratory led by CSAFE Co-Director and leader in criminal justice outcomes from Duke University, Brandon Garrett.

Focusing on projects with a direct impact in North Carolina, the JustScience team seeks to engage with state and local government and community stakeholders to translate research into effective and practical policy. Garrett explains how JustScience Lab’s evidence-based work is making a difference in the field.

How does the JustScience Lab build mutually beneficial relationships with judicial and government officials?

The JustScience Lab sees research as a community service. We did not begin our work with a list of projects that we wanted to complete as researchers. Instead, we spent months reaching out to a wide range of community groups, policy groups, lawyers, lawmakers, and the courts, to listen and to ask how we could help.

We want to do work that fulfills needs and answer questions that the community found important. Every one of our projects has been driven, not by us, but by people in the community that urgently sought answers to pressing questions in criminal justice.

Why is it important that researchers don’t work in isolation, but instead partner with individuals working in criminal justice?

Lawyers can help to identify questions that matter to the legal system and solutions that are feasible in the legal system. Conversely, lawyers are often unfamiliar with scientific methods or the types of solutions that scientific research can bring. That is one of the strengths of collaborations like CSAFE – bringing scientists, lawyers, and practitioners together to solve complex justice system problems.

Why does criminal justice reform need an evidence-based approach driven by data?

Any scientific work must follow the data, but it is often a real challenge to collect data in the criminal justice system because the courts often do not have the funding. As researchers, we have to be willing to get our hands dirty and assist the government in collecting the kind of data that the public could benefit from.

There are some public problems, where data may not make a difference because people have strong normative views and their choices are not evidence-based. For several decades, crime policy in the U.S. has been based on a retributive mindset, grounded in a moral view. It was not primarily about the effectiveness of crime policy. Today, attitudes have changed and that makes data and analysis much more relevant.

In what ways does JustScience Lab research connect to CSAFE efforts in forensic science?

We have a common project of communication at the intersection of law and science – bringing data to criminal justice. In building connections with prosecutors, police, and courts, the work that we are doing in JustScience on criminal justice more broadly, can help us engage with those same stakeholders in CSAFE, as we focus on improving forensic science.

As CSAFE identifies new methods that can benefit the field, we will also have to work to educate legal actors and explain how to use these new tools. Collaborative groups like the JustScience Lab can help with those efforts as well.

How can research groups like CSAFE and JustScience Lab translate these new research-based methods into practice?

There is a need for support for research applied in real-world settings. Not only does research have to illuminate the precise nature of a problem, but there needs to be work done to educate practitioners and develop solutions that can be practically implemented in a cost-efficient way.

Patience is also important because the legal system can operate through the legislative process and through judicial precedent, both of which can move quite slowly. That said, in many areas like wrongful convictions, change has happened far more rapidly than many might have thought two decades ago.

The connection between science and criminal justice has never been more vital, and organizations like CSAFE and JustScience Lab are using a collaborative approach to produce important research in key forensic disciplines.

To learn more about the projects JustScience Lab is working on, review this Duke Law News Story or visit the JustScience Lab website.

The Weight of Evidence Question: Resources to Help Guide the Discussion

When it comes to a piece of evidence, a jury wants to know just how likely it is that a crime scene bullet or shoeprint matches a given suspect. An analysis may almost certainly be correct, but how does an examiner account for the remote possibility that it wasn’t the same gun?

Quantifying the weight of evidence is no simple task, and integrating room for error and uncertainty has historically proven difficult in forensic science.

NIST takes a closer look at how strategies have evolved and shared the newest research in a publicly available collection of influential and seminal papers. The curated content is a valuable resource for anyone interested in learning more about the critical role quantifying evidence plays in the pursuit of justice.

The best way to approach the weight of evidence is often highly debated amongst the forensic science community and researchers. A lively discussion on limitations and implementation of statistical methods for quantifying the weight of evidence can be found by reviewing the 2016 NIST Technical Colloquium: Quantifying the Weight of Forensic Evidence session.

One approach to quantifying evidence, the likelihood ratio, is examined in detail by CSAFE team members and NIST personnel in a 2018 CSAFE webinar. View the panel discussion in the CSAFE education center.

Interested in training on this issue by our expert statisticians? Please contact our team.

NIST Explores New Strategies to Implement Innovation in Crime Laboratories

Research papers abound in forensic science journals each year, and yet often these innovative ideas never find their way into the crime laboratory. What exactly are the barriers, and how can the forensic science community successfully translate research into practice?

At the June 2019, Forensic Science Research Innovation to Implementation Symposium (RI2I), NIST leaders, forensic scientists, researchers and technology transfer experts joined together with laboratory professionals, business leaders and other stakeholders to brainstorm potential solutions.

If you missed the symposium, you can still tune in to hear the different perspectives from plenary sessions. Learn more about potential strategies to pave the way from fundamental research to innovation implementation on the NIST YouTube channel. Day One and Day Two sessions are both available for viewing.

Learn more about NIST forensic science research on their website, and discover how NIST and CSAFE partner to improve forensic science.

Toolmark vs. Firearm Analysis: Do the Same Methods Apply?

Just how closely related are methods for forensic toolmark and firearm examination? Though seemingly quite similar, analysis techniques for one may not be transferrable. CSAFE researchers take a closer look at this question in a new 2019 paper published in the Journal of Forensic Science.

In “Adapting the Chumbley Score to Match Striae on Land Engraved Areas (LEAs) of Bullets,” investigators examine if a statistical method developed for screwdriver markings successfully identifies striations on bullets.

While exploring Chumbley method performance on bullet analysis as opposed to screwdrivers, CSAFE researchers compare error rates. Results show that when judged against toolmarks, error rates increase.

Researchers discovered that parameter changes strongly impact the performance of the test. Next up, the CSAFE team plans to increase the test’s power with bullet-to-bullet comparisons.

Review the full article to learn more about this study. If you’d like a quick snapshot, view the CSAFE poster from AFTE 2018. For more information on CSAFE firearms and toolmark projects, visit our research page.

Influencing Forensic Science for a Decade: Remembering the 2009 NAS Report

10 years ago on February 18, 2009, The National Academies of Science unveiled a monumental report, shedding light on a critical need to reform the forensic sciences. “Strengthening Forensic Science in the United States: A Path Forward” acknowledged the dedication and expertise of professionals serving the forensic science community. However, it highlighted the limitations they face due to lack of adequate resources, sound policies and national support.

Report Brings Several Key Challenges to Light

Over the past several decades, forensic science has played a valuable role in criminal investigation convictions as well as exonerations. The 2009 report recognized progress, yet cautioned giving excessive weight to evidence and testimony derived from imperfect analysis methods. The NAS report outlined key contributing factors to faulty forensic science. Examples are as follows:

Absences of standardization in operational procedures
Lack of uniformity in certification of forensic practitioners or accreditation of crime laboratories
Unevenness in techniques, methodologies, reliability, error rates, research and more across forensic science disciplines
Variations in reliability of expert interpretation of evidence
Lack of rigor to consistently and with high degree of certainty demonstrate a connection between evidence an specific individual or source
Lack of research on established limits and measures of performance to address the impact of variability and bias

NAS Report Increases National Awareness of Forensic Science

The 2009 report launched a national discussion about the necessity of forensic science reform. From increased funding for forensic science research to re-examination of forensic techniques, the report has played a crucial role in courtrooms, legislation and executive actions over the past decade.

In his 2017 Harvard Law Review President Barack Obama stated the “2009 report from the National Academy of Sciences brought to light many of the challenges that the forensic sciences face in reliability and validity.” He, along with the FBI, recognized the need for quality research and technology development in forensic science as the government seeks to reform the criminal justice system.

Examples of governmental initiatives resulting from the goals of the NAS report include the launch of the National Commission on Forensic Science, the release of the President’s Council of Advisors on Science and Technology (PCAST) report, as well as the initiation of CSAFE, a NIST forensic science center of excellence created to address scientific concerns.

Initiation of CSAFE Addresses Strategic Recommendations of 2009 Report

The 2009 NAS report proposed several initiatives moving forward to increase the scientific foundations of forensic evidence. Recommendations included widespread adoption of uniform and enforceable best practices and better training for forensic science professionals. CSAFE has consistently responded to the call-to-action.

“The creation of the Forensic Science Center of Excellence marks a significant milestone in the national effort to establish more rigorous, science-based standards and practices for forensic evidence analysis,” said Willie May, NIST Director Under Secretary of Commerce for Standards and Technology.

CSAFE researchers are committed to long-term projects that revolutionize how forensic scientists and other stakeholders interpret and understand forensic evidence. By undertaking more than 30 active projects in the last five years, CSAFE has made substantial progress — and yet in a field as complex and vital as forensic science, many more foundational and complicated questions remain to be answered.

Looking Ahead: The 2009 NAS Report’s Lasting Impact

Today, the 2009 NAS report initiates collaboration amongst the scientific, forensic, criminal justice and judicial communities to find new solutions. CSAFE Director Dr. Alicia Carriquiry reflects on the report’s impact.

“Among all reports published by the National Research Council, the 2009 report on strengthening the forensic sciences must be among the most impactful. Much remains to be done, but it is undeniable that in the past decade, concepts such as uncertainty, the probability of a coincidental match, reliability and reproducibility of forensic methods and several others have found their way into the conversation about forensic science and its application. This is a good first step toward the goals set forth in the NRC report, as is the fact that collaboration among forensic scientists and the broader scientific community has also become more commonplace.”

Collaboration is at the heart of the CSAFE mission. With a culture strongly rooted in science, robust academic research community, and ample partnerships with forensic science colleagues, the CSAFE team of experts in statistics and forensic science have paved the way for advancements that directly address the concerns raised in the 2009 report. In the years to come, CSAFE, alongside our partners in the forensic and judicial communities aim to provide tangible tools forensic science experts can directly apply to their work.

Probative Value of Forensic Science Conclusions Should be Based on Empirical Data, Not Subjective Impressions

*This blog post was authored by the Innocence Project staff and originally appeared on the Innocence Project website. The piece is republished with the permission of the organization.*

The top statistical society in the United States issued guidelines for the statistically sound expression of the probative value of forensic evidence. The 2009 National Academy of Sciences report on Strengthening Forensic Science in the United States: The Path Forward and the 2016 President’s Council of Advisors on Science and Technology report on Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods have encouraged the use of empirical data and quantitative analyses as the basis of probabilistic statements that link a piece of evidence from a crime scene to a potential source.

The misapplication of forensic science is a contributing factor in 45% of the 362 wrongful convictions overturned by DNA evidence. Among the problems that account for the misapplication of forensic evidence is misleading forensic testimony that overstates or exaggerates the significance of similarities between evidence from a crime scene and evidence from an individual, or oversimplifies the data. In 2015, the FBI and U.S. Department of Justice formally acknowledged that FBI examiners gave flawed testimony regarding the probative value of microscopic hair comparison evidence in over 95% of the first 268 trials in which the evidence was used to ascertain the guilt of a defendant.

“Past errors provide us with opportunities to work with the forensic science community to improve the process and strengthen the use of forensic evidence,” said ASA president Karen Kafadar. “This statement is just one step in that direction, but we hope it will be an important step.”

The American Statistical Association (ASA) guidelines build upon a document guided by the late and esteemed statistician, Dr. Stephen Fienberg, at the National Commission on Forensic Science, but was not passed before the Department of Justice declined to renew its charter. In the absence of databases and empirical evidence, the interpretation of forensic evidence has been based on an examiner’s training and experience. The ASA statement explains why personal impressions are not sufficient for communicating scientific forensic conclusions to the investigators and fact finders who must make critical decisions based on that information.

“The ASA statement being adopted by the entire organization is extremely important because the ASA has an extraordinary reputation and a very large membership,” said Peter Neufeld, co-founder of the Innocence Project, the nonprofit legal organization that exonerates the wrongly convicted through DNA testing and reforms the criminal justice system to prevent future injustice.

“We have plenty of first-hand encounters where forensic examiners relied on their subjective impressions and personal experience instead of statistical data to explain the value of the evidence,” added Neufeld. “As a consequence, there were serious miscarriages of justice, where factually innocent people lost decades of their lives and some of them were sentenced to death. Hopefully policymakers, judges and people who really care about the strength of evidence used to make decisions about life and liberty will take notice and do everything they can to apply rigorous scientific and statistical principles in the future.”

Prominent members of the ASA also serve as lead investigators for the Center for Statistics and Applications in Forensic Evidence (CSAFE), a center of excellence sponsored by the National Institute of Standards and Technology. This research center “works to build a statistically sound and scientifically solid foundation for the analysis and interpretation of forensic evidence to grow competence in the forensic science and legal communities,” providing the foundations that would support valid and reliable interpretation of forensic evidence in the criminal justice system.

To read the ASA guidelines, click here.

To learn more about CSAFE, click here.

NIST Supports Accurate Forensic Science Measurements with Updated Standard Reference Materials

What role do measurements play in analyzing evidence? It’s actually pretty significant, but accuracy is key. When forensic laboratories use tools such as instruments, microscopes or DNA profiling kits it’s important to have physical standards.

The National Institute of Standards and Technology develops standard reference materials (SRMS) to help labs calibrate their analytical instruments. In September 2018, NIST released updated versions of two SRMS.

The Standard Bullet

Every bullet fired from a gun leaves unique markings. If a bullet is recovered at a crime scene, investigators can test-fire a suspect’s weapon to compare these bullet signatures.

A standard bullet is not real, but it looks like a typical bullet you’d see fired from a gun. Each standard bullet contains those markings called striations, simulating the impressions left by a gun.

Firearms examiners are able to use the standard bullet with advanced microscope technology to measure the striations on the NIST standard bullet. This helps test the instrument’s calibration and analysts can compare results to NIST data. A new manufacturing method has led to a more cost effective standard bullet, allowing access to more laboratories.

In a news release, the NIST the physical scientist who led the project stated, “This is one way to catch problems quickly. That way you can diagnose and fix the problem before doing additional casework.”

CSAFE not only owns a standard bullet, but our team’s firearm and ballistic research is leading to new technologies for bullet matching and cartridge case comparison. Find CSAFE tools for analyzing bullet lands or comparing cartridge case images on our resource page. CSAFE Director Dr. Alicia Carriquiry also recently contributed to an article highlighting the need for consistent standards in the firearms examination field.

Human DNA Quantitation

NIST also released a new SRM for Human DNA analysis. While the CSAFE team does not research human DNA, we partner with organizations such as the Innocence Project who work with this type of evidence.

If investigators discover blood or other biological evidence at a crime scene, analysts can extract DNA to create a profile to compare suspects. Accuracy relies on an analyst knowing how much DNA is in the extract prior to processing. NIST’s new standard for creating suspect DNA profiles assists with calibration for this type of measurement.

Learn more about other types of SRMs manufactured by NIST to ensure reliability of scientific evidence. Find additional information on NIST work and its partnership with CSAFE in our January 2018 blog post.

Forensics, Justice, and the Case for Science-Based Decision Making

*The information below is an excerpt from CSAFE researcher and University of California, Irvine’s Professor of Criminology, Law and Society Dr. Simon Cole’s recent blog post “Forensics, Justice, and the Case for Science-Based Decision Making” originally published November 14, 2018 on the Union of Concerned Scientists website. Review the full blog post.

Forensic science—and the language forensic scientists use to talk about their findings–has real-world impacts, sometimes life-or-death impacts, for real people. If the criminal justice system is going to really serve the cause of justice, it needs to be informed by the best available science.

In recent years, some progress has been made toward recognizing the inherently probabilistic nature of all scientific evidence and seeking ways of communicating those probabilities to lay audiences. The ULTRs signal that the DOJ is not yet ready to join that effort. This is unfortunate, given the DOJ’s power and influence.

Scientists don’t need to know anything about forensic science to understand that categorical statements of certainty are not plausible. Any scientist can help by letting the DOJ know that their statements are not scientifically credible and that the opinions of individual scientists and scientific institutions should be taken seriously by the nation’s most important purveyor of justice.

Overstating the certainty of forensic evidence has been implicated in many miscarriages of justice. And it is scientifically wrong. The people who are the ultimate consumers of forensic evidence deserve better.

Does the Age Old Technique of Fingerprinting Need an Update? A Closer Look at this Forensic Science Tool

Just because a technique has been used for over a 100 years, does that make it scientifically sound or reliable? The forensic science community is taking a closer look at this question in regards to fingerprinting, a long used tool in criminal investigations to help convict or rule out suspects.

News website Vox.com recently released a new video “How Reliable is Fingerprint Evidence,” featuring a forensic science expert that walks viewers through how fingerprinting is actually done, and digs a little deeper into the concerns surrounding its use.

CSAFE researcher and Professor of Criminology, Law and Society from University of California, Irvine Simon Cole also weighs in. “(Fingerprinting) is a useful tool that obviously has value, but I think it’s problematic to overstate its value especially in a criminal justice context.”

The video highlights the famous case of Brandon Mayfield and the 2004 Madrid train bombing where a false positive fingerprint identification had major consequences. Fingerprinting requires that investigators visually compare prints for similarity and differences. The Mayfield case clearly shows that due to the reliance on human judgment, error and cognitive bias can affect accuracy.

“I would suggest learning from history and realizing none of these things are going to be error free. So there’s going to be mistakes and errors and screw-ups but also evidence is inherently probabilistic,” Cole recommends.

In the search for more reliable technology, Cole and the team at CSAFE are working to use statistics to address key forensic science challenges in an effort to reduce human error and prevent the conviction of innocent people. Learn more about our research on the CSAFE website. To partner with CSAFE in developing increased scientific foundations for forensic evidence analysis, contact us to discuss collaboration opportunities.