Human Factors in Forensic Science: CSAFE Researchers Explain Cognitive Psychology’s Role

Is the field of science-based solely on logic and facts? A closer look reveals that science is not always as systematic and foolproof as we may think.

Scientists are humans, not robots. As such, the field of science has the potential to be compromised by human error, bias and other cognitive factors.

CSAFE researchers and psychologists Dr. Daniel Murrie and Dr. Sharon Kelley from the University of Virginia describe how human factors can influence the accuracy of scientific work as a whole, and the specific impact they can have on forensic science.

Defining Human Factors

Dr. Murrie explains that human factors are relevant any time that the human mind or behavior is part of the scientific process. “Human factors, including cognitive bias, can influence how we collect and analyze data, or how we communicate the findings,” he said.

The human brain is powerful and typically steers us in the right direction.

“Our minds are very efficient and that usually serves us well,” Murrie said. “Our brains develop many shortcuts or heuristics that help us make reasonably good decisions fast.”

While this approach to decision making seems practical and beneficial, it has a downside.

“There are a few times when that cognitive infrastructure can lead us to take short cuts or jump to conclusions,” Murrie said. “This can leave us vulnerable to bias in ways that aren’t helpful to science, accuracy or objectivity.”

Human Factors in Forensic Evidence Analysis

The 2009 NAS report calling for reform in forensic science has led to more awareness of human factors and driven new research on cognitive bias in evidence analysis.

For instance, when a fingerprint or shoeprint examiner compares the print of a suspect to a print from the crime scene, there is potential for human factors to bias the examiner’s decision on a match.

Dr. Kelley explains how.

“General social and cognitive psychology research indicates that (A) humans often see what they expect to see and that (B) humans tend to seek out and interpret information in a way that supports their pre-existing beliefs,” she said.

For example, forensic science examiner may have had an opinion about the guilt or innocence of a suspect before even beginning the examination. Whether intentional or not, it could lead to biased results.

Dr. Kelley emphasizes that forensic scientists must be aware of potential bias before expressing opinions or decisions on the outcome of evidence analysis.

Why Is Awareness of Bias, not Enough?

“The good news is that awareness of biases in forensic science is probably at an all-time high. The bad news is that people don’t quite know what to do with this new awareness,” Murrie said.

One concern from recent research is the “bias blind spot,” as cognitive psychologists call it.

“People generally recognize that bias is a problem, but they only see it in others—not themselves,” Murrie said.

Kelley explains that for those who do recognize their own biases, it’s hard to change on your own.

“Introspecting and knowing about the bias and then just trying hard not to be biased is not enough,” Kelley said. “We know from research that these biases often aren’t conscious and you can’t just scan yourself and check for them. Biases still creep into our decision making.”

Instead of squinting your eyes and focusing hard not to be biased, Kelley says procedural and systematic changes are needed.

Strategies to Lessen the Impact of Human Factors in Crime Labs

According to Kelley, crime labs are a very heterogeneous group. They differ in their understanding of cognitive bias and strategies to combat the issue.

So how can researchers like Murrie and Kelley help crime labs reduce the impact of human factors in their investigations?

Two strategies CSAFE is working on to reduce bias are blind verification procedures and context management.

Blind verification procedures serve as checks and balances when analyzing evidence.  In this method, a second examiner reviews a case with no information about what the first examiner concluded. Crime laboratories then have two independent decisions to compare. When the two examiners agree, there is more confidence that the analysis is accurate.

Context management involves limiting unnecessary contextual information about the suspect or the crime scene that is irrelevant to the evidence analysis task. For example, when comparing a set of fingerprints, the examiner doesn’t need to know the race or criminal record of the suspect, or even the results of DNA analyses, to do their job. Reducing potentially biasing information increases objectivity in evidence analysis.

Read more about how CSAFE research is addressing context management in crime laboratories.

How Does the Law Impact the Regulation of Forensic Evidence?

Recent advancements in technology have brought forensic science to the forefront in criminal investigations.  While investigators are increasingly relying on the scientific foundations of forensic science, constitutional regulation may not be keeping up.

Historically, the U.S. Supreme Court has not taken a firm stance on the type of oversight the constitution should provide concerning forensic science. However, increased reliance on evidence and analysis techniques in the courtroom is persuading the government to take a closer look.

CSAFE researcher and University of Virginia White Burkett Miller Professor of Law and Public Affairs and Justice Thurgood Marshall Distinguished Professor of Law Brandon Garrett recently released a comprehensive review of the latest issues on constitutional regulation of forensic science.

From the article:

“Despite decades of missed opportunities to adequately regulate forensics, in recent rulings the Supreme Court and lower courts increasingly focus on sound litigation of forensics. In an era of plea bargaining, the accuracy of forensic analysis depends far less on cross-examination at trial, and far more on sound lab techniques, full disclosure of strengths and limitations of forensic evidence to prosecutors and the defense, and careful litigation.”

Learn more in the Washington and Lee Law Review journal.

 

NIST: A Leader in Forensic Science and Valued CSAFE Partner

Imagine a world without standard measurements.  How would we quantify the weight of our food, or the length of material for clothes?  Without a standard system, confusion abounds. In 1901 the U.S. government created the National Institute of Standards and Technology (NIST). NIST is a non-regulatory federal agency within the U.S. Department of Justice.

Building the Foundation of Innovation

NIST’s goal is to promote U.S. innovation and industrial competiveness through advances in measurement in science, standards and technology.  Innovation relies on the ability to observe and measure which allows product control and standard procedure implementation.

In addition to applications in industry and manufacturing, universally accepted definitions of measurement play a key role in scientific advancement. NIST researchers are at the forefront of developing new technology and commercialization.

“From the smart electronic power grid and electronic health records to atomic clocks, innumerable products and services rely in some way on technology, measurement and standards provided by the National Institute of Standards and Technology.”

Benefits of Standard Measurement in Forensic Science

Did you know that standard measurement is key in forensic science? Universal procedures for collection, analysis and interpretation of forensic evidence help protect the innocent and convict the guilty.  NIST focuses on three components of improving forensic science standards.

  • Science- Leads the way in innovative research in many forensic science disciplines to include DNA, fingerprints, digital evidence and more. Experts increase forensic laboratory analytical method validity through physical reference standards and data.
  • Policy- Co-chaired the National Commission on Forensic Science to raise awareness on standards such as accreditation requirements for forensic science service providers.
  • Practice- Directs Organization of Scientific Area Committees for Forensic Science to develop science based standards and guidelines for forensic science disciplines.

CSAFE and NIST- Working together to Transform Forensic Science

NIST is committed to advancing efforts to solve deficiencies in forensic science standards.  In 2015, NIST established CSAFE, a Center for Excellence in forensic science.  NIST continues to provide funding and strategical guidance to CSAFE as the center works to build the scientific foundations of forensic science.  The CSAFE team benefits greatly from the expertise NIST brings to CSAFE research and education initiatives.  Working in close partnership, NIST and CSAFE are developing solutions to better recognize, collect, analyze and interpret evidence.

Sources:

https://www.nist.gov/speech-testimony/importance-basic-research-united-states-competitiveness

https://www.nist.gov/about-nist

https://www.nist.gov/topics/forensic-science

A Look Inside the CSI Effect

The CSI Effect

Are crime scene dramas to blame for unrealistic perceptions of  forensic science in the general public?

CBS launched the first episode of CSI: Crime Scene Investigation in the year 2000, and suddenly forensic science was the latest craze.  What was once a foreign field to many Americans was now at the forefront of our nation’s curiosity.  Nearly 20 years later, viewers are still on the edge of their seats, captivated by stories of crime.

Two important questions to consider:

  1. Does Hollywood get it right?
  2. How are crime dramas influencing real life investigations?

While it may come as a surprise to the everyday citizen, the forensic science community wants you to know that the answers are Hollywood is usually wrong, and television influences public perception more than you might think.

Forensic Science: Silver Screen vs. Real World

CSI, Law and Order and Bones’ investigators solve murders, robberies and more in merely 60 minutes. Evidence collection proceeds without difficulty.  Flashy technology analyzes evidence with ease. Suspects inevitably incriminate themselves.  Juries seem to understand expert witness testimony.  The evidence positively does or does not match a suspect. The court swiftly reaches a verdict.  Justice always prevails.

Actual cases are drastically different from fictional portrayals.  Determining specific events leading up to a crime and bringing the correct person to justice is not nearly as simple as on CSI.

Real crime scenes are messy and evidence can be hard to come by.  In reality, DNA is not always present as crime dramas would have you believe. Many crime dramas highlight sophisticated analysis techniques, but resources may not be available or it may not even be appropriate for certain types of evidence.  In the courtroom, expert witnesses can struggle to convey evidence analysis results in ways that make sense to a non-expert audience, leaving jurors confused and unsure how to proceed.  Often lawyers have to work extra hard to explain that conclusions about suspect identification are not always definitive.

Impacts in the courtroom: The CSI Effect

Exaggerated portrayals of forensic science on television may lead to what is known as The CSI Effect.  First reported by USA Today in 2004, it refers to the effect forensic science television programs potentially have on jurors.

While research has not definitively proven the CSI Effect, trends show that today’s jurors are seeking more unmistakable proof.  Many jurors are looking for sophisticated science to play a role in every trial, and find circumstantial evidence and eyewitness testimony less reliable.

Crime dramas like CSI suggest that deciding if evidence matches a suspect or not is always conclusive. In reality, probability forms the basis of evidence analysis.  This can be hard for jurors to understand.

Many feel that the CSI Effect may be increasing the burden of proof in the courtroom.  However, the forensic science community has a unique opportunity to turn public fascination of the criminal justice system into opportunities for training and education.

Forensic scientists, lawyers and judges are looking to researchers like the CSAFE team and NIST to find the best tools for explaining the weight of evidence to non-expert audiences.  Together, we can help the public recognize fact from fiction and bring the right person to justice.

Learn more about how CSAFE research is increasing the scientific foundations of forensic evidence.

 

4 Questions Answered About Pattern Evidence

Handwriting document

Witnesses of a crime often share their version of the events that led up to the offense.  But often, evidence left at the scene is what helps investigators understand the story.  Through careful collection, testing, analysis and interpretation techniques, investigators attempt to piece together the details of what took place.

A commonly found type of evidence is pattern evidence. Pattern evidence results from a specific pattern left by physical contact from an object and a surface, known as an impression or imprint.  Impressions are three-dimensional markings such as bullet markings, and imprints are two-dimensional, like a fingerprint.

   1. What are the potential sources of pattern evidence?

  • Fingerprints
  • Footwear imprints
  • Bullet marks
  • Tool marks
  • Handwritten documents
  • Blood stains

    2. What type of information might we want to learn from pattern evidence?

  • Shoeprints: Manufacturer, size, style, suspect’s direction and speed of travel
  • Bloodstains: Type of weapon, angle of impact, source of blood, location of victim in the scene

   3. Why is pattern evidence challenging?

Pattern evidence is unique in that it requires comparison of images.  At present, human interpretation forms the basis of pattern evidence analysis. This method leaves examiners open for bias, error and subjectivity that can affect their decisions.  Results may vary between examiners, and juries often struggle to understand the methods an examiner used to arrive at a conclusion.

  4. Why do we need to study pattern evidence?

Due to the current subjectivity inherent in the analysis of pattern evidence, organizations like CSAFE are working to develop standard, reproducible ways of analyzing evidence, and to propose effective ways of communicating results.  Through additional research, new tools can be built to take more precise measurements even under unfavorable circumstances, we can better understand the effects of time and environmental factors on evidence, as well as determine what characteristics make impression evidence unique when compared to all other types of evidence.

Learn more about CSAFE research in pattern evidence.

 

Should the Likelihood Ratio Be Used In the Courtroom? NIST Researchers Weigh In

When investigating and communicating the value of evidence, the forensic science community makes every effort to explain results to juries in objective and understandable ways.

The analysis of evidence involves the consideration of multiple hypotheses.  Let us take DNA evidence for example – What is the probability of observing a genotypic match if the biological sample was left by the suspect?  How likely is a match if someone else left the sample?

When quantifying the results of DNA analysis, forensic science experts often use the likelihood ratio. This technique enables experts to communicate the weight of the evidence with a single number.

Forensic DNA analysis rests on a firm foundation of biology, and the public often understands principles such as genetic inheritance and unique combinations of genetic markers. Thus, confidence in the power of DNA evidence to identify an individual is strong.

Forensic scientists have long since used the likelihood ratio effectively in cases where DNA evidence is prominent. We know how biology works and can successfully use statistics to quantify the strength of the evidence.

But what about other types of evidence?  Can the likelihood help us to quantify the value of evidence such as fingerprints or bullet striations or the chemical composition of glass? That is still up for debate.

National Institute of Standards and Technology (NIST) statisticians Steve Lund and Hari Iyer recently discussed the value of applying the likelihood ratio in the courtroom.  A review of their conclusions is available in a new article published in the NIST Journal of Research.

Evidence analysis and the likelihood ratio applied to DNA evidence is often cut and dry.  However, we can’t say the same for evidence such as bullets or fingerprints.  Why is that?  It is because the fundamental scientific underpinnings of DNA evidence are absent almost everywhere else. We do not have a physical or statistical model for how striations on bullets arise, and we don’t know what determines specific patterns in fingerprints.

The question then becomes-is it still appropriate to use the same statistical tool in areas such as pattern evidence when our level of understanding is not the same? According to Lund and Iyer, the answer is maybe not.  Or rather, not yet.

Because in those cases, the likelihood ratio may rest on assumptions that cannot be verified in practice, a degree of subjectivity is inevitable and it may happen that two experts arrive at different values of the likelihood ratio.  Thus, say Lund and Iyer, when a decision involves human judgment, caution is key.

Without established models on which to base conclusions in evidence beyond DNA, jurors may find it difficult to interpret the results of forensic analysis. Jurors need more information that just the value of a likelihood ratio, Lund and Iyer say.

Lund and Iyer advocate for complete transparency when explaining the value of evidence to members of a jury.  In this light, it is imperative that jurors understand every aspect of the analysis of a specific evidence, and of the methods that were used to draw conclusions.  Forensic scientists must also communicate their level of uncertainty in their results. Armed with additional context, jurors are then better prepared to make informed decisions.

As the groundwork for the statistical treatment of evidence is being laid, Lund and Iyer issue a challenge to the forensic science community.  When communicating the value of evidence in the courtroom, they call for research that does not focus exclusively on the likelihood ratio approach.  More work needs to be done, but they are confident that it is possible to develop more than one framework for the analysis of evidence other than DNA.

Review this NIST news article for more information.  Visit the NIST website to discover more forensic science research.