Skip to content

Implementation and Practice

Overarching GOALS

As new probabilistic and statistical methods are developed they must be implemented by the forensic community. CSAFE is focused on research to facilitate the implementation of methods and identify best practices for their use. Relevant research topics include finding optimal ways to communicate statistical conclusions –– both verbally and graphically, understanding the barriers to widespread implementation, and best practices for forensic practitioners, lawyers and judges.

brandon-garrett_500x500

Brandon Garrett

L. Neil Williams Professor of Law | Co-Director of CSAFE

Duke University

Thompson, William

William C Thompson

Professor

University of California Irvine

SimonColeHi_web

Simon Cole

Professor

University of California Irvine

Additional Team Members

Susan Vanderplas
susan.vanderplas@unl.edu

Nita Farahany
farahany@duke.edu

Gregory Mitchell
greg.mitchell@law.virginia.edu

Hal S. Stern
sternh@uci.edu

Robert Ramotowski
robert.ramotowski@nist.gov

Robert Thompson
robert.m.thompson@nist.gov

Dan Murrie
murrie@virginia.edu

Sharon Kelley
smk8n@virginia.edu

Nicholas Scurich
nscurich@uci.edu

Adele Quigley-McBride
adele.quigleymcbride@duke.edu

Jennifer Teitcher
jennifer.teitcher@duke.edu

 

focus Areas

Statistical evidence can be presented to attorneys, judges and jurors in a range of ways, from verbal descriptions of the weight of evidence to numerical scores to more complicated statistical summaries. CSAFE researchers are conducting studies with various forms of testimony and conclusions to determine if these new forms of testimony are better suited to minimize misinterpretation.

The goal of this project is to better understand how to convey forensic information to laypeople in a way that is accurate and comprehensible. Our research focused initially on terms used to convey forensic conclusions and how jurors understand them. We have studied how to convey proficiency and error rate information and begun to move towards more detailed mock trial scenarios — such as studies of competing experts testifying at trial.

First, in our new work, we need to better understand as a foundational matter what informs jurors regarding the strengths and limits of forensic evidence. Our prior work has given us an initial picture of this problem. In the next set of studies, we need more realistic designs and we need to keep up with developments in the field. We will examine new ways of conveying forensic information to lay jurors as the field moves towards new standards for terminology, including as developed by OSAC, and in some settings, the use of quantitative methods and conclusions. We are also moving towards more detailed designs, including with videos of mock courtroom testimony, and perhaps jury deliberation, to make for more realistic studies. We will examine what explains the varying weight that laypeople place on forensic testimony to better understand how to explain the strengths and limits of that evidence to them.

Second, we need to study new interventions that might change how laypeople evaluate forensic evidence. We will examine newly developed language, such as terminology developed by OSAC, language required by judges in their rulings, and its impact on lay decisionmakers. We need to study what effect jury instructions might have on visual presentations. Relatedly, in addition to studying lay decisionmaking, we will also assess what lawyers currently understand (and do not understand) about the evidence forensic experts currently present (or may, in the future, choose to present) in reports and testimony, so that we can better assess the role that lawyers play in presentation of forensic evidence. Similarly, we will survey judges to better understand their role and needs in this process (as also discussed in ED II). Research on how lawyers understand and misunderstand the language in forensic science reports may help forensic scientists develop more effective reporting language. And this work may help identify undesirable courtroom practices that might mislead jurors and help inform policy regarding how lawyers should conduct themselves at trial. Ultimately, we plan to develop model training for defense lawyers and judges that will also be of value to forensic scientists as they prepare for testimony and interact with legal professionals. 

Third, based on this research, we will develop recommendations for how to better convey strengths and limits of forensic evidence in testimony, to inform new standards for such testimony. We will also develop recommendations for judicial instructions and evidentiary standards regarding forensic conclusions. Finally, we will make recommendations for work that defense lawyers and prosecutors can do to better educate jurors regarding strengths and limits of forensic evidence.

It is a key goal of CSAFE to better understand the impediments to adoption of probabilistic methods among forensic science practitioners and members of the legal community, so that they can be overcome and probabilistic methods can be effectively implemented. It is important to study attitudes of practitioners towards probabilistic analyses including both the benefits and disadvantages that they perceive. CSAFE researchers achieve this insight through surveys, interviews and observations of forensic practitioners in order to assess obstacles and methods to overcome them.

To be useful, the statistical applications developed by CSAFE will have to be adopted by forensic scientists and forensic service providers (FSPs). It is well-known that many forensic scientists and FSPs are skeptical, or even resistant, to the adoption of statistical applications. To facilitate the adoption of statistical applications, we need a better understanding of the organizational cultures of the FSPs that we hope will adopt them. Such understanding can be facilitated by a sociology of forensic science. Further, we also need a better sociological understanding of the discipline of forensic statistics. Many non-statisticians profess themselves bewildered by forensic statistics. While forensic statistics has made efforts to account for itself, it will be helpful to complement these with a sociological account, from an external perspective, about the discipline. An account that locates forensic statistics in its historical and sociological context can only enhance non-statisticians’ understanding of the discipline and nature of the applications it is seeking to implement.

Sociology of science is a well-established discipline that uses the tools of the social sciences to understand the making of scientific knowledge. While most work in sociology of science has focused on more traditional academic disciplines, like physics, biology, medicine, and engineering, there has long been a thriving line of research on forensic science. Sociologists of science are particularly drawn to forensic science because of its proximity to law, which brings two powerful truth-making social institutions (science and law) into close contact.

While there has been some work on the sociology of forensic science, there has been almost no work on the sociology of forensic statistics.

By sociologically analyzing the impact of statistical application on forensic laboratories and the reactions of forensic scientists to statistical applications, the project will allow forensic scientists to progress from a “local” perspective informed by their own experiences and personal interactions to a more “global” perspective informed by the experiences of the discipline as a whole with statistical applications.

The research will draw on the standard tools in the sociology of science. These include social scientific methods, such as interviews, participant-observation and ethnography, as well as methods drawn more from history of science, such analyses of scholarly debates conducted through published literature.

Knowledge Transfer

  • Type

Found 43 Results
Page 1 of 3

A Pioneer in Forensic Science Reform: The Work of Paul Giannelli

Type: Research Area(s):

Published: 2018 | By: Brandon Garrett

Few can say, "I told you so," to our entire criminal justice system. Being right about what is wrong with the use of evidence in criminal cases is not a bad thing, but being able to influence the growing response…

View on Digital Repository


Probabilistic Reporting in Criminal Cases in the United States: A Baseline Study

Type: Research Area(s):

Published: 2020 | By: Simon Cole

Forensic evidence reporting shows a high degree of adherence to prevailing disciplinary standards.  Probabilistic reporting of forensic results remains rare.  Probabilistic reports were mostly subjective verbal assignments of posterior probabilities.

View on Digital Repository


How Can a Forensic Result Be a ‘Decision’? A Critical Analysis of Ongoing Reforms of Forensic Reporting Formats for Federal Examiners

Type: Research Area(s):

Published: 2020 | By: Simon A. Cole

The decade since the publication of the 2009 National Research Council report on forensic science has seen the increasing use of a new word to describe forensic results. What were once called “facts,” “determinations,” “conclusions,” or “opinions,” are increasingly described…

View on Digital Repository


Psychometric Analysis of Forensic Examiner Behavior

Type: Research Area(s): ,

Published: 2019 | By: Amanda Luby

Forensic science often involves the comparison of crime-scene evidence to a known-source sample to determine if the evidence and the reference sample came from the same source. Even as forensic analysis tools become increasingly objective and automated, final source identifications…

View on Digital Repository


Implementation of a Blind Quality Control Program in a Forensic Laboratory

Type: Research Area(s): ,

Published: 2019 | By: Callan Hund

A blind quality control (QC) program was successfully developed and implemented in the Toxicology, Seized Drugs, Firearms, Latent Prints (Processing and Comparison), Forensic Biology, and Multimedia (Digital and Audio/Video) sections at the Houston Forensic Science Center (HFSC). The program was…

View on Digital Repository


CSAFE 2020 All Hands Meeting

Type: Research Area(s): ,,,,,,,,

The 2020 All Hands Meeting was held May 12 and 13, 2020 and served as the closing to the last 5 years of CSAFE research and focused on kicking off new initiatives for the next phase of the center, CSAFE…


Error Rates, Likelihood Ratios, and Jury Evaluation of Forensic Evidence

Type: Research Area(s):

Published: 2020 | By: Brandon Garrett

Forensic examiners regularly testify in criminal cases, informing the jurors whether crime scene evidence likely came from a source. In this study, we examine the impact of providing jurors with testimony further qualified by error rates and likelihood ratios, for…

View on Digital Repository


The Costs and Benefits of Forensics

Type: Research Area(s):

Published: 2020 | By: Brandon L. Garrett

Supreme Court Justice Louis Brandeis famously wrote that states can be laboratories for experimentation in law and policy. Disappointingly, however, the actual laboratories that states and local governments run are not a home for experimentation. We do not have adequate…

View on Digital Repository


What do forensic analysts consider relevant to their decision making?

Type: Research Area(s):

Published: 2019 | By: Brett O. Gardner

In response to research demonstrating that irrelevant contextual information can bias forensic science analyses, authorities have increasingly urged laboratories to limit analysts' access to irrelevant and potentially biasing information (Dror and Cole (2010) [3]; National Academy of Sciences (2009) [18];…

View on Digital Repository


Perceptions and estimates of error rates in forensic science: A survey of forensic analysts

Type: Research Area(s):

Published: 2019 | By: Daniel C. Murrie

Every scientific technique features some error, and legal standards for the admissibility of scientific evidence (e.g., Daubert v. Merrill Dow Pharmaceuticals, Inc., 1993; Kumho Tire Co v. Carmichael, 1999) guide trial courts to consider known error rates. However, recent reviews…

View on Digital Repository


Implementation of a Blind Quality Control Program in Blood Alcohol Analysis

Type: Research Area(s):

Published: 2019 | By: Jackeline Moral

Declared proficiency tests are limited in their use for testing the performance of the entire system, because analysts are aware that they are being tested. A blind quality control (BQC) is intended to appear as a real case to the…

View on Digital Repository


After Uniqueness: The Evolution of Forensic Science Opinion

Type: Research Area(s):

Published: 2018 | By: Alex Biedermann

Big changes are occurring in forensic science, particularly among experts who compare the patterns found in fingerprints, footwear impressions, toolmarks, handwriting, and the like. Forensic examiners are reaching conclusions in new ways and changing the language they use in reports…

View on Digital Repository


Do evidence submission forms expose latent print examiners to task-irrelevant information?

Type: Research Area(s):

Published: 2019 | By: Brett O. Gardner

Emerging research documents the ways in which task-irrelevant contextual information may influence the opinions and decisions that forensic analysts reach regarding evidence (e.g., Dror and Cole, 2010; National Academy of Sciences, 2009; President’s Council of Advisors on Science and Technology,…

View on Digital Repository


The impact of proficiency testing information and error aversions on the weight given to fingerprint evidence

Type: Research Area(s):

Published: 2019 | By: Gregory Mitchell

Fingerprint examiners regularly participate in tests designed to assess their proficiency. These tests provide information relevant to the weight of fingerprint evidence, but no prior research has directly examined how jurors react to proficiency testing information. Using a nationally representative…

View on Digital Repository


How cross-examination on subjectivity and bias affects jurors’ evaluation of forensic science evidence

Type: Research Area(s):

Published: 2019 | By: Thompson, W.C.

Contextual bias has been widely discussed as a possible problem in forensic science. The trial simulation experiment reported here examined reactions of jurors at a county courthouse to cross‐examination and arguments about contextual bias in a hypothetical case. We varied…

View on Digital Repository


How Cross-Examination on Subjectivity and Bias Affects Jurors’ Evaluations of Forensic Science Evidence

Type: Research Area(s):

Published: 2019 | By: William C. Thompson

Contextual bias has been widely discussed as a possible problem in forensic science. The trial simulation experiment reported here examined reactions of jurors at a county courthouse to cross‐examination and arguments about contextual bias in a hypothetical case. We varied…

View on Digital Repository


Litigating Forensics Conclusions and Error Rates

Type: Research Area(s):

Published: 2019 | By: Brandon Garrett

This presentation provides an overview of current challenges in litigating forensics, exploring concepts such as subjectivity in analysis, ambiguity of data, lack of standards proficiency data and the use of error rates and looks at current research to address these…

View on Digital Repository


Probabilistic Reporting in American Criminal Cases: A Baseline Study

Type: Research Area(s):

Published: 2019 | By: Simon Cole

After attending this presentation, attendees will gain an empirically grounded understanding of the current state of probabilistic reporting in six criminalistic disciplines in the United States.

View on Digital Repository


Perceptions and estimates of error rates in forensic science: A survey of forensic analysts

Type: Research Area(s):

Published: 2019 | By: Daniel C. Murrie

The goal of this presentation is to educate attendees about commonly held beliefs of forensic analysts across multiple disciplines regarding the prevalence and acceptability of different types of errors (i.e., false positive and false negative errors) in their field.

View on Digital Repository


The Reliability of Forensics

Type: Research Area(s):

Published: 2019 | By: Brandon Garrett

This presentation examines the reliability of forensic evidence, including limitations of current techniques, how jurors evaluate evidence and the importance of match language, method information and error acknowledgements. The presentation also looks at the impact of proficiency testing and error…

View on Digital Repository


Page 1 of 3

COMMUNITY CALL-TO-ACTION

Want to collaborate with CSAFE on a project. Contact us to share your idea.

Do you have 44.03 seconds?

44.3 Seconds. That is the average amount of time it takes for a visitor to provide site feedback.
Test it yourself by taking the survey.


    A scientist/researcherA member of the forensic science communityA journalist/publicationA studentOther. Please indicate.


    Learn more about CSAFE overall.Discover research CSAFE is undertaking.Explore collaboration opportunities.Find tools and education opportunities.Other. Please indicate.


    YesNo