Statistical Thinking for Forensic Practitioners Short Course – Session 4

This event is scheduled to take place on Friday, November 4, 2022. A registration form can be found below. 

Session 4 Description:

Session 4: Quantitative Tools for Forensic Evidence Part 2 is the final session in the four-part short course, Statistical Thinking for Forensic Practitioners. 

The National Academies report in 2009 and the PCAST report in 2016 encouraged consideration of quantitative approaches to assessing forensic evidence. In this session, we review two of the most often-suggested quantitative approaches, the two-stage approach and the likelihood ratio (Bayes factor) approach. Specific topics include:

  • The two-stage approach to assessing forensic evidence
    • The role of statistical tests in assessing the similarity of two samples
    • Approaches for assessing the relevance of observed similarities
  • Introduction to the likelihood ratio approach
    • Definition and interpretation of the likelihood ratio/Bayes factor
    • Possible applications to different types of evidence (DNA, trace, pattern)
    • Score-based likelihood ratios
    • Sensitivity of the likelihood ratio to modeling choices
  • Results of studies comparing different ways of expressing source conclusions

Presenter:
Hal Stern
Co-Director of CSAFE
Provost, Executive Vice Chancellor, and Chancellor’s Professor – University of California, Irvine

About the Short Course:
Session four is the final session in the four-session short course, Statistical Thinking for Forensic Practitioners. Dr. Hal Stern introduces fundamental concepts from probability and statistics –– motivated by forensic issues –– followed by a detailed investigation of how they apply to assess forensic evidence’s probative value. This short course will be held online in three sessions. Each session builds upon the previous one(s) and recordings will be available in the event registrants are unable to attend one or more of the live sessions. Researchers, collaborators, and members of the broader forensics and statistics communities are encouraged to attend. Short course registrants who attend all sessions will receive a certificate of completion.

Registration:

Statistical Thinking for Forensic Practitioners Short Course – Session 3

This event is scheduled to take place on Friday, October 28, 2022. A registration form can be found below. 

Session 3 Description:

Session 3: Quantitative Tools for Forensic Evidence is the third session in the four-part short course, Statistical Thinking for Forensic Practitioners. 

The National Academies report in 2009 and the PCAST report in 2016 encouraged consideration of quantitative approaches to assessing forensic evidence. In this session, we review two of the most often-suggested quantitative approaches, the two-stage approach and the likelihood ratio (Bayes factor) approach. Specific topics include:

  • The two-stage approach to assessing forensic evidence
    • The role of statistical tests in assessing the similarity of two samples
    • Approaches for assessing the relevance of observed similarities
  • Introduction to the likelihood ratio approach
    • Definition and interpretation of the likelihood ratio/Bayes factor
    • Possible applications to different types of evidence (DNA, trace, pattern)
    • Score-based likelihood ratios
    • Sensitivity of the likelihood ratio to modeling choices
  • Results of studies comparing different ways of expressing source conclusions

Presenter:
Hal Stern
Co-Director of CSAFE
Provost, Executive Vice Chancellor, and Chancellor’s Professor – University of California, Irvine

About the Short Course:
Session three is the third session in the four-part short course, Statistical Thinking for Forensic Practitioners. Dr. Hal Stern introduces fundamental concepts from probability and statistics –– motivated by forensic issues –– followed by a detailed investigation of how they apply to assess forensic evidence’s probative value. This short course will be held online in four sessions. Each session builds upon the previous one(s) and recordings will be available in the event registrants are unable to attend one or more of the live sessions. Researchers, collaborators, and members of the broader forensics and statistics communities are encouraged to attend. Short course registrants who attend all sessions will receive a certificate of completion.

Registration:

Statistical Thinking for Forensic Practitioners Short Course – Session 2

This event is scheduled to take place on Friday, October 21, 2022. A registration form can be found below. 

Session 2 Description:

Session 2: Sampling, Statistics and the Status Quo is the second session in the four-part short course, Statistical Thinking for Forensic Practitioners. 

Statistical ideas such as sampling, point and interval estimation of population quantities, and hypothesis testing have direct application in forensic science. In this session, we start from fundamental ideas about variability (and its sources) in measurements used in forensic analysis, and discuss methods to ameliorate, quantify, model, and interpret variation and uncertainty in the evaluation of forensic evidence. In more detail, we will:

  • Talk about variability and its sources, and introduce ideas such as reproducibility, repeatability, and accuracy.
  • Briefly talk about populations and samples, and describe some sampling methods useful in forensic applications.
  • Introduce the idea of estimation of population quantities such as means and proportions and of methods to report the uncertainty attached to those estimates.
  • Describe how to carry out a test of hypothesis to compare two means and a test of equivalence to compare two means.
  • Demonstrate how statistical concepts can be used in the study of current forensic science practice (e.g., design and analysis of black box studies)

Presenter:
Hal Stern
Co-Director of CSAFE
Provost, Executive Vice Chancellor, and Chancellor’s Professor – University of California, Irvine

About the Short Course:

Session two is the second session in the four-part short course, Statistical Thinking for Forensic Practitioners. Dr. Hal Stern introduces fundamental concepts from probability and statistics –– motivated by forensic issues –– followed by a detailed investigation of how they apply to assess forensic evidence’s probative value. This short course will be held online in four sessions. Each session builds upon the previous one(s) and recordings will be available in the event registrants are unable to attend one or more of the live sessions. Researchers, collaborators, and members of the broader forensics and statistics communities are encouraged to attend. Short course registrants who attend all sessions will receive a certificate of completion.

 

Registration:

Statistical Thinking for Forensic Practitioners Short Course – Session 1

This event is scheduled to take place on Friday, October 14, 2022. A registration form can be found below. 

Session 1 Description:

Probability Concepts and their Relevance to Forensic Science is the first session in the three-session short course, Statistical Thinking for Forensic Practitioners. 

Probability is the mathematical language of uncertainty. Probabilities are used to describe the frequency or likelihood of events or to characterize measurement uncertainty. In this first session, we introduce the laws of probability and their application in forensic settings. Specific topics include:

  • Definition and interpretation of probability
  • Basic laws of probability
  • Conditional probability and independence of events
  • Bayes’ Theorem and Bayesian statistics

Topics are illustrated with examples drawn from forensic science and relevant legal cases.

Presenter:

Hal Stern
Co-Director of CSAFE
Provost, Executive Vice Chancellor, and Chancellor’s Professor – University of California, Irvine

About the Short Course:

Session 1 is the first session in the four-part short course, Statistical Thinking for Forensic Practitioners. Dr. Hal Stern introduces fundamental concepts from probability and statistics –– motivated by forensic issues –– followed by a detailed investigation of how they apply to assess forensic evidence’s probative value. This short course will be held online in four sessions. Each session builds upon the previous one(s) and recordings will be available in the event registrants are unable to attend one or more of the live sessions. Researchers, collaborators, and members of the broader forensics and statistics communities are encouraged to attend. Short course registrants who attend all sessions will receive a certificate of completion.

Registration:

Webinar: Blind Testing in Firearms Examination

CSAFE invites researchers, collaborators, and members of the broader forensics and statistics communities to participate in our Spring 2022 Webinar Series on Thursday, June 23, 2022, from 11:00am-Noon CT. The presentation will be “Blind Testing in Firearms Examination: Preliminary Results, Benefits, Limitations, and Future Directions.” 

Presenter:
Maddisen Neuman
Quality / Research Associate – Houston Forensic Science Center

Presentation Description:
Open proficiency tests created by external vendors are used to meet accreditation requirements and demonstrate examiner competence; however, the tests are limited in the ability to monitor the entire quality management system from evidence submission to reporting of results. As such, many scholars have considered blind proficiency testing to be a more precise testing an individual’s accuracy (1).

In December 2015, the Houston Forensic Science Center (HFSC) implemented a blind quality control (blind QC) program in firearms examination. The intent of the blind QC program is to supplement open proficiency tests by creating mock cases that are submitted to the Firearms section as routine casework, with the expectation that the examiners are unaware that they are receiving test cases. Blind QC cases are created by Firearms section management and submitted by members of the Quality Division. Firearms section management evaluates the mock evidence prior to submission to determine the ground truth and expected results. Once the case is complete, the Firearms section management reviews the results to determine satisfactory completion.

This presentation will outline the results of 51 blind QC cases that were reported between December 2015 and June 2021 resulting in 570 sufficiency and comparison conclusions. No eliminations were reported for true matching pairs, and no identifications were reported for true non-matching pairs; thus, all results were considered satisfactory. In 40.3% (n=225) of the comparisons, the ground truth result was either elimination or identification, but an inconclusive conclusion was reached. Variables such as the examiners assigned to the case, training program, examiner experience level, intended complexity of the case, and evidence type were examined for their contributions to the inconclusive determinations. They type of evidence compared was determined to be the main factor contributing to inconclusive results; specifically, bullets were the main contributors to inconclusive results at a rate of 61.8% (n=168) compared to cartridge cases at a rate of 21.5% (n=57).

The webinar will conclude with a discussion on the rate of inconclusive conclusions, benefits and challenges of the blind QC program, and future directions for blind testing research in firearms examination. While a blind testing program can be challenging to implement, HFSC’s blind QC program demonstrates that the quality management system and procedures used by the Firearms section can obtain accurate and reliable results and provides examiners added confidence in court. Additionally, the blind QC program can be tailored to target specific research questions and provide opportunities for collaboration with other laboratories and researchers.

1. Committee on Identifying the Needs of the Forensic Sciences Community, National Research Council. Strengthening forensic science in the United States: A path forward. Washington, DC, 2009.

 

The webinars are free and open to the public, but researchers, collaborators and members of the broader forensics and statistics communities are encouraged to attend. Each 60-minute webinar will allow for discussion and questions.

Sign up on the form below (Chrome & Safari web browsers work the best):

 

Statistical Thinking for Forensic Practitioners Short Course – Session 3

This event is scheduled to take place on June, 17, 2022. A registration form can be found below. 

Session 3 Description:

Session 3: Quantitative Tools for Forensic Evidence is the third session in the three-session short course, Statistical Thinking for Forensic Practitioners. 

The National Academies report in 2009 and the PCAST report in 2016 encouraged consideration of quantitative approaches to assessing forensic evidence. In this session, we review two of the most often-suggested quantitative approaches, the two-stage approach and the likelihood ratio (Bayes factor) approach. Specific topics include:

  • The two-stage approach to assessing forensic evidence
    • The role of statistical tests in assessing the similarity of two samples
    • Approaches for assessing the relevance of observed similarities
  • Introduction to the likelihood ratio approach
    • Definition and interpretation of the likelihood ratio/Bayes factor
    • Possible applications to different types of evidence (DNA, trace, pattern)
    • Score-based likelihood ratios
    • Sensitivity of the likelihood ratio to modeling choices
  • Results of studies comparing different ways of expressing source conclusions

Presenter:
Hal Stern
Co-Director of CSAFE
Provost, Executive Vice Chancellor, and Chancellor’s Professor – University of California, Irvine

About the Short Course:
Session 3 is the third session in the three-session short course, Statistical Thinking for Forensic Practitioners. Dr. Hal Stern introduces fundamental concepts from probability and statistics –– motivated by forensic issues –– followed by a detailed investigation of how they apply to assess forensic evidence’s probative value. This short course will be held online in three sessions. Each session builds upon the previous one(s) and recordings will be available in the event registrants are unable to attend one or more of the live sessions. Researchers, collaborators, and members of the broader forensics and statistics communities are encouraged to attend. Short course registrants who attend all sessions will receive a certificate of completion.

Registration:

Statistical Thinking for Forensic Practitioners Short Course – Session 2

This event is scheduled to take place on June, 10, 2022. A registration form can be found below. 

Session 2 Description:

Session 2: Sampling, Statistics and the Status Quo is the second session in the three-session short course, Statistical Thinking for Forensic Practitioners. 

Statistical ideas such as sampling, point and interval estimation of population quantities, and hypothesis testing have direct application in forensic science. In this session, we start from fundamental ideas about variability (and its sources) in measurements used in forensic analysis, and discuss methods to ameliorate, quantify, model, and interpret variation and uncertainty in the evaluation of forensic evidence. In more detail, we will:

  • Talk about variability and its sources, and introduce ideas such as reproducibility, repeatability, and accuracy.
  • Briefly talk about populations and samples, and describe some sampling methods useful in forensic applications.
  • Introduce the idea of estimation of population quantities such as means and proportions and of methods to report the uncertainty attached to those estimates.
  • Describe how to carry out a test of hypothesis to compare two means and a test of equivalence to compare two means.
  • Demonstrate how statistical concepts can be used in the study of current forensic science practice (e.g., design and analysis of black box studies)

Presenter:
Hal Stern
Co-Director of CSAFE
Provost, Executive Vice Chancellor, and Chancellor’s Professor – University of California, Irvine

About the Short Course:

Session 2 is the second session in the three-session short course, Statistical Thinking for Forensic Practitioners. Dr. Hal Stern introduces fundamental concepts from probability and statistics –– motivated by forensic issues –– followed by a detailed investigation of how they apply to assess forensic evidence’s probative value. This short course will be held online in three sessions. Each session builds upon the previous one(s) and recordings will be available in the event registrants are unable to attend one or more of the live sessions. Researchers, collaborators, and members of the broader forensics and statistics communities are encouraged to attend. Short course registrants who attend all sessions will receive a certificate of completion.

 

Registration:

Statistical Thinking for Forensic Practitioners Short Course – Session 1

This event is scheduled to take place on June, 3, 2022. A registration form can be found below. 

Session 1 Description:

Probability Concepts and their Relevance to Forensic Science is the first session in the three-session short course, Statistical Thinking for Forensic Practitioners. 

Probability is the mathematical language of uncertainty. Probabilities are used to describe the frequency or likelihood of events or to characterize measurement uncertainty. In this first session, we introduce the laws of probability and their application in forensic settings. Specific topics include:

  • Definition and interpretation of probability
  • Basic laws of probability
  • Conditional probability and independence of events
  • Bayes’ Theorem and Bayesian statistics

Topics are illustrated with examples drawn from forensic science and relevant legal cases.

Presenter:

Hal Stern
Co-Director of CSAFE
Provost, Executive Vice Chancellor, and Chancellor’s Professor – University of California, Irvine

About the Short Course:

Session 1 is the first session in the three-session short course, Statistical Thinking for Forensic Practitioners. Dr. Hal Stern introduces fundamental concepts from probability and statistics –– motivated by forensic issues –– followed by a detailed investigation of how they apply to assess forensic evidence’s probative value. This short course will be held online in three sessions. Each session builds upon the previous one(s) and recordings will be available in the event registrants are unable to attend one or more of the live sessions. Researchers, collaborators, and members of the broader forensics and statistics communities are encouraged to attend. Short course registrants who attend all sessions will receive a certificate of completion.

Registration:

AAFS 2022 Recap: Understanding Juror Comprehension of Forensic Testimony: Assessing Jurors’ Decision Making and Evidence Evaluation

Empty Courtroom

By Samantha Springer, a research assistant at the Center for Statistics and Applications in Forensic Evidence (CSAFE)

 

During the 74th annual American Academy of Forensic Sciences (AAFS) scientific conference, Cassidy Koolmees, a graduate student in legal psychology at Florida International University (FIU), and her colleagues presented a new study entitled Understanding Juror Comprehension of Forensic Testimony: Assessing Jurors’ Decision Making and Evidence Evaluation. This study came after assessing the findings in multiple studies conducted by CSAFE co-director Brandon Garrett et al., who found that a jury’s analysis of forensic testimony was not dependent on the strength of the language used during the testimony. Besides a slight decrease in conviction rates when inconclusive language was used, guilty verdicts remained stable across all conditions that used language indicating a match. This result suggests that the language used to express a match in forensic testimony has little impact on a jury, regardless of the strength of the language or credibility the expert claims.

Building on these findings, Koolmees and her colleagues examined whether jurors could distinguish low-quality testimony from high-quality testimony of forensic experts, using the language guidelines released by the Department of Justice (DOJ) in 2018 as an indicator of quality.

Study participants were put into one of six language-related conditions, where the number of violations of the DOJ language guidelines ranged from zero to five. Participants listened to a full mock trial that included the presentation of forensic evidence. Afterward, they were asked their verdict and how they would rate aspects of the testimony, including confidence in the verdict, clarity of forensic testimony, credibility of the forensic expert, and strength, quality, and usefulness of the forensic evidence.

Most of the dependent variables were found to have no statistical significance between conditions; confidence in the verdict, credibility of the expert, as well as the strength, usefulness, and clarity of the testimony were all consistent across groups.

The only statistically significant difference found between conditions was in the judgment of quality. Only when comparing the conditions of zero guideline violations and four and five violations were there any changes in guilty verdicts, signifying jurors may notice a change in the quality of forensic testimony only when the quality is severely low.

Overall, the study found that, similarly to previous studies, mock jurors are not sensitive to the quality of forensic evidence or to the differences in language used by the experts presenting said evidence. Further research by the FIU group currently being finalized includes versions of the study where jurors are made aware of the DOJ language guidelines before they are presented with expert testimony.

The researchers of the study share CSAFE’s desire for continued education for those involved in criminal trials. Suggestions put forth include simplified jury instructions and a video presentation of instructions. These proposed reforms align with the CSAFE goal of increasing education in forensic evidence for jurors, attorneys, judges, and other relevant parties.

CSAFE supports and oversees substantial contributions to training and education for a wide range of forensic science stakeholders. Explore CSAFE’s available learning opportunities and current training and education research projects at https://forensicstats.org/training-and-education/.

 

Publications referenced by Koolmees in her study:

How Jurors Evaluate Fingerprint Evidence: The Relative Importance of Match Language, Method Information, and Error Acknowledgment
Brandon Garrett and Gregory Mitchell

Mock Jurors’ Evaluation of Firearm Examiner Testimony
Brandon Garrett, Nicholas Scurich and William Crozier

Webinar: Extracting Case-Specific Information from Validation Studies

CSAFE invites researchers, collaborators, and members of the broader forensics and statistics communities to participate in our Spring 2022 Webinar Series on Tuesday, May 10, 2022, from 11:00am-Noon CST. The presentation will be “Extracting Case-Specific Information from Validation Studies.” 

Presenters:
Steve Lund
Mathematical Statistician – National Institute of Standards and Technology
Hari Iyer
Mathematical Statistician – National Institute of Standards and Technology

Presentation Description:
Forensic disciplines often summarize validation studies using average error rates. However, almost every forensic discipline has factors that affect the difficulty of a given case (e.g., quantity and quality of a questioned impression or sample), and average performance metrics fail to reflect the difficulty of the current case. This talk presents an approach to characterize the information a set of validation data provides about method performance in a given case.

The webinars are free and open to the public, but researchers, collaborators and members of the broader forensics and statistics communities are encouraged to attend. Each 60-minute webinar will allow for discussion and questions.

Sign up on the form below (Chrome & Safari web browsers work the best):