Statistical modeling and analysis of trace element concentrations in forensic glass evidence

The question of the validity of procedures used to analyze forensic evidence was raised many years ago by Stephen Fienberg, most notably when he chaired the National Academy of Sciences’ Committee that issued the report The Polygraph and Lie Detection [National Research Council (2003) The National Academies Press]; his role in championing this cause and drawing other statisticians to these issues continued throughout his life. We investigate the validity of three standards related to different test methods for forensic comparison of glass (micro X” role=”presentation” style=”margin: 0px; padding: 0px; border: 0px; font-variant-numeric: inherit; font-variant-east-asian: inherit; font-stretch: inherit; font-size: 14px; line-height: normal; font-family: “Helvetica Neue”, Helvetica, Arial, sans-serif; vertical-align: baseline; display: inline; overflow-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; color: rgb(34, 34, 34); position: relative;”>XX-ray fluorescence (μ” role=”presentation” style=”margin: 0px; padding: 0px; border: 0px; font-variant-numeric: inherit; font-variant-east-asian: inherit; font-stretch: inherit; font-size: 14px; line-height: normal; font-family: “Helvetica Neue”, Helvetica, Arial, sans-serif; vertical-align: baseline; display: inline; overflow-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; color: rgb(34, 34, 34); position: relative;”>μμ-XRF) spectrometry, ICP-MS, LA-ICP-MS], all of which include a series of recommended calculations from which “it may be concluded that [the samples] did not originate from the same source.” Using publicly available data and data from other sources, we develop statistical models based on estimates of means and covariance matrices of the measured trace element concentrations recommended in these standards, leading to population-based estimates of error rates for the comparison procedures stated in the standards. Our results therefore do not depend on internal comparisons between pairs of glass samples, the representativeness of which cannot be guaranteed: our results apply to any collection of glass samples that have been or can be measured via these technologies. They suggest potentially higher false positive rates than have been reported, and we propose alternative methods that will ensure lower error rates.