Skip to content

Combining reproducibility and repeatability studies with applications in forensic science

Journal: Law, Probability and Risk
Published: 2023
Primary Author: Hina Arora
Secondary Authors: Naomi Kaplan-Damary, Hal Stern
Type: Publication
Research Area: Forensic Statistics
View Full Article

Studying the repeatability and reproducibility of decisions made during forensic examinations is important in order to better understand variation in decisions and establish confidence in procedures. For disciplines that rely on comparisons made by trained examiners such as for latent prints, handwriting, and cartridge cases, it has been recommended that ‘black-box’ studies be used to estimate the reliability and validity of decisions. In a typical black-box study, examiners are asked to judge samples of evidence as they would in practice, and their decisions are recorded; the ground truth about samples is known by the study designers. The design for such studies includes repeated assessments on forensic samples by different examiners and additionally, it is common for a subset of examiners to provide repeated assessments on the same evidence samples. We demonstrate a statistical approach to analyse the data collected across these repeated trials that offers the following advantages: i) we can make joint inference about repeatability and reproducibility while utilizing both the intra-examiner and inter-examiner data, ii) we can account for examiner–sample interactions that may impact the decision-making process. We demonstrate the approach first for continuous outcomes such as where decisions are made on an ordinal scale with many categories. The approach is next applied to binary decisions and results are presented on the data from two black-box studies.

Related Resources

Forensic Footwear: A Retrospective of the Development of the MANTIS Shoe Scanning System

Forensic Footwear: A Retrospective of the Development of the MANTIS Shoe Scanning System

There currently are no shoe-scanning devices developed in the United States that can operate in a real-world, variable-weather environment in real-time. Forensics-focused groups, including the NIJ, expressed the need for…
Read More
A Quantitative Approach for Forensic Footwear Quality Assessment using Machine and Deep Learning

A Quantitative Approach for Forensic Footwear Quality Assessment using Machine and Deep Learning

Forensic footwear impressions play a crucial role in criminal investigations, assisting in possible suspect identification. The quality of an impression collected from a crime scene directly impacts the forensic information…
Read More
Enhancing forensic shoeprint analysis: Application of the Shoe-MS algorithm to challenging evidence

Enhancing forensic shoeprint analysis: Application of the Shoe-MS algorithm to challenging evidence

Quantitative assessment of pattern evidence is a challenging task, particularly in the context of forensic investigations where the accurate identification of sources and classification of items in evidence are critical.…
Read More
Computational Shoeprint Analysis for Forensic Science

Computational Shoeprint Analysis for Forensic Science

Shoeprints are a common type of evidence found at crime scenes and are regularly used in forensic investigations. However, their utility is limited by the lack of reference footwear databases…
Read More
PaulHermsen
Paul Hermsen
Laboratory Administrator

Iowa Department of Criminal Investigation Crime Laboratory

Paul Hermsen is the Crime Laboratory Administrator for the Iowa DCI Crime Laboratory located in Ankeny, IA.  He has served in various capacities at the crime laboratory since 1998 including performing casework in drug chemistry, providing scene response for both crime scene and clandestine laboratories, and providing supervision for multiple areas of the laboratory including toxicology, breath alcohol, and evidence functions. Additionally, he served as the Quality Assurance Manager from 2006-2022, leading the laboratory through various iterations of accreditation programs.

Mr. Hermsen is a past-president of the Midwestern Association of Forensic Scientists, Inc. (MAFS) and continues to serve on several committees, having also served as a meeting chair and local host for two annual MAFS meetings. Additionally, he is a technical assessor for ANAB in the areas of seized drugs and scene investigation.

Paul holds a B.S. in Chemistry from the University of Northern Iowa and a Certified Public Manager certificate from Drake University.

MM_190304_1848
Barbara Hervey
Retired, Judge

Court of Criminal Appeals, Texas

Judge Barbara Parker Hervey was elected to the Court of Criminal Appeals in 2000. and served until 2024. Prior to her election, she was an Assistant District Attorney in the Bexar County District Attorney’s Office for 16 years. She is from New Jersey and earned her B .A. from The University of North Carolina at Greensboro and her J.D. from St. Mary’s University School of Law.

Judge Hervey has been an author and speaker for over 450 lectures and legal seminars. She is currently co-chair of the Texas Judicial Commission on Mental Health and is an Adviser for the American Law Institute’s Committee to Reform the Model Penal Code: Sexual Assault and Related Offenses. She is also the chair of the Court’s Grant Committee, which manages $13 million per year to train judges, attorneys, and their staff, and she founded of the Texas Criminal Justice Integrity Unit. Formerly, she served as a Commissioner on the National Commission of Forensic Science and was a member of the Tim Cole Advisory Panel.

Judge Hervey and her husband Richard Langlois reside in San Antonio. They have three children, Edward, Christopher, and Melissa.

Sallie_Keller_blue_a_headshot2019_web
Dr. Sallie Ann Keller
Director

Distinguished Professor in Biocomplexity, Director of the Social and Decision Analytics Division within the Biocomplexity Institute and Initiative at University of Virginia and Professor of Public Health Sciences.

Dr. Keller’s areas of expertise are social and decision informatics, statistical underpinnings of data science, and data access and confidentiality. Dr. Keller’s is a leading voice in creating the science of all data and advancing this research across disciplines to benefit society.Her prior positions include Professor of Statistics and Director of the Social and Decision Analytics Laboratory within the Biocomplexity Institute of Virginia Tech; Academic Vice-President and Provost at University of Waterloo; Director of the Institute for Defense Analyses Science and Technology Policy Institute; the William and Stephanie Sick Dean of Engineering at Rice University; Head of the Statistical Sciences group at Los Alamos National Laboratory; Professor of Statistics at Kansas State University; and Statistics Program Director at the National Science Foundation. Dr. Keller is an Elected Member of the National Academy of Engineering. She has served as a member of the National Academy of Sciences Board on Mathematical Sciences and Their Applications, the Committee on National Statistics, and has chaired the Committee on Applied and Theoretical Statistics. She is fellow of the American Association for the Advancement of Science, elected member of the International Statistics Institute, fellow and past president of the American Statistical Association.
PamKing
Pam King
Judge

Minnesota Third Judicial District Court

Minnesota 3rd Judicial District Court Judge Pam King is chambered in Olmsted County which is in southeastern Minnesota. She presides over civil, criminal, family, juvenile, and probate matters. She was appointed in October 2015, elected in October 2016, and re-elected in 2022. Prior to being appointed to the bench, Judge King worked as an attorney for the Minnesota State Public Defender.  In this role, she handled cases involving complex litigation and/or forensic science. She consulted with public defenders and private defense lawyers on a variety of issues including forensic DNA, forensic pathology, toxicology, and drug chemistry. She was a participant and then facilitated the Minnesota State Public Defender DNA Institute. In 2018 Judge King was awarded the Magnus Mukoro Award for Integrity in Forensic Science by New York City Legal Aid Society. Judge King served as a Commissioner on the National Commission on Forensic Sciences. She is an AAFS fellow, is currently on the Board of Directors and Executive Committee. She is also on the American Academy of Forensic Sciences Academy Standards Board. She is a member of the Minnesota Association of District Court Judges.

JuliaLeighton
Julia Leighton
Retired

Former General Counsel, Public Defender Service for District of Columbia

Julia Leighton is the former general counsel for the Public Defender Service for the District of Columbia (PDS).  As general counsel, Ms. Leighton advised and represented the PDS’s Board of Trustees, the PDS management team, and PDS lawyers on a wide variety of legal matters. In addition to her duties as general counsel, Ms. Leighton was a member of PDS’s Forensic Practice Group and was a 2001 founding member.  In 2014 Ms. Leighton was appointed by the U.S. Attorney General to the National Commission on Forensic Science and was a voting member until the Commission was sunset.  Ms. Leighton was also a member of three subcommittees of the Commission, Accreditation and Proficiency Testing, Interim Solutions, and Reporting and Testimony.  Currently she is a member of the Legal Task Group within NIST’s forensic science standards development entity the Organization of Scientific Area Committees (OSAC).  She is a frequent case consultant and lecturer on challenging forensic evidence and the use of experts in litigation. Prior to becoming PDS’s general counsel, Ms. Leighton spent eleven years litigating criminal cases; eight years as a staff attorney at PDS, and three years as a trial attorney in the Environmental Crimes Section of the U.S. Department of Justice.  Ms. Leighton received her B.A. in Economics from Bowdoin College, Magna Cum Laude, and her J.D. from the Georgetown University Law Center, Cum Laude.

 

Henrymaynardupdatedheadshot2025closeup
Henry Maynard
Chief Operating Officer

Alakai Defense Systems

Henry Maynard is the Chair of the ASCLD Forensic Research Committee and he currently serves as the Chief Operating Officer of Alakai Defense Systems. Prior to working at the lab, Mr. Maynard was a federal contractor that supported Research and Development (R&D) and Forensic Science Training efforts for the Office of Investigative and Forensic Sciences (OIFS) within the National Institute of Justice (NIJ). Before that he was a forensic practitioner at a private laboratory. He has been formally trained in the areas of Forensic Toxicology, Drug Chemistry, and Explosives Analysis. Mr. Maynard is active in the forensic community and maintains memberships with the American Academy of Forensic Sciences (AAFS), the American Chemical Society (ACS), the American Society of Crime Lab Directors (ASCLD), the Council of Forensic Science Educators (COFSE), the International Association of Bomb Technicians and Investigators (IABTI), and Project Management Institute (PMI). Additionally, he serves on the Strategic Advisory Board for the Center for Statistics and Applications in Forensic Evidence (CSAFE) and is the Chair of the Federal Laboratory Needs Technical Working Group (FLN-TWG) Research Subcommittee. He is very active with ASCLD, as he serves as a board member on the ASCLD Board of Directors, Co-Chair of the Forensic Research Committee, creator of the Laboratories and Educators Alliance Program (LEAP), instructor for the ASCLD Leadership Academy Level II, and serves on the Criminal Justice Information Services (CJIS) Advisory Policy Board. Mr. Maynard holds a Bachelor of Science in Biochemistry and a Master of Science in Forensic Science.

Photo - RSR
Mr. Robert Ramotowski
Program Manager for Forensic Science

Special Programs Office, National Institute of Standards and Technology

Robert Ramotowski was awarded Bachelor of Science and Master of Science degrees in Chemistry from the George Washington University in 1993 and 1997, respectively.  He is currently the Program Manager for Forensic Science, Special Programs Office, National Institute of Standards and Technology, Gaithersburg, MD.  He currently oversees a research portfolio that includes projects in firearms and toolmarks, forensic genetics, digital and identification evidence, trace, drugs and toxins, statistics, and biometrics.  He also oversees NIST’s Forensic Science Center of Excellence – the Center for Statistics and Applications in Forensic Evidence (CSAFE).  He has published more than thirty articles on latent print and document chemistry and given more than 100 lectures and workshops in the United States as well as in more than a dozen countries.  Mr. Ramotowski is both author and editor of the 3rd edition of Lee and Gaensslen’s Advances in Fingerprint Technology.  He is a member of the International Association for Identification, Chesapeake Bay Division of the IAI, ASTM International E30, and the American Chemical Society.

Barry-Scheck-Headshot_web
Mr. Barry Scheck
Co-Founder and Special Counsel

Innocence Project

Barry C. Scheck, is a Professor of Law at the Benjamin N. Cardozo School of Law in New York City. In his forty-two years on the Cardozo faculty, he served as the Director of Clinical Education, Co-Director of the Trial Advocacy Programs,and the Jacob Burns Center for the Study of Law and Ethics.He worked for three-years as a staff attorney at The Legal Aid Society in New York City before joining the faculty at Cardozo.

Barry C. Scheck and his colleague Peter Neufeld, Co-Founded and are Special Counsel at the Innocence Project, an independent nonprofit organization closely affiliated with Cardozo Law School, which uses DNA evidence to exonerate the wrongly convicted. The Project also assists police, prosecutors, and defense attorneys in trying to bring about reform in many areas of the criminal justice system, including eyewitness identification procedures, interrogation methods, crime laboratory administration, and forensic science research. In its twenty-eight years of existence, 367 individuals have been exonerated in the United States through post-conviction DNA testing.You can read about each of these cases at www.innocenceproject.org.

Barry C. Scheck is also a partner in the law firmNeufeld, Scheck & Brustin, LLP (formerly Cochran Neufeld & Scheck, LLP), specializing in civil rights and constitutional litigation. The firm is frequently retained by victims of police brutality, pursuing civil rights claims in the courts and institutional reform.

Barry C. Scheck has done extensive trial and appellate litigation in significant civil rights and criminal defense cases. He has published extensively in these areas, including a book with Jim Dwyer and Peter Neufeld entitled, Actual Innocence: When Justice Goes Wrong And How To Make It Right. He has served in prominent positions in many bar associations, including the presidency of the National Association of Criminal Defense Lawyers (2004-2005). He is a former Commissioner (1994-2016) on New York State’s Forensic Science Review Board, a body that regulates all crime and forensic DNA laboratories in the state. He is currently a member of the Legal Resource Committee of the Organization of Scientific Area Committees(OSAC) run by the National Institute of Standards and Technology. From 1998 -2000, he served on the National Institute of Justice’s Commission on the Future of DNA Evidence. In2005 he was a member of the American Judicature Society’s National Commission on Forensic Science and Public Policy.

In 1971, he received his undergraduate degree from Yale University and in 1974 his law degree from Boalt Hall School of Law, University of California at Berkeley.

Peter-Stout-headshot_sqaure
Dr. Peter Stout
CEO and President

Houston Forensic Science Center

Dr. Peter Stout, HFSC’s CEO and president, initially joined the agency in 2015 as its chief operating officer and vice president. He has more than 15 years of experience in forensic science and forensic toxicology. Prior to joining HFSC, Dr. Stout worked as a senior research forensic scientist and director of operations in the Center for Forensic Sciences at RTI International. Dr. Stout also has served as president of the Society of Forensic Toxicologists (SOFT). He represented SOFT in the Consortium of Forensic Science Organizations and has participated in national policy debates on the future of forensic sciences in the United States. Dr. Stout has a doctorate in toxicology from the University of Colorado Health Sciences Center in Denver. Dr. Stout also served as an officer in the U.S. Navy Medical Service Corps.

Jeremy Triplett2023
Jeremy Triplett
Director

Kentucky State Police Central Forensic Laboratory

Jeremy Triplett is the Director of the Kentucky State Police Central Forensic Laboratory in Frankfort, Kentucky, where he oversees the day-to-day operations of the full-service, ISO 17025 accredited laboratory.  Jeremy has more than eighteen years of experience in forensic science with a background in forensic drug chemistry analysis and has testified in local, state, and federal courts.  As former seized drugs technical leader, Jeremy oversaw training programs, policy and procedure revisions, and internal audits for the six KSP seized drug laboratories, statewide, and led the transition of the KSP Seized Drugs section from ASCLD-LAB Legacy accreditation to the ISO 17025 International standard.  Jeremy regularly interfaces with policymakers in Kentucky regarding forensic science, criminal justice, and controlled substances issues facing the Commonwealth.

Jeremy is a Past-President of the American Society of Crime Laboratory Directors (ASCLD), an organization of crime laboratory directors and managers dedicated to providing excellence in forensic science through leadership and innovation.  From 2014-2017, Jeremy served as chairman of the Forensic Science Standards Board of the NIST Organization of Scientific Area Committees.  Jeremy is a member of the American Academy of Forensic Sciences and serves as a technical assessor in seized drugs for ANAB, where he has participated in several assessments of forensic science laboratories inside and outside of the United States.  Jeremy received a B.S. in chemistry from the University of Kentucky and an M.S. in pharmacy from the University of Florida.

Valerio_R_Headshot_2
Raymond Valerio
Director of Forensic Sciences

Queens County District Attorney’s Office, New York

Raymond Valerio has been an Assistant District Attorney in New York City since 2004.  Currently, he is the Director of Forensic Sciences at the Queens County District Attorney’s Office, overseeing all forensic science-based prosecutions handled by the Office.  In this capacity, Mr. Valerio collaborates with the Cold Case Unit, Conviction Integrity Unit, and acts as a liaison to all forensic testing laboratories.   

In 2018, Mr. Valerio received the Thomas E. Dewey Medal from the New York City Bar Association for his accomplishments in forensic science as a prosecutor. In 2020, WIRE Interdisciplinary Journal, a peer-reviewed scientific journal, published Mr. Valerio’s article titled “Likelihood Ratios For Lawyers…I Didn’t Go to Law School for This.”

Mr. Valerio is a member of the Firearm Toolmark and Friction Ridge American Standards Consensus Bodies of the American Academy of Forensic Sciences (AAFS), as well as the National District Attorneys Association (NDAA) Forensic Science Working Group.  Mr. Valerio has participated on various Organization of Scientific Area Committees (OSAC) Scientific Technical Review Panels. He frequently lectures and consults with prosecutors across the country on issues related to forensic evidence.

Mr. Valerio received his B.A from the University of Pennsylvania in 2001, and his J.D. from Temple University School of Law in 2004. 

JodyWolf
Jody Wolf
Chief of Staff

Phoenix Police Department

Jody Wolf is the Chief of Staff for the Phoenix Police Department. 

Over her career, she has worked in both public and private laboratories and was most recently employed by the Phoenix Police Department Crime Laboratory for 22 years. She is an active member of several professional organizations and has been actively involved in the criminal justice community.  She currently serves as the Chair of the Arizona Forensic Science Academy Board and the Chair of the National Forensic Science Academy. She also serves as the Secretariat to the United States Technical Advisory Group to ISO Technical Committee 272 on Forensic Science. She has held many leadership positions within the forensic community including the President of the International Forensic Strategic Alliance (IFSA) and the President for the American Society of Crime Laboratory Directors (ASCLD).

Jody received her Bachelors of Science degrees in Biology and Chemistry from Regis University in Denver, Colorado, and her Masters of Science degree in Chemistry from Arizona State University. She also received her Masters in Business Administration degree with an emphasis in Technology Management from the University of Phoenix.

ArthurAndrade
Arthur Andrade
Criminalist

Miami-Dade Police Department, Forensic Services Division

Arthur Andrade is a Criminalist in the Miami-Dade Police Department, Forensic Services Division. He works in the Firearm and Toolmark Unit of the Forensic Identification Section, where he has been a firearm examiner for over seven years. Mr. Andrade received a Bachelor of Science in Forensic and Investigative Science from West Virginia University in 2015 and is currently a graduate student at Florida International University, pursuing a Professional Science Master’s in Forensic Science. He is an active and certified member of the Association of Firearm and Toolmark Examiners (AFTE), having achieved certification in Firearm Evidence Examination and Identification. Throughout his time with the Miami-Dade Police Department, Mr. Andrade has gained valuable experience in casework and testimony, including a Daubert trial for firearms identification in federal court. He has conducted research related to chamber marks found in firearms chambered for 7.62x39mm, established a firearms identification digital media library, and provided instruction as a primary trainer for new examiners. Mr. Andrade has given multiple presentations for various professional organizations, including the International Association for Identification (IAI), AFTE, and different police departments. Mr. Andrade has served as the Forensic Identification Section’s technical manual and training manual liaison and currently serves as a member of the machining and subclass subcommittee of AFTE.

JoAnnBuscaglia2
JoAnn Buscaglia
Research Chemist

Department of Justice

JoAnn Buscaglia began her career in biometrics through forensic science research and development. In the early 2000s, she served as a research chemist in the FBI’s Laboratory Counterterrorism and Forensic Science Research Unit (CFSRU). While there, she assisted with latent fingerprint research to develop quantitative measurements and assess the scientific basis of identification conclusions in order to address court admissibility challenges to the discipline.

In 2002, Buscaglia began working with automated fingerprint identification system (AFIS) algorithms as a tool for the quantification of fingerprint features. She also wanted to explore the possibility of using image-based and other non-minutia-based algorithms for latent print searching and matching in AFIS. In 2004-2005, she served on an internal FBI Laboratory panel to assess the research needs of the latent print discipline, particularly related to the scientific basis of conclusions.

Buscaglia completed her Ph.D. in 1999, with a dissertation also in the area of trace elemental analysis of materials. Prior to joining the FBI Laboratory, she worked for almost a decade in academia and as a consultant for both private- and public-sector forensic, environmental, and industrial hygiene laboratories. She now has more than 18 years with the FBI Laboratory’s CFSRU and nearly 14 years in biometrics research. Much of her biometric research has been supported through collaboration with the FBI Criminal Justice Information Systems Division Biometrics Center of Excellence.

Buscaglia says her proudest accomplishment in biometrics is leading the FBI Laboratory “Black Box” latent print examiner decision analysis study, which quantified the accuracy, repeatability, and reproducibility of LPE decisions. Black Box was the first large-scale study to estimate error rates for latent print examiners.

SarahChu2
Sarah Chu
Director of Policy and Reform

Perlmutter Center for Legal Justice at Cardozo Law

Sarah Chu, Ph.D., is the Director of Policy and Reform at the Perlmutter Center for Legal Justice at Cardozo Law. As the director of policy and reform, Sarah leads the Perlmutter Center’s forensic science policy initiatives. She served on the Scientific Inquiry and Research Subcommittee of the National Commission on Forensic Science and was the 2021 recipient of the Legal Aid Society’s Magnus Mukoro Award for Integrity in Forensic Science. Prior to joining the Perlmutter Center, Sarah led the Innocence Project’s policy portfolio on forensic science, forensic medicine, and police investigative technologies for 15 years.

A graduate of the University of California, San Diego, Sarah holds bachelor degrees in Biochemistry/Cell Biology, Communication, and a Masters in Biology. She also holds a Masters in Epidemiology from Stanford University and completed her doctorate in Criminal Justice at John Jay College of Criminal Justice/CUNY Graduate Center. Sarah is a member of the 2023-2024 class of the Harvard Medical School Center for Bioethics Fellowship Program.

DianeDo2
Diane Do
Retired

San Diego County Sheriff's Department Regional Crime Laboratory

Diane N. Do retired as a Senior Latent Print Examiner at the San Diego County Sheriff’s Department Regional Crime Laboratory, where she served as the technical leader in the latent print section. Ms. Do has more than 25 years of experience in print examination. In 1999, she created a highly competitive forensic internship program for eligible college students studying forensics, allowing students to get hands-on experience. Ms. Do has been certified by the IAI as a Certified Latent Print Examiner since 2002. She is a Life Member of both the International Association for Identification (IAI) and the Southern California Association of Fingerprint Officers (SCAFO). Ms. Do has been recognized for her excellence in teaching as an Adjunct Professor at Grossmont Community College, where she has been teaching introductory fingerprinting and latent print analysis classes for more than 19 years. Ms. Do has instructed workshops in print examination at the IAI Educational Conference, California State Division IAI Educational Training Seminars, and Southern California Association of Fingerprint Officers Annual Training Conference. Ms. Do is a Certified Technical Assessor for the ANSI National Accreditation Board (ANAB).

da-garza
José Garza
District Attorney

Travis County, Texas

José Garza was elected Travis County District Attorney on November 3, 2020.

As a former federal public defender, immigrant rights activist, and leader of the systemic change organization, Workers Defense Project, José Garza has a unique view into how our broken criminal justice system works and how it impacts our communities. He believes we can fix it together.

José attended law school at Catholic University in Washington, D.C., and worked for Judge Richard W. Roberts in federal district court for the District of Columbia. He returned to Texas to work on the border as an assistant public defender in the first multi-county public defender’s office in Texas at Texas Rio Grande Legal Aid. Subsequently, José served as an assistant federal public defender in the Western District of Texas, where he represented people accused of misdemeanor and felony crimes. In those experiences, he saw firsthand how our system weighs most heavily on people of color, working people and poor people, and immigrant families.

José later returned to Washington, D.C., as Deputy General Counsel for the House Committee on Education and Labor. He went on to work as Special Counsel to the National Labor Relations Board where he represented the agency against legal attacks brought by a Republican Congress. He eventually served Secretary Tom Perez as a senior policy official at the U.S. Department of Labor, where he worked to ensure that working people and people of color were able to find safety and stability in good jobs. 

José brings to the office of the District Attorney extensive experience building power with communities of color, working families, and immigrant communities across Texas. During his tenure as the Executive Director of Workers Defense Project, the organization won significant criminal justice reform in Travis County, won paid sick leave policies in Austin, San Antonio, and Dallas, and advocated for the creation of a Public Defender Office alongside community allies in Travis County.

WillGuthrie
Will Guthrie
Retired, Statistical Engineering Division

National Institute of Standards and Technology

William F. Guthrie received a B.A. degree in mathematics from Case Western Reserve University, Cleveland, OH, in 1987 and an M.S. degree in statistics from The Ohio State University, Columbus, OH, in 1990. He joined the Statistical Engineering Division at the National Institute of Standards and Technology (NIST), Gaithersburg, MD in 1989.

He has collaborated with NIST scientists and engineers in a wide range of areas, applying statistical methods to solve problems in semiconductor and microelectronic metrology, building materials research, and chemical science. His statistical interests include uncertainty assessment, Bayesian statistics, design of experiments, calibration, modern regression methods, and statistical computation.

Forensic-Conference-2019-(3)_square
David Kanaris
Forensic Laboratory Chief

Alaska State Crime Lab

David Kanaris began his career in forensic science in 2005 with the Forensic Science Service in the UK. After four years working as a forensic examiner in forensic biology and footwear impressions, David was seconded to various police forces around the UK. Training them in footwear impression evidence before becoming a footwear intelligence manager with West Midlands Police.

In 2012 David moved to Alaska with his wife where he started working as a Forensic Scientist for the Alaska Scientific Crime Detection Laboratory. In 2014 David re-introduced footwear impressions to the Alaska criminal justice system and was shortly after appointed as an inaugural member and training and competency task group chair of the OSAC Footwear and Tire Subcommittee. David is currently the Chief of the Alaska Scientific Crime Detection Laboratory and the Chair of the OSAC Footwear and Tire Subcommittee. David is an IAI certified Footwear Examiner, has presented at numerous IAI conferences, and in November 2019 presented at the Singapore Police Force annual Forensic Science Conference.

Paul-photo_web
Paul Kish
Forensic Consultant

Paul Erwin Kish Forensic Consultant and Associates

Paul Kish is a forensic consultant based in Corning, NY. He holds B.S. and M.S. degrees from Elmira College.

He has 30 years of experience as a consulting bloodstain pattern and crime scene reconstruction expert. He has consulted on homicide cases in 47 states and 13 countries while presenting expert testimony in 29 states, the District of Columbia, and Canada.

Mr. Kish is an internationally known lecturer on the subject of bloodstain pattern analysis and crime scene reconstruction lecturing throughout the United States, Canada, England, and The Netherlands. He has instructed at 71 week-long basic bloodstain pattern analysis courses educating over 1000 students from 18 countries. He co-authored Principles of Bloodstain Pattern Analysis – Theory and Practice.

Mr. Kish has been actively involved in standards development within the discipline of bloodstain pattern analysis since 2002 when he was one of the founding members of SWGSTAIN. He currently serves on the OSAC Scientific Area Committee on Physics / Pattern Evidence; and the Academy Standards Board’s Bloodstain Pattern Analysis Consensus Body.

He is a Fellow in the American Academy of Forensic Sciences; a Distinguished Member of the International Association of Bloodstain Pattern Analysts; and a Life Associate Member of the International Association for Identification.

AnthonyKoertner2
Anthony Koertner
Latent Print Examiner

U.S. Army Criminal Investigation Laboratory

Anthony Koertner is a Latent Print Examiner at the U.S. Army Criminal Investigation Laboratory (USACIL) located in Atlanta, Georgia.  Mr. Koertner is an Active Member of the International Association for Identification and is certified in both latent print examination and footwear examination. 

Mr. Koertner graduated from the University of Central Florida in 2006 and began his career in latent print and footwear/tire tread examination at the DFSC in 2007.  In 2016 he was selected to be a Science, Mathematics and Research for Transformation Scholar where he completed his Master of Science in Forensic Science from the University of Florida.  Mr. Koertner contributed to the validation of FRStat which was one of the first statistical models to provide statistical support for reported latent print examination conclusions in the United States. 

Mr. Koertner has conducted several domestic and international lectures and workshops to various stakeholders in the forensic community.   Currently he serves as a member of the Footwear and Tire Subcommittee within the Organization of Scientific Area Committees (OSAC) for Forensic Science. 

D717E617-55BE-428E-B4E7-707ACA6AE0F4
Gerald LaPorte
Director

Research Innovation Global Forensic and Justice Center Florida International University

Mr. LaPorte is the Director of Research Innovation at the Global Forensic and Justice Center (GFJC), a pre-eminent program of Florida International University (FIU). The GFJC coordinates all forensic science and criminal justice efforts at FIU, which builds on four established focus areas – academia, industry, technology and international justice. Mr. LaPorte has been employed in various capacities in the forensic sciences since 1993, including the Director of the Office of Investigative and Forensic Sciences at the National Institute of Justice and Chief Research Forensic Chemist at the United States Secret Service. He was the co-chair of the Standards Practices and Protocols Interagency Working Group, under the Office of Science and Technology Policy within the Executive Office of the President of the United States. Mr. LaPorte was selected by the Attorney General of the United States to serve as a Commissioner on the National Commission on Forensic Science. Currently, he serves as the Chairperson on the Forensic Document Examination Subcommittee on the Organization of Scientific Area Committees (OSAC) for Forensic Science, which works to strengthen the nation’s use of forensic science by facilitating the development of technically sound forensic science standards. He has authored numerous publications, including chapters in three forensic science text books, and has testified as an expert witness in International and U.S. Courts over 100 times.

Licht,-Gary-photo_crop
Gary Licht
Criminalist Supervisor

Iowa Division of Criminal Investigation Crime Laboratory

Gary Licht is a Criminalist Supervisor and Forensic Document Examiner at the Iowa DCI Crime Lab in Ankeny. He began his forensic science career in 1987, in the Microanalysis Section of the Florida Department of Law Enforcement’s Orlando Lab, and moved to Iowa in 1994 and trained in forensic document examination. He is a member of Midwestern Association of Forensic Scientists, American Academy of Forensic Sciences, American Society for Testing and Materials, Scientific Working Group for Forensic Document Examination, and the Organization of Scientific Area Committees under the guidance of the National Institute for Science and Technology. Gary is certified by the American Board of Forensic Document Examiners. His purview as a supervisor covers the sectionsspecializing in; documents, firearms and tool marks, latent prints, footwear and tire impressions, crime scene, and photography.

LintonMohammed2
Linton A. Mohammed
Forensic Document Examiner

Forensic Science Consultants, Inc.

Dr. Linton Mohammed has been in the field of Forensic Document Examination for more than 30 years.  He has testified as an expert witness more than 200 times in the US, England, and the Caribbean. He is the author of “Forensic Examination of Signatures”, and co-author of “The Neuroscience of Handwriting: Applications for Forensic Document Examination”. He has published several papers in peer-reviewed journals.

Dr. Mohammed is certified by the American Board of Forensic Document Examiners, Inc, and holds a Diploma in Document Examination from the Chartered Society of Forensic Sciences (formerly The Forensic Science Society).

Dr. Mohammed has conducted or co-presented workshops on signature and document examination in Australia, Brazil, Canada, Croatia, China, Latvia, Poland, Saudi Arabia, Scotland, Turkey, and the United States.

He was the recipient of the Albert S. Osborn Award of Excellence from the American Society of Questioned Document Examiners (2022), the Ordway Hilton Award from the Questioned Documents Section of the American Academy of Forensic Sciences (2019), and the New Horizon Award from the American Board of Forensic Document Examiners, Inc.(2012).

He is a member and Past-President of the American Society of Questioned Document Examiners, Inc, a Fellow of the Questioned Documents Section of the American Academy of Forensic Sciences, a Fellow of the Chartered Society of Forensic Sciences, and a member of the Canadian Society of Forensic Science. He is the Editor of the Journal of the American Society of Questioned Document Examiners, and is also a guest reviewer for several other journals.  

 

Linton currently serves on the Board of Directors of the American Academy of Forensic Sciences and on the Academy Standards Board.

Dr. Mohammed is in private practice in California, United States.

MNally-headshot_web
Melissa Nally
Firearms Examiner

Houston Forensic Science Center

Melissa began her career in forensics in 2001 with the Illinois State Police as a Forensic Scientist Trainee in Firearms Identification. In 2014 she moved to Houston and joined Houston Forensic Science Center as a Senior Firearms Examiner. Melissa has been a member of AFTE since 2004 and is eager to serve as a member of the CSAFE’s Research and Technology Transfer Advisory Board.

Peter-Neufeld_headshot
Peter Neufeld
Co-founder and special counsel

Innocence Project

Peter Neufeld co-founded and is Special Counsel at the Innocence Project, a national non- profit organization which uses DNA evidence to exonerate the wrongly convicted. The Project also assists legislatures and executive branches as well as police, prosecutors, and defense attorneys to bring about reform in criminal justice. In its twenty-eight years of existence, 367 individuals have been exonerated in the United States through post-conviction DNA testing. The Project has been instrumental in the passage of federal legislation and more than a hundred state statutes to improve the quality of justice. You can read about each of these cases at www.innocenceproject.org.

Peter Neufeld is a founding partner in the New York civil rights law firm of Neufeld Scheck & Brustin, LLP. Mr. Neufeld’s focus is constitutional law and civil rights. He has lectured and taught students, lawyers, judges, legislators and scientists on subjects at the intersection of science and criminal justice. Before co-founding the Innocence Project, Mr. Neufeld taught trial advocacy at Fordham University Law School.

From 2014 – 2017, Mr. Neufeld served as a Commission member of the National Commission on Forensic Science, established by US Department of Justice and the National Institute of Standards and Technology. From 1995 to his resignation in 2016, Mr. Neufeld served on the New York State Commission on Forensic Science which regulates the two dozen crime laboratories in the state. He is a member Emeritus of the boards of the Montefiore Medical Center and the Albert Einstein School of Medicine.

A graduate of the University of Wisconsin, Peter Neufeld received his law degree from New York University School of Law.

_NFI9937-C_web
Marjan Sjerps
Principal Scientist Statistician

Professor by special appointment Netherlands Forensic Institute

Prof. dr. Marjan Sjerps received her master degree in mathematics (specialization in statistics/OR) from the Catholic University of Nijmegen, the Netherlands (currently Radboud University Nijmegen) in 1988. She received her Ph.D in 1994 from Leiden University, the Netherlands, based on a dissertation concerning theoretical biology. In 1993 she started working at the Netherlands Forensic Institute (NFI) as a statistician, where she became involved in the emerging area of forensic statistics. Her activities include forensic statistics casework, teaching, consultation, and research. She acted as an expert witness in court on several occasions. She was a member of the NFI principal scientist team in the period 2011-2020. Currently, she is employed as senior scientist at the NFI, and is a member of a team of forensic statisticians. She was appointed professor by special appointment at the University of Amsterdam in February 2010, which was extended in 2015 and 2020. Her research interests are interpretation, evaluation and reporting of forensic evidence: all aspects of probabilistic reasoning in the forensic process, from gathering evidence up to testifying in court. She published over 70 papers in (semi) scientific journals (national and international) and was editor of a book on statistics in law (in Dutch).

harm-van-beek
Harm van Beek
Senior Digital Forensic Scientist

Netherlands Forensic Institute

Dr. Harm van Beek is an experienced Senior Digital Forensic Scientist at the Netherlands Forensic Institute (NFI). His work consists of performing examinations in criminal cases and conducting scientific research in the digital forensic field. Harm is one of the founders of the forensic investigation, innovation and knowledge sharing platform Hansken. He is the technical director of CASE, an international standard for sharing cyber-investigation traces. Harm obtained his Ph.D. degree in formal methods (computer science) at the Eindhoven University of Technology. Before joining the NFI, he was founder and CTO of ISAAC, a company dedicated to developing middleware and software for the Internet.

Kevin-R-Winer_crop
Kevin Winer
Director

Kansas City Missouri Police Department

Kevin Winer is the Director the Kansas City Police Crime Laboratory and is the Vice President and a forensic science consultant for Van Stratton, Winer & Associates, LLC. He holds a B.S. in Chemistry and a B.S. in Honors Cellular and Molecular Biology from the University of Michigan. He has over 19 years of experience as a bloodstain pattern analyst, 21 years as a trace evidence analyst and four years as a DNA analyst. Mr. Winer has attended advanced workshops including Math and Physics for BPA, Fluid Dynamics of Bloodstain Formation and Fabrics-based BPA Workshops. He has been qualified as a BPA expert witness in six states and has provided BPA instruction in ten states. Mr. Winer is the Research Task Group Chair for the OSAC BPA Subcommittee and is a member of the NIJ Impression, Pattern and Trace Technical Working Group. He is a Diplomate of the American Board of Criminalistics.

GregoryFiumara2
Gregory Fiumara
Computer Scientist

Image Group, NIST

Gregory Fiumara has been a Computer Scientist with the National Institute of Standards and Technology (NIST)’s Image Group since 2008. He leads NIST’s world-recognized fingerprint technology evaluations, including MINEX, PFT, SlapSeg, and ELFT. Gregory is an active software developer, maintaining and contributing to open-source projects like NFIQ 2 and Biometric Evaluation framework, the latter earning him a U.S. Department of Commerce Silver Medal. Gregory established a focus area for biometric forensic science in NIST’s Special Programs Office, helping advance metrology that impacts practitioners, including through datasets like NIST’s Special Database 302. He currently serves as editor of ISO/IEC 29794-4.

guttman-2014_web
Barbara Guttman
Manager of the Software Quality Group

NIST

Barbara Guttman is the Manager of the Software Quality Group in NIST’s Information Technology Lab (ITL). Her areas of responsibility include software assurance and computer forensics. In computer forensics, her group runs the National Software Reference Library www.nsrl.nist.gov and the Computer Forensics Tool Testing Project www.cftt.nist.gov. She is also active in both the Scientific Working Group on Digital Evidence and the OSAC DE Subcommittee. She has been working in the area for 20 years. In software assurance, her group runs the Software Assurance Metrics and Tool Evaluation (SAMATE) project including the Static Analysis Tool Exposition and the SAMATE Reference Data Set. Prior to joining the Software Quality Group, she was Associate Director of ITL, Senior Program Analyst to the NIST Director, and worked in computer security and federal information policy.

Herman,-Martin_crop
Dr. Martin Herman
Senior Advisor

NIST

Dr. Martin Herman is a Senior Advisor in the Information Technology Laboratory at NIST. He has research experience in the areas of information technology, image processing, computer vision, and artificial intelligence. He manages the NIST Forensic Footwear Research project which is focused on research towards quantitative analysis for footwear evidence evaluation. He is co-leader of the NIST Cloud Computing Forensic program. He previously served as a senior manager at NIST for 13 years, where he was responsible for a large program in information access technologies, including biometrics, speech processing and human language technologies, multimedia information access, information retrieval, image and video processing technologies, visualization and usability testing, human-computer interfaces, and smart spaces.He is a member of the OSAC Footwear and Tire Subcommittee and a member of the U.S. National Footwear Database Evaluation Study Expert Working Group.He has a PhD in computer science from the University of Maryland.

Hari Iyer
Hari Iyer
Mathematical Statistician

Statistical Engineering, NIST

Hari Iyer was born in Chennai (then known as Madras), India. After graduating from high school he joined St. Xavier’s college in Mumbai and completed his B.Sc degree in Mathematics in 1970. Subsequently he attended the University of Notre Dame in Indiana and completed his MS and his PhD degrees in Mathematics (Theory of Finite Simple Groups) under the direction of Professor W. J. Wong. He was an instructor of Mathematics at the University of Utah from 1975 to 1977.

In June 1977, Hari decided to work with Professor Raj Chandra Bose in the field of Experimental Design, at Colorado State University, and received his PhD in Statistics in 1980. Immediately following this he joined the faculty in the Department of Statistics at Colorado State University in 1980. From 1985 until recently, Hari was a faculty appointee at NIST (Boulder).

From 2007 to 2012 Hari held a position in the information analytics division of Caterpillar Inc. From 2012 to 2014 he was part of the analytics group at CGN Inc. Hari joined NIST (SED Gaithersburg) in February 2014.

During the recent years Hari, collaborating with Jack Wang and Jan Hannig, has contributed to research related to quantification of uncertainty as proposed in the Guide to the Expression of Uncertainty in Measurements (GUM). Much of this research is based on Fiducial Inference Methodology. After moving to NIST (Gaithersburg) Hari has been involved in research in forensic statistics, particularly relating to likelihood ratios for fingerprint and footwear fields (pattern comparison). He is currently collaborating with Steve Lund (SED) on these topics. Hari is also involved in machine learning applications in genomics research with NIST collaborators in Material Measurement Laboratory (MML).

johnlibert
John Libert

NIST

John Libert has worked at NIST for over twenty years, variously engaged in developing measurements for assessment of video quality, performance of electronic displays, and biometric image quality. His current primary focus is development of metrics, protocols, and standard test artifacts for the evaluation of contactless fingerprint acquisition devices and their interoperability with contact devices.

John Lu
John Lu
Mathematical Statistician

Statistical Engineering, NIST

John Lu grew up in northern Anhui Province, China before he went to Beijing to study at the Peking University where he received a BS in mathematics with specialization in probability and statistics in 1986. He came to the US to study at the University of North Carolina, Chapel Hill, and received MS and PhD in statistics, under the guidance of Richard L. Smith. He was a visiting scientist at National Center for Atmospheric Research (NCAR), Boulder, Colorado, where he did post-doctoral research on applying Bayesian statistics to atmospheric sciences under the mentorship of professor Mark Berliner from the Ohio State University. He spent a year as a visiting assistant professor at the Hong Kong University of Science and Technology in Kowloon, Hong Kong, China in 1997-1998.

He was a research scientist doing S-plus statistical software development and research from 1999-2001. He has been at NIST as a mathematical statistician since July 2001. His main research activity has been in interdisciplinary statistical research, working closely with other discipline scientists to address various measurement science problems arising from physical, phantom, or medical products. His experiences of consulting and research cut across many application areas including climatology, material sciences, biotechnology, chemical sciences, information technology, manufacturing, and physical sciences.

Steve_crop
Steve Lund

NIST

Steve Lund graduated from St. Olaf College with majors in math and physics.He received his PhD in statistics from Iowa State University and joined the Statistical Engineering Division at NIST in 2012.While at NIST, Steve has worked on a variety of forensic science projects including comparison algorithms for footwear impression evidence, model uncertainty, probabilistic interpretation, and data visualization.He has served as a member of the OSAC Footwear and Tire Subcommittee and as a delegate to the International Standards Organization Technical Committee for Forensic Science.Steve has also worked on the NIST Review of Scientific Foundations for Firearms Examination.

swofford_henry
Dr. Henry Swofford
Physical Scientist for Forensic Science

Special Programs Office, National Institute of Standards and Technology

Dr. Henry Swofford is a Physical Scientist in the Forensic Science Research Program at the National Institute of Standards and Technology, Gaithersburg, MD.  Swofford is certified by the International Association for Identification (IAI) as a latent print examiner with over 15 years of experience working in an ISO/IEC 17025 accredited forensic science laboratory.

 

Swofford holds a Ph.D. in Forensic Science from the University Lausanne where he focused on the development and implementation of computational algorithms in forensic
fingermark examination, a Master of Science in Forensic Science from the University of Florida, and a Bachelor of Science in Biology with a minor in Chemistry from Georgia State University.

Prior to serving as an independent forensic consultant, he was the Chief of the Latent Print Branch for the United States Army Criminal Investigation Laboratory (USACIL) responsible for overseeing forensic latent print operations in support of the criminal
investigative mission of the U.S. Department of Defense.  Under his leadership, the USACIL became the first laboratory in the United States to implement statistical methods and strengthen the scientific rigor of latent print examinations.  Prior to this, he served as a Quality Assurance Manager, Research Advisor, and Physical Scientist/Latent Print Examiner for the USACIL as well as a Laboratory Technician for the Georgia Bureau of Investigation since 2003.

 

Over his career, Swofford has authored 25 peer review articles and given over 150 professional presentations throughout the United States and International community related to forensic science methods and practices, including applications of statistical methods to latent print examination. He has testified in federal, state, and military courts as an expert and currently serves as the Chair of the Organization for Scientific Area Committees (OSAC)
Friction Ridge Subcommittee; a member of the Board of Directors for the IAI; senior editor for Forensic Science International: Synergy, member of the Editorial Boards for the Journal of Forensic Identification and Journal of Forensic Sciences, and guest editor for Forensic Science International; member of the Academy Standards Board (ASB) Friction Ridge Consensus Body; member of the
Science Advisory Board for the District of Columbia Consolidated Forensic Science Laboratory; and a Fellow of the American Academy of Forensic Sciences (AAFS), among other committees and professional affiliations.

Thompson-1_crop
Robert Thompson
Retired, Forensic Science Research

NIST

Robert M. Thompson is a retired Senior Forensic Science Research Manager with the Special Programs Office-Forensic Sciences at the National Institute for Standards and Technology (NIST) for 12 years. He has 40 years of experience as a Forensic Scientist and Criminalist. He is certified in Criminalistics by the American Board of Criminalistics (ABC) and is a past Chairman and current member – Association of Firearm and Toolmark Examiners (AFTE) Certification Program Committee. He is a Distinguished Member of AFTE and a Fellow of the American Academy of Forensic Sciences.

Mr. Thompson was awarded a Master of Forensic Science degree from The George Washington University in Washington, D.C. and a Bachelor of Science in Forensic Science with Chemistry minor from the California State University in Sacramento, California.

Prior to joining NIST, Mr. Thompson was a Senior Firearms and Toolmark Examiner for the Bureau of Alcohol, Tobacco, Firearms and Explosives (ATF) Forensic Science Laboratories, and a Forensic Scientist and Criminalist in crime laboratories with the Washoe County Sheriff’s Department (Reno, Nevada), the Oregon State Police, and the Genelex Corporation (Seattle, Washington). His court accepted expert testimony includes Firearm/Toolmark Identification, Proximity Testing, Serology and DNA analysis, Drug Analysis, Hair and Fiber Examination, Blood Spatter Reconstruction, Shoe Print Comparison, and Crime Scene/Shooting Reconstruction. Mr. Thompson has testified as an expert witness in numerous Federal and State courts and has active professional affiliations with several regional, national, and international forensic science societies and standards development organizations.

He is published in the Journal of Forensic Sciences; Forensic Science International; Journal for the Association of Firearm and Toolmark Examiners; Proceedings of SPIE – The International Society for Optical Engineering; NIST Journal of Research; Surface Topography: Metrology and Properties, Measurement Science and Technology; Measurement Science Review; Inside ATF; FBI Crime Laboratory Digest; and the Proceedings of Saratov University (Russia).

Craig_072420c
Craig Watson
Manager, Image Group

NIST

Craig Watson is the Manager of the Image Group at the National Institute of Standards and Technology (NIST) and has been at NIST for 29 years. During his time with NIST, he has been engaged in numerous initiatives to improve biometric technologies. This includes extensive work to develop fingerprint testing datasets, fingerprint image compression, and slap fingerprint segmentation (SlapSeg). Additionally, he has expertise with biometric technology evaluations, including the Fingerprint Vendor Technology Evaluation (FpVTE), Proprietary Fingerprint Testing (PFT), and Minutiae Interoperability Exchange Test (MINEX). He has been part of teams that have received a Department of Commerce Gold Medal Award (2003) and a Department of Commerce Silver Medal Award (2014) for their work related to biometric technologies.

alan-bio-pic
Alan Zheng
Mechanical Engineer

NIST

Xiaoyu Alan Zheng is a Mechanical Engineer in Sensor Science Division of the National Institute of Standards and Technology (NIST). He has a B.S. and M.S. in Mechanical Engineering and focuses his research on objective measurements and analysis of 2D/3D ballistics toolmarks. He is currently a member of the Subcommittee on Firearms & Toolmarks in the NIST OSACs, the chair of the Technical Advisor committee for AFTE, and the co-chair of the Technical Working Group for 3D Toolmark Technologies(TWG3D2T).