NIST Seeks Digital Forensics Experts to Participate in Vital ‘Blackbox’ Study

Objectivity and accuracy are the pinnacle of forensic science. Yet everyone can agree: humans make errors — but to what degree when it comes to digital forensic evidence-gathering and analysis?

The National Institute of Standards and Technology (NIST) is launching the first “blackbox” research study to quantify the accuracy of computer and mobile phone forensics and answer this question.

Digital evidence provides an additional layer of potential human error, especially taking into consideration rapidly evolving technologies, and situations when key evidence must be identified and extracted from large volumes of digital data. It is for these reasons that CSAFE too has is working on a mobile app analysis tool EVIHUNTER.

On a broader scale, this NIST study acts as an answer to the 2009 National Academy of Sciences report: Strengthening Forensic Science in the United States: A Path Forward, which calls for blackbox studies to measure reliability of forensic methods that involve human judgement.

Digital evidence, though grounded in technology, certainly relies on the human element. By participating in the NIST study, digital forensic practitioners can help strengthen the future of forensic science by providing a foundation of quantitative probability that can be used by courts and jurors to weigh the validity of presented digital evidence and analysis — as well as inform future studies needed in this realm. Digital forensic experts can answer a question paramount to fulfilling their own goals and missions in their positions: Are our industry sector’s methods accurate and reliable?

The Study Details

Blackbox studies are unique in their anonymity. They assess the reliability and accuracy (right or wrong) of human judgement methods only, without concern for how experts reached their answer. Therefore, the study will not judge individuals and their performance but rather will be aimed to measure the performance of the digital forensics community as a whole.

The study will be conducted online — and enrollment is now open and the test is available for approximately three months.

Digital forensic experts who volunteer for the study will be provided a download of simulated evidence from the NIST website, in the form of one virtual mobile phone and one virtual computer. In roughly a two-hour time commitment, participants will be asked to examine simulated digital evidence and answer a series of questions similar to those that would be expected in a real criminal investigation. Participants will use forensic software tools of their choosing to analyze the forensic images.

Who Can Participate

All public and private sector digital examiners who conduct hard drive or mobile phone examinations as part of their official duties are encouraged to volunteer and participate in this study.

No one individual’s performance or laboratory will be calculated. Rather, NIST will publish anonymized and comprehensive results of the overall performance of the digital forensic expert community and different sectors within that community.

To learn more or to enroll in this vital study to advancing digital forensics forward, visit NIST Blackbox Study for Digital Examiners and follow the simple steps to get started.


[Enroll in NIST Blackbox Study]