Tool induced biases? Misleading data presentation as a biasing source in digital forensic analysis

Abstract

Pattern of life analysis has gained ground in the digital forensics field due to the widespread use of smart devices and systems. At the core of pattern of life analysis are the activity-level traces. These traces require expertise to draw valid inferences regarding coherent narratives of criminal events. Such complex tasks also increase the risks of bias and error. The contextual biases have been examined in a digital forensic context, however, the flaws and misinterpretations related to the interplay between the practitioner and the presented data from various software have not been examined through research.

This study advances this knowledge by examining the flaws or misinterpretations that may occur during such interactions in digital forensic casework. Our experiment conducted a mock murder scenario where pattern of life analysis is necessary to answer investigative questions. Six digital forensics investigators used two different pattern of life analysis tools, Cellebrite and APOLLO, to analyze the data extracted from the victim's iPhone and answer nine core investigative questions. We then evaluated their answers and identified any mistakes, wherein we further explored any errors that were likely caused by data misinterpretation. Both the output from Cellebrite and APOLLO enabled investigative errors due to poor naming conventions, but Cellebrite's lack of context and details of traces contributed to the largest amount of the investigators' errors. Further, the study examines how biases/misinterpretations may possibly be mitigated by combinations of traditional quality measures in digital forensics, such as the dual tool approach and peer review.