Push button forensics – managing the downsides
My post about the value of push button forensics produced a number of interesting comments for which I am quite thankful. A common thread in many of the remarks was that someone needs to understand the the science, logic, and art behind the PBF tools. I absolutely agree. Anyone depending on a technician and a tool alone is doing a disservice to their clients, and will likely fail spectacularly in court.
As one reader put it:
“I think the point that is being missed is this – at the end of the day the goal is to produce admissible evidence. The fact remains that our system generally looks to an expert to introduce digital evidence into court. ”
Harlan made a similar comment, and really got to the heart of the matter:
“The fact is that the questions being asked by customers…was data exfiltrated, did the malware make data exfiltration possible, etc…cannot be answered by a $50/hr “analyst” with a dongle. This approach will work for low hanging fruit, but even a relatively unsophisticated compromise will be improperly and incompletely investigated in this sort of environment.”
A $50/hour analyst with a PBF dongle should not testify in court and their findings alone should not be presented to a client as they lack context and perspective. Their results are only pieces of the larger construct, a construct that should be built and signed off on by people with significantly more experience. A senior examiner can guide a team of less experienced staff using a wide variety of tools, interpret and combine the results into a well constructed report, and sign off on the team’s work product.
Law firms and private investigation firms are but two of many examples of organizations that employ associates to perform many of the simpler tasks involved in preparing cases. Doing so distributes the workload, frees senior staff up for more complex tasks, provides associates with opportunities to learn on the job under the supervision of senior staff, and ensures that work product is reviewed and approved by someone in the firm who is responsible for presenting the case to the court or to the client. The same can hold true in a computer forensics firm, lab, or department. In fact, any firm with more than a few examiners needs to operate in this manner simply for coordination and responsibility purposes. I’m just proposing that the same structure works well to mitigate the risks of using push button forensics.
We build everything from airplanes to software applications to roads out of component parts that are designed to accomplish a specific task but that, standing on their own, have little value. Organizations work in a similar manner, utilizing human components along with their associated skills and tools to streamline many processes and produce better results than one person standing alone could accomplish. Integrate PBF tools and less experienced people into your organization, manage them appropriately, validate the tools, review the results, and let the senior examiners do the heavy lifting with the complex problems, clients, and courts.
Also, I suspect if most people looked around their organization, they’ll see technicians using push button tools as part of the computer forensic process already. Do you have Voom Hard Copy II or a Talon or one of the other hardware imaging solutions? How many button presses does it take to image a drive, and who is usually pushing those buttons? Do you really believe that you’ll need to explain to a client or a court how the Talon creates an E01 image? Your report will say “Imaged the suspect’s drive with a Talon, serial number XXXXX. The hash values reported by the Talon were XXXX and they matched. The Talon was certified to be operating normally during our regular maintenance, conducted per our SOPs.” It is pretty likely that the imaging was performed by a technician, and as was the regularly scheduled testing.
Push button forensics tools are here to stay and they’re already in use in most of our organizations. There clearly are risks to using PBF and inexperienced examiners inappropriately but through sound business practices they can safely contribute to our projects and improve our efficiency in the process.
Integriography: A Journal of Broken Locks, Ethics, and Computer Forensics 
Well said sir!
The objections to PBF that I’ve been reading suffer from one or more of several fatal flaws:
1. They conflate the means used to discover evidence with expert testimony concerning that evidence, the presumption being that PBF discovery cannot be defended – or can too easily be assailed – in court. Even a cursory analysis of court records reveals that this is not true, as many EnCE’s and others can attest. The scene technician who seizes a computer is not an attorney and might know nothing about the law of seizure or rules of evidence other than a scripted procedure for proper seizure, but assuming he follows the procedure the fact that he is not an “expert” on seizure will not spoil the evidence.
2. They assume that case-critical evidence will not be verified. If so, that is entirely the fault of poor lab practice and has nothing to do with the tool that was used to initially reveal the evidence.
3. They assume loss of context. In medicine, your blood may have been analyzed by an automated machine, but it will still be the physician who brings context (i.e. your medical history, symptoms, physical exam findings, lifestyle, other lab tests, and the latest development – “evidence-based medical information” – etc.) to bear in order to interpret those results.
4. They ignore the fact that they themselves are very likely using automated tools already in which they themselves are not expert. I can easily put 20 questions to a forensic expert regarding the Excel application program, or Microsoft Word, that they could not answer.
5. They disregard the indisputable fact that modern systems – and particularly networked systems – are so complex that “expertise” cannot reside in any one individual anyway. One “expert” cannot answer fairly routine questions about the Windows Registry, while I could easily construct a bash shell script that would baffle another “expert”, merely by using some of the more obscure switches that arcane Linux utilities provide. Could they return from the courtroom, study the questions and come up with the answers? Perhaps, but so could many “technicians” who use PBF applications. The notion of “expertise” is highly tenuous at best, particularly in a field where we ourselves admit the deplorable lack of peer-reviewed science. If we wish to carry the notion of “expertise” too far, we will find ourselves in the position of alchemists arguing with flat-Earth people over whose “science” is better.