Halkides: Circular Reasoning and Task-Irrelevant Information In The Mayfield Case

Ed. Note: Chris Halkides has been kind enough to try to make us lawyers smarter by dumbing down science enough that we have a small chance of understanding how it’s being used to wrongfully convict and, in some cases, execute defendants. Chris graduated from the University of Wisconsin-Madison with a Ph.D. in biochemistry, and teaches biochemistry, organic chemistry, and forensic chemistry at the University of North Carolina, Wilmington.

Oregon attorney Brandon Mayfield was arrested because of his alleged involvement in the Madrid Train Bombing on 11 March 2004. The most probative evidence against him was a fingermark on a bag. The FBI had identified Mr. Mayfield as the source of the fingermark, and even a defense expert concurred. The Spanish authorities did not, and within a few weeks Mr. Mayfield was released and given an apology. The Brandon Mayfield case was among the most highly publicized failures of fingermark/fingerprint evidence.

The Mayfield case was beset with several kinds of errors, two of which were circular reasoning and task-irrelevant information. The FBI used a strategy of Analyze, Compare, Evaluate—Verify (ACE-V). The analysis of the fingermark should identify features prior to the comparison with a reference fingerprint. Laura Spinney wrote,

One of the problems with the ACE-V procedure lies in sloppy execution. For example, the protocol calls for the analysis and comparison steps to be separated, with a detailed description of the mark being made before an examiner ever sees an exemplar. This is to prevent circular reasoning, in which the presence of the exemplar inspires the ‘discovery’ of previously unnoticed features in the mark. But this separation doesn’t always happen….

The 2006 report from the Office of the Inspector General stated, “We found evidence that the LPU examiners’ interpretations of some features in LFP 17 were adjusted or influenced during the comparison phase by reasoning “backward” from features that are visible in the Mayfield exemplars.” Brandon L. Garrett wrote, “One examiner changed five of his original seven points, which he had marked on the latent print, only after he looked at Mayfield’s prints.”

In 2004 the FBI convened an international panel of experts who concluded that the verifications that Mr. Mayfield were “tainted” by knowledge of the conclusions of the first examiner. A portion of their report stated, “because of the inherent pressure of such a high-profile case, the power of an IAFIS match in conjunction with the similarities in the candidate print, and the knowledge of the previous examiners’ conclusions (especially since the initial examiner was a highly respected supervisor with many years of experience), it was concluded that subsequent examinations were incomplete and inaccurate. To disagree was not an expected response.” The panel recommended that, “The original examiner’s document should be sealed or withheld from the verifier.”

The 2006 report from the Office of the Inspector General did not clearly agree or disagree with the FBI’s assessment of its problems in the verification procedure. They wrote, “In considering whether the FBI’s verification procedures contributed to the error, the OIG found it significant that the court-appointed expert, Kenneth Moses, reached the same conclusion as the FBI examiners regarding the identification. The pressures that might cause an FBI laboratory examiner to hesitate to dispute an identification by one of his colleagues in the LPU obviously should not have impacted Moses’ impartiality.” Yet they also noted that “…all of the FBI examiners interviewed by the OIG indicated that it is an extremely unusual event [one that had never happened between 2001 and the Mayfield case] for a second examiner to decline to verify an identification. The verifier begins his examination with the knowledge that another FBI examiner has already made the identification. We believe that this information could consciously or subconsciously influence of identification.”

In a 2005 retrospective on known fingermark misattribution cases, Simon Cole wrote, “Indeed, more than half (12/22) of the known misattributions were attested to by more than one examiner. This supports that argument, posited by Haber and Haber, that, if “verification” is not conducted blind, the “verifier” is more likely to ratify misattributions than detect them.” Relative to the OIG report, Itiel Dror and Simon Cole reframed the problem of verification slightly, focusing less on the FBI’s verification procedure and more on the more problem of extraneous information.

The National Commission on Forensic Science adopted a document differentiating between task-relevant versus task-irrelevant information. This document stated, “Information is task-irrelevant if it is not necessary for drawing conclusions about the propositions in question, if it assists only in drawing conclusions from something other than the physical evidence designated for examination, or if it assists only in drawing conclusions by some means other than an appropriate analytic method.”

Citing Itiel Dror’s work, many of the commission’s examples came from fingerprint/fingermark comparisons:

For example, a fingerprint examiner may need information about the surfaces from which the prints were lifted in order to assess whether discrepancies between prints could have been caused by curvature or distortion of one of the surfaces.

Task-irrelevant information includes the suspect’s criminal history, the existence of a confession, the existence of other physical evidence, and “information that another latent print examiner identified the suspect as the source of a print found on a different item at the same crime scene.”

The final example comes close to the question of whether a second examiner should be blind to knowledge of the conclusions of the first examiner without answering it directly. It is difficult to make the argument that the second examiner should know the conclusions of the first, in that the conclusions are the end-product of the analytic method, not the method itself. They are no more task-relevant than the fact that Mr. Mayfield had converted to Islam, knowledge that came to the FBI in a later phase of the investigation.

For Further Reading:
Simon A. Cole “More than Zero: Accounting for Error in Latent Fingerprint Identification” 2004-2005 Journal of Law and Criminology 95(3):985-1078.

Itiel E. Dror and Simon A. Cole “The vision in “blind” justice:
Expert perception, judgment, and visual cognition in forensic pattern recognition,” 2010 Psychonomic Bulletin & Review 17(2):161-167. doi:10.3758/PBR.17.2.161.

Brandon L. Garrett Autopsy of a Crime Lab 2022 University of California Press. ISBN: 9780520389656.

Laura Spinney, “The Fine Print,” 2010 Nature 464:344-346.

Robert B. Stacey, “A Report on the Erroneous Fingerprint Individualization in the Madrid Train Bombing Case,” 2004 Journal of Forensic Identification, 54(6):706-720.

National Commission on Forensic Science “Views of the Commission Ensuring That Forensic Analysis Is Based Upon Task-Relevant Information,” adopted 8 December 2015.

Office of the Inspector General, “A Review of the FBI’s Handling of the Brandon Mavfield Case,” 2006 US Department of Justice.