Cole: Forensic Science and Wrongful Convictions

COLE - FINAL (DO NOT DELETE) 6/2/2012 3:56 PM
Forensic Science and Wrongful

Convictions: From Exposer to
Contributor to Corrector
SIMON A. COLE*
INTRODUCTION
B
randon Garretts book, Convicting the Innocent, makes a number of
important contributions to the scholarly and public discourse on
miscarriages of justice. In this essay, I will focus on the contribution
that is most related to my own research interests: its contribution to our
understanding of the relationship between forensic science and
miscarriages of justice. I will first endeavor to place Garretts contribution
in historical context by briefly tracing the history of discussions about
forensic science and wrongful convictions. I will then highlight in what
way Garretts work has furthered our understanding. I will then discuss
some of the criticisms of Garretts work by advocates of forensic science
and try to explain how data limitations contribute to the difference of
opinion between Garrett and his critics. I will conclude by suggesting a
different, more theoretically grounded way of conceptualizing
miscarriages of justice that might help us move beyond these differences of
opinion. Ultimately, however, my suggestions will be highly speculative:
data limitations, again, will make it difficult to make any strong empirical
inferences about the relationship between forensic science and wrongful
convictions.
* Associate Professor of Criminology, Law and Society, University of California, Irvine;
Ph.D. (Science and Technology Studies), Cornell University; A.B., Princeton University. I am
grateful to Jolle Vuille and Norah Rudin for helpful comments on drafts of this Article. This
material is partially based upon work supported by the National Science Foundation under
grant No. SES-0115305. Any opinions, findings, and conclusions or recommendations
expressed in this material are those of the author and do not necessarily reflect the views of
the National Science Foundation.
711
712 New England Law Review v. 46 | 711
I. Early Studies of Wrongful Conviction
Historically, forensic science and miscarriages of justice were rarely

reflected upon in concert. Certainly, forensic science has been cited as a
contributor to miscarriages of justice since as long ago as the Dreyfus case.1
But until recently, forensic sciencecompared to other issues like
eyewitness identification, perjury, official misconduct, and interrogation
practiceshas tended to take a back seat in discussions of miscarriages of
justice.2 Although the earliest U.S. study of miscarriages of justice
mentioned [t]he unreliability of so-called expert evidence as a
contributor to wrongful convictions,3 most of the early studies that
attempted to systematically identify causes of wrongful conviction
discussed: eyewitness identification; false confessions; police and
prosecutorial misconduct; bad lawyering; race; failures of the discovery
process; and public pressure for a conviction, making scant mention of
forensic science.4 Two Royal Commissions issued reports addressing
problems with forensic science in Australia during the 1980s attracting
little international attention.5 As Schiffer and Champod observed, forensic
science (to convict and to exonerate) is underrepresented and often
wrongly understood in research concerning wrongful convictions.6
1 See, e.g., Laurence H. Tribe, Trial by Mathematics: Precision and Ritual in the Legal Process, 84
HARV. L. REV. 1329, 1333-34 (1971). Tribes characterization of forensic science is perhaps
unjust. See, e.g., D. H. Kaye, Revisiting Dreyfus: A More Complete Account of a Trial by
Mathematics, 91 MINN. L. REV. 825, 827 (2007); Franco Taroni et al., Forerunners of Bayesianism in
Early Forensic Science, 38 JURIMETRICS J. 183, 200 (1998).
2 PAUL ROBERTS & CHRIS WILLMORE, THE ROLE OF FORENSIC SCIENCE EVIDENCE IN CRIMINAL
PROCEEDINGS 1 (1993).
3 EDWIN M. BORCHARD, CONVICTING THE INNOCENT: ERRORS OF CRIMINAL JUSTICE, at xix
(1932).
4
See, e.g., Simon Dinitz, Foreword to C. RONALD HUFF ET AL., CONVICTED BUT INNOCENT:
WRONGFUL CONVICTIONS AND PUBLIC POLICY, at xiv-xv (1996); William O. Douglas, Foreword
to JEROME FRANK & BARBARA FRANK, NOT GUILTY 11-12 (1957); EDWARD D. RADIN, THE
INNOCENTS 231-35 (1964); Talia Roitberg Harmon, Predictors of Miscarriages of Justice in Capital
Cases, 18 JUST. Q. 949, 950 (2001). But see MICHAEL L. RADELET ET AL., IN SPITE OF INNOCENCE:
ERRONEOUS CONVICTIONS IN CAPITAL CASES 148 (1992) (discussing a case where the stray hair
of an unknown person was bottled along with the specimen collected during an autopsy
unbeknownst to the forensic expertcontributing to a miscarriage of justice).
5 Kent Roach, Forensic Science and Miscarriages of Justice: Some Lessons from Comparative
Experience, 50 JURIMETRICS J. 67, 73-74 & n.32 (2009).

6
Beatrice Schiffer & Christophe Champod, Judicial Error and Forensic Science: Pondering the
Contribution of DNA Evidence, in WRONGFUL CONVICTION: INTERNATIONAL PERSPECTIVES IN
MISCARRIAGES OF JUSTICE 33, 45 (C. Ronald Huff & Martin Killias eds., 2008).
2012 Forensic Science and Wrongful Convictions 713
This made some intuitive sense because the characteristics popularly

associated with science seem antithetical to the characteristics of
wrongful convictions. Wrongful convictions were thought to be caused by
unclear, misguided, or fallacious reasoning, but science is supposed to
embody clear, rational reasoning. Wrongful convictions were also thought
to be caused by unjustified biases against people of certain races or classes,
against persons with prior criminal records, or even simply against the
polices preferred suspect.7 By contrast, science is supposed to be objective
and free of bias.8 Wrongful convictions were thought to be caused by
deceitful and otherwise unreliable information given by witnesses,
informants, co-conspirators, and the like.9 But science, goes the truism,
7See BRANDON L. GARRETT, CONVICTING THE INNOCENT: WHERE CRIMINAL PROSECUTIONS

GO WRONG 12, 73 (2011).
8 Rebecca Tsosie, Tribal Environmental Policy in an Era of Self-Determination: The Role of
Ethics, Economics, and Traditional Ecological Knowledge, 21 VT. L. REV. 225, 267 (1996).
9 See Understand the Causes: Eyewitness Misidentification, THE INNOCENCE PROJECT,
never lies. Wrongful convictions were thought to be caused by evidence

that was less reliable than it appearedlike eyewitness identification
evidencebut science is by nature associated, in the popular imagination,
with high reliability, indeed often with certainty.
II. Forensic Science as Exposer of Miscarriages of Justice
The development of forensic DNA profiling during the 1980s caused

people to begin associating forensic science with miscarriages of justice.10
Beginning with Gary Dotson, and then David Vasquez in 1989, post-
conviction DNA testing exposed a number of miscarriages of justice in the
United States.11 Realizing the potential of post-conviction DNA testing to
expose miscarriages of justice, in 1992, American attorneys Peter Neufeld
and Barry Scheck founded the Innocence Project at Cardozo Law School as
a legal clinic dedicated to such testing.12 Over the next two decades, the
Innocence Project and other independent efforts exposed more than 250
wrongful convictions in the United States through post-conviction DNA
testing.13 This set of wrongful convictions has taken on a degree of
significance beyond the parties involved in the underlying cases
themselves. These cases have acquired significance in drawing attention to
the issue of miscarriages of justice, to flaws in the American justice system,
and to capital punishment. In part, their significance derives from their
ability to be packaged and conceptualized as a data set through media
such as reports, books, and the Innocence Projects own website.
Additionally, their significance derives from their ability to achieve
supposed scientific certainty14 or epistemological closure.15 Alleged
miscarriages of justice exposed through post-conviction DNA testing were
less vulnerable to disputes over whether they should be characterized as
http://www.innocenceproject.org/understand/Eyewitness-Misidentification.php (last visited

June 1, 2012); Understand the Causes: Informants, THE INNOCENCE PROJECT, http://www.
innocenceproject.org/understand/Snitches-Informants.php (last visited June 1, 2012).
10 Brandon L. Garrett, Judging Innocence, 108 COLUM. L. REV. 55, 57 (2008).
11
See id. at 63, 74 & n.71 (stating that Gary Dotson was the first person in the United States
to be exonerated by post-conviction DNA testing).
12
Mission Statement, INNOCENCE PROJECT, http://www.innocenceproject.org/about/
MissionStatement.php (last visited June 1, 2012).
13 Facts on Post-Conviction DNA Exonerations, INNOCENCE PROJECT, http://www.innocence
project.org/Content/Facts_on_PostConviction_DNA_Exonerations.php (last visited June 1,

2012) (stating that in the United States, 289 people have been exonerated by post-conviction
DNA testing).
14
See Seth F. Kreimer, Truth Machines and Consequences: The Light and Dark Sides of
Accuracy in Criminal Justice, 60 N.Y.U. ANN. SURV. AM. L. 655, 660 (2005).
15 Jay D. Aronson & Simon A. Cole, Science and the Death Penalty: DNA, Innocence, and the
Debate Over Capital Punishment in the United States, 34 LAW & SOC. INQUIRY 603, 617 (2009).
miscarriages of justice at all. While some post-conviction DNA

exonerations may be challenged, even the most determined innocence
skeptics concede that the vast majority of post-conviction exonerations
were miscarriages of justice.16
III. Forensic Science as Contributor to Miscarriages of Justice
Thus, forensic science was initially perceived as a powerful tool for

exposing wrongful convictions.17 However, the earliest analyses of post-
conviction DNA exonerations as a data set revealed a paradox. Forensic
science was not merely the engine for exposing miscarriages of justice; it
also appeared to be a contributor to miscarriages of justice.18 An analysis of
the first twenty-eight cases of post-conviction DNA exonerations noted:
A majority of the cases involved non-DNA-tested forensic
evidence that was introduced at trial. Although not pinpointing
the defendants, that evidence substantially narrowed the field of
possibilities to include them. Typically, those cases involved
comparisons of nonvictim specimens of blood, semen, or hair at
the crime scene to that of the defendants. Testimony of
prosecution experts also was used to explain the reliability and
scientific strength of non-DNA evidence to the jury.19
Thus, post-conviction DNA exoneration introduced forensic science into
the discourse on miscarriages of justice in two ways: (1) as a tool for
exposing miscarriages of justice in a way that allowed for bypassing debates
over whether the alleged miscarriages of justice were, in fact, miscarriages
of justice; and (2) as a potentially important cause of miscarriages of justice.
Paradoxically, forensic science was little discussed as a cause of
miscarriages of justice until its role was exposedby forensic science.
16 See Barry C. Scheck, Barry Scheck Lectures on Wrongful Convictions, 54 DRAKE L. REV. 587,
604-06 (2006).
17
See EDWARD CONNORS ET AL., U.S. DEPT OF JUSTICE, CONVICTED BY JURIES, EXONERATED
BY SCIENCE: CASE STUDIES IN THE USE OF DNA EVIDENCE TO ESTABLISH INNOCENCE AFTER
TRIAL 2 (1996), available at https://www.ncjrs.gov/pdffiles/dnaevid.pdf.
18 See Clive Walker & Russell Stockdale, Forensic Evidence, in MISCARRIAGES OF JUSTICE: A
REVIEW OF JUSTICE IN ERROR 119, 126-27 (Clive Walker & Keir Starmer eds., 1999).
19 CONNORS ET AL., supra note 17, at 15.
During this same period, a miscarriage of justice crisis arose in the

United Kingdom. Among the most prominent alleged miscarriages of
justice were three 1974 Irish Republic Army (IRA) bombing cases. These
cases resulted in the convictions of the so-called Guildford Four,
Birmingham Six, and Maguire Seven.20 All of the cases involved
explosive residue evidence.21 These cases prompted two official inquiries,
which highlighted the role of forensic science in miscarriages of justice. The
Royal Commission on Criminal Justices 1993 Runciman Report
discussed a number of issues concerning forensic science including: failure
to adhere to objectivity and impartiality; problems with interpretation of
evidence; failure to communicate findings clearly; inequalities between
defense and prosecution resources; defense access to samples; pro-
prosecution bias; expert shopping; and the low accuracy of the residue
detection techniques themselves.22 In 1994, the May Inquiry discussed
the role of forensic science in the Guildford Four and Maguire Seven cases.
The May Inquiry primarily blamed individual forensic scientists for the
20 See RICHARD NOBLES & DAVID SCHIFF, UNDERSTANDING MISCARRIAGES OF JUSTICE: LAW,
THE MEDIA, AND THE INEVITABILITY OF CRISIS 117-22 (2000).
21 Id. at 189. Clive Walker & Carol McCartney, Criminal Justice and Miscarriages of Justice in
England and Wales, in WRONGFUL CONVICTION: INTERNATIONAL PERSPECTIVES ON

MISCARRIAGES OF JUSTICE, supra note 6, at 183, 188.
22 Gary Edmond, Constructing Miscarriages of Justice: Misunderstanding Scientific Evidence in
High Profile Criminal Appeals, 22 OXFORD J. LEGAL STUD. 53, 61 & n.43, 62 (2002).
failings of forensic science in cases involving miscarriages of justice.23

In Canada, the 1998 Kaufman Report discussed the role that
microscopic hair comparison played in the wrongful conviction of Guy
Paul Morin for murder.24 Morin had been exonerated by post-conviction
DNA testing. Among other things, the Kaufman Report noted that both
the overstatement of the probative value of forensics analysts findings and
the failure to disclose contamination problems contributed to Morins
wrongful conviction.25 In 2000, Barry Scheck, Peter Neufeld, and Jim
Dwyer published Actual Innocence, which analyzed the first sixty-two post-
conviction DNA exonerations.26 The book devoted two chapters to forensic
science as a contributor to miscarriages of justice, splitting the issue into
scientific misconduct (White Coat Fraud) and the unreliability of forensic
science (Junk Science).27 The former discussed notorious forensic
vigilantes, such as Fred Zain.28 The latter discussed the unreliability of
microscopic hair analysis and bite mark comparison.29 It also addressed the
need for regulation of forensic laboratories, proficiency testing, clear
reporting of error rates in order for fact-finders to assign weight to forensic
evidence, transparency, and independence from law enforcement.30 In
addition, the book revealed that the Innocence Projects founders became
familiar with forensic DNA profiling by working on the Coakley case,
which involved serology evidence that was misleadingly interpreted.31 The
authors cited serology inclusion as the second leading contributor, after
mistaken identification, to the wrongful convictions exposed by post-
conviction DNA testingcontributing to thirty-two of the sixty-two
wrongful convictions.32 Defective or Fraudulent Science was listed as the
fifth leading cause with twenty-one cases, microscopic hair comparison
was sixth with eighteen cases, other forensic inclusions was the eleventh
leading cause with five cases, and DNA inclusions was the twelfth
leading cause with one case.33
23 Id. at 64.
24 Roach, supra note 5, at 72.
25 Id.
26
See generally BARRY SCHECK ET AL., ACTUAL INNOCENCE: FIVE DAYS TO EXECUTION AND
OTHER DISPATCHES FROM THE WRONGLY CONVICTED (2000).
27 Id. at 107, 158.
28 See id. at 170-71. Fred Salem Zain was a West Virginia state trooper who was in charge
of serology for the states crime laboratory. Id. at 109.
29 Id. at 158-71.
30 Id.
31 See id. at 1-33.
32 SCHECK ET AL., supra note 26, at app. 2.
33 See id.
Michael Saks and Jonathan Koehler cited the role of forensic science as
a contributor to miscarriages of justice to support their claim that forensic
science was not as reliable as it was often claimed to be and reform of
forensic science was urgently needed.34 Saks and Koehler suspected that
Scheck et al.s splitting of forensic science into multiple categories might
obscure the significant role that forensic science plays as a contributor to
miscarriages of justice. As a result, they published a slight reanalysis of the
Innocence Project datawhich at that time represented the first eighty-six
post-conviction DNA exonerationsby aggregating all forensic
contributors into just two categories.35 Errors in forensic science testing
were found to be present in sixty-three percent of cases, second only to
eyewitnesses misidentifications. The giving of false or misleading
testimony by forensic scientists was found to be present in twenty-seven
percent of cases, the fifth most common contributor.36 The articles
placement in the prestigious journal Science helped ignite renewed efforts
to reform forensic science in the United States. Some forensic scientists,
upset by the articles portrayal of their field, questioned the data sampling
techniques, methods, [and] criteria that went into Saks and Koehlers
representation of the forensic sciences as a leading contributor to
miscarriages of justice.37 Saks and Koehler responded that [r]esearch on
DNA exonerations is obviously in its infancy, and we support calls for a
more complete and scientific review of these cases.38
Of course, post-conviction DNA testing is only one method of exposing
miscarriages of justice. In a comprehensive study of U.S. exonerations from
1989, the beginning of the post-conviction DNA exoneration era, to 2003,
Gross and colleagues found a total of 340 exonerations, slightly less than
half of which were exposed by post-conviction DNA testing.39 Gross and
his colleagues devised their own system for categorizing the causes of
these wrongful convictions; forensic science was lumped into their
perjury category. The study found that twenty-four of the wrongful
convictions involved perjury by a forensic scientist.40 This analytic
34 See Michael J. Saks & Jonathan J. Koehler, The Coming Paradigm Shift in Forensic
Identification Science, 309 SCI. 892, 893, 895 (2005).

35 Id. at 892-93.
36 Id. at 892 fig.1.
37 See, e.g., Glenn Langenburg, Letter to the Editor, Questions About Forensic Science, 311 SCI.
607, 607-08 (2006).
38 Michael J. Saks & Jonathan J. Koehler, Questions About Forensic Science, 311 SCI. 607, 608-
09 (2006).
39 Post-conviction DNA testing exposed a total of 144 exonerations. Samuel R. Gross et al.,
Exonerations in the United States, 1989 through 2003, 95 J. CRIM. L. & CRIMINOLOGY 523, 523-24
(2005).
40 Id. at 543.
approach narrowed the apparent contribution of forensic science to

miscarriages of justice.
In 2008, Garrett published an extensive analysis of the first 200 post-
conviction DNA exonerations. He reported that [f]orensic evidence was
the second leading type of evidence supporting these erroneous
convictions, appearing in fifty-seven percent of the cases.41 A more
extensive analysis of the role of forensic science in wrongful convictions
reported that [t]wo hundred thirty-two innocent persons have now been
exonerated by post-conviction DNA testing[] and offered an extensive
analysis of these cases.42
A. The Contribution of Convicting the Innocent
Professor Garretts book, Convicting the Innocent, which extended the

analysis to the first 250 post-conviction DNA exonerations, now stands as
the definitive analysis of the data set of wrongful convictions exposed
through post-conviction DNA testing.43 In this larger study, Garrett found
forensic evidence was used in 185 of the 250 post-conviction DNA
exonerations studiedor seventy-four percent of the time.44 The key
methodological advance in Garretts study was his examination of the trial
and hearing transcripts and case files of all of the post-conviction DNA
exonerations, which he successfully obtained in 220 of the 250 cases.45 Thus,
his study was based on the trial or hearing transcripts and case files, when
available, rather than on the Innocence Projects capsule case summaries.
This is important because it gives us a sense not just of what type of
evidence was present in the case, but what words the forensic analysts
actually said on the witness stand.46 It may be argued that ensuring the
accuracy of the words expert witnesses use to convey the probative value
of their findings is at least as important as ensuring the accuracy of their
forensic techniques.47 In addition, Garretts method allowed him to drill
further into the nature of the forensic testimony. Rather than merely noting
that forensic evidence was involved or implicated in a wrongful
conviction, Garrett was, in many cases, able to report whether the evidence
was inculpatory, exculpatory, or neither; what weight was assigned to it by
41 See Garrett supra note 10, at 64, 81.

42Brandon L. Garrett & Peter J. Neufeld, Invalid Forensic Science Testimony and Wrongful
Convictions, 95 VA. L. REV. 1, 5 (2009).
43 See GARRETT, supra note 7, at 108-14.
44 Id. at 89.
45 See id. at 7.
46 See id. at 89, 90.
47 See Simon A. Cole, Where the Rubber Meets the Road: Thinking About Expert Evidence as
Expert Testimony, 52 VILL. L. REV. 803, 830-31 (2007).
the governments expert witnesses; whether it was challenged by a defense

witness; and whether the governments expert witness characterized the
evidence in a scientifically proper manner.48 He was not, however, able to
measure what weight the fact-finder assigned to the forensic evidence.
Garrett concluded that [m]ost . . . forensic analysis at these trials
offered invalid and flawed conclusions,49 and that he was confronted
with a parade of invalid forensics.50 Garrett sorted the problems with
forensics into two recurring types, which he labeled reliability and
validity.51 The first problem was the use of unreliable forensic
evidence, which Garrett defines as [a] method . . . [that] does not produce
consistent or accurate results.52 Garrett does not report the proportion of
cases that included unreliable forensic evidence.53 More than one type of
unreliable forensic evidence may have occurred in the same case.54 The
second problem is the presenting of invalid conclusions based on a
forensic analysis where the analysis itself might be reliable or
unreliable.55
1. Reservations About Categorizing the Causes of Forensic

Problems
Although it is not central to my argument in this paper, I should

express some reservations about this scheme of categorizing forensic
problems. First, the terminology may be confusing. While reliability and
validity are often used interchangeably in law, in science the two
concepts are sometimes distinguished. When reliability is distinguished
from validity, it is usually in order to make the following distinction:
reliability refers to consistency of outcomes (i.e., the process yields the
same results when performed at different times or by different analysts or
laboratories), whereas validity refers to whether the process measures
what it is intended to measure.56 But, this is not the distinction being drawn
by Garrett. Instead, he conflates the two concepts under the notion of
reliability: A method is unreliable if it does not produce consistent or
48 GARRETT, supra note 7, at 108-14.

49 Id. at 7.
50 Id. at 9.
51 Id. at 89.
52 Id. at 86, 90.
53 Id. at 89-90.
55 Id. at 90.
56 Paul C. Giannelli, The Admissibility of Novel Scientific Evidence: Frye v. United States, a
Half-Century Later, 80 COLUM. L. REV. 1197, 1201 n.20 (1980).
accurate results.57 The best term for this is probably validitythe term
Garrett has reserved for the other category of forensic problemssince
validity is generally understood to include reliability but not vice versa.
Specifically, a process cannot be more valid than it is reliableit cannot
produce accurate results at a greater rate than it produces consistent
results. However, a process can be more reliable than it is validit can
produce consistent results at a greater rate than it produces accurate
results.
Meanwhile, what Garrett calls validity, appears to refer primarily to
making correct inferences from the evidence. As Garrett himself notes:
[T]he term validity refers to whether claims or inferences are supported by
the evidence.58 However, this is the meaning of the term validity as
used in logic rather than the meaning used in science. Thus, it seems to me,
the distinction that Garrett is trying to make would have been clearer had
he chosen different labels for the two recurring types of problems with . . .
forensics.59 For example, he might have labeled them: (1) problems of
validity and (2) problems of inference.
This terminological confusion is exacerbated by some problems with
the integrity of the categories themselves. Garrett views only two forensic
techniques as reliable: serology and DNA profiling.60 This appears to be
because the techniques are not based on a subjective opinion and
because expert witnesses are able to defensibly convey the probative value
of serology or DNA profiling evidence by estimating the rarity of the
biological features found consistent with the suspect.61 However, Garretts
argument is slightly misleading. The issue is not the subjectivity of the
opinion. After all, even opinions about DNA profiles are in some sense
subjective.62 Rather the issue is whether the opinion is based on an
instrumental measurement or a visual analysis. A conclusion that serology
and DNA profiling are reliable ought to be based on: (1) studies that
measure the rate at which users of the technique achieve correct results on
samples of known origin, and (2) as Garrett correctly notes, expert
witnesses ability to provide defensible estimates of the rarity of the
biological features in question, so as to convey the probative value of the
evidence to the fact-finder.63
57 See GARRETT, supra note 7, at 86.

58 See id. at 87.
59 Id. at 89.
60 See id. at 90.
61 See id. at 86.
62See MICHAEL LYNCH ET AL., TRUTH MACHINE: THE CONTENTIOUS HISTORY OF DNA
FINGERPRINTING 190-91, 233 (2008).
63 See GARRETT, supra note 7, at 86.
Garrett labels all of the other forensic techniquesmicroscopic hair

comparison, fingerprint comparison, bite-mark comparison, shoe print
comparison, and voice comparisonas unreliable. He defines
unreliable as follows: A method is unreliable if it does not produce
consistent or accurate results.64 As an example, Garrett offers microscopic
hair comparison, and he notes that [s]tudies have found very high error
rates in hair comparison.65 Strictly speaking, Garretts definition is
imprecise: he is not claiming that hair comparison does not produce
consistent or accurate results, merely that it does not produce consistent
or accurate results at a high enough rate to satisfy his notion of
reliability.66 This may seem like a minor point, but it draws our attention
to an important issue: reliabilityor validityshould not be
conceived as a dichotomous variable in which techniques are reliable or
unreliable. Rather, we should understand that all forensic techniques
should be expected to produce some correct results and some false results;
the important issue is the ratio of correct-to-false results, which we may
term the techniques accuracy. Accuracy may be understood as the
proportion of all results that are correct results. Once we know various
techniques degrees of accuracy, we can exercise value judgments and
describe techniques that surpass some arbitrary threshold of accuracy as
reliable and refer to those that do not as unreliable.
Now that our focus is on accuracy, it becomes clear that Garretts
unreliable category lumps together a variety of techniques for which the
state of scientific knowledge with regard to their accuracy is quite different.
Hair comparison is the only technique for which Garrett explicitly claims
sufficient studies exist so that he can know the accuracy of the technique.67
Garrett claims that these studies show that the accuracy rate is poor. So
hair comparison is deemed unreliable, because he believes that he has
adequate knowledge that its accuracy is poor. The state of scientific
knowledge with regard to the accuracy of all the remaining techniques is
quite different. Garrett offers no evidence that the accuracy of any of these
techniques has been measured and found to be poor. Instead, he cites the
NAS Report, that found these disciplines supported by little rigorous
systematic research to validate . . . basic premises and techniques.68 This
statement by the NAS is not a report of an empirical finding of poor
64 Id.
65 Id.
66 See id.
67 Id. at 86.
68 See COMM. OF IDENTIFYING THE NEEDS OF THE FORENSIC SCI. CMTY., NATL RESEARCH
COUNCIL, STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES: A PATH FORWARD 22
(2009), available at https://www.ncjrs.gov/pdffiles1/nij/grants/228091.pdf.
accuracy. Rather, it is the conclusion of a literature review which has found

a dearth of empirical studies of accuracy.69 In other words, the NAS is
claiming that the accuracy of these techniques has not been sufficiently
studied, but it is not making any substantive claim about the accuracy of
these techniques.
It also becomes clear that Garretts unreliable category contains
techniqueslike hair comparisonfor which he claims that their accuracy
is known to be poor, and techniqueslike shoe print comparisonfor
which he merely claims that their accuracy is unknown. It is not always
obvious in which of these two groups a particular technique belongs. At
least some rudimentary studies exist for most of these techniques. The
issue is whether these studies should be considered so rudimentary that
little or nothing can be inferred about the accuracy of the techniques or
whether the studies should be considered sufficiently robust that such
inferences can be made. For example, do the few studies we have about bite
mark comparison mean that the accuracy of bite mark comparison is
unknown or known to be poor? The situation is further complicated by the
fact that the NASs statement applies to hair comparison as wellthe NAS
appears to view the accuracy of hair comparison as unknown, whereas
Garrett claims it is known to be poor.70
Of course, testifying about a technique whose accuracy is unknown is
problematic as well. If such testimony was proffered at trials, then
government expert witnesses testified at trials about the results of these
techniques without being able to give the fact-finder credible information
about their accuracy. As Garrett shows, in lieu of such credible
information, the governments expert witnesses relied on crude, vaguely
defined vernacular terms like similar, consistent, and match.71 As
Garrett notes, although these terms have no inherent meaning in forensic
science, jurors appear to ascribe them a very high probative value.72
Garrett reports that 185 out of 250 cases (seventy-four percent) had
forensic evidence present, that 169 of these had forensic trial testimony, that
he was able to locate the transcript for 153 of these 169 trials, and that in
69 Id. at 2.
70 Compare id. at 161, with GARRETT, supra note 7, at 86.
71 GARRETT, supra note 7, at 90.
72 See Dawn McQuiston-Surrett & Michael J. Saks, Communicating Opinion Evidence in the
Forensic Identification Sciences: Accuracy and Impact, 59 HASTINGS L.J. 1159, 1188-89 (2008)
(*T+he language employed by forensic experts affects the inferences fact finders draw,
sometimes producing conclusions in the minds of fact finders quite different from what the
expert witnesses purportedly intend.); Dawn McQuiston-Surrett & Michael J. Saks, The
Testimony of Forensic Identification Science: What Expert Witnesses Say and What Factfinders Hear,
33 LAW & HUM. BEHAV. 436, 436 (2009) (discussing how variations in the presentation of
forensic science information affect fact-finders' judgments in a trial).
sixty-one percent of these 153 trials the testimony was invalid.73 The
analysis of this invalid testimony forms Garretts principal contribution
to the discussion of the role forensic science plays in miscarriages of justice.
Elsewhere, Garrett analyzed the invalid testimony in greater detail. He
defined six different categories of invalid testimony: (1) interpreting the
nonprobative evidence as inculpatory; (2) discounting exculpatory
evidence; (3) presenting inaccurate statistics; (4) providing frequencies or
probabilities in the absence of empirical data; (5) providing nonnumerical
statements of probability or frequency despite a lack of any empirical data;
(6) concluding evidence did in fact come from the defendant despite no
empirical data permitting such conclusions.74 Garretts data, therefore,
presents a very complex picture. The forensic evidence he discusses varies
along at least four important dimensions. We have forensic evidence that is
inculpatory, exculpatory, and inconclusive, that reports results from at
least seven different forensic techniques, resulting in trial testimony that is
either valid or invalid in one or more of six different ways, over a twenty-
four-year period. Making generalizations from such data is challenging to
say the least.
2. The Significance of Garretts Findings for Forensic Science
Nonetheless, some important inferences about the state of forensic

science during this twenty-four-year period can certainly be drawn. The
data clearly indicates some of the problems that pervaded forensic science
during this period: the absence of quantitative data with which to convey
the probative value of the evidence to the fact-finder; the reliance, in lieu of
such data, on verbal characterizations that impossibly overstated the
probative value of the evidence, such as stating the defendant was the
source of some forensic traces; the use of faulty statistics in lieu of such
data; and the interpretation of evidence in a manner biased against
defendants.75 The fact that all this forensic evidencederived from
techniques of either unknown or poor accuracy, couched in invalid
testimonywas permitted to be presented to juries indicates a failure of
the gatekeeping responsibility vested in trial courts to ensure the
relevance and reliability of forensic evidence proffered in criminal trials.
The data also suggests poor performance by the prosecutors who

74 Garrett & Neufeld, supra note 42, at 16-20 (discussing six types of invalid forensic science
testimony). For a detailed companion data repository for Convicting the Innocent, see
Convicting the Innocent: Data and Materials, http://www.law.virginia.edu/html/
librarysite/garrett_innocent.htm (last visited June 1, 2012).
75 GARRETT, supra note 7, at 90, 254.
presented invalid testimony and relied upon it in closing arguments,76

defense attorneys who failed to challenge such evidence, and juries that
ignored exculpatory forensic evidence. Certainly, one can conclude from
Garretts data that during this period forensic science was tardy in
conducting studies that would generate data from which to make
defensible inferences about the probative value of evidence.77 Even more,
during this period forensic science was poorly regulated, made few efforts
to eliminate bias from interpretation of evidence, and American courts
permitted all of this without sanction.78
In sum, the great achievement of Convicting the Innocent lies in its
drilling down to the level of the transcript. Garrett is able to show not
merely that forensic evidence was present or even contributed to wrongful
convictions but how the evidence was consciously or unconsciously
distorted to fitand strengthenthe governments theory of the
defendants guilt.
3. The Significance of Garretts Findings for Post-Conviction

DNA Exonerations
Next, we might ask what this data tells us about the role of forensic
science in miscarriages of justice that were exposed by post-conviction DNA
exonerations. The answer, clearly, is that forensic science played an
important role. However, this derives less from simply counting the
proportion of cases in which forensic evidence was present, or even the
proportion of cases in which forensic evidence was inculpatory, or even the
proportion of cases in which the evidence derived from unreliable
techniques or the testimony was invalid. A closer look at Garretts data
clearly shows that forensic science primarily contributed to the
miscarriages of justice that were exposed by post-conviction DNA
exonerations in two ways.
First, serological evidence which ought to have been interpreted as
either excluding the defendant or as having nothing useful to contribute to
the fact-finders perception of the defendants guilt was instead presented
to the fact-finder as inculpatory. This occurred in sixty-seven cases.79
Second, microscopic hair comparison evidence that ought, if used at all, to
have been conveyed to the fact-finder only as failing to exclude the
76 See DANIEL S. MEDWED, PROSECUTION COMPLEX: AMERICA'S RACE TO CONVICT AND ITS
IMPACT ON THE INNOCENT 97 (2012).

78 See id. at 90-91 (explaining that many convictions were based on a range of unreliable
forensic methods, most of which are still in wide use and permitted in criminal courtrooms
today).
79 Id. at 90.
defendant or perhaps as including the defendant, among a very large

population that could have contributed the hair, was instead presented to
the fact-finder as highly incriminating. This occurred in twenty-nine cases.
These cases strongly support the argument made by a number of
scholars that bias is a potentially significant problem for forensic science.80
It is clear that the erroneous interpretations in these cases would not have
been possible were the analysts not aware of police investigators desired
outcome (i.e., who was the suspect) because otherwise they would not
have known when to suppress exculpatory results and when to interpret
nonprobative results as incriminating. As Garrett notes, the bias in these
cases was, without exception, against the defendant, although this
observation would seem to be attributable to the fact that Garrett was
working with a data set of post-conviction exoneration cases.81 It is true
that there were lots of other forms of problematic evidence involving other
types of invalid testimony and other forensic techniques. But few of
these other problems were frequent enough to allow us to draw
generalizations. The extraordinary prevalence of serology and hair
comparison in post-conviction DNA cases emerges strongly from Garretts
data.82
4. The Significance of Garretts Findings for Unexposed

Miscarriages of Justice
While Garretts data tells us a great deal about the contribution of

biased serology and microscopic hair comparison to miscarriages of justice
exposed by post-conviction DNA testing, what the data tell us about the
contribution of forensic science in general to miscarriages of justice is a
more difficult question. The crucial point to keep in mind is that, as many
scholars have noted, post-conviction DNA exonerations are an
unrepresentative data set.83 That is, post-conviction DNA exonerations
represent a set of wrongful convictions that were amenable to exposure
through post-conviction DNA testing.84 This means that biological
80 Itiel E. Dror et al., Contextual Information Renders Experts Vulnerable to Making Erroneous
Identifications, 156 FORENSIC SCI. INTL 74, 75 (2006); Dan E. Krane et al., Sequential Unmasking:
A Means of Minimizing Observer Effects in Forensic DNA Interpretation, 53 J. FORENSIC SCI. 1006,
1006 (2008); D. Michael Risinger et al., The Daubert/Kumho Implications of Observer Effects in
Forensic Science: Hidden Problems of Expectation and Suggestion, 90 CAL. L. REV. 3, 9, 11-12, 30, 41-
42 (2002).
82 Id.
83 Samuel R. Gross, Convicting the Innocent, 4 ANN. REV. L. SOC. SCI. 173, 176 (2008) (And
yet almost everything we know about false convictions in the United States depends on this
small, assorted, messy data set.).
84 See Justin Brooks & Alexander Simpson, Blood Sugar Sex Magik: A Review of Post-
evidence probative of guilt must have been present at the crime scene; that
this evidence was collected and preserved; that it was either not subjected
to DNA testing, subjected to an older, less discriminating method of
analysis, or erroneously interpreted; that the evidence remained available
for testing through post-conviction litigation; and that the crime was
serious enough that it yielded a sentence severe enough that the convict
had an incentive to pursue post-conviction DNA testing.85 In practice, this
means that sexual assault cases are highly over-represented in this data set,
that homicides are over-represented as well, and so are the combination of
the two: rape-murders.86 Thus, Garretts work allows us to conclude that
biased serology and microscopic hair comparison evidence were major
contributors to those miscarriages of justice that occurred in this selective
set of cases.
However, it is clear that there are other miscarriages of justice less
amenable to exposure through post-conviction DNA testing.87 For example,
as I have shown elsewhere, wrongful convictions involving fingerprint
evidence are less amenable to exposure through post-conviction DNA
testing than wrongful convictions involving serology or hair.88
What can we conclude about these cases from Garretts study? The
answer to this question is less clear. One perspective would be to treat
Garretts set of cases as deviantto assume that Garrett has identified a
problematic set of cases which are susceptible to specific problems. For
example, sexual assault prosecutions are especially problematic because
they are crimes that provoke high emotions and with specific evidence
collection problems (e.g., eyewitnesses who are not the victim or
perpetrator are rare). Or, perhaps serology and microscopic hair
Conviction DNA Testing Statutes and Legislative Recommendations, 59 DRAKE L. REV. 799, 805
(2011).
85 See Nicole Dapcic, A Quest for Exculpatory DNA Evidence or a Wild-Goose Chase? Expansion
of Searches for Lost Evidence Under Horton v. Maryland, 37 NEW ENG. J. ON CRIM. & CIV.
CONFINEMENT 77, 83-84 (2011).
86 Gross, supra note 83, at 179 (Once we move beyond murder and rape cases, we know
very little about any aspect of false convictions.); Alexandra Natapoff, Misdemeanors, 85 SO.
CAL. L. REV. (forthcoming 2012) (manuscript at 46) (*T+he innocence movement is centrally
concerned with serious offenses, typically murder and rape, which together comprise a small
fraction of the criminal systemaround 2 percent.); D. Michael Risinger, Innocents
Convicted: An Empirically Justified Factual Wrongful Conviction Rate, 97 J. CRIM. L. &
CRIMINOLOGY 761, 772 (2007).
87 See Gross, supra note 83, at 174-75 (We cannot study an event if we cant tell when it
happens. This is a severe problem for false convictions because, by definition, we dont know
when they occur.).
88 Simon A. Cole, More Than Zero: Accounting for Error in Latent Fingerprint Identification, 95
J. CRIM. L. & CRIMINOLOGY 985, 1025-26 (2005).

comparison are particularly problematic techniques, and cases in which

they were used as evidence were particularly susceptible to error.
Another perspective would be to treat Garretts set of cases as
revealing a deeper set of causes, implying similar problems with different
forensic techniques that are more prevalent in other sets of cases but less
amenable to post-conviction DNA testing. From this perspective, the
invalid testimony concerning serology and hair comparison are
indicative not merely of problems with these two techniques but also of
deeper problems with forensic science and the laws use of it. For example,
Garretts findings are revealing of the tendency of crime laboratories to
exaggerate or distort the probative value of forensic evidence in general and
the legal systems tendency to permit such exaggerations and distortions.89
This perspective would conclude that Garretts data suggests that crime
laboratories probably routinely provide invalid testimony derived from
other forensic techniques, and the courts probably routinely permit that
invalid testimony. We have not been made aware of the problem through
exposed miscarriages of justice only because we do not have a convenient
mechanism for post hoc exposure of miscarriages of justice, like post-
conviction DNA testing. Thus, our greater awareness of problems in
serology and hair comparison is better explained by the fact that we have
improved exposure mechanisms than by the fact that there are more
underlying problems with those techniques.
5. Collinss and Jarviss Critique of Garretts Work
The difference between these two perspectives has been brought into
relief by a critique of Garretts work by two crime laboratory advocates.
Collins and Jarvis contend that the usual methods of analyzing data
derived from post-conviction DNA exonerations, which involve counting
the proportion of cases in which forensic science was involved, overstates
the contribution of forensic science to miscarriages of justice.90 They argue
that the presence of forensic science in a case does not necessarily imply
that forensic science caused the miscarriage of justice.91
This critique certainly has some merit for early analyses which simply
counted whether forensic science was present in post-conviction DNA
exonerations. But Garrett has improved on this method by counting the
number of cases in which forensic science was not merely present but
unreliable or invalid. However, Collins and Jarvis still contend that
Garrett overstates the contribution of forensic science. In their own analysis
89 GARRETT, supra note 7, 89-91.

90John M. Collins & Jay Jarvis, The Wrongful Conviction of Forensic Science, 1 FORENSIC SCI.
POLY & MGMT. 17, 17-18 (2009).
91 See id. at 21-22.
of the post-conviction DNA exoneration data, they conclude that forensic

science played a causal role in the miscarriage of justice in only 32 of the
first 200 post-conviction DNA exoneration cases, or sixteen percent.92 In a
further analysis in which the total number of contributors to the wrongful
conviction, rather than the total number of cases, is treated as the
denominator, they conclude that only eleven percent of the total causes of
wrongful convictions were forensic science.93
Why are these figures so different from Garretts claim that sixty-one
percent of the post-DNA exonerations contained flawed forensic
testimony? The thirty-two cases in which Collins and Jarvis attribute a
causal role to forensic science are limited to cases involving what they call
forensic malpractice.94 This is defined as every conceivable failure that
could be committed, either intentionally or accidentally, by a forensic
scientist or a forensic science facility.95 Based on this definition, Collins
and Jarvis exclude a large number of cases that were included by Garrett.
For example, Collins and Jarvis categorized sixty-nine cases as instances
where [n]on-specific science failed to exclude the defendant.96 Although
it is not clear from Collins and Jarviss article,97 these are probably the
serology and hair comparison cases. The argument is this: serology is
accurate but not very discriminating. Suppose a simple serological analysis
reveals that the perpetrator is blood type Aa type shared with
approximately forty percent of the populationand the suspect is blood
type A. The analysis has failed to exclude the suspect, but proper
understanding of the evidence should make clear that this failure is not
especially probative: an individual picked at random from the population
would have a forty percent chance of not being excluded. Put another way,
an individual with a perfect alibi, who could not possibly have committed
92 Id. at 25.
93 Id.
94 Their coining of this term is criticized in Norah Rudin & Keith Inman, Who Speaks for
Forensic Science?: The Conviction and Exoneration of a Straw Man, CAL. ASS'N OF CRIMINALISTS
NEWS, Fourth Quarter 2008, at 10, 11.
95 Collins & Jarvis, supra note 90, at 24.
96John Collins & Jay Jarvis, The Wrongful Conviction of Forensic Science, CAL. ASSN OF
CRIMINALISTS NEWS, Fourth Quarter 2008, at 15, 21.
97 On January 26, 2012, I wrote to the authors through their website, CRIME LAB REPORT,
http://www.crimelabreport.com (last visited June 1, 2012), where a pre-publication version of

this study appeared. I received two replies. One stated: The e-mail message could not be
delivered because the user's mailfolder is full. The second reply stated: The CRIME LAB
REPORT editors have received your message. Thank you for contacting us. I wrote Mr. Jarvis
at an email address listed on the AMERICAN SOCIETY OF CRIME LABORATORY DIRECTORS
LABORATORY ACCREDITATION BOARD, http://www.ascld-lab.org (last visited June 1, 2012), but
received no response.
the crime, would have a forty percent chance of not being excluded.
Suppose that the same defendant is subsequently convicted, based in
part on the serological evidence. Suppose further that DNA typing is
subsequently developed, the convict requests post-conviction DNA typing,
the test is performed, and it excludes the defendant. DNA profiling is far
more discriminating than blood type. The temptation is to say that the
serological evidence was wrong or false or flawed, but, of course,
this is somewhat unfair. The serologist never said that the suspect was the
perpetrator. True, the serological evidence was used against a defendant
who turned out to be innocent. However, the serologist presumably
allowed for the fact that the suspect may have been among the many
innocent people that a blood-type analysis failed to exclude. The same
argument may be mounted in defense of hair comparison. Microscopic hair
comparison is not considered to be very discriminating.98 Thus, it should
not be surprising that many suspects might be included by microscopic hair
comparison, but excluded by DNA profiling.99
This is a sound argument if the expert witness properly conveys the
probative value of the evidence to the fact-finder. In serology, that would
mean conveying not only that the analysis failed to exclude the defendant,
but also the proportion of innocent persons the test would also be expected
to fail to exclude. For microscopic hair comparison, the problem would be
trickier because responsible estimates of the population frequencies of the
attributes analysts rely upon are not available. Therefore, the analyst
would be forced to employ vague verbal characterizations of the
discriminating power of the observed features. The testimony would be
something like: These attributes of the crime scene hairs were shared by
the sample hair from the suspect; however, many people in the population
also share these attributes.
However, Garretts findings show that the forensic expert witnesses
did not properly convey the probative value of the evidence to the fact-
finder.100 Rather, as noted above, Garrett shows that most of the serology
cases were not simply failures to exclude but cases in which masking
problems meant that the evidence should have been reported as having no
probative value whatsoeverthat the test failed to exclude one-hundred
98 See Walter F. Rowe, The Current Status of Microscopial Hair Comparisons, 2001 SCI. WORLD
868, 876.
99 See Max M. Houck & Bruce Budowle, Correlation of Microscopic and Mitochondrial DNA
Hair Comparisons, 47 J. FORENSIC SCI. 1, 964, 966 (2002); Max M. Houck, Letter to the Editor,
Forensic Science, No Consensus, 20 ISSUES SCI. & TECH. 6, 6-7 (2004); D. Michael Risinger &
Michael J. Saks, A House with No Foundation, 20 ISSUES SCI. & TECH. 35, 35 (2003); Michael J.
Saks & D. Michael Risinger, Baserates, the Presumption of Guilt, Admissibility Rulings, and
Erroneous Convictions, 2003 MICH. ST. L. REV. 1051, 1057.
100 See GARRETT, supra note 7, at 92-93.
percent of the population.101 Or, put another way, that there was a one-
hundred percent probability that the test would fail to exclude an innocent
person. In an additional set of cases, exculpatory evidence was reported as
either non-probative or inculpatory.102 Similarly, Garrett shows that in
many of the hair comparison cases, the probative value of the evidence was
exaggerated.103 It is misleading to categorize cases in which the probative
value of the evidence was skewed against the defendant as [n]on-specific
science failed to exclude the defendant.104
The above cases probably account for the bulk of the difference
between Garretts and Collins and Jarviss counts of the numbers of
wrongful convictions attributable to forensic science. Collins and Jarvis
also exclude cases in which the forensic evidence proffered against the
defendant derived from disciplines . . . rarely, if ever, practiced in
Americas crime laboratories.105 These include voice-print analysis and
dog-scent tracking.106 In my opinion, these analyses fit better in a category
called forensic science than in any other commonly used category of
evidence.107 Certainly, in the categorization typically used for the analysis
of wrongful convictions, dog scent fits better with forensic science than
with any of the other categories. But, in any event, the number of these
cases is small, so this difference in classification accounts for little of the
discrepancy between Garretts numbers and those of Collins and Jarvis.
101 See id. 87, 94-95.

102 See id.
103 See id. at 96-99.
104 See id. at 86.
106 See id. at 15, 20; see also Rudin & Inman, supra note 94, at 10, 12.
107 Voice analysis is a frequent subject of publications in forensic science journals. See, e.g.,
Geoffrey Stewart Morrison, Forensic Voice Comparison and the Paradigm Shift, 49 SCI. & JUST.
298, 301, 307 (2009). Dog scent, though perhaps less frequently, is still present. See, e.g., Lisa M.
Harvey & Jeffery W. Harvey, Reliability of Bloodhounds in Criminal Investigations, 48 J. FORENSIC
SCI. 811, 811 (2003); Gertrud A.A. Schoon, A First Assessment of the Reliability of an Improved
Scent Identification Lineup, 43 J. FORENSIC SCI. 70, 70 (1998); Gertrud A.A. Schoon & J.C.
DeBruin, The Ability of Dogs to Recognize and Cross-Match Human Odours, 69 FORENSIC SCI. INTL
111, 112 (1994). There is also a Scientific Working Group (SWG) for scent dogs called
SWGDOG. See About Scientific Work Groups, SWGFAST, http://www.swgfast.org/About
SWGs.htm (last visited June 1, 2012) (Since the early 1990s, American and International
forensic science laboratories and practitioners have collaborated in Scientific Working Groups . .
. to improve discipline practices and build consensus standards. (emphasis added)); see also
Andrew E. Taslitz, Does the Cold Nose Know? The Unscientific Myth of the Dog Scent Lineup, 42
HASTINGS L.J. 15, 54 (1990) (arguing that dog scent evidence should legally be treated as
scientific expert evidence). But see Rudin & Inman, supra note 94, at 12 (agreeing with Collins
and Jarvis that most criminalists would not consider bloodhounds part of our profession).
In purportingin a clever rhetorical conceitto exonerate forensic

science from its wrongful conviction by Garrett, Collins and Jarvis
identify who they believe to be the true perpetrator: the legal system. As
both a general matter and in specific cases, Collins and Jarvis argue that
forensic science usually gets it right, while the legal system is to blame
for miscarriages of justice.108 They contend that innocence activists like
Garrett prefer to focus on forensic science as a cause of miscarriages of
justice because many of the legislators with whom innocence activists
hope to curry favor are practicing lawyers themselves109 and because
blaming lawyers is boring.110 Thus, in the case of the wrongful conviction
of Ray Krone, for example, Collins and Jarvis blame prosecutorial
misconduct and bad lawyering because a bite mark examiner consulting
for the government concluded that the bite mark excluded Krone. The
government ignored this evidence, sought a second opinion from another
bite mark examiner who implicated Krone, concealed the exculpatory
opinion from the defense, and Krone was convicted. Garrett blames
forensic science because, after all, a forensic scientist provided evidence
that was undoubtedly crucial in persuading the jury to convict Krone.
Collins and Jarvis reply that forensic science got it right because a
forensic scientist did report a conclusion that the evidence excluded
Krone.111
IV. Forensic Science as Corrector of Miscarriages of Justice
Clearly, Krone is an odd case upon which to base the exoneration of

forensic science. If nothing else, the case shows a weakness of bite-mark
analysis given that two different examinerswho were deemed qualified
prior to the examination112reached different conclusions based on the
same data. But the broader conclusion to be drawn from this case is how
artificial it is to treat forensic science and law as separate entities
whose contributing roles in miscarriages of justice must be accounted for
separately, as if the two had nothing to do with one another. It seems,
108 See Collins & Jarvis, The Wrongful Conviction of Forensic Science, CRIME LAB REP. 12 (July
16, 2008), http://www.crimelabreport.com/library/pdf/2008-07,WCFS.pdf.

110 WILLIAM THOMPSON, NEW STUDY EXONERATES FORENSIC SCIENCE 3 (2008), available at
http://www.crimelabreport.com/library/pdf/2008-07,WCFS_Editorial.pdf.
112I note this to preclude the common tactic of invoking a post hoc assessment of
competence to explain the difference of opinion. It is, of course, fallacious to claim, once one
has decided that Krone was innocent, that the examiner who excluded Krone was
competent and the examiner who included him was incompetent. See, e.g., Simon A. Cole,
More Than Zero: Accounting for Error in Latent Fingerprint Identification, 95 J. CRIM. L. &
CRIMINOLOGY 985, 1050 (2005); Edmond, supra note 22, at 82.
instead, that one can only understand forensic science in the context of
its use in law, and one can only understand lawyering in the context of
the evidence that lawyers use to support proof. The disparity between
Garretts position and that of Collins and Jarvis, therefore, like the earlier
dispute between Saks and Koehler and defenders of forensic science,113
illustrates the limitations of blaming discrete causes like eyewitness
identification, false confession, prosecutorial misconduct, bad
lawyering, and forensic science for miscarriages of justice.114 Further,
this dispute illustrates the limitations of trying to quantify the impact of
forensic science on wrongful convictions relative to other causes.115 As
Professor Natapoff notes: This posture assumes that if those discrete
pieces of evidence were stronger, the convictions would be sound. . . .
Rather than focusing on discrete pieces of evidence such as confessions or
fingerprints, innocence skepticism should be aimed at the entire procedural
apparatus.116
To be clear, I am not questioning the value of what we might call the
categorizing project. This project has certainly succeeded in giving us a
rough sense of what the contributors to miscarriages of justice are, and the
data is sufficient to warrant the inference that our criminal justice system
has weaknesses in each of these areas. It is understandable that authors of
books about miscarriages of justice find that the standard categories offer a
convenient way of organizing the material. So, certainly, this categorical
way of thinking about miscarriages of justice has its uses. But even those
who find it useful presumably understand that it is, ultimately, a
simplification.117 It is well understood, moreover, that data limitations
make drawing further inferences about the relative role of each of these
areas or their general accuracy problematic. In real cases, these discrete
causes interact with one another in complex and contradictory ways.118
Finding patterns in these interactions is far more challenging than counting
up cases in which a particular cause was present or contributed to a
113 Beatrice Schiffer, The Relationship Between Forensic Science and Judicial Error: A
Study Covering Error Sources, Bias, and Remedies 124 (2009) (unpublished Ph.D. dissertation,
University of Lausanne) (on file with the New England Law Review), available at
http://www.unil.ch/webdav/site/esc/shared/These_Schiffer.pdf.
114 See id. at 132.
115See id. at 32, 132 (studying the influence of various causes on wrongful convictions and
then concluding that such influence cannot be quantified or empirically determined).
116 Natapoff, supra note 86, (manuscript at 49).
117 See Richard A. Leo, Rethinking the Study of Miscarriages of Justice: Developing a
Criminology of Wrongful Conviction, 21 J. CONTEMP. CRIM. JUST. 201, 213 (2005).
118See George Castelle & Elizabeth F. Loftus, Misinformation and Wrongful Convictions, in
WRONGLY CONVICTED: PERSPECTIVES ON FAILED JUSTICE 17, 30-31 (Saundra D. Westervelt &
John A. Humphrey eds., 2001).
miscarriage of justice.
CONCLUSION
In my conclusion, I want to suggest that one way of moving beyond

the debates like the one between Garrett and Collins and Jarvis is to move
beyond the oversimplified framework of discrete causes and begin the
more challenging task of thinking about miscarriages of justice as the
products of complex interactions between different items of evidence. One
possible criticism of Convicting the Innocent is that it is atheoreticalthat it
is not grounded in a theory of miscarriages of justice. This is a criticism
that, as Professor Leo has noted, applies to almost the entire field of the
study of miscarriages of justice.119 I agree with Professor Leo that the most
promising theoretical approach is that of Lofquist, an approach that is very
similarthough with somewhat different terminology and less grounded
in sociologyto that of Findley and Scott.120 Lofquist draws on Perrows
Normal Accidents Theory to posit a model of miscarriages of justice that
is like a shipping collision.121 Pieces of investigative evidence are like the
pieces of datacollected from instruments, vision, hearing, other
individualsthat are fed to a ships pilot. Sometimes a pilot may form a
false theory of where her ship is. When new evidence that might
potentially disconfirm that theory is presented to the pilot, rather than
revising her theory, she may devise ways of interpreting the new evidence
so as to make it consistent with the erroneous theory. As this process
continues, an entirely erroneous edifice of evidence is constructed. When
the error is finally exposedand the ship runs into the shoalslots of
evidence that may have seemed to have integrity in isolation turns out to
be have been misinterpreted in light of the pilots false theory.
Lofquist suggested thinking about miscarriages of justice in a similar
way, and some scholars have explored this approach as being particularly
applicable to miscarriages of justice involving forensic science.122 If we
apply Lofquists approach to the debate between Garrett and Collins and
Jarvis, we end up with a very different discussion. Our concern is no longer
119 See Leo, supra note 117, at 214.

120William S. Lofquist, Whodunit? An Examination of the Production of Wrongful Convictions,
in CONVICTED: PERSPECTIVES ON FAILED JUSTICE 174, 176 (Sandra D. Westervelt & John A.
Humphrey eds., 2003); see Keith A. Findley & Michael S. Scott, The Multiple Dimensions of
Tunnel Vision in Criminal Cases, 2006 WIS. L. REV. 291, 292-93.
121 See CHARLES PERROW, NORMAL ACCIDENTS: LIVING WITH HIGH-RISK TECHNOLOGIES 3-5
(1999).
122See William C. Thompson, Beyond Bad Apples: Analyzing the Role of Forensic Science in
Wrongful Convictions, 37 SW. U. L. REV. 1027, 1030-33 (2008); see, e.g., Cole, supra note 112, at
1057-58.
whether it was forensic science or bad lawyering that caused the

wrongful convictions. Instead, because we now know123 that the
defendants were innocent, we can conceive of forensic evidence as
potentially disconfirming evidence that, in this case, failed to dissuade the
apparatus of the statethe police, prosecutors, juries, and judgesfrom
their original, erroneous theory of the crime.
There may be a variety of reasons for these failures. In some cases, the
forensic technique may simply have lacked sufficient discriminating
power, as in the case of a properly conducted and reported serological test
that fails to exclude an innocent suspect. Or, a properly conducted forensic
test may have been ignored by state actors, either through willful
misconduct or, more likely, because it contradicted other evidence in which
those actors placed even greater trust. But, in other cases, as Garretts
research shows, it may be the forensic analyst herself who misinterprets the
evidence so that it did not contradict the theory committed to by police and
prosecutors, which the analyst had apparently committed to as well. This
appears to be what occurred in the cases in which serologists ignored the
masking issue and interpreted nonprobative or exculpatory results as
inculpatory. This is why forensic reformers argue for the sequential
unmasking of forensic analyses in order that analysts cannot commit to the
states theory of the crime.124
Rather than asking to what extent forensic science caused
miscarriages of justice, we might ask to what extent forensic science lived
up to its potential to derail impending miscarriages of justice. Even more
than other forms of evidence, people think, quite reasonably, that science
isor should be125an independent actor in the criminal justice system.
The function of forensic science should be to correct false theories
developed by police as well as to support true theories. It therefore seems
reasonable to consider the extent to which forensic science functions as an
independent check on state actors who put forth erroneous theories of
crimes (figures 1-2). Of course, answering such a question empirically will
be quite challenging. As a starting point, we would need to know the
potential of various forensic techniques to correct false theories of crimes.
Even forensic techniques that are considered unreliable, like microscopic
123Well, we do not actually know, but we have very strong reason for believing, and, even
if we are wrong about a handful of cases, it is extremely unlikely that we are wrong about
many of the 250 post-conviction DNA exonerations.
124 See supra note 81 and accompanying text.
125Should be may be more appropriate than is because most crime laboratories in the
United States are part of law enforcement agenciesdespite the recent recommendation of a
National Research Council committee that crime laboratories be independentand this is
unlikely to change. See COMM. OF IDENTIFYING THE NEEDS OF THE FORENSIC SCI. CMTY., NATL
RESEARCH COUNCIL, supra note 68, at 23.
hair comparison, have some such potential. There are, presumably,

innocent suspects who would be excluded by microscopic hair comparison.
But we have little information about the discriminating power of all but a
few forensic techniques.
We would also need data on the number of times forensic evidence
convinced state actors to alter their theory of a crime. To my knowledge,
little such data exists. One anecdotal piece of such data derives from the
FBIs early uses of DNA testing. It was reported that DNA testing excluded
the primary suspect around one-third of the time.126 Indeed, the scarcity of
such data may itself be viewed as an indication of the security and
confidence forensic scientists feel in their role in the criminal justice system
and as an indication of how they conceive that role. If forensic scientists
conceived of demonstrating their ability to correct erroneous theories of
crimes as a way of demonstrating their value to the criminal justice system,
they might have historically made greater efforts to document and count
such instances. A more disturbing interpretation of this fact, of course, is
that forensic scientists know or intuit that their value to the criminal justice
system lies in their ability to support, rather than contradict, the theories of
crimes formed by state actors. This is how we might ideally assess the
contribution of forensic science to wrongful convictions. Of course, we do
not have the data to do such an assessment retrospectively about the post-
conviction DNA exonerations or any other set of past cases. So I do not
wish to fault Garrett for not adopting an approach for which data was not
available. Clearly, Garrett did an admirable job telling us what he could
base on the available data. But, perhaps we could collect such data
prospectively. Setting up data collection in this manner would also
emphasize conceiving of forensic science as an independent check on the
theories of crimes formed by state actors.
126Peter Neufeld & Barry C. Scheck, Commentary in EDWARD CONNORS ET AL., CONVICTED
BY JURIES,EXONERATED BY SCIENCE: CASE STUDIES IN THE USE OF DNA EVIDENCE TO ESTABLISH
INNOCENCE AFTER TRIAL, at xxviii (1996) (noting that out of 8000 sexual assault cases with
conclusive DNA test results, approximately 2000 excluded the primary suspect).

Cole: Forensic Science and Wrongful Convictions

Uploaded by

Copyright:

Available Formats

Cole: Forensic Science and Wrongful Convictions

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cole: Forensic Science and Wrongful Convictions

Uploaded by

Copyright:

Available Formats

COLE - FINAL (DO NOT DELETE) 6/2/2012 3:56 PM

Forensic Science and Wrongful

* Associate Professor of Criminology, Law and Society, University of California, Irvine;

712 New England Law Review v. 46 | 711

I. Early Studies of Wrongful Conviction

Historically, forensic science and miscarriages of justice were rarely

Experience, 50 JURIMETRICS J. 67, 73-74 & n.32 (2009).

2012 Forensic Science and Wrongful Convictions 713

This made some intuitive sense because the characteristics popularly

7See BRANDON L. GARRETT, CONVICTING THE INNOCENT: WHERE CRIMINAL PROSECUTIONS

714 New England Law Review v. 46 | 711

never lies. Wrongful convictions were thought to be caused by evidence

II. Forensic Science as Exposer of Miscarriages of Justice

The development of forensic DNA profiling during the 1980s caused

http://www.innocenceproject.org/understand/Eyewitness-Misidentification.php (last visited

project.org/Content/Facts_on_PostConviction_DNA_Exonerations.php (last visited June 1,

2012 Forensic Science and Wrongful Convictions 715

miscarriages of justice at all. While some post-conviction DNA

III. Forensic Science as Contributor to Miscarriages of Justice

Thus, forensic science was initially perceived as a powerful tool for

716 New England Law Review v. 46 | 711

During this same period, a miscarriage of justice crisis arose in the

England and Wales, in WRONGFUL CONVICTION: INTERNATIONAL PERSPECTIVES ON

2012 Forensic Science and Wrongful Convictions 717

failings of forensic science in cases involving miscarriages of justice.23

718 New England Law Review v. 46 | 711

Identification Science, 309 SCI. 892, 893, 895 (2005).

2012 Forensic Science and Wrongful Convictions 719

approach narrowed the apparent contribution of forensic science to

A. The Contribution of Convicting the Innocent

Professor Garretts book, Convicting the Innocent, which extended the

41 See Garrett supra note 10, at 64, 81.

720 New England Law Review v. 46 | 711

the governments expert witnesses; whether it was challenged by a defense

1. Reservations About Categorizing the Causes of Forensic

Although it is not central to my argument in this paper, I should

48 GARRETT, supra note 7, at 108-14.

2012 Forensic Science and Wrongful Convictions 721

57 See GARRETT, supra note 7, at 86.

722 New England Law Review v. 46 | 711

Garrett labels all of the other forensic techniquesmicroscopic hair

2012 Forensic Science and Wrongful Convictions 723

accuracy. Rather, it is the conclusion of a literature review which has found

724 New England Law Review v. 46 | 711

2. The Significance of Garretts Findings for Forensic Science

Nonetheless, some important inferences about the state of forensic

73 GARRETT, supra note 7, at 89-90.

2012 Forensic Science and Wrongful Convictions 725

presented invalid testimony and relied upon it in closing arguments,76

3. The Significance of Garretts Findings for Post-Conviction

IMPACT ON THE INNOCENT 97 (2012).

726 New England Law Review v. 46 | 711

defendant or perhaps as including the defendant, among a very large

4. The Significance of Garretts Findings for Unexposed

While Garretts data tells us a great deal about the contribution of

2012 Forensic Science and Wrongful Convictions 727

J. CRIM. L. & CRIMINOLOGY 985, 1025-26 (2005).

728 New England Law Review v. 46 | 711

comparison are particularly problematic techniques, and cases in which

5. Collinss and Jarviss Critique of Garretts Work