Mixed Methods Research: A Research Paradigm Whose Time Has Come

Author(s): R. Burke Johnson and Anthony J. Onwuegbuzie

Source: Educational Researcher , Oct., 2004, Vol. 33, No. 7 (Oct., 2004), pp. 14-26
Published by: American Educational Research Association

Stable URL: https://www.jstor.org/stable/3700093

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms

American Educational Research Association is collaborating with JSTOR to digitize, preserve and
extend access to Educational Researcher

This content downloaded from

103.21.125.84 on Tue, 27 Dec 2022 09:07:57 UTC
All use subject to https://about.jstor.org/terms
 I I

Mixed Methods Research:

A Research Paradigm Whose Time Has Come

by R. Burke Johnson and Anthony J. Onwuegbuzie

The purposes of this article are to position mixed methods research writing style using the impersonal passive voice and technical ter-
(mixed research is a synonym) as the natura! complement to tradi- minology, in which establishing and describing social laws is the
tional qualitative and quantitative research, to present pragmatism major focus (Tashakkori & Teddlie, 1998).
Qualitative purists (also called constructivists and interpretivists)
as offering an attractive philosophical partner for mixed methods re-
reject what they call positivism. They argue for the superiority of
search, and to provide a framework for designing and conducting
constructivism, idealism, relativism, humanism, hermeneutics,
mixed methods research. In doing this, we briefly review the para-
and, sometimes, postmodernism (Guba & Lincoln, 1989; Lincoln
digm "wars" and incompatibility thesis, we show some commonali- & Guba, 2000; Schwandt, 2000; Smith, 1983, 1984). These
ties between quantitative and qualitative research, we explain the purists contend that multiple-constructed realities abound, that
tenets of pragmatism, we explain the fundamental principle of mixed time- and context-free generalizations are neither desirable nor
research and how to apply it, we provide specific sets of designs for possible, that research is value-bound, that it is impossible to dif-
ferentiate fully causes and effects, that logic flows from specific
the two major types of mixed methods research (mixed-model de-
to general (e.g., explanations are generated inductively from the
signs and mixed-method designs), and, finally, we explain mixed meth-
data), and that knower and known cannot be separated because
ods research as following (recursively) an eight-step process. A key
the subjective knower is the only source of reality (Guba, 1990).
feature of mixed methods research is its methodological pluralism Qualitative purists also are characterized by a dislike of a de-
or eclecticism, which frequently results in superior research (com- tached and passive style of writing, preferring, instead, detailed,
pared to monomethod research). Mixed methods research will be rich, and thick (empathic) description, written directly and some-
successful as more investigators study and help advance its concepts what informally.
Both sets of purists view their paradigms as the ideal for re-
and as they regularly practice it.
search, and, implicitly if not explicitly, they advocate the in-
compatibility thesis (Howe, 1988), which posits that qualitative
and quantitative research paradigms, including their associated
or more than a century, the advocates of quantitative and methods, cannot and should not be mixed. The quantitative
qualitative research paradigms have engaged in ardent dis- versus qualitative debate has been so divisive that some gradu-
pute.1 From these debates, purists have emerged on both ate students who graduate from educational institutions with an
sides (cf. Campbell & Stanley, 1963; Lincoln & Guba, 1985).2 aspiration to gain employment in the world of academia or re-
Quantitative purists (Ayer, 1959; Maxwell & Delaney, 2004; search are left with the impression that they have to pledge alle-
Popper, 1959; Schrag, 1992) articulate assumptions that are con- giance to one research school of thought or the other. Guba (a
sistent with what is commonly called a positivist philosophy.3 4 leading qualitative purist) clearly represented the purist position
That is, quantitative purists believe that social observations when he contended that "accommodation between paradigms
should be treated as entities in much the same way that physical is impossible ... we are led to vastly diverse, disparate, and to-
scientists treat physical phenomena. Further, they contend that tally antithetical ends" (Guba, 1990, p. 81). A disturbing fea-
the observer is separate from the entities that are subject to ob- ture of the paradigm wars has been the relentless focus on the
servation. Quantitative purists maintain that social science differences between the two orientations. Indeed, the two dom-
inquiry should be objective. That is, time- and context-free gen- inant research paradigms have resulted in two research cultures,
eralizations (Nagel, 1986) are desirable and possible, and real "one professing the superiority of'deep, rich observational data'
causes of social scientific outcomes can be determined reliably and the other the virtues of 'hard, generalizable' . . . data"
and validly. According to this school of thought, educational re- (Sieber, 1973, p. 1335).
searchers should eliminate their biases, remain emotionally de- Our purpose in writing this article is to present mixed meth-
tached and uninvolved with the objects of study, and test or ods research as the third research paradigm in educational re-
empirically justify their stated hypotheses. These researchers have search.5 We hope the field will move beyond quantitative versus
traditionally called for rhetorical neutrality, involving a formal qualitative research arguments because, as recognized by mixed
methods research, both quantitative and qualitative research are
important and useful. The goal of mixed methods research is not
Educational Researcher, Vol. 33, No. 7, pp. 14-26
to replace either of these approaches but rather to draw from the

1| EDUCATIONAL RESEARCHER

This content downloaded from

103.21.125.84 on Tue, 27 Dec 2022 09:07:57 UTC
All use subject to https://about.jstor.org/terms
strengths and minimize the weaknesses of both in single research is most helpful and when and how they should be mixed or com-
studies and across studies. If you visualize a continuum with bined in their research studies.
qualitative research anchored at one pole and quantitative re- We contend that epistemological and methodological pluralism
search anchored at the other, mixed methods research covers the should be promoted in educational research so that researchers are
large set of points in the middle area. If one prefers to think cat- informed about epistemological and methodological possibilities
egorically, mixed methods research sits in a new third chair, with and, ultimately, so that we are able to conduct more effective re-
qualitative research sitting on the left side and quantitative re- search. Today's research world is becoming increasingly inter-
search sitting on the right side. disciplinary, complex, and dynamic; therefore, many researchers
Mixed methods research offers great promise for practicing need to complement one method with another, and all researchers
researchers who would like to see methodologists describe and need a solid understanding of multiple methods used by other
develop techniques that are closer to what researchers actually scholars to facilitate communication, to promote collaboration,
use in practice. Mixed methods research as the third research and to provide superior research. Taking a non-purist or com-
paradigm can also help bridge the schism between quantitative patibilist or mixed position allows researchers to mix and match
and qualitative research (Onwuegbuzie & Leech, 2004a). Meth- design components that offer the best chance of answering their
odological work on the mixed methods research paradigm can specific research questions. Although many research procedures
be seen in several recent books (Brewer & Hunter, 1989; or methods typically have been linked to certain paradigms, this
Creswell, 2003; Greene, Caracelli, & Graham, 1989; Johnson linkage between research paradigm and research methods is nei-
& Christensen, 2004; Newman & Benz, 1998; Reichardt & ther sacrosanct nor necessary (Howe, 1988, 1992). For example,
Rallis, 1994; Tashakkori & Teddlie, 1998, 2003). Much work qualitative researchers should be free to use quantitative meth-
remains to be undertaken in the area of mixed methods research ods, and quantitative researchers should be free to use qualitative
regarding its philosophical positions, designs, data analysis, va- methods. Also, research in a content domain that is dominated
lidity strategies, mixing and integration procedures, and ratio- by one method often can be better informed by the use of mul-
nales, among other things. We will try to clarify the most tiple methods (e.g., to give a read on methods-induced bias, for
important issues in the remainder of this article. corroboration, for complimentarity, for expansion; see Greene et
al., 1989). We contend that epistemological and paradigmatic
Commonalities Among the Traditional Paradigms ecumenicalism is within reach in the research paradigm of mixed
methods research.
Although there are many important paradigmatic differences be-
tween qualitative and quantitative research (which have been fre- Philosophical Issues Debates
quently written about in the Educational Researcher and other
As noted by Onwuegbuzie and Teddlie (2003), some individuals
places), there are some similarities between the various approaches
who engage in the qualitative versus quantitative paradigm debate
that are sometimes overlooked. For example, both quantitative
appear to confuse the logic ofjustification with research methods.
and qualitative researchers use empirical observations to address
That is, there is a tendency among some researchers to treat
research questions. Sechrest and Sidani (1995, p. 78) point out
epistemology and method as being synonymous (Bryman, 1984;
that both methodologies "describe their data, construct explana-
Howe, 1992). This is far from being the case because the logic of
tory arguments from their data, and speculate about why the
justification (an important aspect of epistemology) does not dic-
outcomes they observed happened as they did." Additionally,
tate what specific data collection and data analytical methods re-
both sets of researchers incorporate safeguards into their inquiries
searchers must use. There is rarely entailment from epistemology
in order to minimize confirmation bias and other sources of in-
to methodology (ohnson, Meeker, Loomis, & Onwuegbuzie,
validity (or lack of trustworthiness) that have the potential to
2004; Phillips, 2004). For example, differences in epistemologi-
exist in every research study (Sandelowski, 1986).
cal beliefs (such as a difference in beliefs about the appropriate
Regardless of paradigmatic orientation, all research in the so- logic of justification) should not prevent a qualitative researcher
cial sciences represents an attempt to provide warranted assertions
from utilizing data collection methods more typically associated
about human beings (or specific groups of human beings) and the with quantitative research, and vice versa.
environments in which they live and evolve (Biesta & Burbules, There are several interesting myths that appear to be held by
2003). In the social and behavioral sciences, this goal of under- some purists. For example, on the "positivist" side of the fence,
standing leads to the examination of many different phenomena, the barriers that quantitative educational researchers have built
including holistic phenomena such as intentions, experiences, at- arise from a narrow definition of the concept of "science." 6 As
titudes, and culture, as well as more reductive phenomena such as noted by Onwuegbuzie (2002), modern day "positivists" claim
macromolecules, nerve cells, micro-level homunculi, and bio- that science involves confirmation and falsification, and that
chemical computational systems (de Jong, 2003). There is room these methods and procedures are to be carried out objectively.
in ontology for mental and social reality as well as the more micro However, they disregard the fact that many human (i.e., subjec-
and more clearly material reality. Although certain methodolo- tive) decisions are made throughout the research process and that
gies tend to be associated with one particular research tradition, researchers are members of various social groups. A few examples
Dzurec and Abraham (1993, p. 75) suggest that "the objectives, of subjectivism and intersubjectivism in quantitative research in-
scope, and nature of inquiry are consistent across methods and clude deciding what to study (i.e., what are the important prob-
across paradigms." We contend that researchers and research lems?), developing instruments that are believed to measure what
methodologists need to be asking when each research approach the researcher views as being the target construct, choosing the

OCTOBER 2004 |5

This content downloaded from

103.21.125.84 on Tue, 27 Dec 2022 09:07:57 UTC
All use subject to https://about.jstor.org/terms
specific tests and items for measurement, making score interpre- Fortunately, many (or most?) qualitative researchers and quan-
tations, selecting alpha levels (e.g., .05), drawing conclusions and titative researchers (i.e., postpositivists) have now reached basic
interpretations based on the collected data, deciding what ele- agreement on several major points of earlier philosophical dis-
ments of the data to emphasize or publish, and deciding what agreement (e.g., Phillips & Burbules, 2000; Reichardt & Cook,
findings are practically significant. Obviously, the conduct of 1979; Reichardt & Rallis, 1994). Basic agreement has been
fully objective and value-free research is a myth, even though the reached on each of the following issues: (a) the relativity of the
regulatory ideal of objectivity can be a useful one. "light of reason" (i.e., what appears reasonable can vary across per-
Qualitative researchers also are not immune from constructive sons); (b) theory-laden perception or the theory-ladenness of facts
criticism. Some qualitative purists (e.g., Guba, 1990) openly (i.e., what we notice and observe is affected by our background
admit that they adopt an unqualified or strong relativism, which knowledge, theories, and experiences; in short, observation is not
is logically self-refuting and (in its strong form) hinders the de- a perfect and direct window into "reality"); (c) underdeter-
velopment and use of systematic standards for judging research mination of theory by evidence (i.e., it is possible for more than
quality (when it comes to research quality, it is not the case that one theory to fit a single set of empirical data); (d) the Duhem-
anyone's opinion about quality is just as good as the next per- Quine thesis or idea of auxiliary assumptions (i.e., a hypothesis
son's, because some people have no training or expertise or even cannot be fully tested in isolation because to make the test we
interest in research). We suspect that most researchers are soft rel- also must make various assumptions; the hypothesis is embedded
ativists (e.g., respecting the opinions and views of different peo- in a holistic network of beliefs; and alternative explanations will
ple and different groups). When dealing with human research, continue to exist); (e) the problem of induction (i.e., the recogni-
soft relativism simply refers to a respect and interest in under- tion that we only obtain probabilistic evidence, not final proof in
standing and depicting individual and social group differences empirical research; in short, we agree that the future may not re-
(i.e., their different perspectives) and a respect for democratic ap- semble the past); (f) the social nature of the research enterprise
proaches to group opinion and value selection. Again, however, (i.e., researchers are embedded in communities and they clearly
a strong relativism or strong constructivism runs into problems; have and are affected by their attitudes, values, and beliefs); and
for example, it is not a matter of opinion (or individual reality) (g) the value-ladenness of inquiry (this is similar to the last point
that one should or can drive on the left-hand side of the road in
but specifically points out that human beings can never be com-
Great Britain-if one chooses to drive on the right side, he or she pletely value free, and that values affect what we choose to in-
will likely have a head-on collision, at some point, and end up in vestigate, what we see, and how we interpret what we see).
the hospital intensive care unit, or worse (this is a case where sub-
Pragmatism as the Philosophical Partner
jective and objective realities directly meet and clash). The strong
for Mixed Methods Research
ontological relativistic or constructivist claim in qualitative re-
search that multiple, contradictory, but equally valid accounts of We do not aim to solve the metaphysical, epistemological, axi
the same phenomenon are multiple realities also poses some po- logical (e.g., ethical, normative), and methodological differenc
tential problems. Generally speaking, subjective states (i.e., cre- between the purist positions. And we do not believe that mix
ated and experienced realities) that vary from person to person methods research is currently in a position to provide perfect
and that are sometimes called "realities" should probably be lutions. Mixed methods research should, instead (at this tim
called (for the purposes of clarity and greater precision) multiple use a method and philosophy that attempt to fit together the
perspectives or opinions or beliefi (depending on the specific phe- sights provided by qualitative and quantitative research int
nomenon being described) rather than multiple realities (Phillips workable solution. Along these lines, we advocate considerat
& Burbules, 2000). If a qualitative researcher insists on using the of the pragmatic method of the classical pragmatists (e.g
word reality for subjective states, then for clarity we would rec- Charles Sanders Peirce, William James, and John Dewey) a
ommend that the word subjective be placed in front of the word way for researchers to think about the traditional dualisms th
reality (i.e., as in subjective reality or in many cases intersubjec- have been debated by the purists. Taking a pragmatic and b
tive reality) to direct the reader to the focus of the statement. We anced or pluralist position will help improve communicatio
agree with qualitative researchers that value stances are often among researchers from different paradigms as they attempt
needed in research; however, it also is important that research is advance knowledge (Maxcy, 2003; Watson, 1990). Pragmatis
more than simply one researcher's highly idiosyncratic opinions also helps to shed light on how research approaches can be mix
written into a report. Fortunately, many strategies are recognized fruitfully (Hoshmand, 2003); the bottom line is that research
and regularly used in qualitative research (such as member check- proaches should be mixed in ways that offer the best opportu
ing, triangulation, negative case sampling, pattern matching, ex- ties for answering important research questions.
ternal audits) to help overcome this potential problem and The pragmatic rule or maxim or method states that the curren
produce high-quality and rigorous qualitative research. Finally, meaning or instrumental or provisional truth value (which Jam
qualitative researchers sometimes do not pay due attention to [1995, 1907 original] would term "cash value") of an express
providing an adequate rationale for interpretations of their data (e.g., "all reality has a material base" or "qualitative research is su
(Onwuegbuzie, 2000), and qualitative methods of analyses too perior for uncovering humanistic research findings") is to be d
"often remain private and unavailable for public inspection" termined by the experiences or practical consequences of beli
(Constas, 1992, p. 254). Without public inspection and ade- in or use of the expression in the world (Murphy, 1990). O
quate standards, how is one to decide whether what is claimed is can apply this sensible effects- or outcome-oriented rule throu
trustworthy or defensible? thinking (thinking about what will happen if you do X), prac

16I EDUCATIONAL RESEARCHER

This content downloaded from

103.21.125.84 on Tue, 27 Dec 2022 09:07:57 UTC
All use subject to https://about.jstor.org/terms
cal experiences (observing what happens in your experience when some agreement about the importance of many (culturally de-
you do X), or experiments (formally or informally trying a rule rived) values and desired ends, such as, for example, preventing
and observing the consequences or outcomes). the dropping out of school by adolescents, reducing the use of il-
In the words of Charles Sanders Peirce (1878), the pragmatic licit drugs by children and adolescents, finding effective teaching
method or maxim (which is used to determine the meaning of techniques for different kinds of students, educating children
words, concepts, statements, ideas, beliefs) implies that we should and adults (i.e., increasing their knowledge), helping to reduce
"consider what effects, that might conceivably have practical discrimination in society, and attempting to eliminate or reduce
bearings, we conceive the object of our conception to have. Then mental, learning, and other disabilities. In other words, pragma-
our conception of these effects is the whole of our conception of tism takes an explicitly value-oriented approach to research.
the object" (this quote is found at the end of Section II in How We reject an incompatibilist, either/or approach to paradigm
to Make Our Ideas Clear). Building on Peirce's lead, James (1995, selection and we recommend a more pluralistic or compatibilist
1907 original) argued that "The pragmatic method is primarily approach. Beyond the basic pragmatic method or maxim (i.e.,
a method of settling metaphysical disputes that otherwise might translated in mixed methods research as "choose the combina-
be interminable.... The pragmatic method in such cases is to tion or mixture of methods and procedures that works best for
try to interpret each notion by tracing its respective practical con- answering your research questions") there also is a full philo-
sequences" (p. 18). Extending the works of Peirce and James, sophical system of pragmatism which was systematically devel-
Dewey spent his career applying pragmatic principles in devel- oped by the classical pragmatists (Peirce, James, Dewey) and has
oping his philosophy and in the practice of educating children been refined in newer directions by latter-day neo-pragmatists
(e.g., the Experimental School of Chicago). Dewey (1948, 1920 (e.g., Davidson, Rescher, Rorty, Putnam) (see Menand, 1997;
original) stated that "in order to discover the meaning of the idea Murphy, 1990; Rescher, 2000; Rorty, 2000). To provide the
[we must] ask for its consequences" (p. 132). In short, when reader with a better understanding of the full philosophy of prag-
judging ideas we should consider their empirical and practical matism (for consideration), we have outlined, in Table 1, what
consequences. Peirce, James, and Dewey were all interested in ex- we believe are classical pragmatism's most general and important
amining practical consequences and empirical findings to help in characteristics.
understanding the import of philosophical positions and, im- Although we endorse pragmatism as a philosophy that can
portantly, to help in deciding which action to take next as one help to build bridges between conflicting philosophies, pragma-
attempts to better understand real-world phenomena (including tism, like all current philosophies, has some shortcomings. In
psychological, social, and educational phenomena). Table 2 we present some of these. Researchers who are interested
If two ontological positions about the mind/body problem in applying pragmatism in their works should consider the short-
(e.g., monism versus dualism), for example, do not make a dif- comings, which also need to be addressed by philosophically in-
ference in how we conduct our research then the distinction is, clined methodologists as they work on the project of developing
for practical purposes, not very meaningful. We suspect that a fully working philosophy for mixed methods research. Practic-
some philosophical differences may lead to important practical ing researchers should be reflexive and strategic in avoiding the
consequences while many others may not.7 The full sets of be- potential consequences of these weaknesses in their works.
liefs characterizing the qualitative and quantitative approaches or
paradigms have resulted in different practices, and, based on our
Comparing Qualitative, Quantitative,
and Mixed Methods Research
observation and study, we believe it is clear that both qualitative
and quantitative research have many benefits and many costs. In Mixed methods research is formally defined here as the class o
some situations the qualitative approach will be more appropriate; search where the researcher mixes or combines quantitative and
in other situations the quantitative approach will be more appro- itative research techniques, methods, approaches, concepts or lang
priate. In many situations, researchers can put together insights into a single study. Philosophically, it is the "third wave" or t
and procedures from both approaches to produce a superior prod- research movement, a movement that moves past the parad
uct (i.e., often mixed methods research provides a more workable wars by offering a logical and practical alternative. Philosophic
solution and produces a superior product). We are advocating a mixed research makes use of the pragmatic method and sys
needs-based or contingency approach to research method and of philosophy. Its logic of inquiry includes the use of induc
concept selection. (or discovery of patterns), deduction (testing of theories
Philosophical debates will not end as a result of pragmatism, hypotheses), and abduction (uncovering and relying on
and certainly they should not end. Nonetheless, we agree with best of a set of explanations for understanding one's results)
others in the mixed methods research movement that consider- de Waal, 2001).
ation and discussion of pragmatism by research methodologists Mixed methods research also is an attempt to legitimate
and empirical researchers will be productive because it offers an use of multiple approaches in answering research questions, ra
immediate and useful middle position philosophically and meth- than restricting or constraining researchers' choices (i.e., it re
odologically; it offers a practical and outcome-oriented method dogmatism). It is an expansive and creative form of research
of inquiry that is based on action and leads, iteratively, to further a limiting form of research. It is inclusive, pluralistic, and
action and the elimination of doubt; and it offers a method for plementary, and it suggests that researchers take an eclecti
selecting methodological mixes that can help researchers better proach to method selection and the thinking about and con
answer many of their research questions. Pragmatically inclined of research. What is most fundamental is the research ques
philosophers and researchers also would suggest that we can reach research methods shouldfollow research questions in a way

OCTOBER 2004 |I

This content downloaded from

103.21.125.84 on Tue, 27 Dec 2022 09:07:57 UTC
All use subject to https://about.jstor.org/terms
 Table 1

General Characteristics of Pragmatism

* The project of pragmatism has been to find a middle ground see what works, what solves problems, and what helps us to
between philosophical dogmatisms and skepticism and to find survive. We obtain warranted evidence that provides us with
a workable solution (sometimes including outright rejection) answers that are ultimately tentative (i.e., inquiry provides the
to many longstanding philosophical dualisms about which best answers we can currently muster), but, in the long run,
agreement has not been historically forthcoming. use of this "scientific" or evolutionary or practical epistemol-
* Rejects traditional dualisms (e.g., rationalism vs. empiricism, ogy moves us toward larger Truths.
realism vs. antirealism, free will vs. determinism, Platonic ap- * Endorses a strong and practical empiricism as the path to de-
pearance vs. reality, facts vs. values, subjectivism vs. objec- termine what works.
tivism) and generally prefers more moderate and commonsense * Views current truth, meaning, and knowledge as tentative and
versions of philosophical dualisms based on how well they as changing over time. What we obtain on a daily basis in re-
work in solving problems. search should be viewed as provisional truths.
* Recognizes the existence and importance of the natural or * Capital "T" Truth (i.e., absolute Truth) is what will be the "final
physical world as well as the emergent social and psycholog- opinion" perhaps at the end of history. Lowercase "t" truths
ical world that includes language, culture, human institutions, (i.e., the instrumental and provisional truths that we obtain
and subjective thoughts. and live by in the meantime) are given through experience and
* Places high regard for the reality of and influence of the inner experimenting.
world of human experience in action. * Instrumental truths are a matter of degree (i.e., some estimates
* Knowledge is viewed as being both constructed and based on
are more true than others). Instrumental truth is not "stagnant,"
the reality of the world we experience and live in.
and, therefore, James (1995: 1907) states that we must "be
* Replaces the historically popular epistemic distinction between
ready tomorrow to call it falsehood."
subject and external object with the naturalistic and process-
* Prefers action to philosophizing (pragmatism is, in a sense, an
oriented organism-environment transaction.
anti-philosophy).
* Endorses fallibilism (current beliefs and research conclusions
* Takes an explicitly value-oriented approach to research that is
are rarely, if ever, viewed as perfect, certain, or absolute).
derived from cultural values; specifically endorses shared val-
* Justification comes in the form of what Dewey called "war-
ues such as democracy, freedom, equality, and progress.
ranted assertability."
* Endorses practical theory (theory that informs effective prac-
* According to Peirce, "reasoning should not form a chain which
tice; praxis).
is no stronger than its weakest link, but a cable whose fibers may
be ever so slender, provided they are sufficiently numerous and
* Organisms are constantly adapting to new situations and en-
intimately connected" (1868, in Menand, 1997, pp. 5-6). vironments. Our thinking follows a dynamic homeostatic
* Theories are viewed instrumentally (they become true and process of belief, doubt, inquiry, modified belief, new doubt,
they are true to different degrees based on how well they cur- new inquiry, . . ,in an infinite loop, where the person or re-
rently work; workability is judged especially on the criteria of searcher (and research community) constantly tries to improve
predictability and applicability). upon past understandings in a way that fits and works in the
* Endorses eclecticism and pluralism (e.g., different, even con- world in which he or she operates. The present is always a
flicting, theories and perspectives can be useful; observation, new starting point.
experience, and experiments are all useful ways to gain an un- * Generally rejects reductionism (e.g., reducing culture, thoughts,
derstanding of people and the world). and beliefs to nothing more than neurobiological processes).
* Human inquiry (i.e., what we do in our day-to-day lives as we * Offers the "pragmatic method" for solving traditional philo-
interact with our environments) is viewed as being analogous sophical dualisms as well as for making methodological
to experimental and scientific inquiry. We all try out things to choices.

offers the best chance to obtain useful answers. Many research Gaining an understanding of the strengths and weaknesses of
questions and combinations of questions are best and most fully quantitative and qualitative research puts a researcher in a posi-
answered through mixed research solutions. tion to mix or combine strategies and to use what Johnson and
In order to mix research in an effective manner, researchers first Turner (2003) call the fundamental principle of mixed research.
need to consider all of the relevant characteristics of quantitative According to this principle, researchers should collect multiple
and qualitative research. For example, the major characteristics of data using different strategies, approaches, and methods in such
traditional quantitative research are a focus on deduction, confir- a way that the resulting mixture or combination is likely to re-
mation, theory/hypothesis testing, explanation, prediction, stan- sult in complementary strengths and nonoverlapping weaknesses
dardized data collection, and statistical analysis (see Table 3 for (also see Brewer & Hunter, 1989). Effective use of this principle
a more complete list). The major characteristics of traditional is a major source of justification for mixed methods research be-
qualitative research are induction, discovery, exploration, theory/ cause the product will be superior to monomethod studies. For
hypothesis generation, the researcher as the primary "instrument" example, adding qualitative interviews to experiments as a ma-
of data collection, and qualitative analysis (see Table 4 for a more nipulation check and perhaps as a way to discuss directly the is-
complete list). sues under investigation and tap into participants' perspectives

I| EDUCATIONAL RESEARCHER

This content downloaded from

103.21.125.84 on Tue, 27 Dec 2022 09:07:57 UTC
All use subject to https://about.jstor.org/terms
 Table 2 Table 3

Some Weaknesses of Pragmatism Strengths and Weaknesses of Quantitative Research

* Basic research may receive less attention than applied research Strengths
because applied research may appear to produce more im- * Testing and validating already constructed theories about
mediate and practical results. how (and to a lesser degree, why) phenomena occur.
* Pragmatism may promote incremental change rather than more * Testing hypotheses that are constructed before the data are
fundamental, structural, or revolutionary change in society. collected. Can generalize research findings when the data
* Researchers from a transformative-emancipatory framework are based on random samples of sufficient size.
have suggested that pragmatic researchers sometimes fail to * Can generalize a research finding when it has been repli-
provide a satisfactory answer to the question "For whom is a cated on many different populations and subpopulations.
pragmatic solution useful?" (Mertens, 2003). * Useful for obtaining data that allow quantitative predictions
* What is meant by usefulness or workability can be vague un- to be made.
less explicitly addressed by a researcher. * The researcher may construct a situation that eliminates the
* Pragmatic theories of truth have difficulty dealing with the confounding influence of many variables, allowing one to
cases of useful but non-true beliefs or propositions and non- more credibly assess cause-and-effect relationships.
useful but true beliefs or propositions. * Data collection using some quantitative methods is rela-
* Many come to pragmatism looking for a way to get around tively quick (e.g., telephone interviews).
many traditional philosophical and ethical disputes (this in- * Provides precise, quantitative, numerical data.
cludes the developers of pragmatism). Although pragmatism * Data analysis is relatively less time consuming (using statis-
has worked moderately well, when put under the microscope, tical software).
many current philosophers have rejected pragmatism because * The research results are relatively independent of the re-
of its logical (as contrasted with practical) failing as a solution searcher (e.g., effect size, statistical significance).
to many philosophical disputes. * It may have higher credibility with many people in power
* Some neo-pragmatists such as Rorty (and postmodernists) com- (e.g., administrators, politicians, people who fund programs).
pletely reject correspondence truth in any form, which troubles * It is useful for studying large numbers of people.
many philosophers.
Weaknesses

* The researcher's categories that are used may not reflect

local constituencies' understandings.
and meanings will help avoid some potential problems with the * The researcher's theories that are used may not reflect local
experimental method. As another example, in a qualitative re- constituencies' understandings.
search study the researcher might want to qualitatively observe * The researcher may miss out on phenomena occurring be-
and interview, but supplement this with a closed-ended instru- cause of the focus on theory or hypothesis testing rather
ment to systematically measure certain factors considered im- than on theory or hypothesis generation (called the confir-
portant in the relevant research literature. Both of these examples mation bias).
could be improved (if the research questions can be studied this * Knowledge produced may be too abstract and general for
direct application to specific local situations, contexts, and
way) by adding a component that surveys a randomly selected
individuals.
sample from the population of interest to improve generalizabil-
ity. If findings are corroborated across different approaches then
greater confidence can be held in the singular conclusion; if the
findings conflict then the researcher has greater knowledge and can many other typologies (especially Creswell, 1994; Morgan, 1998;
modify interpretations and conclusions accordingly. In many cases Morse, 1991; Patton, 1990; and Tashakkori & Teddlie, 1998), as
the goal of mixing is not to search for corroboration but rather to well as several dimensions which one should consider when plan-
expand one's understanding (Onwuegbuzie & Leech, 2004b). ning to conduct a mixed research study. For example, it has been
Tables 3 and 4 are specifically designed to aid in the con- noted that one can construct mixed-model designs by mixing
struction of a combination of qualitative and quantitative re- qualitative and quantitative approaches within and across the
search. After determining one's research question(s), one can stages of research (in a simplified view, one can consider a single
decide whether mixed research offers the best potential for an an- study as having three stages: stating the research objective, col-
swer; if this is the case, then one can use the tables as an aid to lecting the data, and analyzing/interpreting the data; see mixed-
help in deciding on the combination of complementary strengths model designs in Johnson & Christensen, 2004; Tashakkori &
and nonoverlapping weaknesses that is appropriate for a partic- Teddlie, 1998). According to Morgan (1998) and Morse (1991),
ular study. Table 5 shows some of the strengths and weaknesses one also may consider the dimension of paradigm emphasis (de-
of mixed methods research, which should aid in the decision to ciding whether to give the quantitative and qualitative compo-
use or not use a mixed methods research approach for a given re- nents of a mixed study equal status or to give one paradigm the
search study. dominant status). Time ordering of the qualitative and quanti-
tative phases is another important dimension, and the phases can
Development of a Mixed Methods be carried out sequentially or concurrently. Our mixed-method
Research Typology designs (discussed below) are based on the crossing of paradigm
Our mixed methods research typologies (mixed-model designs emphasis and time ordering of the quantitative and qualitative
and mixed-method designs) resulted from our consideration of phases. Another dimension for viewing mixed methods re-

OCTOBER 2004 I|E

This content downloaded from

103.21.125.84 on Tue, 27 Dec 2022 09:07:57 UTC
All use subject to https://about.jstor.org/terms
 Table 4

Strengths and Weaknesses of Qualitative Research

Strengths * Qualitative researchers are responsive to changes that oc

* The data are based on the participants' own categories of during the conduct of a study (especially during exten
meaning. fieldwork) and may shift the focus of their studies as a resu
* It is useful for studying a limited number of cases in depth. * Qualitative data in the words and categories of participan
* It is useful for describing complex phenomena. lend themselves to exploring how and why phenome
occur.
* Provides individual case information.
* Can conduct cross-case comparisons and analysis. * One can use an important case to demonst
* Provides understanding and description of people's per- phenomenon to the readers of a report.
sonal experiences of phenomena (i.e., the "emic" or insider's * Determine idiographic causation (i.e., dete
viewpoint). causes of a particular event).
* Can describe, in rich detail, phenomena as they are situated Weaknesses
and embedded in local contexts.
* Knowledge produced may not generalize to other people or
* The researcher identifies contextual and setting factors as
other settings (i.e., findings may be unique to the relatively
they relate to the phenomenon of interest.
few people included in the research study).
* The researcher can study dynamic processes (i.e., docu-
* It is difficult to make quantitative predictions.
menting sequential patterns and change).
* It is more difficult to test hypotheses and theories.
* The researcher can use the primarily qualitative method of
* It may have lower credibility with some administrators and
"grounded theory" to generate inductively a tentative but
commissioners of programs.
explanatory theory about a phenomenon.
* It generally takes more time to collect the data when com-
* Can determine how participants interpret "constructs" (e.g.,
pared to quantitative research.
self-esteem, IQ).
* Data analysis is often time consuming.
* Data are usually collected in naturalistic settings in qualita-
tive research. * The results are more easily influenced by the researcher's
personal biases and idiosyncrasies.
* Qualitative approaches are responsive to local situations,
conditions, and stakeholders' needs.

search is the degree of mixture, which would form a continuum the table. To construct a mixed-method design, the researcher
from monomethod to fully mixed methods. Another dimen- must make two primary decisions: (a) whether one wants to
sion pertains to where mixing should occur (e.g., in the objec- operate largely within one dominant paradigm or not, and
tive[s], methods of data collection, research methods, during (b) whether one wants to conduct the phases concurrently or
data analysis, data interpretation). Yet another important sequentially. In contrast to mixed-model designs, mixed-method
dimension is whether one wants to take a critical theory/ designs are similar to conducting a quantitative mini-study and
transformative-emancipatory (Mertens, 2003) approach or a less a qualitative mini-study in one overall research study. Nonethe-
explicitly ideological approach to a study. Ultimately, the possi- less, to be considered a mixed-method design, the findings must
ble number of ways that studies can involve mixing is very large be mixed or integrated at some point (e.g., a qualitative phase
because of the many potential classification dimensions. It is a
might be conducted to inform a quantitative phase, sequentially,
key point that mixed methods research truly opens up an excit-
or if the quantitative and qualitative phases are undertaken con-
ing and almost unlimited potential for future research.
currently the findings must, at a minimum, be integrated during
Toward a Parsimonious Typology the interpretation of the findings).
of Mixed Research Methods It is important to understand that one can easily create more

The majority of mixed methods research designs canuser

be specific
devel- and more complex designs than the ones shown in Fig-
ures 1 and 2. For example, one can develop a mixed-method de-
oped from the two major types of mixed methods research:
sign that has more stages (e.g., Qual -> QUAN -> Qual); one
mixed-model (mixing qualitative and quantitative approaches
also can design a study that includes both mixed-model and
within or across the stages of the research process) and mixed-
mixed-method design features. The point is for the researcher to
method (the inclusion of a quantitative phase and a qualitative
be creative
phase in an overall research study). Six mixed-model designs areand not be limited by the designs listed in this article.
Furthermore,
shown in Figure 1 (see Designs 2 through 7). These six designs sometimes a design may emerge during a study in
are called across-stage mixed-model designs because the newmixing
ways, depending on the conditions and information that is
obtained.
takes place across the stages of the research process. An example A tenet of mixed methods research is that researchers

of a within-stage mixed-model design would be the useshould mindfully create designs that effectively answer their re-
of a ques-
search questions; this stands in contrast to the common approach
tionnaire that includes a summated rating scale (quantitative
in (qualita-
data collection) and one or more open-ended questions traditional quantitative research where students are given a
tive data collection). menu of designs from which to select.8' 9 It also stands in stark
Nine mixed-method designs are provided in Figure 2. The no-
contrast to the approach where one completely follows either the
tation used (based on Morse, 1991) is explained at the bottom
qualitative
ofparadigm or the quantitative paradigm.

i2| EDUCATIONAL RESEARCHER

This content downloaded from

103.21.125.84 on Tue, 27 Dec 2022 09:07:57 UTC
All use subject to https://about.jstor.org/terms
 Table 5

Strengths and Weaknesses of Mixed Research

Strengths * Can add insights and understanding that might be missed

* Words, pictures, and narrative can be used to add meaning when only a single method is used.
to numbers. * Can be used to increase the generalizability of the results.
* Numbers can be used to add precision to words, pictures, * Qualitative and quantitative research used together produce
and narrative. more complete knowledge necessary to inform theory and
practice.
* Can provide quantitative and qualitative research strengths
(i.e., see strengths listed in Tables 3 and 4). Weaknesses
* Researcher can generate and test a grounded theory.
* Can be difficult for a single researcher to carry out both
* Can answer a broader and more complete range of research
qualitative and quantitative research, especially if two or
questions because the researcher is not confined to a single more approaches are expected to be used concurrently; it
method or approach. may require a research team.
* The specific mixed research designs discussed in this article * Researcher has to learn about multiple methods and ap-
have specific strengths and weaknesses that should be con- proaches and understand how to mix them appropriately.
sidered (e.g., in a two-stage sequential design, the Stage 1 * Methodological purists contend that one should always work
results can be used to develop and inform the purpose and within either a qualitative or a quantitative paradigm.
design of the Stage 2 component). * More expensive.
* A researcher can use the strengths of an additional method * More time consuming.
to overcome the weaknesses in another method by using * Some of the details of mixed research remain to be worked
both in a research study. out fully by research methodologists (e.g., problems of par-
* Can provide stronger evidence for a conclusion through adigm mixing, how to qualitatively analyze quantitative data,
convergence and corroboration of findings. how to interpret conflicting results).

A Mixed Methods Research Process Model pose can be revised when needed. Figure 3 shows several arrows
Our mixed methods research process model comprises leading
eightfrom
dis-later steps to earlier steps indicating that mixed re-
tinct steps: (1) determine the research question; (2) search involves a cyclical, recursive, and interactional process. Re-
determine
whether a mixed design is appropriate; (3) select thecursion can take place within a single study (especially an extended
mixed-
method or mixed-model research design; (4) collectstudy); recursion can also take place across related studies by in-
the data;
forming future
(5) analyze the data; (6) interpret the data; (7) legitimate the research and leading to new or reformulated re-
search
data; and (8) draw conclusions (if warranted) and write the purposes
final and questions.
report. These steps are displayed in Figure 3. Although Three
mixedsteps
re- in the mixed methods research process warrant
search starts with a purpose and one or more researchsome further discussion, especially purpose (Step 2), data analy-
questions,
sis necessar-
the rest of the steps can vary in order (i.e., they are not (Step 5), and legitimation (Step 7). As noted by Greene et al.
ily linear or unidirectional), and even the question and/or pur- are five major purposes or rationales for conducting
(1989), there

Qualitative Research Quantitative Research

Objective(s) Objective(s)

Collect
/ Collect Collect Collect
qualitative quantitative qualitative quantitative
data data data data

Perform Perform Perform Perform Perform Perform Perform Perform

qualitative quantitative
qualitative quantitative
qualitative quantitative qualitative quantitative
analysis analysis analysis analysis analysis analysis analysis analysis
1 2 3 4 5 6 7 8

Designs

Note. Designs 1 and 8 on the outer edges are the monomethod designs. The mixed-model designs are Designs 2, 3, 4, 5, 6, and 7.10

FIGURE 1. Monomethod and mixed-model designs.

OCTOBER 2004 |I

This content downloaded from

103.21.125.84 on Tue, 27 Dec 2022 09:07:57 UTC
All use subject to https://about.jstor.org/terms
 Time Order
Decision
Concurrent Sequential

QUAL + QUAN QUAL -- QUAN

Equal
Status QUAN - QUAL

Paradigm
Emphasis QUAL + quan QUAL - quan
Decision qual - QUAN
Dominant
Status QUAN -) qual
QUAN + qual quan -) QUAL

Note. "qual" stands for qualitative, "quan" stands for quantitative, "+" stands for concurrent, "->" stands
for sequential, capital letters denote high priority or weight, and lower case letters denote lower priority or
weight."

FIGURE 2. Mixed-method design matrix with mixed-method research designs shown

in thefour cells.

mixed methods research: (a) triangulation (i.e., seeking conver- titative data sources. Data integration characterizes the final stage,
gence and corroboration of results from different methods and whereby both quantitative and qualitative data are integrated
designs studying the same phenomenon); (b) complementarity into either a coherent whole or two separate sets (i.e., qualitative
(i.e., seeking elaboration, enhancement, illustration, and clarifi- and quantitative) of coherent wholes.
cation of the results from one method with results from the other The legitimation step involves assessing the trustworthiness of
method); (c) initiation (i.e., discovering paradoxes and contra- both the qualitative and quantitative data and subsequent inter-
dictions that lead to a re-framing of the research question); (d) de- pretations. Frameworks such as the Quantitative Legitimation
velopment (i.e., using the findings from one method to help Model (Onwuegbuzie, 2003; which contains 50 sources of inva-
inform the other method); and (e) expansion (i.e., seeking to ex- lidity for the quantitative component of the mixed methods re-
pand the breadth and range of research by using different meth- search at the data collection, data analysis, and data interpretation
ods for different inquiry components). stages of the study) and the Qualitative Legitimation Model
The mixed methods research process model incorporates (Onwuegbuzie, 2000; Onwuegbuzie, Jiao, & Bostick, 2004;
Onwuegbuzie and Teddlie's (2003) seven-stage conceptualiza- which contains 29 elements of legitimation for the qualitative
tion of the mixed methods data analysis process. According to component of the mixed methods research at the data collection,
these authors, the seven data analysis stages are as follows: (a) data data analysis, and data interpretation stages of the study) can be
reduction, (b) data display, (c) data transformation, (d) data cor- used to assess the legitimacy of the qualitative and quantitative
relation, (e) data consolidation, (f) data comparison, and (g) data phases of the study, respectively. We have begun working on a
integration. Data reduction involves reducing the dimensionality validity or legitimation typology specifically for mixed research
of the qualitative data (e.g., via exploratory thematic analysis, in Onwuegbuzie and Johnson (2004). It is important to note
memoing) and quantitative data (e.g., via descriptive statistics, that the legitimation process might include additional data col-
exploratory factor analysis, cluster analysis). Data display, in- lection, data analysis, and/or data interpretation until as many
volves describing pictorially the qualitative data (e.g., matrices, rival explanations as possible have been reduced or eliminated.
charts, graphs, networks, lists, rubrics, and Venn diagrams) and
The Future of Mixed Methods
quantitative data (e.g., tables, graphs). This is followed (option-
Research in Education
ally) by the data transformation stage, wherein quantitative data
are converted into narrative data that can be analyzed qualitatively Mixed research actually has a long history in research practice b
(i.e., qualitized; Tashakkori & Teddlie, 1998) and/or qualitative cause practicing researchers frequently ignore what is written
data are converted into numerical codes that can be represented methodologists when they feel a mixed approach will best he
statistically (i.e., quantitized; Tashakkori & Teddlie, 1998). Data them to answer their research questions. It is time that meth
correlation involves the quantitative data being correlated with the ologists catch up with practicing researchers! It is now time t
qualitized data or the qualitative data being correlated with the all researchers and research methodologists formally recognize t
quantitized data. This is followed by data consolidation, wherein third research paradigm and begin systematically writing about
both quantitative and qualitative data are combined to create new and using it. In general we recommend contingency theory f
or consolidated variables or data sets. The next stage, data com- research approach selection, which accepts that quantitativ
parison involves comparing data from the qualitative and quan- qualitative, and mixed research are all superior under different c

2|1 EDUCATIONAL RESEARCHER

This content downloaded from

103.21.125.84 on Tue, 27 Dec 2022 09:07:57 UTC
All use subject to https://about.jstor.org/terms
 IF

Note. Circles represent steps (1-8) in the mixed research process; rectangles represent steps in the mixed data analysis process; diamonds represent components.

FIGURE 3. Mixed research process model.

cumstances and it is the researcher's task to examine the specific well as in our sister disciplines in the social and behavioral sci-
contingencies and make the decision about which research ap- ences, as the third major research paradigm.
proach, or which combination of approaches, should be used in As noted by Sechrest and Sidana (1995), growth in the mixed
a specific study. In this article we have outlined the philosophy of methods (i.e., pragmatist) movement has the potential to reduce
pragmatism, we have described mixed research and provided spe- some of the problems associated with singular methods. By uti-
cific mixed-model and mixed-method designs, and we have dis- lizing quantitative and qualitative techniques within the same
cussed the fundamental principle of mixed research and provided framework, mixed methods research can incorporate the
tables of quantitative and qualitative research strengths and weak- strengths of both methodologies. Most importantly, investiga-
nesses to help apply the principle. Also, we have provided a mixed tors who conduct mixed methods research are more likely to se-
methods process model to help readers visualize the process. We lect methods and approaches with respect to their underlying
hope we have made the case that mixed methods research is here research questions, rather than with regard to some preconceived
to stay and that it should be widely recognized in education, as biases about which research paradigm should have hegemony in

OCTOBER 2004 |

This content downloaded from

103.21.125.84 on Tue, 27 Dec 2022 09:07:57 UTC
All use subject to https://about.jstor.org/terms
social science research. By narrowing the divide between quanti- 9 For additional mixed-method designs, see Creswell, Plan
tative and qualitative researchers, mixed methods research has a Guttmann, and Hanson, 2003; Maxwell and Loomis, 2003.
great potential to promote a shared responsibility in the quest for 10 Here is the etiology of Figure 1: As far as we know, Patt

attaining accountability for educational quality. The time has first listed 6 of the mixed model designs (Designs 1, 2, 3, 5, 6
come for mixed methods research. Then Tashakkori and Teddlie (1998) built on this by adding
signs (Designs 4 and 7) that were left out by Patton and the
NOTES some labels to better fit their thinking (e.g., they introduced
mixed model). Finally, in its present form, we first used (in a
1 Thomas Kuhn (1962) popularized the idea of a paradigm. Later,
conference paper) the full set of eight designs identified by T
when he was asked to explain more precisely what he meant by the term,
and Teddlie (1998) while changing some labels to better fit our
he pointed out that it was a general concept and that it included a group
tualization. The term monomethods probably originated in
of researchers having a common education and an agreement on "exem-
and Fiske (1959).
plars" of high quality research or thinking (Kuhn, 1977). In this article,
11 In developing Figure 2, we were probably most influe
by research paradigm we mean a set of beliefs, values, and assumptions
Morgan (1998), Morse (1991), and Tashakkori and Teddlie (1
that a community of researchers has in common regarding the nature
eral of the designs shown in the figure were introduced by Mo
and conduct of research. The beliefs include, but are not limited to, on-
REFERENCES
tological beliefs, epistemological beliefs, axiological beliefs, aesthetic be-
liefs, and methodological beliefs. In short, as we use the term, a research
Ayer, A. J. (1959). Logicalpositivism. New York: The Free Pre
paradigm refers to a research culture. We will be arguing that there is
Biesta, G. J. J., & Burbules, N. C. (2003). Pragmatism and ed
now a trilogy of major research paradigms: qualitative research, quanti-
research. Lanham, MD: Rowman and Littlefield.
tative research, and mixed methods research.
Brewer, J., & Hunter, A. (1989). Multimethod research: A synt
2 Campbell modified his view of qualitative research over time. For
styles. Newbury Park, CA: Sage.
example, based on criticisms by qualitative and case study researchers
Bryman, A. (1984). The debate about quantitative and qualita
of his term "one-shot case study" (which, unfortunately, is still used in
search: A question of method or epistemology? British Journa
several educational research books), Campbell changed this design
ciology, 35, 78-92.
name to the one-group posttest-only design; he madeCampbell,
this change as & Fiske, D. W. (1959). Convergent and discr
D. T.,
part of his endorsement of case study research as an important research
validation by the multitrait-multimethod matrix. Psychologi
approach (e.g., see Campbell's introduction to Yin's case study research
letin, 56, 81-105.
book: Yin, 1984).
Campbell, D. T., & Stanley, J. C. (1963). Experimental and
3 We do not mean to imply that there is anything inherently wrongdesignsfor research. Chicago, IL: Rand McNally.
experimental
with taking an extreme intellectual position. Most of theConstas,
great thinkers
M. A. (1992). Qualitative data analysis as a public ev
in the history of philosophy and science (including social and behavioral of category development procedures. America
documentation
science) were "extreme" for their times. Also, both qualitative and Research
cational quan- Journal, 29, 253-266.
titative philosophies continue to be highly useful (i.e., both have many
Creswell, J. W. (1994). Research design: Qualitative and quantita
advantages when used in their pure forms). proaches. Thousand Oaks, CA: Sage.
4 Positivism is a poor choice for labeling quantitative researchers
Creswell, today
J.W. (2003). Research design: Qualitative, quantita
because positivism has long been replaced by newer philosophies of sci-
mixed approaches. Thousand Oaks, CA: Sage.
ence (Yu, 2003). The term is more of a straw man (easilyCreswell,
knocked J.down)
W, Plano Clark, V. L., Guttmann, M. L., & Hanso
for attack than standing for any actual practicing researchers. A term
(2003). Advanced mixed methods research design. In A.Tas
that better represents today's practicing quantitative researchers
and C. is post- (Eds.), Handbook of mixed methods in social
Teddlie
positivism (Phillips & Burbules, 2000). havioral research (pp. 209-240). Thousand Oaks, CA: Sage.
s Both of the authors of the current article prefer thede
label mixed
Jong, re-(2003). Causal and functional explanations. The
H.L.
search or integrative research rather than mixed methods research. The alter-
Psychology, 13(3), 291-317.
native labels are broader, more inclusive, and more clearly
de paradigmatic.
Waal, C. (2001). On Peirce. Belmont, CA: Wadsworth.
We chose to use the term mixed methods in this article because of its cur-
Dewey, J. (1948, 1920). Reconstruction in philosophy. Boston, M
rent popularity. con Press.
6 Here is a practical definition of science from an educational
Dzurec,research
L. C., & Abraham, J. L. (1993). The nature of inquir
textbook (Johnson & Christensen, 2004) that should being inclusive of
quantitative and qualitative research. Advances in Nursing
quantitative and qualitative research: ". . . the root of the 16,
word science
73-79.
is the Latin scientia, which simply means 'knowledge.' We define J.
Greene, science
C., Caracelli, V. J., & Graham, W. F. (1989). Towar
in this book in a way that is inclusive of the different approaches to framework
ceptual educa- for mixed-method evaluation designs.
tional research. We define science as an approach for the generation of
tional Evaluation and Policy Analysis, 11, 255-274.
knowledge that places high regard for empirical data andGuba,
follows
E. certain
G. (1990). The alternative paradigm dialog. In E. G
norms and practices that develop over time because of their usefulness.
(Ed.), Theparadigm dialog (pp. 17-27). Newbury Park, CA: S
... The ultimate objective of most social, behavioral, and educational
Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation eva
research is improvement of the world or social betterment."Newbury Park, CA: Sage.
7 This is a very interesting empirical question that deserves moreL.T.
Hoshmand, at- (2003). Can lessons of history and logical ana
tention in the literature.
sure progress in psychological science? Theory and Psycholo
8 Note that Shadish, Cook, and Campbell (2002) have 39-44.
attempted to
move quantitative research away from this traditional "menu"
Howe, K.approach.
R. (1988). Against the quantitative-qualitative incompatabil-
In this latest edition of Campbell and Stanley (1963), there
ityisthesis,
increased
or, Dogmas die hard. Educational Researcher, 17, 10-16.
focus on understanding how to construct or create a research design
Howe, K. that
R. (1992). Getting over the quantitative-qualitative debate.
fits a particular situation. American Journal ofEducation, 100, 236-256.

2|[ EDUCATIONAL RESEARCHER

This content downloaded from

103.21.125.84 on Tue, 27 Dec 2022 09:07:57 UTC
All use subject to https://about.jstor.org/terms
James, W. (1995, 1907). Pragmatism. New York: Dover. Onwuegbuzie, A. J., & Johnson, R. B. (2004, April). Validity issues in
Johnson, R. B., & Christensen, L. B. (2004). Educational research: mixed methods research. Paper presented at the annual meeting of the
Quantitative, qualitative, and mixed approaches. Boston, MA: Allyn American Educational Research Association, San Diego, CA.
and Bacon. Onwuegbuzie, A. J., & Leech, N. L. (2004a). On becoming a pragmatic
Johnson, R. B., Meeker, K. M., Loomis, E. J., & Onwuegbuzie, A. J. researcher: The importance of combining quantitative and qualitative
(2004, April). Development of thephilosophical and methodological be- research methodologies. Manuscript submitted for publication.
liefi inventory. Paper presented at the annual meeting of the Ameri- Onwuegbuzie, A. J., & Leech, N. L. (2004b, February). Enhancing the
can Educational Research Association, San Diego, CA. interpretation ofsignificantfindings: The role of mixed methods research.
Johnson, R. B., & Turner, L. A. (2003). Data collection strategies in Paper presented at the annual meeting of the Eastern Educational Re-
mixed methods research. In A.Tashakkori, and C. Teddlie (Eds.), search Association, Clearwater, FL.
Handbook of mixed methods in social and behavioral research (pp. Onwuegbuzie, A. J., & Leech, N. L. (in press). Taking the "Q" out of
297-319). Thousand Oaks, CA: Sage. research: Teaching research methodology courses without the divide
Kuhn, T. S. (1962). The structure ofscientific revolutions. Chicago, IL: between quantitative and qualitative paradigms. Quality e& Quantity:
University of Chicago Press. InternationalJournal ofMethodology.
Kuhn, T. S. (1977). The essential tension: Selected studies in scientific tra- Onwuegbuzie, A. J., & Teddlie, C. (2003). A framework for analyzing
data in mixed methods research. In A. Tashakkori & C. Teddlie
dition and change. Chicago, IL: University of Chicago Press.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly (Eds.), Handbook of mixed methods in social and behavioral research
Hills, CA: Sage. (pp. 351-383). Thousand Oaks, CA: Sage.
Lincoln, Y. S., & Guba, E. G. (2000). Paradigmatic controversies, con- Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd
tradictions, and emerging confluences. In N. K. Denzin, and Y. S. ed.). Newbury Park, CA: Sage.
Lincoln (Eds.), Handbook ofqualitative research (pp. 163-188). Thou- Peirce, C. S. (1878). How to make our ideas clear. Popular Science
sand Oaks, CA: Sage. Monthly, 12, 286-302.
Maxcy, S. J. (2003). Pragmatic threads in mixed methods research in Phillips, D. C. (2004, April). Is mixed methods research an epistemologi-
cal oxymoron? Paper presented at the annual meeting of the American
the social sciences: The search for multiple modes of inquiry and the
end of the philosophy of formalism. In A.Tashakkori & C. Teddlie Educational Research Association, San Diego, CA.
Phillips, D. C., & Burbules, N. C. (2000). Postpositivism and educa-
(Eds.), Handbook of mixed methods in social and behavioral research
tional research. New York: Rowman & Littlefield.
(pp. 51-89). Thousand Oaks, CA: Sage.
Popper, K. R. (1959). The logic ofscientific discovery. New York: Routledge.
Maxwell, J. A., & Loomis, D. M. (2003). Mixed methods design: An
Reichardt, C. S., & Cook, T. D. (1979). Beyond qualitative versus
alternative approach. In A.Tashakkori, and C. Teddlie (Eds.), Hand-
quantitative methods. In T. D. Cook & C. S. Reichardt (Eds.),
book ofmixed methods in social and behavioral research (pp. 241-272).
Qualitative and quantitative methods in evaluation research (pp. 7-32).
Thousand Oaks, CA: Sage.
Newbury Park, CA: Sage.
Maxwell, S. E., & Delaney, H. D. (2004). Designing experiments and
Reichardt, S. S., & Rallis, S. F. (1994). Qualitative and quantitative in-
analyzing data. Mahwah, NJ: Lawrence Erlbaum.
quiries are not incompatible: A call for a new partnership. In C. S.
Menand, L. (1997). Pragmatism: A reader. New York: Vintage.
Reichardt & S. F. Rallis (Eds.), The qualitative-quantitative debate:
Mertens, D. M. (2003). Mixed methods and the politics of human re-
Newperspectives (pp. 85-91). San Francisco, CA: Jossey-Bass.
search: The transformative-emancipatory perspective. In A. Tashakkori
Rescher, N. (2000). Realistic pragmatism: An introduction to pragmatic
& C. Teddlie (Eds.), Handbook of mixed methods in social and be-
philosophy. Albany, NY: State University of New York.
havioral research (pp. 135-164). Thousand Oaks, CA: Sage.
Rorty, R. (2000). Pragmatism. InternationalJournal of Psychoanalysis,
Morgan, D. L. (1998). Practical strategies for combining qualitative and 81,819-823.
quantitative methods: Applications to health research. Qualitative
Sandelowski, M. (1986). The problem of rigor in qualitative research.
Health Research, 3, 362-376.
Advances in Nursing Science, 8(3), 27-37.
Morse, J. M. (1991). Approaches to qualitative-quantitative method-
Schrag, F. (1992). In defense of positivist research paradigms, Educa-
ological triangulation. Nursing Research, 40, 120-123. tionalResearcher, 21(5), 5-8.
Murphy, J. P. (1990). Pragmatism: From Peirce to Davidson. Boulder,
Schwandt, T. A. (2000). Three epistemological stances for qualitative
CO: Westview.
inquiry: Interpretivism, hermeneutics, and social constructionism. In
Nagel, T. (1986). The view fom nowhere. New York: Oxford Univer-
N. K. Denzin & Y. S. Lincoln, Handbook of qualitative research (pp.
sity Press.
189-213). Thousand Oaks, CA: Sage.
Newman, I., & Benz, C. R. (1998). Qualitative-quantitative research Sechrest, L., & Sidana, S. (1995). Quantitative and qualitative methods:
methodology: Exploring the interactive continuum. Carbondale, IL: Is there an alternative? Evaluation and Program Planning, 18, 77-87.
Southern Illinois University Press. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimen-
Onwuegbuzie, A. J. (2000, November). Validity and qualitative re- tal and quasi-experimental designs for generalized causal inference.
search: An Oxymoron? Paper presented at the annual meeting of the Boston, MA: Houghton Mifflin.
Association for the Advancement of Educational Research (AAER), Sieber, S. D. (1973). The integration of fieldwork and survey methods.
Ponte Vedra, Florida.
American Journal of Sociology, 73, 1335-1359.
Onwuegbuzie, A. J. (2002). Positivists, post-positivists, post-structuralists, Smith, J. K. (1983). Quantitative versus qualitative research: An at-
and post-modernists: Why can't we all get along? Towards a framework tempt to clarify the issue. Educational Researcher, 12, 6-13.
for unifying research paradigms. Education, 122(3), 518-530. Smith, J. K. (1984). The problem of criteria for judging interpretive in-
Onwuegbuzie, A. J. (2003). Expanding the framework of internal and quiry. Educational Evaluation and Policy Analysis, 6, 379-391.
external validity in quantitative research. Research in the Schools, 10, Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining
71-90.
qualitative and quantitative approaches. Applied Social Research Meth-
Onwuegbuzie, A. J., Jiao, Q. G., & Bostick, S. . (2004). Library anx-
ods Series (Vol. 46). Thousand Oaks, CA: Sage.
iety: Theory, research, and applications. Lanham, MD: Scarecrow
Tashakkori, A., & Teddlie, C. (Eds.). (2003). Handbook ofmixed meth-
Press.
ods in social and behavioral research. Thousand Oaks, CA: Sage.

OCTOBER 2004 |[

This content downloaded from

103.21.125.84 on Tue, 27 Dec 2022 09:07:57 UTC
All use subject to https://about.jstor.org/terms
Watson, W. (1990). Types of pluralism. The Monist, 73(3), 350-367. ANTHONY J. ONWUEGBUZIE is an Associate Profess
Yin, R. K. (1984). Case study research: Design and methods. Thousand ment of Educational Measurement and Research, Univers
Oaks, CA: Sage. Florida, 4202 East Fowler Avenue, EDU 162, Tampa, FL
Yu, C. H. (2003). Misconceived relationships between logicalpositivism [email protected]. His areas of specialization are di
and quantitative research. Research Methods Forum [On-line]. Re- and under-served populations (e.g., minorities and juvenile
trieved September 2, 2004 from http://www.aom.pace.edu/rmd/ and methodological topics in the areas of quantitative, qual
2002forum.html. mixed methods.

AUTHORS

R. BURKE JOHNSON is a Professor, University of South Alabama, Col- Manuscript received October 14, 20
lege of Education, BSET, 3700 UCOM, Mobile, AL 36688; bjohnson@ Revisions received March I and April 30,
usouthal.edu. His area of specialization is research methodology. Accepted May 12, 2004

Just off the press!

"Teachers Matter: Evidence From

Value-Added Assessments"
(Research Points, Summer 2004)

Read it at
www.aera.net

2| EDUCATIONAL RESEARCHER

This content downloaded from

103.21.125.84 on Tue, 27 Dec 2022 09:07:57 UTC
All use subject to https://about.jstor.org/terms