Southern Cross University

ePublications@SCU
Southern Cross Business School

2009

A sequential mixed model research design: design,

analytical and display issues
Roslyn Cameron
Southern Cross University

Publication details
Cameron, R 2009, 'A sequential mixed model research design: design, analytical and display issues', International Journal of Multiple
Research Approaches , vol. 3, no.2.
The abstract and pdf of the published article reproduced in ePublications@SCU with the permission of e-Content Management
The publisher versions of this article is available at http://mra.e-contentmanagement.com/archives/vol/3/issue/2/

ePublications@SCU is an electronic repository administered by Southern Cross University Library. Its goal is to capture and preserve the intellectual
output of Southern Cross University authors and researchers, and to increase visibility and impact through open access to researchers around the
world. For further information please contact [email protected].
 Copyright © eContent Management Pty Ltd. International Journal of Multiple Research Approaches (2009) 3: 140–152.

A sequential mixed model

research design: Design,
analytical and display issues
R OSLYN CAMERON
Lecturer, School of Commerce and Management, Southern Cross University – Tweed Gold Coast
Campus, Tweed Heads NSW, Australia

ABSTRACT
A new era in research methods is emerging and has been quietly lauded by several emerging authori-
ties in the field of mixed methods research. Like the mythology of the phoenix, mixed methods
research has arisen out of the ashes of the paradigm wars to become the third methodological move-
ment (Cameron & Miller 2007). The fields of applied social science and evaluation are among
those which have shown the greatest popularity and uptake of mixed methods research designs. This
article provides a brief overview of the rise of mixed methods research, its usage in business and
management fields and its relationship to the philosophy of pragmatism. Typologies of mixed meth-
ods research designs are discussed and a case study of a sequential mixed model research design in the
human resource development (HRD) field is presented. Issues related to design, analytical processes
and display arising from utilising this particular mixed method research design are discussed. As a
consequence, the article contains several Tables and Figures which exemplify display options that
may assist those researchers who are considering utilising a mixed method research design.

Keywords: mixed methods, qualitative research, quantitative research, research designs, triangulation

THE PARADIGM WARS can be used in specific situations. In direct oppo-

T he debates surrounding research paradigms

have a long history and were particularly
active in the 1980s. Some commentaries on the
sition to the ‘purists’ are the pragmatists who
argued against a false dichotomy between the
qualitative and quantitative research paradigms
debate contend that the struggle for primacy of and advocate for the efficient use of both
one paradigm over others is irrelevant as each approaches.
paradigm is an alternate offering with its own Proponents of mixed methods research have
merits (Guba 1990: 27). Creswell (1994: 176) been linked to those whom identify with the
identifies several schools of thought in the para- pragmatic paradigm. Historically, pragmatism
digm debate or so-called ‘paradigm wars’. At one can be traced to an early period from 1860–1930
end of the debate are the ‘purists’ who assert para- and the neopragmatic era from 1960 to present
digms and methods should not be mixed. Anoth- (Maxcy 2003). Many mixed methods researchers
er school of thought is identified as the and theorists draw strong associations with mixed
‘situationalists’ who contend that certain methods methodology and pragmatism (Bazeley 2003;

140 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 2, August 2009
 A sequential mixed model research design: Design, analytical and display issues

Greene & Caracelli 1997; Johnson & Onwueg- approach and represent fundamental differences
buzie 2004; Maxcy 2003; Tashakkori & Teddlie between them:
2003). Johnson and Onwuegbuzie (2004: 17)
They are ideal types or idealized, simplified
summarise the philosophical position of mixed
models of more complex arguments. In prac-
method researchers when they say:
tice, few social researchers agree with all parts
We agree with others in the mixed methods of an approach. Often they mix elements
research movement that consideration and dis- from each.
cussion of pragmatism by research methodolo-
Caulley (1994: 4) asserts that positivist
gists and empirical researchers will be
inquiry takes a realist position and involves a
productive because it offers an immediate and
dualist epistemology which requires separation
useful middle position philosophically and
of the researcher to the researched. Postpositivist
methodologically; it offers a practical and out-
inquiry takes a relativist position and allows for
come-orientated method of inquiry that is
multiple constructions of reality and a monist
based on action and leads, iteratively, to fur-
epistemology where the researcher and the
ther action and the elimination of doubt; and
researched interact and are bound together. The
it offers a method for selecting methodological
differences between qualitative and quantitative
mixes that can help researchers better answer
research approaches can be explored more fully
many of their research questions.
in King, Keohane and Verba (1994) and Brady
Pragmatism has a strong philosophical and Collier (2004). Brady and Collier (2004: 5)
foothold in the mixed methods or methodologi- suggest there is a lot of potential for mutual
cal pluralism camps. This paper now looks more learning between qualitative and quantitative
closely at the mixing of qualitative and quantita- researchers:
tive research methods.
…a meaningful discussion of methodology
There exists a complex array of research
must be grounded in the premise that
methodologies utilised that can incorporate
strengths and weaknesses are to be found in
interdisciplinary, transdisciplinary and multidis-
both the qualitative and quantitative
ciplinary approaches. Due to the limitations of
approaches. Regarding the weaknesses…quali-
this paper a detailed discussion of this level of
tative researchers are perhaps ‘handicapped by
complexity is not possible. Therefore, a starting
a lack of quantification and small numbers of
point for the paper will be a brief overview of
observations’, whereas quantitative researchers
three main approaches used in the social sciences
may sometimes suffer from ‘procrustean quan-
(Neuman 2006) and the more simplistic
tification and a jumble of dissimilar cases’.
dichotomy of research methodologies that are
often categorised under two approaches: quanti- Brady and Collier (2004: 195) go on to state
tative (positivist) and qualitative (postpositivist), that the intellectual vitality of the two traditions
each with unique strengths and limitations. Neu- and associated divergences between them, has
man (2006: 80) refers to three ideal types in his produced a ‘major new methodological dialogue’.
discussion of the three approaches found in the Neuman (2006: 177) provides the following
social sciences. The three approaches are: posi- argument in terms of these two methodological
tivist social science; interpretive social science; approaches and argues against the rigid dichoto-
and critical social science. Neuman (2006: 80) my between the two:
acknowledges the levels of complexity in research
choices and approaches and uses this typology as The qualitative and quantitative distinction is
a means to simplify a detailed discussion of each often overdrawn and presented as a rigid

Volume 3, Issue 2, August 2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES 141
 Roslyn Cameron

dichotomy. Too often, adherents of one style THE RISE OF MIXED METHODS
of social research judge the other style on the RESEARCH DESIGNS
basis of the assumptions and standards of their Mixed method research has a short history as an
own style….The well-versed prudent social identifiable methodological movement which can
researcher understands and appreciates each be traced to the early 1980s and has been described
style on its own terms and recognizes the as a ‘quiet’ revolution due to its focus of resolving
strengths and limitations of each. The ultimate tensions between the qualitative and quantitative
goal of developing a better understanding and methodological movements (Tashakkori & Teddlie
explanation of the social world comes from an 2003: 697). Mixed methodologies is an emerging
appreciation of what each has to offer. area with a growing amount of interest across sev-
eral discipline areas and has been particularly pop-
Guba and Lincoln (2005: 200) discuss how ular in the areas of applied social research and
positivists and postpositivists can be reconciled evaluation (Bazeley 2003). Tashakkori and Teddlie
through mixed methods and can be: acknowledge the effects of the residue from the
paradigmatic wars but are very positive in acknowl-
…retrofitted to each other in ways that make
edging the signs of change (2003: 699):
the simultaneous practice of both possible. We
have argued that at the paradigmatic, or …the mixed methods research movement is a
philosophical, level, commensurability positive reaction to this split personality and to
between positivist and postpositivist world the excesses of both the QUAN (quantitative)
views is not possible, but that within each par- and QUAL (qualitative) camps. We believe
adigm, mixed methodologies (strategies) may that mixed methods will eventually pave the
make perfect sense. way for more commonality in research lan-
guage that will benefit both the QUAL and
In particular, qualitative research can be a site
QUAN camps.
of multiple practices where there is no ‘distinct
set of methods or practices that are entirely its Several authorities have been emerging as
own’ (Denzin & Lincoln 2005: 7) and no one mixed methodologist researchers and theorists
method or practice is rated more highly than (Creswell 2003; Greene & Caracelli 1997;
another. Nelson (1992) puts forward the follow- Mertens 2005; Mingers & Gill 1997; Tashakkori
ing argument in terms of a qualitative approach & Teddlie 2003) along with the emergence of
to research: mixed methods chapters in recent research text
books (Creswell 2003; Mertens 2005; McMillan
Qualitative research is an interdisciplinary, & Schumacher 2006). In addition to this a hand-
transdisciplinary, and sometimes counterdisci- book of mixed methods has been published
plinary field. It crosscuts the humanities and (Tashakkori & Teddlie 2003) and a number of
the social and physical sciences. Qualitative textbooks have recently appeared (Bergman
research is many things at the same time. It is 2008; Creswell & Plano Clark 2007; Greene
multiparadigmatic in focus. Its practitioners 2007; Plano Clark & Creswell 2007; Rihoux &
are sensitive to the value of the multimethod Grimm 2006; Teddlie & Tashakkori 2008;
approach. They are committed to the natura- Thomas 2003).
listic perspective and to the interpretive under-
standing of human experience. At the same In the field of management research, Mingers
time, the field is inherently political and and Gill (1997) have been strong advocates for
shaped by multiple ethical and political posi- multimethodology as has Bazeley (2003).
tions (cited in Denzin & Lincoln 2005: 7). Mingers (1997: 9) argues for a strong pluralism

142 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 2, August 2009
 A sequential mixed model research design: Design, analytical and display issues

or multimethodology which takes the position an informed debate between quantitative and
that most if not all intervention situations would qualitative empirical traditions:
be dealt with more effectively with a blend of
…generally speaking the explicitly compara-
methods from different paradigms. Buchanan
tive design is gaining momentum. The choice
and Bryman (2007) draw attention to the con-
of such a strategy often reflects the intention
textual issues surrounding the field of organisa-
of scholars to meet two apparently contradic-
tional research. They see the field as being
tory goals. On the one hand, one seeks to
impacted by three major trends which are identi-
gather in-depth insight in the different cases
fied as:
and capture the complexity of the cases – to
• widening boundaries;
gain intimacy with the cases (Ragin & Becker,
• a multiparadigmatic profile; and
1992). On the other hand, one still wishes to
• methodological inventiveness.
produce some level of generalization (Ragin,
1987). Indeed, in empirical social science,
These trends are impacting on organisational
both case-orientated work and techniques that
research as it is a field where many disciplines
allow one to generalize (typically quantitative,
meet. Organisational research includes such dis-
i.e. statistical, techniques) are useful.
ciplines as human resources, economics, social
psychology, public policy, finance, marketing just The metaphor of the phoenix has been used to
to name a few. Management science researchers illustrate the emergence of the third methodolog-
need to keep abreast of these trends as do man- ical movement that has arisen from the ashes of
agement educators. Buchanan and Bryman the paradigm wars (Cameron & Miller 2007).
(2007: 486) sum this up when they say: Mingers (2003) refers to the ceasefire of the para-
digm wars being announced whilst Johnson and
The paradigm wars of the 1980s have thus
Onwuegbuzie (2004: 14) state very clearly,
turned to paradigm soup, and organisational
‘Mixed methods research: A research paradigm
research today reflects the paradigm diversity
whose time has come’. Nonetheless, it is a devel-
of the social sciences in general. It is not sur-
oping and evolving field and recent studies of the
prising that this epistemological eclecticism
use of mixed methods in the field of information
has involved the development of novel termi-
systems (Mingers 2003), counselling (Hanson,
nology; innovative research methods; non tra-
Creswell, Clark, Petska & Creswell 2005), man-
ditional forms of evidence; and fresh approaches
agement disciplines (Cameron 2008; Hurmerin-
to conceptualization, analysis, and theory
ta-Peltomaki & Nummela 2006), and qualitative
building.
research conducted in Switzerland (Eberle &
Another method gaining momentum in the Elliker 2005) is providing empirical evidence of
social sciences is that of Qualitative Comparative the extent of utilisation of mixed methods in con-
Analysis (QCA). Charles Ragin (1987, 1994) temporary research.
developed a set of systematic case analysis tech-
niques which were born out of his frustrations as MIXED METHOD TYPOLOGIES
a comparative sociologist with the qualitative/ Mixed methods research designs use both quanti-
quantitative split in comparative social science. tative and qualitative approaches in a single
Since then comparative design and analysis has research project to gather or analyse data and sev-
gained momentum. Rihoux (2006: 680) has eral mixed method theorists have developed
mapped recent advances and challenges for QCA mixed method typologies (Creswell 2003;
and related systematic comparative methods. and Creswell & Plano Clark 2007; Greene & Cara-
states there have been new attempts to engage in celli 1997; Mertens 2005; Miles & Huberman

Volume 3, Issue 2, August 2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES 143
 Roslyn Cameron

1994; Morgan 1998; Morse 2003; Tashakkori & addition to this distinction Mertens (2005: 292)
Teddlie 2003). adds parallel and sequential data collection forms
Typologies are the study or systematic classifi- and defines these as:
cation of types that have characteristics or traits
Parallel Form: Concurrent mixed-methods/-
in common and form part of models and theo-
model designs in which two types of data are
ries. Neuman (2006: 55) defines typologies as a
collected and analysed.
way to classify theoretical concepts which is creat-
ed by ‘cross-classifying or combining two or more Sequential Form: One type of data provides a
simple concepts to form a set of interrelated sub- basis for collection of another type of data.
types’. Typologies are used by theorists to assist
them in organising abstract and complex con- Caracelli and Greene (1997) have published
cepts. The mixed method typologies developed extensively on mixed methods in evaluation
by Mertens (2005), Caracelli and Greene (1997), research and have developed a typology of mixed
Creswell and Plano Clark (2007) and Tashakkori methods designs that include three component
and Teddlie (2003) will now be presented. designs and four integrated designs. Table 1
Mertens (2005) has mapped the definitions depicts these in tabular form.
and characteristics of mixed methods and mixed Creswell and Plano Clark (2007) have built on
models designs. Both utilise both qualitative and Creswell’s earlier work in terms of mixed methods
quantitative features. However, Mertens (2005: research designs and have developed a four type
292) distinguishes between mixed method design typology. These four major mixed methods
that uses both methods to answer a research ques- research design types are classified using cate-
tion in a single study as compared to mixed gories associated with variants, timing, weighting
model designs that are part of a larger research and mix. The four designs are: triangulation;
program. These are designed as complementary embedded; explanatory; and exploratory. Table 2
and inform several of the research questions, each summarises the Creswell and Plano Clark (2007)
having a different methodological approach. In mixed methods research designs typology.

TABLE 1: D ESIGNS FOR MIXED METHODS RESEARCH

Component Designs Integrated Designs

Triangulation Iterative
Different methods are used to assess the same Dynamic and ongoing interplay over time between
phenomenon toward convergence and increased the different methodologies associated with
validity. different paradigms. Spiral type design.
Complementary Embedded/nested
One dominant method type is enhanced or clarified One methodology located within another,
by results from another method type. interlocking inquiry characteristics in a framework
of creative tension.
Expansion Holistic
Inquiry paradigms frame different methods that are Highlight the necessary interdependence of different
used for distinct inquiry components. The results methodologies for understanding complex
being presented side-by-side. phenomena fully.
Transformative
Give primacy to the value-based and action-
orientated dimensions of different inquiry traditions.
Mix the value commitments of different traditions
for better representation of multiple interests.
Source: Caracelli & Greene (1997: 23)

144 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 2, August 2009
 A sequential mixed model research design: Design, analytical and display issues

TABLE 2: M AJOR MIXED METHOD DESIGN TYPES

Weighting/
Design Type Timing Mix Notation

Triangulation Concurrent: quantitative and Merge the data during QUAN + QUAL
qualitative at the same time interpretation or analysis
Embedded Concurrent and sequential Embed one type of data QUAN(qual)
within a larger design using Or
the other type of data QUAL(quan)
Explanatory Sequential: Quantitative Connect the data between QUAN  qual
followed by qualitative the two phases
Exploratory Sequential: Qualitative Connect the data between QUAL  quan
followed by quantitative the two phases
Source: Adapted from Creswell & Plano Clark (2007: 85)

Tashakkori and Teddlie (2003) have developed designs of which the sequential mixed model
a very comprehensive typology of mixed methods design has been applied to this research. The
which results in six types of multistrand mixed methodologists also note the parallels between
designs. Mixed method designs involve the mix- this particular type and Creswell’s explanatory and
ing of the quantitative and qualitative approaches exploratory mixed method designs (Tashakkori &
only in the methods stage of a study. Mixed Teddlie 2003: 688).
model designs involve the mixing of the quantita- Mixed methods typologies and research
tive and qualitative approaches in several stages of designs are not without critics and McMillan and
a study. This results in six types of multistrand Schumacher (2006: 401) draw attention to both
mixed designs as depicted in Table 3. The authors the advantages and disadvantages of using mixed
of this typology assert that it is the multistrand methods. They list three disadvantages. The first
mixed methods designs which are the most inno- being the need of the researcher to be proficient
vative and widely used mixed method designs and competent in both qualitative and quantita-
(Tashakkori & Teddlie 2003: 685). Multistrand tive methods. The second disadvantage is the
designs use more than one methodology and are extensive data collection and resources needed to
characterised by three dimensions. They have sin- undertake a mixed method study. The last refers
gle or multiple approaches. They use two meth- to a tendency to use the mixed methods label lib-
ods to answer either exploratory or confirmatory erally to studies which only superficially mix
research inquires. Another dimension is the stages methods. Tashakkori and Teddlie (2003), Bazeley
of integration or the incorporation of both quali- (2003) and Earley (2007) have all attempted to
tative and quantitative data sets. The third dimen- address these issues through advocating for
sion is the procedures for linking the strands research education that explicitly covers mixed
either sequentially or concurrently. These dimen- methods in the research syllabus for novice
sions create six types of multistrand research researchers.

TABLE 3: T WO -D IMENSIONAL FRAMEWORK FOR CONCEPTUALIZING MULTI -STRAND MIXED DESIGNS

Procedure Mixed Method Mixed Model Study

Concurrent Concurrent mixed method design Concurrent mixed model design

Sequential Sequential mixed method design Sequential mixed model design
Conversion Conversion mixed method design Conversion mixed model design
Source: Tashakkori & Teddlie (2003: 687)

Volume 3, Issue 2, August 2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES 145
 Roslyn Cameron

Having outlined the emergence of mixed on program (model) modification, improvement

methods research designs and reviewed some of and program certification and adoption (McMil-
the key typologies, this article will now describe a lan & Schumacher 2006).
research study which utilised the sequential In Phase I of the research a quantitative Learn-
mixed model research design from the Tashakkori ing Survey and qualitative focus groups were con-
and Teddlie (2003) typology of designs. ducted. The Learning Survey was administered to
approximately 250 labour market program
A CASE STUDY OF MIXED METHODS (LMP) participants and was followed by a semi-
RESEARCH DESIGN nar that presented preliminary data to representa-
Recently completed research in the broader field tives of the participating organisations. During
of human resource development (HRD) will be this seminar focus groups were conducted. Phase
presented here as an example of the utilisation of I of the research resulted in the development of a
mixed methods research design. The research model to assist those in career and learning tran-
focused on the learning experiences of those dis- sitions. Phase II of the research encompasses the
advantaged in the labour market and was essen- development and formative evaluation of the
tially a qualitative exploratory study. The research model in the field and utilised a combination of
design is based on a sequential mixed model both qualitative and quantitative data collection
research design that has two phases and research methods. This research process was non-linear
design subtypes within each phase. Phase I and allowed for emergent themes to develop
includes an ex post facto design (retrospective which enabled the most appropriate method for
design) subtype whilst Phase II utilises a com- progressing the research to be chosen. Neuman
bined process and product evaluation design sub- (2006: 152) describes this process very succinctly:
type. Ex post facto research design is defined by
Rather than moving in a straight line, a non-
Cohen and Manion (1989: 176) as:
linear research path makes successive passes
…a method of teasing out possible antecedents through steps, sometimes moving backward
of events that have happened and cannot, and sideways before moving on…It can be
because of this fact, be engineered or manipu- highly effective for creating a feeling for the
lated by the investigator. whole, for grasping subtle shades of meaning,
for pulling together divergent information,
Ex post facto research designs are used in a
and for switching perspectives.
variety of discipline areas and literally means
‘after the fact’, hence the essentially retrospective Figure 1 depicts the overall research design and
nature of the research design. The Learning Sur- design subtypes of the research.
vey in Phase I of the research asked respondents The sequential mixed model design utilised in
to answer questions relating to current and past the research is adapted from Tashakkori and Ted-
learning related activities and skills. A combined dlies’ (2003) typology of multistrand mixed
process and product evaluation design was method research. Figure 2 represents the applica-
utilised in Phase II to evaluate the effectiveness of tion of this design to the research described in
the developed model in the field. The general aim this case study.
of a process evaluation model is to research the The mixed model design allows for the
extent the program (model) achieves its objectives research questions for the second strand (phase)
and products. The aim of a product evaluation of research to emerge from the inferences of the
design is to research the worth of the program first strand (phase) (Tashakkori & Teddlie 2003:
(model) as reflected by process and outcomes. 687). The first strand is usually exploratory and
Combined these two approaches can help decide data collection, analysis and inferences are in one

146 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 2, August 2009
 A sequential mixed model research design: Design, analytical and display issues

Sequential Mixed Model Design approach, as was the case with the Learning Sur-
vey in Phase I of the research which took a quan-
titative approach. The second strand (phase) is
often confirmatory and the new data, its analysis
Phase I Phase II
and inferences are in the other approach, as was
Ex post facto Formative the case with the qualitative approach undertaken
design subtype evaluation design
subtype
in the model development, its testing in the field
and formative evaluation in Phase II of this
Survey Field study
research. The resulting final meta-inferences are
Co mbined made as either confirmatory or disconfirmatory of
Focus groups Process and the inferences made at the end of the two strands
product evaluation
design (phases) (Tashakkori & Teddlie 2003: 688).
Inferences, as used in mixed methods research,
refers to the inferences made from what is studied,
F IGURE 1: R ESEARCH DESIGN OF THE STUDY as opposed to the results of a study. Mixed methods

Phase I
Phase II
Purpose/question Purpose

Quantitative Data Model

Collection Development
Learning Survey & Field Test

Quantitative Data Mixed Method

Analysis Data Collection
Learning Survey

Qual/Quan Data
Data Presentation Analysis
Qualitative Focus
Groups

Inference
:
Inference

Meta-
Inference
Confirmatory

F IGURE 2: S EQUENTIAL MIXED METHOD DESIGN

Source: Adapted from Tashakkori & Teddlie (2003: 688)

Volume 3, Issue 2, August 2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES 147
 Roslyn Cameron

lead to multiple inferences that can either comple- Learning Survey in Phase I inferred a large dis-
ment or confirm each other. In mixed methods crepancy between what the previous research and
research inferences are obtained from each strand literature had found and what was anticipated,
of a mixed method study and are distinguished then any number of research strategies could have
from meta-inferences which are obtained by inte- been employed. In retrospect, had this occurred,
grating the initial inferences. Teddlie and then other mixed method typologies would have
Tashakkori (2003: 35) define inferences as: been considered.
In terms of data alignment and display issues
…an umbrella term to refer to a final outcome
the second phase of the study provided some
of a study. The outcome may consist of a con-
challenges. The second phase involved the testing
clusion about, an understanding of, or an
of the developed model in the field and its evalu-
explanation for an event, (a) behaviour, (a)
ation utilising a combined process product evalu-
relationship, or a case (e.g. in qualitative
ation design. What resulted was a complex blend
research). We use the term ‘inference’ as a
of mixed methods data collection across three
mixed methods term because it may take a
data collection points. Figure 3 provides a visual
variety of meanings ranging between a purely
depiction of the complexity of the mixed meth-
quantitative connotation to a purely qualita-
ods used in this phase and an example of how to
tive connotation.
display this aspect of the research.
In the case of this research the meta-inferences Another dimension of complexity was added
were confirmatory and resulted in the develop- by the use of one of the LMP trainers becoming
ment of a conceptual framework that consists of a a co-researcher or internal evaluator. This
continuum of models of Recognition of Prior enabled a more comprehensive level of triangula-
Learning (RPL), a hierarchy of recognition and a tion to be achieved. For Phase II of the research
model to assist those in career and learning transi- data, investigator, combined levels, time and
tions. methodological triangulation were achieved.
Data triangulation was attained through the
DESIGN, ANALYTICAL AND DISPLAY mixed method data collection techniques from
ISSUES ARISING FROM THE STUDY each phase of the research. Investigator triangula-
The two phase design did not in itself present any tion was achieved through the use of both inter-
major issues. In fact, it allowed for a much need- nal and external evaluators in the formative
ed theoretical framework for the organisation and evaluation of the model in the field. Combined
flow of the research processes. Due to the emer- levels of triangulation were attained through the
gent and cyclical nature of the research study the incorporation of more than one level of analysis
sequential mixed model research design was not (refer to Figure 4). Individual and interactive
fully applied until the end of the first phase of the group responses were collected from the LMP
research. This was due to the fact that the results participants along with organisational responses
or inference of the first phase would, to a large during Phase II. Time triangulation was achieved
degree, determine the research activities and through a longitudinal approach during Phase II,
directions that would follow that phase. This is as opposed to a cross-sectional approach during
not to say that the research design was retrofitted Phase I through the Learning Survey. Method-
to the study, but that the ultimate research design ological triangulation has been incorporated
was not fully known until part way through the through the overall sequential mixed model
research process. Any number of directions could research design of the study, and through the use
have been employed depending on the results of of both qualitative and quantitative data collection
the first phase. For example, if the results of the methods within Phase II of the study. Figure 4

148 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 2, August 2009
 A sequential mixed model research design: Design, analytical and display issues

Initial Pro gram Mid-program End of Pro gram

Evaluation (Week 1) Evaluation (Week 4) Evaluation (W eek 6)

LMP participants: LMP participants: LMP participants:

Participant group Interval self-rating - Participant

interview - QUAL QUAN/Qual Questionnaire -
QUAN/Qual

Learning Survey - (Evaluators meeting - Exploratory group work -

QUAN Week 4) QUAL

Internal Evaluator - Internal Evaluator - External Evaluator -

LMP trainer LMP trainer Researcher

LMP Trainers

Trainer Feedback form

- QUAL

(Evaluation meeting
- Week 8)

External Evaluator -
Researcher

F IGURE 3: P HASE II – M IXED METHOD DATA COLLECTION

QUAN = quantitative data; QUAL = qualitative data; QUAN/Qual = predominantly quantitative with supplementary qualitative

provides a visual depiction of the forms of trian- oped model. The developed model consisted of
gulation achieved during Phase II of the study. four components: self-concept; learning and
Again, this figure provides an example of how recognition; career and life planning; and new lit-
this aspect of the research can be displayed. eracies. These components became the data code
The triangulation achieved in both phases of categories for the qualitative data codes and
the research study is depicted in Figure 5. allowed for a level of alignment and analysis
In terms of data analysis and alignment, the between the results of the quantitative data analy-
decision was made to categorise the qualitative sis in Phase I (the model and its components)
data codes to reflect the structure of the devel- and the different forms of qualitative data collect-

Triangulation of:
• • Data,
• • Investigator,
Theoretical • • Combined levels,
Triangulation • • Time and
• • Methodology
triangulation

Expanded Model Field Test

Literature

F IGURE 4: P HASE II – T RIANGULATION

Volume 3, Issue 2, August 2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES 149
 Roslyn Cameron

Theoretical triangulation

Literature review
Survey Model field
pilot test

Meta-Inference
Phase I – Conceptual Phase II – Model
Learning Survey Fra mew ork:
•• Continuum
•• Model
•• Hierarchy
Triangulation: Triangulation:
•• data •• data
•• combined Phase I – Focus •• investigator
levels Groups •• combined
•• time levels
•• methodological •• time
Triangulation: •• methodological
•• data
•• combined levels
•• methodological

F IGURE 5: T RIANGULATION OF THE STUDY

ed within Phase II of the study (group interview; pragmatism. Several authorities in the area are
exploratory group work). An example of this becoming prominent commentators in the field
from Phase II is depicted in Table 4. This displays and an emerging literature base is developing. An
the alignment between the model components, overview of some of the major mixed methods
the qualitative data codes and the qualitative research design typologies was discussed before
exploratory group work (visual data) collected at presenting and describing research that utilised a
the End of Program data collection point. sequential mixed model design. The case study
allowed for some discussion of issues that utilising
CONCLUSION a mixed method design can present in terms of
This article has mapped the emergence of mixed design, analysis and display options. These issues
methods research as a third methodological move- relate to: the structure such designs can provide for
ment which has resulted from the tensions of the emergent and exploratory approaches to research;
paradigm wars and is related to the philosophy of levels of complexity that can result due to the

TABLE 4: D ATA CODE ALIGNMENT – E XPLORATORY GROUP WORK ( VISUAL DATA )

Exploratory group work

Model Qualitative Data Codes (visual data) – Groups 1–5

1. Self Component Self-esteem/confidence 1, 3, 5

2. Learning and Recognition Component Skills 5
Interaction 1, 3, 4, 5
Self-recognition 1
3. Career and Life Planning Component Employment 1, 5
Direction/challenge 1, 2, 5
Enjoyment 1, 2, 3, 4, 5
4. New Literacies Component Information 1
Financial 1, 5

150 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 2, August 2009
 A sequential mixed model research design: Design, analytical and display issues

blending of mixed methods of data collection Creswell J and Plano Clark V (2007) Designing and
across data collection points; how to align quanti- Conducting Mixed Methods Research. Thousand
tative and qualitative data; and the comprehensive Oaks CA: Sage.
Denzin N and Lincoln Y (Eds) (2005) Strategies of
level of triangulation such approaches can gener- Qualitative Inquiry (2nd edn). Thousand Oaks
ate. Future research is planned to examine and cat- CA: Sage.
egorise the research designs employed by Doctor Earley M (2007) Developing a syllabus for a mixed-
of Business Administration (DBA) candidates methods research course, International Journal of
from an International Centre for Professional Social Research Methodology 10(2): 145-162.
Eberle T and Elliker F (2005) A cartography of
Doctorates at an Australian based University. This
qualitative research in Switzerland, Forum:
planned research will add to the growing body of Qualitative Social Research 6(3): Art 24 September.
knowledge concerning the use of mixed methods Greene JC (2007) Mixed Methods in Social Inquiry.
research designs in the management sciences. San Francisco: Jossey-Bass.
Greene J and Caracelli V (Eds) (1997) Advances in
References Mixed-Method Evaluation: The Challenges and
Bazeley P (2003) Teaching mixed methods, Benefits of Integrating Diverse Paradigms. San
Qualitative Research Journal 3: 117-126. Francisco: Jossey-Bass.
Bergman MM (2008) Mixed Methods Research. Guba E (1990) The Paradigm Dialog. Thousand
Thousand Oaks CA: Sage. Oaks CA: Sage.
Brady H and Collier D (Eds) (2004) Rethinking Guba E and Lincoln Y (2005) ‘Paradigmatic
Social Inquiry Diverse Tools, Shared Standards, controversies, contradictions & emerging
Oxford: Rowman & Littlefield. confluences’ in The Sage Handbook of Qualitative
Buchanan D and Bryman A (2007) Contextualizing Research (3rd edn). Thousand Oaks CA: Sage.
methods choice in organizational research, Hanson W, Creswell J, Clark V, Petska K and
Organizational Research Methods 10(3): 483-501. Creswell J (2005) Mixed methods research
Cameron R (2008) Mixed Methods in Management designs in Counselling Psychology, Journal of
Research, in Proceedings of the 22nd ANZAM Counselling Psychology 52(2): 224-235.
Conference, Australian & New Zealand Academy Hurmerinta-Peltomaki L and Nummela N (2006)
of Management, Auckland, December 2008. Mixed methods in International Business
Cameron R and Miller P (2007) Mixed methods Research: A Value-added Perspective, Management
research: Phoenix of the paradigm wars, in International Review 46(4): 439-459.
Proceedings of the 21st ANZAM Conference, Johnson RB and Onwuegbuzie AJ (2004) Mixed
Australian & New Zealand Academy of methods research: A research paradigm whose time
Management, Sydney, December 2007. has come, Educational Researchers 33(7): 14-26.
Caracelli J and Greene V (1997) ‘Crafting Mixed- King G, Keohane R and Verba S (1994) Designing Soc-
Method Evaluation designs’ in Advances in ial Inquiry, Princeton NJ: Princeton University Press.
Mixed-Method Evaluation: The Challenges and Maxcy S (2003) ‘Pragmatic threads in mixed methods
Benefits of Integrating Diverse Paradigms, Greene J research in the social sciences: The search for mult-
and Caracelli V (Eds) San Francisco: Jossey-Bass. iple modes of inquiry and the end of the philos-
Caulley D (1994) Notes on the basic characteristics of ophy of formalism’ in Tashakkori A & Teddlie C
postpositivist interpretive inquiry, in Neville, Willis (Eds) Handbook of Mixed Methods in Social &
& Edwards (Eds) Qualitative Research in Adult Edu- Behavioral Research. Thousand Oaks CA: Sage.
cation: a colloquium on theory, practice, supervision McMillan J and Schumacher S (2006) Research in
and assessment, Centre for Research in Education Education: Evidence-Based Inquiry (6th edn).
& Work, Adelaide: University of South Australia. Boston: Pearson.
Cohen L and Manion L (1989) Research Methods in Mertens D (2005) Research and Evaluation in
Education (3rd edn). London: Routledge. Education and Psychology: Integrating diversity
Creswell J (1994) Research Design: Qualitative & with quantitative, qualitative, and mixed methods
Quantitative Approaches. Thousand Oaks CA: Sage. (2nd edn). Boston: Sage.
Creswell J (2003) Research design: Qualitative, Miles M and Huberman A (1994) Qualitative data
Quantitative and Mixed Methods Approaches. analysis: An expanded sourcebook, (2nd edn).
Thousand Oaks CA: Sage. Thousand Oaks CA: Sage.

Volume 3, Issue 2, August 2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES 151
 Roslyn Cameron

Mingers J (1997) ‘Multi-paradigm multimethodolgy’ unity and diversity of method. Thousand Oaks
in Mingers J & Gill A (Eds) Multimethodology: The CA: Pine Forge Press.
theory and practice of combining management science Ragin C and Becker HS (1992) What is a Case?
methodologies. West Sussex: John Wiley & Sons. Exploring the Foundations of Social Inquiry. New
Mingers J (2003) The paucity of multimethod York: Cambridge University Press.
research: a review of the information systems Rihoux B (2006) Qualitative Comparative Analysis
literature, Information Systems Journal 13: 233-249. (QCA) and Related Systematic Comparative
Mingers J and Gill A (Eds) (1997) Methods: Recent Advances and Remaining
Multimethodology: The theory and practice of Challenges for Social Science Research,
combining management science methodologies. International Sociology 21(5): 679-706.
West Sussex: John Wiley & Sons. Rihoux B and Grimm H (2006) Innovative Compar-
Morgan D (1998) Practical strategies for combining ative Methods for Policy Analysis: Beyond the Quanti-
qualitative and quantitative methods: Applications tative–Qualitative Divide. New York: Springer.
to health research, Qualitative Health Research 8: Tashakkori A and Teddlie C (Eds) (2003) Handbook
362-376. of Mixed Methods in Social & Behavioral Research.
Morse J (2003) ‘Principles of mixed methods and multi- Thousand Oaks CA: Sage.
method research design’ in Tashakkori A & Teddlie Teddlie C and Tashakkori A (2003) ‘Major Issues and
C (Eds) Handbook of Mixed Methods in Social & Controversies in the use of Mixed Methods in the
Behavioral Research. Thousand Oaks CA: Sage. Social and Behavioral Sciences’ in Tashakkori A &
Neuman W (2006) Social Research Methods: Quali- Teddlie C (Eds) Handbook of Mixed Methods in Soc-
tative and quantitative approaches (6th edn). ial & Behavioral Research. Thousand Oaks CA: Sage.
Boston: Pearson. Teddlie C and Tashakkori A (2008) Foundations of
Plano Clark V and Creswell J (Eds) (2007) The Mixed Mixed Methods Research. Thousand Oaks CA: Sage.
Methods Reader. Thousand Oaks CA: Sage. Thomas RM (2003) Blending Qualitative and
Ragin C (1987) The Comparative Method: Moving Quantitative Research Methods in Theses and
beyond qualitative and quantitative strategies. Dissertations. Thousand Oaks CA: Corwin Press.
California: University of California Press.
Ragin C (1994) Constructing Social Research: The Received 2 May 2008 Accepted 5 November 2008

PROPOSALS FOR SPECIAL ISSUES

International Journal of Multiple Research Approaches (ISSN 1834-0806) is now planning its fifth and sixth years of publication.
Proposals are invited for special issues to appear in volumes 5 and 6 (2011 and 2012) from researchers, supervisors and post-
doctoral students in the areas of health, management, education, psychology, economics, social science and public policy.
Submissions should fall within the scope of the journal, yet may capture emerging methodological fields of philosophy,
education or practice, or a new perspective on an old problem. Proposals which capture cultural approaches to methodology
frames are welcome.
With a primary focus to effectively combine various theoretical frameworks, methodologies and methods to address current
research questions appropriately, the International Journal of Multiple Research Approaches (ISSN 1834-0806) covers:
• Multiple, hybrid, synergistic, integrated, cultural, mixed frameworks, methodologies, data collection, management
qualitative and quantitative empirical research approaches and analysis methods and the different forms of
• Theoretical and conceptual articles on methodological and transformation and representation
ethical dilemmas and advances • Literature reviews – including those from theses – on
• Critical perspectives and proposals for the management of methodological trends/advances
technical issues (eg, software development and data handling) • Articles on methodology education, technologies and
• Discussion of the philosophical issues, practical problems learning techniques
and benefits associated with multiple, hybrid, synergistic, • Practitioner perspectives, experiences from the field and
integrated and cultural approaches including theoretical case applications of methodologies and results
International Journal of Multiple Research Approaches (ISSN 1834-0806) is an international peer-reviewed journal, for timely
publication of global research, scholarship, educational, researcher and practitioner perspectives on multiple, hybrid
(outcome of unusual blending), synergistic (combined effect), integrated and cultural research approaches (be these
Indigenous, institutional, or community based) including the Asian, Arctic, Pacific, Latin American and African regions, as well
as European and North American. Original articles informing multiple and mixed methodological research and practice are
encouraged from a range of disciplines – including all health sciences, social sciences, Indigenous and cultural studies,
psychology, education, organisational science, development studies, political sciences, geography and the creative industries.
Please address special issue proposals to the Editor in Chief: Professor Carol Grbich, at Flinders University School of Medicine
([email protected]) by 1 December 2009. Special issues for 2011 and 2012 will be announced in early March 2010.
eContent Management Pty Ltd, PO Box 1027, Maleny QLD 4552, Australia
Tel.: +61-7-5435-2900; Fax. +61-7-5435-2911
www.e-contentmanagement.com

152 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 2, August 2009