Applied Linguistics --Quantitative, Qualitative and Mixed Methods Huang Lan Department of Humanities and Foreign Language, Hunan Agricultural University, Hunan, 410128, China
ABSTRACT. The purpose of this paper is to compare and critique three traditional research methods: quantitative, qualitative and mixed methods. The article begins with a review from some research papers relating to the comparison of quantitative and qualitative methods. Definitions, Characteristics, strengths and limitations are discussed. Research papers have been examined and analysed in order to critique how these three methods have been applied in applied linguistics. It concludes that the best method depends on the knowledge of researchers, research questions and research purpose.
What is research? Nunan (1992:2-3) defined that it is a process of inquiry and
it formulates questions, problems or hypotheses; we are analysing or interpreting data or evidence which are relevant to answer or prove these questions, problems or hypotheses. The value of doing research facilitates the development of natural and social science. For teachers, the reason for conducting research is to further improve teaching and learning and to understand the teaching and learning process (Zoltan Dorney 2007:16). In social science, there are three common approaches to choose, which are quantitative, qualitative and mixed methods. Researchers often choose either the quantitative or qualitative methods or even combine these two methods. In terms of the selection of methods which has been repeatedly discussed and debated in literature on research methodology. During the 1970s and 80s, there was a period called an era of ‘paradigm wars’ which showed the critique against quantitative research for its dominance for several decades (Katrin 1999:3). There was a rapid growth in quantitative method from early 70s to the mid 80s, which challenged qualitative method (Gao, Li, Lu 2001:1). Hathaway (1995:539) summarises the debate on these two methods,
Published by Francis Academic Press, UK
- 78 - Frontiers in Educational Research ISSN 2522-6398 Vol. 3, Issue 14: 78-85, DOI: 10.25236/FER.2020.031414
which also includes the debate on whether these two methods could be combined. Dorney (2007: 29) argued that the fundamental difference between quantitative and qualitative is striking and they belong to different paradigms. If we mix them, we will lose their essence. Atieno (2009: 13) points out although some social scientific researchers (Lincoln & Guba, 1985; Schwandt,1989)perceive qualitative and quantitative approaches as incompatible, others (Patton, 1990; Reichardt & Cook, 1979;) believe that the skilled researcher can successfully combine these approaches. With mixed methods, biases characteristically associated with one method (quantitative or qualitative) are thought to cancel out (Creswell, 2003). Harwell, M. R. (2011:148) argued that mixed methods shows a promising way for research design for research inquiry.
2. Definition and Characteristics
I looked for the definition of these methods by reviewing seventeen research
papers on the comparison and discussion on quantitative, qualitative and mixed methods. I found that Creswell’s views on these methods were cited in eleven research papers. So in this paper I will apply Creswell’s definition from his book Research Design: Quantitative, Qualitative and Mixed Methods (2nd Edition, 2003), then follows my detailed explanation.
2.1 What is Quantitative Research?
Postpositivist claims for developing knowledge, (i.e., cause and effect thinking, reduction to specific variables and hypotheses and questions, use of measurement and observation, and the test of theories), employs strategies of inquiry such as experiments and surveys, and collects data on predetermined instruments that yield statistical data (Creswell 2003: 7).
2.2 What is Qualitative Research?
Qualitative approach is the inquirer primarily based on constructivist
perspective (i.e., the multiple meanings of individual experience, meanings socially and historically constructed, with an intent of developing a theory or pattern). It also uses strategies of inquiry such as narratives, phenomenologies, ethnographies, grounded theory studies, or case studies. The research collects open-ended, emerging data with the primary intent of developing themes from the date. (Creswell 2003: 7).
2.3 What is Mixed Methods?
Mixed methods approach is defined that ‘the researcher tends to base
knowledge claims on pragmatic grounds. (e.g., consequence-oriented, problem-centred, and pluralistic). It employs strategies of inquiry that involve
Published by Francis Academic Press, UK
- 79 - Frontiers in Educational Research ISSN 2522-6398 Vol. 3, Issue 14: 78-85, DOI: 10.25236/FER.2020.031414
collecting data either simultaneously or sequentially to best understand research
problems. The data collection also involves gathering both numeric information as well as text information so that the final database represents both quantitative and qualitative information. (Creswell 2003: 7). Creswell’s definition combines philosophical ideas, approaches to research and specific procedures. (Creswell, 2003:4) It starts with the knowledge claim which tries to tell ‘how they will learn and what they will learn during their inquiry’(Creswell. 2003:6). Creswell points out that “Postpostivism” challenges the traditional notion of the absolute truth of knowledge which has been mostly discussed by Phillips and Burbules. Postpositivism reflects a deterministic philosophy in which causes probably determine effects or outcomes. (Creswell, 2003:7). Constructivism is the idea from Lincoln and Guba’s Naturalistic Inquiry (1985). It can be summarised that people are trying to understand the world with their own experiences which can develop subjective meanings that are varied and multiple, ‘leading the researcher to look for the complexity of views rather then narrowing meanings into a few categories or ideas’ (Creswell, 2003:8). Pragmatic Knowledge Claims concerns with applications and solutions to problems. Tashakkori and Teddlie (1998) and Patton (1990) who ‘convey the importance for focusing attention on the research problem in social research and then using pluralistic approaches to derive knowledge about problem’ (Creswell, 2003:11). Secondly, it identifies the strategies of inquiry which provide specific directions for procedures in a research design. (Creswell, 1998; Mertens, 1998; Creswell, 2003) The third part considers the research methods such as data collection and data analysis. Creswell (2013:32) further develops the definition by highlighting in more detailed expression that quantitative research tries to test objective theories, to exam variables which can be measured by instrument, to analyse numbered data by using statistical procedures and to generalize and replicate the findings. Qualitative research focuses on the process of natural settings with emerging questions and procedures; analyse data inductively and researcher interprets the meaning of the data. Mix methods research tries to combine two types of data which aims for a better understanding of research questions. So we can see clearly from Table 1 that three approaches show their own characteristics and function in research.
Published by Francis Academic Press, UK
- 80 - Frontiers in Educational Research ISSN 2522-6398 Vol. 3, Issue 14: 78-85, DOI: 10.25236/FER.2020.031414
Fig.1 D Nunan (1992:4) Figure 1.1
Quantitative research Qualitative research
Advocates use of quantitative methods Advocates use of qualitative methods Seeks facts or causes of social Concerned with understanding human phenomena without regard to the behaviour from the actor’s own frame subjective states of the individuals of reference Obtrusive and controlled measurement Naturalistic and uncontrolled observation Objective Subjective Removed from the date: the ‘outsider’ Close to the date: the ‘insider’ perspective perspective Ungrounded, verification-oriented, Grounded, discovery-oriented, inferential, and hypothetical deductive exploratory, expansionist, descriptive, and inductive Outcome-oriented Process-oriented Reliable: ‘hard’ and replicable data Valid: ‘real’, ‘rich’ , and ‘deep’ data Generalisable: multiple case studies Ungeneralisable: single case studies Assumes a stable reality Assumes a dynamic reality
3. Critiques on Strengths and Limitations of Research Methods
Based on the features of these methods, researchers’ selection can be different.
Each method holds their strengths and limitations which need to be considered when we are conducting a research. Many researchers discuss them in some published books and papers.
3.1 Strengths and Limitations of Quantitative Research
Strengths of quantitative research can be summarised into following parts. For
the majority of scholars, objectivity is one of the features and also advantages in quantitative research method. A researcher will draw a conclusion without considering his or her own experience and can put aside his or her biases which is well controlled. (Michael, 2011; Creswell 2013; Vincent 2016). Zoltan concluded that quantitative is regarded as systematic, rigorous process to collect and analyse data. The use of large numbers of people proves the sufficient data to analyse and ‘replicable date that is generalizable to other contexts’ and data can be analysed by using statistical computer software which ‘offer come in-built quality checks and indices that help researchers to decide on the validity of quantitative findings’ Zoltan (2007:34). In terms of researchers, it is less flexible for the research design and data collection process. Its limitations are that quantitative is research driven and researchers collect date which the participants’ voice can not be fully heard so there is not sufficient
Published by Francis Academic Press, UK
- 81 - Frontiers in Educational Research ISSN 2522-6398 Vol. 3, Issue 14: 78-85, DOI: 10.25236/FER.2020.031414
knowledge about the context of participants (Creswell, 2013; Vincent 2016:3).
Hathaway (1995:554) also states that the result of quantitative research has less value due to the overlooking ‘features of phenomena such as the definition of human action in specific settings, the human interest of the actor, and the historical context of the situation’. It is concluded that quantitative studies focus on ‘repetitive and predictable aspects of human behaviour which restrict our views of human beings’ (Cohen & Manion 1996; Salome 2003: 12). Zoltan argues that ‘the general exploratory capacity of quantitative research is rather limited’ (Zoltan 2007:35). Brannen also argues that it fails to catch the meanings that actors’ lives and circumstances’ (Brannen 2005:7).
3.2 Strengths and Limitations of Qualitative Research
The merits of Qualitative research include many aspects. According to Vincent’s
view, participants’ voices can be heard and recorded so it is easier to understand the context of participants. (Creswell 2013; Vincent 2016:3) Zoltan (2007:39; Ochieng, 2009) summarised from several points:(1) exploratory nature which does not depend too much on previous literature; (2) it tries to work on complex situations; (3) it intends to answer why questions, which helps researchers to look for deep understanding of a phenomenon and helps research to do further research in order to broaden our understanding; (4)it is a longitudinal research which is used in applied linguists are dynamic in nature; (5) It is more flexible for researchers to change or adjust when things go wrong. ‘Qualitative methods allow us to accommodate the changes but can also enable us to capitalize on them and produce exciting results’(Zoltan Dorney 2001: 40). (6) Multiple data can be collected. Words to be used so it makes easier for participants to produce convincing data. Based on the discussion above, my critiques on strengths and limitations will be as follows. In regard to strengths of qualitative research, I firstly argue that qualitative is the effective methods to explore the why-question and understand in-depth meanings. Take some qualitative research for example. Ying and King (2011:326)’s research tries to discover ‘second language learning beliefs in study abroad contexts’. Secondly, qualitative research needs to collect multiple data in a long period of time. So multiple data is the guarantee for the validity and reliability of the research. Golombek’s research (1998) shows that there were over 4 months to collect 45 daily nonparticipant class observations, 45 after-class interviews, videotaped the teacher in the final month in class, stimulus recall reports of the videotapes and field notes. For limitations, even though many researchers concern that qualitative research is too subject but I argue that qualitative research adopt some analytical tools to analyse data so when researchers interpret the data, they are based on the scientific way to analyse data rather than just guessing the meaning of the interpretation. Take the research paper from Golombek (1998) for example. The interview date ‘were processed by means of a constant comparative method and inductive analysis procedures’(Golombek 1998:451). So researchers sometime repeatedly check the data then come to the conclusion or interpretation.
Published by Francis Academic Press, UK
- 82 - Frontiers in Educational Research ISSN 2522-6398 Vol. 3, Issue 14: 78-85, DOI: 10.25236/FER.2020.031414
3.3 Strengths and Limitations of Mixed Research
Some researchers argued that mixed methods is a possible trend due to the limitations of quantitative and qualitative methods. Michael (2011: 148) argued that mixed methods shares a ‘promising path’. Mixed methods research allows for the ‘opportunity to compensate for inherent method weaknesses, capitalise on inherent method strengths, and offset inevitable method biases’ (Greene, 2007: xiii, Denzin, 1970; Michael, 2011: 151; Vincent 2016:4)). At the same time mixed methods try to complement and enhance each other (Jonna E. M. Sale; Lynne H. Lohfeld; Kevin Brazil 2002: 48). Some scholars Mouton and Marais (1990:169–170) think it is necessary to mix these methods because neither quantitative nor qualitative methods can fully grasp the complex human beings and ‘the breadth, depth and richness of human life’(Salome 2003:12). There is also disagreement regarding various aspects of mixed methods, such as when mixing should occur. Still other authors have criticized the whole idea of mixed methods which is time-consuming and requires more efforts. (Denzin, 2006; Sale, Lohfeld, & Brazil, 2002; Smith & Hodkinson, 2005; Michael 2011:152). Some researchers criticised that philosophical distinctions have been overlooked, which left the impression that ‘the differences between the two are merely technical’ (Smith and Heshius, 1986; Jonna E. M. Sale; Lynne H. Lohfeld; Kevin Brazil 2002: 44). Some methodologists might argue that ‘a combination of quantitative and qualitative data based on the administration of one research instrument does not represent a true integration of quantitative and qualitative research because one will tend to be subordinate to the other’ (Alan Bryman 2006: 103). Hesse-Biber and Leavy (2006) concerns that whether research can hold both of the research method together? Marxwell and Loomis worried whether we have the framework to guide the way of doing mixed methods. My arguments on merits of mixed methods are as followings. I think mixed method can be worked together as a complementary way. For researcher, they need to think when to combine this methods. Take Baker’s (2014) research for example, he raised the qualitative research questions but he collected qualitative and quantitative data. The research used both methods then the questionnaires with students further support the results of qualitative research. So these two methods combines in data collection and data analysis section. For limitations, I think there are two things we need to consider. Firstly, whether these two methods should be work equally in the research or whether one method should dominate the other? Secondly, as a researcher, whether they can be capable of controlling both methods at the same time.
3.4 Concerns on Quality Criteria
Critiques on these methods show that there is no perfect research method
conducted by researchers. The appearance of one method can be not substituted by another method; On the contrary, they try to complement each other. For researchers,
Published by Francis Academic Press, UK
- 83 - Frontiers in Educational Research ISSN 2522-6398 Vol. 3, Issue 14: 78-85, DOI: 10.25236/FER.2020.031414
no matter what types of research methods they choose, the common goal is to have a good quality of research. Shipman (1988) suggests that four key questions about the quality of research: reliability, validity (internal validity), generalizability (external validity) and credibility. Reliability and validity normally have been considered in quantitative research regarding to measurement. Zoltan points out reliability and validity are significant concepts and ‘are unquestionable fact of life in the quantitative paradigm’(Zoltan, 2001: 49). Some researchers argued that the quality criteria that are suitable for qualitative research should not be the same as quantitative research (Zoltan 2001:49). So my conclusion is that researchers need to reach the quality criteria when they are doing researches. Except reliability and validity, researchers also need to consider the ethical issues in all these methods. No matter in social research or in education, ethical issues need to be considered as long as people involve. Especially in qualitative research, researchers need to do personal interviews or use participant’s diary or take videos to collect data, so researchers need to acquire the agreement with participants rather than do it without permission.
4. Conclusion
This essay starts with the introduction of three traditional research methods by comparing and contrasting their definition, characteristics and critiques of strengths and weaknesses from some researchers and from my standpoints. Based on the critiques, I try to make a research plan for my proposed research topic. The paper concludes that each method has its advantages and disadvantages. Quantitative research approach has dominated in natural and social research areas for quite a long time until the challenge from qualitative research. Then there is a strong tendency for researchers to mix these two methods nowadays due to the fact that both methods can combine together for a better understanding of phenomenon. It is reasonable to think that the purpose for doing research is to discover the fundamental facts and generate theory to make a better improvement of life. So, the best way is to continue the application and exploration of these methods in order to face up with the diversified world. Conducting research is integral to teachers who can solve the issues raised from teaching. It is also beneficial to students who want to improve their English skills.
Acknowledgments
Project Title: Practice and Exploration of the Construction of the “Golden
Course” of College English Quality Development Courses--Taking “General Situation of Chinese Culture” as an example Xiang Jiaotong (2019) No. 291 Project Number: 326.
Published by Francis Academic Press, UK
- 84 - Frontiers in Educational Research ISSN 2522-6398 Vol. 3, Issue 14: 78-85, DOI: 10.25236/FER.2020.031414
References
[1] Atieno, O. P (2009). An Analysis of The Strengths and Limitation of
Qualitative and Quantitative Research Paradigms’, Problems of Education in the 21st Century, no.13, pp.13-15. [2] Bryman, Alan (2006). Integrating Quantitative and Qualitative Research: How Is It Done?’ Qualitative Research, pp. 97-113.
