Knowledge Management & E-Learning: An International Journal
Knowledge Management & E-Learning: An International Journal
Knowledge Management & E-Learning: An International Journal
Mar 2013
ISSN 2073-7904
Jiyou Jia
Peking University, China
Dongfang Xiang
Huiwen Middle School, Beijing, China
Zhuhui Ding, Yuhao Chen, Ying Wang, Yin Bai, Baijie Yang
Peking University, China
Recommended citation:
Jia, J., Xiang, D., Ding, Z., Chen, Y., Wang, Y., Bai, Y., & Yang, B.
(2013). An effective approach using blended learning to assist the average
students to catch up with the talented ones. Knowledge Management & E-
Learning, 5(1), 25–41.
Knowledge Management & E-Learning: An International Journal, Vol.5, No.1. 25
Jiyou Jia*
Department of Educational Technology
Graduate School of Education
Peking University, Beijing, China
E-mail: [email protected]
Dongfang Xiang
Huiwen Middle School
Dongcheng District, Beijing, China
E-mail: [email protected]
Zhuhui Ding
Department of Educational Technology
Graduate School of Education
Peking University, Beijing, China
E-mail: [email protected]
Yuhao Chen
Department of Educational Technology
Graduate School of Education
Peking University, Beijing, China
E-mail: [email protected]
Ying Wang
Department of Educational Technology
Graduate School of Education
Peking University, Beijing, China
E-mail: [email protected]
Yin Bai
Department of Educational Technology
Graduate School of Education
Peking University, Beijing, China
E-mail: [email protected]
26 J. Jia et al. (2013)
Baijie Yang
Department of Educational Technology
Graduate School of Education
Peking University, Beijing, China
E-mail: [email protected]
*Corresponding author
Abstract: Because the average students are the prevailing part of the student
population, it is important but difficult for the educators to help average
students by improving their learning efficiency and learning outcome in school
tests. We conducted a quasi-experiment with two English classes taught by one
teacher in the second term of the first year of a junior high school. The
experimental class was composed of average students (N=37), while the control
class comprised talented students (N=34). Therefore the two classes performed
differently in English subject with mean difference of 13.48 that is statistically
significant based on the independent sample T-Test analysis. We tailored the
web-based intelligent English instruction system, called Computer Simulation
in Educational Communication (CSIEC) and featured with instant feedback, to
the learning content in the experiment term, and the experimental class used it
one school hour per week throughout the term. This blended learning setting
with the focus on vocabulary and dialogue acquisition helped the students in
the experimental class improve their learning performance gradually. The mean
difference of the final test between the two classes was decreased to 3.78, while
the mean difference of the test designed for the specially drilled vocabulary
knowledge was decreased to 2.38 and was statistically not significant. The
student interview and survey also demonstrated the students’ favor to the
blended learning system. We conclude that the long-term integration of this
content oriented blended learning system featured with instant feedback into
ordinary class is an effective approach to assist the average students to catch up
with the talented ones.
Zhuhui Ding, Yuhao Chen, Yin Bai and Baijie Yang are master students in the
Department of Educational Technology, Graduate School of Education, Peking
University.
1. Introduction
Average students in the primary and secondary education are referred to as the students
whose performance in the classroom is normal, while the gifted or talented students
outperform the average ones in the classroom tests. Because the average students are the
prevailing part of the student population, it is an important and difficult task for the
educators to help the average students by stimulating their learning interests, improving
their learning efficacy and the learning outcome that can be achieved in school exams and
tests.
Since the modern computer was born in the 1940s, one of its important
application fields is education at every educational stage from kindergarten to higher
education. Computer Assisted Instruction (CAI ) is one early definition that describes
instruction assisted by computer technology. Though the computer hardware and
software have evolved through several generations from 1940s up to date, this definition
still can designate the nature of computer application in instruction with all kinds of
forms, no matter what it is called, such as Computer Based Education (CBE), Computer
Based Instruction (CBI), electronic learning (e-learning ), Web Based Learning (WBE),
etc. In the new millennium, a new term, called blended learning or blending learning, has
been adopted and widely used to replace the old-fashioned notation CAI and to describe
the instructional design that blends the traditional classroom and Information and
Communication Technology (ICT). Thus in order to ensure the consistency in this paper,
we just use the term CAI to represent all kinds of computer’s application in instruction.
2. Related work
Can CAI help all kinds of students including disabled, average and talented ones improve
their learning outcome and to what extent? This question has drawn great attention since
1950s. A number of meta-analysis studies of CAI analyzed dozens, hundreds or even
thousands of studies dealing with thousands of subjects, and found that CAI generally can
have a more positive effect on learning performance than traditional instructional
approaches (Burns, 1981; Hartley, 1978; Kulik & Kulik, 1991; Liao, 2007; Tamim,
Bernard, Borokhovski, Abrami, & Schmid, 2011; Yueh, Lin, Huang, & Sheen, 2012;
Wakefield, Warren, Rankin, Mills, & Gratch, 2012). For the average or low-performance
students, much research has shown that CAI can have a positive impact on their learning
outcome (Lynch, Fawcett & Nicholson, 2000; O’Byrne, Securro, Jones & Cadle, 2006;
Huang, Yang, & Hwang, 2010).
28 J. Jia et al. (2013)
How can CAI be used to help the low-performance students improve their
learning performance? The answer to this question depends on the learning content, the
learner’s age and other learner characteristics. Despite of disciplinary content and learner
difference, the learning time plays a key role in general. Mann, Shakeshaft, Becker, and
Kottkamp (1999) conducted a study of West Virginia’s Basic Skills/Computer Education
(BS/CE) by analyzing results from a representative sample of 950 fifth-grade students
from 18 elementary schools across the state. The study showed that the longer students
participated in the BS/CE, the higher their test scores on Stanford Achievement Test
(SAT): SAT-9. Ligas (2002) conducted a five-year longitudinal study to examine the
impact of CAI on reading achievement of ‘at-risk’ elementary and middle school students
in Florida. The study found that the students group who used the software for 12 hours or
more outperformed the students group who did not use the software, or used it less than 5
hours, by 7.74 points on the SAT-8 Reading Comprehension average normal curve
equivalent (NCE) scores. Liao (2007) revealed that for the duration variable, the largest
mean ES (1.182) was associated with studies lasting 4–18 hours.
Summarizing the aforementioned literature review, it is inferable that the CAI can
have positive effects on average students’ learning performance, and that the longer
usage of CAI can produce a better performance improvement.
For English instruction as a second language in middle schools, which is often
listed as a core subject, most research has shown the positive effect of CAI on learning
performance within a short duration, for example, several hours within several weeks
(Tsou, Wang, & Li, 2002; Liu, 2009; Liu & Chu, 2010; Chen, Ho, & Yen, 2010;Fujishiro
& Miyaji, 2010). The average or low-performance students’ learning improvement,
compared with the excellent or high-performance ones, varied case by case.
However, we have found few research papers on the long-term integration of CAI
into English instruction in middle schools, for example for a school term, and its effect on
the school test performance of existing classes comprised of average students. Our
previous study (Jia, Chen, Ding, & Ruan, 2012) indicated that the blended learning
setting can facilitate the vocabulary acquisition and improve the students’ examination
performance. The experimental class, starting with a higher pre-test score mean and
participating in the blended learning with one weekly school hour in the computer pool
throughout the experimental term, enlarged its examination score mean difference to the
control class. What will be the result if the experimental class with average students
performs much worse in the pre-test than the control class with talented students? Can the
blended learning help the average students catch up with the talented ones? This is the
key problem reflected in this research paper.
In the instruction of English as a second language, vocabulary acquisition is the
most important foundation, because it is the fundamental prerequisite to the four skills of
a language: listening, speaking, reading and writing. Linguistic experts believe that
vocabulary knowledge and the ability to comprehend text are inextricably linked, and the
breadth and depth of a student's vocabulary is a key forecaster of his/her ability to
understand a wide range of texts (Anderson & Freebody, 1981; Thorndike, 1973). This is
true for both native speakers of English and second language learners (Coady, 1993;
Stoller & Grabe, 1993).
A large amount of researches investigate the vocabulary instruction supported by
emerging technologies in university and college, such as (Chen, Hsieh, & Kinshuk, 2008;
Chen & Chung, 2008; Jones, 2004; Huang & Liou, 2007; Lu, 2008; Peters, 2007), etc.
However, we can only find very few literature studies on computer assisted vocabulary
teaching and learning for formal secondary school students throughout a long time such
Knowledge Management & E-Learning: An International Journal, Vol.5, No.1. 29
as a whole school term. This is just the gap between the theoretical research and the
pedagogical practice which we would like to reduce.
3. System architecture
Research has shown that children can be taught new word meanings through rote
methods involving synonyms and definitions (McKeown, Beck, Omanson, & Pople, 1985;
Stahl, 1983). Moreover, in the case of L2 learners, there is great value in the repetition
and immediate access to definitions for unknown words, especially when those words are
rarely used in English (Stoller & Grabe, 1993). Because technology generally improves
performance if the application directly supports the curriculum content, specifically the
vocabulary learning, we still use the same system, namely CSIEC, as the one described in
(Jia, Chen, Ding, & Ruan, 2012) to support the blended learning for an English class.
This web-based system comprises exercises for every module in a textbook. The exercise
for every module is logically composed of two parts, vocabulary and dialogue, as shown
in Fig. 1.
Fig. 1. The architecture of every module with vocabulary and dialogue assessment
functions
The first part is the course management system that mainly supplies question
banks and quizzes about the vocabulary required in a certain module. The questions and
the quizzes have four features.
The first feature is the multiple choice question and cloze in which a sound file
can be played so that pronunciation and listening based questions can be embedded. For
example, a multiple choice question or a cloze about the spelling and meaning of an
English word or phrase is raised to the students after its pronunciation is played back.
The second feature refers to the randomized items of the quiz based on a question
bank, as well as the randomized sequence of the choice items to a multiple choice
question. This intelligent feature challenges all the students sitting in front of the
computers in a computer pool and doing the same quiz simultaneously.
The third feature is the instant feedback including score, comments and correct
answers after the student submits his or her answers to a quiz. Nevertheless, the scores of
all students in the class can not only be read by the students themselves, but also be
30 J. Jia et al. (2013)
browsed by the teacher. Both the individual feedback and the collective scores can inform
the students and the teacher about the learning outcome, and motivate the students to
compete with each other in the blended learning setting.
The fourth feature is the individualized error set that includes the words and
phrases, with which one student has made mistakes in the multiple choice question and
cloze. A new individualized cloze quiz can be generated based on the words and phrases
within the error set. The student can review the words and phrases with which the
mistakes have been made by doing this cloze quiz.
The second part besides the vocabulary exercises is the dialogue simulation for
specific topics or scenarios defined in the teaching module in the textbook. Two or more
than two roles participate in this kind of dialogue about a specific topic. Two types of
simulation with multiple agent technology have been designed. The first addresses the
talk show of multiple agent characters to role play the dialogue, in which the main
content is semantically the same as the one given in the textbook, but the expressions are
randomly generated according to the predefined script. The second represents the
interactive dialogue with the student participating as a role in it. During the interactive
dialogue the student should input the semantically same or similar expressions as the
textbook in order to ensure the dialogue process. Both in the talk show and the interactive
dialogue, the user can select one of the twelve avatars to represent one role in the
dialogue according to his/her preference. The avatar is in fact a Microsoft agent character
that can speak the text with synthesized voice and carry out some actions. The dialogue
simulation can stimulate the students to participate in the dialogue and learn its content,
and strengthen the listening comprehension.
The two parts, vocabulary and dialogue, are not separated. Some words and
phrases drilled in the first part occur in the corresponding dialogue in the same module.
Through the dialogue simulation the students can understand how the words or phrases
learned are used in the practical dialogue. Wilkins (1972) argued: ‘‘without grammar
very little can be conveyed, without vocabulary nothing can be conveyed.’’ The two parts
are intended to help the average students master the vocabulary and its usage in the
dialogue.
4. Methodology
4.2. Participants
The participants in this research came from two existing classes of Grade one of a junior
school in Capital Beijing, one was an ordinary class and another was an excellent class.
The 34 students in the excellent class were selected from the primary schools in the entire
city with their excellent performance in tests of three main subjects, specifically
mathematics, Chinese language and English language, while the 37 students in the
ordinary class were randomly picked out with a lower performance in tests of the three
main subjects. Therefore in the final exam of the first term of Grade one, which we used
Knowledge Management & E-Learning: An International Journal, Vol.5, No.1. 31
as the pre-test in this research, the excellent class achieved much better scores in the
English subject than the ordinary class. The mean difference between the two classes in
the test, 13.5%, was statistically significant based on the independent sample T-Test
analysis with statistical software SPSS (V16.0), as shown in Table 1.
The two classes’ teacher X was interested in and experienced with computer
assisted language learning. The school managers agreed to our blended learning
experiment for Teacher X’s two classes in the second term of Grade one, and arranged
one school hour in the school class schedule for the ordinary class to be held in one
multimedia computer pool of this school. The initial hope was that our blended-learning
could help the normal students improve their learning outcome, and decrease the
difference between the two classes. We defined the ordinary class as an experimental
class and the school hour in the computer pool as an experimental hour, while the
excellent class as a control class still held its class in a traditional classroom.
Table 1
The English test scores (with the full score 100) of the treatment (average class) and the
control (excellent class)
Midterm Final Vocabulary
Pre test
test test test
Month January April July July
Mean 67.73 77.38 91.21 91.99
Treatment: average class (N=37)
Std. Dev. 14.649 10.364 5.702 7.737
Mean 81.20 87.68 95.00 94.38
Control: talented class (N=34)
Std. Dev. 7.892 4.946 2.256 4.199
Absolute mean difference between two classes 13.48 10.30 3.78 2.38
Relative mean difference compared with control class 16.60% 11.74% 3.98% 2.52%
Difference between the Std. Dev. of the two classes 6.756 5.417 3.445 3.537
Significance of the independent samples T-test between two
0.000 0.000 0.000 0.1894
classes (2-tailed and equal variances assumed)
as usually. On the contrary, the control class still had all its English class in the normal
classroom. While the students in the experimental class reviewed and assessed their
vocabulary and dialogue by using the blended learning system, the students in the control
class did it via traditional approaches without computer support, such as paper-based or
with peers. Except this experimental hour, all the other syllabus design and
implementation of the experimental class and the control class remained the same.
In the computer pool, all the multimedia computers are connected via the Internet,
so that every student can use one computer individually. A computer and a projector can
also be used by the teacher for instructional purposes. In the experimental hour, the
students browsed the website of the CSIEC system via Internet and logged into it with
their own account and password. Then they did the quizzes by themselves. After
submitting the answers the student can read the mark he/she achieved and find the
mistakes and feedback. If the student encountered difficulties by finding the answers,
he/she can look for them in the textbook or get help from the teacher. This search action
strengthened the student’s memory of English words or phrases.
By the blended learning in the computer pool, the teacher was still the leader of
the instructional process. He/she can encourage or affect the students through his/her
speech and body language, and can use the computer projector to show the marks of all
students after they have submitted their answers. This instant feedback motivated the
students to finish the exam more focused and carefully. Because the computers were
connected to the Internet, the teacher’s presence prevented the students from browsing
games or other websites not related to the class.
Fig. 3. The comparison of the test performance of the treatment and the control class
through the school term
In the pre-test, the excellent class performed much better than the ordinary class
with the mean difference 13.5. In April and July there was the midterm exam and the
final test, respectively. All the tests assessed the listening, reading and writing skills of
the examinees. In all the exams, the excellent class still achieved a better performance
than the ordinary class, and the independent sample T-Test analysis with SPSS always
showed statistically significant difference between the means of two classes. However,
the absolute mean difference was decreased gradually from 13.5 to 3.8. The mean
difference of the two classes was decreased by 71.9% throughout the term.
Especially, in order to test the vocabulary acquisition of the students, a vocabulary
test was held in July. Though the excellent class performed better than the ordinary class,
the mean difference was just 2.4, and was statistically not significant based on the
independent sample T-Test analysis with SPSS (p=0.189>0.05).
Historical comparison shows that the final test mean (91.21) of the ordinary class
was 34.7% greater than that of the last term (67.73), while the final-test mean of the
excellent class (95.00) was just 17.0% greater than that of the last term (81.20). Though
both classes reported a statistically significant increase at the 0.000 level, the mean
difference of the absolute score gain (post test score minus pre-test score) between the
treatment and the control is statistically significant at the 0.000 level, as shown in Table 2.
The longitudinal improvement is often used to demonstrate the students’ performance
advancement in educational research. Therefore from the historical view of the student
performance evolution, we can conclude that the ordinary class increased its exam
performance during the experimental term much more significantly than the excellent
class.
As the two classes were in Grade One, their previous test scores were comparable
only in one term, specifically in the first term of Grade One in which both of them existed.
34 J. Jia et al. (2013)
The score mean difference at the beginning of the first term was 12.4. Through the first
term without blended learning, the mean difference was slightly increased to 13.5.
Table 2
The independent sample T-Test analysis with SPSS software for absolute score gain
Class N Mean Standard T-test for equality of means
deviation (Equal variances assumed)
Average 37 23.49 11.30 t = -4.214, Sig. (2-tailed) =.000
Talented 34 13.79 7.53
Because the average score stands for the collective performance of an observed
class as a whole, we reveal the following findings about the treatment and the control.
1) Before the experiment, the examination mean of the excellent class was statistically
significant much more than that of the ordinary class, and this difference already
remained for one term without blended learning.
2) Throughout the experiment both classes improved their test performance
significantly.
3) The ordinary class’s improvement was greater than that of the excellent class so that
the mean difference between the two classes was decreased gradually.
4) Though the mean difference between the two classes in the post test remained
statistically significant, the difference was just 3.8. Compared with the greater
difference 13.5 before the experiment, the difference was decreased by 71.9%.
5) Though the vocabulary test performance of the ordinary class was worse than that of
the excellent class, the difference was statistically not significant more, proved by the
independent sample SPSS T-Test result.
From those findings, and the fact that the only instructional difference between
the two classes was that the ordinary class adopted one school hour blended learning
every week with our CSIEC vocabulary and dialogue system, while the control class did
not, we can come to the conclusion that the blended learning with our CSIEC system, or
the integration of the vocabulary and dialogue assessment system into the ordinary
English instruction, improves the students’ test performance, and especially the
vocabulary acquisition.
“In the cloze question the system requires the exact input of the Chinese meaning
of the English word and punctuations, as defined in the answers, what is too inflexible.”
Other students also complained this problem.
“The interface of the web-based system is very clear and is very easy to use.”
“This blended learning in the computer pool is very helpful for vocabulary
learning. We have to learn the words by heart every week and do not need to review all
the words in a hurry just before the exam.”
“Though the cloze question is harder for me than the single choice question,
which I like to do in fact, the cloze question can help me better remember the new
words.”
“This kind of blended learning in a computer pool is more effective to help me
learn the vocabulary and other content than the learning in a traditional classroom,
because we must concentrate on it, otherwise we cannot get higher scores from the
computer.”
“Just the words learning function is somewhat boring. However, it improves my
learning outcome.”
“I wish to continue to use this system in the next term.”
“Sometimes it is difficult for me to hear the voice in English.”
From these comments and suggestions we are informed that instant scoring and
feedback function can motivate the students to master the vocabulary, and the problems
such as Chinese meaning input and different pronunciation were challenging the students.
outcome. It can also decrease the teachers’ daily heavy burden with the students’
assignment distributing and checking.
Table 4
The coinciding findings both in our research and in Liao (2007)
Variable influencing ES Finding in Liao (2007) Our case
Duration The largest mean ES (1.182) was One term, including 16 school
associated with studies lasting 4–18 hours for the blended learning
hours. in the computer pool.
Grade level The ES associated with junior school First year junior school
subjects (7th–9th graders) was the students, i.e. seventh grade
highest (0.847). students.
Subject The mean ES of language/social English subject as a second
study (0.664) was the second highest. language.
Sample size The small sample (1-50) had the 37
highest ES 0.690.
System type The intelligent CAI software reached Web-based intelligent
the highest ES 1.591. vocabulary and dialogue
assessment system.
the equivalent academic performance in the pre-test, or to divide one class into two
equivalent groups. However, in a practical school setting, it is difficult to find such two
equivalent classes. The idea of class division would cause prejudicial feeling of the
students in the same class, and more importantly, the syllabus design and implementation
with two groups would mean doubled work load for the teacher. Such practical school
settings cannot be neglected in our academic research.
Acknowledgements
The research in this paper is a part of one ongoing project “The blended instruction
research of middle school English subject supported by intelligent tutoring system
CSIEC” (10BYY036), sponsored by the National Social Science Council, China. It is
also sponsored by the Program for New Century Excellent Talents in University (NCET-
10-0179), Ministry of Education, China. We would like to thank all the principals,
teachers and pupils in the junior high school for their voluntary participation in this
project. We are also grateful for anonymous reviewers’ valuable critics, comments and
suggestions.
References
Anderson, R. C., & Freebody, P. (1981). Vocabulary knowledge. In Guthrie, J. T. (Ed.),
Comprehension and Teaching: Research Perspectives (pp.71–117). Newark, DE:
International Reading Association.
Burns, P. (1981). A quantitative synthesis of research findings relative to the pedagogical
effectiveness of computer assisted instruction in elementary and secondary schools.
Dissertation Abstracts International, 42, 2946A.
Chen, C., & Chung, C. (2008). Personalized mobile English vocabulary learning system
based on item response theory and learning memory cycle. Computers & Education,
51(2), 624–645.
Chen, L., Ho, R., & Yen, Y. (2010). Marking strategies in metacognition-evaluated
computer-based testing. Educational Technology & Society, 13(1), 246–259.
Chen, N., Hsieh, S., & Kinshuk, K. (2008). Effects of short-term memory and content
representation type on mobile language learning. Language Learning and Technology,
12(3), 93–113.
Coady, J. (1993). Research on ESL/EFL vocabulary acquisition: Putting it in context. In
T. Huckin, M. Haynes, & J. Coady (Eds.), Second Language Reading and Vocabulary
Learning (pp. 3–23). Norwood, NJ: Ablex Publishing.
Fujishiro, N., & Miyaji, I. (2010). The effects of blended instruction on oral reading
performance and their relationships to a five-factor model of personality. Knowledge
Management & E-Learning, 2(3), 225–245.
Gibbs, G. (1999). Using assessment strategically to change the way students learn. In S.
Brown & A. Glasner (Eds.), Assessment Matters in Higher Education (pp. 41–53).
Buckingham, UK: S.R.H.E. and Open University Press.
Hartley, S. (1978). Meta-analysis of the effects of individually paced instruction in
mathematics. Dissertation Abstracts International, 38(7-A), 4003.
Huang, H., & Liou, H. (2007). Vocabulary learning in an automated graded reading
program. Language Learning and Technology, 11(3), 64–82.
Huang, A. F. M., Yang, S. J. H., & Hwang, G. (2010). Situational language teaching in
ubiquitous learning environments. Knowledge Management & E-Learning, 2(3), 312–
327.
Jia, J., Chen, Y., Ding, Z., & Ruan, M. (2012). Effects of a vocabulary acquisition and
assessment system on students’ performance in a blended learning class for English
Knowledge Management & E-Learning: An International Journal, Vol.5, No.1. 41