AI Future 243
AI Future 243
AI Future 243
To cite this article: Hyangeun Ji, Insook Han & Yujung Ko (2023) A systematic review
of conversational AI in language education: focusing on the collaboration with
human teachers, Journal of Research on Technology in Education, 55:1, 48-63, DOI:
10.1080/15391523.2022.2142873
Review
Introduction
Artificial intelligence (AI) has gained tremendous attention recently as a potential tool to inno-
vate teaching and learning in educational settings. However, the idea of applying AI in education
is not entirely new. Since Turing (1950) explored the mathematical possibility of AI by suggesting
the Turing Machines, the development of AI and research on learning and educational applica-
tions have experienced rises and falls. In particular, Minsky and Papert (1968) pioneered AI
research and applied computational ideas to emulate human psychological processes in machines
so that a computer could solve problems and make decisions based on available information as
humans do. Since then, early research on AI and education focused on developing and testing
intelligent tutoring systems (ITS). Coupled with the pedagogical approach of learning by teaching,
some ITSs have evolved into teachable agents that help students learn while teaching computer
agents (Blair et al., 2007; Leelawong & Biswas, 2008; Silvervarg et al., 2021).
With a recent dramatic increase in the investment in AI industries (Pan, 2016), AI has gained
new momentum. It has driven the advancement of AI technology, such as machine learning,
neural network, natural language processing (NLP) with automatic speech recognition (ASR),
and advanced image processing (Zawacki-Richter et al., 2019) and increased access to commer-
cially available AI devices and mobile applications. Wider and easier access to AI technologies
allowed a potential increase in its application in educational settings. This was supported by a
CONTACT Insook Han [email protected] Department of Teaching and Learning, College of Education and Human
Development, Temple University, 439 Ritter Hall, 1301 Cecil B. Moore Ave, Philadelphia, PA19122-6091, USA
© 2022 ISTE
Journal of Research on Technology in Education 49
significant increase in the number of papers on this topic published since 2015 (Chen et al.,
2020). Indeed, a review by Chen et al. (2020) provided a wide array of educational scenarios,
where recent AI technologies have been applied in administration, instruction, and learning.
The scenarios included personalized intelligent teaching, assessment and evaluation, smart school,
and online and mobile remote education.
One notable example is the rapid growth in the development of chatbots (Adamopoulou &
Moussiades, 2020), a type of conversational AI that allows natural language conversations between
humans and machines by texts or voice. While ITS is often used as an umbrella term for
knowledge-based adaptive tutoring systems that often involve both pedagogical and conversational
abilities, conversational AI refers to a system that emphasizes the conversational abilities of an
ITS whose conversation usually takes place using NLP technology (Fadel et al., 2019). Conversational
AI has been increasingly adopted by educational researchers, particularly in language education,
with its potential to provide learners with chances to interact and support communicative lan-
guage teaching (Fryer et al., 2019; Hapsari & Wu, 2022). With the increasing interest, many
previous reviews that included conversational AI in language learning looked at its broad appli-
cation in education (Pérez et al., 2020). A few recent reviews solely focused on the use of
conversational AI in language learning (Chen et al., 2021; Huang et al., 2021; Liang et al., 2021).
However, these reviews concentrated on the effectiveness of conversational AI in language edu-
cation or viewed conversational AI either as a medium or a replacement of a human teacher.
Acknowledging the limitations of automated interventions with ITSs (Baker, 2016; Roschelle
et al., 2020), this special issue calls for rethinking the ways of using AI for amplifying human
intelligence and developing collaborative partnerships between humans and machines. Compared
to many promises that AI-enabled technologies could bring to language learning settings, few
guidelines are available for teachers on how they utilize AI technologies for enhancing student
learning (Kim et al., 2022). Previous language learning literature defined various teachers’ roles
(Alrayah, 2018; Brandl, 2007), some of which could be supported, enhanced or modified by
using AI technologies. Since the use of AI in classrooms is a rapidly emerging area, we need
to understand how teachers can collaborate with AI technologies while engaging in enacting
instructional roles to support student learning. Therefore, this study aims to explore how a
teacher uses a conversational AI, what instructional roles conversational AI play, and how it
supports or modifies traditional language teachers’ roles.
Previous studies
Potential of conversational AI as a language learning tool
The affordances of conversational AIs can be best supported by the interaction theory, which
emphasizes the language learners’ collaborative efforts to interact and communicate with another
speaker (Chapelle, 2005; Li, 2018; Long, 1996). However, language learners often have limited
chances to practice speaking and receive feedback (Terhune, 2016). Furthermore, chances for
language learners to engage in authentic communication are insufficient (Alharbi, 2015; Jo, 2008).
Recent advances in machine learning, ASR, and NLP technologies have made conversational
AI-integrated language learning a more suitable and economical approach. They provide language
learners access to language learning resources and an authentic environment for communication
in a target language.
Furthermore, conversational AI can reduce foreign language anxiety, a constant concern that
hinders language learning performance and achievement (Horwitz, 2001; Shao et al., 2019;
Teimouri et al., 2019). Shao et al. (2019) emphasized the role of students’ positive emotions such
as hope, pride, and contentment in language learners’ motivation and performance. It was also
argued that teachers should strive to reduce students’ foreign language anxiety in and outside
the language classroom. Among several techniques and interventions to reduce or overcome
language anxiety, drill and repetition (Chen & Chang, 2004; Gregersen et al., 2014), teachers’
positive feedback (Alnuzaili & Uddin, 2020) have been discussed as viable solutions by many
50 H. JI ET AL.
Methods
For the systematic review, we followed the four-phase process described in the Preferred Reporting
Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline (Liberati et al., 2009). The
process involved (a) identification of the possible studies, (b) abstract screening, (c) full-text
assessment for eligibility, and (d) qualitative synthesis of the included studies. Figure 1 provides
an overview of the process and the number of papers included and excluded at each phase.
Search procedures
To identify possible studies, six databases in the educational field (EBSCO, Wiley Online Library,
Science Direct, Web of Science, Taylor and Francis, and ProQuest) were searched. To reduce
publication bias, ProQuest was added to include conference proceedings and dissertations. We
limited the systematic literature search to articles published between January 2015 and January
2022, which is when the AI industry drove a dramatic technological advancement and related
papers increased in number. We conducted full-text searches using comprehensive keywords on
conversational AI (“conversational agents” OR “chatbot” OR “chat bot” OR “intelligent personal”
OR “intelligent agents”) and language learning (“language learning” OR “language teaching” OR
“language classroom” OR “language education”). Following the database search screening, a hand
search on the titles of the reference list from the screened articles was conducted.
Results
Study characteristics
Table 1 describes the overall characteristics of 24 articles included in this review. The number
of articles published each year seemed to be increasing between 2015 and 2021, with only a
slight drop in 2017 and 2018 (see Figure 2). Among the 24 studies, the majority of them were
journal articles (n = 17), followed by conference papers (n = 4), dissertations (n = 2), and a book
chapter (n = 1). Half of the reviewed articles employed a quantitative research design approach
(n = 12), immediately followed by a mixed-method approach (n = 11), and only one study con-
ducted qualitative research (n = 1).
More than half of the studies (n = 13) were performed in formal learning settings during
regular class time. However, five studies were set in informal learning environments, and par-
ticipants used AI technology more freely. Also, another five studies were set in lab environments.
Figure 3 reveals that the reviewed studies were dominantly conducted with learners pursuing
higher education (n = 20). The rest of the studies had participants from elementary education,
secondary education, and adult learning, each with one study that did not specify the education
level of the learners.
In most of the reviewed articles, the target language for learning was English (n = 20), with
learners studying English as a second or foreign language. The participants in the remaining
four articles intended to learn Chinese, French, Japanese, and Russian. In measuring the impact
of language learning with conversational AI, some studies focused solely on one outcome while
others employed multiple outcome measurements, which made duplicate counting for outcome
measurement for each study. In most studies (n = 17), speaking was the main focus, and
54
(2015)
Bao (2019) Journal Article Quantitative Informal learning Adult Learning English Speaking, Perception Conversation partner (oral)
Cai et al. (2020) Conference Paper Quantitative Formal learning Higher Education Japanese Speaking Feedback provider
Divekar (2020) Dissertation Mixed-methods Informal learning Higher Education Chinese Speaking, Listening, Conversation partner (oral)
Vocabulary,
Perception
Dizon (2020) Journal Article Quantitative Formal learning Higher Education English Speaking, Listening, Conversation partner (oral)
Perception
El Shazly (2021) Journal Article Mixed-methods Formal learning Higher Education English Speaking Conversation partner (written/
oral), Resource provider
Fryer et al. (2017) Journal Article Quantitative Formal learning Higher Education English Interest Conversation partner (oral)
Fryer et al. (2019) Journal Article Mixed-methods Lab Higher Education English Listening, Reading, Conversation partner (oral)
Interest, Perception
Fryer et al. (2020) Journal Article Quantitative Formal learning Higher Education English Interest, self-efficacy, Conversation partner (oral)
self-concept
Hassani et al. (2016) Journal Article Quantitative Lab Higher Education English Speaking, Listening Conversation partner (oral),
Evaluator
Hsu et al. (2021a) Journal Article Mixed-methods Formal learning Higher Education English Speaking, Listening, Conversation partner (oral),
Perception Resource provider
Hsu et al. (2021b) Journal Article Quantitative Informal learning Higher Education English Speaking Conversation partner (oral),
Feedback provider
Jeon (2021) Journal Article Mixed-methods Formal learning Elementary English Vocabulary Feedback provider, Needs
Education analyst, Evaluator
Kılıçkaya (2020) Book Chapter Qualitative Formal learning Higher Education English Perception Conversation partner (written)
Kim (2016) Journal Article Quantitative Not specified Higher Education English Speaking, Perception Conversation partner (oral)
Liakin et al. (2014) Journal Article Quantitative Informal learning Higher Education French Speaking Conversation partner (oral),
Feedback provider
Lin and Mubarok Journal Article Mixed-methods Formal learning Higher Education English Speaking, Behaviors Conversation partner (oral),
(2021) Resource provider
McCrocklin (2016) Journal Article Mixed-methods Formal learning Higher Education English Beliefs, Behaviors Conversation partner (oral)
Moussalli and Conference Paper Mixed-methods Lab Not specified English Perception Conversation partner (oral)
Cardoso (2016)
Ruan (2021) Dissertation Quantitative Lab Higher Education English Speaking, Vocabulary Conversation partner (oral),
Feedback provider
Tai and Chen (2020) Journal Article Mixed-methods Formal learning Secondary Education English Speaking, Perception Conversation partner (oral)
Thompson et al. Conference Paper Mixed-methods Formal learning Higher Education English Interest, Perception Conversation partner (oral)
(2018)
Yin and Satar (2020) Journal Article Mixed-methods Informal learning Higher Education English Speaking, Perception Conversation partner (written/
oral)
Journal of Research on Technology in Education 55
perception of language learning with conversational AI was also of interest (n = 11). Other out-
come measurements included listening (n = 5), vocabulary (n = 4), interest (n = 4), behavior (n = 2),
reading (n = 1), grammar (n = 1), self-efficacy (n = 1), self-concept (n = 1), and belief (n = 1).
and songs to learners to aid speaking activities (Lin & Mubarok, 2021). Subsequently, as an
evaluator, conversational AI provided diagnostic information about learners’ vocabulary ability
(Jeon, 2021) or estimated learners’ conversational proficiency level (Hassani et al., 2016).
Furthermore, Hassani et al. (2016) established that conversational AI adjusted its proficiency
level and offered adaptive feedback based on the evaluations it made. Lastly, as a needs analyst,
conversational AI tailored instructions based on the learner’s specific needs (Jeon, 2021). For
example, if a learner needed an implicit prompt, the conversational AI asked the learner to read
the sentence again and guess the meaning. If a learner needed a more explicit prompt, it pro-
vided another sentence that includes the word in a new, more specific context.
Beyond the roles defined by Brandl (2007), two additional roles emerged from the data
analysis. They included the conversation partner role in a written or oral form and the feedback
provider role which provided learners with individualized feedback. The most prominent role
of conversational AI was a conversational partner. Most studies (n = 22, 91.7%) used conversa-
tional AI as a conversational partner for speaking (n = 19) using the learners’ voice input, for
writing tasks (n = 1) using the learners’ text input, or both (n = 2, El Shazly, 2021; Yin & Satar,
2020). In these studies, conversational AIs produced audio and textual outputs with facial
expressions. For example, in El Shazly (2021), language learners in higher education practiced
their pronunciation using a task-oriented dialogue chatbot and extensively interacted with web
chatbots for written conversations. Both chatbots employed NLP and human-like intelligence,
allowing participants to receive acceptable comprehensible input and produce multiple foreign
language outputs. In addition to providing language practice opportunities, conversational AI
was designed as a conversational partner to help language learners with foreign language learning
anxiety (Bao, 2019), willingness to communicate (Ayedoun et al., 2015; Tai & Chen, 2020),
language course/task interest (Fryer et al., 2017; Thompson et al., 2018), language course
self-efficacy (Fryer et al., 2020), language learning interest (Fryer et al., 2019), autonomous
learning behavior (McCrocklin, 2016), and negotiation for meaning (Yin & Satar, 2020).
Another remarkable role identified was feedback provider. In five studies (Cai et al., 2020; Hsu
et al., 2021b; Jeon, 2021; Liakin et al., 2014; Ruan, 2021), conversational AI provided individual
feedback to each learner through implicit/explicit prompts, voice messages, feedback sentences, or
written visuals. Notably, conversational AI in the two studies (Jeon, 2021; Ruan, 2021) modified
its feedback based on the learners’ input. For example, in Ruan (2021), conversational AI provided
adaptive feedback by comparing learners’ responses with the correct response using semantic
similarity, length of the response, and classifying the responses into three performance levels, which
was effective in engaging, motivating learners, and improving language learning outcomes.
Additionally, four studies addressed psychological challenges. El Shazly (2021) reported that
conversational AI in the study may have failed to match the learners’ topical interests and meet
their emotional needs. Similarly, learners in Kılıçkaya (2020) pointed out the lack of emotional
expressions in conversational AI. Furthermore, Fryer et al. (2017) and Fryer et al. (2019) found
that students lost interest in conversing with the chatbot after the initial practice, suggesting
the possibility of a novelty effect. To tackle these challenges, El Shazly (2021) and Fryer et al.
(2019) commonly emphasized the role of teachers to maintain learner interest and provide
support and empathy in social interactions. Finally, technical difficulties were found in Kılıçkaya
(2020), where learners had connection problems and experienced learning application crashes.
Discussion
This systematic review aimed to examine (1) the roles of conversational AI and teachers iden-
tified in the current language learning literature and (2) how collaboration between conversational
AI and language teachers could be achieved to tackle the challenges identified in the literature
and help progress toward intelligence amplification.
of current language instruction identified in previous studies, such as foreign language learning
anxiety (Hsu et al., 2021b), lack of individualized feedback (Luan & Tsai, 2021), or lack of
opportunity for practicing speaking (Muhammad et al., 2020). By providing emotionally safe
learning environments where language learners can practice conversation with individualized
feedback, conversational AI can support communicative practices.
Unlike the prominent roles of conversational AIs, the roles of teachers were not clearly defined
in the majority of studies. This finding is a stark contrast to many arguments that emphasize the
role of teachers in AIEd (Baker, 2016; Chaudhry & Kazim, 2022; Holstein et al., 2020; Kim et al.,
2022; Zheng, 2020). However, our study provides insights into human teachers’ roles in language
classrooms. According to the results of our study, conversational AIs are rarely involved in the
designing, evaluating, and decision-making processes. In these instructional processes, human
intelligence is required for interpreting student data, designing learning activities, or making deci-
sions to adapt to instructional situations emerged during teaching (Cukurova et al., 2019). Therefore,
we expect that teachers continue performing traditional language teacher’s roles, such as presenting
new content, providing learning tasks, facilitating activities, and leading discussions.
Moreover, teachers can play a significant role in addressing the challenges of conversational
AI-integrated language learning identified in the current study. Our findings revealed that the
use of conversational AI without human inputs could render several challenges: interaction
challenges that involve misinterpretation of learner inputs and conversations that are off-topic,
not goal-oriented, or less immersive and social; psychological challenges that fail to meet learners’
interest and emotional needs; and technical difficulties, such as connection problems or appli-
cation crashes. In dealing with these challenges, teachers can play a unique role as a facilitator
and feedback provider by offering meaningful language practices, feedback, and emotional support
as suggested in some of the reviewed studies (Divekar, 2020; El Shazly, 2021; Jeon, 2021).
Similarly, in the report based on the panel discussion on AI and learning, Roschelle et al.
(2020) suggested practical guidelines for classroom orchestration that can be used in the appli-
cation, extension, and follow-up or synthesis phases. Although the report was not tailored for
language learning, the researchers argued that future conversational AI-integrated learning envi-
ronments should focus more on social learning rather than highly individualistic learning. For
example, the report illustrated a middle school classroom lesson designed to explore the 16th
century Venice in a virtual classroom with an AI tour guide and how this guide could assist
human teachers’ group process manager, needs analyst, evaluator, and facilitator roles. It was
suggested that teachers and AI collaborate in forming groups, nurturing better conversations,
grading essays, responding to students’ emotions, and lesson planning. During this collaboration,
human teachers, who have better knowledge about students’ backgrounds and administrative
aspects, could form initial groups, while AIs help teachers by providing anticipated issues and
suggestions. Furthermore, teachers could utilize the AI’s learning analytics data about students’
sequence of behavior and emotions collected in the virtual reality simulation, which otherwise
would have been difficult to pick up.
Conclusion
In contrast to many emphases made by AIEd researchers on intelligence amplification and
human-computer collaboration, our review revealed that the collaboration between conversational
AI and teachers is not strongly supported by the current empirical studies. Nonetheless, researchers
in many reviewed studies were conscious of the challenges of conversational AI-integrated lan-
guage classrooms and provided insights regarding ways wherein teachers and conversational AIs
could collaborate in future studies. Roschelle et al. (2020) expected that stronger orchestration
would be required in future classrooms. Accordingly, it is imperative for language teachers to be
60 H. JI ET AL.
equipped with instructional strategies that maximize human-computer collaboration from design
and implementation to evaluation of language lessons. To support teachers, AIEd researchers
should be able to provide practical guidelines through conducting evidence-based research.
Disclosure statement
The author declares that this study does not present any conflict of interest.
Notes on contributors
Hyangeun Ji is Ph.D. student and research assistant in Science, Math, and Educational Technology at Temple
University. Before coming to Temple, she worked as a computer programmer and EFL teacher in South Korea.
Her research interest includes but is not limited to instructional technology, computer-assisted language learning,
online and virtual learning, and artificial intelligence in education.
Insook Han was Associate Professor in the Department of Teaching and Learning at Temple University and is
currently at Korea University. Her major research interests involve the design of learning environments, embodied
cognition, emerging technologies in classrooms, and pre-service education for technology integration.
Yujung Ko, is Assistant Professor in the Department of Platform EdTech, Division of Social Sciences at Hanyang
Cyber University. Her research interests focus on the educational use of emerging technologies, the design of
learning with technology, and technology integration of pre- and in-service teachers.
ORCID
Hyangeun Ji http://orcid.org/0000-0002-0257-074X
Insook Han http://orcid.org/0000-0002-3344-3356
Yujung Ko http://orcid.org/0000-0002-7535-5549
References
* indicates the selected studies for review
Adamopoulou, E., & Moussiades, L. (2020). Chatbots: History, technology, and applications. Machine Learning
with Applications, 2, 100006. https://doi.org/10.1016/j.mlwa.2020.100006
Journal of Research on Technology in Education 61
* Al-Kaisi, A. N., Arkhangelskaya, A. L., & Rudenko-Morgun, O. I. (2019). Voice assistants as a training tool in
a foreign language class [Paper presentation]. INTED2019–13th International Technology, Education and
Development Conference Proceedings, Valencia. 1236–1246. https://doi.org/10.21125/inted.2019.0399
Alharbi, H. A. (2015). Improving students’ English speaking proficiency in Saudi public schools. International
Journal of Instruction, 8(1), 105–116. https://doi.org/10.12973/iji.2015.818a
Alnuzaili, E. S., & Uddin, N. (2020). Dealing with anxiety in foreign language learning classroom. Journal of
Language Teaching and Research, 11(2), 269–273. https://doi.org/10.17507/jltr.1102.15
Alrayah, H. (2018). The effectiveness of cooperative learning activities in enhancing EFL learners’ fluency. English
Language Teaching, 11(4), 21–31. https://doi.org/10.5539/elt.v11n4p21
* Ayedoun, E., Hayashi, Y., & Seta, K. (2015). A conversational agent to encourage willingness to communicate
in the context of English as a foreign language. Procedia Computer Science, 60, 1433–1442. https://doi.
org/10.1016/j.procs.2015.08.219
Baker, R. S. (2016). Stupid tutoring systems, intelligent humans. International Journal of Artificial Intelligence in
Education, 26(2), 600–614. https://doi.org/10.1007/s40593-016-0105-0
* Bao, M. (2019). Can home use of speech-enabled artificial intelligence mitigate foreign language anxiety –
Investigation of a concept. Arab World English Journal, 5, 28–40. https://doi.org/10.24093/awej/call5.3
Belur, J., Tompson, L., Thornton, A., & Simon, M. (2021). Interrater reliability in systematic review methodolo-
gy: Exploring variation in coder decision-making. Sociological Methods & Research, 50(2), 837–865. https://
doi.org/10.1177/0049124118799372
Bibauw, S., François, T., & Desmet, P. (2019). Discussing with a computer to practice a foreign language: Research
synthesis and conceptual framework of dialogue-based CALL. Computer Assisted Language Learning, 32(8),
827–877. https://doi.org/10.1080/09588221.2018.1535508
Blair, K., Schwartz, D. L., Biswas, G., & Leelawong, K. (2007). Pedagogical agents for learning by teaching:
Teachable agents. Educational Technology & Society, 47, 56–61.
Bushweller, K. (2020, January 7). Teachers, the robots are coming. But that’s not a bad thing. Education Week.
https://www.edweek.org/technology/teachers-the-robots-are-coming-but-thats-not-a-bad-thing/2020/01
Bryant, J., Heitz, C., Sanghvi, S., & Wagle, D. (2020). How artificial intelligence will impact K-12 teachers. McKinsey
& Company.
Brandl, K. (2007). Communicative language teaching in action: Putting principles to work. Pearson.
* Cai, M. Y., Wang, J. Y., Chen, G. D., Wang, J. H., & Yang, S. H. (2020). A digital reality theater with the mech-
anisms of real-time spoken language evaluation and interactive switching of scenario & virtual costumes: Effects on
motivation and learning performance [Paper presentation]. 2020 IEEE 20th International Conference on Advanced
Learning Technologies (ICALT), July (pp. 295–299). IEEE. https://doi.org/10.1109/ICALT49669.2020.00096
Chapelle, C. (2005). Interactionist SLA theory in CALL research. In J.L. Egbert & G.M. Petrie (Eds.), CALL
research perspectives (pp. 53–64). Lawrence Erlbaum Associates.
Chatti, M. A., Muslim, A., Guesmi, M., Richtscheid, F., Nasimi, D., Shahin, A., & Damera, R. (2020). How to
design effective learning analytics indicators? A human-centered design approach [Paper presentation]. In European
Conference on Technology Enhanced Learning, September (pp. 303–317). Springer.
Chaudhry, M. A., & Kazim, E. (2022). Artificial intelligence in education (AIEd): A high-level academic and
industry note 2021. AI and Ethics, 2(1), 157–165. https://doi.org/10.1007/s43681-021-00074-z
Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. IEEE Access. 8, 75264–75278.
https://doi.org/10.1109/ACCESS.2020.2988510
Chen, T. Y., & Chang, G. B. (2004). The relationship between foreign language anxiety and learning difficulties.
Foreign Language Annals, 37(2), 279–289. https://doi.org/10.1111/j.1944-9720.2004.tb02200.x
Chen, X., Zou, D., Cheng, G., & Xie, H. (2021). Artificial intelligence-assisted personalized language learning:
Systematic review and co-citation analysis [Paper presentation]. 2021 International Conference on Advanced
Learning Technologies (ICALT), July (pp. 241–245). IEEE. https://doi.org/10.1109/ICALT52272.2021.00079
Cukurova, M., Kent, C., & Luckin, R. (2019). Artificial intelligence and multimodal data in the service of human
decision‐making: A case study in debate tutoring. British Journal of Educational Technology, 50(6), 3032–3046.
https://doi.org/10.1111/bjet.12829
Dillenbourg, P., & Jermann, P. (2010). Technology for classroom orchestration. In M. S. Khine, & I. M. Saleh
(Eds.), New science of learning (pp. 525–552). Springer Science + Business Media.
* Divekar, R. R. (2020). AI enabled foreign language immersion: Technology and method to acquire foreign lan-
guages with AI in immersive virtual worlds [Doctoral dissertation]. Rensselaer Polytechnic Institute.
Divekar, R. R., Lepp, H., Chopade, P., Albin, A., Brenner, D., Ramanarayanan, V. (2021). Conversational agents
in language education: Where they fit and their research challenges. International Conference on Human-Computer
Interaction (pp. 272–279). Springer.
* Dizon, G. (2020). Evaluating intelligent personal assistants for L2 listening and speaking development. Language
Learning & Technology, 24(1), 16–26.
* El Shazly, R. (2021). Effects of artificial intelligence on English speaking anxiety and speaking performance: A
case study. Expert Systems, 38(3), e12667. https://doi.org/10.1111/exsy.12667
Fadel, C., Holmes, W., & Bialik, M. (2019). Artificial intelligence in education: Promises and implications for
teaching and learning. The Center for Curriculum Redesign.
62 H. JI ET AL.
* Fryer, L. K., Ainley, M., Thompson, A., Gibson, A., & Sherlock, Z. (2017). Stimulating and sustaining interest
in a language course: An experimental comparison of Chatbot and Human task partners. Computers in Human
Behavior, 75, 461–468. https://doi.org/10.1016/j.chb.2017.05.045
* Fryer, L. K., Nakao, K., & Thompson, A. (2019). Chatbot learning partners: Connecting learning experiences,
interest and competence. Computers in Human Behavior, 93, 279–289. https://doi.org/10.1016/j.chb.2018.12.023
* Fryer, L. K., Thompson, A., Nakao, K., Howarth, M., & Gallacher, A. (2020). Supporting self-efficacy beliefs
and interest as educational inputs and outcomes: Framing AI and human partnered task experiences. Learning
and Individual Differences, 80, 101850. https://doi.org/10.1016/j.lindif.2020.101850
Gregersen, T., Macintyre, P. D., & Meza, M. D. (2014). The motion of emotion: Idiodynamic case studies of learn-
ers’ foreign language anxiety. The Modern Language Journal, 98(2), 574–588. https://doi.org/10.1111/modl.12084
Hapsari, I. P., Wu, T. T. (2022). AI chatbots learning model in English speaking skill: Alleviating speaking anx-
iety, boosting enjoyment, and fostering critical rhinking. International Conference on Innovative Technologies
and Learning (pp. 444–453). Springer.
* Hassani, K., Nahvi, A., & Ahmadi, A. (2016). Design and implementation of an intelligent virtual environment
for improving speaking and listening skills. Interactive Learning Environments, 24(1), 252–271. https://doi.org
/10.1080/10494820.2013.846265
Hedayati, M., Reynolds, B., & Bown, A. (2017). Investigating CALL teachers’ contextualised views of their evolving
roles [Paper presentation]. In Proceedings of the 10th International Conference ICT for Language Learning,
October) (pp. 184–188). Libreria Universitaria.
Holstein, K., Aleven, V., Rummel, N. (2020). A conceptual framework for human–AI hybrid adaptivity in edu-
cation. In International Conference on Artificial Intelligence in Education (pp. 240–254). Springer.
Holstein, K., McLaren, B. M., & Aleven, V. (2019). Co-designing a real-time classroom orchestration tool to support
teacher–AI complementarity. Journal of Learning Analytics, 6(2), 27–52. https://doi.org/10.18608/jla.2019.62.3
Horwitz, E. (2001). Language anxiety and achievement. Annual Review of Applied Linguistics, 21, 112–126. https://
doi.org/10.1017/S0267190501000071
* Hsu, H. L., Chen, H. H. J., & Todd, A. G. (2021a). Investigating the impact of the Amazon Alexa on the
development of L2 listening and speaking skills. Interactive Learning Environments, 1–14. https://doi.org/10.1
080/10494820.2021.2016864
* Hsu, M. H., Chen, P. S., & Yu, C. S. (2021b). Proposing a task-oriented chatbot system for EFL learners
speaking practice. Interactive Learning Environments, 1–12. https://doi.org/10.1080/10494820.2021.1960864
Huang, W., Hew, K. F., & Fryer, L. K. (2021). Chatbots for language learning—Are they really useful? A system-
atic review of chatbot‐supported language learning. Journal of Computer Assisted Learning, 38(1), 1–21.
Istrate, A. M. (2018). Artificial intelligence and machine learning—Future trends in teaching ESL and ESP.
eLearning & Software for Education, 2, 471–476.
* Jeon, J. (2021). Chatbot-assisted dynamic assessment (CA-DA) for L2 vocabulary learning and diagnosis.
Computer Assisted Language Learning, 1–27. https://doi.org/10.1080/09588221.2021.1987272
Jo, S. (2008). English education and teacher education in South Korea. Journal of Education for Teaching, 34(4),
371–381. https://doi.org/10.1080/02607470802401594
* Kılıçkaya, F. (2020). Using a chatbot, Replika, to practice writing through conversations in L2 English: A Case
study. In New Technological Applications for Foreign and Second Language Learning and Teaching (pp. 221–238).
IGI Global.
Kim, J., Lee, H., & Cho, Y. H. (2022). Learning design to support student-AI collaboration: Perspectives of
leading teachers for AI in education. Education and Information Technologies, 27, 6069–6104. https://doi.
org/10.1007/s10639-021-10831-6
* Kim, N. Y. (2016). Effects of voice chat on EFL learners’ speaking ability according to proficiency levels.
Multimedia-Assisted Language Learning, 19(4), 63–88.
Kinginger, C. (2001). Sociocultural approaches to teaching and teachers’ research. In R. Z. Lavine (Ed.), Beyond
the boundaries: Changing contexts in language learning (pp. 201–226). McGraw-Hill and Northeast Conference
Reports.
Kukulska-Hulme, A., & Lee, H. (2020). Intelligent assistants in language learning: An analysis of features and
limitations. In K.-M. Frederiksen, S. Larsen, L. Bradley & S. Thouësny (Eds), CALL for widening participation:
Short papers from EUROCALL 2020 (pp. 172–176). Research-publishing.net.
Leelawong, K., & Biswas, G. (2008). Designing learning by teaching agents: The Betty’s Brain system. International
Journal of Artificial Intelligence in Education, 18(3), 181–208.
Li, S. (2018). Corrective feedback in L2 speech production. The TESOL Encyclopedia of English Language Teaching,
1–9.
* Liakin, D., Cardoso, W., & Liakina, N. (2014). Learning L2 pronunciation with a mobile speech recognizer:
French/y/. CALICO Journal, 32(1), 1–25. https://doi.org/10.1558/cj.v32i1.25962
Liang, J. C., Hwang, G. J., Chen, M. R. A., & Darmawansah, D. (2021). Roles and research foci of artificial
intelligence in language education: An integrated bibliographic analysis and systematic review approach.
Interactive Learning Environments, 1–27.
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux,
P. J., Kleijnen, J., & Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses
Journal of Research on Technology in Education 63
of studies that evaluate health care interventions: Explanation and elaboration. Journal of Clinical Epidemiology,
62(10), e1–e34.
* Lin, C. J., & Mubarok, H. (2021). Learning analytics for investigating the mind map-guided AI chatbot approach
in an EFL flipped speaking classroom. Educational Technology & Society, 24(4), 16–35.
Long, M. H. (1996). The role of the linguistic environment in second language acquisition. In W. C. Ritchie &
T. K. Bhatia (eds.), Handbook of research on language acquisition (pp. 413–468). Academic Press.
Luan, H., & Tsai, C. C. (2021). A review of using machine learning approaches for precision education. Educational
Technology & Society, 24(1), 250–266.
Mayring, P. (2000). Qualitative content analysis. In U. Flick, E. Kardoff, & I. Steinke (Eds.), A companion to
qualitative research (pp. 266–269). Sage.
* McCrocklin, S. M. (2016). Pronunciation learner autonomy: The potential of automatic speech recognition.
System, 57, 25–42. https://doi.org/10.1016/j.system.2015.12.013
Minsky, M. L., & Papert, S. A. (1968). Perceptrons: An introduction to computational geometry. MIT Press.
* Moussalli, S., & Cardoso, W. (2016). Are commercial ‘personal robots’ ready for language learning? Focus on
second language speech. In S. Papadima-Sophocleous, L. Bradley, & S. Thouesny (Eds.), CALL communities
and culture – short papers from EUROCALL 2016 (pp. 325–329). Research-publishing.net.
Muhammad, A. F., Susanto, D., Alimudin, A., Adila, F., Assidiqi, M. H., & Nabhan, S. (2020). Developing English
conversation chatbot using Dialogflow [Paper presentation]. 2020 International Electronics Symposium (IES),
September (pp. 468–475). IEEE. https://doi.org/10.1109/IES50839.2020.9231659
Pan, Y. (2016). Heading toward artificial intelligence 2.0. Engineering, 2(4), 409–413. https://doi.org/10.1016/J.
ENG.2016.04.018
Pérez, J. Q., Daradoumis, T., & Puig, J. M. M. (2020). Rediscovering the use of chatbots in education: A sys-
tematic literature review. Computer Applications in Engineering Education, 28(6), 1549–1565. https://doi.
org/10.1002/cae.22326
Roschelle, J., Dimitriadis, Y., & Hoppe, U. (2013). Classroom Orchestration: Synthesis. Computers & Education,
69, 523–526.
Roschelle, J., Lester, J., & Fusco, J. (2020). AI and the future of learning: Expert panel report. Digital Promise.
* Ruan, S. S. (2021). Smart tutoring through conversational interfaces [Doctoral dissertation]. Stanford University.
Shao, K., Pekrun, R., & Nicholson, L. J. (2019). Emotions in classroom language learning: What can we learn
from achievement emotion research? System, 86, 102121. https://doi.org/10.1016/j.system.2019.102121
Silvervarg, A., Wolf, R., Blair, K. P., Haake, M., & Gulz, A. (2021). How teachable agents influence students’
responses to critical constructive feedback. Journal of Research on Technology in Education, 53(1), 67–88. https://
doi.org/10.1080/15391523.2020.1784812
Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook
Messenger. Computers & Education, 151, 103862. https://doi.org/10.1016/j.compedu.2020.103862
* Tai, T. Y., & Chen, H. H. J. (2020). The impact of Google Assistant on adolescent EFL learners’ willingness to
communicate. Interactive Learning Environments, 1–18. https://doi.org/10.1080/10494820.2020.1841801
Teimouri, Y., Goetze, J., & Plonsky, L. (2019). Second language anxiety and achievement: A meta-analysis. Studies
in Second Language Acquisition, 41(2), 363–387. https://doi.org/10.1017/S0272263118000311
Terhune, N. M. (2016). Language learning going global: Linking teachers and learners via commercial Skype-based
CMC. Computer Assisted Language Learning, 29(6), 1071–1089. https://doi.org/10.1080/09588221.2015.1061020
* Thompson, A., Gallacher, A., Howarth, M. (2018). Stimulating task interest: Human partners or chatbots?.
Proceedings of the Future-Proof CALL: Language Learning as Exploration and Encounters, 302–306.
Turing, A. M. (1950). Computing machinery and intelligence. Mind, LIX(236), 433–460. https://doi.org/10.1093/
mind/LIX.236.433
Wang, Y. (2021). When artificial intelligence meets educational leaders’ data-informed decision-making: A cau-
tionary tale. Studies in Educational Evaluation, 69, 100872. https://doi.org/10.1016/j.stueduc.2020.100872
* Yin, Q., & Satar, M. (2020). English as a foreign language learner interactions with chatbots: Negotiation for
meaning. International Online Journal of Education and Teaching, 7(2), 390–410.
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial
intelligence applications in higher education–Where are the educators? International Journal of Educational
Technology in Higher Education, 16(1), 1–27. https://doi.org/10.1186/s41239-019-0171-0
Zeira, J. (1998). Workers, machines, and economic growth. The Quarterly Journal of Economics, 113(4), 1091–1117.
https://doi.org/10.1162/003355398555847
Zhang, R., & Zou, D. (2020). Types, purposes, and effectiveness of state-of-the-art technologies for second and
foreign language learning. Computer Assisted Language Learning, 35(4), 696–742.
Zheng, G. (2020). The optimization and application of blended teaching based on artificial intelligence [Paper
presentation]. 2020 3rd International Conference on Advanced Electronic Materials, Computers and Software
Engineering (AEMCSE), April (pp. 71–74). IEEE. https://doi.org/10.1109/AEMCSE50948.2020.00023