Received July 10, 2020, accepted August 17, 2020, date of publication September 18, 2020, date of current

version October 13, 2020.

Digital Object Identifier 10.1109/ACCESS.2020.3024795

JAICOB: A Data Science Chatbot

DANIEL CARLANDER-REUTERFELT , ÁLVARO CARRERA , CARLOS A. IGLESIAS ,
ÓSCAR ARAQUE , JUAN FERNANDO SÁNCHEZ RADA , AND SERGIO MUÑOZ
Grupo de Sistemas Inteligentes, Universidad Politécnica de Madrid, 28040 Madrid, Spain
Corresponding author: Álvaro Carrera ([email protected])
This work was supported by the Educational Innovation Programme of Universidad Politécnica de Madrid under Grant IE1819.0908.

ABSTRACT The application of natural language to improve students’ interaction with information systems
is demonstrated to be beneficial. In particular, advances in cognitive computing enable a new way of interac-
tion that accelerates insight from existing information sources, thereby contributing to the process of learning.
This work aims at researching the application of cognitive computing in blended learning environments. We
propose a modular cognitive agent architecture for pedagogical question answering, featuring social dialogue
(small talk), improved for a specific knowledge domain. This system has been implemented as a personal
agent to assist students in learning Data Science and Machine Learning techniques. Its implementation
includes the training of machine learning models and natural language understanding algorithms in a human-
like interface. The effectiveness of the system has been validated through an experiment.

INDEX TERMS Cognitive informatics, educational technology, human-computer interaction, machine

learning, natural language processing.

I. INTRODUCTION are: (1) They can actively enhance students’ perfor-

Cognitive computing has grown in the last few years, increas- mances [5], especially in computer science classes [6]; (2)
ing the research and commercial interest in the topic [1]. Con- studying cognitive computing behavior can lead to significant
versational agents have evolved from simple pattern-based results in educational applications, especially in AI-related
programs into rather complex systems, including Natural studies [6]; (3) using a cognitive computing layer for digital
Language Understanding and Machine Learning Techniques, interactions with students can enhance their performances
which have allowed them to be more flexible in maintaining and ease the teachers’ job in managing classes and learning
a conversation. Every day more businesses include chatbots materials [6]; and (4) chatbots are excellent analysis tools,
as a way to interact with consumers to answer requests and as students feel more inclined to send more messages to
FAQs. Natural Language Interface (NLI) increases user sat- chatbots than real people [2].
isfaction and can help to find the information needed in a Compared to other traditional e-learning training, chatbots
more comfortable way than other less sophisticated and time- generate a more positive response from the users [7]. More-
consuming search interfaces [2]. over, there are advantages in this type of learning, such as
Like humans, cognitive systems can use their knowledge interaction, active learning, and sociability [8].
to deduce data meaning based on context [3]. By having the Despite these reasons, these technologies have not been
advantage of computational power, a system like this can widely adopted yet in education, and the ones that have
be even more successful than a human in this kind of task. are usually very rule-based and, therefore, less practical
Though they do not understand the meaning as humans do, and functional. This article presents a modular architec-
the insights these systems provide can be beneficial. As they ture chatbot named Jaicob, adapted to the learning of Data
grow in time, it is expected that they gain abilities such as Science techniques that aims to take advantage of all the
sensing and awareness [4]. benefits for education previously described. It is designed
Some of the benefits of the application of cognitive in a modular way that allows its adaptation to other areas
computing in the development of learning applications of knowledge. It includes a flexible conversation workflow
and is easy to maintain. This contribution has been eval-
The associate editor coordinating the review of this manuscript and uated with real users for a specific use case in a Data
approving it for publication was Tony Thomas. Science class.

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
180672 VOLUME 8, 2020
D. Carlander-Reuterfelt et al.: JAICOB: A Data Science Chatbot

The rest of the paper is organized as follows. Section II significantly outperform the most carefully crafted rule-based
analyses related works about chatbots and the techniques systems [24]. The reason is that they can achieve a more
applied in their development. Section III describes the dif- profound understanding of the intent and the requested infor-
ferent modules of the architecture and how they are inter- mation, thanks to machine learning techniques [25]. The
connected. Section IV describes the evaluation process and most usual and effective approach [26], which is explained
results. Finally, Section V summarizes the learnings of this in greater detail in Section III, is based on intent-entity and
article with conclusions and defining future works. Knowledge Base (KB).
Another aspect to take into account is if they are text or
II. RELATED WORK voice-based. Users tend to use longer sentences with voice-
A comprehensive systematic review of the use of chatbots in based chatbots and prefer reading expanded answers in a text-
education is provided in this recent survey [9]. The authors like manner. However, there is no significant difference in
identify several perspectives for analyzing current research perceived effectiveness, learnability, and humanness between
following the theoretical model of Technology-Mediated text-based and voice-based chatbots [27].
Learning (TML) [10]: structure (input), learning process
(process), and learning outcome (output). Regarding the input III. PROPOSED ARCHITECTURE FOR THE COGNITIVE BOT
perspective, several dimensions have been identified [9]: stu- The first step to design the proposed architecture was to
dent profile, educational settings, and chatbot technology. identify the way students learn and the types of questions.
Learning outputs depend on individual student characteris- Different types of requirements for different types of learning
tics such as personality traits, technological skills as well as (inductive and deductive) [28] were identified due to the
educational and social background [9]. nature of students’ curiosity, and the specifics of the topic.
Some research works claim that chatbot technology is so The following pedagogical solutions were identified:
disruptive that it will eliminate the need for websites and
apps [11]. Chatbots have been used in different educational 1) A definition of a concept is a consequence of the
settings, such as language learning [12], health-related coach- usual teaching style, which is deductive, starting from
ing agents [13], chatbots designed to provide feedback to the main concepts and developing towards the applica-
students [14], programming language learning [15], admin- tions. It is part of the process of learning, but cannot be
istrative support [16] or increase students’ motivation [17]. the whole process. In the Oliver model [29], definitions
These are examples if we don’t take into account open- provide learning content.
domain solutions such as Amazon or Google’s [18], which 2) As stated in [30], the learning of programming tech-
aim to answer any kind of question, instead of a specific area niques can be enhanced by using examples of code
of knowledge. While these types of chatbots are astoundingly using analogy [31] and induction. Also, learning is
ambitious and function with a near-human precision, some- significantly facilitated by examples in initial cod-
times, they come at a very high cost. Closed-domain question ing attempts. Furthermore, surveys suggest that engi-
answering systems benefit from the ability to respond with neering students usually view themselves as inductive
more profound and specific knowledge [19], and also can learners [28]. In the Oliver model [29], examples can
achieve high quality at a lower complexity cost. provide learner support.
Design aspects of chatbots can influence the learning pro- 3) Lastly, the human need for small-talk, such as joking
cess. Flow-based chatbots, like [20]–[22], also called rule- and asking for the weather, must be satisfied to provide
based, can require an extensive database of questions and a more significant communication source [2].
answers and need to have a clear flow of conversation that, With that in mind, the architecture was designed, having
if the user decides not to follow, can result in a bad experience. identified the pedagogical needs of the student. There are
A study on chatbots of this type [23] concludes that they are several steps involved in the process and are explained below
quite limited to human direction and control. These can be and represented in Figure 1.
built with frameworks like Landbot.ai,1 or with simple coding A Knowledge Base (KB) was populated with pertinent
abilities, but require great sophistication to work correctly. information regarding the topic at hand, to satisfy the requests
There is where its limitations lie. An extension to this kind for definitions and examples. The Question Answering (QA)
of bots is button-based, like HelloFreshus,2 that avoids the module is designed to extract meaning from all the data with
possibility of exiting the pre-planned flow. These can work the pedagogical requirements in mind to make sense of that
well but can be very limited in scope and depth. information.
On the other hand, artificial intelligent based chatbots To analyze the students’ question, we use the Speech Act
can better understand student intents. Even the most sim- Classifier. It selects the module where the question must be
ple non-rule-based natural language understanding methods delegated. The way it works will be explained in greater detail
in Section III-B. If small talk is detected, it is passed onto
1 https://landbot.io the Small Talk module or into the QA Module if a question
2 https://chatfuel.com/bot/HelloFreshus regarding Data Science is detected.

VOLUME 8, 2020 180673

 D. Carlander-Reuterfelt et al.: JAICOB: A Data Science Chatbot

TABLE 1. Post classification examples.

examples of implementations is an obvious use case for the

chatbot, and therefore an nearly important part of the KB.
For more complex questions, the use of FAQs solves the
problem. The Machine Learning Mastery site5 used for this
purpose is structured as a list of questions with the answers
associated. It was selected because of the rich and adequate
answer for the project.
FIGURE 1. System architecture.

B. SPEECH ACT CLASSIFIER

Afterward, the modules generate an answer to satisfy the
student request. The answer is sent back to the student, and The speech act classification task involves classifying a spe-
feedback is collected to evaluate and improve the model. cific sentence into a set of predefined speech act categories.
This classification is relevant to the project because it is
indispensable to know the student’s intention [34] and answer
A. KNOWLEDGE BASE
accordingly.
The KB is the place where all the information used by
The dataset [35] used to train the classifier consists
the chatbot is stored. It has been populated from several
of 10567 posts from five different age-oriented chat rooms
online academic resources. Its selection was based on the
at an internet chat site. It is sanitized to protect user privacy.
previously identified student interactions with the chatbot.
The posts were tagged using 15 post categories (Accept,
Glossaries and Frequently Asked Questions (FAQs) of the
Bye, Clarify, Continuer, Emphasis, Greet, No Answer, Other,
topic have been mined using web scraping techniques to pro-
Reject, Statement, System, Wh-question, Yes Answer, Yes/No
vide concept definitions. Regarding searching code snippets,
question, Emotion). Examples of these classes are shown
technical documentation has also been mined.
in Table 1.
This approach benefits from providing students a curated
Since this is a chatbot system that requires a fast
list of pedagogical sources that are credible and useful.
response time, the preprocessing has been simplified to
According to some studies [32], undergraduates tend to use
improve the model’s time complexity while not sacrific-
Google for searching for information, and the usage of aca-
ing relevant performance. Each phrase is processed into
demic resources is low. Thus, our system increases the use of
machine-understandable information using a raw pipeline.
curated academic resources since the bot can enhance their
The overall process is (1) Simple tokenization because
familiarity [33].
n-grams did not present a significant improvement in
The adaptation of the bot to other domains could be made
accuracy, (2) Stemming, and (3) Feature extraction by
replicating the same approach.
vectorization.
According to the categories previously described, the sites
By training and evaluating some of the most popular classi-
that fit the necessities of the definition answering are: (1) Big
fication algorithms, the best one is selected based on the score
Data glossary3 with a list of terms regarding big data. (2)
achieved by a K-Fold. This process can be automated through
Machine Learning glossary4 with a complete glossary of
a grid search that finds the best parameters optimally.
machine learning and statistics terms and definitions.
The scores following are calculated, saving a fourth of the
The documentation sites used to populate the KB for
dataset for testing afterward and using the rest to get these
answering with examples are Pandas Documentation. The
results.
use of the Python Pandas library is widely used when devel-
The scores shown in Table 2 are obtained by performing
oping machine learning models. It is beneficial to have exam-
a 5-Fold and calculating the mean of the scores. Support
ples available for standard implementations of data handling.
Vector Machines, such as the SVC algorithm, achieves higher
This documentation is structured with brief descriptions with
performance.
code examples; and Scikit-Learn Documentation, being the
Using as the training data 3/4 of the dataset and the rest as
library used widely for Machine Learning purposes, Scikit
testing data, we obtain with the SVC a final accuracy score
3 Big Data glossary: https://bigdata-madesimple.com/big-data-a-to-zz-a- of 0.799.
glossary-of-big-data-terminology/
4 Machine Learning glossary: https://www.analyticsvidhya.com/glossary-
of-common-statistics-and-machine-learning-terms/ 5 Site for complex questions: https://machinelearningmastery.com/faq/

180674 VOLUME 8, 2020

D. Carlander-Reuterfelt et al.: JAICOB: A Data Science Chatbot

FIGURE 2. QA architecture.

TABLE 2. Evaluation scores.

FIGURE 3. QA example.

C. QUESTION ANSWERING MODULE

The Question Answering module comes into place when the
user asks for a specific piece of information. These can range
from doubt, a consultation, or documentation clarifications.
FIGURE 4. Definition intent use case.
It must be able to understand what the user is asking for to
retrieve the information effectively.
Using natural language processing techniques, it answers 1) DEFINITION ANSWERING
the question in near real-time. This general-purpose model is When the Answer Type is of the definition type, the mod-
enhanced to attend specific cases to the task at hand, such as ule searches in the Knowledge Base’s Glossary index.
code examples. It searches for a match with the terms in the index. When
The general view of the architecture is defined in Figure 2. a match is found, the corresponding definition is sent as an
The modules involved in the process are the following. answer. Common questions of this type are:
The Process Question module extracts the relevant infor-
• What is a neural network?
mation and intention of the question. The output contains
• Can you give me a definition of overfitting?
a type of question, a type of answer, and a vector with the
relevant ideas. This module is implemented as a DialogFlow agent, with
The Information Retrieval This module receives the an intent to recognize that the user wants a definition. The
question vector and the answer type from the intent is trained with multiple training phrases that can be
question processor as an input. The question vector is, used to ask for a definition. It extracts a term as the slot. These
in essence, a list of keywords ordered by importance. An Elas- slots are recognized thanks to an entity6 defined as all the
ticSearch query is generated to retrieve relevant documents terms available in the Knowledge Base. An example can be
and pieces of information that match the keywords, using this seen in Figure 4.
valuable information.
The Document Parsing module receives and parses the
retrieved information, so it matches the questions intented to 6 An entity is the definition of a type of slot. It can be defined as a list with
generate an answer. all the possible terms for that slot.

VOLUME 8, 2020 180675

 D. Carlander-Reuterfelt et al.: JAICOB: A Data Science Chatbot

FIGURE 5. Example intent use case.

2) EXAMPLE ANSWERING
When the Answer Type is of the example type, we need
a more complex type of search. There is a search across the
documentation text to match the keywords of the query. When
a match is found, the corresponding code snippet is sent to
respond with the appropriate format. Examples of these type FIGURE 6. Small talk examples.
of questions are:
• How is a dataframe defined in Pandas?
TABLE 3. Intent distribution.
• How can I implement a k-fold using scikit?
This module is implemented as a DialogFlow agent with
and intent trained to detect example queries. The slot, in this
case, is more open, so there is no Entity defined. The example
can be of any kind. The result can be seen in Figure 5.

D. SMALL TALK MODULE

According to [36], the users’ satisfaction with a certain chat-
bot is influenced by various factors. By testing which of these E. GRAPHICAL USER INTERFACE
factors were more influential, the results revealed that the The bot needs an identifier to generate a more personal
bot’s human-likeness was significantly correlated with the relationship [37]. Being a Cognitive bot and an intelligent
users’ satisfaction. one, it was decided to be called Just an Artificial Intelligence
It was stated [2] that people were inclined to send more Cognitive Bot (JAICOB).
than twice as many messages to chatbots with a human- In contrast with Jaicob, a general-purpose bot would gain
like interaction compared to other people, contrary to our quality from a text-to-speech transformer, giving it a more
expectations and disconfirming the notion that people feel human appearance. This feature is not the case of Jaicob
less confident or comfortable communicating with chatbots. because it is centered on answering documentation and pro-
Including a module to handle small-talk improves the bot’s gramming related questions. The frequent use of acronyms
human-likeness and makes it more fun and engaging. Instead and code examples in the answers would not make for a
of answering with the fallback answer, if the question is not pleasant listening experience. Instead, the use of text is the
about the topic, it triggers the small-talk module to simulate best option in this case.
human interaction and cleverness. Some examples of the
behavior that the bot can answer are collected in Figure 6. IV. EVALUATION
A. EVALUATION TECHNIQUES
1) IMPLEMENTATION
The evaluation method for Jaicob chatbot is a Partial Least
This module is implemented with Google’s DialogFlow tech-
Squares (PLS) analysis. A detailed example [38] is followed
nology. There is a specially trained agent to provide the
to perform PLS methods. The tool being used is Smart PLS.7
desired output. This agent can detect more than 100 different
The method is based on a questionnaire and requires
intents.
the definition of latent variables to be evaluated, which
Among these intents are some of the provided with the
are abstract variables that are connected to directly mea-
default Small Talk module and some custom ones. The intents
surable variables. These variables’ values are scored by the
are defined to fit the purpose of this project. For example,
when asked what it can do, it responds with directions to ask 7 https://www.smartpls.com/documentation/pls-sem-compared-with-cb-
questions about Data Science. sem

180676 VOLUME 8, 2020

D. Carlander-Reuterfelt et al.: JAICOB: A Data Science Chatbot

TABLE 4. Outer loadings.

responses of the questionnaires. These latent variables can

also have relations, and these can be hypothesized, as shown
in Section IV-C.
Being a conversational interface, the way to test it is with
real users who answer the questionnaire after using the chat-
bot. The number of observations (number of questionnaires
answered by users) should be at least ten times the number of
relations between latent variables.

B. PARTICIPANTS
The experiment was done with 50 participants, all of them
with technical backgrounds. All of them were unaware of the
inner workings of Jaicob. They were asked to use the chatbot
as a tool to answer any questions or doubts that may arise
in understanding Data Science related topics or writing the
corresponding code.
The median of the ages of the participants is 22 years.
A 51% of them were studying a Telecommunication Engi- FIGURE 7. Structural paths in the applied PLS model.
neering Grade and the rest a Master or superior studies.
About their technological background, 54% of the partic- • Behavioral intentions (BI) refers to the recommenda-
ipants had developed and implemented some machine learn- tion of users to others to use the bot.
ing programs. The rest had some basic knowledge in the field. • Satisfaction (SS) refers to the feeling after using the bot.
• Utilitarian value (UV) refers to the value it provides to

C. EXPERIMENT DESIGN the task you are looking to complete.

• Answer Accuracy (AA) refers to the performance in the
As explained in Section III-D, small talk is an essential part
task it was programmed to do.
of the architecture of the chatbot. Therefore, before making These latent variables are not independent, as represented
the measurements, it is taken into account. in Figure 7. They present relations between latent vari-
Five latent variables were defined to evaluate the conver- ables, which are hypothesized and tested. Moreover, the rela-
sational bot: tions between latent variables and questions, summarized
• Social Handling (SH) refers to the personality and in Table 4, are shown in the structural paths of the applied
human-likeness of the bot. PLS Model.

VOLUME 8, 2020 180677

 D. Carlander-Reuterfelt et al.: JAICOB: A Data Science Chatbot

Research [39] suggests that the quality of information on TABLE 5. Discriminant validity.
an e-commerce website has a positive impact on perceived
value. Reference [40] suggests that accurate information
can help users make better decisions, thus improving both
utilitarian values. According to [41], the utilitarian values
increases when the interaction with the process improves.
These hypotheses are proposed:
H1. Perceptions of a better answer accuracy improve util-
itarian value.
User satisfaction is influenced by the human-likeness of the
chatbot [36]. Also, [2] state that people are more inclined to
send messages to a chatbot that handles this type of small-
talk well. A website’s social dimension is another impor-
tant antecedent of perceived value [42]. Research [43], [44]
reveals that there is a direct link between perceived sociability
and satisfaction.
H2. The social handling of the bot improves the overall
satisfaction of the user.
H3. The social handling of the bot improves the utilitarian
value a user perceives.
H4. Good social handling improves the behavioral inten-
tions of users after using the bot.
Utilitarian value is central to user satisfaction and behav-
ioral intentions. If the perceived value is low, the user proba- FIGURE 8. Path coefficients.
bly switches to other sources [39].
H5. A higher perceived answer accuracy value increases
positive behavioral intentions. as shown in Table 4. The PLS analysis also provides us
H6. A higher perceived utilitarian value increases positive with the Composite Reliability of each latent variable. This
behavioral intentions. index surpassed the minimum acceptable value of .70 in all
Perceived utilitarian value also enhances satisfaction [40]. variables, being all over .85.
Research [45] demonstrates that utilitarian value can improve The average variance extracted (AVE) for each variable
the final user satisfaction: must surpass a threshold of .50 [43], [47], and provide a
H7. Perceived utilitarian value has a positive effect on user square root that is much larger than the correlation of the
satisfaction. specific construct with any other construct in the model. All
H8. Perceived answer accuracy has a positive effect on user the latent variables surpass a .70 AVE, as shown in Table 4.
satisfaction. Table 5 shows that the square roots of the AVE (on the
diagonal) are higher than any other values, in support of the
D. RESULTS discriminant validity of the measurement scales [38].
The testers made an average of 15.86 queries per session. The Then, discriminant validity is tested, which indicates the
intent that matched most of the queries was related to code extent to which a given construct (variable) differs from other
example requests, which means that users used the bot for latent constructs. The validity of these variables requires that
what it was intended. After that, there is the Definition intent each measurement item correlates weakly with all constructs,
and then the complex intent. Also, 26.7% of the queries except for which it is theoretically associated. The results in
resulted in small talk handling. The distribution can be seen Table 5 support the validity of the measurement scales.
in Table 3. All the direct hypotheses received support, except for H4,
The results extracted from the PLS modeling, having used as shown in Figure 8. From these results, we can extract
SmartPLS 3.0 [46] meet the requirements, being the sample some insights, such as the impact that Answering Accu-
size ten times the largest number of structural paths directed at racy has on all the other variables. Therefore, the quality
a particular construct in the structural model. There are three of the system and its ability to respond effectively is what
paths directed to Behavioral Intentions and Satisfaction in this makes the difference for overall user Satisfaction, Utilitar-
model, so the minimum sample size should be 30, and the ian Value, and Behavioral Intentions (H1, H5, H8). Also,
sample size is above this minimum. the perceived Utilitarian Value has a positive effect on Behav-
To test the experiment’s internal coherence, and therefore, ioral Intentions and Satisfaction (H6, H7). Surprisingly,
reliability, we look at the outer loadings. These coefficients Social Handling was not significant in positive behavioral
need to meet a threshold for every measure that points to the intentions (H4), contrasting with the Utilitarian Value and
latent variables. All the measures met this reliability index, Satisfaction (H2, H3).

180678 VOLUME 8, 2020

D. Carlander-Reuterfelt et al.: JAICOB: A Data Science Chatbot

V. CONCLUSION [10] R. P. Bostrom, ‘‘Technology-mediated learning: A comprehensive theoret-

The use of chatbots has become prevalent in the last years in ical model,’’ J. Assoc. Inf. Syst., vol. 10, no. 9, pp. 686–714, Sep. 2009.
[11] R. Dale, ‘‘The return of the chatbots,’’ Natural Lang. Eng., vol. 22, no. 5,
shopping, customer support, general assistance, and, though pp. 811–817, Sep. 2016.
less developed, education. The use of chatbots as a form of [12] X. L. Pham, T. Pham, Q. M. Nguyen, T. H. Nguyen, and T. T. H. Cao,
e-learning brings lots of opportunities. ‘‘Chatbot as an intelligent personal assistant for mobile language learning,’’
in Proc. 2nd Int. Conf. Educ. E-Learn. (ICEEL), 2018, pp. 16–21.
This article identified the advantages of cognitive assistants [13] J. Beaudry, A. Consigli, C. Clark, and K. J. Robinson, ‘‘Getting ready for
in education and the corresponding challenges in implemen- adult healthcare: Designing a chatbot to coach adolescents with special
tation. A result is a tool for students with a comfortable health needs through the transitions of care,’’ J. Pediatric Nursing, vol. 49,
pp. 85–91, Nov. 2019.
and usable interface and a human experience. It can provide [14] M. Awais Hassan, U. Habiba, H. Khalid, M. Shoaib, and S. Arshad,
insights and solve doubts about Data Science. The main ‘‘An adaptive feedback system to improve student performance based on
contribution is the adaptation of students’ real pedagogic collaborative behavior,’’ IEEE Access, vol. 7, pp. 107171–107178, 2019.
[15] M. Coronado, C. A. Iglesias, Á. Carrera, and A. Mardomingo, ‘‘A cognitive
needs to the design of the architecture and being flexible in assistant for learning java featuring social dialogue,’’ Int. J. Hum.-Comput.
maintaining a conversation. Stud., vol. 117, pp. 55–67, Sep. 2018.
Teachers can also use it as a tool to identify gaps in the [16] H. T. Hien, P.-N. Cuong, L. N. H. Nam, H. L. T. K. Nhung, and L. D. Thang,
‘‘Intelligent assistants in higher-education environments: The FIT-EBot, a
knowledge of their students. They can also outsource to chatbot for administrative and learning support,’’ in Proc. 9th Int. Symp.
Jaicob the answering of all the questions. The pedagogue is Inf. Commun. Technol. (SoICT), New York, NY, USA, 2018, pp. 69–76,
also an excellent asset to select the most valuable sources of doi: 10.1145/3287921.3287937.
[17] L. Benotti, M. C. Martínez, and F. Schapachnik, ‘‘Engaging high school
information from which Jaicob feeds from, thus providing students using chatbots,’’ in Proc. Conf. Innov. Technol. Comput. Sci. Educ.
a curated source of information instead of a regular Google (ITiCSE), 2014, pp. 63–68.
Search. [18] D. Adiwardana, M.-T. Luong, D. R. So, J. Hall, N. Fiedel, R. Thoppilan,
Z. Yang, A. Kulshreshtha, G. Nemade, Y. Lu, and Q. V. Le, ‘‘Towards
The project was evaluated with a sample of students, a human-like open-domain chatbot,’’ 2020, arXiv:2001.09977. [Online].
achieving very favorable results in usability and original- Available: http://arxiv.org/abs/2001.09977
ity. The experiment confirms that the system can answer [19] H. Doan-Nguyen and L. Kosseim, ‘‘Improving the precision of a closed-
domain question-answering system with semantic information,’’ in Proc.
effectively, that the answer accuracy affects the satisfaction, Coupling Approaches, Coupling Media Coupling Lang. Inf. Retr., Le Cen-
utilitarian value, and behavioral intentions of the user, and tre De Hautes Etudes Internationales D’Informatique Documentaire, 2004,
that proper social handling is significant in satisfaction and pp. 850–859.
[20] B. Heller, M. Proctor, D. Mah, L. Jewell, and B. Cheung, ‘‘Freudbot:
utilitarian value but not in behavioral intentions. An investigation of chatbot technology in distance education,’’ in Proc.
As these technologies evolve, more and more people EdMedia+ Innovate Learn. Assoc. Adv. Comput. Educ. (AACE), 2005,
will study these subjects. Therefore, the future impact pp. 3913–3918.
[21] S. Hussain and G. Athula, ‘‘Extending a conventional chatbot knowledge
of the project is promising, and the affected groups will base to external knowledge source and introducing user based sessions
increase. In future work, to achieve a broader reach in the for diabetes education,’’ in Proc. 32nd Int. Conf. Adv. Inf. Netw. Appl.
areas of knowledge, it is straightforward to place additional Workshops (WAINA), May 2018, pp. 698–703.
[22] J. Jia and W. Chen, ‘‘Motivate the learners to practice English through
information in the Knowledge Base and the corresponding playing with Chatbot CSIEC,’’ in Proc. Int. Conf. Technol. E-Learn. Digit.
Dialogflow intents. Entertainment. Cham, Switzerland: Springer, 2008, pp. 180–191.
[23] S. Roos, ‘‘Chatbots in education: A passing trend or a valuable pedagogical
tool?’’ M.S. thesis, Dept. Inform. Media, Social Sci., Disciplinary Domain
REFERENCES Humanities Social Sci., Uppsala Univ., Uppsala, Sweden, 2018, pp. 1–58.
[1] A. Soffer, D. Konopnicki, and H. Roitman, ‘‘When watson went to work: [24] D. Suendermann, K. Evanini, J. Liscombe, P. Hunter, K. Dayanidhi, and
Leveraging cognitive computing in the real world,’’ in Proc. 39th Int. ACM R. Pieraccini, ‘‘From rule-based to statistical grammars: Continuous
SIGIR Conf. Res. Develop. Inf. Retr. (SIGIR), 2016, pp. 455–456. improvement of large-scale spoken dialog systems,’’ in Proc. IEEE Int.
[2] J. Hill, W. Randolph Ford, and I. G. Farreras, ‘‘Real conversations with arti- Conf. Acoust., Speech Signal Process., Apr. 2009, pp. 4713–4716.
ficial intelligence: A comparison between human–human online conversa- [25] S. Paliwal, V. Bharti, and A. K. Mishra, ‘‘Ai chatbots: Transforming the
tions and human–chatbot conversations,’’ Comput. Hum. Behav., vol. 49, digital world,’’ in Recent Trends and Advances in Artificial Intelligence
p. 245–250, Aug. 2015. and Internet of Things. Cham, Switzerland: Springer, 2020, pp. 455–482.
[3] Y. Chen, J. E. Argentinis, and G. Weber, ‘‘IBM Watson: How cognitive [26] H. T. Hien, P.-N. Cuong, L. N. H. Nam, H. L. T. K. Nhung, and L. D. Thang,
computing can be applied to big data challenges in life sciences research,’’ ‘‘Intelligent assistants in higher-education environments: The FIT-EBot, a
Clin. Therapeutics, vol. 38, no. 4, p. 688–701, Apr. 2016. chatbot for administrative and learning support,’’ in Proc. 9th Int. Symp.
[4] R. High, ‘‘The era of cognitive systems: An inside look at IBM Watson Inf. Commun. Technol. (SoICT), 2018, pp. 69–76.
and how it works,’’ IBM Corp., Redbooks, North Castle, NY, USA, Tech. [27] O. Mangkang and J. Y. Yun, ‘‘A comparative study of self-diagnosis user
Rep., 2012. interfaces for depression: Focusing on usability of paper-based, text-based
[5] M. Coccoli, A. Guercio, P. Maresca, and L. Stanganelli, ‘‘Smarter univer- and voice-based conversational interface,’’ in Proc. Korean HCI Conf.,
sities: A vision for the fast changing digital era,’’ J. Vis. Lang. Comput., 2019, pp. 262–267.
vol. 25, no. 6, pp. 1003–1011, Dec. 2014. [28] R. M. Felder and L. K. Silverman, ‘‘Learning and teaching styles in
[6] M. Coccoli, P. Maresca, and L. Stanganelli, ‘‘Cognitive computing in engineering education,’’ Eng. Educ., vol. 78, no. 7, pp. 674–681, 1988.
education,’’ Big Data, vol. 12, no. 2, p. 15, 2016. [29] R. Oliver, ‘‘Exploring strategies for online teaching and learning,’’ Dis-
[7] S. Kowalski, K. Pavlovska, and M. Goldstein, ‘‘Two case studies in using tance Educ., vol. 20, no. 2, pp. 240–254, Jan. 1999.
chatbots for security training,’’ in Proc. IFIP World Conf. Inf. Secur. Educ. [30] P. Brusilovsky, ‘‘Webex: Learning from examples in a programming
Cham, Switzerland: Springer, 2009, pp. 265–272. course,’’ in Proc. WebNet, vol. 1, 2001, pp. 124–129.
[8] P. Bii, ‘‘Chatbot technology: A possible means of unlocking student poten- [31] P. L. Pirolli and J. R. Anderson, ‘‘The role of learning from examples in
tial to learn how to learn,’’ Educ. Res., vol. 4, no. 2, pp. 218–221, 2013. the acquisition of recursive programming skills,’’ Can. J. Psychol./Revue
[9] R. Winkler and M. Söllner, ‘‘Unleashing the potential of chatbots in Canadienne de Psychologie, vol. 39, no. 2, p. 240, 1985.
education: A state-of-the-art analysis,’’ in Proc. Acad. Manage. Annu. [32] J. R. Griffiths and P. Brophy, ‘‘Student searching behavior and the Web:
Meeting, 2018, pp. 1–40. [Online]. Available: https://www.alexandria. Use of academic resources and Google,’’ Trends, vol. 53, no. 4, p. 539,
unisg.ch/254848/ 2005.
VOLUME 8, 2020 180679
 D. Carlander-Reuterfelt et al.: JAICOB: A Data Science Chatbot

[33] J. Y. Lee, W. Paik, and S. Joo, ‘‘Information resource selection of under- CARLOS A. IGLESIAS received the Telecom-
graduate students in academic search tasks,’’ Inf. Res., Int. Electron. J., munications Engineering degree and the Ph.D.
vol. 17, no. 1, p. n1, 2012. degree in telecommunications from the Uni-
[34] B. Samei, H. Li, F. Keshtkar, V. Rus, and A. C. Graesser, ‘‘Context-based versidad Politécnica de Madrid (UPM), Spain,
speech act classification in intelligent tutoring systems,’’ in Proc. Int. Conf. in 1993 and 1998, respectively. He is currently an
Intell. Tutoring Syst. Cham, Switzerland: Springer, 2014, pp. 236–241. Associate Professor with the Telecommunications
[35] E. N. Forsythand and C. H. Martell, ‘‘Lexical and discourse analysis Engineering School, UPM. He has been leading
of online chat dialog,’’ in Proc. Int. Conf. Semantic Comput. (ICSC), the Intelligent Systems Group, UPM, since 2014.
Sep. 2007, p. 19.
He has been the Principal Investigator on numer-
[36] M. Xuetao, F. Bouchet, and J.-P. Sansonnet, ‘‘Impact of agent’s answers
ous research grants and contracts in the field of
variability on its believability and human-likeness and consequent chatbot
improvements,’’ in Proc. AISB, 2009, pp. 31–36. advanced social and the IoT systems, funded by the regional, national, and
[37] J. Finch, ‘‘Naming names: Kinship, individuality and personal names,’’ European bodies. His main research interests include social computing,
Sociology, vol. 42, no. 4, pp. 709–725, Aug. 2008. multiagent systems, information retrieval, sentiment and emotion analysis,
[38] M. S. Ben Mimoun and I. Poncin, ‘‘A valued agent: How ECAs affect web- linked data, and web engineering.
site customers’ satisfaction and behaviors,’’ J. Retailing Consum. Services,
vol. 26, pp. 70–82, Sep. 2015.
[39] M. Magni, M. Susan Taylor, and V. Venkatesh, ‘‘‘To play or not to play’:
A cross-temporal investigation using hedonic and instrumental perspec-
tives to explain user intentions to explore a technology,’’ Int. J. Hum.-
Comput. Stud., vol. 68, no. 9, pp. 572–588, Sep. 2010.
[40] C. Kim, R. D. Galliers, N. Shin, J.-H. Ryoo, and J. Kim, ‘‘Factors influenc-
ing Internet shopping value and customer repurchase intention,’’ Electron. ÓSCAR ARAQUE received the graduate and mas-
Commerce Res. Appl., vol. 11, no. 4, pp. 374–387, Jul. 2012. ter’s degrees in telecommunication engineering
[41] T. Ahn, S. Ryu, and I. Han, ‘‘The impact of Web quality and playfulness from the Technical University of Madrid (Univer-
on user acceptance of online retailing,’’ Inf. Manage., vol. 44, no. 3, sidad Politécnica de Madrid), Spain, in 2014 and
pp. 263–275, Apr. 2007. 2016, respectively, where he is currently pursuing
[42] J. Lawler, P. Vandepeutte, and A. Joseph, ‘‘An exploratory study of apparel
the Ph.D. degree. He is currently a Teaching Assis-
dress model technology on European Web sites,’’ J. Inf., Inf. Technol.,
tant with the Universidad Politécnica de Madrid.
Organizations, vol. 2, no. 244, pp. 31–46, Nov. 2007.
His research interest includes the application of
[43] S. S. Al-Gahtani, G. S. Hubona, and J. Wang, ‘‘Information technology (IT)
in Saudi Arabia: Culture and the acceptance and use of IT,’’ Inf. Manage., machine learning techniques for natural language
vol. 44, no. 8, pp. 681–691, Dec. 2007. processing. The main topic of his thesis is the
[44] O. Turel, A. Serenko, and N. Bontis, ‘‘User acceptance of hedonic digital introduction of specific domain knowledge into machine learning systems
artifacts: A theory of consumption values perspective,’’ Inf. Manage., in order to enhance sentiment and emotion analysis techniques.
vol. 47, no. 1, pp. 53–59, Jan. 2010.
[45] T. W. Traphagan, Y.-H.-V. Chiang, H. M. Chang, B. Wattanawaha, H. Lee,
M. C. Mayrath, J. Woo, H.-J. Yoon, M. J. Jee, and P. E. Resta, ‘‘Cognitive,
social and teaching presence in a virtual world and a text chat,’’ Comput.
Educ., vol. 55, no. 3, pp. 923–936, Nov. 2010.
[46] C. Ringle, S. Wende, and J.-M. Becker, ‘‘Smartpls 3,’’ SmartPLS GmbH,
Bönningstedt, Germany, Tech. Rep., Jan. 2015.
[47] C. Fornell and D. F. Larcker, ‘‘Structural equation models with unob- JUAN FERNANDO SÁNCHEZ RADA received
servable variables and measurement error: Algebra and statistics,’’ J. the Ph.D. degree from the Universidad Politécnica
Marketing Res., vol. 18, no. 3, pp. 382–388, 1981, doi: 10.1177/ de Madrid (UPM), Spain, in 2020. He is currently
002224378101800313. a Researcher with the Intelligent Systems Group,
Universidad Politécnica de Madrid. His research
DANIEL CARLANDER-REUTERFELT is cur- interests include natural language processing (sen-
rently pursuing the master’s degree in telecom- timent and emotion analysis), social network anal-
munications engineering with the Universidad ysis (social context and graph embedding), the web
Politécnica de Madrid (UPM). He was awarded and distributed systems (interoperability and fed-
an honorary mention for the bachelor’s thesis eration), and semantic technologies (linked data,
(Development of a Cognitive Bot for Data Science ontologies, and knowledge graphs).
Tutoring based on a Big Data Natural Language
Analytics Platform). He has been a part of the
Intelligent Systems Group, UPM, since 2019. His
research interests include intelligent agents, natu-
ral language processing and understanding, and time series prediction.

ÁLVARO CARRERA received the Telecommuni-

cation Engineering degree and the Ph.D. degree SERGIO MUÑOZ received the graduate and mas-
in telecommunications engineering from the Uni- ter’s degrees in telecommunication engineering
versidad Politécnica de Madrid (UPM), Spain, from the Technical University of Madrid (Univer-
in 2010 and 2016, respectively. He is currently an sidad Politécnica de Madrid), Spain, in 2016 and
Assistant Professor with the School of Telecom- 2017, respectively, where he is currently pursu-
munications Engineering, UPM. His research ing the Ph.D. degree. He is currently a Teaching
interests include applying intelligent agents in Assistant with the Technical University of Madrid.
telecommunication networks for critical tasks, His research interests include ambient intelligence
such as fault diagnosis or intrusion detection in and agent-based simulation. The main topic of his
real-time, and in the application of simulation techniques for complex sys- thesis is the adaptation of smart environments to
tems in dynamic environments, such as telecommunication networks and users’ emotions, to enhance well-being and performance.
social networks.

180680 VOLUME 8, 2020