Margie Imperial

Building A Knowledge-Based Chatbot

for Customer Support

Metropolia University of Applied Sciences

Master’s Degree
Degree Programme in Business Informatics
Master’s Thesis

28 November 2022
 Abstract

Author Margie Imperial

Title Building a Knowledge-based Chatbot for Customer Support

Number of Pages 72 pages + 2 appendices

Date 28 November 2022

Degree Master of Business Administration

Degree Programme Business Informatics

Niklas Visanko, Senior Lecturer

Instructor
Zinaida Grabovskaia, Senior Lecturer

The objective of this study is to transform the current simplified version of chatbot into a
smart AI chatbot that can help the case company with lead generation and will take it closer
to having an automated sales funnel.

The data collection for this study includes interviews and discussions in the case company
and a small-scale questionnaire with the customers. The research method for this study is
action research as this requires iterative actions that can help the researcher to plan and
implement in cycles based on the evaluation done at each cycle.

The study started with the current state analysis that helped to identify the issues that the
case company faces within the current chatbot system. The identified issues related to a
considerable manual interference, limited availability in terms of the response time, and lack
of a knowledge base for more informative responses. Therefore, the focus areas of this study
concentrated on improving the current version of the chatbot to implement a knowledge-
based feature that can provide relevant answers 24/7 as it can understand the context and
intent of the website visitors. To find relevant guidance the study explored available
knowledge on the topics of chatbot building and selected especially HubSpot guidance as
best practices to follow when develop the solution.

During the proposal stage, the first version of the chatbot that was developed. During the
testing stage, it was deployed to the case company’s website and is currently in use since
then. The validation stage included validation discussions with the key stakeholders in the
case company that evaluated the testing results and suggested further improvements. The
chatbot that has been launched helps the customer service agents to respond to multiple
inquiries, close deals from the leads generated by the chatbot, and pass their information to
the CRM system to assist the customer service agents further, when needed.

Keywords Chatbot, Artificial Intelligence, Natural Language Processing,

HubSpot
Contents

Glossary

1 Introduction 1

1.1 Business Context 1

1.2 Business Challenge, Objective and Outcome 2
1.3 Thesis Outline 3

2 Method and Material 4

2.1 Research Approach 4

2.2 Research Design 6
2.3 Data Collection and Analysis 8

3 Current State Analysis of the Case Company´s Chatbot 12

3.1 Overview of the Current State Analysis 12

3.2 Description of the Current Chatbot (Design and Functionalities) 13
3.3 Analysis of the Current Chatbot (Design and Functionalities) 15
3.3.1 Internal stakeholders’ perspective 15
3.3.2 Customers’ perspective 19
3.3.1 Selected Focus Areas 21
3.3.2 Summary of responses 22
3.1 Key Findings 23
3.1.1 Fit for Purpose (How successfully it is currently used in customer
service) 24
3.1.2 Fit for Customer Needs (How well it serves the customers) 24
3.1.3 Strengths and weaknesses of the curertn chatbot 24
3.1.4 Direction for improvement 26

4 Existing Knowledge and Best Practice on Building Chatbots 28

4.1 Existing Chatbot Solutions and Benchmarks (How they are designed) 28
4.2 Best Practices on Building a Chatbot 30
4.2 Using Chatbot Scenarios for Chatbot Building 33
4.3 Building a Chatbot in HubSpot (How to build, step by step) 35
4.4 Conceptual Framework of This Thesis 39
5 Building the Chatbot for the Case Company 41

5.1 Overview of the Building Stage 41

5.2 Initial Inputs for Building the Chatbot 41
5.2.1 Planning the Chatbot 43
5.2.2 Building the Chatbot 49
5.2.3 Monitoring the Chatbot 59
5.2.4 Summary of the Initial Proposal 60

6 Validation of the Development 62

6.1 Overview of the Validation Stage 62

6.2 Developments to the Proposal (collecting feedback, Data 3 and implementing
these improvements) 64
6.2.1 Developments to Element 1 of the Initial Build 65
6.2.2 Developments to Elements 2 of the Initial Build 66
6.2.3 Developments to Elements 3 of the Initial Build 66
6.2.4 Final Developments 67
6.3 Final proposal 67

7 Conclusion 69

7.1 Executive Summary 69

7.2 Thesis Evaluation: Objective vs. Outcome 70
7.3 Closing Words 72

References 1

Appendix 1: CSA Results – External Customers 5

Appendix 2: CSA Results – Internal Stakeholders 6

Glossary

ORM Object-relational mapping. The set of rules for mapping objects in a

programming language to records in a relational database, and vice
versa.

DBMS Database management system. Software for maintaining, querying and

updating data and metadata in a database.

AI Artificial Intelligence

NLP Natural Language Processing

 1

1 Introduction

As AI technology is becoming more popular nowadays in digital age, it shows promising

capabilities to improve marketing strategies. According to Vishnoi (2021), about 80% of
customer inquiries can be resolved by AI chatbots as the questions are mostly repetitive
- without the need of customer service representatives. Furthermore, developing an AI-
powered chatbot is beneficial for small businesses because it can handle multiple
queries which allows you to focus on more complex issues.

A virtual agent is a very powerful tool that can interact with customers and potential leads
(Kannan and Bernoff 2019). Chatbot is an automated text chat that provides pre-defined
replies according to the customer’s request. It merely scans all the information available
in the Chatbot knowledge-based library system and based on the client’s messages or
options selected, it will pull out the best answer from the database swiftly. Compared to
a human Customer service agent, chatbot is available 24/7 and can answer customer’s
questions instantly. This is a type of artificial intelligence that once utilized correctly, will
have an impact on the company’s integrity and reputation.

This Thesis aims to improve the current chatbot system of the case company´s website
existing chatbot. The company is Clean Living Oy that provides cleaning services and
other household services to ease the client’s daily lives. AI-powered chatbot can
significantly impact business operations especially in marketing and sales departments.

1.1 Business Context

The case company of this thesis provides cleaning services and other household chores
to ease the client’s daily lives. However, the current chatbot implemented on the website
is a very simplified version which is supported and created with HubSpot to simulate
conversation with potential clients.

The company is a family-owned small enterprise and most of its employees are also
involved in a full-time job and cannot monitor the company´s chatbot activity around the
clock. Therefore, potential leads through the chatbot are often overlooked due to inability
to answer each client’s questions and respond immediately. The chatbot system is
mainly used to attend customer inquiries that could generate more leads to web-site
traffic.
 2

Presently, the company employs 12 full-time cleaners who do their work with
commitment and diligence. Two of the co-founders have professional backgrounds in
the IT world who see the importance of having presence on the worldwide web as
beneficial. The company started by creating a website proving brief information regarding
the company´s services and contact information. Furthermore, the company also
executes its business marketing through HubSpot where the leads and sales are
processed through the marketing platform. The company also advertises our service via
social media platforms such as Facebook and Instagram where the company publishes
active advertisements to target audiences within the range.

One of the company’s goals is to maintain customer satisfaction by providing high-quality

service at each service provided. If in any case, a client is not very much satisfied with
the outcome, he/she is compensated by providing an additional service until the client’s
request is fulfilled. However, although the team is not yet substantial, each of the team
members has a professional background that contributes to the company’s success by
far.

1.2 Business Challenge, Objective and Outcome

The company introduced its first chatbot in 2019 year. The current simplified version of
the chatbot was created through HubSpot which is integrated with the company´s
customer relationship management system. Additionally, HubSpot operates as a sales
platform that provides visibility of all activities throughout the buyer’s personas journey.
So far, the company continues with the same approach in terms of conversion marketing,
as it has recognized a lot of benefits from a chatbot implemented in HubSpot. At the
same time, the company has encountered quite a lot of issues with this current simplified
version of chatbot on its website.

Accordingly, the objective of this thesis is to improve the current chatbot of the company.
This improvement is planned to be done by developing a virtual agent for customer
interactions to gather leads and be attentive at any time to its website users.

Hence, the outcome is a proposal on how to improve the current chatbot of the company
that will be implemented as far as the thesis timeline allows.
 3

1.3 Thesis Outline

The thesis and the development of the chatbot will give a valuable impact to the sales
and marketing of the case company. The company is ready to rip out both the benefits
and potential challenges of introducing an AAI-based chatbot. For example, an AI-
powered chatbot uses Natural Language Processing (NLP) that extracts user-input texts
or human language intelligible to machines. All the translations, keyword extractions and
topic classifications by NLP are then processed by Machine Learning to automate
responses depending on the chatbot user’s intent. Both NLP and Machine Learning are
subsets of AI chatbot which identifies and scans the human’s intent through the chatbot
interaction and provide accurate responses based on the intent. Such benefits and
challenges need to be clarified for a SME company, which will become part of the Thesis.

The thesis will start from the current state analysis based on Data collection 1 in Section
3; then it proceeds to exploring literature and best practice on developing and
implementing chatbot in Section 4; and based on the results from both, the thesis will
propose how the chatbot should be built based on the case company´s needs in Section
5. Finally, in Section 6 the thesis will describe how the chatbot should be implemented
in the case company’s environment. Since the thesis approach is action research, the
implementation part will require several iterations in Section 6. The feedback will be
provided by A/B testers against the pilot version, and it will be taken into consideration
in the next implementation round.
 4

2 Method and Material

This section describes the research approach, research design, and data collection and
analysis methods used in this Thesis. However, the most suitable research approach for
this project is Action Research, this section focuses on why this approach is selected.

2.1 Research Approach

Action research helps to address issues through dealing with context-bound knowledge
(Andersen and Blichfeldt 2006). The action-based research aims to recognize issues
within the organization or operations and as an outcome, to discover the answers and
finally find a possible solution to the problem. Additionally, according to Andersen and
Blichfeldt (2006), one of the responsibilities of the action researcher is to provide
transparent feedbacks or transferrable reports to all other audience/participants involved
regarding the findings, activities, procedures, and outcome of the overall study.

Figure 1 below shows how the elements of the Action research cycle are organized.

Figure 1. An extended action research model (Costello 2011, 9).

 5

As seen from Figure 1, comparing to other research approaches, action research is an

iterative action, so that the researcher can identify if the planned action towards a
problem is effective (done through Observe/Reflect), if not, the Plan must be used and
executed via Action until the expected outcome is achieved. Through the Action research
(AR) cycle, the action researchers will potentially discover a variety of possible solutions
as the researcher explores in a pandora box filled with diverse theory and practices.
(Coughlan and Coghlan 2002.)

In conjunction to the Action research approach, qualitative research makes the preferred
research family. According to Mack et al. (2005), the three most common data collection
methods in qualitative research are observation, interviews and focus groups.
Observation is used when the researcher/observer needs to observe the behavior of the
subject during the process. Observation is also useful if other methods such as surveys
or interviews are not successful. On the other hand, a focus group is conducted when
the target participants belong in a cultural norm of a group to gather overview concerns
from that group. Finally, interviews are a form of data collection where participants are
interviewed directly based on their experiences, views, and perspectives. All mentioned
data collection methods require active participation of participants as their input is
constructive to gain more understanding regarding the issue raised in the research.
(Mack et al. 2005, 31.)

Among different types of interviews, an in-depth semi-structured interview is the most

common of the data collecting methods. This technique is used to gather data from
individuals or groups. Depending on the topic of the interview, the interviewer must select
the right set of people who are associated with the phenomenon by firstly asking general
questions into more detailed ones. To convert the collected interviews into qualitative
material, the interviewer needs to define the questions, record the conducted interview,
and take field notes. This involves keeping track of which questions have and have not
been asked and answered, knowing how to phrase questions and encourage participants
to provide elaborative answers, including being friendly and accommodating in the
contexts of the interview. (Mack et al. 2005.)

In this study, the most applicable approach would be Action research, since this study
does not only include identifying the issues but also the actual development of the
chatbot. The continuous cycle of the Action research would help to identify issues, apply
the most applicable action to solve the issue, reflect on the pilot version iterated and
 6

improve them based on the feedback in the validation period. The input of each
participant in this research is highly valuable and will be considered to tackle the issues
that are faced in conversion marketing. Moreover, the collaboration between the
participants and the action researcher is crucial to determine issues in a particular
circumstance before proceeding to the study.

In this study, face-to-face interviews will be conducted with the internal stakeholders to
acquire a more comprehensive perspective regarding the current implementation of the
chatbot in the case company website. Additionally, follow-up interviews will be performed
at the final stage of the study to ensure that the issues defined at the early stage were
administered and addressed. As the first stage of the Action research is to identify the
problem, this would help to specify the requirements and implement them in the chatbot.
The interviews are scheduled as face-to-face meetings which concentrate on the issues,
they face on the current chatbot system. The questions are pre-defined and depend on
the role and responsibility of the respondent. Any field notes collected during the
interviews will also be checked with the involved participants, this is to ensure quality of
the field notes and this increase reliability.

2.2 Research Design

Figure 1 below shows the research design of this study.

 7

Interview with 4 key stakeholders:

CEO, Managing Director, Sales
Representative, Customer Service
Agent and Customers

Figure 1. Research design of this Thesis.

As shown in Figure 2, the first step of this study is to recognize the objective, and this is
determined by analyzing the case company’s business challenges. Next, Data collection
1 helps to gather the understanding of the current simplified version of the chatbot. This
will occur by interviewing 4 key stakeholders which include the CEO, Managing Director,
Sales Representative and Customer Service Agent of the case company and its
customers. The collected data from Data collection 1 helps to identify the issues
impacting the current marketing and business operations.

Next, literature review and the conceptual framework of how to build a chatbot will be
done. This step will also investigate the advantages and disadvantages of implementing
a chatbot but will focus on finding the best practices for developing chatbots.

From the knowledge obtained from literature and best practice and through the current
state analysis results, an initial proposal of how to build a chatbot will be developed for
the stakeholders. It will be done based on co-creation with the stakeholders and Data
collection 2 to identify the desired outcome with the stakeholders.
 8

Next, this initial proposal will be tested and validated, and an action plan will be built on
how to implement the proposed chatbot in the case company. It will be done with the
help of Data collection 3 used to evaluate the initial proposal (the chatbot build plus the
launch of the real chatbot) and suggest further improvements based on the pilot version
of the implemented chatbot. The improvements or fixes are done in iterations until the
desired outcome is reached.

2.3 Data Collection and Analysis

This study relies on a variety of data sources collected in three data collection rounds.
Table 1 shows details of Data collections 1-3 used in this study.

Table 1. Details of Data collections 1-3 used in this study.

 9

Participants / Data type Topic, description Date, Documented

role length as

Data 1, for the Current state analysis (Section 3 or 4)

1 Respondent 1: Face to face The case company OI May 2020, Web-based

CEO approach and overall sales 1 hour Survey and
and marketing process Field notes
2 Respondent 2: Face to face Interview about current May 2020, Web-based
Managing Interview process related to the 1 hour Survey and
Director respondent experiences Field notes

3 Respondent 3: Face-to-face Interview about current May 2020, Survey and

Sales Interview process based on the 1 hour Field notes
Representative respondent’s experiences as a
sales representative
4 Respondent 4: Face-to-face Interview about current Jan 2022 Web-based
Customer Interview process based on the Survey and
Service respondent’s experiences as a Field notes
Representation sales representative

5 Respondent 5: Survey and Survey created to allow Feb 2022 Web-based

Internal Email customers provide feedback Survey and
Customers conversation regarding the Chatbot’s email
functionality conversation
6 10 Customers Survey Proposal building March Questionnaire
questionnaire 2022, 2 responses
weeks

Data 2, for Proposal building (Section 5)

7 Respondents 1- Workshop/ Proposal building May 2022 Field notes

3: discussion 2 hours

Data 3, from Validation (Section 6)

8 Respondents 1- Group Validation, evaluation of the July, 2022 Field notes

3: interview/ Final Proposal 4 hours and recording
Participants: A/B presentation
testers 1 or 2

As seen from Table 1, data for this Thesis was collected in three rounds. The first round,
collection of Data 1, was conducted primarily for the current state analysis. The focus of
this round is to gather data on the case company’s current use of chatbot. The
respondents are internal stakeholders who have first-hand experience and know-how of
the business operation, overall sales, and marketing process. A face-to-face interview is
scheduled individually to avoid bias. Each respondents’ views and responses are
documented in a field note detailing the questions asked and answers provided.

Additionally, to have a deeper understanding on what were the actual issues or flaws of
the current chat system, it was considered best to consult the end-user to gather their
perspective regarding their experiences. For this study, more than 25 customers were
 10

contacted who were originally approached via the chat system, through email and phone,
if they were willing to participate in the research. However, only end-users that had
successful conversation through the chat system were contacted this time as their
opinions and views were relevant in this matter. The most effective method to gather
their insights was through a survey because many did not have time for a personal face
to face interview for this research. Furthermore, using a Google Form was considered
the most efficient and effective way to process and analyze their responses as, first, the
survey did not ask for any personal information (so using the Google form was judged to
be an appropriate method), and second, it produced a graphical overview of the results.

There was a long break during the stages in data collections, but the objective and goals
of this study and the services provided by the case company, as well as its staff and their
ways of working have remained the same since the initial stages of data collection and,
therefore, the findings were not impacted by the delay. The project timeline was then
adjusted accordingly.

In the next round, Data 2 was collected to gather suggestions from the case company
for developing the proposal. In addition, this data includes proposals for the chatbot
implementation which includes improvements based on the suggestions of the
respondents. Whilst identifying the features that can be implemented in the chatbot
system, Data 2 can also be used to discover if the current platform can handle all the
features as per the case company’s requirements. Furthermore, due to lack of resources
and budget, only features that can be developed in this study are considered and not all
proposed improvements are implemented.

In the third round, Data 3 was collected when conducting validation of the initial proposal.
Data 3 included feedback for the proposal from the case company upon testing the pilot
version of the chatbot. All feedback is welcome and in case of bugs/defects found during
the validation, it will be fixed within the project timeline and requires another round of
regression testing to validate the issue resolved. However, internal stakeholders act as
the A/B testers during the validation phase, their insight on the pilot version is important
to assess if the pilot version matches the case company’s needs.

As seen from the description above, interviews were the primary method of data
collection. The interviews were conducted as semi-structured, face-to-face interviews,
held on the company premises, with questions created in advance. The questions for 3
 11

interviews can be found in Appendix 1. The textual data was analyzed using
Thematic/content analysis.
 12

3 Current State Analysis of the Case Company´s Chatbot

This section discusses the business challenge regarding the current chatbot’s
functionality operating in the CleanLiving Oy’s website. However, this section also
focuses on the summary of the current state analysis and the issues identified during
Data collection 1. First, this section will overview the steps in the CSA and then analyze
and pinpoint the strengths and weaknesses of the current chatbot. The results of the
analysis will point out which areas the current chatbot may negatively impact.

3.1 Overview of the Current State Analysis

The main goal when conducting the CSA was to get an insight into the overall
functionality of the current simplified version of the chatbot. For this end, first, most of
the respondents were interviewed face-to-face individually, and field notes were taken.
The analysis was categorized into two groups: the internal stakeholders´ and the
customer’s perspectives

The first set of interviews was conducted with four internal stakeholders, where the CEO,
managing director, sales rep and customer service agent participated individually for an
hour-length interview. To have a better understanding of how the company’s business
operations and its strategies, the CEO was interviewed first, followed by the Managing
Director who is responsible for day-to-day implementation of the strategies. The Sales
Representative and Customer Service Agent were interviewed next to who are mainly
responsible for handling the chatbot conversations and marketing operation.

In addition, a survey in Google Forms was also collected from the customers to analyze
their views of the problems with the current chatbot and their needs. For this end, appx.
25 customers were also sent out an invitation to participate in this study. However, only
10 agreed to take part as they were simply busy with their personal lives. The customer’s
perspective was important to the case company to gather insights as the main user of
the chatbot. In addition to the fields notes taken during the interview, a survey through
Google Forms was asked for them to fill in to get a more detailed overview of their
viewpoint regarding the current chat system. They were sent out a respective Web-
based survey invitation through Google forms and questions asked about their
experience with the chatbot. The survey included predefined questions and answers for
the customers to easily select an option in the list which translates into a graph later. The
 13

questions were mostly created to analyze their satisfaction regarding their chatbot
interaction before becoming a regular client, and were fully anonymous (therefore, a
Google form was selected). At the end of the survey, they were also asked for their
recommendation and suggestions for the improvement of the chatbot to increase its
efficiency. Based on their responses, the Google Forms report presents the results on
the current chatbot system functionality’s strengths and weaknesses.

3.2 Description of the Current Chatbot (Design and Functionalities)

Before describing the chatbot, this section outlines the organizational structure of the
case company and its ways of working.

Presently, the CEO and Managing Director take full responsibility for the business
operations which includes monitoring progress towards goals, reviewing business
finances and hiring new cleaners. CEO usually monitors the success of the business as
his main responsibility and to ensure that the business goals stay in the right path. On
the hand, the managing director guides the cleaner professionals and other employees
to perform their tasks effectively and efficiently. In case of complaints and issues in the
operations, the managing directly typically finds the right solutions to solve these.
However, the CEO and Managing Director analyze the overall performance metrics of
the company together which includes evaluating sales, profit and customer overall
reviews.

The Sales Representative is responsible for getting new clients, meeting customer needs
and acts as the key point of contact starting from lead generation until the deal is closed.
His daily schedule is to contact any inquiries generated from all communication channels
which mainly come from emails, phone calls and the chatbot. The Customer Service
Agent works alongside him and any service orders that come from the communication
channels are passed onto the Sales Representative. The Customer Service Agent
responds to any type of inquiries and passes those issues to the right personnel, if
needed. There are currently 12 cleaner professionals that perform the main cleaning task
in the company, to deliver high-quality cleaning services and satisfy the customers.

Typically, about 40% of inquiries are acquired from the chat system in which 20% were
order requests. The other communication channels such as email and via phone are
used also to handle orders and inquiries. The website visitors get the case company’s
 14

details and information directly from their website and choose an appropriate channel to
contact the case company.

Figure 2. Current chatbot in the case company.

The current chat system was first launched in 2019, it was created by the CEO when
there was no customer service agent dedicated for customer support. Because the
current chat system does not have any other features than directly assigning each chat
interaction to a live agent, the creation of the chat system was very simple. There was
no testing period as the functionality of the chat system was very straightforward.

A notification is sent to the dedicated agents (including the CEO, Managing Director,
Sales Representative and the Customer Service Agent) with the chat contents. It
displays all the inputs of the website visitor, then the live agent can reply quickly to the
transcript sent. Basically, the transcript link directs the live agent back to the HubSpot
chat inbox where all the other messages from the chat system are accessible. There is
an automated message sent to the website visitor in case there is no online live agent to
assist him. However, this automated message only instructs the user to leave his
personal details, but these details are not collected in the CRM system. The Customer
Service Agent goes through the messages in the Inbox in the morning and acts
 15

accordingly. If there are complaints sent by the existing customers, then this issue is
consulted with the Managing Director. Otherwise, regular inquiries are scanned through,
and she collaborates with the Sales Representative who will be responsible for
contacting the new customers. They divide this task together and ask for guidance from
the management, if needed. Now, their daily work responsibility requires them to monitor
the chat activity during working hours and then the CEO and Managing Director take
over during off hours, especially on weekends and holidays. Monitoring and responding
to chat conversations requires a lot of manual work which also involves passing
information around.

The current chat system allows the end-users to send messages to live customer agents
in real-time. Currently, all the internal stakeholders have a responsibility to respond to
any chat interaction based on their availability and there is no allocated time for those
who will be online on certain period.

In the current chat, the customers are also encouraged to leave their personal contact
information in case there’s no customer agent to attend to their inquiries. As the company
currently employs one customer agent who is responsible for customer service inquiries,
sales, and marketing, he/she may not be able to perform all these tasks simultaneously.
Therefore, each customer interaction within the chat system cannot be attended
immediately. Hence, it may result in losing a potential lead if the end-user is not handled
instantly.

3.3 Analysis of the Current Chatbot (Design and Functionalities)

The analysis was categorized into two groups: the internal stakeholders´ and the
customer’s perspectives.

3.3.1 Internal stakeholders’ perspective

This section analyses the collected results of the internal stakeholders’ responses on the
(1) design and (2) functionalities on the current chatbot.

As for the design and functionalities of the current chatbot (which results in good
customer experience with the chatbot), the customers testified on receiving appropriate
 16

and relevant information during the chat interaction. For the customers, the chatbot is
available through the website that can be reached both via a mobile and from a PC.

At the same time, the internal stakeholder find that the limitation of the functionality is
substantial as it fully requires human action at every inquiry Internal stakeholder testified
on having access to the chat channel through his/her mobile phone and being able to
communicate via a mobile with the potential lead. They are sent a notification to their
emails immediately whenever there is a chat activity that requires attention. Then, they
can respond to the chat messages by logging into their HubSpot account via phone or
PC. In that sense, the chat channel is easily accessible and convenient for the
stakeholders.

The chat system usually sends out a notification to the internal stakeholders’ email
immediately as soon as there is an activity initiated in the chat system. Therefore, in that
case, if the assigned customer service agent is online during that time, any inquiries can
be addressed efficiently.

As for the Response time, Respondent 1 noted that there may be a considerable
difference in a response to different requests, depending on the time of the call:

“I was able to reply quickly to a question sent by a potential lead through

the chat system while having lunch. Immediately, I reserved a cleaning time
for a lead at the latest available time possible with my phone” (Respondent
1).

In some cases, when a potential lead tries to contact through the chat system during off-
hours, this may result in a loss in sales as a customer service agent may not be assisted
right away and there is no way to continue the communication with that lead if her
information is not collected.

“There was an order inquiry for a large residential home that was sent
through the chat system during the weekend and requests for urgent
cleaning. We tried to contact the user on the following working day, but the
user says that she has contacted other cleaning service provider that can
accommodate her needs immediately” (Respondent 2).
 17

There was also a case when other business organizations sent a message through the
chat system but due to lack of resources at that time, the stakeholders were not able to
respond instantly.

“There was a time when a big account was almost missed as we only see
their message 4 hours later, luckily they left their company details, and we
were able to discuss further regarding their request and needs.”
(Respondent 3)

Since other useful information regarding the services is already published on the website,
Questions coming from the leads are somewhat repetitive. The quality measure for FAQs
(Frequently Asked Questions) were only asked by the key stakeholders as they are the
ones who receive the questions. Respondent 4 adds that

“There were a lot of price quotation requests and job applications that the
end-users sent through the chat system. However, because there is no
way to identify whether each interaction is coming from a lead candidate
or a job applicant, it interrupts my focus on other customer service-related
tasks. In addition, we have a large of inquiries written in Finnish language,
as a non-Finnish speaker, it frustrates me that I can’t help the end-user.”
(Respondent 4)

According to Respondent 2, aside from price quotation requests, there were questions
regarding the service itself and mostly random job applications. There are also other
inquiries concerning some more specific details of the service, for example, the cleaning
supplies used for cleaning activities. For example, should the case company provide all
the needed cleaning supplies, or is it included in the cleaning fee already?

Additionally, the interview questions asked were concentrated on (3) the impact of the
simplified version of the chat system on the operational level. Since currently all the
internal stakeholders have a responsibility to respond to any chat interaction based on
their availability (and there is no allocated time for those who will be online on certain
period), all the internal stakeholders are involved, which often interrupts with their other
duties. For example, as the only person who speaks Finnish language is the Sales
Representative (whose main responsibility is conducting Sales and Marketing activities),
 18

his availability is limited when helping the customer service agent whenever there is an
end-user that speaks Finnish.

Thus, when (1) the questions come in the office hours and (2) there is a customer service
agent was experienced enough and knowledgeable, and (3) if the inquiries were related
to, for example, price quotations (i.e., the most frequently asked questions), the business
impact can be minimal.

Additionally, the chatbot makes a very convenient channel for the first contact and asking
for the price without revealing their identity (thus, proving a low threshold to contact the
company and a powerful channel for acquiring new customers). The customers can
conveniently send questions via the chatbot without revealing their identities yet and be
assisted in any concerns they have regarding the services. According to the key
stakeholders, about 45 customers including one-time customers and long-term
customers were acquired from the chat system this year. The customers’ information is
managed using the CRM tool in the HubSpot platform and customers were categorized
based on the communication channels they were generated from. This gives traceability
for the stakeholders which communication channel was effective for them then enhance
their marketing strategies accordingly. For example, if the lead was generated from the
Website or in social media accounts, they will boost their marketing strategy in that
platform to attract more customers.

Table 2 below summarizes the results from the analysis based on the most frequently
mentioned topics by the internal stakeholders.

Table 2. Most frequently mentioned topics by the internal stakeholders (in regard to (1)
design, (2) functionality and (3) impact of the current chatbot.

Topic

1. Design and functionalities (accessibility via phone and pc)

2. Response time (the challenge of responding in “off-hours”)

3. Repetitive questions (with the most needed information being the price
quotations), but also unfiltered (unrelated) questions

4. Impact on daily jobs (interruptions caused by responding to the chatbot)

 19

As seen in Table 2, the most frequently mentioned topics related to the chat system
overall impact and functionality to the key stakeholders. As per the design and
functionalities, one of the key benefits of the chat system is that it notifies the dedicated
live agents immediately and the customer service agent was able to help with the order
request, booked a cleaning schedule and close a recurring cleaning schedule deal.
However, this requires the customer service agent to monitor the chat activity by logging
in the HubSpot Platform during her working hours, otherwise she can access the
conversation inbox through her phone in case she needs to step away from the PC.

3.3.2 Customers’ perspective

This section analyses the customers’ responses on their experience with (1) design and
(2) functionalities on the current chatbot, as well as (3) their needs, hopes and wishes

As for the design and functionalities of the current chatbot (which results in good
customer experience with the chatbot), the customers testified on receiving appropriate
and relevant information during the chat interaction. For the customers, the chatbot is
available through the website that can be reached both, via a mobile and from a PC. The
customers testified that they enjoy having access to the chat channel through his/her
mobile phone and were able to successfully communicate with the company. Also, the
customers can conveniently send questions via the chatbot without revealing their
identities yet and be assisted in any concerns they have regarding the services.

Regarding the Response time, many of the respondents were satisfied with the response
time, as they did not have to wait long. At the same time, a few respondents did not agree
on having a good customer experience. It was due to the fact that their inquiries were
not resolved as quickly as expected (i.e. the response time).

For example, it happened in the situations when a potential lead asks for a price
quotation with her home size provided, but the current online live agent was not
knowledgeable on how to calculate the price based on the service that the potential lead
is interested in. The online live agent then asked the lead to provide her personal
information instead and she was contacted later via email with the price quotation and
details concerning her inquiry.
 20

It became clear from the analysis of the customer responses (as well as records from
the previous responses) that customers expect that the case company already has a
knowledge-based chatbot. For example, one customer sent a message through the chat
around midnight and expected to be replied to immediately. He obviously expected that
the chatbot would be fully automated and he would not disturb anyone by his request.

There was positive feedback regarding the customer service experience as one of the
clients said that the customer service agent was very helpful and friendly, and she
received an appropriate answer regarding her question. She was satisfied with the
solution that the customer service agent provided her with and proceeded to purchase a
cleaning order. In her case, she wanted to know more information regarding the tax
deduction from Vero if her “order” will qualify for the household credit. However, the
Customer Agent directed her to the official link in Vero where actual calculations can be
seen. The lead was then persuaded to proceed with the order as she sees that a big
chunk of the sum is going to be refunded to her through the household credit deduction.

About seven of the ten customers agreed that the response time was quicker than they
had expected. They were able to receive a reply whilst they were still communicating
through the chat system and did not have to wait long. There was a customer when he
initiated a chat on a holiday but was glad that there was an online live agent available
during that time. However, not all inquiries are attended in a timely manner as the key
stakeholders are only human and need time to load their energy during off-hours. Hence,
inquiries during those times either need to wait long until they are contacted back by a
team member, or their inquiry might be missed.

However, there was an odd question that came from a customer, “Do I have to clean the
house before the cleaner comes?” Perhaps she was not sure or embarrassed if the
cleaners found her home too dirty, so she boldly asked this question beforehand. The
customer service agent then confirmed that she does not have to do any preparation
prior to the scheduled cleaning as this is the main task and responsibility of the cleaners
but they would appreciate if she tidied up a little bit and ensure that her important stuffs
are placed in a secured area to avoid any accidents in the future.

In the survey, there was one customer who added that a price calculator would be a
great feature in the chatbot, so they can get a precise price estimate without bothering
the customer service agent. This will then give him enough time for decision making
 21

without revealing his identity yet. It will then reduce the amount of customer interactions
directly with the customer service agent, if all price quotation related matters are handled
by the chatbot. Table 3 summarizes the most frequently mentioned topics by the
customrs.

Table 3. Most frequently mentioned topics by the customers.

Topic

1. Generally Good Customer Experience / Satisfaction

2. Response Time / Customer Interaction Availability (generally evaluated as good,

but with some challenges in the off-hours)

3. Questions Asked (mostly the same questions from the customers, but with some
“odd” questions, too)

4. Wishes and needs (list of services and price quotation)

As shown in Table 3, about six out of ten customers testified having a positive experience
from the chat system and were impressed with the quality of the information received.
Most of the customers were impressed with the response time as mentioned above. But
there were some odd questions that customers were probably shy to ask.

There was a list of features that customers can select from, in the survey, to identify
which features that they find helpful which can be then considered in the development of
the chatbot. The majority chose “Show Services”, “Price Quotation” and “FAQ” from the
list which gives us a good set of recommendations for the proposal.

3.3.1 Selected Focus Areas

The main focus area, based on all inputs from the stakeholders and customers, is to
build a knowledge-based chatbot that will include the strengths of the old chat and will
improve all the weaknesses of the old cha system. This topic is discussed next, in Section
4, so that to find relevant knowledge and best practice for guiding the chatbot building
for the case company.
 22

3.3.2 Summary of responses

Data collection 1 analysis also gave a view on the usefulness, consistency,

responsiveness and quality evaluated by the users from both sides. It also provides
insights into the current chat system’s overall functionality and limitations.

Based on the findings during Data collection 1, the current chatbot has significant impact
on both parties, the internal stakeholders and the customers. Figure 3 below shows the
assessments by the internal stakeholders and external customers of the current chatbot
system´s Quality, Responsiveness, Information Accuracy and Usability.

System Quality

Responsiveness

Information Accuracy

Usability

0 1 2 3 4 5 6 7 8 9

External Customers Internal Stakeholders

Figure 3. Assessment of the current chatbot system´s Quality, Responsiveness, Information

Accuracy and Usability.

As shown in Figure 3, these four features were assessed by the customers and internal
stakeholders’ responses in the survey and in the stakeholder interviews. The responses
gathered in Figure 3 come from two groups of respondents: 4 internal stakeholders and
10 customers. The results show a significant difference between the two groups’
responses to the questions pertaining to the system quality, usability and accessibility of
the chat system. The survey contains a set of Yes/No questions pertaining to the topics
in the above graph. The values presented above are derived from the answers collected
with “Yes” responses.
 23

First, quality was to assess the quality of the information the customers received if they
were contented with the information and their inquiry was fulfilled. For example, the
customers can conveniently send questions via the chatbot without revealing their
identities yet and be assisted in any concerns they have regarding the services. Some
complex issues such as existing customer complaints can also be handled effectively in
the chatbot.

In terms of responsiveness, about half of the internal stakeholders do not believe the
system is responsive enough, since they cannot respond to the chat interaction as soon
as possible due to simply not being available during that time. Some customers provided
valuable feedback following a chat interaction which rated the chat experience to a
positive one. While some customers, we’re not quite pleased with the overall experience
because they were not responded quickly as they expected.

In terms of the information provided by internal stakeholders in response to questions,

the customers were most pleased with this. It was evaluated high, as seen from Figure
3 above. This is, most likely, because the customer service agent was experienced
enough and knowledgeable if inquiries were related to price quotations (i.e., the most
frequently asked questions). On the other hand, the situation might have been different
due to language limitations, for example, or absence of the experienced and
knowledgeable stakeholder to respond.

For the usability measure, the chat system platform was easily accessible with mobile
and PC by the website visitors and because it performs like any messaging channel, it
works as if the user is communicating in any communication platform. In that case, the
website visitors do not need to learn how to use it. However, usability did not get a higher
score, perhaps some customers incorporated their views in responsiveness measure as
they were not responded quickly.

3.1 Key Findings

This section summarizes the key findings form the current state analysis.
 24

3.1.1 Fit for Purpose (How successfully it is currently used in customer service)

As seen from the analysis above, the chatbot currently boosts operational efficiency by
offering convenience to both internal employees and potential clients. According to the
interviews conducted with the internal stakeholders, they can instantly provide answers
to the client by chatting on their phone conveniently. As the chatbot connects directly to
the live agent, whoever will be available at the time when an interaction was initiated, will
then continue the conversation with the potential lead. When a potential lead has been
assisted successfully and decides to avail themselves of a service, she/he will be then
considered as a client.

3.1.2 Fit for Customer Needs (How well it serves the customers)

Usually, the clients use the chatbot for sending messages during the off-hours: night or
weekends when it is unconventional to contact via phone calls. They can conveniently
send questions without revealing their identities yet and be assisted in any concerns they
have regarding the services. It is also a channel, where leads request for price
quotations depending on the size of their property and services that they are interested
in. In addition, leads can view the list of frequently asked questions and answers
corresponding to them. Some complex issues such as existing customer complaints can
also be handled effectively in the chatbot.

3.1.3 Strengths and weaknesses of the curertn chatbot

The findings can be grouped by identifying the strengths and weaknesses of the current
chat system’s functionality. Figure 4 below summarizes the key points of the current
chatbot in terms of strengths and weaknesses.
 25

Strength Weakness

A low threshold for the first contact with the Lost sales: no availability of customer service
company & asking for quotation agent in off-hours

At the end of the conversation, information of the

Engaging and interactive platform lead is not gathered and customer satisfactory
was not asked after the conversation

Dissastifed with the current simplified versfion of

Can handle complex issues the chatbot (very manual! cannot handle e.g.
repetitive questions, off-hours, ect)

Lead is often transformed into a customer after a Live agent is not knowledgeable on certain areas
successful chat interaction / Finnish language was used by the end-user

Customer is then enrolled in a defined workflow

Figure 4. Strengths and weaknesses of the simplified chat system functionality.

First, although the limitation of the functionality is substantial as it fully requires human
action at every inquiry, it is believed to be functional enough to resolve complex issues,
according to the Internal Stakeholders. Secondly, they are also able to communicate
directly to the potential lead by responding through their mobile phones at any time of
the day, but this also requires them to step away from whatever they are doing to be able
to assist the lead. After successful customer interaction, the customer journey continues,
and the potential lead will be transformed as a client and will be enrolled to a relevant
workflow for future promotional advertisements. Third, it is functional enough to solve
most of the concerns or issues by having a conversation just like in a simple messaging
channel.

One of the main weaknesses of the current chat system’s functionality is there are cases
when a potential lead sends a message during off-hours, and no one was available to
respond. Hence, this may result in a dissatisfied lead. Another weakness is when the
assigned customer service agent is not too knowledgeable on the area where the
potential lead is asking about for e.i price quotations and if the message was in Finnish
language. In addition, questions coming from the end-users are usually repetitive, hence
it may be tiresome for a customer service agent to reply to the same questions
constantly.
 26

3.1.4 Direction for improvement

Therefore, about 30% of the internal customers see the advantage of transforming the
simplified chat system into a functional knowledge-based chatbot.

When it comes to improving the chat system, Respondents 1-4 acknowledged the need
for improvement as they see the benefits of having a knowledge-based chatbot on the
customer journey. Respondent 2 admits:

“It would be great to know what’s the intention of each chat interaction at
the beginning of the conversation so we can better prepare and respond
efficiently with appropriate and relevant information” (Respondent 2).

The internal stakeholders also laid out some improvement suggestions and added
features to the chat system that can handle manageable inquiries such as:

- Booking a Cleaning Schedule

- Price Quotations

- Show Services

- Leave a Contact Request

- Feedback gathering

- Multilingual Chatbot

- FAQ

- Careers.

All the listed items above are from the suggestions of the key stakeholders, mostly
because these have been a hot topic in the chat inquiries. Most of the inquiries are asking
about the services for example what is the minimum duration to be able to book a
cleaning service as the case company requires to have at least 3 hours of minimum
cleaning hours regardless of the property size. It is important that both parties agree on
 27

the hours’ estimation, before confirming the cleaning schedule to avoid confusion in the
future.

For example, if an order request was sent in the chat system, ideally the website visitor
wants to know the most available time that matches her preferred schedule. Instead of
manually asking the customer service agent what the available time slots for the cleaning
are, a booking system gives a view to all the available times that she would like to book
a time with. The price quotation is a great feature to give price details before proceeding
to book a cleaning schedule.

When the customer service agent recently joined the marketing/customer service team,
she started receiving similar questions about the cleaning supplies. Previously, she was
a cleaning professional, so she wasn’t aware of what was included in the cleaning fees.
However, with the help of the sales representative, they listed down all frequently asked
questions with relevant answers as an internal document and she goes through this
material in case there’s a question that is not familiar with.

Leave a Contact Request feature is merely for lead generation, for the customer service
agent to follow up with the client regarding her initial inquiry. In addition, the customer
service agent doesn’t speak Finnish, hence all the Finnish inquiries are passed onto the
Finnish-speaking Sales Representative for further discussions. The customer service
agent wishes to have a Finnish-speaking chatbot to solve these inquires, so she doesn’t
need to forward these issues to the sales representative.
 28

4 Existing Knowledge and Best Practice on Building Chatbots

This section discusses the existing knowledge when developing an effective knowledge
based chatbot solutions. However, it can potentially solve the issues identified in Section
3 if the objective and the overall design of the chatbot follow best practices when building
it.

As there are numerous amounts of service providers that offer Chatbot solutions without
programming it from scratch, Section 4.1 covers some of the most used platforms when
creating a bot. It also provides some golden rules and what characteristics are needed
when implementing one, to make it as engaging with the leads in their first interaction.
Section 4.2 explains the chatbot scenario which begins right from clicking the chatbot
icon on the website, through providing quick and efficient answers to their questions and
asking for the user’s feedback. Lastly, a step-by-step guide is shown on Section 4.3 on
how to build a knowledge based chatbot in HubSpot and to utilize the platform to create
an efficient and innovative chatbot.

4.1 Existing Chatbot Solutions and Benchmarks (How they are designed)

Nowadays, creating a conversational chatbot does not require advanced coding skills as
many platforms allow users to develop a chatbot for lead generation without technically
writing a line of code. It can be integrated into communication channels easily and start
deploying it as a customer service agent. “At the heart of chatbot technology lies natural
language processing or NLP, the same technology that forms the basis of the voice
recognition systems used by virtual assistants such as Google Now, Apple’s Siri, and
Microsoft’s Cortana.” (Shewan, 2022). Conversational chatbot are becoming popular in
digitalized businesses partly because developing them from scratch is not needed and
some platforms offer the ability to create actions with their drag-and-drop interfaces.

According to Singh et al. (2019), there are two types of Chatbot Development
Approaches: AI-Based and Menu-Based Approaches. In an AI-Based Approach
Chatbot, when the end-user inputs or selects an element shown by the Chatbot, the NLP
engine extracts the information by the user input then decides the outcome of the
conversation from the data system. On the other hand, a menu-based approach Chatbot
presents a menu to the user to choose from, then those menu items are directed to the
 29

following actions, the advantage of having a menu-based Chatbot is that the response
is accurate as the design for the chatflow. (Singh et al. 2019.)

“Chatbots – also known as “conversational agents” – are software applications that mimic
written or spoken human speech for the purposes of simulating a conversation or
interaction with a real person.” (Shewan, 2022). Today, as chatbot technology is
becoming more advanced, it is difficult to identify whether it is an automated chatbot or
a human speaking with the end-user.

Below is the list of the most used chatbot frameworks that can be integrated when
building own channels.

Chatbot allows to create a customizable lead-generation Chatbot that can be added to

the website as a widget. It gathers the potential lead’s data and information and prompts
the user to subscribe to their Newsletter at the end of the conversation. In addition, to
shorten the sales cycle, prospects can be directed to the live customer service agent for
further discussions. Sales processes are automated as it collects data of the potential
leads, and these data are sent to the CRM system. (Chatbot 2022.)

HubSpot Chatbot Builder offers an easy-to-use Chatbot builder which can be set up with
no coding skills. It can pull data directly from the Customer Relationship Management
system to generate personalized messages depending on the target audience and vice
versa, all data reflected from the Chatbot interactions are passed to the CRM. For
businesses who would like to have a book a meeting functionality, HubSpot Chatbot
Builder is a great option as it can automate this task. To create a knowledge-based
operated chatbot, publish a knowledge-based library from which the chatbot collects the
most appropriate answer depending on the end-user’s input. (HubSpot Chatbot Builder
2022.)

Intercom offers “Resolution Bot” that is customizable, but the pricing depends on the size
of the business size and needs. It can be integrated easily to many communication
channels and can be developed multilingually. However, Resolution Bot can be tailored
and can define specific answers for each targeted group of audience for example, the
Chatbot can send tailored response to website visitors that use different devices or plans.
(Intercom 2022.)
 30

Salesforce Einstein uses machine learning and Natural Language Processing, like all
other Chatbots, to recognize and process the end-users’ questions. However, their AI
powered Chatbots built right into the Salesforce Platform which gives smart predictions
by processing the past data and apply those predictions and use existing rules and pass
the case to the respective live agents. (Salesforce Einstein 2022.)

WP-Chatbot is suitable a WordPress website, WP-Chatbot is a plugin that can be

installed in just one-click. With its easy-to-use platform, it is easy to create automatic and
manual chatbots that can be integrated with Facebook. Conversation History is easily
accessible through Facebook and team members can access the messages while
marketing business page on Facebook. (WordPress-Plugin Chatbot 2022.)

There are many platforms that offer to create AI chatbots with no programming required.
However, testing different solutions is recommended to find the right solution that fits the
business needs and intentions. If a business uses a CRM system application, it is best
to utilize their applications so it can be easily integrated to the CRM system when
qualifying leads. The team members can then work alongside the AI Chatbot for contact
and lead management.

4.2 Best Practices on Building a Chatbot

In order to create an AI Chatbot, business practice has developed strategies and

approaches to follow before implementing them into a business. The golden rules will
help to avoid most common chatbot mistakes that can potentially give negative
impressions to website visitors. Therefore, this section focuses on how to create an
effective chatbot strategy that needs to be considered before developing a chatbot.

First comes the Planning stage. According to Mousumi (2021), the planning stage is an
important step to define the business objectives and goals at this stage. Will the Chatbot
be specifically designed as a customer support or to help qualify leads? These objectives
must be clearly defined prior to building a Chatbot to avoid having an unreliable and
inconsistent Chatbot. List the business issues and needs to identify the focus areas and
determine the purpose of the Chatbot. Rank all the issues identified and pick up the most
important area that needs to be resolved or automate with a Chatbot.
 31

Once the objective and goals are defined, create a simplified mind-map or a
conversational flow and plan a chatbot story. Basically, creating a chatbot story visually
tells how each chatbot scenario is built and it defines the scope of each feature (Singh,
Rama Subramanian and Shiva, 2019). It begins from a start-to-end conversation and
each possible chatbot scenario should be covered at any instance. Refer to Section 4.3.
Third, find the best solution where the chatbot can be developed with, refer to Section
4.1.

Second comes the Building stage. In order to build an effective and engaging chatbot,
be mindful of the chatbot’s greeting (Mousumi 2021) and it should reflect the company’s
values and brand. As this is the first interaction of the potential lead with the company, it
is necessary to have a good impression to build integrity and trust right from the
beginning of their customer journey. According to (Alburger, 2022) adding a little humor
to the greetings and responses can give a relaxed atmosphere but don’t go overboard
and ensure that the personality reflects the tone and voice of the organization. Be
creative and use a catchy welcome message then provide a message right away to
motivate the website visitor. Below is an illustration on how the “welcome greeting can
be modified”

Figure 5. HubSpot, Create a Live Chat, Welcome Greetings (HubSpot 2022).

One can define the Chatbot’s personality by allowing to communicate in informal or

formal language to the targeted audiences but be transparent that they are
communicating with a Chatbot instead of a Live customer service agent. Because the
 32

website visitors might get frustrated if they do not receive the best solution from the
Chatbot. (Cornell, 2022.)

Adding buttons are a good way to help the website visitors go through the Chatbot story
for example, if they would like to know about the services or Request a Booking directly.
The example is a menu-based approach as per (Singh et al. 2019.)

Figure 6. HubSpot Knowledge Base, Use if/then branches in bots (HubSpot 2022).

With the options provided, the website visitor will achieve their goal without writing
anything on the chat. Behind those buttons, the user is then directed to their respective
chatbot stories. The next step is to add context to each button so they know what will
happen to them after clicking any of the options available. (Alburger, 2022.)

Lastly, it is recommended to define the correct fallback message in case the chatbot
cannot pull any answers to match the user’s questions. This is useful if the chatbot is a
knowledge-based bot that requires user’s input but remember to instruct the user what
to do in this case. Give an option to go back to the beginning and try again. The example
is AI-based approach. (Singh et al. 2019.)
 33

Figure 7. HubSpot Chatbot 2022, Choose your bot action, Book a Meeting (HubSpot 2022).

Next is the Monitoring stage. Before deploying chatbot to communication channels,

testing it is one of the most important phases. This will help verify if the chatbot is working
as expected and the chatbot stories supposedly lead the website visitors to the right
actions. It is important that other team members can help and act as beta testers to also
provide constructive feedback as they can see many things that were unnoticed during
the development phase. This is a good chance to improve in case of inconsistencies
found during testing. Once the chatbot is up and running, observe and get to know the
users. This will give insights regarding their interests and update the chatbot
implementation accordingly. (Cornell, 2022.)

4.2 Using Chatbot Scenarios for Chatbot Building

Business practitioners (HubSpot 2022, SAP 2020 etc.) suggest that the key aspects of
the chatbot need to be identified first, the business needs and what is the reason why a
chatbot is needed. As per Jaramanan (2020). “In the bot design process, the Use Case
design stage focuses on building the conversational workflows that will perform actions
for the user. This helps in automating all the use cases and results in the bot responding
relevantly to the users.” Determining each user cases for the chatbot is an important step
as this will visually identify each chat stories from the start to the end of the conversation.
 34

Figure 8 below shows a sample of use case when developing a Chatbot with SAP
Conversational AI bot-building platform

Figure 8. Example of a chatbot used to track parcels (SAP 2020).

Figure 8 above shows how the chat user can track his parcel by providing a valid tracking
number to the chatbot. The relevant status of the parcel is then displayed from the API
used to pull the information from. If the user doesn’t want to proceed to find out the parcel
status, he is directed back to the first question, all user cases including negative use
case must also be covered, to avoid any fallbacks when the chatbot is deployed. “In this
phase, we need to focus on the nominal flow. In other words, a straightforward scenario
where everything goes according to plan. The user asks all the right questions, and the
bot understands and provides them with exactly the info they asked for.” (Jaramanan
2020.)
 35

4.3 Building a Chatbot in HubSpot (How to build, step by step)

HubSpot (2022) recommends using conditional logic to create a more interactive

experience for the website visitors. When building a Chatbot in HubSpot, HubSpot
recommends connecting the chat channel to the conversation inbox and the tracking
code must be added to the website if the chatbot is decided to be available on the
website. After this, the chat sessions are assigned to team members based on their
availability and also to control who will be receiving incoming messages. (HubSpot
2022.)

Figure 9. Choosing a Bot, Offline Bot (HubSpot Chatbot 2022).

Figure 10 below shows how to customize the welcome greetings which will be first
displayed to the website visitor. Depending on the target audience, you can assign a
formal or informal message to greet your users. (HubSpot 2022.)
 36

Figure 10. Customizing a welcome message (HubSpot Chatbot 2022.)

Other way to do it is by sending simple messages which do not require a response yet
to introduce the chatbot first then ask a question about the website visitor to gather initial
information before a live agent needs to take over the conversation. If the chat session
is then abandoned, the live agent can follow-up with the website visitor by contacting her
with the details provided. (HubSpot 2022.)

Figure 11. Choose a bot action “Book a Meeting” (HubSpot Chatbot 2022).
 37

Figure 11 above shows how the chatbot collects initial information before going deeper
with the conversation.

Figure 12. Providing Actions (HubSpot Chatbot 2022).

Figure 12 above shows how action-based responses are then added to identify the intent
of the website visitor. Based on the intent of the user, she will select the relevant action
in order to proceed to the following actions until her intention is fulfilled. (HubSpot 2022.)
 38

Figure 13. If/then branch (HubSpot 2022).

The next step is assigning the conditions for the If/then branch to direct the website
visitors to a specific path. For example, website visitors are sent to different bot actions
depending on the selected action, if there was a “Yes” or “No” options and the website
visitor chooses “Yes” to the chat question “Would you like to be connected to the live
agent?” . Once “Yes” is selected she will then directed to the assigned live agent.
(HubSpot 2022.)

Figure 14. Showing a consent banner to the visitor on the exit intent (HubSpot 2022).

Figure 14 above shows that the website visitor needs to enable the cookie consent text
and it is also a consent to collect chat cookies. The user is informed with a message why
the chat cookies are used. To maintain the chat conversation while the website visitor
goes to the other pages, the cookie request must be toggled so the chat session does
not get aborted and timed out. (HubSpot 2022.)

Once the chatbot is ready and all actions including its conditions are assigned
successfully, the chatbot can be deployed to the assigned communication channel. Bot’s
performance can be then analyzed weather it has been helpful in the customer ends or
not. (HubSpot 2022.)
 39

Figure 15. A Dashboard to analyze the bot’s performance (HubSpot 2022).

To determine how many chat sessions were established and how many of them reached
until the end of the conversation, the chat session rate is seen in the Dashboard in the
HubSpot platform. The traceability between the lead generated from the chatbot system
can be easily accessed here and a detail on sessions aborted and their time spent
interacting with the chatbot. (HubSpot 2022.)

4.4 Conceptual Framework of This Thesis

This sub-section contains a visual representation of this study’s conceptual framework

and its elements that contain identified tools and suggestions on how to building a
Chabot.

As presented in Figure 16 below, the first step was to Select the platform for building a
chatbot by analyzing existing AI Chatbot solutions, which refers to Section 4.1. Although
there are many platforms that offer easy to use chatbot builder to create successful
chatbot with no coding skills required, it is best to try out a few chatbot platforms before
finally publishing it to the public. Before analyzing and selecting which one is suitable for
the business needs, business practitioners suggest to make a list of all the business
issues that a company is currently facing and find the most relevant chatbot platform that
can solve those issues. AI chatbot can typically automate tasks which can eliminate
human effort and boost the team members’ productivity. The chatbot solutions provided
in Section 4.1 are the most used chatbot platforms that are customizable and can be
integrated to communication channels easily.

Second, before developing a chatbot, it is important to Select relevant strategies before

embarking on the building. As chatbot represents the company’s values, the first step
would be to ensure that that the chatbot can also communicate formally or informally to
align with the company’s values and brand. The Planning stage helps recognize how a
chatbot should be implemented and how it is expected to work. However, the Building
stage is the actual development of the chatbot where the personality of it can be
 40

assigned. Lastly, the Monitoring stage helps to update the chatbot depending on the data
that has been collected from the website visitors.

Figure 16. Conceptual framework of this Thesis.

The third element of the CF is to determine and Use chatbot scenario depending on the
findings in the planning stage whether the chatbot works as a customer support or to
qualify leads. A sample of user stories are provided for each chatbot story that was
illustrated in Section 4.2.1 and this is then presented to the key stakeholders when
planning the chatbot features. However, this illustration will be used as a pattern when
building the chatbot for the case company.

Lastly, the developers will need a step-by-step guide for Building a chatbot. This study
has selected HubSpot for such guidance as shown in Section in 4.3, due to particularly
clearly showing how to easily create a chatbot by HubSpot.

Next, Section 5 applies this guidance developed in the conceptual framework shown in
Figure 16 for building the actual AI Chatbot for the case company.
 41

5 Building the Chatbot for the Case Company

This section merges the results of the current state analysis and the conceptual
framework towards the building of the AI Chatbot using Data 2.

5.1 Overview of the Building Stage

This section presents the steps in building the AI Chatbot with the findings identified in
CSA whilst applying all elements established on Section 4. As the current simplified
version of chatbot seeks improvements as per the output from CSA.

The results of CSA show that the current chatbot implicates several concerns that were
time, customer and budget related issues. First, the Internal Stakeholders admitted that
hiring a customer service agent that could be available responding to chat messages
24/7, is highly costly and unreasonable. Some external customers have complaint that
the resolution time was rather slower than expected. This results in low customer
satisfaction and customer data collection is overlooked if the customer was not attended
to on time.

Second, before transforming the chatbot into an AI knowledge based chatbot, it’s
important to also get familiar with other chatbot platforms that could be considered other
than sticking with HubSpot as in the initial plan. However, the Internal Stakeholders
agreed to stay with HubSpot after other chatbot platforms were proposed. Third, a
meeting was held among the Internal Stakeholders to identify what features are the most
important to implement in the chatbot. Therefore, Illustration 1 in Section 4.2 was also
presented to them, to have a visual idea on how the chatbot stories will flow and what
elements can be added. They came up with suggestions on how the chatbot should be
named, other additional features and setting the chatbot into a multilingual so it’s able to
interact with website visitors in Finnish as well.

5.2 Initial Inputs for Building the Chatbot

For Data Collection 2, an interview was conducted to collect suggestions from key
stakeholders following the findings from CSA that were presented to them. These inputs
will be put into consideration whilst producing a plan on how and when these features
 42

will be implemented. Each key stakeholder gave their own proposition that were affecting
certain business areas and believe that improving the chatbot could potentially resolve
those issues. The main inputs are listed in the below table according to each key
stakeholders’ suggestions.

Table 4. Key stakeholder suggestions (findings of Data 2) for Proposal building in relation to
findings from the CSA (Data 1) and the Conceptual framework.

Key focus Input from Suggestions from Description of their suggestion (in
area from literature and best stakeholders for detail)
CSA (from practice (CF) on the Proposal,
Data 1) the topics of: summary
(from Data 2)

1.Improving 1.Selecting the a) Developing a According to the CEO, if the

the simplified platform chatbot which can answers are automated, it can
version of provide price eliminate such mundane tasks
Chatbot quotation and lead from their side as any machine is
2.Selecting the
system and generation. proven to work quickly if utilized
strategies
its limitation correctly.

3. Using chatbot
2. Building a b) Booking system, The Sales Representative and
scenarios
Job opportunities, Customer Service Agent believe
knowledge-
List of Services, that a responsive booking system
based
4.Building an chatbot’s ability to developed within a chatbot can
version of
automated communicate both effectively ease their work as their
the chatbot
knowledge-based in English and effort to manually book a time for
chatbot (based on Finnish a client is reduced. In addition, a
a library for the separate query for job applicants
frequently asked as this subject is handled
question) separately.

c) List of Frequently According to the Managing

Director, she heavily suggested to
Asked Question
have a list of Frequently Asked
Question where leads can go
through and find the most suitable
solution regarding their concern.
Basically, the data from the FAQ
are also applied within the chatbot
System.

As seen from Table 4, the CEO suggests having a price quotation feature as there are
times when a live agent was not up to date with the current pricing of the service.
Therefore, automating the response for the chatbot will reduce human effort on providing
price quotation depending on the property size of the client. For sales and marketing
purposes, one of the main advantages of having an AI Chatbot is that it generates leads
by helping them turn their interest into sales. Therefore, the CEO believes that if the AI
chatbot can qualify leads and collect data, it will boost their marketing approach.
 43

Basically, their personal information is then passed onto the CRM system and continue
their customer journey from there.

Because the sales representative and customer service agent take full responsibility with
the chatbot activity, they have primary experience on what the inquiries were mostly
about. According to the customer service agent, about 40% of the inquiries were
regarding services in which similar questions were asked frequently. About 20% of the
website Visitors, asked for a cleaning scheduling and 20% for price quotations. In
addition, they have also received many job applications, which is about 20% of the
inquiries from chatbot. The customer service agent added, that if each activity from the
chatbot can be grouped so it doesn’t clutter their inboxes for example, having all job
applications in a separate group so once there is a job opportunity available, they can
easily scan each application and find the suitable candidate for the job.

A meeting was held with the managing director to list all the frequently asked questions
and prepare an answer for the selected questions. Ideally, this set of questions and
answers will be saved under the Knowledge Base Library in HubSpot where the chatbot
retrieves information from.

5.2.1 Planning the Chatbot

Step 1 in Planning: Developing the Chatbot scenarios

Below is an illustrated mind-map on how the customer journey flows in scenarios where
the end-users may lead when choosing elements on the chatbot system of the case
company. Ideally, these chatbot scenarios presented below are used as a guide when
developing the AI Chatbot for this study in Section 5.
 44

Figure 17. Chatbot scenarios with possible chat stories.

The Start Point represents as the beginning of the chatbot interaction established, the
chatbot will then immediately respond by providing the list of options where the user can
select from. It will show a pre-defined list which requires an action from the user to
proceed further. Each scenario will be explained below:

Chatbot scenarios

- Scenario A. User wants to be familiarized with the services that the company is
providing. Firstly, Bot welcomes the user and is shown four elements such as
Show Services, Price Quotation, FAQ, Support, Careers and Request a Booking.
User then selects “Show Services”, Bot quickly pulls a set of elements that were
defined in the library which contains the list of services. Based on the selection,
Bot shows the details and price in a short, summarized form and then the user is
given an option to proceed to Schedule a Cleaning or price quotation. The user
is then asked whether she/he wants to be contacted by the sales agent or wants
to contact the sales agent. The website visitor is then also given an option to
 45

proceed with the booking system. At the end of the conversation, feedback will
be collected from the user to assess if the inquiry is solved by the provided
information. The feedback is later utilized to further develop the chatbot system.

- Scenario B. User selects “Price Quotation” from the list of elements, Bot will then
ask the size of the property that needs cleaning. A rough price estimate of the
service will be then shown to the user then she is given an option if she/he wants
to be contacted by the sales agent or wants to directly contact the agent instead.
The Website visitor can also proceed to the main booking system and choose the
service she/he would like to avail himself. Like in all other scenarios, feedback is
collected at the end of the conversation.

- Scenario C. If the user has an idea of what to ask, she can proceed to the FAQ
option and input keywords regarding her questions. For example, by writing a
“holiday” text, the chatbot pulls the relevant information from the knowledge base
library and displays the most appropriate answer to a question.

- Scenario D. User can directly be connected to the live agent or currently online
customer service agent and continue their communication further. At the end of
the conversation, the user can give feedback based on the chat experience.

- Scenario E. In the case of job applicants. They can choose “Careers” from the
options provided where they can leave their personal information and their
resume in pdf form. Later, if there is a job position available, the team member
will scan through the applications and contact the right person suitable for the
job.

- Scenario F. If the user selects “Request a Booking” option, this will direct the user
to the right website URL to proceed to the booking system. There is a separate
booking system for each service as the length of minimum cleaning hours varies
with each service. The Customer Service Agent who is responsible for task
assignment to the cleaners can find the right and available cleaner based on the
service and time booked.

Step 2: Implementing a Knowledge Base Chatbot in HubSpot

 46

The chatbot solution chosen to implement is a knowledge-based chatbot is with HubSpot

as the simplified version of the chatbot is developed with it. This section focuses on how
to build a Knowledge-based chatbot with HubSpot with a step-by-step guide.

Step 1: Create an account in HubSpot and connect a channel to the conversation’s

inbox. Then connect the website to the chatbot by adding the tracking code to the
external pages where the chatbot widget is going to be displayed.

Step 2: In the HubSpot Account, click Conversations > Chatflow and a new page
appears.

Step 3: Click the Create chatflow in the upper right corner.

Step 4: Select Website if it needs to be integrated with the website or Facebook

Messenger if the chatflow is intended for Facebook. Click Set up Chat at the pop-up
message.

Step 5: Personalize the accent color of the chatbot, the color chosen will be
visible in the icon and the header’s background

Step 6: Customize welcome greetings as this is the first message that the website
visitor sees, make it as engaging as possible. Give a name for the chatbot in case
personal name is not used. Other details can be modified later.

Figure 18. Chatbot, “Welcome Greetings”.

 47

Step 7: Select a time frame for when the chatbot should be available. If this relies on
the Live Agent’s availability, adjust the time availability according to their working
hours. Otherwise, the chatbot can be available 24/7 by clicking the check box “Chat is
available 24/7”.

Figure 19. Chatbot, Availability behavior.

Step 8: Proceed to preview and install the code snippets by copying the tracking code
to the website.

Step 9: Alternatively, if there’s one chatflow previously created. When creating a new
one through the Create Chatflow in Step 2, it shows template options, and each has
features already built with. Select the most appropriate bot template according to the
needs.

• Live Chat – Connects directly to the online customer service agent

• Knowledge base + live chat – Use this template to share knowledge base
articles to the website Visitors. This is very useful if there is a list of frequently
asked questions to eliminate time spent responding to repetitive questions.
• Concierge bot – live chat and knowledge base templates are combined in the
concierge bot.
• Qualify leads bot – gather information of the potential lead by asking their
reason on visiting the website.
• Meetings Bot – share a meeting link where they can conveniently book a time
with the team.
 48

• Tickets Bot – Use this template as a visitor support inquiry. They can create a
ticket regarding their issues then update the tickets when they are addressed.
• Offline Bot – If the chatbot is set to be available only during business working
hours when a customer service agent is online. Use this template to gather
website visitor’s email when it’s offline so the customer service agent can later
contact the visitor for further discussions.
• Start from scratch –choose this to build the bot from scratch then later bot
actions or features as it is developed.

Step 10: For demo purposes, if the Knowledge base + live chat template is chosen, this
prompt to first publish at least 3 articles where chatbot can retrieve the data/information
from.

Figure 20. Chatbot: Knowledge Base articles.

The chatbot scenarios and stories were created during the Planning stage, that is used
as a guideline when implementing each feature and its relevant chat scenarios in the
chatbot. The features considered were from the suggestions of the key stakeholders
which includes Price Quotation, Show Services, Booking System, Support, Careers,
FAQ and multilingual chatbot. However, these features are developed in two separate
sprints due to time constraints and resources. The features are prioritized based on the
risk level and possible impact to the company’s operation.
 49

5.2.2 Building the Chatbot

After the account has been created following the steps in Section 4, the building of the
AI Chatbot starts for the case company.

Step 1: Assigning a Name and Personalizing a Welcome Greeting

The key Stakeholders chose the name “Mari” for their Bot and a welcome greeting is
assigned afterwards. As part of the best practices, Bot should be engaging and
informative to ensure that it has the ability to help the website visitor.

Figure 21. Chatbot scenarios with possible chat stories (“Select a Reason”, on the right side,
and “Chatbot Output” on the right side).

In order to list the elements that will be presented to the Website Visitors, the features
suggested in Section 5.1 can be added as action bases responses. Basically, it prompts
 50

the user to select an option before proceeding to the following pages. The website
visitors will be then directed to their targeted action from the quick replies selected.

Step 2: Assigning If/ then branches

Figure 22. Chatbot scenarios with possible chat stories (“If/Then branches”, on the right side,
and “Assigning the next action” on the right side).

Each action should be connected to their respective actions in the conversation through
the following if/then branches. Basically, the purpose of the if/then branch is to assign
the next action if the website’s visitor selected “Show services” as an example above in
Figure 14. Assign all of the quick-replies options to their respective actions.

Step 3: Assigning if/then branches for quick replies.

As seen in Step 2, the steps done here are similar. If the action contains a set of quick
replies, they should also be assigned to the following actions correctly. Basically, if the
web visitor clicks “Home Cleaning” from the list, she/he will be directed the Home
 51

Cleaning Action. Figure 15 illustrates how it is visually shown to the Website Visitor.

Figure 23. Show Services.

However, the back end of these actions is implemented as in the figure below. Each
cleaning service contains cleaning description and pricing; hence it is better to have an
individual action-based response for each.
 52

Figure 24. Back-end of “Show services”.

Step 4: Price quotation

As the HubSpot Chatbot does not have a calculator widget, the price estimates should
be calculated individually depending on the property size. These calculations come from
the internal document produced by the Case Company during Data Collection 2. The
calculation varies from the size range as seen on the figure below.

Figure 25. Price Quotation.

Assuming the website visitor selects 115-129m2 as the property size, chatbot then
displays the corresponding calculation as in Figure 17 8. The estimated price is shown
and a tip on how to claim a tax credit benefit for household expenses in Vero. Key
stakeholders requested this special message to be included so that the website visitor is
aware of such benefits which may influence her/him to continue purchasing a cleaning
service.
 53

Figure 26. Price Quotation Sample.

In addition, as in any other action bases responses seen at the earlier steps. There is
also an option to Request a Booking, Leave Contact Request and Call Us forms. Once,
the website Visitor process to the “Leave Contact Request” form, her contact details are
collected, and she will be transformed as a Lead at this point. An online customer service
agent will contact the lead for further discussions.
 54

Figure 27. Chatbot scenarios with possible chat stories (“Leave Contact Request1/2”, on the
right side, and “Leave Contact Request 2/2” on the right side).

The personal information collected in Figure 18 is passed on to the CRM system, a team
member will then enroll in this lead to the relevant workflow templates for marketing
purposes.

Step 5: Request a Booking

When a website Visitor is lead to the Request a Booking action from any action-based
response, she/he is shown a link that directs to the Booking System in the Case
Company’s website
 55

Figure 28. Chatbot scenarios with possible chat stories (“Request a Booking Action”, on the
right side, and “Booking System” on the right side).

HubSpot Chatbot platform does not support a booking system widget and the built-in
book a meeting feature must be used instead. But because of the built-in booking feature
can’t be integrated to the Case company’s existing booking system. The key
stakeholders have decided to include a link towards the booking system rather than
utilizing the one in HubSpot.

Step 6: Support
 56

If the Website Visitor wishes to speak directly to an online customer service agent, a
notification is sent to the live agent that there is a conversation request initiated by the
website visitor. The chatbot will automatically gather the website visitor’s personal
information in case there is no live agent available, and to continue the lead generation.

Figure 29. Support – Connect to the Live Agent.

Step 7: Knowledge Base

To have the knowledge base library feature in HubSpot, the account user is required to
upgrade to the Sales Hub Professional Plan, the pricing depends on the company size
and its users. As mentioned in Table 2 in Section 5.2, the development for Knowledge
Base functionality will be implemented on Version 2 of the chatbot if the key stakeholders
agree to upgrade the current subscription plan. However, HubSpot allows a 14-day trial
to try the Knowledge Base feature, we were able to publish articles within the Knowledge
Base library and test them out. The output results are then proposed to the Key
Stakeholders. The knowledge base is used for Frequently Asked Questions and all the
questions were literally asked by the customers either through chatbot interactions or
from other communication channels.
 57

Before creating a knowledge base library, Hubspot prompts to create at least three
articles. Go to HubSpot Account -> Service and Knowledge Base then click the Create
article button on the right-hand side of the page.

Figure 30. Knowledge Base – Cancellation policy.

This is one of the most asked questions by the customers, hence this is used as an
example for this demonstration purposes. After the answer to the Title or Question is
provided, click Settings and assign a Category for this article, then publish it. Once, at
least 3 articles in the KB are published, continue activating it in the chatbot. Go back to
the chatflows under the Conversations tab and enable the Knowledge Base icon as seen
in Figure 23 below. This activates the KB functionality and applies Artificial Intelligence
to the chatbot.

Figure 31. Activating Knowledge Base.

 58

HubBot, short for HubSpot Chatbot, sends a message on what to write as the main topic
of the website visitor’s concern. For example, if the user types in “policy”, the most
matching and appropriate article is pulled from the KB library and displayed as a
message to the user, with a link to the corresponding published article.

Figure 32. Chatbot scenarios with possible chat stories (“Keyword”, on the right side, and
“HubBot pulls the article” on the right side).

Figure 33. Published article.

 59

As many articles in the Knowledge Base Library can be added as needed, and as much
information saved in the library as the company wishes. If the knowledge base is
extensive, Mari, the chatbot, can be able to “answer” to any type of questions regarding
the service.

5.2.3 Monitoring the Chatbot

At the Monitoring stage, the stakeholders chatbot performance is analyzed for example
what is the hot topic in the services list so we can adjust our marketing strategies
accordingly. Because the leads generated are automatically collected in the CRM
system, they enroll each lead in a defined workflow so they can be sent promotional
newsletters in the future. Then in case they find some inconsistencies in the chatbot, we
will fix it immediately.

Step 1: Conversation activities

Going through all conversation activities to identify what was successful
and what went wrong. The advantage of finding out what is the most
purchased service is essential especially if the inquiries did not successfully
turn into an order. A marketing strategy can be planned and boost this
service in all social media accounts to attract customers. The messages
received can be also seasonal, for example during springtime, customers
are very keen to maintain their garden for the upcoming summer season.
The case company can act ahead and promote yard cleaning on the
website or in the chatbot contents.

Step 2: Lead-Generation
All leads generated from the Chatbot are enrolled to a defined workflow
and this is depending on their intent during the first interaction. For
example, if the lead was interested in home cleaning, she would be then
sent a newsletter that was appropriate to her initial intent. The newsletter
regarding window cleaning or deep sofa cleaning will be sent to her so she
is aware of the service and some promotional discounts to attract her to
purchase additional service.
 60

Step 3: Defects found in Production

If there are bugs or defects found by the time the chatbot was deployed. All
bugs found in Production are prioritized and addressed immediately. The
chatbot needs constant improvement to remain relevant. If there are
suggestions coming from the stakeholders and customers, they will be then
proposed to the stakeholders and identify which features are doable.

Step 4: Analyze Chatbot Performance

HubSpot has a dashboard to analyze metrics in the Action Completion chat
visualization. There is a chart where each of the actions of both are
analyzed, based on their completion rate. For example. How many chat
interactions were started in a particular date range and how many chat
sessions were completed and abandoned. With this data, the case
company will have a visibility on how many sessions were successful and
how many were failed and were not completed.

5.2.4 Summary of the Initial Proposal

The key stakeholders’ inputs were taken into consideration and prioritized for each item
based on their importance. Section 5.3.1 showed the actual development of the AI
Chatbot in the HubSpot Chatbot platform, and this was the initial version that needed
beta testers to perform validation tasks. Every chat transcript was sent to the live agents
as a Follow-up transcript email after each chat conversation.

The stages of the chatbot development are shown in Table 5 below.

Table 5. Stages in the chatbot development: summary.

Stage Content of the stage

1 Planning, step 1 Developing the Chatbot scenarios

2 Planning, step 2 Implementing a Knowledge Base Chatbot in HubSpot

3 Building, Step 1 Assigning a Name and Personalizing a Welcome

Greeting

4 Building, Step 2 -3 Assigning if/then branches for quick-replies

 61

5 Building, Step 4 Price Quotation

6 Monitoring, Step 1 Conversation activity

7 Monitoring Step 2 Lead Generation

This initial version of the chatbot building plan developed at this stage needed to go
through validation which happens in Section 6 below since the bugs or defects found
during the validation will be addressed prior to the deployment of the chatbot, version 1.

Next, section 6 reports on the testing results and validation of the developed chatbot.
 62

6 Validation of the Development

This section reports on the results of the validation stage and points to developments to
the initial chatbot building plan. At the end of this section, the Final Chatbot building plan
and the action plan are presented.

6.1 Overview of the Validation Stage

This section validates the initial chatbot building plan developed in Section 5. First, the
key stakeholders were invited to validate the initial chatbot building plan, the features
and its limitations were explained transparently beforehand. Additionally, other team
members and professional cleaners participated in the validation phase. Second,
because chatbot required to be deployed on the case company’s website for the other
participants to validate it (so, it was deployed). It was decided to activate it within a certain
time when participants can access it at the same time (for the first round of texting for 5
hours; for the second round of testing for another 5 hours; and for the third round, for yet
another 5 hours). Third, after the testing rounds the internal stakeholders´ comments and
feedbacks were collected about testing Mari, the chatbot. The goal of the validation stage
was to ensure that Mari works as expected and all elements developed in it do not have
any uncertainties.

Figure 34. Mari the Chatbot.

 63

The features for the chatbot were suggested by the key stakeholders earlier (in the CSA,
Section 3). In the Planning and Building stages, these items summarized in Table 6
below, were implemented into the chatbot. Table 6 also includes a timeline on when they
were released (or expected to be deployed) in production.

Table 6. Scope and timeline for the chatbot development, testing, and releases.

Features Version Development Testing (in Release

(in Section 5) Section 6) Date

1 Price Quotation 1.0 May 1 – June July 1, 2022 Aug 1, 2022

30, 2022
2 Show services

3 Booking System

4 Support

5 Careers

6 Frequently Asked 2.0 January 1 – Feb 1, 2023 Mar 1, 2023

Questions/Knowledge base January 31,
2023
7 Multilingual (Finnish)

Earlier in the development stage, together with the key stakeholders, the items were
prioritized based on their importance and features that are in Version 1.0 were set as
high priority. The timeframe for this project was defined by various factors, time and
resource as the development and testing stages involve plenty of time and effort. The
release date was about 1 month after the testing, as the key stakeholders and other team
members in the case company rigorously tested the AI Chatbot during this time. All bugs
or inconsistencies found at this stage were addressed urgently until the final release
date.

In addition, having a Knowledge Base functionality in the AI Chatbot required an

acquisition fee in HubSpot which required the account owner to upgrade his/her plan into
a Sales Hub Professional plan where Knowledge Base functionality was supported.
Features in version 2.0 is planned to be built later if the case company considers
proceeding with the upgraded plan. On the other hand, the chatbot elements and
contents can be translated into Finnish without having the upgraded plan.
 64

Thus, the items listed in Table 6 were grouped into two project sprints which meant that
the deliverables and milestones of each version would be reached separately. This
project timeline for each sprint was planned together with the key stakeholders and
revised once again after the testing stage.

6.2 Developments to the Proposal (collecting feedback, Data 3 and implementing

these improvements)

Data Collection 3 concentrated on identifying improvements to the Initial Build in Section

5, what the experts say should be further developed in the Initial chatbot building plan.
The key points in the testing results and feedback are as follows.

Table 7. Summary of the testing results and further development suggestions (findings of
Data 3) for the Initial proposal.

Element 1 of Parts commented Description of the comment/ Development to

the Initial in Validation feedback by experts (in detail) the Initial
Build proposal
identified as
faulty in
Testing

1 Grammatical a) Grammar errors The experts suggested going Update the

errors and in Bot’s responses through each response contents and fix
messages and in the articles together and revising the grammatical
seem to be answers in each article as well. errors in each
lengthy response and in
the articles.
Lengthy
messages will
be shortened.

UI -issue Articles in FAQ is The full content of an article is Assign each

not categorized not displayed. There was a article to one
case when the answers to an category for
article was cut off. example “FAQ”
which lists all
the articles
under it

2 Incorrect Calculations after When the 60-69m2 property Ensure that

price selecting a property size was selected, it showed each price
calculations size led to another the price estimate for 85- calculation per
calculation that is 92m2. property size
intended for outputs correct
another property price estimates,
size update it
accordingly.
 65

3 Elements in Action-based The experts spotted an error in Connect the

action-based response such as one of the Action-based Leave Contact
response do Leave Contact responses for Leave Contact Request form to
not lead Request action Request action. They were not respective
anywhere form. led anywhere else, and the actions, when
chat conversation ended. applicable.

The findings from Data collection 3 including the inputs of the experts/beta testers are
discussed in the sub-sections below.

6.2.1 Developments to Element 1 of the Initial Build

There were quite a lot of grammatical errors in the Bot’s responses, the CEO noticed
that some messages were lengthy and seem to be boring to read. In addition, there are
also grammar mistakes in the articles published in the Knowledge Base library. There
was also redundant information in the articles, refer to the Figure below.

Figure 35. Duplicate article.

These duplicate articles were displayed in the chatbot when the user types the keyword
“same cleaner”. The first one did not display the full answers and got cut off on the
second sentence. On the other hand, the second question got duplicated because this
wasn’t assigned in the correct category in the first place.

All UI related issues and grammatical errors were corrected with the key stakeholders.
They were present or contacted directly while fixing these issues.
 66

6.2.2 Developments to Elements 2 of the Initial Build

This element is set to high-priority and requires immediate fixing because if this is missed
in Production, it may give incorrect information to the website visitors that may lead to
disappointments later. Each price calculation element were gone thoroughly and
ensured that each price estimates action was mapped in the correct property size. The
customer service agent volunteered to record the user story done when going through
the price quotation elements then assign PASS/FAIL for each estimates shown. It will
allows the case company to have a visibility of which element requires fixing for the final
development.

6.2.3 Developments to Elements 3 of the Initial Build

One expert tried to proceed through the price quotations until clicking to the “Leave
Contact Request” but the action did not lead him anywhere but was sent a confirmation
message that his contact information was collected. Figure 36 shows how it was.

Figure 36. Leave Contact Request, navigational issue.

As seen in Figure 36 above, it seems that the chatbot did not collect the necessary
personal information of the user after clicking the “Leave Contact Request”. Ideally, the
Leave Contact Request action should ask for the user’s personal details as in the figure
shown in Figure 21 in Section 5.2.2. Rather, the user is shown a fallback message. This
issue was investigated together to identify the root cause.
 67

6.2.4 Final Developments

After the experts had finished the validation and verification, the chatbot was de-activated
for the time-being until all necessary fixes are applied. There’s an adequate time allotted
for bug fixes before finally deploying the final version of the AI chatbot. The key
stakeholders agreed to have a meeting to fix grammatical errors together and go through
each article and verify that they are assigned to one category only which is “FAQ”.

As the defect in the price quotation feature was evaluated as critical and required extra
attention, the sales representative and the customer service agent agreed to take part in
finding uncertainties under the price quotation actions and to re-validate this feature once
fixed.

For the Leave Contact Request navigation issue, the Managing Director agreed to check
all actions mapped to the Leave Contact Request action and to examine what’s the
underlying issue. Basically, any action-based response connected to the Leave Contact
Request must be directed to the correct action element.

6.3 Final proposal

These stages of the chatbot building were used for building the chatbot:

Stage Content of the stage

1 Planning, step 1 Developing the Chatbot scenarios

2 Planning, step 2 Implementing a Knowledge Base Chatbot in HubSpot

3 Building, Step 1 Assigning a Name and Personalizing a Welcome

Greeting

4 Building, Step 2 -3 Assigning if/then branches for quick-replies

5 Building, Step 4 Price Quotation

6 Bug fixes Developments to Element 1 2 3 which impacted Stage

1-7 in Section 5.2.4

7 Validation round Another round of validation stage to ensure that the

defects are addressed prior to deployment
 68

8 Monitoring, Step 1 Conversation activity

9 Monitoring Step 2 Lead Generation

There was a slight update from the Initial Proposal, Section 5.2.4, as key stakeholders
spotted some bugs in the beta version of the chatbot. Ideally, all necessary bug fixes
need to be addressed prior to the release of chatbot v1.0. Therefore, the key
stakeholders agreed to help with the grammatical and navigational issues and re-test the
features on the second round before the deployment. All the inputs in Data Collection 3
are taken into consideration in the final proposal. The Monitoring Stage will occur after
the chatbot has been deployed to monitor the website visitor’s activity and the lead
generated.

In addition, the development of the features for Version 2 is set to start in January 2023,
which is beyond the timeframe of this thesis, as there are many factors to consider before
these can be implemented.
 69

7 Conclusion

This section contains the summary of this study including the objectives, goals and the
Research Method used. The milestones achieved in this study are derived from the
findings in the CSA and the research done in the Conceptual Framework.

7.1 Executive Summary

The objective of this study was to transform the current simplified version of chatbot into
a smart AI chatbot that can help the case company with lead generation and will take it
closer to having an automated sales funnel. At the beginning of the study, the issue within
the simplified version of the chatbot was obvious for the key stakeholders. They also felt
clearly that a smart AI chatbot can boost the marketing of their business in the digital
world. There was an internal decision made that the case company would need a better
bot assistant for their customer support.

Before proceeding to development, there were factors that needed to be identified first.
This means identifying the issues that they currently face in the chatbot system through
conducting the current state anaysis. The data collection for this study included
interviews and discussions in the case company and a small-scale questionnaire with
the customers. The research method for this study was action research as this study
required iterative actions to plan and implement several cycles based on the evaluation
done at the previous stage.

The current state analysis helped to identify the issues that the case company faced
within the current chatbot system. The identified issues related to a considerable manual
interference, limited availability in terms of the response time, and lack of a knowledge
base for more informative responses, among other issues. Therefore, the focus areas of
this study concentrated on improving the current version of the chatbot in order to
implement a knowledge-based feature that can provide relevant answers 24/7 as it can
understand the context and intent of the website visitors.

To find relevant guidance the study explored available knowledge on the topics of
chatbot building and selected especially HubSpot guidance as best practices to follow
when develop the solution. The most relevant elements of the available lnpesklge and
best practice were picked up and merged into the conceptual framework that helped to
 70

guide the next steps as for how the chatbot features can be implemented by following
the best practices. Since the case company already decided earlier to utilize the HubSpot
Chatbot platform as the case company’s CRM system, the platform for chatbot building
was naturally associated wiht the HubSpot application. It was also a point to get
acquainted with and become aware of other chatbot solutions that serve similar
functionalities.

During the proposal stage, the first version of the chatbot that was developed. The
chatbot scenarios were presented to the key stakeholders to have a visual idea on how
the chatbot scenarios flow. On the other hand, the features were considered that were
gathered from the needs form stakeholders as inputs from Data Collections 1 and 2. It
was then agreed to group these features into two project sprints due to time limitations
and lack of resources in terms of actual development. The features specified in Version
1 were developed at the initial solution building stage, Section 5. Later on, the initial
version was piloted and tested by the key stakeholders in the production environment
and the chatbot was de-activated once they have completed the validation. Their
feedback was gathered immediately after, and the action plan for fixing each defect found
was determined in Section 6. The key stakeholders gathered in a meeting to resolve the
issues together. After that, a round of validation discussions was conducted prior to the
release of the final version of the AI chatbot called Mari. The validation discussions with
the key stakeholders evaluated the testing results and suggested further improvements
to benefit the case company with a better version of the chatbot.

During the testing stage, it was deployed to the case company’s website and is currently
in use there since then. The chatbot that has been launched helps the customer service
agents to respond to multiple inquiries, close deals from the leads generated by the
chatbot, and pass their information to the CRM system to assist the customer service
agents further, when needed.

7.2 Thesis Evaluation: Objective vs. Outcome

The objective of this study was to transform the current simplified version of chatbot into
a smart AI chatbot that can help the case company with lead generation and will take it
closer to having an automated sales funnel. The first version of the chatbot was deployed
according to the project timeline. Since then, the case company has received inquiries
 71

through the chatbot that help the company to collect qualify leads, as was initially
intended when staring this thesis.

The key stakeholders of the company are very satisfied with the results. The customer
service agent added that it has boosted her productivity at work as she can focus on
other tasks. According to the customer service agent, she now spends about 2 hours in
total a day monitoring the chatbot activity. The amount of time spent on monitoring has
reduced a lot compared to when the simplified chat system was in use. She goes through
the chatbot inbox and follow-up to the customers and take actions. For the meantime,
the contents for the Frequently Asked Questions will be added as a sub-page under their
company website. In case there is an inquiry or question that has been frequently asked,
she can manually copy the information from the page added and paste it to the website
visitor.

As a result of the chatbot developed in this thesis, the case company was able to close
deals from the generated leads and can see potential benefits in their sales operations
if the AI chatbot is fully operational. It has reduced their time monitoring the chatbot
activity as it can provide real-time assistance. The feedback from Data Collection 3 had
shown a significant impact on the overall functionality of chatbot version 1. We were able
to spot major issues and diminish negative issues that the website visitor might see.
Once the knowledge base feature is implemented within the chatbot, it will lessen all the
manual tasks by the Customer Sales Agent as the chatbot can then handle complex
issues or inquiries with no help from their side.

The outcome of this study has given a positive impact on the case company sales and
operational level. On the other hand, the thesis researcher herself learned a lot through
this journey despite all the challenges she have faced. There were times when giving up
seemed to be the only option. It was also difficult to collect data from the customers, so
the thesis researcher tried contacting them via phone and email, and many agreed but
some declined due to not having extra time to complete the survey. They were also
worries that their personal details were going to be shared across other platforms,
therefore, the survey form created (Google Forms) purposefully didn’t collect any other
information from the respondents.
 72

7.3 Closing Words

I independently selected the most suitable topic that could potentially help my
professional career growth while boosting sales of our family-run business. Not only
because I have been fascinated with how chatbot works which can tremendously impact
any business operations. I have also grown an interest how they are implemented in
platforms that do not require too much programming.

Moreover, upon attending the Digital Sales and Marketing Automation course offered in
Metropolia, I discovered HubSpot’s capabilities more in-depth and got more
understanding of how it can handle complex functions when implementing a knowledge-
based chatbot. The course basically gave me a go-ahead signal that it is possible for a
non-programmer professional to build an AI knowledge-based chatbot. Although minimal
programming is required for example, when adding simple syntax and code snippets to
the platform, this has encouraged me more to take the challenge and continue learning.

In addition, I have tried to look around our competitor’s website to check if they have
chatbots implemented on their website, but I cannot find one. This means we are at the
right time to implement a well-functioning AI chatbot to the case company’s website for
their competitive advantage.
 1

References

Agarwal (2022). 6 Best Practices For Chatbots. Available at:

https://outgrow.co/blog/best-practices-for-chatbots

Alburger, (2022). Chatbots. Chatbot Best Practices: The Ultimate Guide in 2022.
Accessed from: https://www.hubtype.com/blog/chatbot-best-practices .
(Accessed on May 2022)

Alexander (2002). Resolution Bot explained.Available at:

https://www.intercom.com/help/en/articles/2408936-resolution-bot-
explained

Andersen, J. P. and Blichfeldt (2006). Research Design: Creating a Wider Audience for
Action Research: Learning from Case-Study Research. Journal of
Research Practice. Vol. 2, Iss. 1, Article D2.

Case Company (2021). Case Company website. CleanLiving Oy. Available from:
http://www.cleanliving.fi (Accessed 01 Jan 2021).

Chatbot (2002). Build, test and refine. Available at: https://www.chatbot.com/

Cornell (2022). ProProfs Chat. 10 Chatbot Best Practices for a Business. Blog.
Available at: https://www.proprofschat.com/blog/chatbot-best-practices/
(Accessed 01 Jan 2021).

Costello, P. (2011). Effective Action Research 2nd Edition: Developing Reflective

Thinking and Practice, eBook.

Coughlan, P. and Coghlan, D. (2002) "Action research for operations management",

International Journal of Operations & Production Management, Vol. 22 Iss.
2, pp.220-240, Available at: https://doi.org/10.1108/01443570210417515
(Accessed 23, Jan 2021)
 2

Dilmegani, C. (2017), Top 12 Benefits of Chatbots in 2020: The Ultimate Guide , AI

Multiple. Available from: https://research.aimultiple.com/chatbot-benefits/
(Accessed 05 December 2020)

Haan, M. (2002). 7 Best Chatbots (November 2022). Available at: 7 Best Chatbots (02
November 2022)

Hubspot (2002). Free Chatbot Builder Software. Available at: Free Chatbot Builder
Software (01 November 2022)

HubSpot (2002). HubSpot Knowledge Base. Choose your bot actions. Available from :
https://knowledge.hubspot.com/conversations/a-guide-to-bot-actions
(Accessed 15 September, 2022)

HubSpot (2002). HubSpot Knowledge Base. Create a Live Chat. Available from:
https://knowledge.hubspot.com/chatflows/create-a-live-chat#1-build-create-
a-welcome-message (Accessed 15 September, 2022)

HubSpot (2002). HubSpot Knowledge Base. Use If/then branches in bots. Available
from: https://knowledge.hubspot.com/conversations/use-if-then-branches-
with-chatflows (Accessed 15 September, 2022)

Jaramanan (2020). How to Design a Perfect Use Case For Your Chatbot [Chattbot
Methodology Part 2/3]Available from :
https://blogs.sap.com/2020/05/28/how-to-design-a-perfect-use-case-for-
your-chatbot-chatbot-methodology-part-2-3/

Kananen, J. (2013). Design Research (Applied Action Research) as Thesis Research.

A practical guide for thesis research. Publications of JAMK University of
Applied Sciences, 146.

Kannan, P.V. and Bernoff J. (2019), Does Your Company Really Need a Chatbot?
Harvard Business Law. Available from: https://hbr.org/2019/05/does-your-
company-really-need-a-chatbot (Accessed 05 December 2020)

Khan, R. and Das A. (2018). Build Better Chatbots: A Complete Guide to Getting
Started with Chatbots. Chapter 1, Introduction to Chatbots.
 3

Mack, N., Woodsong, C., McQueeen, K., Guest, G., and Namey, E. (2005). Qualitative
Research Methods: A Data Collector’s Field Guide. Available at:
https://www.fhi360.org/sites/default/files/media/documents/Qualitative%20R
esearch%20Methods%20-
%20A%20Data%20Collector%27s%20Field%20Guide.pdf (Accessed 23,
Jan 2021)

Merisalo, S. (2018). Developing a Chatbot for Customer Service to Metropolia UAS

Student Affairs Office. Master’s Thesis.

Mousumi (2021) .11 Best Practices for Building Chatbots. Available at:
https://www.kommunicate.io/blog/best-chatbot-practices/

Nguyen Tung Son (2019). Potential effects of chatbot technology on customer support:
A case study. Master’s Thesis.

Nilsson, N. J. (1998). Artificial Intelligence, What is AI? Chapter 1, eBook.

Pathania, A. and Guzman, I. (2016), Accenture Interactive, Chatbots in Customer

Service, pp 3-6. Available from:
https://www.accenture.com/t00010101T000000__w__/br-
pt/_acnmedia/PDF-45/Accenture-Chatbots-Customer-Service.pdf
(Accessed 05 December 2020)

Ramasubramanian, S. and Shivam (2019). Building an Enterprise Chatbot: Work with

Protected Enterprise Data Using Open Source Frameworks. Chapter 3,
Chatbot Development Essentials. (Accessed on 12, February 2021)

Roldos, I. (2020). “NLP, AI, and Machine Learning: What’s The Difference?” Blog.
Available at: https://monkeylearn.com/blog/nlp-ai/ (Accessed 01, Feb 2021)

Salesforce (2002). Einstein for Service: AI-Powered Customer Service. Available at:
https://www.salesforce.com/content/dam/web/en_us/www/documents/datas
heets/einstein-for-service-data-sheet.pdf

Shewan (2022). 10 of the Most Innovative Chatbots on the Web. Available at:
https://www.wordstream.com/blog/ws/chatbots
 4

Vishnoi, L. (2020), Acquire, 10 Chatbot Benefits That Can Improve Your Business.
Available at: https://acquire.io/blog/chatbot-benefits-for-business/
(Accessed 05 December 2020)

Vishnoi, L. (2021). “Chatbot vs. Live Chat: Which is Winning the Customer Service
Game and Why?” Available at: https://acquire.io/blog/chatbot-vs-live-chat/ .(
Accessed 01, Feb 2021)

Wordpress (2002) . WP-Chatbot for Messenger Plugin. Available at: WP-Chatbot for
Messenger.

Zaboj (2022). How to build your first chatbot. Available at:

https://www.chatbot.com/help/build-your-chatbot/how-to-build-your-
chatbot/#how-to-add-a-chatbot-to-your-website
 5

Appendix 1: CSA Results – External Customers

 6

Appendix 2: CSA Results – Internal Stakeholders