See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/360456896

RECOMMENDATION SYSTEMS, MASTER SEMINAR REPORT DEPARTMENT OF

COMPUTER SCIENCE, INSTITUTE OF INFORMATICS, GAZI UNIVERSITY.

Technical Report · March 2020

CITATIONS READS

0 23

2 authors:

Abdur Rehman Anwar Qureshi M. Ali Akcayol

Gazi University Gazi University
4 PUBLICATIONS   0 CITATIONS    114 PUBLICATIONS   1,000 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Graded Field Classification Based Web Search Engine with Semantic Results Matching and Pre-ranking Algorithm View project

A comprehensive analysis of e-learning studies in Turkey, problems and suggestions View project

All content following this page was uploaded by Abdur Rehman Anwar Qureshi on 08 May 2022.

The user has requested enhancement of the downloaded file.

 1

RECOMMENDATION SYSTEMS (ÖNERİ SİSTEMLERİ)

Abdur Rehman Anwar Qureshi

198005101

MASTER SEMINAR REPORT

DEPARTMENT OF COMPUTER SCIENCE

GAZI UNIVERSITY
INSTITUTE OF INFORMATICS

MARCH 2020
ANKARA
 2

(MSc. Seminar Report)

Abdur Rehman Anwar Qureshi

198005101

GAZI UNIVERSITY
DEPARTMENT OF COMPUTER SCIENCE,
INSTITUTE OF INFORMATICS

Abstract

The Recommendation System (RS) is used to show relevant product items to the customers
according to the customer interest area. The RS is widely used in web search engines, mobile
devices. E-commerce websites, desktop applications, social media platforms, etc. The RS
significantly increases the business of e-commerce giants, prevents a user from incorrect inputs in
search engines, helps the user to find relevant content in social media platforms, assists the user in
acquiring desired outcomes in the mobile devices. We have investigated RS and their applications
form various aspects i.e. time, location, seasonal, context, social media, others, and intended to
propose the deep learning based query auto-completion approach for the RS, which enhances the
quality and efficiency of existing RS systems.

Keywords: Recommendation systems, recommendation engines, query auto-completion, string

completion, and e-commerce.
 3

Acknowledgments

First of all, I am grateful to Almighty Allah for his blessings upon me, Who gave me the ability
and courage to complete my thesis seminar report on time. He is the only one who I always looked
at in the event of happiness and trouble and He always helped me in the time of need. With His
blessing upon me, I have completed my work.

I would specially thank my MSc thesis supervisor Prof. Dr. M.Ali Akcayol who remains the source
of guidance during this all period. He gives me continuous advice on the content of this report and
thesis objectives. He gives me a lot of time to guide in each and, every step of this seminar report.

I would like to thank my parents who supported me in all my endeavors and saw a successful
person in me. Last but not least I would like to thank all my friends whose silent support led me to
complete this task and enjoy the MSc studies at the university.

Abdur Rehman Anwar Qureshi

198005101
Ankara Turkey
 4

Contents
Abstract .......................................................................................................................................... 2
Acknowledgments ......................................................................................................................... 3
1. Introduction ........................................................................................................................... 6
2. Recommendation Systems .................................................................................................. 10
3. Approaches to Recommendation Systems ......................................................................... 13
3.1. Content-Based Recommendation Systems ................................................................. 13
3.2. Collaborative Recommendation Systems ................................................................... 15
3.3. Hybrid Recommendations Systems ............................................................................ 20
3.4. Knowledge-based Recommendation Systems ............................................................ 22
4. Challenges of Recommendation Systems .......................................................................... 24
4.1. Cold Start Problem ...................................................................................................... 24
4.2. Ethical Issues or Privacy Concerns ............................................................................ 25
4.3. Fraud Challenge ........................................................................................................... 26
4.4. Data Sparsity Problem ................................................................................................. 26
4.5. Scalability ...................................................................................................................... 28
4.6. Diversity ........................................................................................................................ 28
5. Evaluation of Recommendation Systems .......................................................................... 29
6. Recommendation Systems Applications ............................................................................ 31
6.1. Query Recommendations ............................................................................................ 32
6.2. Types of Recommendation Systems Applications ..................................................... 35
6.2.1. E-Government ....................................................................................................... 35
6.2.2. E-Commerce or E-Shopping or M-Commerce .................................................. 35
6.2.3. E-Library ............................................................................................................... 35
6.2.4. E-Learning ............................................................................................................. 36
6.2.5. E-Tourism .............................................................................................................. 36
6.2.6. E-Resource Services .............................................................................................. 36
7. Conclusions........................................................................................................................... 37
8. References............................................................................................................................. 38
 5

“Knowledge is knowing that a tomato is a fruit. Wisdom is knowing not

to put it in a fruit salad.”–Brian O’Driscoll
 6

1. Introduction
The trend of shopping is entirely changed due to the revolution of information and communications
technology. People prefer e-commerce shopping rather than traditional physical shopping.
Recently novel virus diseases named COVID-191 outbroke and affected the world economy
drastically. However, the usage of e-commerce is exponentially increased and became an integral
part of daily life. Global e-commerce sales reached $2.928 trillion in 2018 and raised by 20.7 %
to $3.535 trillion in 2019 and economists also expected that e-commerce sales would approach $5
trillion by 2021 [30].
The volume of data and information is exponentially increasing each second on the various kind
of platforms over the internet. In this scenario, it is very important to find relevant information
from such a large amount of data. The Recommendation Systems (RS) plays a vital role to find
relevant information according to the customers' preferences. The objective of RS is to
personalized online product or service recommendations. It has been observed that RS is the
integral sale strategy of e-commerce based companies. Due to the wide usage and commercial
importance of RS, the research community has been working on various aspects of RS, however,
it is still a challenging area of research in the domain of information retrieval.
The usage of RS is tremendously increased in e-commerce websites, search engines, social media
applications, mobile devices, mobile applications, etc. The big tech giants do huge investments in
this application of information processing areas to compete with others. Due to the commercial
importance of RS, the research community plays a vital role in recent years. It has been observed
that like other areas, deep neural networks are also widely used in RS for the good results and high
accuracy rate of prediction. We have performed literature studies as follows:
Jian Wei et al. [1] proposed the RS approach based on collaborative filtering and deep learning
techniques, and especially deal cold start problem of RS systems. The proposed approach enhances
the RS systems efficiency and quality of outcomes especially in the scenario of the cold start
problems by using collaborative filtering and machine learning algorithms. They extracted
content-based features from the logs and used deep learning neural networks to formulate a
recommendation list.
Abhay E. Patil et al. [2] proposed the RS approach based on collaborative filtering and association
rule mining. In this proposed approach, the collaborative filtering predicts customer choices while
the association rule mining is used to know the frequent pattern in the logs. Due to the help of
collaborative filtering and association rule mining techniques, the proposed RS gives good quality
recommendations over data sparsity and cold start problems. This RS especially developed for a
recommendation of books to the online customer.

1
World Health Organization (WHO) briefed COVID-19 https://www.who.int/emergencies/diseases/novel-
coronavirus-2019/events-as-they-happen (accessed on 16 March 2020)
 7

Shaowei Wang et al. [3] proposed an automatic tag RS named ENTAGREC++ for software
information sites. The proposed approach acquires wisdom from tag logs and predicts relevant
tags for new entries. The ENTAGREC++ system based on inference components i.e Bayesian,
frequentist, user information, and tags components. They stated that this approach produces good
outcomes over four different software information sites i.e. Ask Different, Ask Ubuntu, Super
User, and, Stock Overflow.
Baolin Yi et al. [4] proposed a novel RS approach named Deep Matrix Factorization (DMF) based
on deep learning and collaborative filtering. The DMF approach combines user and item
information, and implicit feedback embedding is used to transform the high dimensional
information into the vector-based dominant features. This approach predicts user interest with the
help of latent factors.
Zhenhua Huang et al. [5] proposed the systematic RS approach to grab the passengers in the area
of smart transportation by using taxi GPS logs and deep neural networks. The proposed approach
consists of the deep learning model, which has advantages over memorization and enhances the
efficiency of grabbing online taxi passengers. This RS learns from the taxi drivers and predicts to
graph passengers based on road conditions and maximum profit for the taxi drivers.
SRS Reddy et al. [6] proposed the content-based movie RS by using genre correlation. The
proposed approach predicts movies on the base of the type of genre, which customers like to watch.
The content-based filtering technique is used to calculate genre correlation for the outcomes. The
proposed system finds a specific genre against each customer and then suggest a list of
recommended movie.
Wang Zhou et al. [7] proposed the RS approach based on deep learning by exploring customer
interests. This approach explores the interests of each customer and acquires aspects of text
information with the help of the convolutional neural network and apply convolutional matrix
factorization to produce the candidate list. The proposed approach increases the efficiency of the
system and generates a top-N recommendation list according to customer relevance even in the
data-sparse and cold start problem. But this approach does not deal with customer interest changes
over time.
L. Zheng et al. [8] used the reviews information given by customers to deal with data sparsity
problem and enhance the efficiency of RS. The proposed Deep Cooperative Neural Networks
model based on two parallel neural network layers. The first network determines the behavior and
preferences of a customer by exploring the reviews while another network is used to find properties
and characteristics of relevant items. The shared layer is used to communicate between these two
networks to allow learned latent factors. Experimental outcomes showed a given approach based
on neural networks tremendously improved the quality of RS over the various dataset.
T. Bansal et al. [9] used the latent factor model of collaborative filtering to give relevant
recommendations over the new and unseen text. The proposed approach based on deep recurrent
neural networks for the transformation of text sequence into a latent vector with collaborative
filtering. The results generated by simple linear models are not satisfactory due to which multi-
task learning has been preferred. The transformation of text to the vector enable to use of gradient
 8

descent technique and give space to apply multi-task learning. The multi-task learning technique
enhances the quality of outcomes remarkably.
R. Catherine et al. [10] proposed neural nets based RS approach, which combines latent
representations of reviews of customers and target items. The approach performs well even in case
of new customer or item. The proposed approach named Transformational Neural Networks,
which is the advanced form of neural network RS by adding an extra latent layer for the estimation
of new customers and target items. Experimental results showed this approach significantly
enhance the efficiency of RS over new customers.
S. Cao et al. [11] proposed the deep learning and collaborative filtering based approach for online
news RS by using the stacked auto-encoder technique. The stacked auto-encoder technique
acquires the constructive features in low dimension form from actual infrequent user-item
matrices. The experimental results showed that this approach improved the RS efficiency and
quality of recommended items.
Jianming He et al. [16] proposed a novel approach to find social information of potential users,
which can be used in RS. They extracted data from an online social network named Yelp.com and
formed as a dataset. They proposed that the website should have a very interactive user interface
with various social network features i.e. LinkedIn, Google, Facebook, and Twitter to grab potential
user attention. The website shows a sign-up dialog box or a sign-in with any existing social
network account. From both ways, personal information of potential users has been extracted.
A. Rzheuskyi et al. [22] worked on Ukraine-based virtual libraries archives. The RS named virtual
reference proposed and implemented to facilitate the worker. The proposed RS composed into
three modules i.e. start crawling, statistic module, and bibliography module. Experimental results
showed that this approach helps the users to give satisfactory results in a very short time interval.
Nabil S et al. [23] investigated various RS approaches and proposed combined proposed based on
the demographic filtering, collaborative filtering, content filtering, user profile, sentiment analysis
of social information for the efficient system. The objective of this approach is to prevent from
cold start problem, generate the most relevant items for the customer, minimize the search
duration, and improve the efficiency of the system.
Yongfeng Zhang et al. [24] worked on the conversational search and recommendation where the
customer and system can interact multiple times with each other. They highlighted the importance
of conversational search and recommendation and proposed the System Ask-User Respond
recommendation framework for the e-commerce products by using Multi-Memory Network
(MMN) architecture, which is trained by reviews of customers. The proposed system first ask
relevant questions from the customers to understand the needs of customers and then generate
outcome accordingly. Experimental results showed that conversational search and
recommendation gives better performance than traditional recommendation techniques.
Hwangbo et al. [25] proposed and implemented collaborative filtering based RS for the Korean
fashion industry. They deal with both online and offline customers and respective products. The
proposed approach named K-RecSyc uses an item-based collaborative algorithm and unifies click
 9

data from online products and sale data from offline products in the direction of knowing
preferences of customers. Experimental results showed that the proposed approach is more
efficient than traditional recommendation techniques in the case of seasonal and fashion
recommendations by using customer click data.
Youdong Yun et al. [26] stated that the collaborative filtering technique is widely used in various
applications of recommendation systems but the result is not remarkable due to insignificant
performance. As a consequence, they proposed a novel collaborative filtering based RS by using
review data of products and evaluated the proposed model on Amazon review data.
T. Rutkowski et al. [19] proposed the content-based RS by using a neuro-fuzzy approach for the
relevant recommendations. This approach pre-processed past content into a variable for the
construction of the neuro-fuzzy system. The MovieLens 20M2 dataset has been in this approach.
The experimental results showed that the neuro-fuzzy approach performs better than the deep-
learning approach. It has been stated that this is the first successful implementation of the neuro-
fuzzy content-based RS in the literature studies.

2
The Movie Lens 20M dataset is available here. https://grouplens.org/datasets/movielens/20m/
 10

2. Recommendation Systems
The significantly increase the usage of technology medium for various business and services
purposes, play a vital role in the development of the RS domain. The role of the user feedbacks in
the web and mobile applications acts as a catalyst in the area of RS. The essential job of RS is to
used various kinds of information to conclude user interests and preferences for the particular
product, item, or service as in Fig 13. The recommendation has been generated for the user and
each recommendation operation has an item or the list of items. Consequently, the RS uses the
user previous session, interaction history, and contextual information for the identification of user
interest and preferences to predict user future choices.

Fig 1. The user-based and item-based filtering of RS according to customer preferences.

The online user activities carry the various kinds of intra-similarities between user and items and
the basic concept of recommendation is to identify such activities and predict the user's future
possible choices as outcomes. The user-rating metric is used to trace and identify those intra-
similarities in a data-driven manner and then the proposed model uses those intra-similarity
features to predict relevant items for each user. Literature studies of RS showed that if there is
enough user information then there would be more appropriate recommendations.
The usage of RS systems is tremendously increasing in recent years because of the advancement
of artificial intelligence techniques in real life. The RS based applications play a vital role to assist
the naïve user with the help of artificial intelligence algorithms. Recommendation feature-based
e-commerce websites and mobile applications predict relevant product items for the customer with
the help of customer likeness or dislike ness, comments, and responses on other product items.
Some systems also use social media information, demographic information, and other appropriate
information for the prediction of product items. The RS plays a vital role to find relevant
information according to the customers' preferences. Recommendation feature-based search

3
RS filtering https://medium.com/@cfpinela/recommender-systems-user-based-and-item-based-collaborative-
filtering-5d5f375a127f (accessed on 16 March 2020)
 11

website prevents a user from incorrect inputs and formulates user intended queries.
Recommendation feature also helps to find a relevant prediction list over the location, time,
demography, etc. As we know mobile devices have the lowest keypad, which increases the
chances of incorrect input or mistyping, however, recommendation features in mobile devices and
mobile applications accommodate the mobile customer to step up the human-computer interaction.
The RS problem can be defined in various ways, but two of them are described as follows:
• Prediction problem: The first problem is to predict the rating value by using the user-item
combination. It is supposed dataset shows the user has preferences for the item and a is
user and b is items then the incomplete metric would be a×b. In the scenario, the existing
values are used in the training while the missing value would be predicted by using the
trained model.

• Ranking problem: The trained model has been predicted the missing values but then the
question comes how to display them to the particular user. The top-k items have been
chosen based on the user preferences for the display purpose. This problem is also known
as the k-top recommendation problem or ranking formulation problem.
The main goal of RS is to increase products or services sales and used by various kinds of digital
traders. That is why; RS is the part of each company's marketing strategy. The business-centric
based goals of RS are as follows:
1. Relevance: The most important goal is to generate the user relevant recommendation list
based on user interests.

2. Novelty: The generated recommendation list must be interesting and attractive to the
particular user.

3. Serendipity: The user gets the unexpected recommended items and consequently the user
would happy because of system intelligence. The outcome should be positive for achieving
this goal of RS.

4. Diversity: The system should be enough able to include the behavioral factor of each in the
recommendation list. This goal shows the diversification of RS.

5. User usability: The user should not be offended by the recommendation list. The system
must provide easy access to the recommendation items.
The aforementioned goals are essential however, there are many other goals of RS as well.
Websites with RS features improve the satisfaction and retention of their customers. Take the
example of Amazon based RS that recommends the relevant recommendations to the customer
and the customer is satisfied with the website. There are more chances that the same customer will
come again and would buy more products. In this way, Amazon's revenue is tremendously
increased.
 12

The exploration of a few big tech giants based RS are as follows:

• GroupLens Recommendation System: GroupLens team was the first one who introduced
the recommendation system and contributed to the literation studies. This RS trained by
using the ratings of Usenet4 News based readers and then predicted new articles whether
customers would have liked predicted articles or not. The predicted articles were not read
by customers before. This group also prepared datasets5 for RS which are widely used in
RS research.

• Amazon Recommendation System: Amazon has a robust e-commerce RS and was the first
e-commerce company that introduced RS. Amazon started usage of recommendation from
e-book selling and now Amazon uses RS in her all products. The Amazon-based RS uses
both implicit and explicit features of customers to predict the recommendation.

• Facebook Friend Recommendations: Facebook is social networking sites and Facebook

recommendations are different than others. Facebook recommends possible friendship
recommendations to its users based on various factors such as mutual friends, same
profession, demographics attributes, location, etc. The objective of friend
recommendations over Facebook is to increase the social networks of each customer that
significantly increase the growth of Facebook.

• Netflix Recommendation System: Netflix was the small mail-order digital movie company
and from the last 10 years, it is one of the biggest online movie streaming companies.
Netflix provides online movies and television shows streaming throughout the world and
allows each customer to give a rating of each episode based on the 5 point scale. Netflix
also saves customer session history, contextual information, etc. These aspects of
information are used in the Netflix movie recommendation. Netflix organization also
organized various Netflix Prize Contest6 which allowed RS-based research community to
represent their work.

• Google News Personalization System: Google recommends the news to its customer based
on the user clicks history. The click model has been utilized for the generation of the
recommendations. The clicks of each user have been collected with the help of user
accounts associated with google. Google treats the news articles as an outcome while the
ratings of each article have been decided based on the number of clicks. This is our intuition
about the google recommendation algorithm, although various other techniques have been
utilized for this accomplishment of this purpose.

4
The details of Usenet News can be found here. https://usenet-news.net/ (accessed on 21 May 2020)
5
The GroupLens based datasets can be found here. https://grouplens.org/datasets/ (accessed on 21 May 2020)
6
The details of Netflix Price Contest can be found here. https://www.netflixprize.com/ (accessed on 21 May 2020)
 13

3. Approaches to Recommendation Systems

The RS depends on the various type of information filtering. Information filtering deals with
information that is liked by users to find something interesting or useful for predictions and
recommendations. An information filtering judges users based on user-profiles by filtering and
then, as a result, give relevant information to the user. The approaches to RS are classified into
content-based, collaborative filtering, hybrid, and knowledge-based as described in Fig 2.

Fig 2. Classification of Recommendation Systems approaches

The detailed explanation of each approach of RS are as follows:
3.1. Content-Based Recommendation Systems
It is one of the most famous techniques of RS. The content refers to the items, products, services
that are liked or responded by the customer. Then the recommendation list has been generated
according to customer existing content. Take the example of a YouTube user who likes to learn
cooking skills. This user always searches for cooking related stuff then YouTube automatically
shows a series of cooking-related videos [17-18].
The content-based RS foundation lies in the customer preferences in the past and similar to past
preferences, the new product has been recommended. The process of content-based RS is shown
in Fig 2. The customer experience represents her preferences in the shape of Product A and
Product B. The two products can extract product features from the (dairy products) and (dry fruits
products) respectively. The features of product C (Eating Products) is similar to product A and
product B, as a consequence, the product C can be recommended to the customer. It can be seen
that the interest model in the content-based RS plays a vital role.
The various kinds of similarity measures have been used to compute the similarity of two items.
The Euclidean and cosine similarity are the most famous ones are given below in equations 1 and
2 respectively.
 14

Fig 3. The content-based RS according to customer preferences.

The Euclidean dissimilarity is as follows:
n
dissimilarity ( a, b) =  (a
i =1
i − bi ) 2 (1)

Where the a and b are two items. The outcome of the Euclidean dissimilarity used to determine
the ranking of recommendation items.
The cosine similarity is as follows:
n

x i  yi
similarity ( a , b ) = i =1
(2)
n n

x
i =1
2
i y i =1
2
i

Where the a and b are two items. The cosine similarity finds the closeness between two items.
Content-based Filtering requires a large amount of information on items, features, aspects rather
than user interaction and feedback. It is also known as cognitive filtering. It predicts items based
on the comparison between items and user profiles. Usually, content-based filtering uses text
documents as an information source. There are various techniques for the learning of user-profiles
such as relevance feedback, neural network, Bayesian classifier, and genetic algorithm.
Achakulvisut T et al. [20] proposed content-based RS for scientific publication material. This
approach works for the articles and scholarly materials recommendations. It has been stated that
the recommendation of scholarly materials is different than the recommendation of movies, music,
and other recommendations. The NeuroScience Conference dataset has been used in this work.
The proposed approach used the Latent Semantic Analysis (LSA) to get the topics where the
 15

variance value is high, and the Rachio Algorithm used to approximate nearest neighbor search and
formulate content-based recommendations. The proposed algorithm is summarized as follows:
i. Dataset has n conference papers P={P1, P2, … , Pn}.
ii. Data preprocessing has been completed to exclude English stop words and performed
stemming.
iii. Transform papers to matrix M by using the term frequency, TF-IDF, or log entropy.
Exclude rare words and high TF-IDF value words.
iv. Apply Latent Semantic Analysis to get high variance topics.
v. Apply the nearest-neighbor model
vi. Compute user preferences vector
vii. Generate the suggestion of relevant papers
SRS Reddy et al. [6] proposed the movie RS by using content filtering. The content filtering does
the behavioral analysis of customers and by using such experiences new item has been
recommended to that customer. The objective of this RS is to generate movie recommendations
with the help of similarity of genres. The MovieLens 20M dataset and R tools for analysis have
been in this approach. The proposed approach is summarized as follows:
i. Load the genres dataset and build a data frame where row shows the id of movie and
column shows the genre value of that movie.
ii. Generate the list of all available genres in the dataset
iii. Check the genre value in the data frame and if the value is present then assign 1 to the genre
matrix.
iv. Load the rating sheet to transform in the rating matrix. If the rating is greater than 3 then
write 1 in the rating matrix, or if the rating is equal to 3 or less than 3 then write -1 in the
rating matrix.
v. Compute the dot product of the genre matrix and the rating matrix to get the result matrix.
vi. Transform the result matrix into the binary format. If the value is in minus then write 0,
otherwise 1 in all cases.
vii. Compute the Euclidean distance among the present users and other users.
viii. Display the minimal distance rows that are recommendations for the present user.

3.2. Collaborative Recommendation Systems

It is another famous technique of RS. It is the most successful recommendation system technique
and widely used by Facebook, Amazon, eBay, et. The collaborative based RS investigates the
items, products, services, etc with the help of feedback from other customers. Such systems filter
the other customer profiles for the prediction of the current customer. The collaborative based RS
gather the customer profile and then find the relationship between former customer profiles and
current customer profile with the help of similarity model. The essential steps of each collaborative
filtering recommendation are described in a processes diagram in Fig 4.
 16

Fig 4. The processes diagram of collaborative Recommendation System

 17

The collaborative RS can be divided into i. model-based and ii. memory-based.

1. Model-based RS: Model-based collaborative RS describes the user-item interactions and
the model is trained by the user-item interaction metric. Both supervised and unsupervised
machine learning techniques are used in the creation of models and training and prediction
phases must be separated. Model-bases RS includes models such as regression model,
support vector machine, neural networks, Bayes classifiers, and others.

2. Memory-based RS: Memory-based collaborative RS based on the similarities between user

and item extracted during the user-item interaction. There is no model of user-item
interaction however, it based on the facts related to similar users. There two types of
memory-based RS i.e. i. user-based collaborative filtering and ii. item-based collaborative
filtering where the user-based computes the ratings by using neighborhood iems while
item-based computes rating by using similar user ratings. Memory-based RS is also known
as the neighborhood-based RS.
The advantage of collaborative-based RS over content-based RS is a collaborative technique also
performs well for the new customer. Consequently, the collaborative technique is widely used in
cold start problems.
Collaborative Filtering (CF) based on historical data and required historical preferences of users
on the set of items. It is a technique used by various prediction systems including RS. CF has a
narrow sense and a general sense. CF based applications use large data sets due to which CF
methods are applied to various kinds of data such as e-commerce data, bank transaction data, etc.
CF is the personalized recommendation technology. It uses the nearest neighbor technique for the
calculation of distance among a user and user preference. CF helps to do automatically content
analysis of information and generating new information according to user behavior. CF especially
emphasized on the potential interest of the users. Zheng et al. [12] stated that CF-based systems
use network information service, which is based on the evaluation of recommended information
to the users. CF only gives accurate results when the system has accurate information about the
user.

CF based RS systems are capable to determine which group of users are common to the given user.
The CF method is a choice of k-neighbors to the user, where wish to generate predictions. This
process is achieved by a similarity measure, which required only the most relevant information.
Jesus et al. [13] proposed a model that improves results by using contextual information and
categorizing rating values. The main idea of this paper is that the contribution of an item to the
similarity assigned to two users should not be as absolute, but rather as relative to the vote awarded
to this item by all users in the system. The CF has been used for the acquiring of desired results.
Zheng et al. [12] stated that the collaborative filtering based system not only analyzes the similarity
between items but also learn from the history of the targeted users and similarity between users'
social behaviors due to which, these systems can predict very relevant items. They stated that CF
systems based on certain information evaluation of other users to be predicted for the current user.
The system specifies users, which have similar interests with the current user and then predicts
items for the current user.
 18

CF is widely used for making recommendations list online and offline as well. The data sparsity
and scalability are two major are issues in traditional CF techniques. Zhenzhen Fan et al. [14]
presented a hybrid user-item collaborative filtering method to address these issues and would
produce more personalized query suggestions with better rating quality. Luo et al. [15] stated that
the choices of users and items are combined to generate a user-item grading matrix in the CF-
based system, where huge grading denotes strong choices. CF systems are developed on the
neighborhood-based model and the latent factor model.

CF uses the user-item rating matrix and relevant matching has been obtained by the similarity
function as follows:

sim(a, b) =
 iI ab
(rai − ra )  ( rbi − rb )

 iI ab
(rai − ra ) 2   iI ab
(rbi − rb ) 2
(3)

The outcome of the similarity function is used for making recommendations or predictions of
relevant items. After that, computation among users or items can be measured by several similarity
functions such as Pearson Correlation Coefficients (PCC) as follows:

pti =r+
 uU nei
sim(t , u )  ( rui − ru )
 uU nei
| sim(t , u ) | (4)

When the similarity is calculated, neighboring users are selected as a similar group and forecasting
rating of new items for each similar user group.

Youdong Yun et al. [26] proposed the recommendation system and developed by using the
customer reviews dataset over the e-commerce sites and assumed that such reviews are represented
user preferences. The hybrid strategy of user evaluation score and opinion mining based text-
scores used for the performing of recommendations. This hybrid strategy fetches statistics and
produces quantified outcomes based on user evaluation score and user review text. The proposed
recommendation system method has six models. The first model is the foundation of the proposed
method and depends only on existing data without applying the opinion mining technique. The
resting five models used the opinion mining technique and the general formula has been derived
for the performing of recommendations as follows.

Applied opinion mining data with additional = Score + Text Score (5)

After that, different mathematical manipulations have been applied to drive the mathematical
equation of each model.

Zarzour H et al. [27] stated that traditional collaborative filtering based recommendation systems
are not efficient due to the huge volume of data as data is increasing exponentially over a single
 19

second. Consequently, two collaborative filtering based recommendation systems algorithms are
proposed. These two algorithms used improved k-mean clustering techniques to increase the
performance and efficiency of recommendation systems. The objective of the k-mean algorithm is
to keep high the inter-point similarity of the same cluster as much as possible while low the intra-
point similarity of clusters. Further, a detailed explanation of each algorithm is given below
explicitly.
The first collaborative recommended system is solely based on k-mean clustering. However, the
general k-mean algorithm has been transformed into personalization for dealing with the unknown
rating and active user of the recommendation system. This algorithm 1 is summarized as follows:

(i) Input matrix of user-item rating data and active user

(ii) Calculate the similarity form Pearson Correlation Coefficient technique and generate
a similarity matrix as an outcome
(iii) Pick randomly initial clustering point for the active user as a matrix from the dataset
(iv) Compute the distance between centers and objects to assign the objects to assign
objects to the nearest cluster
(v) Compute the average of each cluster as a new partition centers
(vi) Utilize the new partition centers to readjust points into new clusters
(vii) Repeat the last two processes until the algorithm converge into a stable position
(viii) Compute similarity between active users and clusters
(ix) Pick the first n similar clusters of the active user
(x) Compute the values of the prediction of an active user to each cluster
(xi) Pick the N-top items of similar users as recommendations
(xii) Display the recommendations
The Principal Component Analysis (PCA) has been used in the second algorithm. The PCS did the
reduction of dimension to improve the recommendation system accuracy for a large volume of
data. After that, the k-mean clustering technique has been applied. The combination of a PCA
method and the k-mean clustering technique made the collaborative recommendation system more
efficient. This algorithm 2 is summarized as follows:
(i) Input matrix of user-item rating data and active user
(ii) Calculate the similarity form Pearson Correlation Coefficient technique and generate a
similarity matrix as an outcome
(iii) Normalize the data
(iv) Compute convenience of the corresponding matrix
(v) Compute the eigenvectors of the corresponding matrix
(vi) Translate the data into terms of principal components by using matrix multiplication
(vii) Use obtained principal components matrix as dataset and pick randomly initial
clustering centers which are the initial value of active user
(viii) Compute the distances between center and objects to assign objects to the nearest
cluster
(ix) Compute the average of each cluster as a new partition centers
(x) Utilize the new partition centers to readjust points into new clusters
 20

(xi) Repeat last processes until the algorithm converge into a stable position
(xii) Compute similarity between active users and clusters
(xiii) Compute similarity between active users and clusters
(xiv) Pick the first n similar clusters of the active user
(xv) Compute the values of the prediction of an active user to each cluster
(xvi) Pick the N-top items of similar users as recommendations
(xvii) Display the recommendations
The collaborative filtering technique used in the collaborative RS. It is the most famous and
effective approach towards the RS. It solves the cold start problem of RS that is one of the biggest
challenges in this domain. Although the collaborative technique is most effective, it also suffers
various challenges like customer privacy issues, fraud challenge, and others. It seems that there is
always a tradeoff of privacy concerns of customers data while using the collaborative technique in
the RS.
3.3. Hybrid Recommendations Systems
The hybrid RS based on the hybrid filtering that extracted features from two or more than two
different RS techniques such as content-based, collaborative, demographic, etc. Individual RS
generates inaccurate recommendations in the case of content and rating sparsity. The objective of
the hybrid approach is to overcome the limitation of each technique. The literature studies showed
that hybrid RS leverage the benefits of each technique to design and develop a robust system. The
generic framework of hybrid RS is shown in Fig 5.

Fig 5. Generic Framework of Hybrid Recommendation System

 21

In Fig 4, it has been seen that the hybrid RS has multiple RS techniques for the generating of
recommendation. The variable “N” in the last technique of hybrid RS shows the most number of
techniques used in the particular hybrid approach and the range of N starts from the zero to the
real positive number. Mathematically it can be represented as 0≤N≤R+. The generic hybrid
approach communicates with the learning profile process in two ways and then display the
recommendations.
Tarus, J. K et al. [35] proposed the hybrid RS for the recommendation of the online learning
materials. It has been stated that traditional RS techniques are not reliable in case of data sparsity
problem. The proposed hybrid approach based on the content-awareness, collaborative
algorithms, and the sequential pattern mining algorithm. The content-awareness and sequential
pattern mining are used to develop learning profile and collaborative algorithms use that learning
profile for the prediction of the recommendations.
Logesh, R et al. [36] proposed the hybrid RS approach in the direction of e-tourism. The goal of
this approach is to develop the hybrid RS based on customer preferences and ratings by using
collaborative filtering. The proposed hybrid approach based on the three phases as follows:
i. In the first phase, the Expectation-Maximization (EM) algorithm makes the cluster of use
ratings and each cluster used to find the similarity between users with the help of the
Pearson Coefficient Correlation (PCC).

ii. In the second phase, the opinion mining technique used to compute the major factors of
tourism from the database to construct the user preferences profile and POI influence.
After that, the proposed approach predicts the tourist places by using customer profile,
customer-based similarity, POI-based similarity, and POI influence.

iii. In the third phase, the customer personalization such as time, current location, mobile
cache history, and other profile characteristics are considered as customer profile and
used in the generation and ranking of the top-N recommendations.
Kim, D et al. [37] proposed the deep hybrid RS approach based on the document context
awareness, convolutional neural network (CNN), and probabilistic matrix factorization (PMF).
The proposed hybrid approach has been described in three phases.
i. In the first phase, the CNN analyzes the characteristics of each document and grab the
contextual information to formulate document latent vectors. The acquired document latent
factor vectors would be used in the proposed model. This CNN based on the four layers as
follows:

o Embedding Layer: It calculates the dense numeric matrix by using the input
document and that matric would be used as a document in the next convolution
layer.
 22

o Convolutional Layer: It finds the contextual features by using the document that
comes from the previous layer.

o Pooling Layer: It takes the variable-length document that comes from the
convolutional layer to find the fix-length feature vector by using pooling operation.

o Output Layer: The objective of output layers is to transform all features which are
extracted in previous layers into a particular task.

ii. In the second phase, the robust convolutional matrix factorization model has been
introduced. The ratings and document descriptions have been utilized in the introduced
model with the help of CNN and PMF for generating recommendations.

iii. The third phase describes the optimization methodology of the proposed hybrid
recommendation model.

3.4. Knowledge-based Recommendation Systems

The other types of RS such as content-based, collaborative, or hybrid, take more information for
the recommendation. What kind of information, each method take, we discussed above already. It
has also been noticed that the above-mentioned RS types are not given satisfactory results for the
customized RS includes financial services, real state, expensive importing products, etc. In this
scenario, the knowledge-based RS has been used because it seeks each user's requirements
individually. The utilization of knowledge bases and ground truth of such customized applications
is essential for such complex systems and that is a reason such systems are known as knowledge-
based RS.
Knowledge-based RS generates recommendations based on explicit user requirements and
attributes. The knowledge-based RS approach focuses on domain knowledge and not rely on the
data and contextual information like other RS approaches. The objective of the knowledge-based
RS is to find similarity-based retrieval. Customers with only fundamental information can find
relevant recommendations and help them to understand the information space in the knowledge-
based systems. Several studies related to the knowledge-based RS have been found in the literature
studies, however, one of them are discussed here.
Knowledge-based RS has two different types. The first one is the constraint-based
recommendation systems and another one is the case-based recommendation systems. The
knowledge-based RS categorization has been taken place based on the methodology used in for
the user interaction, and the knowledge bases that support the user interaction. The exploration of
both knowledge-based RS types as follows:
• Constraint-based Recommendation Systems: Constraint-based RS deals with the specific
customer requirements for generating the recommendations by using the domain-specific
rules. Those requirements of specific customers are also known as constraints of the
 23

system. The recommendations have been repeated unless the customer gets the desired
results [54].

• Case-based Recommendation Systems: Case-based RS deals with the specific cases

mentioned by the customer. The recommendations have been generated based on the
specific cases by using the similarity matrics that defines that domain-specific strategy and
domain-knowledge [55].
Both types of Knowledge-based RS provides the opportunity of requirements changing to the
customer.
Vijayakumar, V. et al. [50] proposed the knowledge-based RS in the domain of tourism. The
proposed approach formulates the travel list based on the domain points of interest. The most
relevant destinations have been predicted by using the domain points of interest knowledge bases.
The temporal features also included in the proposed approach to enhance the quality of
recommendations and increase the number of visitors. The heuristic-based algorithm has been used
for the implementation of the proposed approach.
To conclude the approaches towards the RS section, the conceptual understanding of the RS
approaches are described in table 1.
Table 1: The conceptual understanding of Recommendation Systems approaches
Approach Objective Input
Content-based RS Generate recommendations User personal, contextual,
based on the target user demographic information.
content information.
Collaborative filtering RS Generate recommendations Target user ratings and
based on the target user existing users ratings.
ratings, community ratings,
and other collaborative
approaches.
Hybrid RS Generate recommendations The mixture of user personal
based on user contextual information, ratings and
information, ratings of community ratings
community, and other RS
methods.
Knowledge-based RS Generate recommendations Domain knowledge and user-
based on explicit user specific requirements
requirements and domain
knowledge.
 24

4. Challenges of Recommendation Systems

The RS is very effective and beneficial to find relevant information through large data. However,
the practical implementation of RS showed different kinds of challenges have occurred. The
theoretical approaches of RS seem most effective but the implementation phase suffers the various
challenges. The RS challenges include the cold start problem, ethical issues, fraud schemes, data
sparsity problem, scalability, diversity, and many others. The mentioned RS challenges are
described below.
4.1. Cold Start Problem
The cold start problem is one of the major problems in the direction of RS. It has occurred when
the new users join the system without information history. In this scenario, the RS systems are not
able to generate recommendations for such new users. Table 2 shows the essential understanding
of the cold-start problem as follows:
Table 2: Basic understanding of the cold start problem
Items
Complete cold-start No ratings
Incomplete cold-start Partial ratings
Non-cold-start Absolute ratings

It has noticed that the most effective collaborative filtering technique of RS suffers the limitation
of the cold start problem. Many researchers proposed various solutions for catering to this problem
in the RS. However, the cold start problem is still an open research area.
Hawashin, B et al. [38] proposed the cold start problem solution based on the group interests in
the direction of RS. The proposed approach focused on the hidden interests of the group where the
news users belong to formulate recommendations for them. The proposed solution is summarized
as follows:
• Identify the domain factors that can affect the new user interests.
• Create an account of a new user based on her profile attributes.
• Use new user profile attributes to find the group of existing users that is most similar to the
new user.
• Extract the group interests based on the contextual data of a new user and avoid the rating
history of new users as it is insufficient.
The proposed solution avoided the item-to-item and user-to-user similarities and worked in the
finding of hidden interests and behavior of a new user based on her contextual profile. It seems a
novel method in the literature contribution of cold start problems.
Kang, S et al. [39] proposed the cross-domain RS based on semi-supervised learning. The proposed
approach deals with the cold-start problem of RS. Firstly, the proposed model computes the user-
to-user and user-to-item similarities to construct the latent vectors of users and items. After that,
the semi-supervised based mapping function is used to estimate the ground-truth of existing users
and their interaction with items. Finally, the latent vectors of existing users utilized to determine
 25

the neighbors of new users by using a multi-hop neighborhood inference technique. Experimental
results showed that the proposed model is better than supervised learning and significantly
improves the accuracy of recommendation to the cold-start users.
4.2. Ethical Issues or Privacy Concerns
The collaborative RS is entirely based on the customer data due to which major challenge of such
recommendation systems is privacy and security threats of customer information. It is essential to
take customers in confidence related usage of her personal information for prediction
recommendations and prevent a third party or malicious attacker to use customer information for
wrong means. Therefore, the research community pays attention to the privacy-preserving RS.
Marcello Paolo Scipioni [28] worked and analyzed the user privacy of the location-based RS. The
author focused on the arising privacy issues related to location-based RS and proposed a new
privacy-aware approach to perform location-based RS based on homomorphic encryption.
However, homomorphic encryption is very inefficient and slow. Frey et al. [29] worked on the
usage of blockchain in the RS for the strong privacy and security of users as well as the system.
This was the first time in literature studies of RS when blockchain technology was used to deal
with the privacy and security concerns of users and systems. The proposed approach uses secure
multiparty computation; privacy-preserving computation of cryptography and supported by
blockchain technology. The blockchain system (BS) is the interaction channel between customer
and company and all activities have occurred over BS. The company does not extract customer
information without permission and the customer aware of which information has been extracted
by the company.
Omar A.A et al. [31] proposed a secure and privacy-preserving RS method by using blockchain
technology. The distributed ledger attributes of blockchain technology, which store customer data
transactions, guarantee privacy concerns of customer data manipulation by a third party or
malicious attacker. In this approach, the customer has control over her personal information and
the company would not be able to such information without customer permission, and the company
has to pay some incentives while utilizing such information for a recommendation.
Arora M et al. [32] highlighted the importance of deep learning, artificial intelligence, and
blockchain technology in the e-commerce business and proposed a secure RS method. The whole
system can be categorized into the data layer, business layer, and application layer respectively.
The data layer validates user input by using artificial intelligence and extracts various attributes
with the help of deep learning. After that, the business layer stores a user-item relation in a block
of blockchain technology and generates recommendations by using the k-mean clustering
algorithm. Finally, the application layer shows all recommendation predictions to the particular
customer. This strategy prevents privacy concerns and threats of misusing personal data by third
parties or malicious attackers.
F. Casino et al. [33] worked on the blockchain-based privacy-preserving RS and stated
decentralized RS systems protect user privacy and threat but such systems are inefficient and slow.
As a consequence, blockchain technology has been used in decentralized RS and proposed new
RS architecture. This architecture uses the locality-sensitive hashing algorithmic technique for the
 26

classification of data and generating recommendations. Empirical results showed the accuracy of
the new architecture is the same other decentralized RS but efficiency is significantly improved
due to the use of blockchain technology.
Frey et al. [34] worked on shopping experience in the domain of e-commerce and the proposed
blockchain-based RS approach. This approach not only protects customer information privacy but
as well as allows a customer to control her personalized information. The proposed method based
on the blockchain storage network with a bitcoin protocol and tamper-proof way is used to secure
multiparty computation of data.
Consequently, customer privacy and security concerns have arisen and traditional cryptography
techniques are not sufficient to protect customer information without affecting system performance
and efficiency. The usage of blockchain technology makes it possible to design and develop a
secure and efficient Collaborative RS for e-commerce, and leverage more advantages of customer
trust value, satisfaction, and retention. But the proposed blockchain approaches are not applied
practically on real-world RS applications. Due to this reason, it is still an open research area.
4.3. Fraud Challenge
The commercial websites widely use the RS for the financial benefits and marketing objectives.
However, some vendors use unethical and fraud schemes to cheat with the benefits of RS for their
financial gains. The fraud can be performed with two types of attacks as follows:
i. Push attacks: Some vendors falsely increase their product ratings, positive reviews to
grab the attraction of customers.

ii. Nuke attacks: Some vendors try to reduce the competitor product ratings, positive
reviews, and involve in spreading out false information about the competitor products.
These kinds of fraud schemes in the RS are known as shilling attacks. The research community
proposed many solutions to prevent such attacks however, it is still an open research area in the
development of the RS.
Yuan, W et al. [40] proposed the neural-network-based detection of shilling attacks approach by
using the user rating history and latent features. The content features of users and items used for
the learning purpose and input profiles are categorized into attack profile and normal profile. The
first label assigned with the attack profile and second label assigned with all profiles except attack
profile. Then the latent feature vector includes the user vector and item vector, which are inputs of
the neural matrix decomposition model. This model is the combination of matrix decomposition
and multi-layer perception and responsible for the detection of attacks.
4.4. Data Sparsity Problem
The sparsity of data is a major challenge for acquiring better accuracy and performance of RS. The
RS used in commercial websites (e.g. Amazon, eBay, Netflix) has to deal with the massive large
itemsets. Apart from the cold-start user, even the active user only buys one percent item from such
large itemsets. If the active user wants to purchase one movie and there are 10 million movies on
the Netflix website then imaging the sparsity of the user-item matric. In the scenario, the
 27

neighborhood algorithms are not able to compute the distance for the recommendations, and the
system becomes poor and ineffective. The basic understanding of the data sparsity problem is
shown in Table 3.
Table 3: Customer-Item metric
Item
2 4
3
Customer

5 1
2
1
2
5

It has been noticed that data sparsity problems encountered in the collaborative RS. However, the
content-based and RS solve the data sparsity problem because of content-based RS based on the
contextual information of the user. The hybrid RS approach also solves the data sparsity problem
because content-based RS is the part of the hybrid approach. The reduced coverage and neighbor
transitivity also come under the data sparsity problem. The research community proposed various
theoretical approaches to solve this problem for collaborative RS but the real-world
implementation of such approaches is still an open research area.
Deng, J. et al. [41] stated that data sparsity is a major challenge for the effective collaborative RS
and proposed the K-medoids clustering-based RS for solving the data sparsity problem. The
proposed approach makes it possible to use the rating information for the selection of the cluster
center by using the probability distribution. The improved Kullback–Leibler (KL) divergence
method is used for the computation of the distance between two items. This method calculates the
distance of the cluster center to efficiently determines the different items.
Choi, J. W. et al. [42] proposed a clustering-based collaborative RS approach to deal with the data
sparsity and scalability problems respectively. However, the summarization of the data sparsity-
based solution has been described here. The proposed method does the clustering based on the
customer contextual information and then predicts the particular customer preferences by using
similar cluster-based customers' preferences and completes the user-item matric. The proposed
approach works as follows:
• Apply the k-means clustering on the complete row where the particular customer occurred
in customer-item matric.
• Describe neighbors of a particular customer by using a similar cluster.
• Complete the matric.
• Use collaborative RS techniques to recommend an item on the based of a filled matrix.
It has been stated that the proposed method prevents additional preference rating scores.
 28

4.5. Scalability
The RS techniques suffer the scalability problem when the existing number of users and items
significantly increased. In this situation, the execution time of each operation is exponentially
increased because the utilizing of computing resources is directly proportional to the data.
Consequently, the performance of the RS algorithms is ineffective and inaccurate. The scalability
problem depends on the performance and configuration of the computer. The literature studied
showed that the dimensional reduction method used to overcome the scalability problem in the
direction of RS. The scalability problem is also a popular challenging problem due to the massive
demand for big data applications. The researchers proposed various solutions to control the
scalability problem, however, it is still an open research area.
Choi, J. W. et al. [42] proposed a clustering-based collaborative RS approach to deal with the data
sparsity and scalability problems respectively. However, the summarization of the scalability-
based solution has been described here. The proposed method uses a clustering technique to reduce
the data space for overcoming the scalability problem. The proposed approach works as follows:
• Construct the matric of existing customers and items.
• Apply the k-mean clustering on the constructed matric.
• Apply the collaborative RS algorithms on the k-clusters.
• Display the recommendations.
It has been stated that the proposed method prevents the exponential growth of operation time.
4.6. Diversity
Diversity is the user usability and user experience characteristics of RS which includes the variety
of items based on the individual user requirements [43]. It has been noticed that each user has
different requirements of diversity that make it challenging for the RS algorithm to generate
recommendations according to each user's requirements. Diversity of the RS can be divided into
user diversity and item diversity respectively.

• User Diversity: Each user has different interests, behaviors, preferences, attributes, etc.
The personalized RS algorithms used such characteristics for the recommendations.

• Item Diversity: Item diversity shows the candidate recommendations that recommend to
each uses differently. Many approaches have been proposed for the item diversification in
the literature studies.

Literature studies showed that diversification of recommended items to the user badly affected the
performance and accuracy of RS. That is the reason, it is still an open research area in the direction
of RS.

Nava Tintarev et al. [44] proposed the diversity-aware recommendation model. The proposed
model predicts the diverse selection of news for each user. The diversity-aware recommendation
model works on the two principles as follows:

i. Each user has different requirements.

 29

ii. The items' recommendations must fulfill each user's requirements.

It has been stated that experimental results maximize the diversity of content without damaging
system performance and efficiency.

5. Evaluation of Recommendation Systems

The set of recommendation lists has been generated but how would be performance and
effectiveness of that list examined. Such questions encountered in the evaluation of RS. The
evaluation of RS is different from the classification and regression evaluation where the only
missing target variables have been predicted. The RS has data-driven properties of observed and
expected entities in the matrix form. Through the intuition, it has been stated that the evaluation
of RS is the generalization of the classification evaluation. As a result, the modified versions of
the classification evaluation are used in the evaluation of RS [45].

There are various factors such as features of a dataset, RS technique, aim of recommendations, etc
play an important role in the selection of the evaluation matric. The evaluation of RS can be
classified into two categorize into prediction accuracy metrics and classification accuracy metrics
[46].
1. Prediction accuracy metrics: It includes the Mean Absolute Error (MAE), the Root Mean
Square Error (RMSE), etc. Each of them is described below.

o Mean Absolute Error (MAE): It is the average absolute dissimilarity between

recommendations and actual data. Mathematically, it can be defined as,

1
MAE =
N
 Au ,i − Bu ,i (6)

Where Au,i is the recommendations, u is a user, i is an item, and Bu,i is the actual
data. N is the total number of samples.

o Root Mean Square Error (RMSE): It is equal to the square root of the average
absolute dissimilarity between recommendations and actual data. Mathematically,
it can be defined as,

 (Re commendation i − Actuali ) 2

RMSE = i =1
(7)
N
Where N is the total number of samples.

2. Classification accuracy metrics: It includes confusion metric, accuracy, precision, recall,

F-measures, etc. Each of them is described below.
 30

o Confusion metric: It generates the output to give the complete performance of the
proposed approach. The essential structure of the confusion metric is given in Table
4.
Table 4: Understanding of Confusion Metric
PREDICTED VALUES TOTAL
+ -
C C
TP FN N+
C+ True False Actual
ACTUAL

Positive Negative Positive

VALUES
(Hits) (Miss) Numbers
FP TN N-
C- False True Actual
Positive Negative Negative
(Miss) (Hits) Numbers
Nˆ+ Nˆ- N
TOTAL Predicted Predicted Total
Positive Negative Example
Numbers Numbers Numbers

Where C+ and C- are the class labels of predicted and actual values of the samples.
o Precision: It is a measure of exactness by calculating the relevant items retrieved
from all retrieved items. The formula of the precision of the proposed model is as
follows:
TruePositive
Pr ecision = (8)
TruePositve + FalsePositive

o Recall: It is a measure of completeness by calculating the relevant items retrieved

from all retrieved items. The formula of the recall of the proposed model is as
follows:
TruePositive
Re call = (9)
TruePositve + FalseNegative

o F-measures: It is used to get a single comparison value from the precision and
recall. It helps to check the test accuracy of the proposed approach. F-measure also
describes as the harmonic mean of precision and recall.

2(Pr ecision  Re call )

F − measures = (10)
Pr ecision + Re call
There are various other evaluation metrics, evaluation scores, online, and offline evaluation
methods in the literature studies of the RS. However, right now we are not sure which evaluation
technique or techniques would be used in our thesis. We intended to choose appropriate evaluation
method or methods as per the specification of our future proposed algorithm.
 31

6. Recommendation Systems Applications

It is justified to say that RS-based applications have eaten the growth of traditional businesses and
services. All such practical applications of RS are part of human life and drop the significant
impact individual and collective daily routine of humans throughout the world. The research
community, big tech giants, and various governmental institutions work on the practical
applications of RS to improve them gradually. The goal of this research includes the practical
implementation of RS-based applications and improve the performance, efficiency, user usability,
etc.
The application of RS includes web search, mobile search, music, advertisement, trade, e-
commerce, etc. The customized methods are used for the adaption of RS in these applications. It
is the diversity of RS to identify and trace the user interests and preferences through the various
user-identification mechanisms. The RS formulates the inappropriate, irrelevant, and inaccurate
recommendations in the absence of the user-identification mechanism.
The RS-based applications face specific challenges as follows:
1. Query recommendation: It is exciting to know how query logs are used in the
recommendation system to predict the queries for the customer. The query
recommendation is not only dependent on the contextual information of the customer but
also dependent on the session history of the customer, current trends, temporal factor, and
several other features. The customer-identification mechanism is not enough in the case of
query recommendation. It has been stated that query recommendation is not a
personalization problem. These are the reasons behind the complexity of the query
recommendation challenge for the RS-based application.

2. Online platform content and news personalization: The online platform identifies the
customer who revisits the website by using robust customer-identification mechanisms. In
this scenario, the contextual information of the customer has been used for the
personalization recommendations. The news personalization engines use customer-
identification mechanisms to find relevant and interesting news for that customer so that
the same customer would revisit the website. The news personalization based on customer
behavioral activities such as which and how many times an item has been clicked by the
specific customer. This kind of RS-based application focuses on the implicit feedback of
the customer like clicks. The implementation of customer-identification mechanisms raises
customer privacy and data breaches concerns.

3. Computational advertising: Computational advertising is one of the practical applications

of RS. The goal of computational advertising is to find relevant things (like products,
article, e-book, touristic places, etc.) over the internet by using customer search history.
Take the example of Google search, which shows the relevant advertisement of various
business-centric companies based on the user search. In this way, Google generates a large
 32

amount from those business-centric companies. The RS is directly involved in the

computation advertising and it is one of the challenging areas for the research community.

4. Reciprocal recommendation system: It is a unique challenge in the direction of RS-based

applications where both user and item have preferences, predefined requirements and
recommendations have been occurred to find the best match. The typical example of a
reciprocal recommendation system is an online dating website where both men and women
try to find the best match for romantic purposes. The link-prediction methods are used in
the reciprocal recommendation system.
The research community proposed various solutions to solve all the aforementioned challenges.
However, we would like to investigate the query recommendation challenge briefly and would
review the baseline and improved approaches in the literature studies of this area.
6.1. Query Recommendations
Modern search engines are more responsive and interactive for the naïve user. It has been studied
that if the user would not get what she wants within the first 20 seconds then there is more
probability user leaves that application and move to another one. This is a kind of human-computer
interaction problem (it may also be known as user usability problem) and has been solved by
adding the query recommendations feature in the search engines. The baseline algorithm of query
recommendations is described in the processes diagram in Fig 6. The clustering technique has been
used in this baseline algorithm.
Literature studies of the query recommendations showed that researchers proposed various
approaches in this direction, however, some of them are discussed below.
Zhang, Z et al. [52] proposed the query recommendation approach by using the query logs. The
proposed approach based on two methods i.e. sequential search behavior, and traditional content-
based similarity. The first method deals with customer search behavior while the second method
relies on the contextual information of the customer. It has stated that classification or clustering
techniques in the direction of query recommendation, assume the initial value of the number of
classes or number of clusters. However, the proposed hybrid approach is more adaptive and
accommodate the seasonal trends factor in the recommendation process.
Baeza-Yates, R et al. [51] proposed the query recommendations method by using the query logs
for search engines. The proposed method takes an input query and predicts the list of queries as an
outcome. The clustering technique is used that identifies a semantical similar group of queries and
build clusters. The clustering uses the previous search queries of the same user from the query logs
of the search engine. The proposed query recommendation method works as follows:
 33

Fig 6. The processes diagram of clustering based query recommendation algorithm

1. The proposed approach only examines the queries that are in respective query logs and the
submitted query might be the same however, query sessions always different. The query
 34

session based on the existing query text and URL clicks represents as follows:

QuerySessions := (queryText , (clickedURL)*) (11)

2. After that finds the cluster of existing queries that is similar to the submitted query. Then
the rank score of each query in a similar cluster has been calculated. The rank score
computation based on the two factors as follows:

o The similarity of query: The similarity between the query and submitted query can
be found by using the term-weight vector of each query. Each term-weight vector
based on the number of terms occurrences and number of clicks.

o The support of query: The support of the query defines the relevancy of each query
with the submitted query. It has been calculated as the total documents in a similar
cluster divided by the user interested documents. In this case, interested documents
are those which were clicked by the user.

3. Display the related queries to the user.

The aforementioned two literature studies deal when the submitted query is related to existing
query logs. But what happens when the submitted query rarely matches the query logs. Such a
problem is known as the long-tail query problem in the domain of the query recommendations.
Many researchers proposed different approaches to deal with the long-tail query problem and one
of them is described here.

Huang, Z.et al. [53] worked to handle the long-tail query problem when the customer search history
and existing query logs are insufficient to perform the query recommendations. They found that
entities of the submitted query play important to deal with the long-tail query problem. The
extracted entities from the submitted query would be used to suggest new query recommendations
by using the knowledge base of the proposed approach and clicks logs of the particular customer.
 35

6.2. Types of Recommendation Systems Applications

I have investigated types of RS applications and development and found that there are various
kinds of RS applications; however, I would like to write about six different areas of RS applications
as follows:
6.2.1. E-Government
The significant increase in internet users does impact government institutions' services and
responsibilities. The various governments of the world are going in the direction of digital
transformation or digitalization. The development of e-government RS is one of challenge
in the digital transformation. The e-government RS can be categorized into government to
citizen services such as issuing national identity cards7, and government to business service
such as a fast train8. It included the applications interface personalization and adaption, and
services recommendation.

6.2.2. E-Commerce or E-Shopping or M-Commerce

The trend of e-commerce or e-shopping or m-commerce is significantly increased in recent
years. However, the positive impact of the COVID-19 over e-commerce is unbelievable.
The unique and mature RS are used in the e-commerce area by the biggest e-commerce
companies such as Amazon9, eBay10, Walmart11. The RS systems are widely used in
Turkish-based e-commerce companies such as the getir12, the hepsiburada13. Such RS
systems help users to find the relevant products based on overall best selling items,
customer demographics, customer personalization, etc. The business-to-consumer (B2C)
model has been used in e-commerce RS systems.

6.2.3. E-Library
With a wide range of books and scholarly materials on the e-library, it seems hard to find
relevant information and scholarly material. However, the RS makes it possible to find
relevant materials according to user profiles and preferences. E-library RS aims to generate
recommendations based on user profile and her preferences by using library scholarly
materials. It has been noticed that big research journals-based digital libraries such as
ACM14, Elsevier15, SAGE16, and many others use RS to show relevant journals. When the

7
The detail of issuing national identify card for the citizen of Republic of Turkey can be found from here.
https://www.nvi.gov.tr/tc-kimlik-karti (accessed on 5th May 2020)
8
The detail of Turkish fast train services can be found here.
https://ebilet.tcddtasimacilik.gov.tr/view/eybis/tnmGenel/tcddWebContent.jsf (accessed on 5th May 2020)
9
The Amazon online shopping is one of biggest e-commerce application. https://www.amazon.com/
10
The eBay is e-commerce company. https://www.ebay.com/
11
The Walmart is e-commerce company. https://www.walmart.com/
12
The getir is Turkish-based e-commerce company. https://www.getir.com/
13
The hepsiburada is largest Turkish e-commerce company. https://www.hepsiburada.com/
14
The details of ACM digital library can be found here. https://dl.acm.org/
15
The details of Elsevier digital library can be found here. https://www.elsevier.com/solutions/eLibrary
16
The details of SAGE digital library can be found here . https://us.sagepub.com/en-us/nam/digital-library-
products
 36

user finds a particular journal then such a system also provides a list of relevant journals
on the small dialog box over the same research page. These are the practical applications
of e-library RS.
6.2.4. E-Learning
The term MOOC (Massive Open Online Course) has been used more in the educational
institute and professional organization. The concept of e-learning educational is widely
applicable in the global village. The RS systems used in e-learning education to find the
relevant courses, certification, and many other webinars and workshops. The big e-learning
organizations such as Udemy17, Udactiy18, edX19, IBM Cognitive Class20, and others
leveraging RS systems to grab massive numbers of e-students.
6.2.5. E-Tourism
Before the proceeding of actual tourism, people prefer to search over the internet about the
destination, visiting places, accommodation, transportation, restaurant, etc. The RS
systems play an important role to find relevant information for the customer. E-tourism RS
formulate suggestions and recommendations based on customer personalization, most
visited places, and positive ratings. Some RS systems work in the direction of destination
suggestions and some work for the transportation, hotel, restaurant recommendation. The
largest companies such as the Trivago21, Expedia group22 , etc provide tourism suggestions
to their customer. The business-to-customer (B2C) model has been used in the e-tourism
industry.
6.2.6. E-Resource Services
The recommendation of movies, documents, music, datasets, etc is included in e-resources
services that are uploaded by other users over the particular platform. The typical examples
are Youtube23, SoundCloud24, Kaggle25, ResearchGate26, and many others. Some
customers upload e-resources and other customers grab such resources according to their
interest. The customer-to-customer (C2C) model has been used in the e-resource services.
The RS systems can be applied to various kinds of industries such as entertainment, e-content,
service-oriented, e-group activities, e-business, e-developer platform, etc [21,47]. The
aforementioned areas of RS demonstrated the real-world applications of RS. These studies directly

17
The Udemy is e-learning platform for students and professionals. https://www.udemy.com/
18
The Udacity is e-learning platform. https://www.udacity.com/
19
The edX is Harvard University based free e-learning platform. https://www.edx.org/
20
The Cognitive Class is IBM sponsored e-learning platform. https://cognitiveclass.ai/courses
21
The trivago provides e-hotel reservation. https://www.trivago.com.tr/
22
The expedia group is world travel platform. https://www.expediagroup.com/
23
The Youtube is online video sharing platform. https://www.youtube.com/
24
The SoundCloud is online audio distribution platform. https://soundcloud.com/
25
The Kaggle is online community of data scientists and machine learning practitioners. https://www.kaggle.com/
26
The Research Gate is commercial social networking for researchers and scientists.
https://www.researchgate.net/
 37

support the researchers to understand the applications and development of RS. In this thesis, we
intended to work on e-commerce or e-shopping area.

7. Conclusions
The volume of information is exponentially increasing in each second on various kinds of
platforms over the internet. The RS makes it possible to find the relevant information from such a
large amount of data. That is the reason the RS becomes a fundamental part of the marketing
strategy of each business-centric digital organization. The online user activities carry the intra-
similarity features of a customer against product items and RS used those features to predict the
future possible choice of the same customer. The approaches towards the RS are classified into
content-based, collaborative filtering, hybrid, and knowledge-based. Though the usage of the RS
approaches proves the significant increase in user satisfaction, user usability but RS domain still
suffers several challenges such as cold-start problem, privacy concerns, fraud, data sparsity,
scalability, novelty, diversity, synonymy, etc. Prediction accuracy and classification accuracy
metrics are used for the evaluation of the RS approaches. The application of RS includes web
search, mobile search, music, advertisement, trade, e-commerce, etc.
We investigated used approaches and techniques in the direction of the RS domain. The RS
systems are very effective for business-centric companies and it is an essential part of their sales
strategy. The companies not only extract customer contextual information (like age, gender,
location, time, likeness, dislike ness, search, shopping, click histories and credit card frequency),
but also community ratings, seasonal trends, domain knowledge, etc. It has been noticed that the
usage of approach depends on the type of business, business short and long-term objectives, and
several other factors. Most of the proposed approaches (like privacy-secure RS, trend-aware RS,
attack-resistant RS, etc.) are in theoretical form only and still not implemented. The challenge for
the research community is to implement the proposed approaches without any tradeoff on the
performance and effectiveness of the RS. We think so robust and effective RS method based on
several approaches due to the diversification of this domain.
 38

8. References
1. Jian Wei, Jianhua He, Kai Chen, Yi Zhou, Zuoyin Tang. Collaborative filtering and deep
learning based recommendation system for cold start items. Expert Systems with Applications
Volume 69, 1 March 2017, Pages 29-39.
2. Abhay E. Patil, Simran Patil, Karanjit Singh, Parth Saraiya, Aayusha Sheregar. ONLINE BOOK
RECOMMENDATION SYSTEM USING ASSOCIATION RULE MINING AND
COLLABORATIVE FILTERING. IJCSMC, Vol. 8, Issue. 4, April 2019, pg.83 – 87.
3. Shaowei Wang, David Lo, Bogdan Vasilescu, Alexander Serebrenik. EnTagRec++: An
Enhanced Tag Recommendation System for Software Information Sites. Article in Empirical
Software Engineering. July 2017.
4. Baolin Yi, Xiaoxuan Shen, Hai Liu, Zhaoli Zhang, Wei Zhang, Sannyuya Liu, and Naixue
Xiong. Deep Matrix Factorization With Implicit Feedback Embedding for Recommendation
System. IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 15, NO. 8,
AUGUST 2019.
5. Zhenhua Huang, Guangxu Shan, Jiujun Cheng, Jian Sun. TRec: an efficient recommendation
system for hunting passengers with deep neural networks. The Natural Computing Applications
Forum 2018.
6. SRS Reddy, Sravani Nalluri, Subramanyam Kunisetti, S. Ashok and B. Venkatesh. Content-
Based Movie Recommendation System Using Genre Correlation. Smart Intelligent Computing
and Applications. Smart Innovation, Systems and Technologies, vol 105. Springer, Singapore.
7. WANG ZHOU, JIANPING LI, MALU ZHANG, YAZHOU WANG, FADIA SHAH. Deep
Learning Modeling for Top-N Recommendation With Interests Exploring. Article in IEEE Access
September 2018.
8. L. Zheng, V. Noroozi, and P. S. Yu, ``Joint deep modeling of users and items using reviews for
the recommendation,'' in Proc. 10th ACM Int. Conf. Web Search Data Mining, 2017, pp. 425434.
9. T. Bansal, D. Belanger, and A. Mccallum, ``Ask the GRU: Multi-task learning for deep text
recommendations,'' in Proc. 10th ACM Conf. Rec- commander Syst., 2016, pp. 107114.
10. R. Catherine and W. Cohen. (Jun. 2017). ``TransNets: Learning to transform for the
recommendation.'' [Online]. Available: https://arxiv.org/ abs/1704.02298
11. S. Cao, N. Yang, and Z. Liu, ``Online news recommender based on stacked auto-encoder,'' in
Proc. IEEE/ACIS 16th Int. Conf. Comput. Inf. Sci. (ICIS), May 2017, pp. 721726.
12. QiPing Zheng, Yi Wang, 2012. Using formal concept analysis and collaborative filtering to
the recommendation system in e-commerce. Journal of Convergence Information Technology.
Vol. 7 Issue 10, p389.

13. Jesus Bobadilla, et al., 2012. A collaborative filtering similarity measure based on singularities.
Information Processing & Management Volume 48, Issue 2, March 2012, Pages 204-217.
 39

14. Zhenzhen Fan, Nitin Pradeep Kumar, 2015. Hybrid user-item based collaborative filtering.
Procedia Computer Science Volume 60, Pages 1453-1461.
15. Xin Luo, et al., 2011. Improving k nearest-neighborhood-based collaborative filtering via
similarity support. International Journal of Digital Content Technology and its Applications
5(7):248-256.
16. Jianming He and Wesley W. Chu. A Social Network-Based Recommender System (SNRS).
2012.
17. M. J. Pazzani and D. Billsus, “Content-based recommendation systems,” in The adaptive web.
Springer, 2007, pp. 325–341.

18. R. Yera Toledo and L. Martinez, “Fuzzy tools in recommender systems: A survey,” vol. 10,
pp. 776 – 803, 03 2017.
19. T. Rutkowski, J. Romanowski, P.Woldan, P. Staszewski, R. Nielek, and L. Rutkowski, "A
Content-Based Recommendation System Using Neuro-Fuzzy Approach," 2018 IEEE International
Conference on Fuzzy Systems (FUZZ-IEEE), Rio de Janeiro, 2018, pp. 1-8.
20. Achakulvisut T, Acuna DE, Ruangrong T, Kording K (2016) Science Concierge: A Fast
Content-Based Recommendation System for Scientific Publications. PLoS ONE 11(7): e0158423.
https://doi.org/10.1371/journal.pone.0158423
21. Jie Lu, Dianshuang Wu, Mingsong Mao, Wei Wang, Guangquan Zhang. Recommender system
application developments: A survey. Decision Support Systems Volume 74, June 2015, Pages 12-
32.
22. A. Rzheuskyi, N. Kunanets and M. Stakhiv, "Recommendation System “Virtual Reference”,"
2018 IEEE 13th International Scientific and Technical Conference on Computer Sciences and
Information Technologies (CSIT), Lviv, 2018, pp. 203-206.
23. Nabil S., Elbouhdidi J., Chkouri M.Y. (2020) Toward a New Performing Recommendation
System. In: Ezziyyani M. (eds) Advanced Intelligent Systems for Sustainable Development
(AI2SD’2019). AI2SD 2019. Advances in Intelligent Systems and Computing, vol 1105. Springer,
Cham.
24. Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, W. Bruce Croft. Towards Conversational
Search and Recommendation: System Ask, User Respond. CIKM '18: Proceedings of the 27th
ACM International Conference on Information and Knowledge Management October 2018 Pages
177–186.
25. Hwangbo, Hyunwoo et al. “Recommendation system development for fashion retail e-
commerce.” Electron. Commer. Res. Appl. 28 (2018): 94-101.
26. Youdong Yun, Danial Hooshyar, Jaechoon Jo, and Heuiseok Lim. Developing a hybrid
collaborative filtering recommendation system with opinion mining on purchase review. Journal
of Information Science 1–14 The Author(s) 2017.
 40

27. Zarzour H., Maazouzi F., Soltani M., Chemam C. (2018) An Improved Collaborative Filtering
Recommendation Algorithm for Big Data. In: Amine A., Mouhoub M., Ait Mohamed O., Djebbar
B. (eds) Computational Intelligence and Its Applications. CIIA 2018. IFIP Advances in
Information and Communication Technology, vol 522. Springer, Cham.
28. Marcello Paolo Scipioni. Towards Privacy-Aware Location-Based Recommender Systems.
2011.
29. Frey, Remo M., Dominic Wörner and Alexander Ilic. “Collaborative Filtering on the
Blockchain: A Secure Recommender System for e-Commerce.” AMCIS (2016).
30. Global E-commerce 2019 https://www.emarketer.com/content/global-ecommerce-
2019#targetText=Despite%20its%20obviously%20strong%20growth,20%25%20threshold%20be
ginning%20in%202020 (accessed 16 March 2020)

31. Omar A.A., Bosri R., Rahman M.S., Begum N., Bhuiyan M.Z.A. (2019) Towards Privacy-
preserving Recommender System with Blockchains. In: Wang G., Bhuiyan M., De Capitani di
Vimercati S., Ren Y. (eds) Dependability in Sensor, Cloud, and Big Data Systems and
Applications. DependSys 2019. Communications in Computer and Information Science, vol 1123.
Springer, Singapore.
32.Arora M., Chopra A.B., Dixit V.S. (2020) An Approach to Secure Collaborative Recommender
System Using Artificial Intelligence, Deep Learning, and Blockchain. In: Choudhury S., Mishra
R., Mishra R., Kumar A. (eds) Intelligent Communication, Control, and Devices. Advances in
Intelligent Systems and Computing, vol 989. Springer, Singapore.
33.F. Casino and C. Patsakis, "An Efficient Blockchain-Based Privacy-Preserving Collaborative
Filtering Architecture," in IEEE Transactions on Engineering Management.
34. Frey, Remo Manuel, Denis Vuckovac, and Alexander Ilic. “A Secure Shopping Experience
Based on Blockchain and Beacon Technology.” RecSys Posters (2016).
35. Tarus, J. K., Niu, Z., & Kalui, D. (2018). A hybrid recommender system for e-learning based
on context awareness and sequential pattern mining. Soft Computing, 22(8), 2449-2461.
36. Logesh, R., & Subramaniyaswamy, V. (2019). Exploring hybrid recommender systems for
personalized travel applications. In Cognitive informatics and soft computing (pp. 535-544).
Springer, Singapore.
37. Kim, D., Park, C., Oh, J., & Yu, H. (2017). Deep hybrid recommender systems via exploiting
document context and statistics of items. Information Sciences, 417, 72-87.
38. Hawashin, B., Mansour, A., Kanan, T., & Fotouhi, F. (2018, October). An efficient cold start
solution based on group interests for recommender systems. In Proceedings of the First
International Conference on Data Science, E-learning and Information Systems (pp. 1-5).
 41

39. Kang, S., Hwang, J., Lee, D., & Yu, H. (2019, November). Semi-Supervised Learning for
Cross-Domain Recommendation to Cold-Start Users. In Proceedings of the 28th ACM
International Conference on Information and Knowledge Management (pp. 1563-1572).
40. Yuan, W., Xiao, Y., Jiao, X., & Ming, Y. (2019, November). Neural Network Detection of
Shilling Attack Based on User Rating History and Latent Features. In 2019 International
Conference on Machine Learning, Big Data and Business Intelligence (MLBDBI) (pp. 229-232).
IEEE.
41. Deng, J., Guo, J., & Wang, Y. (2019). A Novel K-medoids clustering recommendation
algorithm based on a probability distribution for collaborative filtering. Knowledge-Based
Systems, 175, 96-106.
42. Choi, J. W., Yun, S. K., & Kim, J. B. (2019, May). Improvement of Data Sparsity and
Scalability Problems in Collaborative Filtering Based Recommendation Systems. In International
Conference on Applied Computing and Information Technology (pp. 17-31). Springer, Cham.
43. Frontiers of Recommendation Systems and Diversity Article
https://towardsdatascience.com/frontiers-of-recommendation-systems-diversity-
f668adad502c (accessed on 19th May 2020)

44. Nava Tintarev. 2017. Presenting Diversity Aware Recommendations. In Proceedings of The
FATREC Workshop on Responsible Recommendation, Como, Italy., August 31, 2017.

45. Aggarwal, C. C. (2016). Recommender systems (Vol. 1). Cham: Springer International
Publishing, (available here https://www.springer.com/gp/book/9783319296579 and accessed
on 18th May 2020)

46. Herlocker, J. L., Konstan, J. A., Terveen, L. G., & Riedl, J. T. (2004). Evaluating collaborative
filtering recommender systems. ACM Transactions on Information Systems (TOIS), 22(1), 5-53.

47. Raghuwanshi, S. K., & Pateriya, R. K. (2019). Recommendation Systems: Techniques,

Challenges, Application, and Evaluation. In Soft Computing for Problem Solving (pp. 151-164).
Springer, Singapore.

50. Vijayakumar, V., Vairavasundaram, S., Logesh, R., & Sivapathi, A. (2019). Effective
knowledge-based recommender system for tailored multiple points of interest
recommendation. International Journal of Web Portals (IJWP), 11(1), 1-18.

51. Baeza-Yates, R., Hurtado, C., & Mendoza, M. (2004, March). Query recommendation using
query logs in search engines. In International conference on extending database technology (pp.
588-596). Springer, Berlin, Heidelberg.

52. Zhang, Z., & Nasraoui, O. (2006, May). Mining search engine query logs for query
recommendation. In Proceedings of the 15th international conference on World Wide Web (pp.
1039-1040).
 42

53. Huang, Z., Cautis, B., Cheng, R., Zheng, Y., Mamoulis, N., & Yan, J. (2018). Entity-based
query recommendation for long-tail queries. ACM Transactions on Knowledge Discovery from
Data (TKDD), 12(6), 1-24.

54. A. Felfernig and R. Burke. Constraint-based recommender systems: technologies and research
issues. International Conference on Electronic Commerce, 2008.

55. Burke, R. (2000). Knowledge-based recommender systems. Encyclopedia of library and

information systems, 69(Supplement 32), 175-186.

View publication stats