Credibility in Social Media Opinions, News, and Health Information-A Survey

Advanced Review
Credibility in social media:

opinions, news, and health
information—a survey
Marco Viviani* and Gabriella Pasi
In the Social Web scenario, where large amounts of User Generated Content dif-
fuse through Social Media, the risk of running into misinformation is not negligi-
ble. For this reason, assessing and mining the credibility of both sources of
information and information itself constitute nowadays a fundamental issue. Cred-
ibility, also referred as believability, is a quality perceived by individuals, who are
not always able to discern with their cognitive capacities genuine information from
the fake one. For this reason, in the recent years several approaches have been pro-
posed to automatically assess credibility in Social Media. Most of them are based
on data-driven models, i.e., they employ machine-learning techniques to identify
misinformation, but recently also model-driven approaches are emerging, as well
as graph-based approaches focusing on credibility propagation. Since multiple
social applications have been developed for different aims and in different con-
texts, several solutions have been considered to address the issue of credibility
assessment in Social Media. Three of the main tasks facing this issue and consid-
ered in this article concern: (1) the detection of opinion spam in review sites, (2) the
detection of fake news and spam in microblogging, and (3) the credibility assess-
ment of online health information. Despite the high number of interesting solu-
tions proposed in the literature to tackle the above three tasks, some issues remain
unsolved; they mainly concern both the absence of predefined benchmarks and
gold standard datasets, and the difficulty of collecting and mining large amount of
data, which has not yet received the attention it deserves. © 2017 John Wiley & Sons, Ltd
How to cite this article:
WIREs Data Mining Knowl Discov 2017, e01209. doi: 10.1002/widm.1209
INTRODUCTION running into misinformation is not negligible. For

this reason, assessing and mining the credibility of
N owadays, in the Web 2.0 era, the interaction

between users is promoted by a number of
Social Media that facilitate the establishment of
both pieces and sources of information constitutes
nowadays a fundamental issue for users. Credibility,
also referred as believability, is a concept that has
social relationships, and the diffusion of information been studied since ancient times, and in different
in the form of User Generated Content (UGC). In this research fields, such as communication, psychology,
context, users are able to publish directly textual and social sciences. Recent studies have identified
reviews or opinions, pictures, videos, and others, credibility as a quality perceived by individuals, who
almost without any form of trusted external control. in most cases do not have the necessary means to dis-
It clearly emerges how, in this situation, the risk of cern with their own cognitive capacities genuine
information from fake one. With respect to the ‘off-
*Correspondence to: [email protected] line’ world, where people traditionally reduce uncer-
Department of Informatics, Systems and Communication (DISCo), tainty about credibility based either on the reputation
University of Milano-Bicocca, Milano, Italy of the source of information (e.g., experts and/or
Conflict of interest: The authors have declared no conflicts of inter- opinion leaders), or on personal trust based on first-
est for this article. hand experiences, in the digital realm the assessment
© 2017 John Wiley & Sons, Ltd 1 of 25

Advanced Review wires.wiley.com/dmkd
of credibility is often more complex than in previous

media contexts. Online, the multiplicity of sources
involved in the dissemination of contents, their possi-
ble anonymity, the absence of standards for informa- The Post-Truth Era
tion quality, the ease in manipulating and altering
The Oxford Dictionary has recently declared
the information, the lack of clarity of the context, post-truth to be its international word of the
and the presence of many potential targets of credi- year for 2016. It is defined by the dictionary as
bility evaluation (i.e., the content, the source, and the an adjective ‘relating to or denoting circum-
medium), make urgent the need of developing sys- stances in which objective facts are less influen-
tems for helping users in automatically assess credi- tial in shaping public opinion than appeals to
bility of information and information sources. emotion and personal belief.’ Although the
In computer science, numerous are the concept of post-truth has been in existence for
approaches that have been proposed to tackle this the past decade, the editors of the Oxford Dic-
issue. Credibility perceptions are evaluated in terms tionary have observed a spike in frequency in
of multiple characteristics connected to the source of 2016 compared to previous years (an increase
information, the content, and the media across which of about 2000% in the use of the term). This
information diffuses. Most of the state-of-the-art term has been particularly employed in the con-
approaches discussed in this article are based on text of the EU referendum on ‘Brexit’ in the
data-driven models, i.e., they employ machine- United Kingdom, and in the context of the
learning techniques that classify pieces and/or sources presidential election in the United States.
of information as credible and not credible. Recently, Social Media are today seen as a long-equal
also model-driven approaches based on Multicriteria and democratic instrument for exchange and
Decision Making (MCDM) are emerging, which sharing of information; despite this, they can
focus on aggregation schemes to assess an overall turn into palaces of mirrors in which everyone
credibility estimate. Finally, graph-based approaches seeks and finds possible confirmations to their
opinions and reflections. This way, people see
exploiting the social structure of connected entities
only themselves, their anger and discomfort,
have been recently investigated; they mainly focus on
without a critical verification of the posted con-
the concepts of credibility and/or trust propagation.
tent. In an interesting article in the Washington
Since the Social Web is characterized by multiple
Post, the echo-chamber phenomenon on Social
social applications, which have been developed to Media1 is perfectly described: there is no truth
address different aims and in different contexts, the of the facts, because each has selected and
solutions that have been proposed to address the cred- receives only the news and comments with
ibility assessment issue in Social Media change which he or she agrees in advance. In the near
depending on the considered scenario. Three of the future it will be more and more crucial for both
main tasks that have tackled this issue are: (1) the traditional and Social Media to address the
detection of opinion spam in review sites, (2) the credibility assessment issue, facing the growing
detection of fake news and spam in microblogging, importance and weight of the phenomenon
and (3) the credibility assessment of online health formed by the swirling spread of misinforma-
information. In this article, we provide a review of the tion, with the subsequent advent of the post-
main and more significant approaches that have been truth.
designed to address the above tasks, by discussing
their pros and cons. Despite the high number of inter-
esting proposed solutions, some open issues still con- been in turn associated with believability, trustwor-
cern the problem of the absence of predefined thiness, perceived reliability, expertise, accuracy, and
benchmarks and gold standard datasets, and the prob- with numerous other concepts or combinations of
lem of collecting and mining large amount of data, them.2 In particular, the attention raised by both
which has not yet received the attention it deserves. credibility and credibility assessment has gradually
moved from traditional communication environ-
ments, characterized by interpersonal and persuasive
communication, to mass communication and interac-
CREDIBILITY
tive mediated communication, with particular refer-
In the field of communication, credibility is one of ence to online communication.2,3 In this section, we
the oldest concepts that have been investigated. Over first provide a brief historical background on the
the years, depending on the context, credibility has concept of credibility; then we illustrate its
2 of 25 © 2017 John Wiley & Sons, Ltd

WIREs Data Mining and Knowledge Discovery Credibility in social media
application to computer science, with particular ref- accurate, reputable, competent, and current. In other
erence to online information. We present the open words, a source is trustworthy for being honest, care-
issues connected to credibility assessment in UGC, ful in choice of words, and disinclined to deceive.11
where information is generated and shared at a high Information is trustworthy when it appears to be reli-
rate and big volumes through Social Media. able, unbiased, and fair.12 When considering these
characteristics, the impact of the delivery medium
can change the perception that people have about
An Overview on Credibility sources of information and information itself. Some-
One of the oldest sources involving the concept of times, depending on the context, the medium diffus-
credibility is the Phaedrus (around 370 BC) by Plato, ing information can be misleadingly interpreted as
a dialogue between Socrates, the protagonist, and the source of information. Thus, one important ques-
Phaedrus, his interlocutor. In this work, Plato tackled tion to be tackled nowadays is whether new media
one aspect of the concept of credibility of the infor- introduce new factors into credibility assessment.13
mation source, i.e., the speaker’s knowledge of truth In particular, according to Metzger and Flanagin,14 a
(or at least the resemblance of truth offered by the great attention must be paid in conceptualizing credi-
speaker to the audience). Later on, Aristotle in his bility in the digital realm. This issue will be discussed
Rhetoric (fourth century BC) identified another in detail in the next section.
group of source credibility characteristics, connected
to the ethos—Greek term for ‘character’—of the
communicator. Coming to the modern era, since the From Offline to Online Credibility
1950s researchers in the fields of psychology and As previously illustrated, the problem of assessing
communication have been interested in defining and information credibility has a long history and
studying source credibility. The research undertaken involves different research fields; therefore, it does
by Hovland and colleagues4–6 constitutes the first not refer only to online accessible information. In the
systematic work about credibility and mass media. ‘offline’ world, long before the birth and spread of
The authors studied in particular the credibility of the Web, users have always been confronted with the
mass communication messages, by investigating how problem of trusting information obtained via differ-
people evaluated these messages coming from ‘high ent kinds of media. Although it has been shown that
credibility’ and ‘low credibility’ sources. From the the cognitive skills that people need to assess infor-
results of their studies it emerged that, even if the mation credibility are not dependent on the scenario
audience was more influenced by ‘high credibility’ (i.e., offline or online),15 these skills often depend on
sources, the information was equally learned from the way in which people get this information. In real
both source types. In fact, as illustrated by Fogg and life, people usually deal with sources of information
Tseng,7 credibility is a perceived quality of the infor- that can be (1) organization-oriented, i.e., provided
mation receiver, and it is composed of multiple by well known organizations such as reputed news-
dimensions. In this sense, the process of assessing papers or popular enterprises; (2) independent,
perceived credibility involves different characteristics, i.e., provided by third parties such as no profit orga-
which can be connected to: (1) the source of the mes- nizations or individuals considered as experts in a
sage, (2) the message itself, i.e., its structure and its given field; (3) interpersonal, i.e., based on direct
content, and (3) the media used to diffuse the communication and knowledge among individuals.
message.8 In these three cases, a common practice by which
Most researchers agree that there are at least people have traditionally reduced uncertainty about
two key dimensions connected to (perceived) credibil- credibility include judgment based on both the repu-
ity: expertise and trustworthiness.6,9 With particular tation of the source of information or of traditional
reference to source credibility, expertise is ‘the per- information intermediaries such as experts and/or
ceived knowledge, skill, and experience of the opinion leaders—cases (1) and (2), and the personal
source.’7 It allows people to measure to which extent trust based on first-hand experiences with the infor-
a communicator is capable of making correct state- mation providers—case (3).
ments. Trustworthiness refers to which degree an Since nowadays a huge amount of information
audience perceives the statement made by a commu- is distributed online, and the Web represents for
nicator to be valid.6 This aspect is strictly connected many people the primary means to stay informed,
to message credibility, which allows to measure, several of these traditional intermediaries have been
according to Liu,10 to what extent users think infor- removed through a process of ‘disintermediation.’16
mation (i.e., the message) is truthful, unbiased, In this way, according to Flanagin and Metzger,15

digital media ‘have in many ways shifted the burden reason, there is nowadays the need of developing
of information evaluation from professional gate- interfaces, tools or systems that are designed to help
keepers to individual information consumers.’ Thus, users in automatically or semi-automatically assess
the assessment of credibility in the online environ- information credibility. In the next section, we illus-
ment is often more complex with respect to previous trate the approaches that have been proposed so far
media contexts, for a number of different reasons. A to assess credible information in the Social Web.
first issue is connected to ‘the multiplicity of sources
embedded in the numerous layers of online dissemi-
nation of content,’17 and to the fact that these APPROACHES TO CREDIBILITY
sources and connected information messages become ASSESSMENT
confused in information seekers’ minds almost imme- Social Media refers to a plethora of different Web
diately after having performed online search.18 Other 2.0 Internet-based applications, which take the form
issues concern the absence of standards for informa- of blogs, social networks, forums, microblogs, photo
tion quality, the ease in manipulating and altering sharing, video sharing, review sites, and so
the information, the lack of clarity of the context, on. Depending on both the context and the aim to
and the presence of many potential targets of credi- which a given social application is developed, several
bility evaluation interacting together in users’ percep- different characteristics can concur to the assessment
tions, i.e., the content, the source, the medium.14 of information credibility. In this literature survey,
we review in particular the main approaches that
have been proposed for: (1) opinion spam detection,
Credibility and Social Media with particular reference to fake review detection in
The previously described issues to assess credibility review sites, (2) fake news and spam detection in
are exacerbated in the Web 2.0 context, where users microblogging sites, and (3) credibility assessment of
are encouraged to interact and collaborate with each online health information, in particular in healthcare
other through Social Media, by playing the role of social forums.
creators of UGC in a virtual community.19 In this
context, personal knowledge is replaced by virtual
relationships, through which it is easier and inexpen- Opinion Spam Detection in Review Sites
sive for users to directly exchange information. Many The first studies on opinion spam detection were pro-
studies in behavioral psychology have shown that posed by Jindal and colleagues in the late 2000s,28–30
people have an innate tendency to share information in conjunction with the spread of both opinion-
in virtual communities, whether or not explicit sharing sites and the first social platforms allowing
benefits are involved.20–22 In such a scenario, the users to diffuse their opinions in the form of reviews
process of evaluating the most credible information concerning given entities, e.g., products, restaurants,
differs both from traditional media and Web 1.0 services, and so forth. Opinion spam covers mali-
technologies. cious activities attempting to mislead either human
Evaluating information credibility in the Social readers, or automated opinion mining and sentiment
Web deals with the analysis of both UGC and their analysis systems. These fraudulent activities concern
authors’ characteristics,23 and to the intrinsic nature both the formulation of undeserving positive opi-
of Social Media platforms.24 Generally speaking, this nions to promote some target entities, and the provi-
means to take into account characteristics of credibil- sion of false-negative opinions to some other entities,
ity both connected to information (i.e., UGC) and to in order to damage their reputation. In this context,
information sources, as well as to the social relation- opinion spam can have different forms; in most cases,
ships connecting the involved entities. In this context, it concerns fake reviews, but it can also be in the
the users’ credibility perceptions are usually based on form of fake comments, fake blog and social network
crowd consensus, which is typical in interacting postings, and deceptive messages.
communities.25,26 This section focuses in particular on fake
Even if credibility is a characteristic perceived review detection in review sites (e.g., TripAdvisor,
by individuals, credibility assessments should not be Yelp, Amazon, etc.); in last years, this task has
up to users, especially in the online environment.3 In received a growing attention within the scientific
fact, humans have only limited cognitive capacities to community, and several approaches have been pro-
effectively evaluate the information they receive, posed to date. In fact, online reviews represent one of
especially in situations where the complexity of the the consumer’s primary factors in choosing a prod-
features to be taken into account increases.27 For this uct/service, and there is an increasing interest of

companies in customers’ online opinions. This can, In the second case, we can deal with characteristics
in fact, help companies to reshape their business by connected to the behavior of the reviewer, i.e., behav-
adapting products, services, and marketing to con- ioral features, and with features connected to the
sumer preferences.31 For this reason, fake reviews user’s profile (if available). In addition, even if they
can damage both the buyer, whose choice is guided have been less used in the literature so far, other fea-
to nonoptimal decisions, and the business, which not tures connected to (3) the network structure repre-
only looses consumer’s trust, but also risks to misin- sented by the relationships among users or other
terpret users’ needs. entities can be taken into account; they can be
A well-known spam review categorization has defined as social features. Finally, another category
been provided by Jindal and Liu,30 according to whom of less used characteristics includes product features,
there are generally three main types of fake reviews: directly linked to (4) the product/service reviewed.
While fake review detection approaches are based in
• Untruthful reviews: deliberately providing particular on linguistic, behavioral, meta-data and
undeserving positive or negative reviews to product features, behavioral, and social features are
some entities in order to promote or to scuttle nowadays used in most cases to detect spammers and
them, misleading readers or opinion mining group spammers.
systems; Two literature surveys on opinion spam detec-
tion in review sites have been recently published.31,32
• Reviews on brands: considered as spam because
Aim of this section is to describe, among the
they focus specifically on the brands, the manu-
approaches that have been well presented and sum-
facturers or the sellers of the products, by disre-
marized in the above surveys, the ones that have
garding the specific products or services;
demonstrated to be the more effective. In addition,
• Nonreviews: occurring in the form of advertise- we provide an overview of the most recent
ments or other kinds of posts that do not con- approaches, and with respect to previous surveys we
tain opinions (e.g., questions, answers, and organize the presentation of the state of the art by
random texts). focusing on the aim for which the approaches ana-
lyzed have been proposed: (1) fake review detection,
The majority of the approaches proposed in the liter- and (2) spammers/group spammers detection. Within
ature for fake review detection consider the first cate- each of the above classes of approaches, we focus on
gory of reviews (i.e., untruthful reviews), for two the features that they employ.
main reasons32: (1) they undermine the integrity of
the online review system or site in exam; (2) it is Fake Review Detection
almost impossible to distinguish between fake and As previously outlined, fake review detection con-
genuine reviews by only reading their content. There- cerns, in most cases, the identification of untruthful
fore, to build automatic or semi-automatic methods reviews within an opinion-sharing system. Based on
for the recognition of untruthful reviews, both data- the features on which the proposed approaches
driven and model-driven approaches have been pro- focus, we categorize them into purely content-based
posed and developed, based on data mining and and multi-feature-based approaches. Content-based
machine-learning techniques, and MCDM, respec- approaches are based on linguistic features, while
tively. Besides the approaches that directly identify multi-feature-based approaches employ multiple kinds
fake reviews, some other approaches have been pro- of features among those previously described. The fol-
posed to the aim of capturing spammers and group lowing subsections summarize the approaches in each
spammers who spread untruthful reviews in opinion- of the above categories.
sharing platforms.
Whatever the purpose, to directly identify false Content-Based Approaches
reviews or users that spread them, both strategies Several approaches proposed in the literature are
usually consider (simultaneously or not) a certain based exclusively on features extracted from the text
number of characteristics, namely features, that can of reviews. Different linguistic features can be consid-
be evaluated in terms of credibility. They can be ered; the simplest ones are either individual words or
extracted from both (1) the review and (2) the n-grams, (i.e., groups of n words), which can be
reviewer. In the former case, it is possible to deal either weighted or unweighted.30,33 For a complete
with linguistic features, associated with the content overview on the multiple kinds of linguistic features
of the review, and meta-data features, i.e., additional that have been proposed in the literature, refer to
information connected to the textual information. Crawford et al.32 The majority of purely content-

based approaches are based on supervised learning et al.44 employ this content-based representation to
techniques.34–37 ascertain possible ‘concept substitutions’ in untruth-
Ott et al.34 use three groups of features to train ful reviews: if the semantic content of a review is
Nave Bayes (NB) and support vector machine (SVM) mainly generated from another review, this suggests
classification algorithms. For each review, the that both reviews may not truly reflect writers’ opi-
authors define linguistic features that are based on nions. To evaluate the likelihood that a review has
the frequencies of each parts-of-speech (POS) tag,38 been generated by another one, the Kullback–Leibler
and they test differences in POS distribution between divergence45 is employed. To convert this measure to
fake and nonfake reviews based on the genre of a spam score for each review, the authors us a linear
reviews. In addition, they consider psycholinguistic normalization method. Finally, they employ an
deception detection by using the Linguistic Inquiry SVM-base method to classify spam and genuine
and Word Count (LIWC) software,39 which assigns reviews, by obtaining a declared 81% precision
80 psycholinguistic meanings to 4500 keywords. (it should be noticed that the precision of the Jindal
Finally, they consider ‘classical’ n-gram feature sets, and Liu method,30 which employs multiple kinds of
i.e., unigrams, bigrams, and trigrams. The evaluation features, is 85%). Lau et al.46 extend the previous
of this approach is based on a gold standard dataset work by proposing a semantic language model
of genuine reviews collected from review sites and (SLM)-based approach. By using both the semantic
fake reviews generated by means of the Amazon relationships captured from WordNet, and high-
Mechanical Turk (AMT). With respect to considering order concept associations discovered via the pro-
only simple n-gram features, this approach achieves posed text mining method, the SLM method detects
better results, even if it does not outperform models duplicate untruthful reviews even if their wordings
considering multiple features of different nature are not the same. The proposed model achieves a
(i.e., in addition to linguistic ones).30 In a later declared true positive rate of over 95% in fake
work,35 the same authors focus on the optimization review detection. Both approaches previously
of the supervised classification method, obtaining described44,46 employ as a gold standard a real-world
slightly better result with respect to their previous dataset collected from Amazon, and manually labeled
approach. On the same dataset, Banerjee and Chua36 by human experts.
use logistic regression (LR) to build a classification Generating gold standard datasets for evalua-
model by considering the readability of a review tions is a complex task in the fake review detection
(i.e., its complexity and reading difficulty), the distri- context.47 In fact, as it emerges from the presentation
bution of POS tags, and the review writing style, of the approaches illustrated so far, current methods
i.e., positive cues, perceptual words, and usage of of collecting reviews labeled with respect to credibil-
future tense. Fusilier et al.37 use both character n- ity are neither effective (AMT) nor practical (when
grams and word n-grams obtaining the best results human intervention is required). Therefore, in the
with character n-grams with values for n of 4 and absence of large amounts of labeled examples of
5, respectively, by using NB and an SVM classifier deceptive and truthful opinions, some approaches
on the Ott et al.’s dataset.34 have proposed the use of semi-supervised learning
It is worth to be noticed that Mukherjee techniques, by considering only a few examples of
et al.40,41 and Sun et al.42 observed that using the deceptive opinions and a set of unlabeled data. Spe-
AMT for producing fake reviews causes inaccurate cifically, these methods are based on the so-called
results in review spam detection, because fake positive-unlabeled (PU) learning.48 Hereafter, we
reviews generated ad hoc (e.g., by means of AMT) describe those PU-based methods acting on linguistic
are not meaningful in spam detection techniques features. Ren et al.’s research work on fake review
when compared with real reviews collected from detection49 assume the presence of some differences
opinion-sharing sites. In other words, the employed in the language structure and sentiment polarity
dataset is biased because the motivations and the between deceptive reviews and truthful ones.50 The
psychological states of mind of people involved in authors identify a set of features related to the text of
the AMT crowdsourcing activity can be quite differ- the review and combine two clustering algorithms to
ent from those of the professional spammers in the identify deceptive reviews. PU learning is implemen-
real world. This explains the good results obtained ted in a subsequent approach: MPIPUL,49 which
by previous approaches in fake review detection identifies deceptive reviews based on both Latent
based only on linguistic features. Dirichlet Allocation (LDA)51 and SVM. Experiments
Other works propose the use of language mod- are performed on a gold standard dataset obtained
els43 to represent the textual content of reviews. Lai from the ATM dataset by Ott et al.,34 but the

approach is only evaluated against other PU algo- (described by Ott et al.34), and a real-world review
rithms, and not w.r.t. state-of-the-art fake review dataset (where reviews are automatically classified
detection approaches. A similar approach has been and labeled by Yelp in recommended and-not recom-
recently proposed by Fusilier et al.52; with respect to mended). The obtained results are similar to those
the original PU learning algorithm proposed by Liu reported in previous studies: using only n-gram fea-
et al.53 to classify text documents, this approach is tures performs well on the AMT dataset, but when
more conservative when selecting the negative exam- used on a real world dataset (i.e., Yelp in this case)
ples (i.e., not deceptive opinions) from the the approach performs significantly worse. Once
unlabeled ones. again, this proves the intrinsic bias in the ground
truth constructed by means of the AMT.
Multiple-Feature-Based Approaches Li et al.55 propose an approach constituted by
In the literature, the use of multiple features of differ- two phases. First, the authors investigate the impact
ent nature has proven to be a more effective method that review- and reviewer-based features have on
than those based on a unique feature type fake review detection. The supervised NB method
(i.e., linguistic features). In fact, experienced review employed for this task achieves, as expected, signifi-
spammers write so good fake reviews to make them cant improvement compared with heuristic methods;
indistinguishable from truthful reviews, even when it has been tested on a small manually labeled dataset
they are read by experts in review spam from Epinions. To boost the performance of the
detection.31,54 approach, by considering a larger number of unla-
In the first research works on opinion spam beled data, the authors employ a semi-supervised
detection,29,30 this aspect was already clear. In their cotraining algorithm, which demonstrated its effec-
2008 work,30 Jindal and Liu proposed an approach tiveness with respect to supervised NB. Another
to detect duplicate and near-duplicate reviews, by semi-supervised approach for fake review detection
observing that fake reviews are often duplicates of a has been proposed by Li et al.56; in their model, the
small number of reviews created by spammers, which authors take into consideration a subset of the fea-
are used as templates to generate false information tures proposed by the previously described
on different products or services. To estimate the approaches, and implement a PU learning algorithm
similarity between reviews and to obtain duplication to classify Chinese reviews. The review dataset is
scores, the authors propose to use bigrams and the extracted from the review hosting site dianping.com,
Jaccard Index. By considering heterogeneous features which is the Chinese equivalent of Yelp. Even if it
(i.e., connected to the review and its content, to the does not propose any automatic approach for opin-
reviewer, and to the reviewed product), this approach ion spam detection in review sites, the work by Luca
applies LR to detect spam reviews on a manually and Zervas57 is interesting in the sense that it ana-
labeled dataset composed of 470 fake reviews from lyzes multiple kinds of features as well as the motiva-
Amazon, identified by means of their computed tions (often based on economic incentives) connected
duplication scores. This method has been evaluated to the process of issuing fake reviews. The awareness
by means of the area under ROC curve (AUC), a of the importance of features in terms of credibility
standard measure used in machine learning for asses- can lead to the development of model-driven
sing the model quality, obtaining a 78% AUC value approaches instead of data-driven ones.
by considering hybrid features with respect to the Recently, Viviani and Pasi58 have proposed a
63% AUC value obtained by using only textual fea- MCDM approach for fake review detection, where
tures. A study by Mukherjee et al.41 found that when multiple kinds of features, both review- and
considering abnormal behavioral features of reviewer-based are considered and aggregated by
reviewers, the obtained results are better than those suitable functions to obtain an overall credibility esti-
obtained by using only the linguistic features of the mate. This method is based in particular on quanti-
reviews. The authors consider for example in their fier guided aggregations of credibility scores
approach the maximum number of reviews written associated with the considered features: at least k or
by a customer, the amount of positive reviews with at least k important features must characterize a
respect to negative ones, the review length, the review to classify it as true rather than fake. The
reviewer deviation in assigning ratings with respect labeled dataset used for evaluation purposes has been
to the general rating consensus, and the content simi- crawled from Yelp, as in the work by Mukherjee
larity. Supervised learning approaches for deceptive et al.41 The outcome of this work is that an MCDM-
review detection are defined and evaluated, by using based approach can achieve the same results
both the AMT (AMT) fake reviews dataset obtained with the best machine-learning classifiers.

Spammer Detection group, group size, etc.). By considering these features,

With respect to fake review detection, the approaches it is possible to provide an effective classification of
that have been proposed for spammer and group reviewers into spam groups and nonspam groups. In
spammers detection are more recent. In most cases, this work, multiple experts were employed to create
they focus on behavioral and social features, and a labeled group opinion spammer dataset, consisting
nowadays they are increasingly exploiting the graph- of 2412 labeled spam and nonspam groups.
based structure that represents the relationships
among various entities (e.g., reviews, reviewers,
stores, and products). Graph-Based Approaches
These approaches exploit the graph-based nature of
the relationships that can occur between entities to
Behavioral-Based Approaches be evaluated in opinion spam/spammer detection.
The approach by Lim et al.59 involves the use of The first graph-based methods to detect review spam-
behavioral heuristics to detect spammers, for exam- mers were proposed by Wang et al.66,67 These
ple their attitude to target specific products or prod- approaches consider three entities as nodes of the
uct groups in order to maximize their impact, and graph: the review, the reviewer and the store (i.e., the
the fact that spammers deviate from the other reviewed entity). The authors explore how the inter-
reviewers in their ratings of products. Considering actions between nodes in the graph can be used to
these aspects, the authors propose a scoring method identify spam, and propose an iterative computation
for ranking reviewers by implementing some mea- model to identify suspicious reviewers. They obtain
sures that capture spamming behaviors on an Ama- data for experiments by crawling resellerratings.com,
zon review dataset. A subset of highly suspicious one of the largest hosts of store reviews. First, they
reviewers is then identified by human experts, and a let the proposed algorithms identify highly suspicious
regression model is employed to automatically score spammer candidates; then, human judges examine
reviewers. The authors found that deleting reviews these candidates to decide how many reviewers are
written by suspicious reviewers has a high impact on indeed suspicious. The evaluation of this approach
the rating of the target products. As pointed out by produces both good precision and good human eva-
Heydari et al.,31 one of the possible issues of this luators agreement.
approach is that, by considering the rating deviation The approach described by Akoglu et al.68 pro-
of a reviewer with respect to the product average poses a framework, FraudEagle, for detecting spam-
rank as an evidence of anomaly, there is the risk to mers and fake reviews in online review sites. This
ignore that this could simply represent a different framework exploits the connectivity structure linking
personal experience of the reviewer with respect to users, products, and reviews, by observing that spam-
others. Fei et al.60 propose the idea of measuring the mers are usually linked to good/bad products with
‘burstiness’ that characterizes reviews generated by negative/positive fake reviews, and vice versa. For
spammers to identify fake reviews. Bursts are reviews this reason, it is possible to ‘sign’ with sentiment the
that become very popular and receive great attention network edges, and to apply a signed inference algo-
in a given time period and/or in a certain area. Due rithm extending LBP to infer the class labels of users,
to their rapid diffusion, they become suspicious as products, and reviews. However, with respect to
well as their authors. A kernel density estimation prior work,67 it does not improve the spam/spammer
(KDE)61 technique is used to detect review bursts, detection accuracy. In a recent work, Ye and Ako-
and both behavioral features for spammers and fea- glu69 propose GroupStrainer, a two-step method to
tures extracted from review bursts are considered. To discover group spammers and their targeted pro-
identify in an automatic way spam reviewers, the ducts. The considered review network is a bipartite
authors employ a Markov random field (MRF)62,63 graph consisting of reviewer nodes connected to
model coupled with a loopy belief propagation product nodes through review relations. The authors
(LBP)64 algorithm, on the Jindal et al.’s30 dataset. define a Network Footprint Score (NFS) measure to
Mukherjee et al.65 consider the particular case quantify the likelihood of products of being spam
of spammers who act with other spammers to targets. Then, a clustering phase identify highly simi-
achieve a given goal. They present Group Spam lar spammers (i.e., group spammers) on a 2-hop sub-
Rank (GSRank), an approach for group spammers graph induced by top ranked products (i.e., products
detection, which exploits some behavioral features having the highest NFS values). In the experiments,
connected to groups of spammers (e.g., content simi- the authors compare the proposed method to various
larity between members, content similarity among a graph anomaly detection methods on synthetic

datasets, and they study its performance on two large a subset of both recommended and non-recom-
real-world datasets (Amazon and iTunes). mended reviews automatically labeled by some
Other recent approaches focusing on group review sites, namely Yelp and Dianping.41,56,57 This
spammer detection are proposed by Choo et al.70 constitutes a quite strong assumption, since these
and Wang et al.71 In the former approach, the review sites do not disclose their policy for detecting
authors explore the community structures to distin- nonrecommended (fake) reviews, and even in such a
guish spam communities from nonspam ones with case the filtered reviews would be affected by some
sentiment analysis on user interactions. Through uncertainty related to the applied decision process. It
experiments over a crawled Amazon dataset, the is clear that none of the solutions proposed so far
authors obtain comparable results with respect to can be considered fully reliable, and that new
state-of-the-art approaches, and they find—not approaches to the evaluation of credibility are neces-
surprisingly—that strong positive communities are sary. To tackle the problem of opinion spam detec-
more likely to be opinion group spammers. The latter tion from a different perspective, recent approaches
approach illustrates the ‘loose’ group spam problem, focus on identifying spammers and group spammers
i.e., each group member is not required to review in Social Media, by using both behavioral character-
every target product. This problem is addressed by istics of reviewers and the social graph connecting
using bipartite graph projection. The authors pro- multiple entities (i.e., users, products, and reviews) in
pose a set of group spam indicators to measure the review sites. Table 1 lists the approaches to fake
‘spamicity’ of a loose group of spammers, and they review and spammer detection, by summarizing the
design an algorithm to identify highly suspicious main characteristics of each approach.
loose group spammers in a divide and conquer man-
ner. Experimental results show that the proposed
approach achieves both target goals, i.e., the detec- Fake News Detection in Microblogging
tion of loose group spammers, and the detection of Microblogging is a well-established broadcast
group spammers, with better results with respect to medium that allow users ‘to exchange small elements
frequent itemset mining (FIM)72 approaches. of content such as short sentences, individual images,
or video links.’74 These short messages are com-
Discussion monly referred as microblog posts, microposts, or
Several approaches presented in this section have status updates. Nowadays, thanks to the success of
been defined in the last years to tackle the problem of the microblogging platform Twitter,a they are also
opinion spam detection in review sites. As previously simply referred as tweets. In Social Media classifica-
illustrated, most of the proposed approaches are tion, microblogs stand in fact between traditional
based on supervised or semi-supervised techniques blogs and social networking sites, and they are char-
that make use on multiple kinds of characteristics acterized by both a high degree of self-presentation,
that can be related to credibility. One of the main and a medium to low degree of social presence and
open issues that these approaches do not solve is the media richness.75
lack of a valuable gold standard to train and evaluate This preamble on microblogging is necessary to
the proposed classifiers; in fact, in most cases, pseudo underline that, due to its sharing characteristics and
fake reviews have been employed rather than real to the fact that messages display a wide variety of
fake ones.73 Some approaches consider duplicate content, the credibility assessment of information on
reviews as fakes, not considering that similar review this kind of medium may concern different aspects.
contents might be produced by honest reviewers; As outlined in the literature,76 short messages in
moreover, spammers commonly copy the text of microblogging can correspond both to the so called
truthful reviews to generate fakes.31 The human conversation items (e.g., chats, personal updates, gos-
intervention for labeling reviews can be inapplicable sips, etc.), which concern the user and its circle of
to big amount of data, and it is debatable as also the friends, and to news items, representing more general
reliability of the human assessors may be questioned. and relevant information.77–79 This latter kind of
Even the usage of crowdsourcing platforms such as information can influence a broader community, in
the AMT for producing fake reviews has been proved particular when microblogs deal with the well known
unreliable, due to the fact that the motivations and trending topics, which can be considered ‘headline
the psychological states of mind of people using the news or persistent news.’80 In this context, Rubin
AMT can be quite different from those of profes- et al.81 identify three types of fake news: exposed
sional spammers in the real world.40,42 Recent fraudulent journalistic writing, humorous fakes, and
approaches have proposed to use as a gold standard large-scale hoaxes. This latter category is the one that

TABLE 1 | Summarization of the Proposed Approaches for Opinion Spam Detection in Review Sites
Authors Year Focus Approach
30
Jindal and Liu 2008 Fake review detection Multifeature-based
Lai et al.44 2010 Fake review detection Content-based
59
Lim et al. 2010 Spammer detection Behavioral-based
Li et al.55 2011 Fake review detection Multifeature-based
Ott et al.34 2011 Fake review detection Content-based
Wang et al.66 2011 Fake review/Spammer detection Graph-based
Lau et al.46 2012 Fake review detection Content-based
65
Mukherjee et al. 2012 Spammer/Group spammer detection Behavioral-based
Wang et al.67 2012 Fake review/Spammer detection Graph-based
68
Akoglu et al. 2013 Fake review/Spammer detection Graph-based
Fei et al.60 2013 Fake review/Spammer detection Behavioral-based
41
Mukherjee et al. 2013 Fake review detection Multifeature-based
Banerjee and Chua36 2014 Fake review detection Content-based
56
Li et al. 2014 Fake review detection Multifeature-based
Ren et al.49 2014 Fake review detection Content-based
Choo et al.70 2015 Group spammer detection Graph-based
37
Fusilier et al. 2015 Fake review detection Content-based
Fusilier et al.52 2015 Fake review detection Content-based
71
Wang et al. 2015 Group spammer detection Graph-based
Ye and Akoglu69 2015 Group spammer detection Graph-based
57
Lucas and Zervas 2016 Feature analysis Content-based
Viviani and Pasi58 2016 Fake review detection Multifeature-based
is better suited to be spread on Social Media like topics, and they classify these topics as credible or
Twitter, and the one that has a higher impact on not credible based on multiple features extracted
society due to the speed with which hoaxes diffuses from them. In particular, the authors identify mes-
in microblogging. sage-based features, focusing on the content of the
In recent years, a number of different message (e.g., length, syntax, sentiment, etc.), user-
approaches have been proposed for assessing credi- based features, connected to the user who posted the
bility in microblogging, by also considering its time- tweet (e.g., number of friends and followers), topic-
sensitive nature. In the literature, these methods can based features, obtained by aggregating the previous
be broadly classified into two categories82,83: classifi- ones (e.g., fraction of positive or negative sentiment),
cation-based approaches, and propagation-based propagation-based features, related to the propaga-
approaches, which exploit the network structure of tion tree built by considering retweets (e.g., depth of
users and tweets. In addition, a number of survey- the retweet tree). This approach is based on super-
based studies have been presented, as it will be dis- vised learning, and in order to assess its effectiveness,
cussed later in this article. the authors have built a suitable dataset by collecting
2500 trending topics via TwitterMonitor,85 keeping
Classification-Based Approaches only those cases containing at most 10,000 tweets
Castillo et al. were among the first to tackle the prob- each. The labeling of the dataset was carried out in
lem of information credibility on microblogging sites, two phases: in a first phase some evaluators
Twitter in particular,79,84 in a structured way, by (by means of the AMT) were requested to assess
using classification-based approaches. In their first whether the collected tweets concerned news about
work,79 the authors focus on automatic methods for specific events or rather they were ‘conversation’
assessing the credibility of a given ‘time-sensitive’ set topics. Then, for the subset of tweets identified as
of tweets, i.e., a trending topic. Specifically, they ana- news, a second group of evaluators were requested to
lyze microblog posts (messages) related to trending label them as credible or not credible. By performing

a threefold validation strategy and by applying differ- each tweet individually, and at predicting user credi-
ent classifiers (i.e., SVM, decision trees, decision bility. Not surprisingly, as already demonstrated in
rules, and Bayesian networks), the authors obtained the previously described work,79 the features that
good results in identifying ‘newsworthy’ topics, while consider the underlying network and the dynamics
using a J48 decision tree86 they reached a 86% accu- connected to information flows are better indicators
racy in credibility classification with respect to a ran- of credibility in microblogs than linguistic features.
dom predictor. In this paper, a feature analysis is Another work that assesses the credibility of
also provided, to illustrate the contribution that dif- individual tweets is the one by Gupta and Kumara-
ferent types of features give in terms of credibility guru.88 In this work, the authors use the SVM rank
assessment. The outcomes of this work are: (1) - algorithm89 (a variant of the SVM algorithm that is
message- and user-based features are not enough to used to solve certain ranking problems via learning
effectively classify credible trending topics; (2) in gen- to rank90), to illustrate that ranking tweets based on
eral credible news are propagated by authors who Twitter features (i.e., content- and user-based) can
previously wrote a considerable amount of messages; help in assessing the credibility of tweets about an
(3) propagation-based features (having many reposts) event. To this purpose, tweets are labeled by human
are particularly effective. Moreover, the authors out- annotators. To evaluate the effectiveness of the pro-
line that tweets not including URLs are in most cases posed approach, the standard normalized discounted
related to noncredible news, while tweets including cumulative gain (NDCG) metric91 is employed. In a
negative sentiment are related to credible content. subsequent work, Gupta et al.92 extend the previous
In another paper, Castillo et al.84 discuss more work and propose TweetCred, a real-time system in
broadly the problem of information credibility in the form of a Chrome extension. TweetCred takes a
microblogging services. By presenting a case study direct stream of tweets as input, and it computes the
about information propagation and news event credi- credibility for each tweet on a scale of 1 (low credi-
bility in Twitter, first they summarize their prior bility) to 7 (high credibility). This research work is
work,79 and then they redesign the learning scheme. based on a semi-supervised ranking model using
A first supervised classifier is used to decide if an SVM Rank for assessing credibility, based on tweets
information cascade corresponds to a ‘newsworthy’ related to six high impact crisis events of 2013. With
event. A second supervised classifier is employed to respect to prior work, in this paper a more exhaus-
decide if this news event has to be considered as cred- tive and comprehensive set of features is used: 45 fea-
ible or not. Both classifiers are trained over labeled tures categorized as tweet meta-data features, tweet
data, obtained using crowdsourcing tools. By intro- content features, user-based features, network fea-
ducing a label named ‘unsure’ to identify tweets that tures, linguistic features, and external resource fea-
do not belong neither to news nor to chat messages, tures. The ground truth was obtained through
these ‘unsure’ tweets are removed from the training crowdsourcing: human assessors labeled around
dataset, thus improving the performance of the 500 tweets randomly selected per event. This system
approach. provides useful insights on how credibility evaluation
Kang et al.87 propose an approach for recom- models evolve over time. As in the case of opinion
mending credible information connected to specific spam detection, recent approaches are focusing on
topics in Twitter. The authors define three models the identification of spammers also in the context of
that focus on different credibility indicators: (1) fea- fake news detection. Galán-Garca et al.93 focus on
tures extracted from the content of tweets related to the detection of troll profiles in Twitter. Their
specific topics (e.g., length of tweet, number of URLs, assumption is that, since ‘every trolling profile is fol-
number of mentions, etc.), (2) features connected to lowed by the real profile of the user behind the trol-
the network-structure connecting users and tweets ling one […] it is possible to link a trolling account
(e.g., ‘follow’ relationships, retweets, etc.), and to the corresponding real profile of the user behind
(3) features from both the previous approaches that the fake account, analyzing different features present
are combined in different ways to predict credible in the profile, connections’ data and tweets’ charac-
information and credible information sources. Each teristics, including text, using machine-learning algo-
model is evaluated by using a J48 tree-based learning rithms.’ Using manually-selected genuine profiles,
algorithm which is trained on a set of tweets that are and collecting at least 100 genuine tweets per profile,
manually annotated with credibility scores. Differ- the authors compared the performance of different
ently from Castillo et al.,79 who propose an algo- classification algorithms (i.e., Random Forests, J48,
rithm that acts on groups of ‘newsworthy’ tweets, the k-Nearest-Neighbor, Sequential Minimal Optimiza-
approach described in this paper aims at classifying tion, and NB) on the selected features: the content of

the tweet published by the user, the time of publica- automatically reporting them highly questioned
tion, the language and geolocation, and the Twitter information.
client. Even if this approach reached ‘only’ 68.47% With respect to Mendoza et al.,95 the approach
of accuracy in its best result, this work constitutes proposed by Seo et al.96 studies how to identify
one of the first significant approaches that try to sources of rumors when there is a limited view on the
directly detect trolls in Social Media and microblogs rumor provenance, and how to determine whether a
in particular, and it has been applied to a real case piece of information is a rumor or not. The method
study, for the identification of the responsible of is based on the assumption that rumors are initiated
cyberbullying in a school in the city of Bilbao from only a small number of sources, whereas truth-
(Spain). Also Gupta and Kaushal94 propose an inte- ful information can be observed and originated by a
grated approach for spammer detection, which com- large number of unrelated individuals. This approach
bines three learning algorithms, i.e., NB, Clustering, relies on the use of network monitors, i.e., indivi-
and Decision Trees, with the aim of improving spam- duals who have heard a particular piece of informa-
mer detection accuracy. The considered features are tion (from their social neighborhood), but who do
followers/followees, URLs, spam words, replies, and not want disclose neither who told it to them, nor
hashtags. The algorithms accuracy in the detection of when they learned it. If a monitor receives a rumor,
non-spammers is about 99.1%, but the accuracy of it is called positive monitor; negative monitor other-
detecting spammers reaches the 68.4%, which is the wise. To find a rumor source, the authors base their
same result obtained by Galán-Garca et al.93 approach on the intuition that the source must be
close to positive monitors and far from negative ones.
Propagation-Based Approaches For this reason, the authors introduce four metrics:
The approaches that focus on the concept of propa- the number of reachable positive monitors, the sum
gation for assessing the credibility of tweets/news, of distances to reachable positive monitors, the num-
usually consider the propagation of rumors (false ber of reachable negative monitors, and the sum of
claims) in microblogs, by exploiting the network distances to reachable negative monitors. By comput-
structure constituted by retweets, and the social ing these metrics for each node, it is possible to sort
graph constituted by followers and followees. Fur- all the nodes in the network; in the resulting sorted
thermore, also trust propagation on the social graph list, the first node is the top suspect source of the
can be assessed. Mendoza et al.95 explore the behav- rumor. In addition, to identify if a piece of informa-
ior of Twitter users under an emergency situation tion is a rumor, the authors propose two greedy stra-
(the 2010 earthquake in Chile); a preliminary study tegies based on the set of monitors that received the
on the dissemination of false rumors and confirmed information. A first strategy tries to assign as many
news is reported. On a crawled dataset containing positive monitors as possible to each source, produ-
tweets and other user-related information, the cing this way large greedy information propagation
approach analyzes the characteristics of the social trees. To solve this problem, a second strategy esti-
network of the community surrounding the topic, mates the disparity between actual propagation trees
and how trending topics propagate. The considered and the ones constructed by the above greedy strat-
characteristics are the relation between the number egy. To evaluate the proposed approach, a case study
of followers/followees, the number of tweets each involving a real social network crawled from Twitter
user posts, and the retweet activity during the first is reported. Using the experimental data, the authors
hours of the emergency situation. From this analysis, evaluate how accurately LR can classify rumor and
it emerges that the network topology characteristics nonrumors. The proposed approach shows good
remain unchanged with respect to normal circum- potential to help users in identifying rumors and their
stances, and that the vocabulary used in critical situa- sources.
tions exhibits a low variance. Concerning credibility, Gupta et al.97 propose an approach for the
and the spread of rumors through the network, the assessment of the credibility of news events on Twit-
authors have manually selected some confirmed ter. Starting from the approach described by Castillo
truths, i.e., reliable news items confirmed by reliable et al.,79 they introduce some new features to improve
sources, and some false rumors, i.e., baseless rumors it. The authors then claim that this classification-
that emerged during the crisis. The outcome of this based approach is neither entity- nor network-aware,
credibility study is that rumors tend to be questioned because events are described by features originally
much more than confirmed news by the Twitter com- related to tweets and users. To address this issue, the
munity; therefore, the authors suggest that micro- authors propose an approach constituted by two
blogs could implement methods to warn people by modules: (1) a Basic Credibility Analyzer, namely

BasicCA, and (2) an Event Graph Optimization the approach evaluates the trustworthiness of each
Credibility Analyzer, namely EventOptCA. BasicCA tweet and each user. To do this, the authors evaluate
acts on a graph constituted by users, tweets, and similarity between features, under the assumption
events. At each iteration, each node shares its credi- that a candidate tweet (and the user who wrote it) is
bility value (learned from a classifier) with its neigh- considered trustworthy if its features do not conflict
bors only. Since the assumption of BasicCA is that with the features of trustworthy news. Then, by
credible entities are strictly connected, every iteration means of four propagation rules defined on the social
helps in mutually enhancing the credibility of genuine graph, the trustworthiness of tweets and users is
entities and reducing the credibility of nongenuine refined and propagated. The evaluation of the
ones, via propagation. EventOptCA enhances similarity-based trust evaluation method is based on
BasicCA by supposing that similar events have simi- two datasets: a manually labeled set, and the dataset
lar credibility scores. Thus, it performs event credibil- provided by Castillo et al.79; both datasets are also
ity updates on a graph of events whose edges are employed to identify emerging events. Based on the
weighted with event similarity values. The experi- resulting precision and f-score values, this method
ments are conducted using 457 news events extracted outperforms classification-based supervised learning
from two tweet feed datasets: a first dataset from approaches.
Castillo et al.,79 and another crawled datases, whose
events are manually labeled as social gossip or news, Survey-Based Studies
using 250 of the news events (of which 167 labeled These studies do not propose algorithms for the auto-
as credible) in their study. On an average, the pro- matic or semi-automatic classification of fake news
posed approach outperforms the classifier-based or spammers in Social Media. They are valuable
approach discussed by the authors. since they employ survey-based studies, i.e., involving
The work just described is taken as a reference a representative sample of people who are asked to
by Jin et al.,83 who first discuss its limitations. In fill out questionnaires and/or to execute some tasks,
fact, the authors believe that it is not always true that to investigate the contribution that credibility factors
credible users provide credible tweets with a high have in the overall process of credibility judgment by
probability; furthermore, they are convinced that users in Twitter and microblogs in general.
considering an event as a whole as only constituted Morris et al.99 present survey results regarding
by one kind of information (i.e., genuine or fake) is users perceptions of tweet credibility. The authors
debatable. For this reason, they propose a hierarchi- conduct some experiments with a limited number of
cal credibility propagation network with three layers: persons involved, where several features are system-
a message layer, a subevent layer and an event layer. atically manipulated to assess their impact on credi-
The assumption behind this choice is that the hierar- bility ratings of tweets. The outcome of this research
chical structure of message to subevent, and subevent work is that the analysis of the content alone does
to event, can model their relations and the process of not allow users to judge tweet credibility; instead,
credibility propagation; furthermore, with a subevent they are more influenced by heuristics such as user
layer, deeper semantic information can be revealed names or tweet topics when making credibility
for an event. Credibility propagation is modeled as a assessments, and less by profile picture images. This
graph optimization problem. To validate the effec- latter result has been recently denied by Kang
tiveness of the proposed model, two datasets on Sina et al.100; in their work, the authors identify the
Weibo (the leading microblogging platform in important cues that contribute to information being
China), were collected: one with random fake news perceived as credible in microblogging, by reporting
in a year and truthful news at the same time; another on a demographic survey followed by a user experi-
with both fake and truthful news related to the same ment with 101 people. This study investigates how
topic. Experiments on both datasets showed the such cues are portable across different microblogs
effectiveness of the proposed model in terms of both platforms, i.e., Twitter and Reddit. The results of this
accuracy and f-score improvements with respect to study show that meta-data and image type elements
baseline methods. are, in general, the strongest influencing factors in
Zhao et al.98 consider the issue of automatically credibility assessment.
estimating the trustworthiness of messages and users The fact that multiple kinds of features consid-
in Twitter for a specific topic domain. In particular, ered together leads to better results in evaluating per-
the authors consider the relationships between users/ ceived credibility has been experimentally confirmed
tweets and multiple characteristics (i.e., textual, spa- by automatic classification-based approaches previ-
tial, and temporal features) connected to them. First, ously described, both for fake news and fake reviews

detection. An analysis of the distribution of the dif- influence that this second category of messages can
ferent types of salient features in Twitter is discussed have on a broader community, the research propo-
by O’Donovan et al.,101 together with the impor- sals have focused on approaches that allow the iden-
tance of considering the context in which these fea- tification of fake news and spammers/group
tures are employed. A theoretical proposal for a spammers diffusing them. The techniques that have
contextual credibility approach has been illustrated been proposed so far do not differ substantially from
by AlMansour et al.82; it aims at examining the those proposed for fake review detection; they focus
effects that culture, topic variations, language, and so on multiple kinds of characteristics connected to
on, have on assessing credibility. Cultural differences users, messages, topics, social graphs, and they apply
and other contextual aspects affecting credibility per- some machine-learning techniques or propagation-
ceptions have been studied by Yang et al.,102 who based approaches to identify pieces or sources of mis-
point out the differences between China and United information. With respect to the opinion spam sce-
States users in rating the credibility of tweets. From nario, it is easier in fake news detection to evaluate
this study, it emerges that Chinese people are appar- the effectiveness of the proposed approaches, since
ently more willing to trust microblogs as information the credibility of news and events can be verified
diffusion media, and to accept as reliable the content (at least a posteriori) by experts and traditional
produced by anonymous or pseudo-anonymous media. Nevertheless, also in this case, in many
sources. approaches the volume of the datasets used for
Differently from the previous ones, the work by experimental evaluations remains rather low.
Sikdar et al.103 illustrates that survey-based methods Table 2 summarizes the research works that have
can be extremely noisy and that results may vary from been proposed so far by illustrating, for each
survey to survey. In large-scale online user surveys, approach, its focus and the category to which it
e.g., AMT, people providing credibility ratings usually belongs.
ignore message senders and other external informa-
tion such as the underlying network context, and they
provide in this way biased credibility ratings. For this Credibility Assessment of Online Health
reason, the authors make a proposal to construct a Information
stable ground truth, collecting two datasets on the According to the Pew Research Centerb
same topic, but from different perspectives. To do this, (a nonpartisan ‘fact tank’ that informs the public
two different surveys are provided to users, in which about the issues, attitudes and trends shaping Amer-
subjects are confronted with different information ica and the world), about 59% of Americans have
about the same tweets. This allows to test to which searched the Web frequently for health information
degree the credibility judgments across different sur- in 2013. In particular, the 35% of U.S. adults
veys are comparable, and how the survey-based declared to have used online health information to
method influences the results. The authors also con- try to figure out their medical condition or that of
sider two different ways to quantify retweets, overall someone they know. In addition to this, the 41% of
and at the time of the message, capturing the impor- the so called ‘online diagnosers’ reported their condi-
tance of the message at two different time granulari- tion confirmed by a clinician.
ties. This proposal represents an interesting Similarly, a recent EU-wide survey104 showed
contribution to the problem of generating a consen- that Europeans too are very sensitive to online infor-
sual and reliable ground truth that, as we have already mation about health and healthy lifestyles. Six of
discussed in the section illustrating opinion spam 10 EU citizens go online when looking for health
detection approaches, remains one of the most signifi- information, 90% of those said that the Web helped
cant obstacles in evaluating the effectiveness of meth- them to improve their knowledge about health-
ods for credibility assessment in different contexts. related topics.
Despite this interest, and the huge amount of
Discussion health information available online, the extent to
As in the case of opinion spam detection in review which people are capable to benefit from this infor-
sites, different approaches have been proposed to mation largely depends on their level of health liter-
identify misinformation in microblogging. In this acy. According to the World Health Organization,105
medium, the messages that are spread between users ‘health literacy represents the cognitive and social
can correspond both to conversation items skills which determine the motivation and the ability
(e.g., chats, personal comments, and gossips) and of individuals to gain access to, understand and use
news items (e.g., trending topics). Due to the information in ways which promote and maintain

TABLE 2 | Summarization of the Proposed Approaches for Fake News Detection in Microblogging
Authors Year Focus Approach
95
Mendoza et al. 2010 Rumor propagation Propagation-based
Castillo et al.79 2011 Trending topic credibility Classification-based
84
Castillo et al. 2012 Trending topic credibility Classification-based
Gupta and Kumaraguru88 2012 Event credibility Classification-based
97
Gupta et al. 2012 Event credibility Propagation-based
Kang et al.87 2012 Tweet and source credibility Classification-based
Morris et al.99 2012 Perceived credibility factors Survey-based
101
O’Donovan et al. 2012 Perceived credibility and context Survey-based
Seo et al.96 2012 Rumor and source credibility Propagation-based
Galán-Garca et al.93 2013 Troll detection Classification-based
Sikdar et al.103 2013 Ground truth construction Survey-based
Yang et al.102 2013 Perceived credibility and context Survey-based
AlMansour et al.82 2014 Perceived credibility and context Survey-based
92
Gupta et al. 2014 Tweet credibility Classification-based
Jin et al.83 2014 Tweet and source credibility Propagation-based
Gupta and Kaushal94 2015 Spammer detection Classification-based
100
Kang et al. 2015 Perceived credibility factors Survey-based
Zhao et al.98 2016 Topic-focused tweet credibility Propagation-based
good health.’ More recently, based on 17 different if online health information is perceived as credible,
definitions identified in the literature, the following people will invest more time in searching and using
definition has been provided by Sørensen et al.106: different types of online health-related services,
‘Health literacy is linked to literacy and entails peo- avoiding to use a significant amount of cognitive
ple’s knowledge, motivation and competencies to efforts and knowledge that would represent a barrier
access, understand, appraise, and apply health infor- for health information usage.110
mation in order to make judgments and take deci- In the last years, the environments in which
sions in everyday life concerning healthcare, disease people have searched for and/or consumed online
prevention and health promotion to maintain or information related to health has dramatically chan-
improve quality of life during the life course.’ This ged. From traditional Web search engines and spe-
definition is more focused on the individual with cialized healthcare platforms, users has moved to
respect to the previous one, which is more centered Social Media platforms,111,112 appreciating the fact
on the public health perspective. In fact, over the that in this kind of media it is possible to exchange
years, a socially contextualized view of individuals personal experience about their own medical or
has emerged, and the health promotion has shifted health-related issues. In this changing context, it is
from simply encouraging people to adopt healthy necessary to briefly recall that credibility has tradi-
behaviors and avoid unhealthy ones, to have more tionally been defined as the believability of informa-
control over health, as single individuals and tion messages and sources, as perceived by the
communities.107 information receiver.113 Thus, depending on the envi-
Being able of assessing the reliability, validity, ronment, the way in which users can perceive and
and in general the credibility of health information evaluate both the health source and message credibil-
can be considered one of the aspects connected to ity changes. Before the advent of technologies related
health literacy.108 As it may be easily guessed, the to the Web 2.0, the focus of research studies in this
health information that individuals find online is field was mainly on the investigation of the credibility
likely used to assess and diagnose personal diseases; of Websites spreading medical and health-related
for this reason, inaccurate or misleading medical information. Nowadays, with the rapid diffusion of
information would cause negative and harmful con- Social Media and the increasing volume of health
sequences for users’ health.16,109 In addition to this, information shared in these applications, new open

issues and a new (and very little explored) research of health-related information credibility in Social
field have emerged. Media, where the characteristics of the medium for
information sharing have a lower impact in the per-
Health Information Credibility in Websites ceived credibility assessment, especially with respect
The study of the reliability of healthcare Websites to the information source.120
and the credibility of the information they contain
has been investigated especially between 2000 and Health Information Credibility in Social Media
2010, and it has concerned both patients and the In general, individuals are inclined to participate in
health sector.114,115 In particular, the content of Social Media to share their stories, experience, and
health-related Websites is potentially unsafe if its knowledge, since social interactions offer them a
quality has not been properly addressed.116 sense of satisfaction to be a member of a commu-
Over the years, researchers have demonstrated nity.121 This is even more true in such a special and
that there are several Website features that can personal context like the one represented by online
boost the perceived credibility of information in the communities related to health. These communities
healthcare context, such as a professional-appearing can either be small or large groups of individuals
interface design, the ease of navigation, the presence who share the same health problems, as well as pro-
of endorsements, etc.113 In their systematic review fessional healthcare services allowing people to inter-
of health Website credibility evaluation,117 Eysen- act online.26 This represents for users the possibility
bach et al. show that, among the characteristics that to have an immediate satisfaction of their needs,
influence credibility, there are: accuracy, i.e., the since they can share their knowledge and symptoms;
degree of concordance of the information provided users can post/read reviews about health products,
with generally accepted medical practice, complete- medicines, and doctors, thus forming online support
ness, i.e., the portion of topics covered by the Web- groups or self-help groups. Social Media also consti-
site, readability, and design, i.e., the visual tutes a valuable opportunity for the health sector to
appearance. The article by Freeman and Spyrida- improve services.25 In fact, using Social Media,
kis109 is one of the first studies summarizing and health care providers can post health information not
analyzing in a complete way the factors influencing only as text, but also in various accessible forms,
the credibility of information in health-related Web- such as images and videos. This way, Social Media
sites, and the review works by Metzger,3 and Rieh empowers health care consumers,122 by providing
and Danielson118 provide a more general view on them with immediate access to an incredible amount
Website credibility, discussing at the same time some of health information and a variety of perspectives
health-related issues. on health topics,112 which they would never have
In a more recent study, Rains and Donnerstein been able to access before, since owned by health
Karmikel119 examine the relationship between Web- care providers.
sites’ structural features and message characteristics, It is immediately clear to the reader how these
in terms of perceived credibility. In addition, in benefits also bring with them a particularly important
assessing Website credibility, they study the impact issue. The direct access to medical information and
of individuals’ navigation orientation, i.e., searching the disappearance of experienced intermediaries,
or surfing. As already proved by previous state-of- makes the problem of verifying the credibility of the
the-art approaches, the authors show that structural obtained information a key factor in the healthcare
features (e.g., a street address, privacy policy state- scenario. Here, more than in other environments, a
ments, third-party endorsements, and the inclusion careful evaluator is necessary to affirm that the
of a navigation menu or images) are effectively posted information is trustworthy, and not danger-
related to perceptions of Website credibility. Fur- ous for the patient.16 Despite the importance of this
thermore, they illustrate how characteristics con- problem, and the fact that it has been previously
nected to the message (e.g., statistics, quotes, and addressed in many research fields such as psychology
identification of authorship) are positively associ- and social sciences,115,123–125 few are the approaches
ated with the evaluation of a given health topic in that have been so far proposed in computer science
terms of credibility. An interesting outcome of this to assess, automatically or semi-automatically, the
work is that these characteristics may have an influ- credibility of health-related information in Social
ence beyond the particular Website on which they Media. With respect to healthcare Websites, where
are presented, and may be extended to individuals’ erroneous comments can be corrected by third
credibility perceptions about a health topic in gen- parties,126 and some structural features can be posi-
eral. This represents an interesting link to the study tively connected to the credibility of the Website

itself,119 online communities are characterized by dif- online user-generated health information. The aim is
ferent features and strategies connected to the evalua- to synthesize the findings of past research about the
tion of information credibility. It is possible, for source and content credibility relationship. As also
example, to provide ratings with respect to pieces illustrated in the above studies,128,129 what emerges
and sources of information, as well as to analyze fea- is that the original sources of online health informa-
tures connected to both user profiles and other tion generally do not have a great impact on the per-
related information,26 and to the UGC. ceived information credibility, especially in Social
The paper by Oh et al.127 investigates the issue Media. In fact, when the health-related content is
of quality of online health answers in social question generated by laypersons, it maintains the potential of
answering (QA) systems, considering also source being perceived as highly credible. This does not hap-
credibility. This paper presents a preliminary user pen in traditional Websites, where only the health
study involving 40 participants who have rated information created by experts is perceived as highly
400 health answers each with a 5-point Likert scale credible. This means that in the social-oriented health
according to 10 evaluation criteria. The paper high- context, individuals trust the content generated by
lights the positive role that experts play in helping laypersons, due to a particular characteristic of this
users and patients to find reliable health information kind of environments, i.e., homophily.132 When the
in Social Media. Another preliminary work by Syn source has low credibility, homophily influences the
and Kim128 addresses the impact of information credibility perceptions that users have on informa-
sources on users’ credibility perceptions. The paper tion. As seen so far, several are the works in the liter-
discusses in particular the way in which credibility ature that have made an analysis of the factors that
affects, in Facebook, the activities related to informa- influence the perception of health-related information
tion generation and usage by users. According to the credibility in Social Media. However, few are the
authors, these activities include the actions that peo- approaches that have been proposed up to now to
ple do to fill the gaps between needs and solutions in assess the credibility of health information in an
problematic situations connected to health, automatic or semi-automatic way. Abbasi et al.133
e.g., search and retrieval, browsing, monitoring, and propose a focused crawling method, ssmCredible, to
so on. The approach is based on an online survey collect credible medical content from Websites, for-
study, involving a limited number of people. The ums, blogs, and social networking sites, by consider-
findings of this work include that, although users ing source and sentiment features pertaining the
perceive certain sources of health information as medical content. For a given candidate URL, the sys-
unreliable, they still access them to find information; tem computes a credibility score that is used both to
this finding also emerged in prior work.129 The paper rank candidate credible URLs, and to filter out non-
also shows that the fact of identifying on Facebook credible URLs; the technique is based on a graph
the sources of information more easily with respect propagation algorithm that leverages credibility
to traditional Websites, has a positive impact on the information from various online medical data-
users’ perception of information credibility. Leder- bases. Each credible URL is then evaluated in terms
man et al.130 provide qualitative and quantitative of relevance: text classifiers are applied to its par-
studies on how users assess the credibility of UGC in ent pages, by considering linguistic features
Online Health Forums (OHF), with particular refer- (i.e., words and n-grams incorporating medical and
ence to source and message characteristics. 159 parti- sentiment lexicons) learned from a set of manually
cipants were recruited using the AMT, and the labeled Web pages known to be relevant or irrele-
analysis was performed by requesting these partici- vant. Comparative evaluations have been per-
pants to complete a 10-min online survey on their formed against previous focused crawlers and
credibility perceptions. From this work, it emerges baseline Web spiders,134 showing the effectiveness
that two kinds of information are generated in OHF: of the proposed approach, in particular on blogs.
scientific information and experimental information. Weitzel et al.135 propose a method, Reputation
When a message explicitly refers to external scientific Rank, to measure the source reputation in the
content, the health literacy of users becomes less health-related context by exploiting social interac-
important, and they are more likely to depend on tions in Twitter. This approach focuses on retweet-
objective standards; instead, when dealing with sub- ing as a spreading information mechanism, which
jective or conflicting experimental information, users is considered as a form of endorsement for both
are more willing to focus on crowd consensus.131 Ma the message and the originating user. The authors
and Atkin120 in their recent work provide the first model a Retweet-Network as a directed weighted
complete meta-analysis of perceived credibility of graph, where users are the nodes and retweets are

the edges. A direct edge is generated if a user uj Discussion

retweets the content of another user ui. Weights on With respect to opinion spam and fake news detec-
edges represents the strength of trust ties, and are tion, a limited number of approaches have been pro-
computed as the ratio between the number of posed in the health-related context to assess the
retweets received by a user ui from a user uj, and credibility of online information, especially in Social
the total number of ui retweets in the network, plus Media. Table 3 summarizes the main approaches that
a ‘discount rate’ representing the presence or have been proposed so far; some of them consider the
absence of relationships (e.g., the ‘follow’ relation- issue of Website credibility, while in the Social Media
ship) among users. The Rank Reputation approach scenario the majority of them concerns the analysis of
proposed by the authors ranks the reputation of the characteristics influencing users’ perception of
the sources of information based on a set of cen- credibility with respect to health information. Only
trality measures136 that exploit the Retweet-Net- few works propose concrete techniques to assess
work modeled as previously described. sources and/or information credibility in Social Media.
The most complete work appeared so far in This shows how this problem, although it is of para-
this domain is the one by Mukherjee et al.,137 mount importance, has to be fully investigated yet.
who propose a method for the automatic assess-
ment of the credibility of both user-generated
medical statements in users’ posts and their CONCLUSIONS
authors. This approach considers both linguistic- In the post-truth era, Social Media often become the
and user-based features, since it focuses on the vehicle of the spread of misinformation and hoaxes.
assumption that statement credibility strongly Individuals who must form their own opinion, and
depends on linguistic objectivity, i.e., the general who are often locked inside their ‘eco-chambers,’ do
quality of the language in the user’s post, and on not have the necessary instruments and cognitive
user trustworthiness, represented by her/his status abilities to assess the level of credibility of pieces and
and engagement in the community (e.g., the num- sources of information with which they come into
ber of questions and answers posted, the number contact. While this may not have serious conse-
of provided ‘thanks,’ etc.). These factors are mod- quences in the case of the spread of rumors of little
eled through an MRF model, where the random importance, it may lead to serious problems when
variables are constituted by users, their posts, and consumers are directed to choose products and ser-
the medical statements contained within posts. vices based on fake reviews, or when false news
Evaluations are made on 15,000 users and 2.8 (e.g., political news) that influence public opinion are
million posts crawled from healthboards.com, and diffused, or when individuals run into inaccurate
on a ground truth provided by a medical portal. medical information which is likely to jeopardize
The results produced by the application of the their health. For this reason, the interest in studying
proposed model are compared with state-of-the- possible ways of helping users in assessing the level
art baselines, illustrating an increase in accuracy of credibility of online information is particularly
with respect to SVM. important, especially in the Social Web scenario. The
TABLE 3 | Summarization of the Proposed Approaches for Assessing Credibility of Online Health Information
Authors Year Focus Contribution
117
Eysenbach et al. 2002 Website credibility Credibility perception analysis
Freeman and Spyridakis109 2004 Website credibility Credibility perception analysis
119
Rains and Donnerstein Karmikel 2009 Website credibility Credibility perception analysis
Oh et al.127 2012 Credibility in social media Credibility perception analysis
Abbasi et al.133 2013 Credibility in social media Crawling credible UGC
Syn and Kim128 2013 Credibility in social media Credibility perception analysis
130
Lederman et al. 2014 Credibility in social media Credibility perception analysis
Mukherjee et al.137 2014 Credibility in social media UGC credibility assessment
135
Weitzel et al. 2014 Credibility in social media Source reputation assessment
Ma and Atkin120 2016 Credibility in social media Credibility perception analysis

concept of credibility has been studied since Aristotle credibility perceptions can be evaluated in terms of
(fourth century BC), and more recently in many dif- multiple characteristics, which may be associated
ferent research fields, such as communication, psy- with sources of information, shared contents, and
chology, and social sciences. An interesting aspect media across which information diffuses. In this sur-
connected to credibility is that, according to several vey, we have presented the more significant state-of-
studies, it represents a perceived quality of indivi- the-art proposals, which include: data-driven
duals, who form their opinion in a way to minimize approaches, model-driven approaches, and
their cognitive effort, especially online. In everyday approaches that exploit the graph-based structure
life, people usually reduce the uncertainty about cred- connecting entities in social platforms to propagate
ibility based on information easy to obtain in the credibility and trust. In particular, we have taken into
‘offline’ world: the reputation of the source of infor- consideration that the Social Web is characterized by
mation, the presence of trusted intermediaries such as multiple social applications, each of which is
experts and/or opinion leaders, personal trust based intended for different audiences and aims. This
on first-hand experiences. Conversely, the digital means that the solutions that have been proposed to
realm is characterized by a process of ‘disintermedia- assess credibility in Social Media change depending
tion’; in this context, the multiplicity of sources on the considered scenario. In this article, we have
involved in information dissemination, their possible therefore considered some context where the spread
anonymity, the absence of standards for information of false information may generate serious conse-
quality, the ease in manipulating and altering con- quences for the individuals involved. In particular,
tents, the lack of clarity of the context, and the pres- we have presented state-of-the-art approaches
ence of many potential targets of credibility addressing three main tasks: (1) the detection of
evaluation (i.e., the content, the source, and the opinion spam in review sites, (2) the detection of fake
medium), make credibility assessment a more com- news and spam in microblogging, and (3) the credi-
plex task with respect to ‘offline’ media. bility assessment of online health information.
Due to the above reasons, in the last years Table 4 summarizes and compares the three above
numerous approaches in the literature have tackled tasks with respect to their focus, proposed
the issue of the automatic assessment of information approaches, and open issues in the assessment of
credibility, in particular in Social Media. Individuals’ information credibility.
TABLE 4 | Summarization of the Focus, Proposed Approaches, and Open Issues of Each of the Three Scenarios Considered in the Credibility
Assessment Issue
Assessment of the Credibility of Information in Social Media

Opinions News Health Information
Focus
Fake review detection Perceived credibility factors Perceived credibility factors
Spammer detection Rumor propagation Health Website credibility
Group spammer detection Spammer detection Health-related UGC credibility assessment
Tweet/Event/Trending topic credibility
Approaches
Classification-based Classification-based Classification-based
Content-based Multifeature-based Multifeature-based
Behavioral-based Propagation-based Propagation-based
Multifeature-based Graph-based Graph-based (Crawling)
Propagation-based Survey-based Survey-based
Behavioral-based
Graph-based
Open issues
Almost impossible to hire human The impact of the content alone is not Only few automatic or semi-automatic
experts for assessing the credibility sufficiently investigated approaches are proposed
of ‘opinions’, such as reviews The definition of news, topic, event is not Higly context-dependent scenario
Absence of convincing gold standard always rigorous Possibly harmful scenario
datasets Small datasets are considered Small datasets are considered
Small datasets are considered

As discussed in the article, and as it emerges automatically assess the credibility of online health
from Table 4, among the important open issues information, even if we think that this constitutes
characterizing the credibility assessment problem, one of the most challenging areas of research in the
the most urgent are the absence of predefined coming years.
benchmarks and gold standard datasets
(in particular, for fake review detection), and the
problem of collecting and mining large amounts of
data. In the big data era, this latter problem has
NOTES
a
not yet received the attention it deserves. Further- https://twitter.com
b
more, only few approaches have been proposed to http://www.pewresearch.org/
REFERENCES
1. Quattrociocchi W, Scala A, Sunstein CR. Echo Press, 1983. J Am Soc Inf Sci 1984, 35:254–255.
Chambers on Facebook, June 13, 2016. Available: https://doi.org/10.1002/asi.4630350410.
http://ssrn.com/abstract=2795110. 12. Hilligoss B, Rieh SY. Developing a unifying frame-
2. Self CS. Credibility. In: Salwen MB, Stacks DW, eds. work of credibility assessment: construct, heuristics,
An Integrated Approach to Communication Theory and interaction in context. Inf Process Manage 2008,
and Research. 2nd ed. Routledge: Taylor and Francis 44:1467–1484. https://doi.org/10.1016/j.ipm.2007.
Group; 2008, 435–456. https://doi.org/10.4324/ 10.001.
9780203887011.
13. Wathen CN, Burkell J. Believe it or not: factors influ-
3. Metzger MJ. Making sense of credibility on the web: encing credibility on the web. J Am Soc Inf Sci Tech-
models for evaluating online information and recom- nol 2002, 53:134–144.
mendations for future research. J Am Soc Inf Sci
Technol 2007, 58:2078–2091. 14. Metzger MJ, Flanagin AJ. Credibility and trust of
information in online environments: the use of cogni-
4. Hovland CI, Lumsdaine AA, Fred D. Experiments on
tive heuristics. J Pragmat 2013, 59(Part B):210–220.
Mass Communication. Studies in Social Psychology in
https://doi.org/10.1016/j.pragma.2013.07.012.
World War II, vol. 3. Princeton, NJ: Princeton Uni-
versity Press; 1949. 15. Flanagin AJ, Metzger MJ. Digital Media and Youth:
5. Hovland CI. Changes in attitude through communica- Unparalleled Opportunity and Unprecedented
tion. J Abnorm Soc Psychol 1951, 46:424. Responsibility. Cambridge, MA: MIT Press; 2008,
5–28. Available at: http://www.mitpressjournals.org/doi/
6. Hovland CI, Janis IL, Kelley HH. Communication
pdfplus/10.1162/dmal.9780262562324.005. Accessed
and Persuasion. New Haven, CT: Yale University
January 1, 2017.
Press; 1953.
7. Fogg BJ, Tseng H. The elements of computer credibil- 16. Eysenbach G. Credibility of Health Information and
ity. In: Proceedings of the SIGCHI Conference on Digital Media: New Perspectives and Implications for
Human Factors in Computing Systems, Pittsburgh, Youth. In: Metzger MM, Flanagin AJ, eds. Digital
PA, USA, 15–20 May, 1999. New York, NY: ACM; Media, Youth, and Credibility. Cambridge, MA: The
1999, 80–87. MIT Press; 2008, 123–154.
8. Metzger MJ, Flanagin AJ, Eyal K, Lemus DR, 17. Sundar SS. The main model: a heuristic approach to
McCann RM. Credibility for the 21st century: inte- understanding technology effects on credibility. In:
grating perspectives on source, message, and media Metzger MJ, Flanagin AJ, eds. Digital Media, Youth,
credibility in the contemporary media environment. and Credibility. Cambridge, MA: The MIT Press;
Ann Int Commun Assoc 2003, 27:293–335. 2008, 73–100.
9. Pornpitakpan C. The persuasiveness of source credi- 18. Eysenbach G, Köhler C. How do consumers search
bility: a critical review of five decades’ evidence. for and appraise health information on the world
J Appl Soc Psychol 2004, 34:243–281. https://doi. wide web? Qualitative study using focus groups, usa-
org/10.1111/j.1559-1816.2004.tb02547.x. bility tests, and in-depth interviews. BMJ 2002,
10. Liu Z. Perceptions of credibility of scholarly informa- 324:573–577.
tion on the web. Inf Process Manage 2004, 19. Kaplan AM, Haenlein M. Users of the world, unite!
40:1027–1038. The challenges and opportunities of social media. Bus
11. Bernier CL. Second-hand knowledge. An inquiry into Horiz 2010, 53:59–68. DOI: https://doi.org/10.1016/
cognitive authority. Patrick Wilson. Greenwood j.bushor.2009.09.003.

20. Warneken F, Tomasello M. Altruistic helping in 35. Ott M, Cardie C, Hancock J. Estimating the preva-
human infants and young chimpanzees. Science 2006, lence of deception in online review communities. In:
311:1301–1303. Proceedings of the 21st International Conference on
21. Olson KR, Spelke ES. Foundations of cooperation in World Wide Web, Lyon, France, 16–20 April, 2012.
young children. Cognition 2008, 108:222–231. New York, NY: ACM; 2012, 201–210.
22. Warneken F, Tomasello M. The roots of human 36. Banerjee S, Chua AY. Applauses in hotel reviews:
altruism. Br J Psychol 2009, 100:455–471. genuine or deceptive?. In: Science and Information
Conference (SAI), London, UK, 27–29 August, 2014.
23. Moens M-F, Li J, Chua T-S, eds. Mining User Gener- IEEE; 2014, 938–942.
ated Content, Social Media and Social Computing.
Boca Raton, FL: Chapman and Hall/CRC; 2014. 37. Fusilier DH, Montes-y Gómez M, Rosso P,
Cabrera RG. Detection of opinion spam with charac-
24. Safko L. The Social Media Bible: Tactics, Tools, and
ter n-grams. In: International Conference on Intelli-
Strategies for Business Success. 2nd ed. Hoboken,
gent Text Processing and Computational Linguistics.
NJ John Wiley & Sons Publishing; 2010. ISBN:
Springer; 2015, 285–294.
0470623977.
25. Hajli MN. Developing online health communities 38. Hengeveld K. Parts of speech. In: Fortescue M,
through digital media. Int J Inf Manage 2014, Harder P, Kristoffersen L, eds. Layered Structure and
34:311–314. Reference in a Functional Perspective. Amsterdam:
John Benjamins; 1992, 29–55.
26. Hajli MN, Sims J, Featherman M, Love PE. Credibil-
ity of information in online communities. J Strateg 39. Pennebaker JW, Francis ME, Booth RJ. Linguistic
Market 2015, 23:238–253. Inquiry and Word Count: Liwc 2001, vol. 71. Mah-
way, NJ: Lawrence Erlbaum Associates; 2001, 2001.
27. Lang A. The limited capacity model of mediated mes-
sage processing. J Commun 2000, 50:46–70. 40. Mukherjee A, Venkataraman V, Liu B, Glance N.
Fake review detection: classification and analysis of
28. Liu B. Web Data Mining: Exploring Hyperlinks,
real and pseudo reviews. Technical Report No. UIC-
Contents, and Usage Data. Springer-Verlag Berlin
CS-03-2013, 2013.
Heidelberg Springer Science & Business Media; 2007.
41. Mukherjee A, Venkataraman V, Liu B, Glance NS.
29. Jindal N, Liu B. Review spam detection. In: Proceed-
What yelp fake review filter might be doing?. In:
ings of the 16th International Conference on World
ICWSM, 2013.
Wide Web, Banff, Alberta, Canada, 8–12 May, 2007.
New York, NY: ACM; 2007, 1189–1190. 42. Sun H, Morales A, Yan X. Synthetic review spam-
30. Jindal N, Liu B. Opinion spam and analysis. In: Pro- ming and defense. In: Proceedings of the 19th ACM
ceedings of the 2008 International Conference on SIGKDD International Conference on Knowledge
Web Search and Data Mining, Palo Alto, CA, USA, Discovery and Data Mining. New York, NY: ACM;
11–12 February, 2008. New York, NY: ACM; 2008, 2013, 1088–1096.
219–230. 43. Croft B, Lafferty J. Language Modeling for Informa-
31. Heydari A, Tavakoli M, Salim N, Heydari Z. Detec- tion Retrieval, vol. 13. Dordrecht: Springer Science &
tion of review spam: a survey. Expert Syst Appl Business Media; 2013.
2015, 42:3634–3642. 44. Lai CL, Xu KQ, Lau RYK, Li Y, Jing L. Toward a
32. Crawford M, Khoshgoftaar TM, Prusa JD, language modeling approach for consumer review
Richter AN, Al Najada H. Survey of review spam spam detection. In: 2010 I.E. 7th International Con-
detection using machine learning techniques. J Big ference on E-Business Engineering, 2010, 1–8.
Data 2015, 2:23. https://doi.org/10.1186/s40537- 45. Kullback S, Leibler RA. On information and suffi-
015-0029-9. ciency. Ann Math Stat 1951, 22:79–86.
33. Li F, Huang M, Yang Y, Zhu X. Learning to identify
46. Lau RYK, Liao SY, Kwok RC-W, Xu K, Xia Y, Li Y.
review spam. In: IJCAI Proceedings—International
Text mining and probabilistic language modeling for
Joint Conference on Artificial Intelligence, vol 22, Bar-
online review spam detection. ACM Trans Manage
celona, Catalonia, Spain, 16–22 July, 2011. Menlo
Inf Syst 2012, 2:25:1–25:30. https://doi.org/10.1145/
Park, CA: AAAI Press; 2011, 2488–2493.
2070710.2070716.
34. Ott M, Choi Y, Cardie C, Hancock JT. Finding
47. Mukherjee A, Liu B, Wang J, Glance N, Jindal N.
deceptive opinion spam by any stretch of the imagina-
Detecting group review spam. In: Proceedings of the
tion. In: Proceedings of the 49th Annual Meeting of
20th International Conference Companion on World
the Association for Computational Linguistics:
Wide Web. New York, NY: ACM; 2011, 93–94.
Human Language Technologies, vol 1, Portland, OR,
USA, 19–24 June, 2011. Stroudsburg, PA: Associa- 48. Li X, Liu B. Learning to classify texts using positive
tion for Computational Linguistics; 2011, 309–319. and unlabeled data. In: IJCAI, vol 3. 2003, 587–592.

49. Ren Y, Ji D, Zhang H. Positive unlabeled learning for Research and Development in Information Retrieval.
deceptive reviews detection. In: EMNLP, 2014, New York, NY: ACM; 2005, 472–479.
488–498. 64. Murphy KP, Weiss Y, Jordan MI. Loopy belief prop-
50. Yafeng R, Lan Y, Donghong J. Deceptive reviews agation for approximate inference: an empirical
detection based on language structure and sentiment study. In: Proceedings of the Fifteenth Conference on
polarity. J Front Comput Sci Technol 2014, Uncertainty in Artificial Intelligence. Morgan Kauf-
8:313–320. mann Publishers Inc.; 1999, 467–475.
51. Blei DM, Ng AY, Jordan MI. Latent 65. Mukherjee A, Liu B, Glance N. Spotting fake
Dirichlet allocation. J Mach Learn Res 2003, reviewer groups in consumer reviews. In: Proceedings
3:993–1022. of the 21st International Conference on World Wide
52. Fusilier DH, Móntes-y-Gómez M, Rosso P, Web. New York, NY: ACM; 2012, 191–200.
Cabrera RG. Detecting positive and negative decep- 66. G. Wang, S. Xie, B. Liu, and S. Y. Philip. Review
tive opinions using pu-learning. Inf Process Manage graph based online store review spammer detection.
2015, 51:433–443. DOI: https://doi.org/10.1016/j. In: 2011 I.E. 11th International Conference on Data
ipm.2014.11.001. Mining. IEEE; 2011, 1242–1247.
53. Liu B, Lee WS, Yu PS, Li X. Partially supervised clas- 67. Wang G, Xie S, Liu B, Yu PS. Identify online store
sification of text documents. In: ICML (Citeseer), vol review spammers via social review graph.
2, 2002, 387–394. ACM Trans Intell Syst Technol 2012, 3:61.
54. Abbasi A, Zhang Z, Zimbra D, Chen H, 68. Akoglu L, Chandy R, Faloutsos C. Opinion fraud
Nunamaker JF Jr. Detecting fake websites: the contri- detection in online reviews by network effects. In:
bution of statistical learning theory. MIS Q 2010, ICWSM, vol 13; 2013, 2–11.
34:435–461.
69. Ye J, Akoglu L. Discovering opinion spammer groups
55. Li F, Huang M, Yang Y, Zhu X. Learning to identify
by network footprints. In: Joint European Conference
review spam. In: Proceedings of the Twenty-Second
on Machine Learning and Knowledge Discovery in
International Joint Conference on Artificial Intelli-
Databases. Springer; 2015, 267–282.
gence, IJCAI’11, vol 3, Barcelona, Catalonia, Spain.
AAAI Press; 2011, 2488–2493, ISBN 978-1-57735- 70. Choo E, Yu T, Chi M. Detecting Opinion Spammer
515-1. Groups Through Community Discovery and Senti-
ment Analysis. Cham: Springer International Publish-
56. Li H, Liu B, Mukherjee A, Shao J. Spotting fake
ing; 2015, 170–187. https://doi.org/10.1007/978-3-
reviews using positive-unlabeled learning. Comput
319-20810-7_11. ISBN: 978-3-319-20810-7.
Sist 2014, 18:467–475.
71. Wang Z, Hou T, Song D, Li Z, Kong T. Detecting
57. Luca M, Zervas G. Fake it till you make it: reputa-
review spammer groups via bipartite graph projec-
tion, competition, and yelp review fraud. Manage Sci
tion. Comput J 2015, 59:bxv068.
2016, 62:3412–3427.
58. Viviani M, Pasi G. Quantifier guided aggregation for 72. Borgelt C. Frequent item set mining. WIREs Data
the veracity assessment of online reviews. Int J Intell Mining Knowl Discov 2012, 2:437–456.
Syst 2016, 32:481–501. 73. Li H, Chen Z, Mukherjee A, Liu B, Shao J. Analyzing
59. Lim E-P, Nguyen V-A, Jindal N, Liu B, Lauw HW. and detecting opinion spam on a large-scale dataset
Detecting product review spammers using rating via temporal and spatial patterns. In: International
behaviors. In: Proceedings of the 19th ACM Interna- AAAI Conference on Web and Social Media, 2015,
tional Conference on Information and Knowledge 634–637.
Management. New York, NY: ACM; 2010, 939–948. 74. Kaplan AM, Haenlein M. The early bird catches the
60. Fei G, Mukherjee A, Liu B, Hsu M, Castellanos M, news: nine things you should know about micro-blog-
Ghosh R. Exploiting burstiness in reviews for review ging. Bus Horiz 2011, 54:105–113.
spammer detection. In: ICWSM, vol 13. 2013, 75. Kaplan AM, Haenlein M. Users of the world, unite!
175–184. The challenges and opportunities of social media. Bus
61. Botev ZI, Grotowski JF, Kroese DP, et al. Kernel den- Horiz 2010, 53:59–68.
sity estimation via diffusion. Ann Stat 2010, 76. Harcup T, O’Neill D. What is news? Galtung and
38:2916–2957. ruge revisited. J Stud 2001, 2:261–280.
62. Cross GR, Jain AK. Markov random field texture 77. Java A, Song X, Finin T, Tseng B. Why we twitter:
models. IEEE Trans Pattern Anal Mach Intell 1983, understanding microblogging usage and communities.
PAMI-5:25–39. In: Proceedings of the 9th WebKDD and 1st SNA-
63. Metzler D, Croft WB. A Markov random field model KDD 2007 Workshop on Web Mining and Social
for term dependencies. In: Proceedings of the 28th Network Analysis. New York, NY: ACM;
Annual International ACM SIGIR Conference on 2007, 56–65.

78. Naaman M, Boase J, Lai C-H. Is it really about me?: 91. Järvelin K, Kekäläinen J. Cumulated gain-based eval-
message content in social awareness streams. In: Pro- uation of ir techniques. ACM Trans Inf Syst 2002,
ceedings of the 2010 ACM Conference on Computer 20:422–446.
Supported Cooperative Work. New York, NY: ACM; 92. Gupta A, Kumaraguru P, Castillo C, Meier P.
2010, 189–192. Tweetcred: A real-time Web-based system for asses-
79. Castillo C, Mendoza M, Poblete B. Information credi- sing credibility of content on Twitter. In: Proceedings
bility on twitter. In: Proceedings of the 20th Interna- of 6th International Conference on Social Informatics
tional Conference on World Wide Web. New York, (SocInfo), Barcelona, Spain, 2014.
NY: ACM; 2011, 675–684. 93. Galán-Garca P, de la Puerta JG, Gómez CL, Santos I,
80. Kwak H, Lee C, Park H, Moon S. What is Twitter, a Bringas PG. Supervised machine learning for the
social network or a news media?. In: Proceedings of detection of troll profiles in twitter social network:
the 19th International Conference on World Wide application to a real case of cyberbullying. In: Inter-
Web. New York, NY: ACM; 2010, 591–600. national Joint Conference SOCO’13-CISIS’13-
ICEUTE’13. Springer; 2014, 419–428.
81. Rubin VL, Chen Y, Conroy NJ. Deception detection
for news: three types of fakes. Proc Assoc Inf Sci 94. Gupta A, Kaushal R. Improving spam detection in
Technol 2015, 52:1–4. online social networks. In: 2015 International Con-
ference on Cognitive Computing and Information
82. AlMansour AA, Brankovic L, Iliopoulos CS. A model
Processing (CCIP). IEEE; 2015, 1–6.
for recalibrating credibility in different contexts and
languages—a twitter case study. Int J Digital Inf 95. Mendoza M, Poblete B, Castillo C. Twitter under cri-
Wirel Commun 2014, 4:53–62. sis: can we trust what we RT? In: Proceedings of the
First Workshop on Social Media Analytics.
83. Jin Z, Cao J, Jiang Y-G, Zhang Y. News credibility New York, NY: ACM; 2010, 71–79.
evaluation on microblog with a hierarchical propaga-
tion model. In: 2014 I.E. International Conference on 96. Seo E, Mohapatra P, Abdelzaher T. Identifying
Data Mining. IEEE; 2014, 230–239. rumors and their sources in social networks. In: SPIE
Defense, Security, And Sensing. International Society
84. Castillo C, Mendoza M, Poblete B. Predicting infor- for Optics and Photonics; 2012, 83891I–83891I.
mation credibility in time-sensitive social media.
Internet Res 2012, 23:560–588. https://doi.org/10. 97. Gupta M, Zhao P, Han J. Evaluating event credibility
1108/IntR-05-2012-0095. on twitter. In: SDM. SIAM; 2012, 153–164.
85. Mathioudakis M, Koudas N. TwitterMonitor: trend 98. Zhao L, Hua T, Lu C-T, Chen I-R. A topic-focused
detection over the Twitter stream. In: Proceedings of trust model for twitter. Comput Commun 2016,
the 2010 ACM SIGMOD International Conference 76:1–11. DOI: https://doi.org/10.1016/j.comcom.
on Management of data. New York, NY: ACM; 2015.08.001.
2010, 1155–1158. 99. Morris MR, Counts S, Roseway A, Hoff A,
Schwarz J. Tweeting is believing?: understanding
86. Bhargava N, Sharma G, Bhargava R, Mathuria M.
microblog credibility perceptions. In: Proceedings of
Decision tree analysis on j48 algorithm for data min-
the ACM 2012 Conference on Computer Supported
ing. Proc Int J Adv Res Comput Sci Softw Eng 2013,
Cooperative Work, CSCW ’12, Seattle, Washington,
3:1114–1119.
USA. New York, NY: ACM; 2012, 441–450, ISBN
87. Kang B, O’Donovan J, Höllerer T. Modeling topic 978-1-4503-1086-4. DOI: https://doi.org/10.1145/
specific credibility on twitter. In: Proceedings of the 2145204.2145274.
2012 ACM International Conference on Intelligent
100. Kang B, Höllerer T, O’Donovan J. Believe it or not?
User Interfaces. New York, NY: ACM; 2012,
analyzing information credibility in microblogs. In:
179–188.
Proceedings of the 2015 IEEE/ACM International
88. Gupta A, Kumaraguru P. Credibility ranking of Conference on Advances in Social Networks Analysis
tweets during high impact events. In: Proceedings of and Mining, 2015. New York, NY: ACM; 2015,
the 1st Workshop on Privacy and Security in Online 611–616.
Social Media. New York, NY: ACM; 2012, 2.
101. O’Donovan J, Kang B, Meyer G, Höllerer T,
89. Cao Y, Xu J, Liu T-Y, Li H, Huang Y, Hon H-W. Adalii S. Credibility in context: an analysis of feature
Adapting ranking SVM to document retrieval. In: distributions in twitter. In: 2012 International Con-
Proceedings of the 29th Annual International ACM ference on Privacy, Security, Risk and Trust (PAS-
SIGIR Conference on Research and Development in SAT), 2012 International Conference on Social
Information Retrieval. New York, NY: ACM; 2006, Computing (SocialCom). IEEE; 2012, 293–301.
186–193. 102. Yang J, Counts S, Morris MR, Hoff A. Microblog
90. Trotman A. Learning to rank. Inf Retr 2005, credibility perceptions: comparing the USA and
8:359–381. China. In: Proceedings of the 2013 Conference on

Computer Supported Cooperative Work. New York, online health message forums. Int J Med Inform
NY: ACM; 2013, 575–586. 2012, 81:36–44.
103. Sikdar S, Kang B, O’Donovan J, Höllerer T, Adah S. 117. Eysenbach G, Powell J, Kuss O, Sa E-R. Empirical
Understanding information credibility on twitter. In: studies assessing the quality of health information for
2013 International Conference on Social Computing consumers on the world wide web: a systematic
(SocialCom). IEEE; 2013, 19–24. review. JAMA 2002, 287:2691–2700.
104. European Commission. Flash Eurobarometer 404— 118. Rieh SY, Danielson DR. Credibility: a multidiscipli-
TNS Political & Social. Technical Report. European nary framework. Annu Rev Inf Sci Technol 2007,
Commission; 2014. 41:307–364.
105. World Health Organization, Kickbusch I, Nutbeam 119. Rains SA, Karmikel CD. Health information-
D. Health Promotion Glossary. Geneva: World seeking and perceptions of website credibility:
Health Organization; 1998. examining web-use orientation, message character-
istics, and structural features of websites. Comput
106. Sørensen K, Van den Broucke S, Fullam J, Doyle G,
Hum Behav 2009, 25:544–553, including the Spe-
Pelikan J, Slonska Z, Brand H. Health literacy and
cial Issue: State of the Art Research into Cognitive
public health: a systematic review and integration of
Load Theory. DOI: https://doi.org/10.1016/j.chb.
definitions and models. BMC Public Health
2008.11.005.
2012, 12:1.
120. Ma TJ, Atkin D. User generated content and credibil-
107. Stokols D. Translating social ecological theory into
ity evaluation of online health information: a meta
guidelines for community health promotion.
analytic study. Telemat Inform 2016, 34:472–486.
Am J Health Promot 1996, 10:282–298.
121. Casaló LV, Flavián C, Guinalu M. New members’
108. Chinn D. Critical health literacy: a review and critical integration: key factor of success in online travel com-
analysis. Soc Sci Med 2011, 73:60–67. munities. J Bus Res 2013, 66:706–710.
109. Freeman KS, Spyridakis JH. An examination of fac- 122. Househ M, Borycki E, Kushniruk A. Empowering
tors that affect the credibility of online health infor- patients through social media: the benefits and chal-
mation. Tech Commun 2004, 51:239–263. lenges. Health Inform J 2014, 20:50–58.
110. Xiao N, Sharman R, Rao HR, Upadhyaya S. Factors 123. Spence PR, Lachlan KA, Westerman D, Spates SA.
influencing online health information search: an Where the gates matter less: ethnicity and perceived
empirical analysis of a national cancer-related survey. source credibility in social media health messages.
Decis Support Syst 2014, 57:417–427. Howard J Commun 2013, 24:1–16.
111. De Choudhury M, Morris MR, White RW. Seeking 124. Syed-Abdul S, Fernandez-Luque L, Jian W-S, Li Y-C,
and sharing health information online: comparing Crain S, Hsu M-H, Wang Y-C, Khandregzen D,
search engines and social media. In: Proceedings of Chuluunbaatar E, Nguyen PA, et al. Misleading
the 32nd Annual ACM Conference on Human Fac- health-related information promoted through video-
tors in Computing Systems. New York, NY: ACM; based social media: anorexia on youtube. J Med
2014, 1365–1376. Internet Res 2013, 15:e30.
112. Li Y, Wang X, Lin X, Hajli M. Seeking and sharing 125. Berry TR, Shields C. Source attribution and credibil-
health information on social media: a net valence ity of health and appearance exercise advertise-
model and cross-cultural comparison. Technol Fore- ments: relationship with implicit and explicit
cast Social Change. In press. attitudes and intentions. J Health Psychol 2014,
113. Metzger MJ, Flanagin AJ. Online Health Information 19:242–252.
Credibility. In: Thompson TL, ed. Encyclopedia of 126. Dutta-Bergman MJ. The impact of completeness and
Health Communication. Thousand Oaks, CA: SAGE; web use motivation on the credibility of e-health
2011, 976–978. information. J Commun 2004, 54:253–269.
114. Rice RE. Influences, usage, and outcomes of internet 127. Oh S, Yi YJ, Worrall A. Quality of health answers in
health information searching: multivariate results social q&a. Proc Am Soc Inf Sci Technol
from the pew surveys. Int J Med Inform 2006, 2012, 49:1–6.
75:8–28. 128. Syn SY, Kim SU. The impact of source credibility on
115. Adams SA. Revisiting the online health information young adults’ health information activities on face-
reliability debate in the wake of web 2.0: an inter- book: preliminary findings. Proc Am Soc Inf Sci
disciplinary literature and website review. Int J Med Technol 2013, 50:1–4.
Inform 2010, 79:391–400. 129. Kwan MYW, Arbour-Nicitopoulos KP, Lowe D,
116. O’Grady L, Wathen CN, Charnaw-Burger J, Betel L, Taman S, Faulkner GE. Student reception, sources,
Shachak A, Luke R, Hockema S, Jadad AR. The use and believability of health-related information. J Am
of tags and tag clouds to discern credible content in Coll Health 2010, 58:555–562.

130. Lederman R, Fan H, Smith S, Chang S. Who can you 134. Pant G, Srinivasan P. Learning to crawl: comparing
trust? Credibility assessment in online health forums. classification schemes. ACM Trans Inf Syst 2005,
Health Policy Technol 2014, 3:13–25. 23:430–462.
131. Festinger L. A theory of social comparison processes. 135. Weitzel L, de Oliveira JPM, Quaresma P. Measuring
Hum Relat 1954, 7:117–140. the reputation in user-generated-content systems
based on health information. Procedia Comput Sci
132. McPherson M, Smith-Lovin L, Cook JM. Birds of a 2014, 29:364–378.
feather: homophily in social networks. Annu Rev
Sociol 2001, 27:415–444. 136. Freeman LC. Centrality in social networks conceptual
clarification. Social Netw 1978, 1:215–239.
133. Abbasi A, Fu T, Zeng D, Adjeroh D. Crawling 137. Mukherjee S, Weikum G, Danescu-Niculescu-
credible online medical sentiments for social intel- Mizil C. People on drugs: credibility of user state-
ligence. In: 2013 International Conference on ments in health communities. In: Proceedings of the
Social Computing (SocialCom). IEEE; 2013, 20th ACM SIGKDD International Conference on
254–263. Knowledge Discovery and Data Mining. New York,
NY: ACM; 2014, 65–74.

Credibility in Social Media Opinions, News, and Health Information-A Survey

Uploaded by

Copyright:

Available Formats

Credibility in Social Media Opinions, News, and Health Information-A Survey

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Credibility in Social Media Opinions, News, and Health Information-A Survey

Uploaded by

Copyright:

Available Formats

Advanced Review

Credibility in social media:

INTRODUCTION running into misinformation is not negligible. For

N owadays, in the Web 2.0 era, the interaction

© 2017 John Wiley & Sons, Ltd 1 of 25

of credibility is often more complex than in previous

2 of 25 © 2017 John Wiley & Sons, Ltd

© 2017 John Wiley & Sons, Ltd 3 of 25

4 of 25 © 2017 John Wiley & Sons, Ltd

© 2017 John Wiley & Sons, Ltd 5 of 25

6 of 25 © 2017 John Wiley & Sons, Ltd

© 2017 John Wiley & Sons, Ltd 7 of 25

Spammer Detection group, group size, etc.). By considering these features,

8 of 25 © 2017 John Wiley & Sons, Ltd

© 2017 John Wiley & Sons, Ltd 9 of 25

10 of 25 © 2017 John Wiley & Sons, Ltd

© 2017 John Wiley & Sons, Ltd 11 of 25

12 of 25 © 2017 John Wiley & Sons, Ltd

© 2017 John Wiley & Sons, Ltd 13 of 25

14 of 25 © 2017 John Wiley & Sons, Ltd

© 2017 John Wiley & Sons, Ltd 15 of 25

16 of 25 © 2017 John Wiley & Sons, Ltd

© 2017 John Wiley & Sons, Ltd 17 of 25

the edges. A direct edge is generated if a user uj Discussion

18 of 25 © 2017 John Wiley & Sons, Ltd

Assessment of the Credibility of Information in Social Media

© 2017 John Wiley & Sons, Ltd 19 of 25

20 of 25 © 2017 John Wiley & Sons, Ltd

© 2017 John Wiley & Sons, Ltd 21 of 25

22 of 25 © 2017 John Wiley & Sons, Ltd

© 2017 John Wiley & Sons, Ltd 23 of 25

24 of 25 © 2017 John Wiley & Sons, Ltd

© 2017 John Wiley & Sons, Ltd 25 of 25

You might also like