Journal of Information Security and Applications 55 (2020) 102644

Contents lists available at ScienceDirect

Journal of Information Security and Applications

journal homepage: www.elsevier.com/locate/jisa

A Guiding Framework for Vetting the Internet of Things

Fatma Masmoudi a, Zakaria Maamar b, Mohamed Sellami *, c, Ali Ismail Awad d, e, f,
Vanilson Burégio g
a
College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University, Alkharj, 11942, Saudi Arabia
b
College of Technological Innovation, Zayed University, Dubai, UAE
c
Samovar, Télécom SudParis, Institut Polytechnique de Paris, Paris, France
d
Department of Computer Science, Electrical and Space Engineering, Luleå University of Technology, Luleå 97187, Sweden
e
Electrical Engineering Department, Faculty of Engineering, Al-Azhar University at Qena, Qena 83513, Egypt
f
Centre for Security, Communications and Network Research, University of Plymouth, Plymouth PL4 8AA, UK
g
Department of Computing, Federal Rural University of Pernambuco, Recife, Brazil

A R T I C L E I N F O A B S T R A C T

Keywords: Like any emerging and disruptive technology, multiple obstacles are slowing down the Internet of Things (IoT)
Internet of Things expansion for instance, multiplicity of things’ standards, users’ reluctance and sometimes rejection due to pri­
Security vulnerabilities vacy invasion, and limited IoT platform interoperability. IoT expansion is also accompanied by the widespread
Vetting
use of mobile apps supporting anywhere, anytime service provisioning to users. By analogy to vetting mobile
Atomic/composite duties
apps, this paper addresses the lack of principles and techniques for vetting IoT devices (things) in preparation for
their integration into mission-critical systems. Things have got vulnerabilities that should be discovered and
assessed through proper device vetting. Unfortunately, this is not happening. Rather than sensing a nuclear
turbines steam level, a thing could collect some sensitive data about the turbine without the knowledge of users
and leak these data to third parties. This paper presents a guiding framework that defines the concepts of,
principles of, and techniques for thing vetting as a pro-active response to potential things vulnerabilities.

1. Introduction connected devices set to top 20 billion by 2023 [3].

In conjunction with the mobile apps “fever”, we observe some early
In the 21st century, security triad formed by confidentiality, integ­ signs of another “fever” that the Internet of Things (IoT) could end-up
rity, and availability (CIA) in addition to user privacy, are among many catching. According to Gartner, 6.4 billion connected things were in
pressing concerns that the Information and Communication Technology use in 2016, and this number is projected to reach 20.8 billion by 2020
(ICT) community is actively examining [1]. The number of misuse and [4], [5]. Accordingly, the total economic impact of IoT will reach be­
fraudulent cases related to the ICT are on the rise, suggesting the need tween $3.9 trillion and $11.1 trillion per year by 2025 [6]. This rapid
for an immediate revision of existing practices, techniques, and tools. increase in the number of things needs to be closely monitored to avoid a
According to the Australian Institute of Criminology, the estimated “boom” in privacy breaches, identity thefts, confidentiality deceptions,
value of internal fraud against the Commonwealth has steadily etc.
increased from $1.9 million to $3 million between 2008 and 2011. To Things in an IoT ecosystem have some unique features that make
mitigate some of these cases, an option is to vet ICT applications prior to them different from other components such as reduced size, restricted
their integration into day-to-day (sometimes critical) business opera­ connectivity, continuous mobility, limited energy, and constrained
tions. By vetting, we mean ensuring their safety and compliance with storage [7]. On top of these features, things’ plethora of uses lead to the
relevant regulations. Mobile apps exemplify ICT applications, whose generation of rich and large volume of (unstructured) data that need to
rapid and uncontrolled widespread use has become a major concern to be processed in compliance with requirements such as safety, security
policy makers. As of September 2019, there were 2.7 million apps posted and privacy. The complexity of these features mixed with these different
on Google Play Store and 2.46 million apps posted on Apple’s App Store, uses raises a number of challenges related to how safe things are or
which are the 2 leading app stores in the world [2], with the number of should be. In this context, 𝒱etting IoT(𝒱IoT) is appropriate for ensuring

* Corresponding author.
E-mail address: [email protected] (M. Sellami).

https://doi.org/10.1016/j.jisa.2020.102644
Received 20 March 2020; Received in revised form 27 August 2020; Accepted 5 October 2020
Available online 19 October 2020
2214-2126/© 2020 Elsevier Ltd. All rights reserved.
F. Masmoudi et al. Journal of Information Security and Applications 55 (2020) 102644

things’ “good conducts” before allowing them to collect, process, and apps’ administrators, phone users, and application developers to detect
distribute data without raising any concerns. However, there is a limited and prevent malware threats. They highlight the need for a strict app
number of R&D initiatives on 𝒱IoT, which is accentuated by the heavy vetting on the server-side for removing any malicious apps from the
dependence of ICT practitioners on the security claims of things’ ven­ market using cloud-based scanning engines. Finally, He et al. conclude
dors [8]. by suggesting new research directions related to the issues of (1) how to
By analogy to mobile apps vetting, we aim to develop a guiding systematically collect data of emerging malware; (2) how to develop
framework for 𝒱IoT. The objective is to verify things’ “good conducts” techniques and tools to associate new attacks with those already
and identify the vulnerabilities that could lead to misconducts. Examples recorded; (3) how to uncover unknown malware; and (4) how to make a
of vulnerabilities could be the excessive use of resources, data sharing paradigm shift that will involve advertisers as defenders, too. The lack of
with unauthorized parties, and changes in initial configurations. In this datasets about new vulnerable applications is also discussed by Nagap­
paper and in line with our ongoing agenda on IoT [9,10], we refer to pan and Shihab in [14]. They address the need for more sophisticated
things’ actions as duties and categorize them into atomicand composite 1. lightweight tools that would prevent malicious code from getting into
This categorization permits the fine-tuning of the vetting according to smartphones through mobile app stores.
each duty’s characteristics and whether the duties originate from the In [15], Chen et al. propose mass vetting (MassVet), a lightweight
same or different things. Our contribution is manifold, including (i) the mechanism that is capable of quickly discovering unknown malice at a
definition of vetting in the context of IoT, (ii) the identification of vul­ large scale (e.g., Google Play Store). The authors use an app’s view
nerabilities for atomic duties and composite duties, (iii) the analysis of structure and navigation relations among views to produce a view graph
the consequences of vulnerabilities on the completion of duties, and (iv) whose nodes correspond to view items (active widgets) and arcs be­
the demonstration of the vetting through a proof of concept. tween nodes show navigation paths. These graphs are further compared
The rest of this paper is organized as follows. Section 2 discusses with those that original apps on the market have.
initiatives for vetting mobile apps. Section 3 identifies existing gaps in The fields of mobile computing (exemplified with mobile apps) and
vetting things. Section 4 details the process of preparing things for IoT (exemplified with hundreds types or models of device) present many
vetting. This consists of identifying their duties along with analyzing the similarities in terms of affordability, wide use, and limited control; thus,
impact of vulnerabilities on these duties that things are expected to they should benefit from each other. Guidelines for vetting mobile apps
achieve. Prior to concluding in Section 6, conceptual details about the already exist and would constitute a good source for developing the
framework along with the results of the experiments are presented in same for IoT. The next section identifies the gaps in current research
Section 5. related to vetting IoT.

2. Vetting mobile apps 3. Gaps in vetting IoT

According to the US National Institute of Standards and Technology Compared to vetting mobile apps (Section 2), there is a major gap in
(NIST) [12], there is a need to secure mobile apps from vulnerabilities (𝒱IoT). In addition to Section 1’s obstacles that undermine IoT, this gap
and defects; apps are used by all people and all organizations. To satisfy is also due to the nature of IoT. IoTmixes physical processes with cyber
this need, a strict vetting process would ensure that mobile apps comply connectivity, making it different from other software-related disciplines
with an organization’s security requirements and are to a certain extent [16]. The first set of references that we reviewed are more concerned
free of (serious) vulnerabilities. These requirements could be related to with the security and privacy of IoT applications than with developing a
origin, data sensitivity, and target environment and could be decom­ comprehensive guide for vetting things that would reveal their vulner­
posed into (i) general with respect to some existing standards and best abilities. Meanwhile, the second set of references run tests to identify
practices, such as those specified by NIAP, OWASP, MITRE, and NIST, vulnerabilities of IoT devices that are already in operation [16]. There is
and (ii) specific with respect to some internal policies, regulations, and a consensus that IoT vastly impacts the way we view, use, and interact
guidelines. Factors that could be the cause of app vulnerabilities include with smart devices [17]. However, security remains a concern that could
design flaws and programming errors, which could have been inserted turn IoT misuse into a serious problem; such devices collect and use
intentionally or inadvertently [12]. Depending on the risk tolerance of users’ personal data without their permissions. McKinsey argues that
an organization, some vulnerabilities might be more serious than others, security may represent the greatest obstacle to IoT growth [18]. In [19],
suggesting the need for a contextualized vetting process. Vetting could Creager discusses methods for detecting IoT devices’ suspicious activ­
occur throughout an app’s lifecycle, which consists of development, ities. Devices are monitored for proper behavior, and those that show
acquisition, and deployment stages and would cover correctness testing, signs of having been interfered can have their behaviors mitigated and
source and binary code testing, and static and dynamic testing. their security issues eliminated.
In [8], Quirolgico et al. mention that millions of apps for mobile In [20], Celik et al. examine the security and privacy of IoT appli­
devices are available through commercial stores and open repositories. cations using a program-analysis technique. They use the examples of
Because of their low cost and widespread, the threats of the vulnera­ unlocked doors when nobody is at home and turned-off heaters in cold
bilities of these apps could be far greater than those of traditional weather. Both examples illustrate risks for the safety of people and their
computers. Because some vulnerabilities of mobile apps are unique, the assets, calling for appropriate and prompt measures. Despite those risks,
study insists on the urgency of developing a quick and cost efficient Celik et al. do not target IoT but, IoT platforms that are used to develop
vetting process. them and the IoT applications that manage them. The considered IoT
In [13], He et al. analyze malware attacks on smartphones and platforms are Samsung’s SmartThings, Apple’s HomeKit, OpenHAB,
suggest a two-level defense strategy that aims at preventing malware Amazon AWS IoT, and Android Things.
from entering smartphones and determining whether appropriate tools In line with the work of Celik et al., Fernandes et al. [21] look into
should detect and remove them. However, this may not always be suc­ data protection in the context of IoT, in general, and mobile apps asso­
cessful due to the rapid changes in malware behavior and therefore, ciated with IoT, in particular. They argue that permission-based access
more sophisticated tools need to be quickly developed while considering control over sensitive data is not enough once an app gains control over
smartphones’ limitations. He et al. recommend collaboration between data. Apps should make their data-use patterns explicit so that these
patterns are enforced (deviations are not allowed) at runtime when
sensitive data is used. Fernandes et al.’s system, known as FlowFence,
1
Atomic/Composite duty is similar to Component/Composite service adop­ enables robust and efficient flow control between sources and sinks in
ted by the service computing community [11] IoT applications by creating a data-handling mechanism for privacy

2
F. Masmoudi et al. Journal of Information Security and Applications 55 (2020) 102644

Table 1
Summary of the common reviewed state-of-the-art research that highlights the scopes, contexts, and main contributions. means fully covered, means partially
covered, and □ means uncovered.
Scope Context

Literature Security Privacy Vetting Software (Apps) Hardware IoT Contributions

He et al. [13] □ □ □ Future directions

Nagappan and Shihab [14] □ □ □ Reviews and recommendations
Chen et al. [15] □ □ □ MassVet technique
Celik et al. [20] □ Challenges and opportunities
Fernandes et al. [21] □ □ FlowFence system
Palavicini Jr. et al. [23] □ Semi-automatic firmware vetting
Siboni et al. [24] Security testbed framwork

protection purposes. FlowFence focuses partially on data leakage pre­ addition, the framework uses different open source tools to simulate real
vention where the overall IoT device behavior is not considered. operational environments and to run security tests. These tools may
In a 2018 report by KEYFACTOR [22], the authors discuss cases of need a vetting process for itself, continuous updates, and special
IoT devices that have been subject to attacks although these devices configuration from device-to-device, which makes the proposed frame­
were critical to humans’ lives. Vetting IoT devices would have helped work complex.
prevent or at least reduce such cases by revealing their vulnerabilities Table 1 summarizes the scopes, contexts, and contributions of the
ahead of time. In the healthcare domain, in 2017, the US Food and Drug reviewed state-of-the-art research. The table clearly demonstrates that
Administration (FDA) recalled 465K pacemakers after discovering se­ very few papers focus on the IoT vetting process by introducing security
curity flaws that could allow hackers to drain device batteries or send testing frameworks. The rest of the research either focuses on mobile
malicious instructions to modify a patient’s heartbeat. Vetting pace­ apps or introduces a partial vetting process or partially covers security
makers could have prevented such cases too. Similar news were reported and privacy aspects. In common, the reviewed literature considers
in the automotive industry when a Jeep Cherokee was hijacked turning testing IoT devices that are already in operation, although this research
off the transmission while the vehicle was on the freeway. aims for a vetting framework that detects security vulnerabilities before
Among the different works that we reviewed, the works of Palavicini taking the device into operation.
Jr. et al. [23] and Siboni et al. [24] overlap with our objectives. On the
one hand, Palavicini Jr. et al. apply symbolic analysis to vet, in a 4. Preparing things for vetting
semi-automated way, Industrial IoT (IIoT) firmware using angr, a UC
Santa Barbara binary analysis framework [25], and Mecanical Phish, a According to Crews and Mangal [27], the massive arrival of smart­
component from the same university’s cyber reasoning system, to phones and mobile apps has triggered a “mini-revolution” in the soft­
perform the semi-automated analysis of IIoT. The authors mention that ware engineering discipline. Some concepts, principles, and practices,
embedded systems and IIoT devices are rapidly increasing in number such as those related to testing, have been adjusted. Touchscreen ges­
and complexity. As a result, cyber-physical attacks have become omni­ tures, location awareness, and orientation need to be tested differently.
present, causing economic and physical damages. They also mention The same is valid when testing and vetting IoT smart devices. We expect
that the firmware for these systems has become difficult to analyze when that IoT features in terms of reduced size, restricted connectivity,
searching for malicious functionalities. Their approach consists of 3 continuous mobility, limited energy, constrained storage, and additional
steps: preparation of the firmware image for loading into the angr features that Kamrani et al. [28] discuss will trigger a similar “revolu­
framework, emulation for verification of discovered vulnerabilities, and tion”. “The things in the Internet of Things (IoT) can get personal. They can
analysis of the firmware sample “angr style”. be in your home, your car, and your body. They can make your living and
On the other hand, Siboni et al. [24] discuss a security testbed for IoT working space smart, and they can be dangerous to your health, safety, and
devices that iTrust Lab has developed. First, they report on the experi­ liberty. ⋅⋅⋅ Is our future a brave new world or a dystopian nightmare? Who
ence of using an IoT search engine, SHODAN [26], to discover vulner­ decides?” [29].
abilities of IoT devices. It is worth noting that this experience targeted
devices that were already in operation, while we insist that vetting aims 4.1. Overview
at detecting vulnerabilities prior to putting devices into operation. The
proposed testbed emulates different types of testing environments that By analogy to the NIST definition of the app vetting process [12], the
simulate the activity of multiple sensors and perform predefined and IoT vetting process would be a sequence of activities that an organiza­
customized security tests along with advanced security testing analysis. tion would carry out to declare if a thing is “clean” with respect to some
Siboni et al. consider 4 security aspects of IoT devices: architecture, IoT safety and security requirements. Our sequence of activities repre­
which investigates attacks on hardware and software; network connec­ sented in Fig. 1 would consist of (i) defining things’ duties that would be
tivity, which investigates attacks on data distribution; data collection, subject to vetting, (ii) identifying the vulnerabilities that would affect
which that investigates data collection with regard to privacy invasion these duties, (iii) analyzing the impact of these vulnerabilities on these
and information theft; and, finally, countermeasures and mitigation, duties, and (iv) developing guidelines and/or recommending techniques
which that investigate how to reduce the security and privacy risks that to address these vulnerabilities. The first two steps in Fig. 1 are already
IoT devices could cause. According to Siboni et al., the proposed discussed in [9,10]. Therefore, this article focuses on the impact of
framework requires further improvements to support the full opera­ vulnerabilities on duties to initiate the development of guidelines
tional capability so that the entire IoT environment is considered. In against these vulnerabilities.

Fig. 1. General representation of the 𝒱IoTprocess.

3
F. Masmoudi et al. Journal of Information Security and Applications 55 (2020) 102644

1. Sensing, Actuating, Communicating: sensed data are passed on to

actuating; and the data that result from actuation are passed on to
communicating for distribution.
2. Sensing, Actuating: sensed data are passed on to actuating; and the
data that result from actuation are finals.
3. Sensing, Communicating: sensed data are passed on to communi­
cating for distribution.
4. Actuating, Communicating: data that result from actuating are
passed on to communicating for distribution.

Fig. 3 is the duty model representation referring to one super-class,

Duty, which would describe the common characteristics of all the
duties, such as identity (id_duty), name (name_duty), type (type), and re­
sources to consume when performing the duty (res). Duty is specialized
Fig. 2. Atomic duties of a thing. into Sensing, Actuating, and Communicating. Sensing class describes ele­

Fig. 3. Duty model.

4.2. Duties of things ments such as frequency (freq_sensing) and sensed data (data). Actuating
class describes elements such as necessary time (latency) and actions
In a previous work [10,30], we identified 3 atomic duties that would (actions) required to take over data (data). Communicating class describes
capture a thing’s capabilities in terms of sensing (collecting and tempo­ elements such as time (timestamp), performed action (action), which
rarily storing data), actuating (processing/acting upon data), and could be either “receive” or “send”, and the involved things (things) in
communicating (sharing/distributing data). A duty is either enabled or this duty. Data class of a thing (thing) describes elements such as the
disabled ((0,1) in Fig. 2) according to the requirements and needs of the attribute (attribute, e.g., response time), value (value_data), and validity
under-development IoT applications. Our duty categorization overlaps (validity). Resource class describes elements such as identity (id_res), name
to a certain extent with first, Celik et al.’s sensor-computation-actuator (name_res), and value (value_res). Finally, the Thing class defines the
cycle that structures the design of IoT systems’ apps [20] and second, the characteristics of the thing that performs duties such as identity
IoT device capabilities of [31]. (id_thing) and description (des_thing).
Simply put, a thing senses the cyber-physical surrounding so that it
generates and (temporarily) stores (raw) data; a thing actuates data
including those that are sensed; and a thing communicates with the 4.3. Analysis of vulnerabilities of duties
cyber-physical surrounding the sensed and/or actuated data. Accepting
data and/or commands from external parties (e.g., other things) is also As per Fig. 1, our 𝒱IoTprocess proceeds with defining the duties of
taken care of by the communicating duty, but this process is not further things and then, identifying potential vulnerabilities that could nega­
discussed in this report. It is worth noting that a thing’s sensing, actu­ tively impact these duties. Although some vulnerabilities that could be
ating, and communicating duties can be composed together as per the sources of attacks have already been identified in the context of the
following 4 representative cases (other cases, such as Communicating, OWASP IoT project [32] (Table 2), we find these vulnerabilities generic
Actuating and Communicating, Actuating, Sensing are not discussed):

4
F. Masmoudi et al. Journal of Information Security and Applications 55 (2020) 102644

Table 2 Table 3
Common security vulnerabilities in the IoT paradigm in the context of the Sample of vulnerabilities of things per duty.
OWASP IoT project. Type ID Description
Vulnerability Attack surfaces Summaries
Atomic V1s Changing sensing frequency(ies) without approval
Username Administrative Ability to collect a set of valid usernames V2s Tampering sensing’s approved resource consumption-level so,
enumeration interface by interacting with the authentication this over consumption gets unnoticed
mechanism V3s Infinite suspension of sensing duty
Device Web V4a Changing actuating frequency without approval
interface
V5a Altering inputs purposely
Cloud interface
Mobile V6a Infinite suspension of actuating duty
applications V7c Communicating differently from the claimed frequency
Weak passwords Administrative Ability to set account passwords to ’1234’ V8c Using a different protocol from what is claimed
interface or ’123456’, for example
V9c Unauthorized party proceeds with communicating using a
Device Web Usage of pre-programmed default
non-acceptable device
interface passwords c
V10 Interrupting communicating without resumption
Cloud interface
Mobile Composite SA
V11 Changing sensing frequency(ies) without approval or (un)
applications synchronized sensing and actuating though it is not mandatory
Account lockout Administrative Ability to continue sending
SA
V12 Interrupting sensing or actuating without resumption
interface authentication attempts after 3 - 5 failed SA
V13 Altering sensed input, which would impact the sensed input to
login attempts actuate
Device Web V14
SC Changing sensing frequency without approval, which would
interface impact the frequency of the sensed content to communicate
Cloud interface SC
V15 Interrupting sensing or communicating without resumption
Mobile Unauthorized party proceeds with sensing and communicating
V16
SC
applications configuration using a non-acceptable device
Unencrypted Device network Network services are not properly AC Changing actuating frequency(ies) without approval or (un)
V18
services services encrypted to prevent eavesdropping or synchronized actuating and communicating though it is not
tampering by attackers mandatory
⋮ ⋮ ⋮ AC Interrupting actuating or communicating without resumption
V19
AC
V20 Altering received input

and not related to the duties of things. To address this limited focus, we AC
V21 Unauthorized party proceeds with communicating using a
non-acceptable device
have developed different questions2 that would expose vulnerabilities, SCC
V22 Change in sensing or communicating frequency or resource
as per Table 3, where Vix is the ith vulnerability related to either an atomic consumption of the sensing duty
or a composite duty x, and x ⊂ {s, a, c, sac, sa, sc, ac} (s: sensing, a: SCC
V23 Infinite suspension of sensing duty or the communicating duty
actuating, and c: communicating). Only the questions for atomic duties in the sensing duty or the communicating duty in one of the
are presented. two things
SCC
V24 Altering received input
Questions (Q) that can be raised during the vetting of sensing include
but are not limited to Q1: does sensing target living (e.g., persons) and/
or non-living things (e.g., rooms)? What is the sensing about (e.g., communicating use? Also, is this protocol secured? Q2: what interaction
ambient temperature, wind speed, and heartbeat)? Q2: does sensing mode does communicating use (e.g., synchronous versus asynchronous
target indoor, outdoor, or both? Q3: what is the frequency of sensing (e. and point-to-point versus multi-point?) Q3: what is the frequency of
g., continuously, at regular intervals, or trigger-based)? Q4: who con­ communicating (e.g., continuously, at regular intervals, or trigger-
figures sensing (e.g., frequencies, service periods, or authorized re­ based)? Q4: who configures communicating in terms of frequencies,
cipients)? Does configuration need to occur from a specific location and/ mode, service, etc.? Does configuration have to happen from a specific
or using a specific device? Q5: what is the resource consumption level of location and/or using a specific device? Q5: to whom does thing send
sensing? Also, is there any threshold that would indicate over­ data (e.g., persons, systems, or both)? Are there guarantees that data
consumption and hence, trigger alarms? And, Q6: are there traces of recipients do not share it further without approval? Q6: is data
tracking sensing using logs, for example? If yes, how are the traces communicated immediately with authorized parties or cached for later
safeguarded? sharing? Also, if data is classified, what measures are used for achieving
Questions (Q) that can be raised during the vetting of actuating this? Q7: what is the resource consumption level of communicating?
include but are not limited to Q1: can a thing cancel and/or compensate Also, is there any threshold that would indicate over-consumption and
the outcomes of actuating? If yes, does it need any approval? Q2: what is hence, trigger alarms? And, Q8: are there traces of tracking communi­
the frequency of actuating (e.g., continuously, at regular intervals, or cating using logs, for example? If yes, how are these traces safeguarded?
trigger-based)? Q3: who configures actuating in terms of frequencies,
service periods, etc.? Does configuration have to happen from a specific
location and/or using a specific device? Q4: what inputs does actuating 4.4. Impact of vulnerabilities on duties
require? What outputs does actuating produce? Q5: are there traces of
tracking actuating using logs, for example? If yes, how are these traces In this part of the paper, we examine the impact of vulnerabilities on
safeguarded? And, Q6: what is the resource consumption level of actu­ atomic and composite duties. To this end, we capture the successful
ating? And, is there any threshold that would indicate overconsumption completion of a duty using a state diagram and then, enrich this diagram
and hence, trigger alarms? with special notations to illustrate how a duty could deviate from this
Questions (Q) that can be raised during the vetting of communicating completion because of vulnerabilities. We see deviations as vulner­
include but are not limited to Q1: what interaction protocol does ability’s side effects.

4.4.1. Atomic duties

2
The questions are part of a tip-sheet similar to the one available at https:// Sensing. Fig. 4 is a state diagram that tracks the progress of the
tinyurl.com/thpzmc3. sensing duty towards successful completion. Using this diagram’s

5
F. Masmoudi et al. Journal of Information Security and Applications 55 (2020) 102644

Fig. 4. Enriched state diagram of the sensing duty.

Fig. 5. Enriched state diagram of the actuating duty.

Fig. 6. Enriched state diagram of the communicating duty.

states and transitions, we develop what we refer to as normal-progress consumption level (V2s ), and we associate indefinite on-hold with
cycle of a duty (plain lines in Fig. 4). We also enrich this diagram 1 vulnerability, namely, infinite suspension of sensing (V3s ).
with necessary states and/or transitions to represent deviations from Actuating. Similar to sensing, we develop the normal-progress cycle of
this cycle that correspond to side effects of vulnerabilities (dashed the actuating duty (plain lines in Fig. 5) and identify the side effects
lines in Fig. 4). of vulnerabilities that this duty could be subject to (dashed lines in
On the one hand, in a normal-progress cycle, the states are not- Fig. 5).
activated (the sensing duty is not available), activated (the sensing In a normal-progress cycle, on the one hand, the states are not-
duty is being configured), operated (a composed state where the activated (the actuating duty is not available), activated (the actu­
sensing is taking place in terms of reading and buffering3), done (the ating duty is being configured), operated (a composed state where the
sensing is complete), and interrupted (the sensing is put on-hold and actuating is taking place in terms of receiving and processing data),
then resumed). On the other hand, the transitions connecting these done (the actuating is complete), and interrupted (the actuating is put
states together are activation, start, sending, interruption, resumption, on-hold and then resumed). On the other hand, the transitions con­
and completion. necting these states together are activation, sending, start, interruption,
Vulnerability’s side effects correspond to 2 states, namely, acti­ resumption, and completion.
vated (the sensing duty is misconfigured) and interrupted (the sensing Vulnerability’s side effects correspond to 3 states, namely, acti­
is put on-hold indefinitely), and 1 transition, namely, misconfigura­ vated (the actuating duty is being misconfigured), processed (the
tion. We associate misconfiguring sensing with 2 vulnerabilities, actuating operates differently), and interrupted (the actuating is put
namely, change in sensing frequency (V1s ) and tampering resource- on-hold indefinitely), and 2 transitions, namely, misconfiguration and
condition-unmet. We associate misconfiguring actuating with 1
vulnerability, i.e., change in actuating frequency (V4a ); we associate
operating differently of the actuating duty with 1 vulnerability, i.e.,
3
We adjust the sensing behavior reported in [33].

6
F. Masmoudi et al. Journal of Information Security and Applications 55 (2020) 102644

Fig. 7. Enriched state diagram of the sensing,actuating composite duty.

Fig. 8. Enriched state diagram of the sensing,communicating composite duty.

altering received input (V5a ); and we associate indefinite on-hold with 4.4.2. Composite duties
1 vulnerability, i.e., the infinite suspension of actuating (V6a ).
Communicating. Like sensing and actuating, Fig. 6 shows the (Sensing,Actuating). Fig. 7 shows the normal-progress cycle of the
normal-progress cycle of the communicating duty (plain lines in sensing,actuating composite duty (plain lines in Fig. 7) and the side
Fig. 6) and the side effects of vulnerabilities that this duty could be effects of vulnerabilities that this composite duty could be subject to
subject to (dashed lines in Fig. 6). (dashed lines in Fig. 7).
In a normal-progress cycle, on the one hand, the states are not- In a normal-progress cycle, on the one hand, the states are S:not-
activated (the communicating duty is not available), activated (the activated (the sensing duty is not available), S:activated (the sensing
communicating duty is being configured), operated (a composed state duty is being configured), S:read (the sensing duty is taking place in
where the communicating is taking place in terms of receiving and terms of reading), A:processed (the actuating duty takes over from the
sending data), done (the communicating is complete), and interrupted sensing duty after receiving the sensed data for processing), A:done
(the communicating is put on-hold and then, resumed). On the other (the actuating duty is complete), and interrupted (either the sensing
hand, the transitions connecting these states together are activation, duty or the actuating duty is put on-hold and then resumed). On the
start, relay, interruption, resumption, and completion. other hand, the transitions connecting these states together are
Vulnerability’s side effects correspond to 3 states, namely, acti­ activation, start, sending, interruption, resumption, and completion.
vated (the communicating duty is being misconfigured), sent (the Vulnerability’s side effects correspond to 3 states, namely, S:acti­
community duty shares data with unauthorized party), and inter­ vated (the sensing duty is misconfigured), S/A: interrupted (either the
rupted (the communicating is put on-hold indefinitely), and 2 tran­ sensing duty or the actuating duty is put on-hold indefinitely), and A:
sitions, namely, misconfiguration and unacceptable-party. We associate processed (the actuating operates differently), and 2 transitions,
misconfiguring communicating with a change in communicating namely, S:misconfiguration and A:condition unmet. We associate mis­
frequency (V7c ) and a change in communicating protocol (V8c ); we configuring sensing with a change in sensing frequency (V11 SA
); we
associate the misuse operation of the thing’s duty with a connected associate indefinite on-hold with 1 vulnerability, namely, the infinite
unauthorized party (V9c ); and we associate the indefinite on-hold suspension of sensing or actuating (V12 SA
); and we associate operating
c
with the infinite suspension of communicating (V10 ). differently of the actuating duty with 1 vulnerability, namely,
SA
altering sensed input (V13 ).

Fig. 9. Enriched state diagram of the sensing,communicating,communicating composite duty.

7
F. Masmoudi et al. Journal of Information Security and Applications 55 (2020) 102644

Fig. 10. Modules of the 𝒱IoTframework.

(Sensing,Communicating). Fig. 8 shows the normal-progress cycle of activated (the sensing duty is misconfigured), C1: received (the
the sensing,communicating composite duty (plain lines in Fig. 8) and communicating duty in the first thing shares data with unauthorized
the side effects of vulnerabilities that this composite duty could be party), C2: received (the communicating duty in the second thing
subject to (dashed lines in Fig. 8). shares data with unauthorized party) and S/C1/C2: interrupted
In a normal-progress cycle, on the one hand, the states are S:not- (either the sensing duty or the communicating duty in one of the two
activated (the sensing duty is not available), S:activated (the sensing things is put on-hold indefinitely), and 3 transitions, namely, S:mis­
duty is being configured), S:read (the sensing duty is taking place in configuration, C1: unacceptable party and C2: unacceptable party. We
terms of reading), C:received (the communicating duty is taking place associate misconfiguring sensing with the change in sensing or
in terms of receiving data), and S/C: interrupted (either the sensing communicating frequency, or resource consumption (V22 SCC
); we
duty or the communicating duty is put on-hold and then, resumed), associate an indefinite on-hold with 1 vulnerability, i.e., the infinite
and C:done (the communicating duty is complete). On the other suspension of sensing duty or the communicating duty within the
hand, the transitions connecting these states together are activation, sensing duty or the communicating duty in one of the two things
start, sending, interruption, resumption, and completion. SCC
(V23 ), and operating differently of the communicating duty in one of
Vulnerability’s side effects correspond to 3 states, namely, S:acti­ the two things with altering received input (V24SCC
).
vated (the sensing duty is misconfigured), C:received (the communi­
cating duty shares data with unauthorized party) and S/C:interrupted 5. Guiding framework for vetting things
(either the sensing duty or the communicating duty is put on-hold
indefinitely), and 2 transitions, namely, S:misconfiguration and C: This section presents our proposed 𝒱IoTframework (Section. 5.1)
unacceptable party. We associate misconfiguring sensing with the and, then, the system that was implemented accordingly (Section. 5.2).
SC
change in sensing frequency by an unauthorized person (V14 ); we
associate indefinite on-hold with 1 vulnerability, i.e., infinite sus­ 5.1. Architecture
SC
pension of sensing or communicating (V15 ); and we associate the
misuse operation of the communicating duty with a connected un­ Fig. 10 identifies the modules that constitute our framework for
SC
authorized party (V16 ). vetting things. Vetting ensures that a thing’s behavior is in line with its
(Sensing,Communicating,Communicating). Fig. 9 shows the expected behavior (Section 4.4) when tested against vulnerabilities that
normal-progress cycle of the sensing, communicating, communi­ its duties can be subject to (Section 4.3). The progress of completing the
cating composite duty (plain lines in Fig. 9) and the side effects of vetting in the framework goes over 5 stopovers. safe
vulnerabilities that this composite duty could be subject to (dashed In stopover , the vet engineer enriches the description of the thing
lines in Fig. 9). to vet with details about its atomic/composite duties and then, publishes
In a normal-progress cycle, on the one hand, the states are S:not- this description on the repository of things. In stopover ②, the vet engi­
activated (the sensing duty is not available), S:activated (the sensing neer deploys the thing on the testbed allowing to run different tests and
duty is being configured), S:read (the sensing duty is taking place in to support interactions with peers and the Tester module. The deploy­
terms of reading data), C1:received (the actuating duty is taking place ment is taken care by the deployer module that also extracts the thing’s
in the first thing in terms of receiving data), C1/C2:synchronized (the descriptions of duties from the repository of things and submits them to
communicating duty is synchronized between the first and the sec­ the tester module, thereby initiating the third stopover ③. The tester
ond things), C2:received (the actuating duty is taking place in the module (i) retrieves the list of known vulnerabilities for the thing based
second thing in terms of receiving data), C2:done (the communicating on its duties, (ii) selects the necessary tests4 associated with these vul­
duty is complete) and S/C1/C2:interrupted (either the sensing duty or nerabilities and finally, (iii) runs the selected tests. During the tests, the
the communicating duty in one of the two things is put on-hold and testbed is constantly monitored thanks to the tester module that tracks
then resumed). On the other hand, the transitions connecting these and stores both the states of the thing and the events in a dedicated log.
states together are activation, sending, start, interruption, resumption,
relay,and synchronizing.
Vulnerability’s side effects correspond to 4 states, namely, S: 4
Tests could be automatic, semi-automatic, or manual.

8
F. Masmoudi et al. Journal of Information Security and Applications 55 (2020) 102644

Fig. 11. Things deployed in the Node-RED testbed of the 𝒱IoTframework.

Fig. 12. JSON request submission via MQTTLens.

Upon completing the tests that correspond to stopover ④, the following other parts (i.e., the tester and the inspector) were implemented using
occurs; the inspector module pulls the behavior of the under-vetting Python language (Spyder Tool V3.7 [36]).
thing from the log and its expected behavior as defined in Section 4.4. According to Fig. 11 (the testbed), we used six node types (i)
Then, it compares the behavior built upon the tests to the expected one: mqtt input node allows users to configure an MQTT service and the topic
if no vulnerabilities’ side-effects are detected then the thing is declared to publish about. As a result, a node could receive messages with the
safe; otherwise, it is declared unsafe. exact same JSON string that was sent via MQTTLens (as shown in
Fig. 12). (ii) JSON node which parses messages that MQTT input node
5.2. Implementation and experiments generates when it receives an MQTT message and converts it to a
JavaScript object. (iii) function node receives a message object as input
A system demonstrating the 𝒱IoTframework was developed using and can return 0 or many message objects as output. A message object
Python 3, Java 14, JavaScript, MQTTLens [34] and Node-RED [35]. must have at least a payload property (msg.payload). (iv) debug node
MQTTLens is an add-on for the Chrome browser allowing to publish displays the received messages within the flow in the panel. (v) text node
messages to an MQTT broker, to subscribe to MQTT topics and to receive shows messages within the flow on a slider and as a text string. Finally,
messages. This latter is a flow-based programming paradigm in which a (vi) change node changes a message payload or adds new properties. All
variety of processes called “nodes” are connected together to form ap­ calls to the testbed’s flow are tracked and stored by the Tester in the log
plications called “flows”, instead of writing code. Node-RED runs on of the active states database consisting on a CSV file. Next, the Inspector
local workstations, the cloud, and edge devices. It has become an ideal module compares the stored active states to the expected ones to check
tool for the Raspberry Pi and other low-cost hardware. Node-RED run­ the possible existence of any vulnerability.
time is built on top of Node.js and takes its full advantage in terms of To illustrate a vulnerability, we considered sensing duty and set its
event-driven, non-blocking I/O model. It can be used to build and deploy frequency to every time interval. A vulnerability would consist of
applications due to its browser-based editor by wiring hardware devices, communicating differently from the claimed frequency (V1s ) by using the
APIs and online services. Node-RED implements the testbed and in­ “change” node. Thus, during the experiment, the Python program
cludes an MQTT server so that IoT devices could be easily connected via related to sensing duty was launched in order to check whether the
port 1883 using Transport Layer Security (TLS). We considered sensors stored states were as expected. If not, it displayed a notification of a
as things to vet. The vet engineer interacts with the sensors running on vulnerability. In the following, we considered 2 series of vulnerability-
the testbed using JSON requests via the deployer module (i.e., related experiments: vetting duration and vetting efficiency. These ex­
MQTTLensTool). The way a sensor was implemented consists of 3 con­ periments run on Windows 10 Asus TUF Gaming i7-8750H @ 2.206Hz,
nected objects: living.lights, bed.alarm, and kitchen.aircondition. They were 12Go. Before all, the 𝒱IoT engineer sends a JSON request to a living.
created and deployed by the Node-RED server. On top of Node-RED, lights’s sensor via MQTTLens Tool as per Fig. 12.

9
F. Masmoudi et al. Journal of Information Security and Applications 55 (2020) 102644

framework in terms of precision/recall is explained by the presence of

the inspectorthat browses instantly the active states of the connected
things, where each is checked separately wether it is as expected or not.

6. Conclusion

This paper discussed the importance of vetting things prior to their

integration into systems, some of which are critical. The objective is to
verify claims of providers about the safety of things as some could be
subject to vulnerabilities that could be taken advantage of for non-
beneficial practices. The IoT paradigm imposes a new way of how
determining security, confidentiality, identity, and privacy should be
approached. By analogy to mobile apps vetting, we identified the vul­
nerabilities that could impede things’ duties from successful completion
and then illustrated how these vulnerabilities could be detected. The
Fig. 13. Latency-based 𝒱IoTframework performance. duties are referred to as sensing, actuating, and communicating; they

Table 4
Precision/recall results.
Vulnerabilities 1 2 3 4 5 6 7 8 9 10 Total

TV 1 1 0 1 1 1 1 1 1 1 9
FV 0 1 0 1 2 0 0 0 0 0 4
FN 0 0 1 0 0 0 0 0 0 0 1
Precision 1 0.5 0 0.5 0.33 1 1 1 1 1 0.9
Recall 1 1 0 1 1 1 1 1 1 1 0.9

1. Vetting duration: to assess the framework’s time performance for have been examined separately in the context of atomic duties and then
detecting vulnerabilities. We computed the time of sending a request combined, in the context of composite duties.
and the time of detecting a vulnerability. We considered 10 requests The outcome of vetting either ensures thing’s trustworthiness when
with an 2-second time interval between each one, where each their behaviors meet the expected execution cycles or declares them as
request was tampered by changing frequency vulnerability. Then, we unsafe. Future work will target completing the testbed by vetting more
recorded the time that the inspector took to process the vetting so, things and more duties, whether atomic and composite, and examining
that vulnerabilities were detected. As per Fig. 13, we noticed that the concept of provable vetting so that things would be accountable for
launching violations does not affect much the vetting latency since the duties they perform.
the active states are checked by the Inspector module.
2. Vetting efficiency: to assess the frameworks capacity to detect vul­ Author Statement
nerabilities based on precision and recall. The precision expresses the
ability of the Inspector module to detect correct (i.e., that actually We the authors of the manuscript A Guiding Framework for Vetting
happened) vulnerabilities only. It is defined by calculating the ratio the Internet of Things declare that this manuscript is original, has not
of the number of true vulnerabilities among the launched vulnera­ been published before and is not currently being considered for publi­
bilities as per Eq. (1), where TV is the number of true vulnerabilities, cation elsewhere.
and FV is the number of detected false vulnerabilities. We confirm that the manuscript has been read and approved by all
TV named authors and that there are no other persons who satisfied the
Precision = (1) criteria for authorship but are not listed. We further confirm that the
TV + FV
order of authors listed in the manuscript has been approved by all of us.
We understand that the Corresponding Author is the sole contact for
The recall is defined as the ratio of true vulnerabilities over the the Editorial process. He/she is responsible for communicating with the
total number of detected vulnerabilities by the Inspector, as per Eq. other authors about progress, submissions of revisions and final
(2), where FN is the number of missing true vulnerabilities that are approval of proofs.
not detected. It represents the probability of detecting all true vul­ Fatma Masmoudi, Zakaria Maamar, Mohamed Sellami, Ali Ismail
nerabilities among the detected vulnerabilities. Awad and Vanilson Burégio
TV
Recall = (2)
TV + FN
Declaration of Competing Interest

In these experiments, we relied on the vulnerabilities used in the The Authors, Fatma Masmoudi, Zakaria Maamar, Mohamed Sellami,
first series (Fig. 13). Then, we observed the behavior of the Inspector Ali Ismail Awad, and Vanilson Arruda Burégio, declare that there is no
module in terms of number of detected true/false vulnerabilities, and conflict of interest.
the number of non-detected ones to determine the precision and
recall values (Table 4). References

As per Table 4, we noted that the 𝒱IoT framework is efficient by [1] Gunduz MZ, Das R. Cyber-security on smart grid: Threats and potential solutions.
achieving a precision close to 1 in most of vulnerabilities (same for recall Computer Networks 2020;169:107094. https://doi.org/10.1016/j.
comnet.2019.107094.
values). We also noticed that there is only one missing vulnerability [2] Statista. Mobile app usage—Statistics & Facts. 2019.[Online; accessed 20-March-
among 10 (the third vulnerability). The efficiency of the 𝒱IoT 2020]; https://www.statista.com/topics/1002/mobile-app-usage/

10
F. Masmoudi et al. Journal of Information Security and Applications 55 (2020) 102644

[3] Ashford W. Less than half of firms able to detect IoT breaches. 2019.[Online; [20] Celik ZB, Fernandes E, Pauley E, Tan G, McDaniel P. Program analysis of
accessed 16-June-2020]; https://www.computerweekly.com/news/252455834 commodity iot applications for security and privacy: Challenges and opportunities.
/Less-than-half-of-firms-able-to-detect-IoT-breaches-study-shows ACM Comput Surv 2019;52(4). https://doi.org/10.1145/3333501.
[4] Middleton P, Tully J, Kjeldsen P. Forecast: The internet of things, worldwide, 2013. [21] Fernandes E, Paupore J, Rahmati A, Simionato D, Conti M, Prakash A. FlowFence:
2013.Report of Gartner. [ Online; accessed 19-August-2020]; http://www.gartner. Practical Data Protection for Emerging IoT Application Frameworks. Proceedings
com/doc/2625419/ of the 25th USENIX Security Symposium (USENIX Security’2016). 2016.Austin,
[5] Ko E, Kim T, Kim H. Management platform of threats information in iot TX, USA
environment. Journal of Ambient Intelligence and Humanized Computing 2018;9 [22] KEYFACTOR. The Impact of Unsecured Digital Identities. 2020.[Online; accessed
(4):1167–76. 08-March-2020]; https://www.keyfactor.com/
[6] DZone. The Internet of Things, Application, Protocols, and Best Practices. Tech. [23] Palavicini Jr. G, Bryan J, Sheets E, Kline M, San Miguel J. Towards Firmware
Rep.. 2017.https://dzone.com/guides/iot-applications-protocols-and-best-practi Analysis of Industrial Internet of Things (IIoT) - Applying Symbolic Analysis to IIoT
ces(visited in May 2017) Firmware Vetting. Proceedings of the 2nd International Conference on Internet of
[7] Elrawy MF, Awad AI, Hamed HFA. Intrusion detection systems for IoT-based smart Things, Big Data and Security, Porto, Portugal (IoTBDS’2017). 2017. p. 470–7.
environments: a survey. Journal of Cloud Computing 2018;7(1):21. https://doi. https://doi.org/10.5220/0006393704700477.
org/10.1186/s13677-018-0123-6. [24] Siboni S, Sachidananda V, Meidan Y, Bohadana M, Mathov Y, Bhairav S, et al.
[8] Quirolgico S, Voas J, Kuhn R. Vetting Mobile Apps. IT Professional 2011;13(4). Security testbed for internet-of-things devices. IEEE Transactions on Reliability
[9] Maamar Z, Kajan E, Asim M, Baker Shamsa T. Open challenges in vetting the 2019;68(1):23–44. https://doi.org/10.1109/TR.2018.2864536.
internet-of-things. Internet Technology Letters 2019;2(5):e129. https://doi.org/ [25] Shoshitaishvili Y, Wang R, Salls C, Stephens N, Polino M, Dutcher A, et al. SoK:
10.1002/itl2.129. (State of) The Art of War: Offensive Techniques in Binary Analysis. Proceedinmgs
[10] Qamar A., Asim M., Maamar Z., Saeed S., Baker T.. A quality-of-things model for of the Symposium on Security and Privacy, San Jose, CA, USA (SP’2016). 2016.
assessing the internet-of-things’ nonfunctional properties. Transactions on p. 138–57. https://doi.org/10.1109/SP.2016.17.
Emerging Telecommunications Technologiese366810.1002/ett.3668. [26] The search engine for the Internet-connected devices. [Online; accessed 19-August-
[11] Service Research Challenges and Solutions for the Future Internet - S-Cube - 2020] https://www.shodan.io.
Towards Engineering, Managing and Adapting Service-Based Systems. In: [27] Crews B., Mangal S.. IoT and it’s Impact on Testing. [Online; accessed 19-July-
Papazoglou M, Pohl K, Parkin M, Metzger A, editors. Lecture Notes in Computer 2020]; https://www.getzephyr.com/resources/whitepapers/iot-and-its-impact
Science, 6500. Springer; 2010. -testing.
[12] Ogata M, Franklin J, Voas J, Sritapan V, Quirolgico S. Vetting the Security of [28] Kamrani F, Wedling M, Rodhe I. Internet of Things: Security and Privacy Issues.
Mobile Applications. DRAFT NIST Special Publication 800-163 (Revision 1). April Tech. Rep. FOI Swedish Defence Research Agency, Defence and Security, Systems
2019. https://doi.org/10.6028/NIST.SP.800-163r1.[Online accessed 20-March- and Technology; 2019.
2020] [29] Orman H. You Let That In? IEEE Internet Computing 2017;21(3).
[13] He D, Chan S, Guizani M. Mobile Application Security: Malware Threats and [30] Maamar Z, Baker T, Sellami M, Asim M, Ugljanin E, Faci N. Cloud vs edge: Who
Defenses. IEEE Wireless Communications 2015;22(1). serves the Internet-of-Things better? Internet Technology Letters 2018;1(5):e66.
[14] Nagappan M, Shihab E. Future Trends in Software Engineering Research for Mobile https://doi.org/10.1002/itl2.66.
Apps. Proceedings of the 23rd IEEE International Conference on Software Analysis, [31] NIST, NSA. CISCO IoT Industrial Ethernet and Connected Grid Switches running
Evolution, and Reengineering (SANER’2016) vol. 5.. 2016.Suita, Osaka, Japan IOS. 2018.Downloaded in November Report Number: CCEVS-VR-10692-2015,
[15] Chen K, Wang P, Lee Y, Wang X, Zhang N, Huang H, et al. Finding Unknown Malice March 11 2016
in 10 Seconds: Mass Vetting for New Threats at the Google-play Scale. Proceedings [32] OWASP. OWASP Internet of Things. 2020.[Online; accessed 08-March-2020];
of the 24th USENIX Conference on Security Symposium (SEC’2015). 2015. https://owasp.org/www-project-internet-of-things/
Washington, D.C., USA [33] Costa B, Pires P, Delicato F, Li W, Zomaya A. Design and Analysis of IoT
[16] Ali B, Awad AI. Cyber and physical security vulnerability assessment for IoT-Based Applications: A Model-Driven Approach. Proceedings of the 2016 IEEE 14th
smart homes. Sensors 2018;18(3):817. https://doi.org/10.3390/s18030817. International Conference on Dependable, Autonomic and Secure Computing, the
[17] Hassan AM, Awad AI. Urban transition in the era of the Internet of Things: Social 14th International Conference on Pervasive Intelligence and Computing, and the
implications and privacy challenges. IEEE Access 2018;6:36428–40. https://doi. 2nd International Conference on Big Data Intelligence and Computing and Cyber-
org/10.1109/ACCESS.2018.2838339. Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech’2016).
[18] Bauer H, Burkacky O, Knochenhauer C. Security in the Internet of Things. 2020. 2016.Auckland, New Zealand
[Online; accessed 08-March-2020]; http://www.mckinsey.com/ [34] MQTTLens. MQTTLens. 2020.[Online; accessed 19-July-2020]; https://chrome.
[19] Creager L., How can Anomalous IoT Device Activity be Detected? http://www. google.com/webstore/detail/mqttlens/hemojaaeigabkbcookmlgmdigohjobjm
tinyurl.com/y9v5lgfq Downloaded in November 2018. [35] Red N. Node Red. 2020.[Online; accessed 19-July-2020]; https://nodered.org/
[36] Spyder. Spyder Website. 2020.[Online; accessed 22-July-2020]; https://www.
spyder-ide.org/

11