Big IoT Data Analytics - Architecture, Opportunities, and Open Research Challenges

This article has been accepted for publication in a future issue of this journal, but has not been
fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2017.2689040, IEEE Access
Big IoT Data Analytics: Architecture,

Opportunities, and Open Research Challenges
Mohsen Marjani, Fariza Nasaruddin*, Abdullah Gani, Fellow, IEEE, Ahmad Karim, Ibrahim Abaker
Targio Hashem*, Aisha Siddiqa, Ibrar Yaqoob Member, IEEE
Abstract—Voluminous amounts of data have been produced The widespread popularity of IoT has made big data
since the past decade as the miniaturization of Internet of analytics challenging because of the processing and
things (IoT) devices increases. However, such data are not collection of data through different sensors in the IoT
useful without analytic power. Numerous big data, IoT, and
analytics solutions have enabled people to obtain valuable environment. The International Data Corporation (IDC)
insight into large data generated by IoT devices. However, report indicates that the big data market will reach over
these solutions are still in their infancy, and the domain lacks a US$125 billion by 2019 [3]. IoT big data analytics can be
comprehensive survey. This study investigates state-of-the-art defined as the steps in which a variety of IoT data are
research efforts directed toward big IoT data analytics. The examined [4] to reveal trends, unseen patterns, hidden
relationship between big data analytics and IoT is explained. correlations, and new information [5]. Companies and
Moreover, this study adds value by proposing a new
architecture for big IoT data analytics. Furthermore, big IoT individuals can benefit from analyzing large amounts of
data analytic types, methods, and technologies for big data data and managing huge amounts of information that can
mining are discussed. Numerous notable use cases are also affect 1businesses [6]. Therefore, IoT big data analytics aims
presented. Several opportunities brought by data analytics in to assist business associations and other organizations to
IoT paradigm are then discussed. Lastly, open research achieve improved understanding of data, and thus, make
challenges, such as privacy, big data mining, visualization, and efficient and well-informed decisions. Big data analytics
integration, are presented as future research directions.
enables data miners and scientists to analyze huge amounts
Index Terms— Big data, Internet of things, Data analytics, of unstructured data that can be harnessed using traditional
Distributed computing, Smart city. tools [5]. Moreover, big data analytics aims to immediately
extract knowledgeable information using data mining
I. INTRODUCTION techniques that help in making predictions, identifying
The development of big data and the Internet of things (IoT) recent trends, finding hidden information, and making
is rapidly accelerating and affecting all areas of decisions [7].
technologies and businesses by increasing the benefits for Techniques in data mining are widely deployed for both
organizations and individuals. The growth of data produced problem-specific methods and generalized data analytics.
via IoT has played a major role on the big data landscape. Accordingly, statistical and machine learning methods are
Big data can be categorized according to three aspects: (a) utilized. IoT data are different from normal big data
volume, (b) variety, and (c) velocity [1]. These categories collected via systems in terms of characteristics because of
were first introduced by Gartner to describe the elements of the various sensors and objects involved during data
big data challenges [2]. Immense opportunities are collection, which include heterogeneity, noise, variety, and
presented by the capability to analyze and utilize huge rapid growth. Statistics [8] show that the number of sensors
amounts of IoT data, including applications in smart cities, will be increased by 1 trillion in 2030. This increase will
smart transport and grid systems, energy smart meters, and affect the growth of big data. Introducing data analytics and
remote patient healthcare monitoring devices. IoT into big data requires huge resources, and IoT has the
capability to offer an excellent solution. Appropriate
Mohsen Marjani, Abdullah Gani, Aisha Siddiqa, Ibrahim Abaker Targio resources and intensive applications of the platforms are
Hashem, and Ibrar Yaqoob are with Department of Computer System and
Technology, Faculty of Computer Science and Information Technology, provided by IoT services for effective communication
University of Malaya, Kuala Lumpur, Malaysia among various deployed applications. Such process is
([email protected]). Fariza Nasaruddin is with Department of
Information System, Faculty of Computer Science and Information suitable for fulfilling the requirements of IoT applications,
Technology, University of Malaya, Kuala Lumpur, Malaysia. and can reduce some challenges in the future of big data
Ahmad Karim is with Department of Information Technology, Bahauddin
Zakariya University, Multan, Pakistan. analytics. This technological amalgamation increases the
possibility of implementing IoT toward a better direction.
Moreover, implementing IoT and big data integration
1
2169-3536 (c) 2016 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2017.2689040, IEEE Access
solutions can help address issues on storage, processing, areas may collect various kinds of data, such as
data analytics, and visualization tools. It can also assist in geographical, astronomical, environmental, and logistical
improving collaboration and communication among various data.
objects in a smart city [9]. Application areas, such as smart
ecological environments, smart traffic, smart grids, A large number of communication devices in the IoT
intelligent buildings, and logistic intelligent management, paradigm are embedded into sensor devices in the real
can benefit from the aforementioned arrangement. Many world. Data collecting devices sense data and transmit these
studies on big data has focused on big data management; in data using embedded communication devices. The
particular, big data analytics has been surveyed [10, 11]. continuum of devices and objects are interconnected
However, this survey focused on IoT big data in the context through a variety of communication solutions, such as
of the analytics of a huge amount of data. The contributions Bluetooth, WiFi, ZigBee, and GSM. These communication
of this survey are as follows. devices transmit data and receive commands from remotely
controlled devices, which allow direct integration with the
a) State-of-the-art research efforts conducted in terms physical world through computer-based systems to improve
of big data analytics are investigated. living standards.
b) An architecture for big IoT data analytics is
proposed. Over 50 billion devices ranging from smartphones, laptops,
c) Several unprecedented opportunities brought by sensors, and game consoles are anticipated to be connected
data analytics in the IoT domain are introduced. to the Internet through several heterogeneous access
d) Credible use cases are presented. networks enabled by technologies, such as radio frequency
e) Research challenges that remain to be addressed identification (RFID) and wireless sensor networks. [15]
are identified and discussed. mentioned that IoT could be recognized in three paradigms:
These contributions are presented from Sections 3 to 6. The Internet-oriented, sensors, and knowledge [16]. The recent
conclusion is provided in Section 7. adaptation of different wireless technologies places IoT as
the next revolutionary technology by benefiting from the
II. OVERVIEW OF IOT AND BIG DATA full opportunities offered by Internet technology.
An overview of IoT technologies and big data is provided B. Big data
before the discussion.
The volume of data generated by sensors, devices, social
A. IoT media, health care applications, temperature sensors, and
various other software applications and digital devices that
IoT offers a platform for sensors and devices to
continuously generate large amounts of structured,
communicate seamlessly within a smart environment and
unstructured, or semi-structured data is strongly increasing.
enables information sharing across platforms in a
This massive data generation results in ―big data‖ [17].
convenient manner. The recent adaptation of different
Traditional database systems are inefficient when storing,
wireless technologies places IoT as the next revolutionary
processing, and analyzing rapidly growing amount of data
technology by benefiting from the full opportunities offered
or big data [18]. The term ―big data‖ has been used in the
by the Internet technology. IoT has witnessed its recent
previous literature but is relatively new in business and IT
adoption in smart cities with interest in developing
[19]. An example of big data-related studies is the next
intelligent systems, such as smart office, smart retail, smart
frontier for innovation, competition, and productivity;
agriculture, smart water, smart transportation, smart
healthcare, and smart energy [12, 13]. McKinsey Global Institute [20] defined big data as the size
of data sets that are a better database system tool than the
IoT has emerged as a new trend in the last few years, where usual tools for capturing, storing, processing, and analyzing
mobile devices, transportation facilities, public facilities, such data [18]. ―The Digital Universe‖ study [21] labels big
and home appliances can all be used as data acquisition data technologies as a new generation of technologies and
equipment in IoT. All surrounding electronic equipment to architectures that aim to take out the value from a massive
facilitate daily life operations, such as wristwatches, volume of data with various formats by enabling high-
vending machines, emergency alarms, and garage doors, as velocity capture, discovery, and analysis. This previous
well as home appliances, such as refrigerators, microwave study also characterizes big data into three aspects: (a) data
ovens, air conditioners, and water heaters are connected to sources, (b) data analytics, and (c) the presentation of the
an IoT network and can be controlled remotely. Ciufo [14] results of the analytics. This definition uses the 3V‘s
stated that these devices ―talk‖ to one another and to central (volume, variety, velocity) model proposed by Gartner [2].
controlling devices. Such devices deployed in different The model highlights an e-commerce trend in data
management that faces challenges to manage volume or size
of data, variety or different sources of data, and velocity or linearly with the rapid increase in computational resources
speed of data creation. Some studies declare volume as a [19].
main characteristic of big data without providing a pure
definition [22]. However, other researchers introduced Big data analytics processes consume considerable time to
additional characteristics for big data, such as veracity, provide feedback and guidelines to users, whereas only a
value, variability, and complexity [23, 24]. The 3V‘s model, few tools [29] can process huge data sets within reasonable
or its derivations, is the most common descriptions of the amount of processing time. By contrast, most of the
term ―big data.‖ remaining tools use the complicated trial-and-error method
to deal with massive amounts of data sets and data
III. BIG DATA ANALYTICS heterogeneity [30]. Big data analytics systems exist. For
example, the Exploratory Data Analysis Environment [31]
Big data analytics involves the processes of searching a
is a big data visual analytics system that is used to analyze
database, mining, and analyzing data dedicated to improve
complex earth system simulations with large numbers of
company performance [25]. data sets.
Big data analytics is the process of examining large data A. Existing analytics systems
sets that contain a variety of data types [4] to reveal unseen Different analytic types are used according to the
patterns, hidden correlations, market trends, customer requirements of IoT applications [32]. These analytic types
preferences, and other useful business information [5]. The are discussed in this subsection under real-time, off-line,
capability to analyze large amounts of data can help an memory-level, business intelligence (BI) level, and massive
organization deal with considerable information that can level analytics categories. Moreover, a comparison based on
affect the business [6]. Therefore, the main objective of big analytics types and their levels is presented in Table 1.
data analytics is to assist business associations to have
Real-time analytics is typically performed on data collected
improved understanding of data, and thus, make efficient
from sensors. In this situation, data change constantly, and
and well-informed decisions. Big data analytics enables data
rapid data analytics techniques are required to obtain an
miners and scientists to analyze a large volume of data that analytical result within a short period. Consequently, two
may not be harnessed using traditional tools [5]. existing architectures have been proposed for real-time
analysis: parallel processing clusters using traditional
Big data analytics require technologies and tools that can relational databases and memory-based computing
transform a large amount of structured, unstructured, and platforms [33]. Greenplum [34] and Hana [35] are examples
semi-structured data into a more understandable data and of real-time analytics architecture.
metadata format for analytical processes. The algorithms
used in these analytical tools must discover patterns, trends, Off-line analytics is used when a quick response is not
and correlations over a variety of time horizons in the data required [32]. For example, many Internet enterprises use
[26]. After analyzing the data, these tools visualize the Hadoop-based off-line analytics architecture to reduce the
findings in tables, graphs, and spatial charts for efficient cost of data format conversion [36]. Such analytics
decision making. Thus, big data analysis is a serious improves data acquisition efficiency. SCRIBE [37], Kafka
challenge for many applications because of data complexity [38], Time-Tunnel [39], and Chukwa [40] are examples of
and the scalability of underlying algorithms that support architectures that conduct off-line analytics and can satisfy
such processes [27]. the demands of data acquisition.
Talia (2013) highlighted that obtaining helpful information Memory-level analytics is applied when the size of data is
from big data analysis is a critical matter that requires smaller than the memory of a cluster [32]. To date, the
scalable analytical algorithms and techniques to return well- memory of clusters has reached terabyte (TB) level [41].
timed results, whereas current techniques and algorithms are Therefore, several internal database technologies are
inefficient to handle big data analytics. Therefore, large required to improve analytical efficiency. Memory-level
infrastructure and additional applications are necessary to analytics is suitable for conducting real-time analysis.
support data parallelism. Moreover, data sources, such as MongoDB [42] is an example of this architecture.
high-speed data stream received from different data sources,
have different formats, which makes integrating multiple BI analytics is adopted when the size of data is larger than
sources for analytics solutions critical [28]. Hence, the the memory level, but in this case, data may be imported to
challenge is focused on the performance of current the BI analysis environment [43]. BI analytic currently
algorithms used in big data analysis, which is not rising supports TB-level data [32]. Moreover, BI can help discover
strategic business opportunities from the flood of data. In traditional databases [44]. Massive analytics uses the
addition, BI analytics allows easy interpretation of data Hadoop distributed file system for data storage and
volumes. Identifying new opportunities and implementing map/reduce for data analysis. Massive analytics helps create
an effective strategy provide competitive market advantage the business foundation and increases market
and long-term stability. competitiveness by extracting meaningful values from data.
Moreover, massive analytics obtains accurate data that
Massive analytics is applied when the size of data is greater leverage the risks involved in making any business decision.
than the entire capacity of the BI analysis product and In addition, massive analytics provides services effectively
TABLE 1: COMPARISON OF DIFFERENT ANALYTICS TYPES AND THEIR LEVELS
Existing
Analytic Types/Level Specified Use Advantages/Category
Architectures/Tools
+Parallel processing
clusters using
Real time[33] To analyze the large amounts of data +Greenplum traditional
generated by the sensors +HANA databases memory
based
computing platforms
+Efficient Data
+Scribe acquisition
To use for the
+ Kafka +Reduce the cost of
Offline [36] Applications where there is no high
+Timetunnel data
requirements on response time
+Chukwa format conversion
To use where the total
Memory level [41] data volume is smaller than the
+MongoDB +Real time
maximum
Memory of the cluster
Business intelligence level To use when the data +Both offline and
+Data analysis plans.
[43] scale surpasses the Online
memory level
To use when data scale is
totally surpassed the
Massive level [44] +Mostly belong to
capacity of business +MapReduce
Offline
intelligence products and traditional
databases
B. Relationship between IoT and big data analytics ―things,‖ big data implementations will necessitate
Big data analytics is rapidly emerging as a key IoT initiative performing lightning-fast analytics with large queries to
to improve decision making. One of the most prominent allow organizations to gain rapid insights, make quick
features of IoT is its analysis of information about decisions, and interact with people and other devices. The
―connected things.‖ Big data analytics in IoT requires interconnection of sensing and actuating devices provide the
processing a large amount of data on the fly and storing the capability to share information across platforms through a
data in various storage technologies. Given that much of the unified architecture and develop a common operating
unstructured data are gathered directly from web-enabled picture for enabling innovative applications.
Fig. 1. Relationship between IoT and big data analytics
The need to adopt big data in IoT applications is Accordingly, statistical and machine learning methods are
compelling. These two technologies have already been utilized. The evolution of big data also changes analytics
recognized in the fields of IT and business. Although, the requirements. Although the requirements for efficient
development of big data is already lagging, these mechanisms lie in all aspects of big data management [30],
technologies are inter-dependent and should be jointly such as capturing, storage, preprocessing, and analysis; for
developed. In general, the deployment of IoT increases the our discussion, big data analytics requires the same or faster
amount of data in quantity and category; hence, offering the processing speed than traditional data analytics with
opportunity for the application and development of big data minimum cost for high-volume, high-velocity, and high-
analytics. Moreover, the application of big data technologies variety data [45].
in IoT accelerates the research advances and business
models of IoT. The relationship between IoT and big data, Various solutions are available for big data analytics, and
which is shown in Figure 1, can be divided into three steps advancements in developing and improving these solutions
to enable the management of IoT data. The first step are being continuously achieved to make them suitable for
comprises managing IoT data sources, where connected new big data trends. Data mining plays an important role in
sensors devices use applications to interact with one analytics, and most of the techniques are developed using
another. For example, the interaction of devices such as data mining algorithms according to a particular scenario.
CCTV cameras, smart traffic lights, and smart home Knowledge on available big data analytics options is crucial
devices, generates large amounts of data sources with when evaluating and choosing an appropriate approach for
different formats. This data can be stored in low cost decision making. In this section, we present several methods
commodity storage on the cloud. In the second step, the that can be implemented for several big data case studies.
generated data are called ―big data,‖ which are based on Some of these analytics methods are efficient for big IoT
their volume, velocity, and variety. These huge amounts of data analytics. Diverse and tremendous size data sets
data are stored in big data files in shared distributed fault- contribute more in big data insights. However, this belief is
tolerant databases. The last step applies analytics tools such not always valid because more data may have more
as MapReduce, Spark, Splunk, and Skytree that can analyze ambiguities and abnormalities [7].
the stored big IoT data sets. The four levels of analytics start
from training data, then move on to analytics tools, queries, We present big data analytics methods under classification,
and reports. clustering, association rule mining, and prediction
categories. Figure 2 depicts and summarizes each of these
C. Big data analytics methods categories. Each category is a data mining function and
involves many methods and algorithms to fulfill information
Big data analytics aim to immediately extract extraction and analysis requirements. For example,
knowledgeable information that helps in making Bayesian network, support vector machine (SVM), and k-
predictions, identifying recent trends, finding hidden nearest neighbor (KNN) offer classification methods.
information, and ultimately, making decisions [7]. Data Similarly, partitioning, hierarchical clustering, and co-
mining techniques are widely deployed for both problem- occurrence are widespread in clustering. Association rule
specific methods and generalized data analytics. mining and prediction comprise significant methods.
5
Big Data Analytics Clustering is another data mining technique used as a big
Classification Clustering
data analytics method. Contrary to classification, clustering
uses an unsupervised learning approach and creates groups
for given objects based on their distinctive meaningful
features [56]. As we have presented in Figure 2 that
grouping a large number of objects in the form of clusters
makes data manipulation simple. The well-known methods
used for clustering are hierarchical clustering and
Prediction Association Rule
partitioning. The hierarchical clustering approach keeps
combining small clusters of data objects to form a
hierarchical tree and create agglomerative clusters. Divisive
clusters are created in the opposite manner by dividing a
single cluster that contains all data objects into smaller
appropriate clusters [57].
Market analysis and business decision making are the most

significant applications of big data analytics. The process of
association rule mining involves identifying interesting
Fig. 2. Overview of big data analytics methods relationships among different objects, events, or other
entities to analyze market trends, consumer buying
Classification is a supervised learning approach that uses behavior, and product demand predictions (see Figure 2).
prior knowledge as training data to classify data objects into Association rule mining [58] focuses on identifying and
groups [46]. A predefined category is assigned to an object, creating rules based on the frequency of occurrences for
and thus, the objective of predicting a group or class for an numeric and non-numeric data. Data processing is
object is achieved (see Figure 2). Finding unknown or performed in two manners under association rules. First,
hidden patterns is more challenging for big IoT data. sequential data processing uses priori-based algorithms,
Furthermore, extracting valuable information from large such as MSPS [59] and LAPIN-SPAM [60], to identify
data sets to improve decision making is a critical task. A interaction associations. Another significant data processing
Bayesian network is a classification method that offers approach under association rule is temporal sequence
model interpretability. Bayesian networks are efficient for analysis, which uses algorithms to analyze event patterns in
analyzing complex data structures revealed through big data continuous data.
rather than traditional structured data formats. These
networks are directed acyclic graphs, where nodes are Predictive analytics use historical data, which are known as
random variables and edges denote conditional dependency training data, to determine the results as trends or behavior
[47]. Naïve, selective naïve, semi-naïve Bayes, and Bayes in data. SVM and fuzzy logic algorithms are used to identify
multi-nets are the proposed categories for classification relationships between independent and dependent variables
[48]. and to obtain regression curves for predictions, such as for
natural disasters. Furthermore, customer buying predictions
Analyzing data patterns and creating groups are efficiently and social media trends are analyzed through predictive
performed using SVM, which is also classification approach analytics [61] (see Table 2). In the case of big data
for big data analytics. SVM utilizes statistical learning analytics, processing requirements are modified according
theory to analyze data patterns and create groups. Several to the nature and volume of data. Fast data access and
applications of SVM classification in big data analytics mining methods for structured and unstructured data are
include text classification [49], pattern matching [50], health major concerns related to big data analytics. Furthermore,
diagnostics [51], and commerce. Similarly, KNN is data representation is a significant requirement in big data
typically designed to provide efficient mechanisms for analytics. Time series analysis reduces high dimensionality
finding hidden patterns from big data sets, such that associated with big data and offers representation for
retrieved objects are similar to the predefined category [52]. improved decision making. Research related to time series
Using cases further improve the KNN algorithm for representation includes ARMA [62], bitmaps [63], and
application in anomaly detection [53], high-dimensional wavelet functions [64].
data [54], and scientific experiments [55]. Classification has
other extensions while adopting a large number of artificial The big data analytics methods discussed in this section are
intelligence and data mining techniques. Consequently, widely adopted in many application areas of big data, such
classification is one of the widespread data mining as disaster management, healthcare, business, industry, and
techniques for big data analytics. e-governance. In Table 2, we present the application areas
of big data mining functionalities that are elaborated in this association rule-based data analytics methods are applicable
section, ‗‘ is used to show the support for an application to industry and e-governance and are well adopted in
whereas ‗-‘ denotes that it is not obvious whether the healthcare, e-commerce, and bioinformatics. Predictive
method supports to an application or not. In particular, analytics are useful for disaster and market predictions,
Table 2 shows that classification methods are suitable for whereas time series analysis is used in disaster forecasting,
medical imaging, industry, speech recognition, natural medical imaging, speech recognition, social network
language processing, and e-governance. Clustering and analysis, and e-governance.
TABLE 2: APPLICATIONS OF BIG DATA MINING FOR I OT
Applications
Disaster management
Healthcare
Medical Imaging
Human Genetics
Market Analysis
Industry
Speech Recognition
Bioinformatics
NLP
Social Network Analysis
e-governance
Method
Classification [46] - -  - -   -  - 
Clustering [57] -      -  -  
Association rule[58, 65] -  - -   -  - - 
Prediction [61]  - - -  - - - -  -
Time Series [62] [63] [64]  -  - - -  - -  
has support
- not obvious
D. IoT architecture for big data analytics representation with IoT as the unifying architecture.
However, the current architecture focuses on IoT with
The architectural concept of IoT has several definitions
regard to communications. To our knowledge, our proposed
based on IoT domain abstraction and identification. It offers
architecture, which integrates IoT and big data analytics,
a reference model that defines relationships among various
has not been studied in the current literature. Figure 3
IoT verticals, such as, smart traffic, smart home, smart
illustrates the IoT architecture and big data analytics. In this
transportation, and smart health. The architecture for big
figure, the sensor layer contains all the sensor devices and
data analytics offers a design for data abstraction.
the objects, which are connected through a wireless
Furthermore, this standard provides a reference architecture
network. This wireless network communication can be
that builds upon the reference model. Many IoT
RFID, WiFi, ultra-wideband, ZigBee, and Bluetooth. The
architectures are found in the literature [66] [67] [13]. For
IoT gateway allows communication of the Internet and
example, [13] offered an IoT architecture with cloud
various webs. The upper layer concerns big data analytics,
computing at the center and a model of end-to-end
where a large amount of data received from sensors are
interaction among various stakeholders in a cloud-centric
stored in the cloud and accessed through big data analytics
IoT framework for better comparison with the proposed IoT
applications. These applications contain API management
architecture. This architecture is achieved by seamless
and a dashboard to help in the interaction with the
ubiquitous sensing, data analytics, and information
processing engine.
API Management Dashboard
Big
Big data
data analytic
analytic
Cloud storage
IoT Gateway
Network devices
IoT
IoT Devices
Devices
Fig. 3. IoT architecture and big data analytics
A novel meta-model-based approach for integrating IoT such as smart grids, tank levels, and water flows, and silos
architecture objects is proposed. The concept is semi- stock calculation, in which processing takes a long time
automatically federated into a holistic digital enterprise even on a dedicated and powerful machine [68]. A smart
architecture environment. The main objective is to provide meter is a device that electronically records consumption of
an adequate decision support for complex business, electric energy data between the meter and the control
architecture management with the development of system. Collecting and analyzing smart meter data in IoT
assessment systems, and IT environment. Thus, environment assist the decision maker in predicting
architectural decisions for IoT are closely connected with electricity consumption. Furthermore, the analytics of a
code implementation to allow users to understand the smart meter can also be used to forecast demands to prevent
integration of enterprise architecture management with IoT. crises and satisfy strategic objectives through specific
pricing plans. Thus, utility companies must be capable of
IV. USE CASES high-volume data management and advanced analytics
This section presents a number of use cases for big IoT data designed to transform data into actionable insights.
analytics. Although the use cases are relevant to IoT
applications, the choices have been guided for the ones that B. Smart transportation
are most commonly used in IoT applications and for the
A smart transportation system is an IoT-based use case that
amount of data that can be generated for analytics.
aims to support the smart city concept. A smart
transportation system intends to deploy powerful and
A. Smart metering advanced communication technologies for the management
Smart metering is one of the IoT application use cases that of smart cities. Traditional transportation systems, which are
generates a large amount of data from different sources, based on image processing, are affected by weather
conditions, such as heavy rains and thick fog. Consequently,
8
the captured image may not be clearly visible. The design of D. Smart agriculture
an e-plate system [69] using RFID technology provides a
Smart agriculture is a beneficial use case in big IoT data
good solution for intelligent monitoring, tracking, and
analytics. Sensors are the actors in the smart agriculture use
identification of vehicles. Moreover, introducing IoT into
case. They are installed in fields to obtain data on moisture
vehicular technologies will enable traffic congestion
level of soil, trunk diameter of plants, microclimate
management to exhibit significantly better performance than
condition, and humidity level, as well as to forecast
the existing infrastructure. This technology can improve
weather. Sensors transmit obtained data using network and
existing traffic systems in which vehicles can effectively
communication devices. These data pass through an IoT
communicate with one another in a systematic manner
gateway and the Internet to reach the analytics layer shown
without human intervention.
in Fig. The analytics layer processes the data obtained from
Satellite navigation systems and sensors can also be applied the sensor network to issue commands. Automatic climate
in trucks, ships, and airplanes in real time. The routing of control according to harvesting requirements, timely and
these vehicles can be optimized by using the bulk of controlled irrigation, and humidity control for fungus
available public data, such as traffic jams, road conditions, prevention are examples of actions performed based on big
delivery addresses, weather conditions, and locations of data analytics recommendations.
refilling stations. For example, in case of runtime address
change, the updated information (route, cost) can be E. Smart grid
optimized, recalculated, and passed on to drivers in real
The smart grid is a new generation of power grid in which
time. Sensors incorporated into these vehicles can also
managing and distributing electricity between suppliers and
provide real-time information to measure engine health,
consumers is upgraded using two-way communication
determine whether equipment requires maintenance, and
technologies and computing capabilities to improve
predict errors [70].
reliability, safety, efficiency with real-time control, and
monitoring [73, 74]. One of the major challenges in a power
C. Smart supply chains system is integrating renewable and decentralized energy.
Embedded sensor technologies can communicate Electricity systems require a smart grid to manage the
bidirectionally and provide remote accessibility to over 1 volatile behavior of distributed energy resources (DERs)
million elevators worldwide [71]. The captured data are [75]. However, most energy systems have to follow
used by on- and off-site technicians to run diagnostics and governmental laws and regulations, as well as consider
repair options to make appropriate decisions, which result in business analysis and potential legal constraints [76]. Grid
increased machine uptime and enhanced customer service. sensors and devices continuously and rapidly generate data
Ultimately, big IoT data analytics allows a supply chain to related to control loops and protection and require real-time
execute decisions and control the external environment. processing and analytics along with machine-to-machine
IoT-enabled factory equipment will be able to communicate (M2M) or human-to-machine (HMI) interactions to issue
within data parameters (i.e., machine utilization, control commands to the system. However, the system must
temperature) and optimize performance by changing fulfill visualization and reporting requirements.
equipment settings or process workflow [72]. In-transit
visibility is another use case that will play a vital role in F. Smart traffic light system
future supply chains in the presence of IoT infrastructure.
The smart traffic light system consists of nodes that locally
Key technologies used by in-transit visibility are RFIDs and
interact with IoT sensors and devices to detect the presence
cloud-based Global Positioning System (GPS), which
of vehicles, bikers, and pedestrians. These nodes
provide location, identity, and other tracking information.
communicate with neighboring traffic lights to measure the
These data will be the backbone of supply chains supported
speed and distance of approaching transportation means and
by IoT technologies. The information gathered by
manage green traffic signals [77]. IoT data gathered using
equipment will provide detailed visibility of an item shipped
the system require real-time analytics processing to perform
from a manufacturer to a retailer. Data collected via RFID
necessary tasks, such as changing the timing cycles
and GPS technologies will allow supply chain managers to
according to traffic conditions, sending informative signals
enhance automated shipment and accurate delivery
to neighboring nodes, and detecting approaching vehicles
information by predicting time of arrival. Similarly,
that use IoT sensors and devices to prevent long queues or
managers will be able to monitor other information, such as
accidents. Moreover, smart traffic light systems can send
temperature control, which can affect the quality of in-
their collected IoT data to cloud storage for further
transit products.
analytics. Table 3 presents the use cases of IoT big data
analytics.
TABLE 3: COMPARISON OF IOT BIG DATA ANALYTICS USE CASES

Big data
IoT Data
Use cases Benefits analytics
devices source
applications
Smart metering [68] Predict electricity consumption Sensors Text Hadoop
Smart Improve existing traffic system by which vehicles can Text,
Sensors, Hadoop, Spark,
transportation[69] effectively communicate with one another in a systematic video,
cameras Hive
[70] manner without human intervention audio
Sensors,
Smart supply chains Allow a supply chain to execute decisions and control the Text,
mobile Hadoop
[71] [72] external environment image
devices
Smart agriculture Obtain moisture level of soil, trunk diameter of plants, Text,
Sensors Hadoop
[12, 13] microclimate condition and humidity level; forecast weather image
Smart grid [73, 74] Improves reliability, safety, and efficiency, along with real-
Sensors Text Hadoop
[75] [76] time control and monitoring
Video,
Smart traffic [77] Detect the presence of vehicles, bikers, and pedestrians Cameras Hadoop, Spark
image
As shown in Table 3, most use cases are related to M2M V. OPPORTUNITIES

communication technologies and decrease the role of human
IoT is currently considered one of the most profound
interaction. However, the technologies use prediction
transitions in technology. Current IoT provides several data
methods and decision-making techniques to improve real-
analytics opportunities for big data analytics. Figure 4
time control, monitoring, and performance. Textual data are
shows the examples of use cases and opportunities
among the common data types generated by IoT devices,
discussed in Sections 4 and 5.
which are mostly sensors and cameras. Text-based data are
suitable for analysis by distributed file systems, such as
Hadoop.
E-commerce
Retail & Logistics Smart cities

Big IoT data analytics
Healthcare
Fig. 4. Example of use cases and opportunities for big IoT data analytics architecture
A. E-commerce making. Big IoT data exhibit heterogeneity, increasing

Big IoT data analytics offers well-designed tools to process volume, and real-time data processing features. The
real-time big data, which produce timely results for decision convergence of big data with IoT brings new challenges and
10
opportunities to build a smart environment. Big IoT data VI. OPEN CHALLENGES AND FUTURE DIRECTIONS
analytics has widespread applications in nearly every IoT and big data analytics have been extensively accepted
industry. However, the main success areas of analytics are by many organizations. However, these technologies are
in e-commerce, revenue growth, increased customer size, still in their early stages. Several existing research
accuracy of sale forecast results, product optimization, risk challenges have not yet been addressed. This section
management, and improved customer segmentation. presents several challenges in the field of big IoT data
analytics.
B. Smart cities
Big data collected from smart cities offer new opportunities A. Privacy
in which efficiency gains can be achieved through an Privacy issues arise when a system is compromised to infer
appropriate analytics platform/infrastructure to analyze big or restore personal information using big data analytics
IoT data. Various devices connect to the Internet in a smart tools, although data are generated from anonymous users.
environment and share information. Moreover, the cost of With the proliferation of big data analytics technologies
storing data has been reduced dramatically after the used in big IoT data, the privacy issue has become a core
invention of cloud computing technology. Analysis problem in the data mining domain. Consequently, most
capabilities have made huge leaps. Thus, the role of big data people are reluctant to rely on these systems, which do not
in a smart city can potentially transform every sector of the provide solid service-level agreement (SLA) conditions
economy of a nation. Hadoop with YARN resource regarding user personal information theft or misuse. In fact,
manager has offered recent advancement in big data the sensitive information of users has to be secured and
technology to support and handle numerous workloads, protected from external interference. Although temporary
real-time processing, and streaming data ingestion. identification, anonymity, and encryptions provide several
ways to enforce data privacy, decisions have to be made
C. Retail and logistics with regard to ethical factors, such as what to use, how to
IoT is expected to play a key role as an emerging use, and why use generated big IoT data [7].
technology in the area of retail and logistics. In logistics, Another security risk associated with IoT data is the
RFID keeps track of containers, pallets, and crates. In heterogeneity of the types of devices used and the nature of
addition, considerable advancements in IoT technologies generated data, such as raw devices, data types, and
can facilitate retailers by providing several benefits. communication protocols. These devices can have different
However, IoT devices generate large amounts of data on a sizes and shapes outside the network and are designed to
daily basis. Thus, powerful data analytics enables communicate with cooperative applications. Thus, to
enterprises to gain insights from the voluminous amounts of authenticate these devices, an IoT system should assign a
data produced through IoT technologies. Applying data non-repudiable identification system to each device.
analytics to logistic data sets can improve the shipment Moreover, enterprises should maintain a meta-repository of
experience of customers. Moreover, retail companies can these connected devices for auditing purposes. This
earn additional profit by analyzing customer data, which can heterogeneous IoT architecture is new to security
predict the trends and demands of goods. By looking into professionals, and thus, results in increased security risks.
customer data, optimizing pricing plans and seasonal Consequently, any attack in this scenario compromises
promotions can be planned efficiently to maximize profit. system security and disconnects interconnected devices.
In the context of big IoT data, security and privacy are the
D. Healthcare key challenges in processing and storing huge amounts of
Recent years have witnessed tremendous growth in smart data. Moreover, to perform critical operations and host
health monitoring devices. These devices generate private data, these systems highly rely on third party
enormous amounts of data. Thus, applying data analytics to services and infrastructure. Therefore, an exponential
data collected from fetal monitors, electrocardiograms, growth in data rate causes difficulty in securing each and
temperature monitors, or blood glucose level monitors can every portion of critical data. As previously discussed,
help healthcare specialists efficiently assess the physical existing security solutions (Karim, 2016 #86) are no long
conditions of patients. Moreover, data analytics enables applicable to providing complete security in big IoT data
healthcare professionals to diagnose serious diseases in their scenarios. Existing algorithms are not designed for the
early stages to help save lives. Furthermore, data analytics dynamic observation of data, and thus, are not effectively
improves the clinical quality of care and ensures the safety applied. Legacy data security solutions are specifically
of patients. In addition, physician profile can be reviewed designed for static data sets, whereas current data
by looking into the history of treatment of patients, which requirements are changing dynamically (Lafuente, 2015).
can improve customer satisfaction, acquisition, and Thus, deploying these security solutions is difficult for
retention. dynamically increasing data. In addition, legislative and
regulatory issues should be considered while signing SLAs.
11
With regard to data generated through IoT, the following APIs is necessary to avoid interoperability and reliability
security problems can emerge [78]: (a) timely updates - problems. (b) Second, devices must be well protected while
difficulty in keeping systems up to date, (b) incident communicating with peers. (c) Third, devices should be
management - identifying suspicious traffic patterns among hardcoded with the best security practices to protect against
legitimate ones and possible failure to capture unidentifiable common security and privacy threats.
incidents, (c) interoperability - proprietary and vendor-
B. Data mining
specific procedures will pose difficulties in finding hidden
or zero day attacks, (d) and protocol convergence - although Data mining methods provide efficient and best-fitting
IPv6 is currently compatible with the latest specifications, predictive or descriptive solutions for big data that can also
this protocol has yet to be fully deployed. Therefore, the be generalized for new data [79]. The evolution of big IoT
application of security rules over IPv4 may not be data and cloud computing platforms has brought the
applicable to protecting IPv6. challenges of data exploration and information extraction
At present, no answer can address these challenges and [80]. However, for the overall big IoT data architecture,
manage the security and privacy of interconnected devices. Figure 5 presents the primary challenges related to
However, the following guidelines can overcome these processing and data mining.
adversities. (a) First, a true open ecosystem with standard
Knowledge
Data Processing
Discovery
Volume Complexity Sequential
Accessiblity Parallel
Correctness Data Source

Heterogeneity Data Type
Fig. 5. Big data mining issues in IoT
Exhaustive data reads/writes: The high-volume, high- devise algorithms remains to provide compatibility with the
velocity, and high-variety qualities of big IoT data challenge latest parallel architectures. Moreover, synchronization
exploration, integration, heterogeneous communication, and issues may occur in parallel computing, while information is
extraction processes. The size and heterogeneity of data exchanged within different data mining methods. This
impose new data mining requirements, and diversity in data bottleneck of data mining methods has become an open
sources also poses a challenge [81-83]. Furthermore, issue in big IoT data analytics that should be addressed.
compared with small data sets, large data sets comprise
C. Visualization
more abnormalities and ambiguities that require additional
preprocessing steps, such as cleansing, reduction, and Visualization is an important entity in big data analytics,
transmission [23, 84]. Another issue lies in the extraction of particularly when dealing with IoT systems where data are
exact and knowledgeable information from the large generated enormously. Furthermore, conducting data
volumes of diverse data. Consequently, obtaining accurate visualization is difficult because of the large size and high
information from complex data requires analyzing data dimension of big data. This situation shows underlying
properties and finding association among different data trends and a complete picture of parsed data. Therefore, big
points. data analytics and visualization should work seamlessly to
Researchers have introduced parallel and sequential obtain the best results from IoT applications in big data.
programming models and proposed different algorithms to However, visualization in the case of heterogeneous and
minimize query response time while dealing with big data. diverse data (unstructured, structured, and semi-structured)
Moreover, researchers have selected existing data mining is a challenging task. Designing visualization solution that is
algorithms in different manners to (a) improve single source compatible with advanced big data indexing frameworks is
knowledge discovery, (b) implement data mining methods a difficult task. Similarly, response time is a desirable factor
for multi-source platforms, and (c) study and analyze in big IoT data analytics. Consequently, cloud computing
dynamic data mining methods and stream data [85]. Hence, architectures supported with rich GUI facilities can be
parallel k-means algorithm [86] and parallel association rule deployed to obtain better insights into big IoT data trends
mining methods [65] are introduced. However, the need to [87].
12
Different dimensionality reduction methods have been D. Integration

introduced as a result of complex and high-dimensional big Integration refers to having a uniform view of different
IoT data [88, 89]. However, these methods are unsuitable formats. Data integration provides a single view of the data
for all types of presented data. Similarly, when fine-grained arriving from different sources and combines the view of
dimensions are visualized effectively, the probability to data [96]. Data integration includes all processes involved in
identify observable correlations, patterns, and outliners is collecting data from different sources, as well as in storing
high [90]. Moreover, data should be kept locally to obtain and providing data with a unified view. For each moment,
usable information efficiently because of power and different forms of data are continuously generated by social
bandwidth constraints. In addition, visualization software media, IoT, and other communication and
should run with the concept of reference locality to achieve telecommunication approaches. The produced data can be
efficient outcome in an IoT environment. Given that the categorized into three groups: (a) structured data, such as
amount of big IoT data is increasing rapidly, the data stored in traditional database systems, including tables
requirement of enormous parallelization is a challenging with rows and columns; (b) semi-structured, such as HTML,
task in visualization. Thus, to decompose a problem into XML, and Json files; and (c) unstructured data, such as
manageable independent tasks to enforce concurrent videos, audios, and images. Good data offer good
execution of queries is a challenge for parallel visualization information; however, this relationship is only achieved
algorithms [91]. through data integration [97]. Integrating diverse data types
At present, most big data visualization tools used for IoT is a complex task in merging different systems or
exhibit poor performance results in terms of functionality, applications [98]. Overlapping the same data, increasing
scalability, and response time. To provide effective performance and scalability, and enabling real-time data
uncertainty-aware visualization during the visual analytics access are among the challenges associated with data
process, avoiding uncertainty imposes a considerable integration that should be addressed in the future.
challenge [32]. Furthermore, several important issues are
addressed [92], such as (a) visual noise - most data set Another challenge is to adjust structures in semi-structured
objects are closely related to one another, and thus, users and unstructured data before integrating and analyzing these
may perceive different results of the same type; (b) types of data [99]. Information, such as entities and
information loss - applying reduction methods to visible relationships, can be extracted from textual data by using
data sets can cause information loss; (c) large image available technologies in the eras of text mining, machine
observation - data visualization tools have inherent learning, natural processing, and information extraction.
problems with respect to aspect ratio, devise resolution, and However, new technologies should be developed to extract
physical perception limits; (d) frequently changing image - images, videos, and other information from other non-text
users will not notice rapid data changes in an output; and (e) formats of unstructured data [99]. Text mining is expected
high performance requirements - high performance to be conducted by applying several specialized extractors
requirements are imposed because data are generated on the same text. Hence, managing and integrating different
dynamically in an IoT environment. Moreover, methods extraction results from a certain data source require other
supported by advanced analytics enable interactive graphics techniques [100].
on laptops, desktops, or mobile devices, such as
smartphones and tablets [93]. VII. CONCLUSION
Real-time analytics is another consideration highlighted in The growth rate of data production has increased drastically
IoT architectures. Several guidelines on visualization in big over the past years with the proliferation of smart and
data are presented [94], such as (a) data awareness, i.e., sensor devices. The interaction between IoT and big data is
appropriate domain expertise, (b) data quality - cleaning currently at a stage where processing, transforming, and
data using information management or data governance analyzing large amounts of data at a high frequency are
policies, (c) meaningful results - data clustering is used to necessary. We conducted this survey in the context of big
provide high-level abstraction such that the visibility of IoT data analytics. First, we explored recent analytics
smaller groups of data is possible, and (d) outliers should be solutions. The relationship between big data analytics and
removed from the data or treated as a separate entity. [95] IoT was also discussed. Moreover, we proposed an
suggested that visualization should adhere to the following architecture for big IoT data analytics. Furthermore, big
guidelines: (a) the system should provide special attention data analytics types, methods, and technologies for big data
to metadata, (b) visualization software should be interactive mining were presented. Some credible use cases were also
and should require maximum user involvement, and (c) provided. In addition, we explored the domain by discussing
tools should be built based on the dynamic nature of the various opportunities brought about by data analytics in the
generated data. IoT paradigm. Several open research challenges were
discussed as future research directions. Finally, we
concluded that existing big IoT data analytics solutions
13
remained in their early stages of development. In the future, http://eecatalog.com/caciufo/2014/07/15/iot-

real-time analytics solution that can provide quick insights gateway-adlink/.
will be required. 15. Atzori, L., A. Iera, and G. Morabito, The internet
of things: A survey. Computer networks, 2010.
ACKNOWLEDGMENT 54(15): p. 2787-2805.
16. Hsieh, H.-C. and C.-H. Lai. Internet of things
architecture based on integrated plc and 3g
This work is fully funded by Malaysian Ministry of Higher communication networks. in Parallel and
Education under University Malaya Research Grant Distributed Systems (ICPADS), 2011 IEEE 17th
(UMRG) – Project/Program UM.0000168/HRU.RP.IT and International Conference on. 2011. IEEE.
RP029D-14AET. 17. Kambatla, K., et al., Trends in big data analytics.
Journal of Parallel and Distributed Computing,
Reference 2014. 74(7): p. 2561-2573.
18. Manyika, J., et al., Big data: The next frontier for
1. Tiainen, P., New opportunities in electrical innovation, competition, and productivity. 2011.
engineering as a result of the emergence of the 19. Hashem, I.A.T., et al., The rise of ―big data‖ on
Internet of Things. 2016. cloud computing: Review and open research
2. Beyer, M., Gartner Says Solving'Big issues. Information Systems, 2015. 47: p. 98-115.
Data'Challenge Involves More Than Just Managing 20. Ali, W.B., Big Data-Driven Smart Policing: Big
Volumes of Data. Gartner. Archived from the Data-Based Patrol Car Dispatching. Journal of
original on, 2011. 10. Geotechnical and Transportation Engineering,
3. Gantz, J. and D. Reinsel, Extracting value from 2016. 1(2).
chaos. IDC iview, 2011. 1142: p. 1-12. 21. Gantz, J. and D. Reinsel, THE DIGITAL
4. Mital, R., J. Coughlin, and M. Canaday. Using Big UNIVERSE IN 2020: Big Data, Bigger Digital
Data Technologies and Analytics to Predict Sensor Shadows, and Biggest Growth in the Far East.
Anomalies. in Proceedings of the Advanced Maui Study report, IDC, 2012.
Optical and Space Surveillance Technologies 22. Borkar, V., M.J. Carey, and a.C. Li., Inside "Big
Conference, held in Wailea, Maui, Hawaii, Data management": Ogres,Onions, or Parfaits? . In
September 15-18, 2014, Ed.: S. Ryan, The Maui Proceedings of the 15th International Conference
Economic Development Board, id. 84. 2015. on Extending Database Technology, EDBT ‘12,,
5. Golchha, N., Big Data–The information revolution. 2012: p. 3-14.
IJAR, 2015. 1(12): p. 791-794. 23. Gani, A., et al., A survey on indexing techniques
6. Russom, P., Big data analytics. TDWI Best for big data: taxonomy and performance
Practices Report, Fourth Quarter, 2011. evaluation. Knowledge and Information Systems,
7. Tsai, C.-W., et al., Big data analytics: a survey. 2016. 46(2): p. 241-284.
Journal of Big Data, 2015. 2(1): p. 1-32. 24. Paul, A., et al., Video search and indexing with
8. Chen, M., et al., Related Technologies, in Big reinforcement agent for interactive multimedia
Data. 2014, Springer. p. 11-18. services. ACM Trans. Embed. Comput. Syst.,
9. Khan, Z., A. Anjum, and S.L. Kiani. Cloud based 2013. 12(2): p. 1-16.
big data analytics for smart future cities. in 25. Kwon, O., N. Lee, and B. Shin, Data quality
Proceedings of the 2013 IEEE/ACM 6th management, data usage experience and
international conference on utility and cloud acquisition intention of big data analytics.
computing. 2013. IEEE Computer Society. International Journal of Information Management,
10. Russom, P., Big data analytics. TDWI Best 2014. 34(3): p. 387-394.
Practices Report, Fourth Quarter, 2011: p. 1-35. 26. Oswal, S. and S. Koul. Big Data Analytic and
11. LaValle, S., et al., Big data, analytics and the path Visualization On Mobile Devices. in Proceedings
from insights to value. MIT sloan management of National Conference on New Horizons in IT-
review, 2011. 52(2): p. 21. NCNHIT. 2013.
12. Al Nuaimi, E., et al., Applications of big data to 27. Candela, L., D. Castelli, and P. Pagano, Managing
smart cities. Journal of Internet Services and big data through hybrid data infrastructures.
Applications, 2015. 6. ERCIM News, 2012. 89: p. 37-38.
13. Gubbi, J., et al., Internet of Things (IoT): A vision, 28. Assunçaoa, M.D., et al., Big Data Computing and
architectural elements, and future directions. Clouds: Challenges, Solutions, and Future
Future Generation Computer Systems, 2013. 29(7): Directions. arXiv preprint arXiv:1312.4722, 2013.
p. 1645-1660. 29. Singh, D. and C.K. Reddy, A survey on platforms
14. Ciufo, C.A. Industrial equipment talking on the for big data analytics. Journal of Big Data, 2014.
IoT? Better get a gateway (device). 2014 [cited 2(1): p. 1.
2016 7-8-2016]; Available from:
14
30. Siddiqa, A., et al., A Survey of Big Data 46. Estivill-Castro, V., Why so many clustering
Management: Taxonomy and State-of-the-Art. algorithms: a position paper. ACM SIGKDD
Journal of Network and Computer Applications, Explorations Newsletter, 2002. 4(1): p. 65-75.
2016. 47. Bielza, C. and P. Larrañaga, Discrete Bayesian
31. Steed, C.A., et al., Big data visual analytics for network classifiers: a survey. ACM Computing
exploratory earth system simulation analysis. Surveys (CSUR), 2014. 47(1): p. 5.
Computers & Geosciences, 2013. 61: p. 71-82. 48. Chen, F., et al., Data mining for the internet of
32. Chen, C.P. and C.-Y. Zhang, Data-intensive things: literature review and challenges.
applications, challenges, techniques and International Journal of Distributed Sensor
technologies: A survey on Big Data. Information Networks, 2015. 2015: p. 12.
Sciences, 2014. 275: p. 314-347. 49. Luss, R. and A. d‘Aspremont, Predicting abnormal
33. Pfaffl, M.W., A new mathematical model for returns from news using text classification.
relative quantification in real-time RT–PCR. Quantitative Finance, 2015. 15(6): p. 999-1012.
Nucleic acids research, 2001. 29(9): p. e45-e45. 50. Melin, P. and O. Castillo, A review on type-2
34. Waas, F.M. Beyond conventional data fuzzy logic applications in clustering, classification
warehousing—massively parallel data processing and pattern recognition. Applied soft computing,
with Greenplum database. in International 2014. 21: p. 568-577.
Workshop on Business Intelligence for the Real- 51. Soualhi, A., K. Medjaher, and N. Zerhouni,
Time Enterprise. 2008. Springer. Bearing health monitoring based on hilbert–huang
35. Färber, F., et al., SAP HANA database: data transform, support vector machine, and regression.
management for modern business applications. IEEE Transactions on Instrumentation and
ACM Sigmod Record, 2012. 40(4): p. 45-51. Measurement, 2015. 64(1): p. 52-62.
36. Cheng, M., et al., Mu rhythm-based cursor control: 52. Larose, D.T., k‐Nearest Neighbor Algorithm.
an offline analysis. Clinical Neurophysiology, Discovering Knowledge in Data: An Introduction
2004. 115(4): p. 745-751. to Data Mining, 2005: p. 90-106.
37. Castro, M., et al., SCRIBE: A large-scale and 53. Su, M.-Y., Real-time anomaly detection systems
decentralized application-level multicast for Denial-of-Service attacks by weighted k-
infrastructure. IEEE Journal on Selected Areas in nearest-neighbor classifiers. Expert Systems with
communications, 2002. 20(8): p. 1489-1499. Applications, 2011. 38(4): p. 3492-3498.
38. Kreps, J., N. Narkhede, and J. Rao. Kafka: A 54. Muja, M. and D.G. Lowe, Scalable nearest
distributed messaging system for log processing. in neighbor algorithms for high dimensional data.
Proceedings of the NetDB. 2011. IEEE Transactions on Pattern Analysis and
39. Notsu, H., et al. Time-tunnel: Visual analysis tool Machine Intelligence, 2014. 36(11): p. 2227-2240.
for time-series numerical data and its extension 55. Hu, C., et al., Data-driven method based on particle
toward parallel coordinates. in International swarm optimization and k-nearest neighbor
Conference on Computer Graphics, Imaging and regression for estimating capacity of lithium-ion
Visualization (CGIV'05). 2005. IEEE. battery. Applied Energy, 2014. 129: p. 49-55.
40. Rabkin, A. and R.H. Katz. Chukwa: A System for 56. Srivastava, K., et al., Data mining using
Reliable Large-Scale Log Collection. in LISA. hierarchical agglomerative clustering algorithm in
2010. distributed cloud computing environment.
41. Hong, S. and H. Kim. An analytical model for a International Journal of Computer Theory and
GPU architecture with memory-level and thread- Engineering, 2013. 5(3): p. 520.
level parallelism awareness. in ACM SIGARCH 57. Berkhin, P., A survey of clustering data mining
Computer Architecture News. 2009. ACM. techniques, in Grouping multidimensional data.
42. Chodorow, K., MongoDB: the definitive guide. 2006, Springer. p. 25-71.
2013: " O'Reilly Media, Inc.". 58. Gosain, A. and M. Bhugra. A comprehensive
43. Jourdan, Z., R.K. Rainer, and T.E. Marshall, survey of association rules on quantitative data in
Business Intelligence: An Analysis of the data mining. in Information & Communication
Literature 1. Information Systems Management, Technologies (ICT), 2013 IEEE Conference on.
2008. 25(2): p. 121-131. 2013. IEEE.
44. Bifet, A., et al., Moa: Massive online analysis. The 59. Fitzwater, M., Efficient mining of maximal
Journal of Machine Learning Research, 2010. 11: sequential patterns using multiple samples. 2005.
p. 1601-1604. 60. Yang, Z. and M. Kitsuregawa. LAPIN-SPAM: An
45. Mukhopadhyay, A., et al., A survey of improved algorithm for mining sequential pattern.
multiobjective evolutionary algorithms for data in 21st International Conference on Data
mining: Part I. Evolutionary Computation, IEEE Engineering Workshops (ICDEW'05). 2005. IEEE.
Transactions on, 2014. 18(1): p. 4-19. 61. Gandomi, A. and M. Haider, Beyond the hype: Big
data concepts, methods, and analytics. International
15
Journal of Information Management, 2015. 35(2): 76. Neureiter, C., et al. A Standards-based Approach
p. 137-144. for Domain Specific Modelling of Smart Grid
62. Kalpakis, K., D. Gada, and V. Puttagunta. Distance System Architectures. in Proceedings of the 11th
measures for effective clustering of ARIMA time- International Conference on System of Systems
series. in Data Mining, 2001. ICDM 2001, Engineering (SoSE), Kongsberg, Norway. 2016.
Proceedings IEEE International Conference on. 77. Bonomi, F., et al. Fog computing and its role in the
2001. IEEE. internet of things. in Proceedings of the first
63. Kumar, N., et al. Time-series Bitmaps: a Practical edition of the MCC workshop on Mobile cloud
Visualization Tool for Working with Large Time computing. 2012. ACM.
Series Databases. in SDM. 2005. SIAM. 78. Steinklauber, K. Data Protection in the Internet of
64. Ryan, D., High performance discovery in time Things. 2014 [cited 2016 20 June]; Available
series: techniques and case studies. 2013: Springer from: https://securityintelligence.com/data-
Science & Business Media. protection-in-the-internet-of-things.
65. Wu, X. and S. Zhang, Synthesizing high-frequency 79. Mukhopadhyay, A., et al., A survey of
rules from different data sources. IEEE multiobjective evolutionary algorithms for data
Transactions on Knowledge and Data Engineering, mining: Part I. IEEE Transactions on Evolutionary
2003. 15(2): p. 353-367. Computation, 2014. 18(1): p. 4-19.
66. Duan, R., X. Chen, and T. Xing. A QoS 80. Hu, T., et al. A survey of mass data mining based
architecture for IOT. in Internet of Things on cloud-computing. in Anti-counterfeiting,
(iThings/CPSCom), 2011 International Conference Security, and Identification. 2012. IEEE.
on and 4th International Conference on Cyber, 81. Sun, Y., et al., Mining knowledge from
Physical and Social Computing. 2011. IEEE. interconnected data: a heterogeneous information
67. Zhang, Y., et al., ICN based Architecture for IoT. network analysis approach. Proceedings of the
IRTF contribution, October, 2013. VLDB Endowment, 2012. 5(12): p. 2022-2023.
68. Darby, S., Smart metering: what potential for 82. Chen, M., et al., Itinerary planning for energy-
householder engagement? Building Research & efficient agent communications in wireless sensor
Information, 2010. 38(5): p. 442-457. networks. IEEE Transactions on Vehicular
69. Rahman, T.A. and S.K.A. Rahim. RFID vehicle Technology, 2011. 60(7): p. 3290-3299.
plate number (e-plate) for tracking and 83. Zhang, D., et al., A Taxonomy of Agent
management system. in Parallel and Distributed Technologies for Ubiquitous Computing
Systems (ICPADS), 2013 International Conference Environments. TIIS, 2012. 6(2): p. 547-565.
on. 2013. IEEE. 84. Chen, M., V.C. Leung, and S. Mao, Directional
70. Sherly, J. and D. Somasundareswari, INTERNET controlled fusion in wireless sensor networks.
OF THINGS BASED SMART Mobile Networks and Applications, 2009. 14(2): p.
TRANSPORTATION SYSTEMS. 2015. 220-229.
71. Tohamy, N., What you need to know about the 85. Wu, X., et al., Data mining with big data. IEEE
Internet of Things. MHD Supply Chain Solutions, transactions on knowledge and data engineering,
2015. 45(3): p. 32. 2014. 26(1): p. 97-107.
72. Pettey, C. Five Ways the Internet of Things Will 86. Su, K., et al., A logical framework for identifying
Benefit the Supply Chain. 2015 [cited 2016; quality knowledge from different data sources.
Available from: Decision Support Systems, 2006. 42(3): p. 1673-
http://www.gartner.com/smarterwithgartner/five- 1683.
ways-the-internet-of-things-will-benefit-the- 87. Wang, L., G. Wang, and C.A. Alexander, Big data
supply-chain-2/. and visualization: methods, challenges and
73. Yan, Y., et al., A survey on smart grid technology progress. Digital Technologies, 2015.
communication infrastructures: Motivations, 1(1): p. 33-38.
requirements and challenges. IEEE 88. Azar, A.T. and A.E. Hassanien, Dimensionality
communications surveys & tutorials, 2013. 15(1): reduction of medical big data using neural-fuzzy
p. 5-20. classifier. Soft computing, 2015. 19(4): p. 1115-
74. Bera, S., S. Misra, and J.J. Rodrigues, Cloud 1127.
computing applications for smart grid: A survey. 89. Popov, V.L. and M. Heß, Method of
IEEE Transactions on Parallel and Distributed dimensionality reduction in contact mechanics and
Systems, 2015. 26(5): p. 1477-1494. friction. 2015: Springer.
75. Dethlefs, T., et al. Energy Service Description for 90. Donalek, C., et al. Immersive and collaborative
Capabilities of Distributed Energy Resources. in data visualization using virtual reality platforms. in
DA-CH Conference on Energy Informatics. 2015. Big Data (Big Data), 2014 IEEE International
Springer. Conference on. 2014. IEEE.
16
91. Hank Childs, B.G., Will Schroeder, Jeremy

Meredith, Kenneth Moreland, Christopher Sewell,
Torsten Kuhlen, E. Wes Bethel, Research
Challenges for Visualization Software. Computer,
2013. 46(5): p. 34-42.
92. Gorodov, E.Y.e. and V.V.e. Gubarev, Analytical
review of data visualization methods in application
to big data. Journal of Electrical and Computer
Engineering, 2013. 2013: p. 22.
93. Center", I.I. Big Data Visualization: Turning Big
Data Into Big Insights [while paper]. 2013 [cited
2016 June 20]; Available from:
https://future.transport.nsw.gov.au/wp-
content/uploads/2016/02/big-data-visualization-
turning-big-data-into-big-insights.pdf.
94. Inc, S.I. Five big data challenges and how to
overcome them with visual analytics. 2013 [cited
2016 June 20]; Available from:
https://www.sas.com/resources/asset/five-big-data-
challenges-article.pdf.
95. Simon, P., The visual organization: data
visualization, Big Data, and the quest for better
decisions. 2014: John Wiley & Sons.
96. Ahamed, B.B., T. Ramkumar, and S. Hariharan.
Data Integration Progression in Large Data Source
Using Mapping Affinity. in Advanced Software
Engineering and Its Applications (ASEA), 2014
7th International Conference on. 2014.
97. Liu, J. and X. Zhang, Data integration in fuzzy
XML documents. Information Sciences, 2014. 280:
p. 82-97.
98. Ma‘ayan, A., et al., Lean Big Data integration in
systems biology and systems pharmacology.
Trends in pharmacological sciences, 2014. 35(9):
p. 450-460.
99. Divyakant Agrawal, P.B., Elisa Bertino, Susan
Davidson, Umeshwar Dayal, Michael, et al.,
Challenges and Opportunities with Big Data: A
community white paper developed by leading
researchers across the United States. Whitepaper,
Computing Community Consortium, 2012.
100. Rakesh Agrawal, A.A., Philip A. Bernstein, Eric A.
Brewer, Michael J. Carey,, et al., The Claremont
Report on Database Research. 2009. 52(6): p. 56-
65.
17

Big IoT Data Analytics - Architecture, Opportunities, and Open Research Challenges

Uploaded by

Copyright:

Available Formats

Big IoT Data Analytics - Architecture, Opportunities, and Open Research Challenges

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Big IoT Data Analytics - Architecture, Opportunities, and Open Research Challenges

Uploaded by

Copyright:

Available Formats

What are the main challenges discussed in analyzing big IoT data?

What are the main challenges discussed in analyzing big IoT data?

What are some of the notable use cases presented for big IoT data analytics?

What are some of the notable use cases presented for big IoT data analytics?

This article has been accepted for publication in a future issue of this journal, but has not been

Big IoT Data Analytics: Architecture,

Fig. 1. Relationship between IoT and big data analytics

Market analysis and business decision making are the most

Social Network Analysis

API Management Dashboard

Fig. 3. IoT architecture and big data analytics

TABLE 3: COMPARISON OF IOT BIG DATA ANALYTICS USE CASES

As shown in Table 3, most use cases are related to M2M V. OPPORTUNITIES

Retail & Logistics Smart cities

A. E-commerce making. Big IoT data exhibit heterogeneity, increasing

Volume Complexity Sequential

Correctness Data Source

Fig. 5. Big data mining issues in IoT

Different dimensionality reduction methods have been D. Integration

remained in their early stages of development. In the future, http://eecatalog.com/caciufo/2014/07/15/iot-

91. Hank Childs, B.G., Will Schroeder, Jeremy

You might also like