1

Machine Learning in Wireless Sensor Networks:

Algorithms, Strategies, and Applications
Mohammad Abu Alsheikh1,2 , Shaowei Lin2 , Dusit Niyato1 and Hwee-Pink Tan2
1
School of Computer Engineering, Nanyang Technological University, Singapore 639798
2
Sense and Sense-abilities Programme, Institute for Infocomm Research, Singapore 138632
arXiv:1405.4463v2 [cs.NI] 19 Mar 2015

Abstract—Wireless sensor networks monitor dynamic environ- The following two classical definitions capture the essence of
ments that change rapidly over time. This dynamic behavior is machine learning:
either caused by external factors or initiated by the system de-
signers themselves. To adapt to such conditions, sensor networks 1) The development of computer models for learning pro-
often adopt machine learning techniques to eliminate the need cesses that provide solutions to the problem of knowl-
for unnecessary redesign. Machine learning also inspires many edge acquisition and enhance the performance of devel-
practical solutions that maximize resource utilization and prolong oped systems [2].
the lifespan of the network. In this paper, we present an extensive
literature review over the period 2002-2013 of machine learning 2) The adoption of computational methods for improving
methods that were used to address common issues in wireless machine performance by detecting and describing con-
sensor networks (WSNs). The advantages and disadvantages of sistencies and patterns in training data [3].
each proposed algorithm are evaluated against the corresponding
problem. We also provide a comparative guide to aid WSN Applying these definitions to WSNs, we see that the promise
designers in developing suitable machine learning solutions for of machine learning lies in exploiting historical data to im-
their specific application challenges. prove the performance of sensor networks on given tasks with-
Index Terms—Wireless sensor networks, machine learning, out the need for re-programming. More specifically, machine
data mining, security, localization, clustering, data aggregation, learning is important in WSN applications for the following
event detection, query processing, data integrity, fault detection, main reasons:
medium access control, compressive sensing.
1) Sensor networks usually monitor dynamic environments
that change rapidly over time. For example, a node’s lo-
I. INTRODUCTION cation may change due to soil erosion or sea turbulence.
It is desirable to develop sensor networks that can adapt
W ireless sensor network (WSN) is composed typically and operate efficiently in such environments.
A of multiple autonomous, tiny, low cost and low power
sensor nodes. These nodes gather data about their environment
2) WSNs may be used for collecting new knowledge about
unreachable, dangerous locations [4] (e.g., volcano erup-
and collaborate to forward sensed data to centralized backend tion and waste water monitoring) in exploratory applica-
units called base stations or sinks for further processing. The tions. Due to the unexpected behavior patterns that may
sensor nodes could be equipped with various types of sensors, arise in such scenarios, system designers may develop
such as thermal, acoustic, chemical, pressure, weather, and op- solutions that initially may not operate as expected.
tical sensors. Because of this diversity, WSNs have tremendous System designers would rather have robust machine
potential for building powerful applications, each with its own learning algorithms that are able to calibrate itself to
individual characteristics and requirements. Developing effi- newly acquired knowledge.
cient algorithms that are suitable for many different application 3) WSNs are usually deployed in complicated environ-
scenarios is a challenging task. In particular, WSN designers ments where researchers cannot build accurate mathe-
have to address common issues related to data aggregation, matical models to describe the system behavior. Mean-
data reliability, localization, node clustering, energy aware while, some tasks in WSNs can be prescribed using
routing, events scheduling, fault detection and security. simple mathematical models but may still need complex
Machine learning (ML) was introduced in the late 1950’s algorithms to solve them (e.g., the routing problem
as a technique for artificial intelligence (AI) [1]. Over time, [5], [6]). Under similar circumstances, machine learning
its focus evolved and shifted more to algorithms which are provides low-complexity estimates for the system model.
computationally viable and robust. In the last decade, machine 4) Sensor network designers often have access to large
learning techniques have been used extensively for a wide amounts of data but may be unable to extract important
range of tasks including classification, regression and density correlations in them. For example, in addition to ensur-
estimation in a variety of application areas such as bioinfor- ing communication connectivity and energy sustainabil-
matics, speech recognition, spam detection, computer vision, ity, the WSN application often comes with minimum
fraud detection and advertising networks. The algorithms and data coverage requirements that have to be fulfilled by
techniques used come from many diverse fields including limited sensor hardware resources [7]. Machine learning
statistics, mathematics, neuroscience, and computer science. methods can then be used to discover important corre-
 2

lations in the sensor data and propose improved sensor comparison of their strengths and weaknesses. In particular,
deployment for maximum data coverage. we provide a comprehensive overview which groups these
5) New uses and integrations of WSNs, such as in cyber- recent techniques roughly into supervised, unsupervised and
physical systems (CPS), machine-to-machine (M2M) reinforcement learning methods. Another distinction between
communications, and Internet of things (IoT) technolo- our survey and earlier works is the way that machine learning
gies, have been introduced with a motivation of support- techniques are presented. Our work discusses machine learning
ing more intelligent decision-making and autonomous algorithms based on their target WSN challenges, so as to
control [8]. Here, machine learning is important to encourage the adoption of existing machine learning solutions
extract the different levels of abstractions needed to in WSN applications. Lastly, we build on existing surveys
perform the AI tasks with limited human intervention and go beyond classifying and comparing previous efforts, by
[9]. providing useful and practical guidelines for WSN researchers
However, there are a few drawbacks and limitations that and engineers who are interested in exploring new machine
should be considered when using machine learning techniques learning paradigms for future research.
in wireless sensor networks. Some of these are: The rest of the paper is organized as follows:
• Section II introduces the reader to machine learning
1) As a resource limited framework, WSN drains a con-
siderable percentage of its energy budget to predict the algorithms and themes that will be referred to in later
accurate hypothesis and extract the consensus relation- sections. Simple examples will be given in the context of
ship among data samples. Thus, the designers should WSNs.
• In Section III, we review existing machine learning
consider the trade-off between the algorithm’s compu-
tational requirements and the learned model’s accuracy. efforts to address functional issues in WSNs such as
Specifically, the higher the required accuracy, the higher routing, localization, clustering, data aggregation, query
the computational requirements, and the higher energy processing and medium access control. Here, an issue is
consumptions. Otherwise, the developed systems might functional if it is essential to the basic operation of the
be employed with centralized and resource capable wireless sensor network.
• Section IV investigates machine learning solutions in
computational units to perform the learning task.
2) Generally speaking, learning by examples requires a WSNs for fulfilling non-functional requirements, i.e.
large data set of samples to achieve the intended general- those which determine the quality or enhance the per-
ization capabilities (i.e., fairly small error bounds), and formance of functional behaviors. Examples of such
the algorithm’s designer will not have the full control requirements include security, quality of service (QoS)
over the knowledge formulation process [10]. and data integrity. In this section, we also highlight some
unique efforts in specialized WSN applications.
During the past decade, WSNs have seen increasingly • Section V outlines major difficulties and open research
intensive adoption of advanced machine learning techniques. problems for machine learning in WSNs.
In [11], a short survey of machine learning algorithms ap- • Finally, we conclude in Section VI and present a compar-
plied in WSNs for information processing and for improving ative guide with useful paradigms for furthering machine
network performance was presented. A related survey that learning research in various WSN applications.
discussed the applications of machine learning in wireless
ad-hoc networks was published in [12]. The authors of [13]
discussed applications of three popular machine learning al- II. INTRODUCTION TO MACHINE LEARNING IN
gorithms (i.e., reinforcement learning, neural networks and WIRELESS SENSOR NETWORKS
decision trees) at all communication layers in the WSNs. In Usually, sensor network designers characterize machine
contrast, specialized surveys that touch on machine learning learning as a collection of tools and algorithms that are
usage in specific WSN challenges have also been written. used to create prediction models. However, machine learning
For instance, [14], [15] addressed the development of efficient experts recognize it as a rich field with very large themes
outlier detection techniques so that proper actions can be taken, and patterns. Understanding such themes will be beneficial to
and some of these techniques are based on concepts from those who wish to apply machine learning to WSNs. Applied
machine learning. Meanwhile, [16] discusses computational to numerous WSNs applications, machine learning algorithms
intelligence methods for tackling challenges in WSNs such as provide tremendous flexibility benefits. This section provides
data aggregation and fusion, routing, task scheduling, optimal some of the theoretical concepts and strategies of adopting
deployment and localization. Here, computational intelligence machine learning in the context of WSNs.
is a branch of machine learning that focuses on biologically- Existing machine learning algorithms can be categorized by
inspired approaches such as neural networks, fuzzy systems the intended structure of the model. Most machine learning
and evolutionary algorithms [17]. algorithms fall into the categories of supervised, unsupervised
Generally, these early surveys concentrated on reinforce- and reinforcement learning [18]. In the first category, machine
ment learning, neural networks and decision trees which were learning algorithms are provided with a labeled training data
popular due to their efficiency in both theory and practice. set. This set is used to build the system model representing
In this paper, we decided instead to include a wide variety the learned relation between the input, output and system
of important up-to-date machine learning algorithms for a parameters. In contrast to supervised learning, unsupervised
 3

Neurons
learning algorithms are not provided with labels (i.e., there (computational units)
is no output vector). Basically, the goal of an unsupervised RSSI
learning algorithm is to classify the sample sets to different Location_x
groups (i.e., clusters) by investigating the similarity between TDOA
the input samples. The third category includes reinforcement Location_y
learning algorithms, in which the agent learns by interacting TOA
with its environment (i.e., online learning). Finally, some
Angle Location_z
machine learning algorithms do not naturally fit into this clas-
sification since they share characteristics of both supervised
and unsupervised learning methods. These hybrid algorithms Fig. 1: Illustration example of node localization in WSNs in
(often termed as semi-supervised learning) aim to inherit the 3D space using supervised neural networks.
strengths of these main categories, while minimizing their
weaknesses [19].
This section is mainly to introduce the reader to the al- 2) Decision tree (DT): It is a classification method for
gorithms that will be referred to in later sections. Moreover, predicting labels of data by iterating the input data through a
examples will be given to demonstrate the process of adopting learning tree [39]. During this process, the feature properties
machine learning in WSNs. In Sections III and IV, such details are compared relative to decision conditions to reach a specific
will be omitted. For interested reader, please refer to [18], [20] category. The literature is very rich with solutions that use
and references therein, for thorough discussions of machine DT algorithm to resolve different WSNs’ design challenges.
learning theory and its classical concepts. For example, DT provides a simple, but efficient method to
identify link reliability in WSNs by identifying a few critical
features such as loss rate, corruption rate, mean time to failure
A. Supervised Learning (MTTF) and mean time to restore (MTTR). However, DT
works only with linearly separable data and the process of
In supervised learning, a labeled training set (i.e., predefined building optimal learning trees is NP-complete [40].
inputs and known outputs) is used to build the system model. 3) Neural networks (NNs): This learning algorithm could
This model is used to represent the learned relation between be constructed by cascading chains of decision units (e.g.,
the input, output and system parameters. In this subsection, perceptrons or radial basis functions) used to recognize non-
the major supervised learning algorithms are discussed in the linear and complex functions [9]. In WSNs, using neural
context of WSNs. In fact, supervised learning algorithms are networks in distributed manners is still not so pervasive due to
extensively used to solve several challenges in WSNs such the high computational requirements for learning the network
as localization and objects targeting (e.g., [21], [22], [23]), weights, as well as the high management overhead. However,
event detection and query processing (e.g., [24], [25], [26], in centralized solutions, neural networks can learn multiple
[27]), media access control (e.g., [28], [29], [30]), security and outputs and decision boundaries at once [41], which makes
intrusion detection (e.g., [31], [32], [33], [34]), and quality of them suitable for solving several network challenges using the
service (QoS), data integrity and fault detection (e.g., [35], same model.
[36], [37]). We consider a sensor node localization problem (i.e., de-
1) K-nearest neighbor (k-NN): This supervised learning termining node’s geographical position) as an application
algorithm classifies a data sample (called a query point) based example of neural network in WSNs. Node localization can be
on the labels (i.e., the output values) of the near data samples. based on propagating angle and distance measurements of the
For example, missing readings of a sensor node can be pre- received signals from anchor nodes [42]. Such measurements
dicted using the average measurements of neighboring sensors may include received signal strength indicator (RSSI), time
within specific diameter limits. There are several functions of arrival (TOA), and time difference of arrival (TDOA)
to determine the nearest set of nodes. A simple method is as illustrated in Figure 1. After supervised training, neural
to use the Euclidean distance between different sensors. K- network generates an estimated node location as vector-valued
nearest neighbor does not need high computational power, as coordinates in 3D space. Related algorithms to neural networks
the function is computed relative to local points (i.e., k-nearest include self-organizing map (or Kohonen’s maps) and learning
points, where k is a small positive integer). This factor coupled vector quantization (LVQ) (see [43] and references therein for
with the correlated readings of neighboring nodes makes k- an introduction to these methods). In addition to function es-
nearest neighbor a suitable distributed learning algorithm for timation, one of the important applications of neural networks
WSNs. In [38], it has been shown that the k-NN algorithm is for big data (high-dimensional and complex data set) tuning
may provide inaccurate results when analyzing problems with and dimensionality reduction [44].
high-dimensional spaces (more than 10-15 dimensions) as the 4) Support vector machines (SVMs): It is a machine learn-
distance to different data samples becomes invariant (i.e., the ing algorithm that learns to classify data points using labeled
distances to the nearest and farthest neighbors are slightly sim- training samples [45]. For example, one approach for detecting
ilar). In WSNs, the most important application of the k-nearest malicious behavior of a node is by using SVM to investigate
neighbor algorithm is in the query processing subsystem (e.g., temporal and spatial correlations of data. To illustrate, given
[24], [25]). WSN’s observations as points in the feature space, SVM
 4

Support vectors
PCA 1
The highest-variance
PCA 2 direction

Class B
Class A

Feature 2
Feature 2

The transformed data

coordinates
Separation margin
Feature 1 Feature 1

Fig. 2: An example of non-linear support vector machines. The original data coordinates
Fig. 3: A simple 2D visualization of the principal component
analysis algorithm. It is important to note that the potential of
divides the space into parts. These parts are separated by the PCA algorithm is high mainly when dealing with high-
as wide as possible margins (i.e., separation gaps), and new dimensional data [62].
reading will be classified based on which side of the gaps they
fall on as shown in Fig. 2. An SVM algorithm, which includes
optimizing a quadratic function with linear constraints (that is, node clustering problem are (a) randomly choose k nodes to
the problem of constructing a set of hyperplanes), provides an be the initial centroids for different clusters; (b) label each
alternative method to the multi-layer neural network with non- node with the closest centroid using a distance function; (c) re-
convex and unconstrained optimization problem [39]. Potential compute the centroids using the current node memberships and
applications of SVM in WSNs are security (e.g., [33], [34], (d) stop if the convergence condition is valid (e.g., a predefined
[46], [47], [48]) and localization (e.g., [49], [50], [51]). For a threshold for the sum of distances between nodes and their
detailed discussion of the SVM theory, please refer to [45]. perspective centroids), otherwise go back to step (b).
5) Bayesian statistics: Unlike most machine learning algo- 2) Principal component analysis (PCA): It is a multivariate
rithms, Bayesian inference requires a relatively small number method for data compression and dimensionality reduction
of training samples [52]. Bayesian methods adapt probability that aims to extract important information from data and
distribution to efficiently learn uncertain concepts (e.g., θ) present it as a set of new orthogonal variables called principal
without over-fitting. The crux of the matter is to use the current components [62]. As shown in Fig. 3, the principal compo-
knowledge (e.g., collected data abbreviated as D) to update nents are ordered such that the first component corresponds to
prior beliefs into posterior beliefs p(θ|D) ∝ p(θ)p(D|θ), the highest-variance direction of the data, and so on for the
where p(θ|D) is the posterior probability of the parameter other components. Hence, the least-variance components can
θ given the observation D, and p(D|θ) is the likelihood of be discarded as they contain the least information content.
the observation D given the parameter θ. One application of For example, PCA reduces the amount of transmitted data
Bayesian inference in WSNs is assessing event consistency among sensor nodes by finding a small set of uncorrelated
(θ) using incomplete data sets (D) by investigating prior linear combinations of original readings. Furthermore, the
knowledge about the environment. However, such statistical PCA method simplifies the problem solving by considering
knowledge requirement limits the wide adoption of Bayesian only few conditions in very large variable problems (i.e.,
algorithms in WSNs. A related statistical learning algorithm tuning big data into tiny data representation) [63]. A thorough
is Gaussian process regression (GPR) model [53]. discussion of the PCA theory (e.g., the eigenvalue, eigenvector,
and covariance matrix analysis) is given in [62].
B. Unsupervised Learning
Unsupervised learners are not provided with labels (i.e.,
C. Reinforcement Learning
there is no output vector). Basically, the goal of an unsu-
pervised learning algorithm is to classify the sample set into Reinforcement learning enables an agent (e.g., a sensor
different groups by investigating the similarity between them. node) to learn by interacting with its environment. The agent
As expected, this theme of learning algorithms is widely used will learn to take the best actions that maximize its long-term
in node clustering and data aggregation problems (e.g., [54], rewards by using its own experience. The most well-known
[55], [56], [57], [58], [59], [60]). Indeed, this wide adoption reinforcement learning technique is Q-learning [64]. As shown
is due to data structures (i.e., no labeled data is available) and in Fig. 4, an agent regularly updates its achieved rewards based
the desired outcome in such problems. on the taken action at a given state. The future total reward
1) K-means clustering: The k-means algorithm [61] is used (i.e., the Q-value) of performing an action at at a given state
to recognize data into different classes (known as clusters). st is computed using Eq. (1).
This unsupervised learning algorithm is widely used in sensor
node clustering problem due to its linear complexity and
simple implementation. The k-means steps to resolve such Q (st+1 , at+1 ) = Q (st , at ) + γ (r (st , at ) − Q (st , at )) (1)
 5

(st ) (at ) 1

a
Initial state Taken action 6 5 3 6 6

dat
5 3

Loc 6.5
bal

4
1.2
Agent

6.5
3.5
Enviroment

al d
Glo
6.5
4
(e.g., sensor node) 4

ata
2 6
New state (st+1) Achieved reward r (at , st ) 4
3.5
4 4
3.5
1 4 1

3
2
1.2

4
Fig. 4: A Visualization of the Q-learning method. 1.2

3
3

3
2 2 2
3 2 3 5 3 5 3 2

(a) Original graph. (b) Traditional routing. (c) Simplified problems us-
where r(st , at ) denotes the immediate reward of performing ing machine learning.
an action at at a given state st , and γ is the learning rate
that determines how fast learning occurs (usually set to value Fig. 5: An example of a sensor network routing problem using
between 0 and 1). This algorithm can be easily implemented a graph along with each path routing cost, traditional spanning
in a distributed architecture like WSNs, where each node tree routing, and the generated sub-problems using machine
seeks to choose actions that are expected to maximize its long learning that require only local communication to achieve
term rewards. It is important to note that Q-learning has been optimal routing (i.e., require only single-hop neighborhood
extensively and efficiently used in WSN routing problem (e.g., information exchange).
[65], [66], [67], [68]).

III. FUNCTIONAL CHALLENGES considering only their local neighbors, thus achieving low
cost, efficient and real-time routing.
In the design of WSNs, it is important to consider power • Meet QoS requirements in routing problem using rela-
and memory constraints of sensor nodes, topology changes, tively simple computational methods and classifiers.
communication link failures, and decentralized management.
Machine learning paradigms have been successfully adopted Figures 5a and 5b illustrate a simple sensor network routing
to address various functional challenges of wireless sensor problem using a graph, and the traditional spanning tree rout-
networks such as energy aware and real-time routing, query ing algorithm, respectively. To find the optimal routing paths,
processing and event detection, localization, node clustering the network nodes have to exchange their routing information
and data aggregation. with each other. In the other side, Figure 5c demonstrates how
machine learning reduces the complexity of a typical routing
problem by only considering neighboring nodes’ information
A. Routing in WSNs that will be used to predict the full path quality. Each node will
Designing a routing protocol for WSNs has to consider independently perform the routing procedures to decide which
various design challenges such as energy consumption, fault channels to assign, and the optimal transmission power. As
tolerance, scalability, and data coverage [6]. Sensor nodes are we will discuss in this subsection, such mechanism is proven
provided with limited processing capabilities, small memory to provide a near optimal routing decision with a very low
and low bandwidth. Traditionally, it is common to formulate computational complexity.
a routing problem in wireless sensor networks as a graph In this subsection, a wide range of machine learning-
G = (V, E), where V represents the set of all nodes, and based routing protocols developed for WSNs are described.
E represents the set of bidirectional communication channels Table I provides a summary and comparison of these routing
connecting the nodes. Using this model, the routing problem protocols. The column “Scalability” implies the solutions’
can be defined as the process of finding the minimum cost capability to route data in large scale networks.
path starting at the source vertex, and reaching all destination 1) Distributed regression framework: In [69], Guestrin et
vertices, by using the available graph edges. This path is al. introduced a general framework for sensors data modeling.
actually a spanning tree T = (V, E) whose vertices include This distributed framework relies on the network nodes for
the source (i.e., a root node) and destinations (i.e., leaf nodes fitting a global function to match their own measurement.
that do not have any child nodes). Solving such a tree with The nodes are used to execute a kernel linear regression in
optimal data aggregation is found to be NP-hard, even when the form of weighted components. Kernel functions map the
the full topology is known [5]. training samples into some feature space to facilitate data
Machine learning allows a sensor network to learn from manipulation (refer to [71], [72] for an introduction to kernel
previous experiences, make optimal routing actions and adapt methods). The proposed framework exploits the fact that the
to the dynamic environment. The benefits can be summarized readings of multiple sensors are highly correlated. This will
as follows: minimize the communication overhead for detecting the struc-
• Able to learn the optimal routing paths that will result in ture of the sensor data. Collectively, these results serve as an
energy saving and prolonging the lifetime of dynamically important step in developing a distributed learning framework
changing WSNs. for wireless networks using linear regression methods. The
• Reduce the complexity of a typical routing problem by main advantages of utilizing this algorithm are the good fitting
dividing it into simpler sub-routing problems. In each results, and the small overhead of the learning phase. However,
sub-problem, nodes formulate the graph structures by it cannot learn non-linear and complex functions.
 6

TABLE I: Summary of wireless sensor network routing protocols that adopt machine learning paradigms.
M ACHINE D ISTRIBUTED
R OUTING
T OPOLOGY LEARNING OVERHEAD S CALABILITY D ELAY / QOS
PROTOCOLS
ALGORITHM ( S ) CENTRALIZED
Distributed regression kernel linear
Flat / multi-hop Low Limited High Distributed No
[69] regression
SIR [70] Flat / multi-hop SOM High Limited Moderate Hybrid Yes
Q-MAP multicast
Flat /multi-hop Q-learning Low Moderate High Distributed No
[65]
Hierarchical /
RLGR [66] Q-learning Low Good Low Distributed No
geographic routing
Flat / geographic
Q-Probabilistic [68] Q-learning Low Limited High Distributed Yes
routing
FROMS [67] Flat - multi-hop Q-learning High Limited Moderate Distributed No

Neuron
(computational units) networks. Basically, the Q-MAP multicast routing algorithm is
designed to guarantee reliable resource allocation. A mobile
Output layer ad hoc network may consist of heterogeneous nodes, where
(3×4 QoS matrix) different nodes have different capabilities. In addition, it is not
feasible to maintain a global, up-to-date knowledge about the
Synapsis
(weight matrix)
whole network structure. The multicast routes are determined
in two phases. The first phase is “Join Query Forward” that
Input
discovers an optimal route, as well as updates the Q-values
layer (a prediction of future rewards) of the Q-learning algorithm.
Duty cycle
Throughput

Error rate
Latency

The second phase, called “Join Reply Backward”, creates

the optimal path to allow multicast transmissions. Using Q-
learning for multicast routing in mobile ad hoc networks can
Fig. 6: The SOM construction of the SIR algorithm, where
reduce the overhead for route searching. However, energy
routing link is selected based on the multi-hop path QoS
efficiency is the key requirement for WSNs, so Q-MAP needs
metrics (latency, throughput, error rate, and duty cycle) and
to be modified for WSNs (e.g., considering hierarchical and
the Dijkstra’s algorithm [70].
geographic routing).
The Federal Communications Commission (FCC) has ded-
2) Data routing using self-organizing map (SOM): Bar- icated the frequency band from 3.1 to 10.6 GHz (7,500 MHz
bancho et al. [70] introduced “Sensor Intelligence Routing” of spectrum) for the use of unlicensed ultra-wideband (UWB)
(SIR) by using SOM unsupervised learning to detect optimal communication [73]. UWB is a technique for transmitting
routing paths as illustrated in Fig. 6. SIR introduces a slight bulky data for short distances using a wide spectrum of
modification on the Dijkstra’s algorithm to form the network frequency bands with relatively low power. In [66], Dong et
backbone and shortest paths from a base station to every al. used a similar idea as [65] to enhance geographic routing
node in the network. During route learning, the second layer in UWB equipped sensor networks. “Reinforcement Learning
neurons compete with each other to reserve high weights in based Geographic Routing” (RLGR) protocol considers the
the learning chain. Accordingly, the weights of the winning sensor node energy and delay as metrics for formulating the
neuron and its neighboring neurons are updated to further learning reward function. This hierarchical geographic routing
match the input patterns. Clearly, the learning phase is a highly uses the UWB technology for detecting the nodes’ locations,
computational process due to the neural network generation where only the cluster heads are equipped with UWB devices.
task. As a result, it should be performed within a resourceful Moreover, each node uses a simple look-up table to maintain
central station. However, the execution phase does not incur the information about its neighbors (as location and energy
computational cost, and can be run on the network nodes. of the neighbors are needed during network learning). These
As a result, this hybrid technique (i.e., a combination of the information are exchanged between nodes using short “hello”
Dijkstra’s algorithm and the SOM model) takes into account messages to learn the best routing actions. The main benefit
the QoS requirements (latency, throughput, packet error rate, of using reinforcement learning in routing is that it does
and duty cycle) during the process of updating neurons’ not require information about the global network structure to
weights. The main obstacles of applying such an algorithm achieve an acceptable routing solution.
are the complexity of the algorithm and the overhead of the In [68], Arroyo-Valles et al. introduced “Q-Probabilistic
learning phase in the case that the network’s topology and Routing” (Q-PR), an enhanced geographic routing algorithm
setting change. for WSNs that learns from previous routing decisions (e.g., to
3) Routing enhancement using reinforcement learning (RL): select the routing path that has the highest delivery rate over
In multicast routing, a node sends the same message to several the past period of time). This protocol differs from RLGR
receivers. Sun et al. [65] demonstrated the use of Q-learning [66] in the QoS support. Depending on the importance of
algorithm to enhance multicast routing in wireless ad hoc messages, expected delivery rate, and the power constraints, Q-
 7

PR determines the optimal routes using reinforcement learning

and a Bayesian decision model. This algorithm discovers the
next hop during the message routing time (i.e., an on-line 1
opeartion). A Bayesian method is used to handle the decision 2 Cluster B
Cluster A
of transmitting the packets to the set of candidate neighbor
nodes, taking into account the data importance, nodes’ profiles,
and expected transmission and reception energy. 3
4
Förster and Murphy [67] also introduced an enhancement
to routing in WSN using reinforcement learning. A novel 5
Gateway
technique for exchanging node local information as a feedback 6
response to other nodes, named “Feedback Routing for Opti- Cluster C
mizing Multiple Sinks in WSN” (FROMS) is introduced. The
9 8 7
main advantage of FROMS is to allow efficient routing from
multiple sources to multiple sinks. The Q-values are initialized Base station Working node
based on the hop counts to every node in the network. The Dead node (non-functional node)
hop counts can be collected using short “hello messages”,
Cluster-head node
exchanged between the nodes at earlier stages of the network
deployment. FROMS extends the basic mechanism of RLGR
Fig. 7: Data aggregation example in a clustered architecture,
[66] by assuming that all nodes can directly communicate with
where the nodes are marked as working, dead and cluster
their neighbors.
heads.
The key disadvantage of reinforcement learning-based rout-
ing algorithms is the limited recognition of future knowledge
(i.e., inability to look ahead). Therefore, the algorithms are
not suitable for highly dynamic environments as they require 1) Large scale network clustering using neural network:
a long time to learn optimal routes. Hongmei et al. [79] discussed the development of self-
managed clusters using neural networks. This scheme targets
B. Clustering and Data Aggregation the clustering problem in large scale network with short
In large scale energy-constrained sensor networks, it is transmissions radii in which centralized algorithms may not
inefficient to transmit all data directly to the sink [74]. One work efficiently. However, for large transmission radii, the
efficient solution is to pass the data to a local aggregator performance of this algorithm is close to that of centralized
(known as a cluster head) which aggregates data from all the algorithms in terms of efficiency and quality of service.
sensors within its cluster and transmits to the sink. This will 2) Electing a cluster head using decision trees: Ahmed
typically result in energy savings. There are several works et al. [80] applied a decision tree algorithm to solve the
that have discussed the optimal selection of the cluster head cluster head election problem. This approach uses several
(i.e., cluster head election process), such as in [75], [76], [77]. critical features while iterating the input vector through the
Taxonomy and comparison of classical clustering algorithms decision tree such as distance to the cluster centroids, battery
are presented in [78]. level, the degree of mobility, and the vulnerability indications.
Figure 7 represents the cluster-based data aggregation from The simulation reveals that this scheme enhances the overall
sources to a base station in WSNs. In this case, there could be performance of cluster head selections when compared to the
some faulty nodes which must be removed from the network. “Low Energy Adaptive Clustering Hierarchy” (LEACH) [87]
Such faulty nodes may generate incorrect readings that could algorithm.
negatively affect the accuracy of the overall operation of the 3) Gaussian process models for sensor readings: Gaussian
network. Principally, ML techniques improve the operation of process (GP) is a combination of random variables (stochastic
node clustering and data aggregation as follows: variables) that is parameterized using mean and covariance
• Usage of machine learning to compress data locally functions. Ertin [81] presented a scheme for initializing prob-
at cluster heads by efficiently extracting similarity and abilistic models of the readings based on Gaussian process
dissimilarity (e.g., from faulty nodes) in different sensors’ regression. Comparatively, Kho et al. [82] also extended
readings. Gaussian process regression to adaptively sample sensor data
• Machine learning algorithms are employed to efficiently depending on its importance. Focusing on energy consump-
elect the cluster head, where appropriate cluster head tion, [82] studied a trade-off between computational cost and
selection will significantly reduce energy consumption solution optimality. Broadly speaking, Gaussian process mod-
and enhance the network’s lifetime. els are preferable in the problems with small training data sets
Table II compares data aggregation and node clustering (less than a few thousand samples) and for predicting smooth
solutions. The column “Balancing energy consumption” in- functions [53]. However, WSN designers must consider the
dicates whether the protocol distributes computationally in- high computational complexity of such methods when dealing
tensive tasks into all nodes while considering the remaining with large scale networks.
energy information. The column “Topology aware” indicates 4) Data aggregation using self-organizing map (SOM):
the requirement for full network topology knowledge. The SOM algorithm is an unsupervised, competitive learning
 8

TABLE II: Compassion of different machine learning-based data aggregation and node clustering mechanisms.
M ACHINE BALANCING
T OPOLOGY
M ECHANISMS LEARNING C OMPLEXITY ENERGY CON - D ELAY OVERHEAD
AWARE
ALGORITHM ( S ) SUMPTION
Large scale network clustering [79] NNs Moderate Yes High Low Yes
Cluster head election [80] DT Low Yes Low Low Yes
Gaussian process models for censored
Moderate No Moderate Moderate No
sensor readings [81] GPR
Adaptive sampling [82] High Yes High High No
Clustering using SOM and sink distance
SOM Moderate No High Moderate Yes
[54]
Online data compression [83] LVQ High No High High Yes
Data acquisition using compressive sensing
High Yes High High Yes
[55], [56]
PCA
Transmission reduction [57] Moderate No High High Yes
Consensus-based distributed PCA [58] Moderate Yes High High No
Lossy data compression [84] Moderate No Moderate High Yes
Collaborative signal processing [60] Low Yes Moderate Moderate No
k-means
Advanced surveillance systems [59] Moderate Yes Low Low Yes
Role-free clustering [85] Q-learning Low No Low Low No
Decentralized learning for data latency [86] RL Moderate Yes Low Low No

method for mapping from high dimensional spaces to low the training samples, will never take part in the competition.
dimensions. Lee et al. [54] proposed a novel network architec- Therefore, it is important to develop algorithms that are robust
ture called “Cluster-based self-Organizing Data Aggregation” against outliers. By the same token, LVQ is suitable for
(CODA). In this architecture, the nodes are able to classify representing big data set by few vectors [43].
the aggregated data using a self-organizing algorithm. The 6) Data aggregation using principal component analysis:
winning neuron j ∗ , that has a weight vector w(t) closest to We begin by introducing two important algorithms that are
the input vector x(t), is defined as: efficiently used in combination with principal component
analysis (PCA) to enhance data aggregation in WSNs.
j ∗ = arg min kxj (t) − wj (t)k , j = 1, ..., N (2) • Compressive sensing (CS) has been recently explored to
j
replace the traditional scheme of “sample then compress”
where N represents the number of neurons in the second layer. with “sample while compressing”. CS explores sparsity
Further, the winning node and its neighbors are updated as property of signals to recover the original signal from
follows: few random measurements. A simple introduction to CS
is provided in [88].
wj (t + 1) = wj (t) + h (t) (xj (t) − wj (t)) (3) • Expectation-maximization (EM) [89] is an iterative

where w (t) and w (t + 1) represent the values of a neuron at algorithm composed of two steps, i.e., an expectation (E)
time t and t+1, respectively. In addition, h (t) is the Gaussian step and a maximization (M) step. During its E-step, EM
neighborhood function given as: formulates the cost function while fixing the current ex-
pectation of the system parameters. Subsequently, the M-
!
1 kj ∗ − jk
2 step recomputes parameters that minimize the estimation
h (t) = √ exp − . (4) error of the cost function.
2πσ 2σ 2 (t)
Masiero et al. [55], [56] developed a method for estimating
Using CODA for data aggregation will result in enhancing distributed observations using few collected samples from
the quality of data, saving network energy, and reducing the a WSN. This solution is based on the PCA technique to
network traffic. produce orthogonal components used by compressive sensing
5) Applying learning vector quantization for online data to reconstruct the original readings. Moreover, this method
compression: While the above methods require a complete is independent of the routing protocol due to its ability to
knowledge about the network topology, some algorithms may estimate data spatial and temporal correlations. Similarly,
not have such a restriction. For example, Lin et al. [83] in- Rooshenas et al. [57] applied PCA to optimize the direct
troduced a technique called “Adaptive Learning Vector Quan- transmission of readings to a base station. PCA results in
tization” (ALVQ) to accurately retrieve compressed versions considerable traffic reduction by combining nodes’ collected
of readings from the sensor nodes. Using data correlation and data into fewer packets. This distributed technique is executed
historical patterns, ALVQ uses the LVQ learning algorithm in intermediate nodes to combine all the incoming packets
to predict the code-book using past training samples. The instead of forwarding them to destinations.
ALVQ algorithm minimizes the required bandwidth during Equally important, Macua et al. [58] introduced distributed
transmission, and enhances the accuracy of original reading consensus-based methods for data compression using PCA and
recovery from the compressed data. maximum likelihood of the observed data. These methods
The crucial disadvantage of using LVQ for online data are “Consensus-based Distributed PCA” (CB-DPCA) which
aggregation is that dead neurons, that are far away from relies on exploring the eigenvectors of local covariance ma-
 9

trices, and “Consensus-based EM Distributed PCA” (CB-EM-

Query (2) Broadcast query
DPCA). The latter uses a distributed EM algorithm. These ize
itial processing and
In aggregate results
methods adopt the consensus algorithm [90] to predict the (1
) unit

probability distribution of the data, and hence calculate the rega

te
Agg Event
global dominant eigenvectors using only local communication System (3) esults Assess event
r
controller accuracy
parameters (i.e., single hop communications). CB-DPCA and
CB-EM-DPCA can be tuned to provide a trade-off between [2]
Ala
the achieved approximation quality and the communication rm Event
[1] Declare Event
detection unit n
cost by adjusting the consensus round parameter. For example, the detectio
to increase the algorithm accuracy, the number of consensus
rounds should be increased which will increase the computa-
tional requirements of the algorithm. Fig. 8: Event detection and query processing enhancement
Recently, Fenxiong et al. [84] have tackled the problem of using machine learning methods by assessing event validity
data compression using PCA by transforming the data from a and delimiting queried areas. System controller initiates query
high dimensional space to a lower one. The data is collected that is spread by the query processing unit to intended nodes.
over time, and then it is transmitted from each node to its In contrast, events are detected by nodes to monitor specific
corresponding cluster head. At the cluster head, the data matrix signs within the monitored area.
is compressed to eliminate the data redundancy. The data
compression is achieved by ignoring principal components
node executes the learning algorithm locally to optimize the
through which the data has the least variation values.
data aggregation without the need for a central control station.
High computational requirement is the main issue of
Consequently, the efficiency of the whole network is enhanced
PCA-based data aggregation solutions. Other than increas-
with smaller learning transmission overhead. The approach
ing throughput, these solutions elegantly cope with the high
saves the node energy budget during data gathering process,
dimensionality of collected data by keeping only important
and hence prolong the network lifetime.
information (data dimensionality reduction).
7) Collaborative data processing through k-means algo-
C. Event Detection and Query Processing
rithm: Li et al. [60] addressed the fundamental concepts for
distributed detection and tracking of a single target using Event detection and query processing are considered to be
sensor networks. “Collaborative Signal Processing” (CSP) is functional requirements of any large scale sensor network.
a framework for information gathering from the monitored This introduces the need for trustworthy event scheduling and
environment. Additionally, this algorithm can track multiple detection with minimal human intervention. Monitoring in
targets using classification techniques such as SVM and k- WSNs can be classified as: event-driven, continuous, or query-
nearest neighbors. driven [6]. Figure 8 illustrates event detection and query pro-
Classical surveillance systems have to collect massive data cessing operations in WSNs. Fundamentally, machine learning
from surveillance cameras. Together with the requirement offers solutions to restrict query areas and assess event validity
of highly complex computation and analysis process, this for efficient event detection and query processing mechanisms.
introduces the need for more practical methods. Therefore, This adoption will result in the following benefits:
Tseng et al. [59] proposed “Integrated Mobile Surveillance • Learning algorithms enable the development of efficient
and Wireless Sensor System” (iMouse) which adopts powerful event detection mechanisms with limited requirements
mobile sensors to enhance traditional surveillance systems. of storage and computing resources. Besides they are
iMouse divides the monitored sites into a number of clusters able to assess the accuracy of such events using simple
using the k-means unsupervised learning algorithm. Each clus- classifiers.
ter will be repeatedly monitored by only one mobile sensor. • Machine learning facilitates the development of effective
Although these ideas (using k-means for data processing) query processing techniques for WSNs, that determine
are appealing because of the straightforward implementations the search regions whenever a query is received without
and low complexity, they are still sensitive to outliers and to flooding the whole network.
initial seed selections. The design of effective event detection and query process-
8) Role-free clustering: In [85], Förster and Murphy in- ing solutions has recently received increased attention from
troduced the WSN cluster formulation method called “Role- WSNs research community. The simplest techniques rely on
Free Clustering with Q-Learning for Wireless Sensor Net- defining a strict threshold value for the sensed phenomenon
works” (CLIQUE). Instead of performing an election process, and alarming the system manager of any violations. However,
CLIQUE enables each node to investigate its ability to function in most recent applications of WSNs, event and query pro-
as a cluster-head node. This is achieved through the use of Q- cessing units are often complicated and require more than a
learning algorithm in combination with some dynamic network predefined threshold value. One such emerging technique is to
parameters such as energy levels. use machine learning to develop advanced event detection and
9) Decentralized learning for data latency: Mihaylov et query processing solutions. Table III presents a comparison of
al. [86] addressed the problem of high data latency in random functional aspects of different machine learning-based event
topology sensor networks using reinforcement learning. Each detection and query processing solutions for WSNs.
 10

TABLE III: Comparison of functional aspects of different machine learning-based event detection and query processing solutions
for WSNs.
M ACHINE
D ATA D ELIVERY
A PPROACHES LEARNING C OMPLEXITY C HARACTERISTICS
M ODELS
ALGORITHM ( S )
Event region detection [91] Low Fault-tolerant event region detection
Bayesian Event-driven
Activity recognition [92] Moderate Real-time activity recognition
In-network query processing [24] Low In-network query processing
k-NN Query-driven
Query processing in 3D space [25] Moderate Enhance 3D space query processing
Real-time and lightweight forest fire
Forest fire detection [26] NNs Event-driven Moderate
detection
Distributed event detection [27] DT Event-driven Low Disaster prevention system
Query optimization and dimensionality
Query optimization [93] PCA Query-driven High
reduction

First sensor data [95] and references therein for requirements and challenges
of such systems). Moreover, the use of WSNs for forest fire
Preprocessing HMM
detection can achieve better performance than using satellite-
based solutions while costing much less. Yu et al. [26]

Activity type
Preprocessing HMM presented a real-time forest fire detection scheme based on a
. Naive
. . Bayes neural network method. Data processing will be distributed
.
. . to cluster heads, and only important information will be
.
Last sensor data . . aggregated to a final decision maker. Although the idea is
Preprocessing HMM creative and beneficial to the environment, the classification
task and system core are hardly interpretable when introducing
such systems to decision makers.
Fig. 9: Human activity recognition using the hidden Markov
3) Query processing through k-nearest neighbors: K-
model and the naive Bayes classifier [92].
nearest neighbor query is considered as a highly effective
query processing technique in WSNs. For example, Winter
et al. [24] developed an in-network query processing solution
1) Event recognition through Bayesian algorithms: Krish- using the k-nearest neighbor algorithm, namely the “K-NN
namachari and Iyengar [91] investigated the use of WSNs for Boundary Tree” (KBT) algorithm. Each node that is aware of
detecting environmental phenomenon in a distributed manner. its location will determine its k-NN search region whenever a
Readings will be considered as faulty if their values exceed a query is received from the application manager.
specific threshold. This study employs decentralized Bayesian Correspondingly, Jayaraman et al. [25] extended the query
learning that detects up to 95 percent of the faults, and will processing design of [24]. “3D-KNN” is a query processing
result in recognizing the event region. It is important to note scheme for WSNs that adopts the k-nearest neighbor algo-
that Chen et al. [94] provided corrections to several errors rithm. This approach restricts the query region to bound at
related to the distributed Bayesian algorithms that have been least k-nearest nodes deployed within a 3D space. In addition,
derived in [91]. In summary, these corrections result in en- signal-to-noise ratio (SNR) and distance measurements are
hanced error and performance calculations for the distributed used to refine the k-nearest neighbor.
Bayesian algorithm proposed in [91]. The primary concerns of such k-NN-based algorithms for
Additionally, Zappi et al. [92] presented a real-time ap- query processing are the requirement of large memory foot-
proach for activity recognition using WSNs that accurately print to store every collected sample and the high processing
detects body gesture and motion. Initially, the nodes, that are delay in large scale sensor networks.
spread throughout the body, detect the organ motion using an 4) Distributed event detection for disaster management
accelerometer sensor with three axis measurements (positive, using decision tree: Bahrepour et al. [27] developed decision
negative and null), where these measurements are used by a tree-based event detection and recognition for sensor network
hidden Markov model (HMM) to predict the activity at each disaster prevention systems. The main application of this
sensor. Sensor activation and selection rely on the sensor’s decentralized mechanism is the fire detection in residential
potential contributions in classifier accuracy (i.e., select the areas. Most noteworthy, the final event detection decision is
sensors that provide the most informative description of the made by using a simple vote from the highest reputation nodes.
gesture). To generate a final gesture decision, a naive Bayes 5) Query optimization using principal component analysis
classifier is used to combine the independent node predictions (PCA): Malik et al. [93] optimized traditional query process-
so as to maximize the posterior probability of the Bayes ing in WSNs using data attributes and PCA, thus reducing the
theorem. The architecture of the proposed system is shown overhead of such a process. PCA has been used to dynamically
in Fig. 9. detect important attributes (i.e., dominant principal compo-
2) Forest fire detection through neural network: WSNs nents) among the whole correlated data set. Figure 10 shows
were actively used in fire detection and rescue systems (see the workflow of the proposed algorithm in four fundamental
 11

Base station

(3) Diffuse the

optimized query

(1) SQL query (4) Sensory data

(2) Optimized query

d3
(PCA components)
d2

Wireless sensor
network d4
Historical data

d1
Database management d5
and optimization system

Fig. 10: Workflow of the query optimization and reduction

system using PCA proposed in [93] Unknown node.

Beacon node (anchor node).

steps. In Step 1, the structured query language (SQL) request, Fig. 11: Localization using few beacon nodes by utilizing ma-
which contains the human intelligible attributes, is sent to chine learning algorithms and other signal strength indicators
the database management and optimization system. At the (reformulated from [96]).
database management and optimization system, the original
query is optimized where the high-variance components are
extracted from historical data using the PCA algorithm (Step
using machine learning algorithms in sensor node localization
2). Then, the optimized query is diffused to the wireless sensor
process can be summarized as follows:
network to extract the sensory data as shown in Steps 3
• Converting the relative locations of nodes to absolute
and 4, respectively. Later, the original attributes (i.e., human
intelligible attributes) can be extracted from the optimized ones using few anchor points. This will eliminate the
attributes by reversing the process of PCA. need for range measurement hardware to obtain distance
As a result, this algorithm guarantees 25 percent improve- estimations.
• In surveillance and object targeting systems, machine
ment in energy saving of the network nodes while achieving
93 percent of accuracy rates. However, this enhancement is at learning can be used to divide the monitored sites into a
the cost of accuracy of the collected data (as some of the data number of clusters, where each cluster represents specific
location indicator.
components will be ignored). Therefore, this solution may not
be ideal for the applications with high accuracy and precision We begin by defining some terms that are widely used in WSN
requirements. localization literature, as illustrated in Figure 11.
• Unknown node is a node that cannot determine its
current location.
D. Localization and Objects Targeting • Beacon node (or anchor node) is any node that is able
Localization is the process of determining the geographic to recognize its location by using positioning hardware
coordinates of network’s nodes and components. Position or from its manual placement. In most systems, the
awareness of sensor nodes is an important capability, since beacon node is used as a reference point to estimate the
most sensor network operations are typically based on the coordinates of other unknown nodes.
location [96]. In most large scale systems, it is financially • Received signal strength indication (RSSI) is an indi-
infeasible to use global positioning system (GPS) hardware in cator of the received signal strength, used to represent
each node for this purpose. Moreover, GPS service may not be transmission performance or distance.
available in the observed environment (e.g., indoor). Relative Next, we discuss some seminal WSN localization tech-
location measurement is sufficient for certain uses. However, niques that use machine learning and summarize our reviews
by using the absolute locations for a small group of nodes, in Table IV.
relative locations can be transformed into absolute ones [97]. 1) Bayesian node localization: Morelande et al. [21] used
In order to enhance the performance of proximity based local- a Bayesian algorithm to develop a localization scheme for
ization, additional measurements relying on distance, angle or WSNs using only few anchor points. This study focuses on
a hybrid of them can be used. Distance measurements can be the enhancement of progressive correction [109], which is a
obtained by utilizing various techniques such as RSSI, TOA, method for predicting samples from likelihoods to get closer to
and TDOA. Furthermore, angle of the received signal can be the posterior likelihood. The proposed algorithm is efficiently
measured using compasses or special smart antennas [98]. A applicable for node localization in large scale systems (i.e.,
valuable introduction about the basics of different range-based networks with a few thousands of nodes). The idea of using
localization techniques is provided in [42]. the Bayesian algorithm for localization is appealing as it can
Sensor nodes may encounter changes in their location handle incomplete data sets by investigating prior knowledge
after deployment (e.g., due to movement). The benefits of and probabilities.
 12

TABLE IV: Summary of localization algorithms in WSNs that adopt machine learning concepts and their prime advantages.
The column “Applications” specifies the targeted application(s) of the proposed solution (either general-purpose or a specific
application).
M ACHINE
D ISTRIBUTED / B EACON /
S TUDIES DESCRIPTION LEARNING C OMPLEXITY A PPLICATIONS
CENTRALIZED B EACON - LESS
ALGORITHM ( S )
Bayesian node localization [21] Moderate Centralized Beacon General-purpose
Bayesian
Location-aware activity recognition
Moderate Centralized Beacon Smart homes
[22]
Localization based on NNs [23] High Centralized Beacon General-purpose
Soft localization [99] NNs Moderate Distributed Beacon General-purpose
Localization based on NNs [100] High Distributed Beacon General-purpose
Area localization [51] Moderate Distributed Beacon General-purpose
SVM
Localization using SVM [50] Moderate Distributed Beacon General-purpose
Localization using SVR [49] SVR Moderate Distributed Beacon General-purpose
Target classification and information
DT Low Distributed Beacon Acoustic WSNs
fusion [101]
Underwater surveillance system [102] Moderate Centralized Beacon Submarine surveillance
Sensor placements [103] GP Low Distributed Beacon Node deployment
Spatial Gaussian process regression
GPR Moderate Distributed Beacon Collective node motion
[104]
Localization in 2D space [105] Moderate Distributed Beacon Large scale WSNs
Localization using SOM [106] SOM Low Centralized Beacon-less General-purpose
Distributed localization [107] Moderate Distributed Beacon-less General-purpose
Path determination [108] RL Low Distributed Beacon Mobile nodes

2) Robust location-aware activity recognition: Lu and Fu class, the neural network is adopted to predict the sensor
[22] addressed the problem of sensor and activity localization location by using RSSI measurements from all anchor nodes
in smart homes. The activities of interest include using the as an input vector. In the same way, Chagas et al. [100] applied
phone, listening to the music, using the refrigerator, studying, neural networks for WSNs localization with RSSI as an input
etc. In such applications, designers need to comply with both to the learning network.
human and environment constraints in a convenient and easily The main advantage of these NNs-based localization al-
operated way. The proposed framework, named “Ambient gorithms is their ability to provide coordinates in the form
Intelligence Compliant Object” (AICO), facilities the human of continuous-valued vectors (e.g., coordinates in 3D space).
interaction with the home electric devices in a more intelligent However, unlike statistical or Bayesian alternatives, neural
manner (e.g., automatic power supply management). At its network is a non-probabilistic method. This fact limits the
core, AICO uses multiple naive Bayes classifiers to determine designers’ certainty about precision of unknown node’s pre-
the resident’s current location and evaluate the reliability of dicted coordinates, and hence restricts their ability to manage
the system by detecting any malfunctioned sensors. Although the cost of localization errors.
this method provides a robust mechanism for localization, it 4) Localization using support vector machine (SVM): The
is still application-dependent and the designers must predefine SVM technique has been widely used for node localization in
a set of supported activities in advance. This is because the WSNs, where having a self-positioning device to each sensor
used learning features are selected and evaluated manually is infeasible. As an illustration, Yang et al. [51] developed
depending on the activities and the domain of interest. To over- a mobile node localization scheme by employing SVM and
come this limitation in this centralized system, we recommend connectivity information capabilities. In its initial step, the
investigating unsupervised machine learning algorithms for proposed method has to detect node movement using their
automatic feature extraction such as the deep learning methods radio frequency oscillation such as RSSI metric. For movement
[9] and the non-negative matrix factorization algorithm [110]. detection, SVM will be executed to provide the new location.
3) Localization based on neural network: Shareef et al. Similar to [51], Tran and Nguyen [50] proposed “Local-
[23] compared three localization schemes that are based on ization Based on Support Vector Machines” (LSVM) method
different types of neural networks. In particular, this study con- for node localization in WSNs. To achieve its design goals
siders WSN localization using multi-layer perceptron (MLP), and given an appropriate training data, LSVM adopts several
radial basis function (RBF), and recurrent neural networks decision metrics such as connectivity information and indica-
(RNN). In summary, the RBF neural network results in the tors. Even though LSVM offers distributed localization in a
minimum error at the cost of high resource requirements. fast and effective manner, its performance is still sensitive to
In contrast, MLP consumes the minimum computational and outliers in training samples.
memory resources. 5) Localization using support vector regression (SVR):
Likewise, Yun et al. [99] adopted a similar design, in which Limited resources and high data dimensionality impede the
two classes of algorithms for sensor node localization using wide adoption of SVR learning in WSNs. Therefore, Kim et al.
RSSI from anchor nodes are proposed. The first class utilizes [49] developed the idea of using lightweight implementation of
the fuzzy logic system and genetic algorithm. In the second SVR by dividing the original regression problem into several
 13

sub-problems. Basically, the algorithm starts by dividing the input layer connected to the 2 neurons of the output layer.
network into a set of sub-networks, thus a small number of In particular, the input layer is formulated using the spatial
data has to be processed by each regression algorithm (i.e., coordinates of 8 anchor nodes surrounding the unknown node.
SVR’s sub-predictors). Then, the learned hypothesis models of After a sufficient training, the output layer is used to represent
the sub-predictors are combined together using a customized the unknown node’s spatial coordinates in a 2D space. The
ensemble combination technique. Thus, in addition to its low main disadvantage of this scheme is that the nodes should
computational requirements and robustness against noisy data, be distributed uniformly and equally spaced throughout the
this solution converges to the preferred solution with low monitored area.
computational requirement. Unlike traditional methods that require absolute locations
6) Decision tree-based localization: Based on decision tree of a few nodes to find the positions of the unknown nodes,
learning, Merhi et al. [101] developed an acoustic target Giorgetti et al. [106] introduced a localization algorithm that
localization method for WSNs. Exact locations of targets are is only based on connectivity information and the SOM algo-
determined using the time difference of arrival (TDOA) metric rithm. The developed method is highly suitable for networks
in a spatial correlation decision tree. Also, this work proposed with limited resources, as it does not require a GPS-enabled
the design of “Event Based MAC” (EB-MAC) protocol, that device. However, since this is a centralized algorithm, each
enables event-based localization and targeting in acoustic node transmits the information of its neighbors to the central
WSNs. The proposed framework was implemented using a processing unit to determine the adjacency matrix and hence
MicaZ board that supports ZigBee 802.15.4 specifications for the node’s location. Similarly, Hu and Lee [107] presented
personal area networks. a scheme that provides node localization service in WSNs
Using the GPS functionality to support localization in without the need for anchor nodes. The algorithm is based on
underwater wireless sensor network’s applications may not be SOM, and it operates efficiently for any number of nodes. The
feasible due to the propagation limitation of the GPS signal contribution of [107] over [106] is that the proposed algorithm
through water [111]. Erdal et al. [102] developed a system distributes the computation tasks to all nodes in the network,
for submarine detection in underwater surveillance systems, which eliminates the needs for a central unit and minimizes
so that a randomly deployed node finds its location in the 3D the transmission overhead of the algorithm.
space based on beacon node coordinates. Each monitoring unit 10) Path determination using reinforcement learning: Li et
consists of a sensor that is fixed with a cable to a surface buoy. al. [108] developed a reinforcement learning-based localiza-
Data is collected using the buoys, where they are transmitted to tion method for WSNs, called “Dynamic Path determination of
the central processing unit. At the central unit, a decision tree Mobile Beacons” (DPMB), suitable for real-time management
classifier is used to recognize any submarines in the monitored of the mobile beacons. The mobile beacon (MB), which is
sites. aware of the physical location during its movement, will be
7) Sensor placements through Gaussian processes: Krause used to determine the positions of large number of sensor
et al. [103] provided an optimized solution to sensor place- nodes. In brief, the states of the Q-learning algorithm are
ment in applications with spatially correlated data such as used to represent the different positions of the MB, and the
temperature monitoring systems. One interesting feature of this algorithm target is to cover all the sensors in the monitored
solution is the development of a lazy learning scheme based area (i.e., all the sensors should hear a location update message
on Gaussian process model for the investigated phenomenon. from the MB at some stages). The entire operation will be run
Lazy learning algorithms store training samples and delay the in the mobile beacon, and hence, this will save the resources
major processing task until a classification request is received. of the unknown nodes. However, as a centralized method,
Moreover, this solution aims to achieve robustness against the entire system will fail in the event of mobile beacon
node failures and model ambiguity when choosing optimal malfunctions.
locations for sensors.
8) Spatial Gaussian process regression: Gu and Hu [104] E. Medium Access Control (MAC)
developed a distributed protocol for collective node motion. In WSNs, a number of sensors cooperate to efficiently
This approach employs distributed Gaussian process regres- transfer data. Therefore, designing MAC protocols for WSNs
sion (DGPR) to predict optimal locations for mobile nodes’ poses different challenges from typical wireless networks, as
movements. Traditional Gaussian process regression (GPR) well as energy consumption and latency [112]. Also, the duty
algorithms have computational complexity of O(N 3 ), where cycle (i.e., fraction of time that a sensor node is active) of
N is the size of samples. However, this solution adopts the node has to be controlled to conserve energy. Therefore,
a sparse Gaussian process regression algorithm to reduce the MAC protocols have to be modified to support efficient
such computational complexity. Each node will execute the data transmission and reception of the sensor nodes. A com-
regression algorithm independently using only spatiotemporal prehensive survey of MAC protocols in WSNs is provided in
information from local neighbors. [113].
9) Localization using self-organizing map (SOM): Given Recently, machine learning methods have been used to
some anchor positions, Paladina et al. [105] introduced the enhance the performance of MAC protocols in WSNs. Specif-
SOM-based positioning solution for WSNs consisting of thou- ically, this is achieved through the following points:
sands of nodes. The proposed scheme is executed in each • Machine learning can be used to adaptively determine
node with a simple SOM algorithm that consists of a 3x3 the duty cycle of a node using the transmission history
 14

of the network. In particular, the nodes, which are able to Success Collision Collision Success Success
predict when the other nodes’ transmissions will finish,
can sleep in the meantime and wake up (to transmit data)

Q-values
just when the channel is expected to be idle (i.e., when no 0 0 0 0.1 0 -0.1 -0.01 0.1 -0.1
other node is transmitting). For WSNs, many factors, such
Frame 1 Frame 2 Frame 3
as energy consumption and latency, are more important
than fairness when designing MAC protocols. Fig. 12: An example of the Q-values of a node over three
• Achieving secured data transmission by combining the
frames in a WSN that employs ALOHA-QIR to manage
concepts of machine learning and MAC protocols. Such medium access [114].
MAC layer security schemes are independent of the
proposed application and are able to iteratively learn
sporadic attack patterns. in a frame-based structure. RL-MAC adaptively determines the
Table V gives a brief comparison between MAC protocols slot length, duty cycle and transmission active time depending
reviewed in this subsection. The column “Synchronization” in- on the traffic load and the channel bandwidth.
dicates whether the protocol assumes that time synchronization Similarly, Chu et al. [112] integrated slotted ALOHA and
is achieved externally, and “Adaptivity to changes” indicates Q-Learning algorithms to introduce a new MAC protocol for
the ability to handle topology changes such as nodes failure. WSNs, called “ALOHA and Q-Learning based MAC with
1) Bayesian statistical model for MAC: Kim and Park [28] Informed Receiving” (ALOHA-QIR). ALOHA-QIR inherits
presented a contention-based MAC protocol for managing the features of both ALOHA and Q-Learning to achieve the
active and sleep times in WSNs. Instead of continuously benefits of simple design, low resource requirements and low
sensing the medium, this scheme utilizes a Bayesian statistical collision probability. During their transmission frames, nodes
model to learn when the channel can be allocated, and hence broadcast their future transmission allocation such that other
save network energy. Furthermore, in its basic design, this nodes can sleep during reserved frame. The Q-value map
scheme is targeted for CSMA contention-based protocols such in each node represents the willingness for slot reservation,
as “Sensor MAC” (S-MAC) [116], and “Timeout MAC” (T- where the node with higher Q-value will attain the right of
MAC) [117]. slot allocation and hence transmission of its own data. Figure
2) Neural network-based MAC: Time division multiple 12 demonstrates the steps of updating the Q-values over three
access (TDMA)-based protocols employ periodic time frames frames of a node that is allowed to transmit a maximum of two
to separate the medium access of different nodes. This process packets in each frame. Initially, the Q-values are initialized
requires a central unit to broadcast a transmission schedule to zero, i.e., Q (f rame#0) = {0, 0, 0}. Upon successful
in case of topology changes. Shen and Wang [29] proposed transmission, the Q-value of each time slot is updated using the
a solution to broadcast the transmission schedule in TDMA Q-learning update rule given by Eq. (1) where the learning rate
using a fuzzy hopfield neural network (FHNN) technique. is set to 0.1. Upon successful transmission, the reward value
Time slots are distributed among the nodes in a network is equal to +1, and it is -1 for a failed transmission. Certainly,
while maximizing the cycle length, preventing any potential the nodes will decide to transmit data using the time slots with
transmission collisions and reducing the processing time. the maximum Q-values.
In the same way, Kulkarni and Venayagamoorthy [30] Although the idea of using reinforcement learning for duty
presented an innovative CSMA-based MAC solution, that can cycle management is appealing because of its distributed
prevent denial-of-service (DoS) attacks in WSNs. Denial-of- operation and small memory and computational resource re-
service is a type of attacks that generates huge useless traffic quirement, it may result in high collision rates during the initial
(i.e., flood the network), thus preventing the delivery of useful exploration phases.
data. In such cases, attackers exploit the limitations of WSNs 4) Adaptive MAC layer: In a variety of modern applica-
such as limited bandwidth and buffering capabilities. The tions, such as in healthcare and assisted living systems, WSNs
proposed solution is based on neural network learning to are used to directly share the collected data with the users’
prevent flooding the network with untruthful data traffic by mobile phones. This introduces new design challenges that
investigating the network properties and variations such as are related to the dynamic communication patterns and service
packet request rate and average packet waiting time. Conse- requirements over time. Sha et al. [115] studied this problem
quently, the MAC layer will be blocked if the neural network at the MAC layer, hence proposing the “Self-Adapting MAC
output exceeds a predefined threshold level. More importantly, Layer” (SAML) design. SAML is composed of two main
only nodes in the affected sites will be blocked, as this solution components: The “Reconfigurable MAC Architecture” (RMA)
is designed to work in a distributed manner. to switch between the different MAC protocols, and the
3) Duty cycle management using reinforcement learning: MAC engine that is used to learn the suitable MAC protocol
Liu and Elhanany [114] employed a reinforcement learning for the current network conditions. The learning process is
technique to introduce RL-MAC, an adaptive MAC protocol performed using the decision tree classier as illustrated in
for WSNs. Basically, RL-MAC reduces energy usage and Fig. 13. The learning features of the decision tree are: inter
increases throughput by optimizing the duty cycle of the packet interval (IPI) and received signal strength indication
network node. Similar to S-MAC [116] and T-MAC [117], RL- (RSSI) statistical parameters (i.e., the mean and the variance),
MAC synchronizes node’s transmission on a common schedule the application QoS requirements (reliability, energy usage,
 15

TABLE V: Comparison of MAC protocols.

M ACHINE
A DAPTIVITY
MAC PROTOCOLS LEARNING C OMPLEXITY C ATEGORY T YPE S YNCHRONIZATION
TO CHANGES
ALGORITHM ( S )
Bayesian Statistical
Bayesian Moderate Contention–Based CSMA Good No
Modeling [28]
Broadcast scheduling [29] Low Contention–Free TDMA Weak Yes
NNs
NN-based secure MAC [30] Moderate Contention–Based CSMA/CA Average No
RL-MAC [114] Moderate Contention–Based CSMA Good Yes
RL
ALOHA-QIR [112] Low Contention–Free Slotted Aloha Weak Yes
Adaptive MAC layer [115] DT High Hybrid Hybrid Good Yes

Sensor measurement
Compatibility (e.g., temperature, Anomalies readings
Not requireed Requireed
pressure ...etc.)

IPI variation ZigBee

>20 <=20
Expected readings
Adaptive
TDMA
1st priority
Energy Latency PDR

Pure
Environment IPI mean
TDMA
Clean Noisy >=140 <=140

Pure
IPI mean IPI mean 2nd priority Sensor’s location
TDMA
Latency
indicator
<=170 >170 <=780 >780 Energy

RI- Box- RI- Box- Pure RI-

MAC MAX MAC MAX TDMA MAC Fig. 14: An example of anomaly detection in phenomena
monitoring sensor system using machine learning clustering
Fig. 13: The decision tree classifier used to select the optimal and classification techniques (data set in euclidean space).
MAC algorithm in the SAML architecture [115].

network. Figure 14 presents the general concept of anomaly

and latency), packet delivery rate (PDR), and the traffic detection in phenomenon monitoring sensor system using
pattern. The supported MAC protocols are Pure TDMA [118], machine learning clustering and classification algorithms. In
Adaptive TDMA [119], Box-MAC [120], RI-MAC [121], and this example, machine learning techniques classify the data
ZigBee [122]. Even though the SAML scheme provides an into two correct reading regions. Since most observations lie
adaptive MAC solution in dynamic environments, it introduces in these two regions, the points that are inconsistent (e.g., from
a level of complexity and additional expense into the designed an attack) with these regions are considered as anomalies.
systems. Machine learning algorithms have been employed to detect
outlying and misleading measurements. Simultaneously, sev-
IV. NON-FUNCTIONAL CHALLENGES eral attacks could be detected by analyzing well-known ma-
Non-functional requirements include specifications that are licious activities and vulnerabilities. Basically, WSN security
not related to the basic operational behavior of the system. enhancements by adopting machine learning techniques will
For example, WSN designers may need to ensure that the result in the following earnings:
proposed solution is always capable of providing up-to-date • Save node’s energy and significantly expand WSN life-
information about the monitored environment. This section time by preventing the transmission of the outlier, mis-
provides a comprehensive review of recent machine learning leading data.
advances that have been adopted to achieve non-functional • Enhance network reliability by eliminating faulty and
requirements in WSNs such as security, quality of service, malicious readings. In the same way, avoiding the dis-
and data integrity. Moreover, this section also highlights some covery of unexpected knowledge that will be converted
unique efforts in specialized WSN applications. Such studies to important, and often critical actions.
could inspire researchers to a variety of WSN applications that • Online learning and prevention (without human interven-
can be improved using machine learning techniques. tion) of malicious attacks and vulnerabilities.
In this subsection, we explore various machine learning-based
A. Security and Anomaly Intrusion Detection algorithms addressing the security issue in WSNs. Table VI
The major challenge to implement security techniques in summarizes the reviewed methods in this subsection. The
WSNs is the limited resource constraints [14]. Moreover, column “Predicting missing data” indicates the ability of
some attack methods aim to produce unexpected, mistaken the proposed solution to provide predictions for any missing
knowledge, by introducing misleading observations to the sensors’ readings.
 16

TABLE VI: Summary of wireless sensor network outlier detection techniques that adopt machine learning paradigms.
M ACHINE P REDICTING D ISTRIBUTED
S TUDIES LEARNING MISSING / C OMPLEXITY A IM ( S )
ALGORITHM ( S ) DATA CENTRALIZED
Outlier detection using Bayesian
Bayesian Yes Distributed Low Outlier detection
belief networks [31]
Outlier detection using k-NN [32] k-NN Yes Distributed Moderate Distributed outlier detection
Detecting selective forwarding Detect black hole and selective
Centralized Moderate
attacks using SVM [33] forwarding attacks
Distributed outlier detection using
SVM No Distributed Low Outlier detection
SVM [34]
Online outlier detection [46] Centralized Moderate Online outlier detection
Intrusion detection using immune
Intrusion detection system [47] Centralized High
and SVM algorithms
Linear outlier detection [48] Distributed Moderate Adaptive outlier detection
Analyzing attacks with SOM
SOM No Distributed Moderate Detect anomalous behaviors
[123]

1) Outlier detection using Bayesian belief network: Janaki- detection algorithms. This outlier detector is similar to the
ram et al. [31] used Bayesian belief networks (BBNs) to method introduced in [34].
develop an outlier detection scheme. Given that the majority of Artificial immunity algorithm is a computationally intelli-
node’s neighbors will have similar readings (i.e., temporal and gent algorithm for problem solving inspired by the biological
spatial correlations), it is reasonable to use this phenomenon to immunity systems [124]. The biological immunity systems
build conditional dependencies among nodes’ readings. BBNs automatically generate the immune body (antibody) against the
infer the conditional relationships among the observations to antigen (e.g., a virus) through the cell fission. In [47], Chen et
discover any potential outliers in the collected data. Further- al. extended the basic idea of using SVM for detecting intru-
more, this method can be used to evaluate missing values. sion by combining it with immunity algorithm. In summary, an
2) Outlier detection using k-nearest neighbors: Branch et immune algorithm was introduced as a preprocessing step for
al. [32] developed an in-network outlier detection method the sensor data, that will be used by SVM to detect intruders.
in WSNs using k-nearest neighbors. Moreover, any missing Furthermore, Zhang et al. [48] also investigated the temporal
nodes’ readings will be replaced by the average value of and spatial correlations of the collected readings using a one-
the k-nearest nodes. However, such non-parametric, k-NN- class SVM learning algorithm to develop an outlier detection
based algorithm requires large memory to store every collected method. This study adopts an ellipsoidal one-class SVM that
readings from the monitored environment. can be solved using linear optimization instead of the quadratic
3) Detecting selective forwarding attacks using support optimization problem in traditional SVM methods.
vector machine (SVM): In black hole attacks, malicious nodes The main advantages of these SVM-based methods are
send misleading “Routing Reply” (RREP) messages when- their good performance (efficient learning) and ability to learn
ever the malicious nodes receive “Route Request” (RREQ) non-linear and complex problems. However, they still suffer
messages, indicating that routes to the destinations are found. from a scalability issue to large data set due to their high
Accordingly, source nodes will stop the process of route computational and large memory requirements [45].
discovery, and will ignore other RREP messages. Therefore, 5) Analyzing attacks with self-organizing map (SOM):
malicious nodes will drop all network’s messages, while the Avram et al. [123] addressed the issue of detecting network
source nodes assume that their packets were delivered to the attacks in wireless ad hoc networks using self-organizing
destination. Kaplantzis et al. [33] presented packet dropping map unsupervised learning. Learning the weights are obtained
attack prevention technique based on one class support vector through statistical analysis of the input data vectors. The main
machine classifier. The proposed scheme is capable of de- issue of this scheme is the complexity in determining input
tecting black hole attacks and selective forwarding attacks. weights. Moreover, SOM-based algorithms are not suitable for
Basically, routing information, bandwidth and hop count are detecting attacks in very large and complex data sets (i.e., large
used to determine the malicious nodes in the network. scale sensor network).
4) Outlier detection using support vector machine (SVM):
By using a quarter-sphere centered at the origin, the drawback B. Quality of Service, Data Integrity and Fault Detection
of high computational requirements of traditional SVM could Quality of service (QoS) guarantees high-priority delivery
be alleviated. For instance, Rajasegarar et al. [34] introduced a of real-time events and data. In the context of WSNs, there
one-class quarter-sphere SVM anomaly recognition technique. are potential multi-hop transmissions of data to the end user,
The motivation of this distributed scheme is to distinguish in addition to distributing queries from a system controller to
anomalies in data while minimizing communication overhead. the network nodes [125]. WSNs suffer from energy and band-
In [46], Yang et al. tackled the design of an online outlier width constraints that limit the quantity of information to be
detection method using quarter-sphere SVM. The unsuper- transmitted from a source to destination nodes. Furthermore,
vised learning method investigates the local data to reduce data aggregation and dissemination in WSNs can be faulty
the computational complexity of traditional SVM-based outlier and unreliable [4]. These issues coupled with random network
 17

topologies introduce an important challenge for designing presented a real-time algorithm to determine a set of nodes
reliable algorithms for such networks. The state of the art and that are capable of handling information processing tasks such
general QoS requirements in WSNs have been reviewed in as assessing the accuracy of collected sensor readings and
[126]. predicting the missing readings. This algorithm provides a
In the following, we review the latest efforts of using probabilistic Gaussian process based iterative implementation
machine learning techniques to achieve specific QoS and data that is trained to re-use previous experience (i.e., the historical
integrity constraints. In brief, this adoption results in the data) and maintain a reasonable training data size. Yet, the
following advantages: posterior distribution of an observed environmental variable x
• Different machine learning classifiers are used to recog- (e.g., sea surface temperature) is calculated using the general-
nize different types of streams, thus eliminating the need ized multivariate Gaussian distribution given by Eq. (5).
for flow-aware management techniques.
• The requirements for QoS guarantee, data integrity and
 
1 1 T
fault detection depend on the network service and ap- p (x|µ, K, I) , √ exp − (x − µ) K −1 (x − µ)
det 2πK 2
plication. Machine learning methods are able to handle (5)
much of this while ensuring efficient resource utilization, where µ, K are the prior mean and covariance of the vari-
mainly bandwidth and power utilization. able x, respectively. Further, I denotes the historical data (a
Table VII summarizes the methods that are reviewed in this sequence of time-stamped samples) that is updated online to
subsection. The column “Characteristics” indicates features or consider the new sequentially collected observations.
qualities belonging to each study. 4) QoS provisioning using reinforcement learning: Oufer-
1) QoS estimation using neural network: Recently, there is hat and Mellouk [128] introduced a QoS task scheduler for
growing interest in estimating and improving the performance adaptive multimedia sensor networks based on Q-learning
of WSNs. For example, Snow et al. [35] introduced a method technique. This scheduler significantly enhances the network
to estimate a sensor network dependability metric using a neu- throughput by reducing the transmission delay. Comparatively,
ral network method. Dependability is a metric that represents Seah et al. [129] considered coverage as a QoS metric in
availability, reliability, maintainability, and survivability of a WSNs that represents how efficiently the area of interest will
sensor network. Several attributes are used to estimate such be observed. A Q-learning method was used to develop a
a metric including mean time between failure (MTBF) and distributed learner that is able to find weakly monitored sites.
mean time to repair (MTTR). These sites can be resolved in future re-deployment stages.
Moustapha and Selmic [36] introduced a dynamic fault It is important to note that energy harvesting has not been
detection model for WSNs. This model captures the nodes’ considered in the above QoS mechanisms. Conversely, Hsu et
dynamic behavior and their effects on other nodes. In addi- al. [130] introduced a QoS-aware power management scheme
tion, neural network learning, which is trained using back- for WSNs with energy harvesting capabilities, namely “Re-
propagation method, was used for node identification and fault inforcement Learning based QoS-aware Power Management”
detection (a similar idea as in [35]). This study results in an (RLPM). This scheme is able to adapt to the dynamic levels of
effective nonlinear sensor model that suits applications with nodes’ energy in systems with energy harvesting capabilities.
fault detection requirements. QoS-aware RLPM employs reinforcement learning to attain
2) MetricMap (link quality estimation framework): Link QoS awareness and to manage nodes’ duty cycle under the
quality measurement tools may provide inaccurate and un- energy restriction. Furthermore, Liang et al. [131] designed
stable readings across different environments due to different “Multi-agent Reinforcement Learning based multi-hop mesh
conditions such as signal variations and interference [132]. Cooperative Communication” (MRL-CC) to be a structure
As a result, Wang et al. [37] presented MetricMap, a link modeling tool for QoS provisioning in WSNs. Basically,
quality estimation framework using supervised learning meth- MRL-CC is adopted to reliably assess the data in a cooperative
ods. MetricMap enhances the MintRoute [133] protocol by manner. Moreover, MRL-CC might be used to examine the
adopting online and offline learning methods, such as decision impact of traffic load and node mobility on the whole network
tree learners, to derive link quality indicators. This framework performance.
uses several local features to build the classification tree such
as the received signal strength indicator (RSSI), transmission
buffer size, channel load, and the forward and backward prob- C. Miscellaneous Applications
abilities. The forward probability pf (l) is defined as the ratio This subsection presents miscellaneous and unique research
of the received to the total transmitted packets over the link l, efforts that are not discussed previously.
whereas the backward probability pb (l) is calculated over the 1) Resource management through reinforcement learning:
reverse path. The local features are preferred over the global Shah and Kumar [134] presented the “Distributed Independent
one as they can be found without costly communications Reinforcement Learning” (DIRL) algorithm that utilizes local
with far away nodes. Experiments reveal that up to three information and application constraints to optimize various
times improvement in data delivery rate over basic MintRoute tasks over time while minimizing energy consumption of
method can be achieved. the network. Each sensor node learns the minimum required
3) Assessing accuracy and reliability of sensor nodes us- resources to perform its scheduled tasks, and maximizes its
ing multi-output Gaussian processes: Osborne et al. [127] future rewards by finding optimal parameters of the intended
 18

TABLE VII: Summary of quality of service, data integrity and fault detection solutions.
M ACHINE
A PPROACHES LEARNING C OMPLEXITY C HARACTERISTICS
ALGORITHM ( S )
System’s dependability [35] High Estimate the dependability metric
NNs
Fault detection [36] Moderate Dynamic fault detection model
MetricMap [37] DT Low Link quality estimation
Assessing accuracy and reliability metrics
GP Moderate Information processing tasks
[127]
A QoS scheduler [128] Low QoS task scheduler for adaptive multimedia sensor networks
Uncertainty and coverage factors [129] Moderate Investigate converge problems
RL
QoS-aware power management in energy harvesting sensor
QoS-aware power management [130] Low
nodes
QoS provisioning [131] Low A structure modeling tool for QoS provisioning

has to consider the limited resource constraints. For example,

Paladina et al. [137] proposed to use SOM to ensure reliable
Task priorities
clock synchronization for large scale networks. Nodes predict
and Q-value table near-optimal estimation of the current time without having a
central timing device and with limited storage and computing
ve

Targeted user resources. However, this method assumes a uniform deploy-

Mo

ment of the nodes over the monitored area, as well as the same
Node 1
transmission powers for all nodes.
4) Air quality monitoring using neural networks: Posto-
time=0 Node 2 Node 3 time=8 lache et al. [138] proposed a neural networks-based method
for measuring air pollution levels using inexpensive gas sensor
Base station 1- Aggregate 2- Sample
3- Transmit 4- Receive
nodes, while eliminating the effects of temperature and humid-
5- Sleep ity on sensor readings. This solution detects the air quality and
gas concentration using neural networks implemented using
Fig. 15: An example of task management using the DIRL
JavaScript (JS). As a result, the solution is able to distribute
middleware algorithm: Object tracking application [134].
processing between web server and end user computers (i.e.,
a combination of client and server side scripts).
5) Intelligent lighting control using neural networks: Gao
application. As a typical case, consider an object recognition
et al. [139] introduced a new standard for lighting control in
application, as shown in Fig. 15, which consists of five
smart building using the neural network algorithm. A radial
fundamental tasks: (a) aggregate two or more readings into
basis function (RBF) neural network is used to extract a
a single reading, (b) transmit a message to the next hop, (c)
new mathematical expression, called “Illuminance Matrix” (I-
receive incoming messages, (d) sample and take readings, and
matrix), to measure the degree of illuminance in the lighted
(e) put the radio into sleep mode. These tasks must be executed
area. Fundamentally, in the field of lighting control, converting
in some priority to maximize the network lifetime, where the
the collected data from the photosensors to a form that is
network do not have such knowledge of priority as there is
suitable for digital signal processing is a crucial issue and
no static schedule for the events. For example, a node does
can highly affect the performance of the developed system.
not have the knowledge of when the object is going to move
The article shows that using the I-matrix scheme can achieve
near to it to start taking samples. Here, the DIRL algorithm
about 60% more accuracy compared to the standard methods.
can be used to generate the required knowledge of priority by
using the Q-learning algorithm and after specifying the set of
reward and price value for each task. V. FUTURE APPLICATIONS OF MACHINE
2) Decision tree-based animals behavior classification: LEARNING IN WIRELESS SENSOR NETWORKS
WSNs were applied in many applications such as environ- Although machine learning techniques have been applied to
mental and habitat monitoring [135]. As an example, Nadimi many applications in WSNs, many issues are still open and
et al. [136] employed decision tree to accurately classify need further research efforts.
the behavior of a herd of animals (active or inactive) using
parameters such as the pitch angle of the neck and movement
velocity. The advantages of the proposed solution for animals A. Compressive Sensing and Sparse Coding
behavior classification are the simple implementation and low In practice, a large number of sensor measurements are
complexity due to the use of a few critical features. usually required to maintain a desired detection accuracy. This
3) Clock synchronization using self-organizing map: Clock introduces several challenges to network designers such as
synchronization between sensor nodes is an important process, network management and communication issues. Given that
since most operations of the nodes must be consistent with 80 percent of the nodes’ energy is consumed while sending
each other. Moreover, the design of such methods for WSNs and receiving data [140], data compression and dimensionality
 19

Gateway
reduction techniques can be used to reduce transmission and Similarity
F
hence prolong the network lifetime. G C decrease
E D
Traditional data compression techniques may result in extra
energy consumption due to their high computational and B A
memory requirements. In [141], Barr and Asanović studied the Similarity
tradeoff between energy consumption in data transmission and increase
compression. This study approximates the efficiency threshold E
of data compression in WSNs to be 1 bit data reduction using
F B
485-1267 ADD instructions. D
Even though compressive sensing can be recast as a linear
program, it still not applicable for on-node compression. As A
a result, it is important to apply and extend the basic concept
of compressive sensing to meet the resource constraint of G
WSNs. For more on the theoretical performance of decentral- C
ized compressive sensing, please refer to [142], [143], [144].
Examples of similar emerging techniques include independent Fig. 16: Hierarchical clustering of network’s nodes based
component analysis, dictionary learning, non-negative matrix on data spatial and temporal correlations in a temperature
factorization and singular value decomposition. monitoring system.

B. Distributed and Adaptive Machine Learning Techniques for

WSNs D. Detecting Data Spatial and Temporal Correlations Using
Hierarchical Clustering
Distributed machine learning techniques suit limited re-
source devices such as WSNs. Compared to centralized Hierarchical clustering is an unsupervised learning algo-
learning algorithms, distributed learning methods require less rithm that aims to build a hierarchy of clusters. Basically,
computational power and smaller memory footprint (i.e., they hierarchical clustering algorithms generate decomposition of
do not need to consider the whole network information). the set of objects, which could be a set of sensor nodes in
The decentralized learning techniques enable the nodes to WSNs. Broadly speaking, hierarchical clustering can provide
rapidly adapt their future behavior and predictions in tune an emerging clustering technique in WSNs using some clus-
with the current environment conditions. For such reasons, tering criteria such as spatial and temporal correlations of
distributed and adaptive learning algorithms are adequate for readings. Figure 16 illustrates such hierarchically clustered
in-network processing of data while avoiding exhausting the network based on spatial and temporal correlations of readings
nodes with high computational tasks [145]. Examples of recent in a temperature monitoring system. In this example, Cluster
online learning algorithms include “Adaptive Regularization C is formed by combining Clusters A and B, and so on for
of Weights” (AROW) [146], “Improved Ellipsoid Method the rest of the clusters in the network.
for Online Learning” (IELLIP) [147] and “Soft Confidence- The study of data correlation based on hierarchical cluster-
Weighted” (SCW) [148]. Kotecha et al. [149] studied some ing method will provide simple methods for energy saving. In
distributed classification algorithms for WSNs. such formations, only one node from each cluster is activated
at a time to cover and monitor the whole cluster area. Typical
C. Resource Management Using Machine Learning methods of hierarchical clustering include “Balanced Iterative
Reducing and Clustering using Hierarchies” (BIRCH) [151]
Energy saving is a crucial issue in developing efficient and “Clustering Using Representatives” (CURE) [152].
WSNs algorithms and techniques. This design goal can be
achieved using two main techniques, namely, by enhancing
communication related protocols (e.g., routing and MAC pro- VI. CONCLUSIONS
tocols design) and by detecting nonfunctional and energy Wireless sensor networks are different from traditional
wasteful activities. The first technique includes physical, MAC network in various aspects, thereby necessitating protocols
and networking layer protocols. As it is discussed in this and tools that address unique challenges and limitations. As
survey, this technique has been widely studied and enhanced a consequence, wireless sensor networks require innovative
using machine learning algorithms. The second technique solutions for energy aware and real-time routing, security,
focuses on decreasing the consumed energy in minor and scheduling, localization, node clustering, data aggregation,
nonfunctional requirements. For example, sensor nodes will fault detection and data integrity. Machine learning provides
consume their energy when over-listening to other nodes’ a collection of techniques to enhance the ability of wire-
transmissions [150]. Accordingly, such operations unneces- less sensor network to adapt to the dynamic behavior of
sarily increase the active time of the nodes (i.e., increase its surrounding environment. Table VIII summarizes studies
nodes’ duty cycle). The nodes that are equipped with machine that have adopted machine learning methods to address these
learning techniques will be able to optimize their resource challenges from distinct research areas.
management and power allocation operations under those From the discussion so far, it became clear that many design
circumstances. challenges in wireless sensor networks have been resolved
 20

TABLE VIII: Summary of publications resolving various WSN challenges by the adoption of machine learning techniques.
F UNCTIONAL N ON - FUNCTIONAL
E VENT SECURITY Q O S, DATA
C LUSTERING L OCALIZATION
DETECTION AND INTEGRITY
ROUTING AND DATA AND OBJECTS MAC
AND QUERY INTRUSION AND FAULT
AGGREGATION TARGETING
PROCESSING DETECTION DETECTION

[91], [94],
BAYESIAN STATISTICS [21], [22] [28] [31]
[92]
K - NEAREST NEIGHBORS [24], [25] [32]
[23], [99],
N EURAL NETWORK [79] [26] [29], [30] [35], [36]
[100]
[33], [34],
[49], [50],
S UPPORT VECTOR MACHINES [46], [47],
[51]
[48]
D ECISION TREE [80] [101], [102] [27] [115] [37]
G AUSSIAN P ROCESSES [81], [82] [103], [104] [127]
[105], [107],
S ELF - ORGANIZING MAP [70] [54] [123]
[106]
V ECTOR QUANTIZATION [83]
[55], [56],
P RINCIPAL COMPONENT [57], [58], [93]
[84]
K - MEANS ALGORITHM [59], [60]
[65], [66], [114], [128], [129],
R EINFORCEMENT LEARNING [85], [86] [108] [134]
[67], [68] [112] [130], [131]

using several machine learning methods. In this paper, an [11] M. Di and E. M. Joo, “A survey of machine learning in wireless
extensive literature review over the period 2002-2013 on such sensor netoworks from networking and application perspectives,” in
6th International Conference on Information, Communications Signal
studies was presented. In summary, adopting machine learning Processing, 2007, pp. 1–5.
algorithms in wireless sensor networks has to consider the [12] A. Forster, “Machine learning techniques applied to wireless ad-hoc
limited resources of the network, as well as the diversity of networks: Guide and survey,” in 3rd International Conference on
Intelligent Sensors, Sensor Networks and Information. IEEE, 2007,
learning themes and patterns that will suit the problem at hand. pp. 365–370.
Moreover, numerous issues are still open and need further [13] A. Förster and M. Amy L, Machine learning across the WSN layers.
research efforts such as developing lightweight and distributed InTech, 2011.
[14] Y. Zhang, N. Meratnia, and P. Havinga, “Outlier detection techniques
message passing techniques, online learning algorithms, hier- for wireless sensor networks: A survey,” IEEE Communications Sur-
archical clustering patterns and adopting machine learning in veys & Tutorials, vol. 12, no. 2, pp. 159–170, 2010.
resource management problem of wireless sensor networks. [15] V. J. Hodge and J. Austin, “A survey of outlier detection methodolo-
gies,” Artificial Intelligence Review, vol. 22, no. 2, pp. 85–126, 2004.
[16] R. Kulkarni, A. Förster, and G. Venayagamoorthy, “Computational
intelligence in wireless sensor networks: A survey,” IEEE Communi-
R EFERENCES cations Surveys & Tutorials, vol. 13, no. 1, pp. 68–96, 2011.
[17] S. Das, A. Abraham, and B. K. Panigrahi, Computational intelligence:
Foundations, perspectives, and recent trends. John Wiley & Sons,
[1] T. O. Ayodele, “Introduction to machine learning,” in New Advances
Inc., 2010, pp. 1–37.
in Machine Learning. InTech, 2010.
[18] Y. S. Abu-Mostafa, M. Magdon-Ismail, and H.-T. Lin, Learning from
[2] A. H. Duffy, “The “what” and “how” of learning in design,” IEEE data. AMLBook, 2012.
Expert, vol. 12, no. 3, pp. 71–76, 1997. [19] O. Chapelle, B. Schlkopf, and A. Zien, Semi-supervised learning. MIT
[3] P. Langley and H. A. Simon, “Applications of machine learning and press Cambridge, 2006, vol. 2.
rule induction,” Communications of the ACM, vol. 38, no. 11, pp. 54– [20] S. Kulkarni, G. Lugosi, and S. Venkatesh, “Learning pattern
64, 1995. classification-a survey,” IEEE Transactions on Information Theory,
[4] L. Paradis and Q. Han, “A survey of fault management in wireless sen- vol. 44, no. 6, pp. 2178–2206, 1998.
sor networks,” Journal of Network and Systems Management, vol. 15, [21] M. Morelande, B. Moran, and M. Brazil, “Bayesian node localisation
no. 2, pp. 171–190, 2007. in wireless sensor networks,” in IEEE International Conference on
[5] B. Krishnamachari, D. Estrin, and S. Wicker, “The impact of data Acoustics, Speech and Signal Processing, 2008, pp. 2545–2548.
aggregation in wireless sensor networks,” in 22nd International Con- [22] C.-H. Lu and L.-C. Fu, “Robust location-aware activity recognition
ference on Distributed Computing Systems Workshops, 2002, pp. 575– using wireless sensor network in an attentive home,” IEEE Transactions
578. on Automation Science and Engineering, vol. 6, no. 4, pp. 598–609,
[6] J. Al-Karaki and A. Kamal, “Routing techniques in wireless sensor 2009.
networks: A survey,” IEEE Wireless Communications, vol. 11, no. 6, [23] A. Shareef, Y. Zhu, and M. Musavi, “Localization using neural
pp. 6–28, 2004. networks in wireless sensor networks,” in Proceedings of the 1st
[7] K. Romer and F. Mattern, “The design space of wireless sensor International Conference on Mobile Wireless Middleware, Operating
networks,” IEEE Wireless Communications, vol. 11, no. 6, pp. 54–61, Systems, and Applications, 2008, pp. 1–7.
2004. [24] J. Winter, Y. Xu, and W.-C. Lee, “Energy efficient processing of
[8] J. Wan, M. Chen, F. Xia, L. Di, and K. Zhou, “From machine-to- k nearest neighbor queries in location-aware sensor networks,” in
machine communications towards cyber-physical systems,” Computer 2nd International Conference on Mobile and Ubiquitous Systems:
Science and Information Systems, vol. 10, pp. 1105–1128, 2013. Networking and Services. IEEE, 2005, pp. 281–292.
[9] Y. Bengio, “Learning deep architectures for AI,” Foundations and [25] P. P. Jayaraman, A. Zaslavsky, and J. Delsing, “Intelligent processing of
Trends in Machine Learning, vol. 2, no. 1, pp. 1–127, 2009. k-nearest neighbors queries using mobile data collectors in a location
[10] A. G. Hoffmann, “General limitations on machine learning,” pp. 345– aware 3D wireless sensor network,” in Trends in Applied Intelligent
347, 1990. Systems. Springer, 2010, pp. 260–270.
 21

[26] L. Yu, N. Wang, and X. Meng, “Real-time forest fire detection with [48] Y. Zhang, N. Meratnia, and P. J. Havinga, “Distributed online outlier
wireless sensor networks,” in International Conference on Wireless detection in wireless sensor networks using ellipsoidal support vector
Communications, Networking and Mobile Computing, vol. 2, 2005, machine,” Ad Hoc Networks, vol. 11, no. 3, pp. 1062–1074, 2013.
pp. 1214–1217. [49] W. Kim, J. Park, and H. Kim, “Target localization using ensemble
[27] M. Bahrepour, N. Meratnia, M. Poel, Z. Taghikhaki, and P. J. Havinga, support vector regression in wireless sensor networks,” in Wireless
“Distributed event detection in wireless sensor networks for disaster Communications and Networking Conference, 2010, pp. 1–5.
management,” in 2nd International Conference on Intelligent Network- [50] D. Tran and T. Nguyen, “Localization in wireless sensor networks
ing and Collaborative Systems. IEEE, 2010, pp. 507–512. based on support vector machines,” IEEE Transactions on Parallel and
[28] M. Kim and M.-G. Park, “Bayesian statistical modeling of system Distributed Systems, vol. 19, no. 7, pp. 981–994, 2008.
energy saving effectiveness for MAC protocols of wireless sensor [51] B. Yang, J. Yang, J. Xu, and D. Yang, “Area localization algorithm
networks,” in Software Engineering, Artificial Intelligence, Networking for mobile nodes in wireless sensor networks based on support vector
and Parallel/Distributed Computing, ser. Studies in Computational machines,” in Mobile Ad-Hoc and Sensor Networks. Springer, 2007,
Intelligence. Springer Berlin Heidelberg, 2009, vol. 209, pp. 233– pp. 561–571.
245. [52] G. E. Box and G. C. Tiao, Bayesian inference in statistical analysis.
[29] Y.-J. Shen and M.-S. Wang, “Broadcast scheduling in wireless sensor John Wiley & Sons, 2011, vol. 40.
networks using fuzzy hopfield neural network,” Expert Systems with [53] C. E. Rasmussen, “Gaussian processes for machine learning,” in in:
Applications, vol. 34, no. 2, pp. 900 – 907, 2008. Adaptive Computation and Machine Learning. Citeseer, 2006.
[30] R. V. Kulkarni and G. K. Venayagamoorthy, “Neural network based [54] S. Lee and T. Chung, “Data aggregation for wireless sensor networks
secure media access control protocol for wireless sensor networks,” using self-organizing map,” in Artificial Intelligence and Simulation,
in Proceedings of the 2009 International Joint Conference on Neural ser. Lecture Notes in Computer Science. Springer Berlin Heidelberg,
Networks, ser. IJCNN’09. Piscataway, NJ, USA: IEEE Press, 2009, 2005, vol. 3397, pp. 508–517.
pp. 3437–3444. [55] R. Masiero, G. Quer, D. Munaretto, M. Rossi, J. Widmer, and M. Zorzi,
[31] D. Janakiram, V. Adi Mallikarjuna Reddy, and A. Phani Kumar, “Data acquisition through joint compressive sensing and principal com-
“Outlier detection in wireless sensor networks using Bayesian belief ponent analysis,” in Global Telecommunications Conference. IEEE,
networks,” in 1st International Conference on Communication System 2009, pp. 1–6.
Software and Middleware. IEEE, 2006, pp. 1–6. [56] R. Masiero, G. Quer, M. Rossi, and M. Zorzi, “A Bayesian analysis
[32] J. W. Branch, C. Giannella, B. Szymanski, R. Wolff, and H. Kargupta, of compressive sensing data recovery in wireless sensor networks,”
“In-network outlier detection in wireless sensor networks,” Knowledge in International Conference on Ultra Modern Telecommunications
and information systems, vol. 34, no. 1, pp. 23–54, 2013. Workshops, 2009, pp. 1–6.
[33] S. Kaplantzis, A. Shilton, N. Mani, and Y. Sekercioglu, “Detecting [57] A. Rooshenas, H. Rabiee, A. Movaghar, and M. Naderi, “Reducing
selective forwarding attacks in wireless sensor networks using support the data transmission in wireless sensor networks using the principal
vector machines,” in 3rd International Conference on Intelligent Sen- component analysis,” in 6th International Conference on Intelligent
sors, Sensor Networks and Information. IEEE, 2007, pp. 335–340. Sensors, Sensor Networks and Information Processing. IEEE, 2010,
[34] S. Rajasegarar, C. Leckie, M. Palaniswami, and J. Bezdek, “Quarter pp. 133–138.
sphere based distributed anomaly detection in wireless sensor net- [58] S. Macua, P. Belanovic, and S. Zazo, “Consensus-based distributed
works,” in International Conference on Communications, 2007, pp. principal component analysis in wireless sensor networks,” in 11th
3864–3869. International Workshop on Signal Processing Advances in Wireless
[35] A. Snow, P. Rastogi, and G. Weckman, “Assessing dependability of Communications, 2010, pp. 1–5.
wireless networks using neural networks,” in Military Communications [59] Y.-C. Tseng, Y.-C. Wang, K.-Y. Cheng, and Y.-Y. Hsieh, “iMouse: An
Conference. IEEE, 2005, pp. 2809–2815 Vol. 5. integrated mobile surveillance and wireless sensor system,” Computer,
[36] A. Moustapha and R. Selmic, “Wireless sensor network modeling using vol. 40, no. 6, pp. 60–66, 2007.
modified recurrent neural networks: Application to fault detection,” [60] D. Li, K. Wong, Y. H. Hu, and A. Sayeed, “Detection, classification,
IEEE Transactions on Instrumentation and Measurement, vol. 57, no. 5, and tracking of targets,” IEEE Signal Processing Magazine, vol. 19,
pp. 981–988, 2008. no. 2, pp. 17–29, 2002.
[37] Y. Wang, M. Martonosi, and L.-S. Peh, “Predicting link quality using [61] T. Kanungo, D. M. Mount, N. S. Netanyahu, C. D. Piatko, R. Sil-
supervised learning in wireless sensor networks,” ACM SIGMOBILE verman, and A. Y. Wu, “An efficient k-means clustering algorithm:
Mobile Computing and Communications Review, vol. 11, no. 3, pp. Analysis and implementation,” IEEE Transactions on Pattern Analysis
71–83, 2007. and Machine Intelligence, vol. 24, no. 7, pp. 881–892, 2002.
[38] K. Beyer, J. Goldstein, R. Ramakrishnan, and U. Shaft, “When is [62] I. T. Jolliffe, Principal component analysis. Springer verlag, 2002.
“nearest neighbor” meaningful?” in Database Theory. Springer, 1999, [63] D. Feldman, M. Schmidt, C. Sohler, D. Feldman, M. Schmidt, and
pp. 217–235. C. Sohler, “Turning big data into tiny data: Constant-size coresets for
[39] T. O. Ayodele, “Types of machine learning algorithms,” in New k-means, PCA and projective clustering,” in SODA, 2013, pp. 1434–
Advances in Machine Learning. InTech, 2010. 1453.
[40] S. R. Safavian and D. Landgrebe, “A survey of decision tree classifier [64] C. Watkins and P. Dayan, “Q-learning,” Machine Learning, vol. 8, no.
methodology,” IEEE Transactions on Systems, Man and Cybernetics, 3-4, pp. 279–292, 1992.
vol. 21, no. 3, pp. 660–674, 1991. [65] R. Sun, S. Tatsumi, and G. Zhao, “Q-MAP: A novel multicast routing
[41] R. Lippmann, “An introduction to computing with neural nets,” ASSP method in wireless ad hoc networks with multiagent reinforcement
Magazine, IEEE, vol. 4, no. 2, pp. 4–22, 1987. learning,” in Region 10 Conference on Computers, Communications,
[42] W. Dargie and C. Poellabauer, Localization. John Wiley & Sons, Ltd, Control and Power Engineering, vol. 1, 2002, pp. 667–670 vol.1.
2010, pp. 249–266. [66] S. Dong, P. Agrawal, and K. Sivalingam, “Reinforcement learning
[43] T. Kohonen, Self-organizing maps, ser. Springer Series in Information based geographic routing protocol for UWB wireless sensor network,”
Sciences. Springer Berlin Heidelberg, 2001, vol. 30. in Global Telecommunications Conference. IEEE, 2007, pp. 652–656.
[44] G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of [67] A. Förster and A. Murphy, “FROMS: Feedback routing for optimizing
data with neural networks,” Science, vol. 313, no. 5786, pp. 504–507, multiple sinks in wsn with reinforcement learning,” in 3rd International
2006. Conference on Intelligent Sensors, Sensor Networks and Information.
[45] I. Steinwart and A. Christmann, Support vector machines. Springer, IEEE, 2007, pp. 371–376.
2008. [68] R. Arroyo-Valles, R. Alaiz-Rodriguez, A. Guerrero-Curieses, and
[46] Z. Yang, N. Meratnia, and P. Havinga, “An online outlier detection tech- J. Cid-Sueiro, “Q-probabilistic routing in wireless sensor networks,” in
nique for wireless sensor networks using unsupervised quarter-sphere 3rd International Conference on Intelligent Sensors, Sensor Networks
support vector machine,” in International Conference on Intelligent and Information. IEEE, 2007, pp. 1–6.
Sensors, Sensor Networks and Information Processing. IEEE, 2008, [69] C. Guestrin, P. Bodik, R. Thibaux, M. Paskin, and S. Madden,
pp. 151–156. “Distributed regression: An efficient framework for modeling sensor
[47] Y. Chen, Y. Qin, Y. Xiang, J. Zhong, and X. Jiao, “Intrusion detection network data,” in 3rd International Symposium on Information Pro-
system based on immune algorithm and support vector machine in cessing in Sensor Networks, 2004, pp. 1–10.
wireless sensor network,” in Information and Automation, ser. Com- [70] J. Barbancho, C. León, F. Molina, and A. Barbancho, “A new QoS
munications in Computer and Information Science. Springer Berlin routing algorithm based on self-organizing maps for wireless sensor
Heidelberg, 2011, vol. 86, pp. 372–376. networks,” Telecommunication Systems, vol. 36, pp. 73–83, 2007.
 22

[71] B. Scholkopf and A. J. Smola, Learning with kernels: Support vector networks",” IEEE Transactions on Computers, vol. 54, no. 9, pp. 1182–
machines, regularization, optimization, and beyond. Cambridge, MA, 1183, 2005.
USA: MIT Press, 2001. [95] K. Sha, W. Shi, and O. Watkins, “Using wireless sensor networks
[72] J. Kivinen, A. Smola, and R. Williamson, “Online learning with for fire rescue applications: Requirements and challenges,” in IEEE
kernels,” IEEE Transactions on Signal Processing, vol. 52, no. 8, pp. International Conference on Electro/information Technology, 2006, pp.
2165–2176, 2004. 239–244.
[73] G. Aiello and G. Rogerson, “Ultra-wideband wireless systems,” IEEE [96] H. Liu, H. Darabi, P. Banerjee, and J. Liu, “Survey of wireless indoor
Microwave Magazine, vol. 4, no. 2, pp. 36–47, 2003. positioning techniques and systems,” IEEE Transactions on Systems,
[74] R. Rajagopalan and P. Varshney, “Data-aggregation techniques in sen- Man, and Cybernetics, Part C: Applications and Reviews, vol. 37, no. 6,
sor networks: A survey,” IEEE Communications Surveys & Tutorials, pp. 1067–1080, 2007.
vol. 8, no. 4, pp. 48–63, 2006. [97] J. Wang, R. Ghosh, and S. Das, “A survey on sensor localization,”
[75] G. Crosby, N. Pissinou, and J. Gadze, “A framework for trust-based Journal of Control Theory and Applications, vol. 8, no. 1, pp. 2–11,
cluster head election in wireless sensor networks,” in 2nd IEEE 2010.
Workshop on Dependability and Security in Sensor Networks and [98] A. Nasipuri and K. Li, “A directionality based location discovery
Systems, 2006, pp. 10–22. scheme for wireless sensor networks,” in Proceedings of the 1st ACM
[76] J.-M. Kim, S.-H. Park, Y.-J. Han, and T.-M. Chung, “CHEF: Cluster International Workshop on Wireless Sensor Networks and Applications.
head election mechanism using fuzzy logic in wireless sensor net- ACM, 2002, pp. 105–111.
works,” in 10th International Conference on Advanced Communication [99] S. Yun, J. Lee, W. Chung, E. Kim, and S. Kim, “A soft computing
Technology, vol. 1. IEEE, 2008, pp. 654–659. approach to localization in wireless sensor networks,” Expert Systems
[77] S. Soro and W. Heinzelman, “Prolonging the lifetime of wireless sensor with Applications, vol. 36, no. 4, pp. 7552–7561, 2009.
networks via unequal clustering,” in 19th IEEE International Parallel [100] S. Chagas, J. Martins, and L. de Oliveira, “An approach to localization
and Distributed Processing Symposium, 2005, pp. 4–8. scheme of wireless sensor networks based on artificial neural networks
[78] A. A. Abbasi and M. Younis, “A survey on clustering algorithms for and genetic algorithms,” in 10th International Conference on New
wireless sensor networks,” Computer communications, vol. 30, no. 14, Circuits and Systems. IEEE, 2012, pp. 137–140.
pp. 2826–2841, 2007. [101] Z. Merhi, M. Elgamel, and M. Bayoumi, “A lightweight collaborative
[79] H. He, Z. Zhu, and E. Makinen, “A neural network model to minimize fault tolerant target localization system for wireless sensor networks,”
the connected dominating set for self-configuration of wireless sensor IEEE Transactions on Mobile Computing, vol. 8, no. 12, pp. 1690–
networks,” IEEE Transactions on Neural Networks, vol. 20, no. 6, pp. 1704, 2009.
973–982, 2009. [102] E. Cayirci, H. Tezcan, Y. Dogan, and V. Coskun, “Wireless sensor
[80] G. Ahmed, N. M. Khan, Z. Khalid, and R. Ramer, “Cluster head selec- networks for underwater survelliance systems,” Ad Hoc Networks,
tion using decision trees for wireless sensor networks,” in International vol. 4, no. 4, pp. 431–446, 2006.
Conference on Intelligent Sensors, Sensor Networks and Information [103] A. Krause, A. Singh, and C. Guestrin, “Near-optimal sensor placements
Processing. IEEE, 2008, pp. 173–178. in gaussian processes: Theory, efficient algorithms and empirical stud-
[81] E. Ertin, “Gaussian process models for censored sensor readings,” in ies,” The Journal of Machine Learning Research, vol. 9, pp. 235–284,
14th Workshop on Statistical Signal Processing. IEEE, 2007, pp. 2008.
665–669. [104] D. Gu and H. Hu, “Spatial Gaussian process regression with mobile
[82] J. Kho, A. Rogers, and N. R. Jennings, “Decentralized control of sensor networks,” IEEE Transactions on Neural Networks and Learning
adaptive sampling in wireless sensor networks,” ACM Transactions on Systems, vol. 23, no. 8, pp. 1279–1290, 2012.
Sensor Networks (TOSN), vol. 5, no. 3, pp. 19:1–19:35, 2009. [105] L. Paladina, M. Paone, G. Iellamo, and A. Puliafito, “Self organizing
[83] S. Lin, V. Kalogeraki, D. Gunopulos, and S. Lonardi, “Online in- maps for distributed localization in wireless sensor networks,” in 12th
formation compression in sensor networks,” in IEEE International IEEE Symposium on Computers and Communications, 2007, pp. 1113–
Conference on Communications, vol. 7. IEEE, 2006, pp. 3371–3376. 1118.
[84] C. Fenxiong, L. Mingming, W. Dianhong, and T. Bo, “Data com- [106] G. Giorgetti, S. K. S. Gupta, and G. Manes, “Wireless localization
pression through principal component analysis over wireless sensor using self-organizing maps,” in Proceedings of the 6th International
networks,” Journal of Computational Information Systems, vol. 9, no. 5, Conference on Information Processing in Sensor Networks, ser. IPSN
pp. 1809–1816, 2013. ’07. New York, NY, USA: ACM, 2007, pp. 293–302.
[85] A. Förster and A. Murphy, “CLIQUE: Role-free clustering with q- [107] J. Hu and G. Lee, “Distributed localization of wireless sensor networks
learning for wireless sensor networks,” in 29th IEEE International using self-organizing maps,” in IEEE International Conference on
Conference on Distributed Computing Systems, 2009, pp. 441–449. Multisensor Fusion and Integration for Intelligent Systems, 2008, pp.
[86] M. Mihaylov, K. Tuyls, and A. Nowe, “Decentralized learning in 284–289.
wireless sensor networks,” in Adaptive and Learning Agents, ser. [108] S. Li, X. Kong, and D. Lowe, “Dynamic path determination of mobile
Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2010, beacons employing reinforcement learning for wireless sensor local-
vol. 5924, pp. 60–73. ization,” in 26th International Conference on Advanced Information
[87] W. B. Heinzelman, “Application-specific protocol architectures for Networking and Applications Workshops, 2012, pp. 760–765.
wireless networks,” Ph.D. dissertation, Massachusetts Institute of Tech- [109] C. Musso, N. Oudjane, and F. Le Gland, “Improving regularised parti-
nology, 2000. cle filters,” in Sequential Monte Carlo methods in practice. Springer,
[88] M. Duarte and Y. Eldar, “Structured compressed sensing: From theory 2001, pp. 247–271.
to applications,” IEEE Transactions on Signal Processing, vol. 59, [110] Y.-X. Wang and Y.-J. Zhang, “Non-negative matrix factorization: A
no. 9, pp. 4053–4085, 2011. comprehensive review,” IEEE Transactions on Knowledge and Data
[89] A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood Engineering, vol. 25, no. 6, pp. 1336–1353, 2013.
from incomplete data via the EM algorithm,” Journal of the Royal [111] H.-P. Tan, R. Diamant, W. K. Seah, and M. Waldmeyer, “A survey
Statistical Society. Series B (Methodological), pp. 1–38, 1977. of techniques and challenges in underwater localization,” Ocean Engi-
[90] M. H. DeGroot, “Reaching a consensus,” Journal of the American neering, vol. 38, no. 14, pp. 1663–1676, 2011.
Statistical Association, vol. 69, no. 345, pp. 118–121, 1974. [112] Y. Chu, P. Mitchell, and D. Grace, “ALOHA and q-learning based
[91] B. Krishnamachari and S. Iyengar, “Distributed bayesian algorithms medium access control for wireless sensor networks,” in International
for fault-tolerant event region detection in wireless sensor networks,” Symposium on Wireless Communication Systems, 2012, pp. 511–515.
IEEE Transactions on Computers, vol. 53, no. 3, pp. 241–250, 2004. [113] A. Bachir, M. Dohler, T. Watteyne, and K. K. Leung, “MAC essen-
[92] P. Zappi, C. Lombriser, T. Stiefmeier, E. Farella, D. Roggen, L. Benini, tials for wireless sensor networks,” IEEE Communications Surveys &
and G. Tröster, “Activity recognition from on-body sensors: Accuracy- Tutorials, vol. 12, no. 2, pp. 222–248, 2010.
power trade-off by dynamic sensor selection,” in Wireless Sensor [114] Z. Liu and I. Elhanany, “RL-MAC: A reinforcement learning based
Networks. Springer, 2008, pp. 17–33. MAC protocol for wireless sensor networks,” International Journal of
[93] H. Malik, A. Malik, and C. Roy, “A methodology to optimize query Sensor Networks, vol. 1, no. 3, pp. 117–124, 2006.
in wireless sensor networks using historical data,” Journal of Ambient [115] M. Sha, R. Dor, G. Hackmann, C. Lu, T.-S. Kim, and T. Park, “Self-
Intelligence and Humanized Computing, vol. 2, pp. 227–238, 2011. adapting MAC layer for wireless sensor networks,” Technical Report
[94] Q. Chen, K.-Y. Lam, and P. Fan, “Comments on "Distributed Bayesian WUCSE-2013-75, Washington University in St. Louis, Tech. Rep.,
algorithms for fault-tolerant event region detection in wireless sensor 2013.
 23

[116] W. Ye, J. Heidemann, and D. Estrin, “An energy-efficient MAC [137] L. Paladina, A. Biundo, M. Scarpa, and A. Puliafito, “Self organizing
protocol for wireless sensor networks,” in 21st Annual Joint Conference maps for synchronization in wireless sensor networks,” in New Tech-
of the IEEE Computer and Communications Societies, vol. 3, 2002, pp. nologies, Mobility and Security, 2008, pp. 1–6.
1567–1576 vol.3. [138] O. Postolache, J. Pereira, and P. Girao, “Smart sensors network for air
[117] T. van Dam and K. Langendoen, “An adaptive energy-efficient MAC quality monitoring applications,” IEEE Transactions on Instrumenta-
protocol for wireless sensor networks,” in Proceedings of the 1st tion and Measurement, vol. 58, no. 9, pp. 3253–3262, 2009.
International Conference on Embedded Networked Sensor Systems, ser. [139] Y. Gao, Y. Lin, and Y. Sun, “A wireless sensor network based on the
SenSys ’03. New York, NY, USA: ACM, 2003, pp. 171–180. novel concept of an I-matrix to achieve high-precision lighting control,”
[118] K. Klues, G. Hackmann, O. Chipara, and C. Lu, “A component- Building and Environment, vol. 70, pp. 223–231, 2013.
based architecture for power-efficient media access control in wireless [140] N. Kimura and S. Latifi, “A survey on data compression in wireless
sensor networks,” in Proceedings of the 5th International Conference sensor networks,” in International Conference on Information Technol-
on Embedded Networked Sensor Systems. ACM, 2007, pp. 59–72. ogy: Coding and Computing, vol. 2, 2005, pp. 8–13 Vol. 2.
[119] C. Doerr, M. Neufeld, J. Fifield, T. Weingart, D. C. Sicker, and [141] K. C. Barr and K. Asanović, “Energy-aware lossless data compression,”
D. Grunwald, “MultiMAC-an adaptive MAC framework for dynamic ACM Transactions on Computer Systems, vol. 24, no. 3, pp. 250–291,
radio networking,” in International Symposium on New Frontiers in Aug. 2006.
Dynamic Spectrum Access Networks. IEEE, 2005, pp. 548–555. [142] J. Haupt, W. Bajwa, M. Rabbat, and R. Nowak, “Compressed sensing
[120] D. Moss and P. Levis, “BoX-MACs: Exploiting physical and link layer for networked data,” IEEE Signal Processing Magazine, vol. 25, no. 2,
boundaries in low-power networking,” Computer Systems Laboratory pp. 92–101, 2008.
Stanford University, 2008. [143] J. Luo, L. Xiang, and C. Rosenberg, “Does compressed sensing
[121] Y. Sun, O. Gurewitz, and D. B. Johnson, “RI-MAC: A receiver-initiated improve the throughput of wireless sensor networks?” in International
asynchronous duty cycle MAC protocol for dynamic traffic loads in Conference on Communications, 2010, pp. 1–6.
wireless sensor networks,” in Proceedings of the 6th ACM Conference [144] S. Feizi, M. Medard, and M. Effros, “Compressive sensing over
on Embedded Network Sensor Systems. ACM, 2008, pp. 1–14. networks,” in 48th Annual Allerton Conference on Communication,
[122] Z. Alliance, “Zigbee-2007 specification,” Online: Control, and Computing (Allerton), 2010, pp. 1129–1136.
http://www.zigbee.org/Specifications/ZigBee/Overview.aspx, 2007. [145] J. B. Predd, S. Kulkarni, and H. V. Poor, Distributed learning in
[123] T. Avram, S. Oh, and S. Hariri, “Analyzing attacks in wireless ad wireless sensor networks. IEEE, 2006, vol. 23, no. 4, pp. 56–69.
hoc network with self-organizing maps,” in 5th Annual Conference on [146] K. Crammer, A. Kulesza, and M. Dredze, “Adaptive regularization of
Communication Networks and Services Research, 2007, pp. 166–175. weight vectors,” Machine Learning, vol. 91, no. 2, pp. 155–187, 2013.
[124] L. N. De Castro and J. Timmis, Artificial immune systems: A new [147] L. Yang, R. Jin, and J. Ye, “Online learning by ellipsoid method,” in
computational intelligence approach. Springer, 2002. Proceedings of the 26th Annual International Conference on Machine
[125] G. J. Pottie and A. Pandya, Quality of service in wireless sensor Learning, ser. ICML ’09. New York, NY, USA: ACM, 2009, pp.
networks. John Wiley & Sons, Inc., 2008, pp. 401–435. 1153–1160.
[126] D. Chen and P. K. Varshney, “QoS support in wireless sensor networks: [148] J. Wang, P. Zhao, and S. C. Hoi, “Exact soft confidence-weighted
A survey,” in International Conference on Wireless Networks, vol. 233, learning,” in Proceedings of the 29th International Conference on
2004. Machine Learning, 2012, pp. 121–128.
[127] M. A. Osborne, S. J. Roberts, A. Rogers, S. D. Ramchurn, and N. R. [149] J. H. Kotecha, V. Ramachandran, and A. Sayeed, “Distributed mul-
Jennings, “Towards real-time information processing of sensor network titarget classification in wireless sensor networks,” IEEE Journal on
data using computationally efficient multi-output Gaussian processes,” Selected Areas in Communications, vol. 23, no. 4, pp. 703–713, 2005.
in Proceedings of the 7th International Conference on Information [150] X. Jiang, J. Taneja, J. Ortiz, A. Tavakoli, P. Dutta, J. Jeong, D. E. Culler,
Processing in Sensor Networks. IEEE Computer Society, 2008, pp. P. Levis, S. Shenker et al., “An architecture for energy management in
109–120. wireless sensor networks,” SIGBED Review, vol. 4, no. 3, pp. 31–36,
[128] N. Ouferhat and A. Mellouk, “A QoS scheduler packets for wireless 2007.
sensor networks,” in International Conference on Computer Systems [151] T. Zhang, R. Ramakrishnan, and M. Livny, “BIRCH: An efficient data
and Applications, 2007, pp. 211–216. clustering method for very large databases,” ACM SIGMOD Record,
[129] M. Seah, C.-K. Tham, V. Srinivasan, and A. Xin, “Achieving cov- vol. 25, no. 2, pp. 103–114, Jun. 1996.
erage through distributed reinforcement learning in wireless sensor [152] S. Guha, R. Rastogi, and K. Shim, “CURE: An efficient clustering
networks,” in 3rd International Conference on Intelligent Sensors, algorithm for large databases,” in Proceedings of the 1998 ACM SIG-
Sensor Networks and Information. IEEE, 2007, pp. 425–430. MOD International Conference on Management of Data, ser. SIGMOD
[130] R. Hsu, C.-T. Liu, K.-C. Wang, and W.-M. Lee, “QoS-aware power ’98. New York, NY, USA: ACM, 1998, pp. 73–84.
management for energy harvesting wireless sensor network utilizing
reinforcement learning,” in International Conference on Computational
Science and Engineering, vol. 2. IEEE, 2009, pp. 537–542.
[131] X. Liang, M. Chen, Y. Xiao, I. Balasingham, and V. C. M. Leung,
“A novel cooperative communication protocol for QoS provisioning
in wireless sensor networks,” in 5th International Conference on
Testbeds and Research Infrastructures for the Development of Networks
Communities and Workshops, 2009, pp. 1–6.
[132] N. Baccour, A. Koubaa, L. Mottola, M. A. Zuniga, H. Youssef, C. A.
Boano, and M. Alves, “Radio link quality estimation in wireless sensor
networks: A survey,” ACM Transactions on Sensor Networks (TOSN),
vol. 8, no. 4, p. 34, 2012.
[133] A. Woo, T. Tong, and D. Culler, “Taming the underlying challenges
of reliable multihop routing in sensor networks,” in Proceedings of the
1st International Conference on Embedded Networked Sensor Systems,
ser. SenSys ’03. New York, NY, USA: ACM, 2003, pp. 14–27.
[134] K. Shah and M. Kumar, “Distributed independent reinforcement learn-
ing (DIRL) approach to resource management in wireless sensor
networks,” in Internatonal Conference on Mobile Adhoc and Sensor
Systems, 2007, pp. 1–9.
[135] A. Mainwaring, D. Culler, J. Polastre, R. Szewczyk, and J. Anderson,
“Wireless sensor networks for habitat monitoring,” in Proceedings of
the 1st ACM International Workshop on Wireless Sensor Networks and
Applications. ACM, 2002, pp. 88–97.
[136] E. Nadimi, H. T. Søgaard, and T. Bak, “Zigbee-based wireless sensor
networks for classifying the behaviour of a herd of animals using
classification trees,” Biosystems Engineering, vol. 100, no. 2, pp. 167–
176, 2008.