MODULE 4 (9 HOURS)

Data Collection, storage and computing Using a Cloud Platform-Introduction, Cloud

Computing Paradigm for Data Collection, Storage, Computing-Cloud
Computing Paradigm,
Cloud Deployment Models-Everything as a Service and Cloud Service Models-SaaS, PaaS,
IaaS, DaaS. Cloud based platforms-XIVELY, NIMBITS.
IoT Physical Devices & Endpoints-IoT Device-Building blocks –Raspberry-Pi -Board- Linux
on Raspberry-Pi-Raspberry-Pi Interfaces (serial, SPI, I2C). Raspberry Pi interfacing and
programming examples using python (LED, switch, sensor, serial, SPI, I2C devices).
Controlling GPIO outputs and displaying sensor readings using web interface/cloud (Python
programming is required only for assignments and projects and not for examinations. Other
end nodes and platforms can also be used).

DATA COLLECTION, STORAGE AND COMPUTING USING A CLOUD

PLATFORM
A few conventional methods for data collection and storage are as follows:
● Saving devices data at a local server for the device nodes
● Communicating and saving the devices’ data in the files locally on removable media,
such as micro SD cards and computer hard disks
● Communicating and saving the data and results of computations in a dedicated data
store or coordinating node locally
● Communicating and saving data at a local node, which is a part of a distributed DBMS
● Communicating and saving at a remote node in the distributed DBMS
● Communicating on the Internet and saving at a data store in a web or enterprise server
● Communicating on the Internet and saving at data centre for an enterprise. Cloud is a
new generation method for data collection, storage and computing.

Devices or sensors network data collection at a device local-server, local files, dedicated data
store, at a coordinating node, a local node of a distributed DBMS, Internet-connected server of
data centre, server or distributed database nodes or a cloud infrastructure.

CLOUD COMPUTING PARADIGM FOR DATA COLLECTION, STORAGE AND

COMPUTING

Different methods of data collection, storage and computing are shown in Figure.

The figure shows (see next page)

(i) Devices or sensor networks data collection at the device web server,
(ii) Local files,
(iii) Dedicated data store at coordinating node,
(iv) Local node in a distributed DBMS,
(v) Internet-connected data centre,
(vi) Internet-connected server,
(vii) Internet-connected distributed DBMS nodes, and (viii)Cloud infrastructure and
services.

1
Cloud computing paradigm is a great evolution in Information and Communications
Technology (ICT). The new paradigm uses XAAS at the Internet connected clouds for
collection, storage and computing.

CLOUD COMPUTING PARADIGM

• Cloud computing means a collection of services available over the Internet. Cloud
delivers the computational functionality.
• Cloud computing deploys infrastructure of a cloud-service provider.
• The infrastructure deploys on a utility or grid computing or webservices environment
that includes network, system, grid of computers or servers or data centres.
• Just as we—users of electricity—do not need to know about the source and underlying
infrastructure for electricity supply service, similarly, a user of computing service or
application need not know how the infrastructure deploys or the details of the
computing environment.
• Just as the user does not need to know Intel processor inside a computer, similarly, the
user uses the data, computing and intelligence in the cloud, as part of the services.

Cloud Platform Services

Cloud platform offers the following:

● Infrastructure for large data storage of devices, RFIDs, industrial plant machines,
automobiles and device networks
2
 ● Computing capabilities, such as analytics, IDE (Integrated Development Environment)
● Collaborative computing and data store sharing

Cloud Platform Usages

• Cloud platform usages are for connecting devices, data, APIs, applications and services,
persons, enterprises, businesses and XAAS.
• The following Equation describes a simple conceptual framework of the Internet Cloud:
Internet Cloud + Clients = User applications and services with ‘no boundaries and
no walls’ …

• An application or service executes on a platform which includes the operating system (OS),
hardware and network.
• Multiple applications may initially be designed to run on diversified platforms (OSs,
hardware and networks).
• Applications and services need to integrate them on a common platform and running
environment.
• Cloud storage and computing environment offers a virtualised environment, which refers to
a running environment made to appear as one to all applications and services, but in fact
physically two or more running environments and platforms may be present.

Virtualisation

• A characteristic of virtualised environment is that it enables applications and services to

execute in an independent execution environment (heterogeneous computing environment).
• Each one of them stores and executes in isolation on the same platform, though in fact, it may
actually execute or access to a set of data centres or servers or distributed services and
computing systems.
• The applications or services which are hosted remotely and are accessible using the Internet
can easily be deployed at a user application or service in a virtualised environment, provided
the Internet or other communications are present.
• Virtualisation enables provisioning for storage, network functions, server and desktop in
execution environment of multi-platforms and servers.
• Applications need not be aware of the platform, just Internet connectivity to the platform,
called cloud platform, is required. The storage is called cloud storage.
• The computing is called cloud computing. The services are called cloud services in line with
the web services which host on web servers.
• Virtualisation of storage means user application or service accesses physical storage using
abstract database interface or file system or logical drive or disk drive, though in fact storage
may be accessible using multiple interfaces or servers.
• For example, Apple iCloud offers storage to a user or user group that enables the sharing of
albums, music, videos, data store, editing files and collaboration among the user group
members.

3
 • Network Function Virtualisation (NFV) means a user application or service accesses
the resources appearing as just one network, though the network access to the resources
may be through multiple resources and networks.
• Virtualisation of server means user application accesses not only one server but in
fact accesses multiple servers.

• Virtualised desktop means the user application can change and deploy multiple
desktops, though the access by the user is through their own computer platform (OS)
that in fact may be through multiple OSs and platforms or remote computers.

CLOUD COMPUTING FEATURES AND ADVANTAGES

• On demand self-service to users for the provision of storage, computing servers,

software delivery and server time
• Resource pooling in multi-tenant model
• Broad network accessibility in virtualised environment to heterogeneous users, clients,
systems and devices
• Elasticity
• Massive scale availability
• Scalability
• Maintainability
• Homogeneity
• Virtualisation
• Interconnectivity platform with virtualised environment for enterprises and
provisioning of in-between Service Level Agreements (SLAs)
• Resilient computing
• Advanced security
• Low cost

CLOUD COMPUTING CONCERNS

● Requirement of a constant high-speed Internet connection

● Limitations of the services available
● Possible data loss
● Non delivery as per defined SLA specified performance
● Different APIs and protocols used at different clouds
● Security in multi-tenant environment needs high trust and low risks
● Loss of users’ control

CLOUD DEPLOYMENT MODELS

Following are the four cloud deployment models:
1.Public cloud: This model is provisioned by educational institutions, industries, government
institutions or businesses or enterprises and is open for public use.

4
2.Private cloud: This model is exclusive for use by institutions, industries, businesses or
enterprises and is meant for private use in the organisation by the employees and associated
users only.
3.Community cloud: This model is exclusive for use by a community formed by institutions,
industries, businesses or enterprises, and for use within the community organisation, employees
and associated users. The community specifies security and compliance considerations.
4.Hybrid cloud: A set of two or more distinct clouds (public, private or community) with
distinct data stores and applications that bind between them to deploy the proprietary or
standard technology.

Cloud platform architecture is a virtualised network architecture consisting of a cluster of

connected servers over the data centres and Service Level Agreements (SLAs) between them.

A cloud platform controls and manages resources, and dynamically provisions the networks,
servers and storage. Cloud platform applications and network services are utility, grid and
distributed services. Examples of cloud platforms are Amazon EC2, Microsoft Azure, Google
App Engine, Xively, Nimbits, AWS IoT, CISCO IoT, IOx and Fog, IBM IoT Foundation, TCS
Connected Universe Platform.

EVERYTHING AS A SERVICE AND CLOUD SERVICE MODELS

• Cloud connects the devices, data, applications, services, persons and business.
• Cloud services can be considered as distribution service—a service for linking the
resources (computing functions, data store, processing functions, networks, servers and
applications) and for provision of coordinating between the resources.

Cloud computing can be considered by a simple equation:

Cloud Computing = SaaS + Paas + IaaS + DaaS …
5
 SaaS: SaaS means Software as a Service. The software is made available to an application or
service on demand. SaaS is a service model where the applications or services deploy and host
at the cloud,and are made available through the Internet on demand by the service user. The
software control, maintenance, updation to new version and infrastructure, platform and
resource requirements are the responsibilities of the cloud service provider.

PaaS : PaaS means Platform as a Service. The platform is made available to a developer of an
application on demand. PaaS is a service model where the applications and services develop
and execute using the platform (for computing, data store and distribution services) which is
made available through the Internet on demand for the developer of the applications. The
platform, network, resources, maintenance, updation and security as per the developers’
requirements are the responsibilities of the cloud service provider.

IaaS: IaaS means Infrastructure as a Service. The infrastructure (data stores, servers, data
centres and network) is made available to a user or developer of application on demand.
Developer installs the OS image, data store and application and controls them at the
infrastructure.

IaaS is a service model where the applications develop or use the infrastructure which is made
available through the Internet on demand on rent (pay as per use in multi-tenancy model) by a
developer or user. IaaS computing systems, network and security are the responsibilities of the
cloud service provider.

DaaS : DaaS means Data as a Service. Data at a data centre is made available to a user or
developer of application on demand. DaaS is a service model where the data store or data
warehouse is made available through the Internet on demand on rent (pay as per use in multi
tenancy model) to an enterprise. The data centre management, 24×7 power, control, network,
maintenance, scale up, data replicating and mirror nodes and systems as well as physical
security are the responsibilities of the data centre service provider.

IOT CLOUD-BASED DATA COLLECTION, STORAGE, COMPUTING USING

XIVELY

• Xively is an open-source platform for Arduino which is an opensource prototyping platform

that provides connectivity with web deploying Internet.
• Xively is a commercial PaaS for the IoT/M2M.It is used as a data aggregator and data mining
website often integrated into the Web of Things.
• Xively is an IoT PaaS for services and business services. The platform supports the REST,
WebSockets and MQTT protocols and connects the devices to Xively Cloud Services. There
are native SDKs for Android,Arduino, ARM mbed, Java, PHP, Ruby and Python languages.

6
Developers can use the workflow of prototyping, deployment and management through the
tools provided by Xively
Xively PaaS services offers the following features:
● It enables services, business services platform which connects the products, including
collaboration products, Rescue, Boldchat, join.me, and operations to the Internet.
● Data collection in real-time over the Internet.
● Data visualisation for data of connected sensors to IoT devices.
● Graphical plots of collected data.
● It generates alerts.
● Access to historical data.
● It supports Java, Python and Ruby, and Android platform.
● It generates feeds which can be real-world objects of own or others.
● It supports the ARM mBedTM-based, Arduino-based and other hardware-platformbased
IoT devices, and HTTP-based APIs which are easy to implement on device hardware
acting as clients to Xively web services, and connect to the web service and send data.
● It supports REST.

✓ A user creates an account with Xively when deploying Xively APIs for the data
collection and other functions. An API key from my settings is necessarily copied.
Xively APIs enable interface with Python, HTML5, HTML5 server, tornado,
webSocket, webSocket Server and WebSockets and interface with an RPC (Remote
Procedure Call).
✓ Devices get an online presence. For example, an Arduino climate logging client that
can be accessed via a browser or mobile using Xively.
✓ Xively is an open-source platform that enables IoT devices or sensors network to
connect the sensor data to the web, Xively provides services for logging, sharing and
displaying sensor data of all kinds using an HTTP based API.
✓ Xively is based on the concept of users, feeds, data streams, data points and triggers. A
feed is typically a single location (e.g. a house), and data streams are of individual
sensors associated with that location (for example, ambient lights, temperatures, power
consumption).
➢ Pull or Push (Automatic or Manual Feed) Methods for IoT Devices Data
Xively provides two modes for data capture, viz. a pull method (automatic feed type) where
data is collected from an http server, and a push method (manual feed type) where data is
written to Xively using an http client.

➢ Data Formats and Structures

Number of data formats and structures enable interaction, data collection and services with
Xively. The support exists for JSON, XML and CSV (a tabular, spreadsheet, excel, data
numbers and text with a comma-separated values in file).

➢ Private and Public Data Access

A free account supports up to 10 sensor feeds updated in near real time and the data is stored
for up to 3 months. Interactive graphs provided by the service can be embedded onto the mobile
7
 giving the application access to data anywhere. The application can even use other user’s data
feeds as inputs.

➢ Data streams, Data points and Triggers

Xively enables data streams, data points and triggers. Data stream means continuous sensed
data flow over the Internet. Data points mean data values, whereas trigger means action on a
state change. For example, ambient light sensing below a threshold value near a group of
streetlights or data point reaching at threshold, maximum or minimum or set point.

➢ Creating and Managing Feeds

Sensors or IoT devices network, such as a group of streetlights, can feed (also called
environment) data to Xively over the Internet. Individual devices can send the data streams for
the sensed parameters to Xively over Internet.

➢ Visualising Data
Xively is a platform which captures data over the Internet in real time and provides graphing,
alerts and historical data access. Xively enables visualising data of feeds and data streams.
Xively allows the manual as well as automatic feeds.

IOT CLOUD BASED DATA COLLECTION, STORAGE AND COMPUTING SERVICES

USING NIMBITS

➢ Nimbits enables IoT on an open source distributed cloud.

➢ Nimbits cloud PaaS deploys an instance of Nimbits Server at the device nodes. Nimbits
functions as an M2M system data store, data collector and logger with access to historical data.
➢ Nimbits architecture is a cloud-based Google App Engine.
➢ Nimbits server is a class hierarchy com.nimbits.server.system.ServerInfo of java.lang.Object.

NIMBITS PaaS services offer the following features

➢ Edge computing locally on embedded systems, built up of local applications It runs the
rules and pushes important data up to the cloud running when connected over Internet
and an instance of Nimbits Server hosts at the device nodes which is then enabled
➢ It supports multiple programming languages, including Arduino, new Arduino library,
push functions from Arduino cloud, JavaScript, HTML or the Nimbits io Java library
➢ Nimbits server functions as a backend platform Nimbits data point can relay data
➢ between the software systems, or hardware devices such as Arduino, using the cloud as
a backend
➢ An open-source Java library called nimbits.io enables easy development of JAVA, web
and Android solutions Nimbits data, alerts, messages on mobile)
➢ It provides a rule engine for connecting sensors, persons and software to the cloud and
one another Rules can be for calculations, statistics, email alerts, xmpp messages, push
notifications and more

8
 ➢ It provides a data logging service and access, and stores the historical data points and
data objects
➢ Storage in any format that can be serialised into a string, such as JSON or XML
➢ It filters the noise and important changes sent to another larger central instance
➢ It processes a specific type of data and can store it
➢ Time or geo stamping of the data
➢ Nimbits clients provide over Internet, data collection in real time, charts, chart and
Graphical plots of collected data and data entry Data visualisation for data of connected
sensors to IoT devices
➢ Supports the alerts subscription, generation and sending in real time over the Internet
➢ It creates streams of data objects and stores them in a data point series
➢ Data accessibility and monitoring from anywhere, and is used to shape the behaviour
of connected devices and software
➢ It supports the mBedTM Arduino, Raspberry Pi based and other hardware platform
based IoT devices

Figure shows connected devices, sensor nodes, network data points, Nimbits server,
deployment at the device network nodes, and networked with the Nimbits Server (SaaS and
IaaS services) at cloud for applications and services

9
 Data Points
➢ A data point means a collected value of a sensor in a group of sensors. Data points organize
the data in a number of ways.
For example, points can have child points (child points mean subpoints; for example, if light
level is a data point then light on or off is a child point and light level above or below the
threshold can be another child point)
➢ Points can be in the folders The folders can go as deep as like in a tree (Tree means a
folder having several subfolders, a subfolder having several subfolders, till the leaf subfolder
➢ Any type of document can upload and organise them with the points Files can be shared
publicly or with the connections
➢ A subscription data feed is a special point for each user that logs system messages, events,
alerts from other points which are subscribed by a service and more

Data Channels
➢ A user can create a data feed channel which shows the system events and messages that
also shows data alerts which are subscribed to show up in the feed
➢ The user can subscribe to the data point of other users also, and configure the
subscription(s) to send messages to the feed The user can observe the idle, high or low
alerts here in real time
➢ The user data feed is just another Nimbits data point

Jabbing
Jabbing means pushing the alerts or messages down quickly or pushing repeatedly Each type
of alert or message is assigned a Jabber ID, called JID Each JID consists of three main parts,
viz the node identifier ( domain identifier ( and resource identifier ( A JID is written in notations
as < JID>::==[<node>”@”]<domain>[“/”<resource>.

Subscriptions
Subscriptions are alternative to the configurations of alert (an alert configuration means
specifying when a point became idle, high, low using the point property menu). A subscription
to a point creates on configuring the subscription how an application is programmed to get
alerts (Twitter, Email or other) and what events trigger the alert (new data, high or other alerts).

Summary point
A user creates a summary point which can compute averages, minimum, maximum, standard
deviations, variance and sums of another point on a specific time interval basis

Calculations
A user can create calculation objects for a point The objects can organise in a tree and a user
can apply many formulas for a single data point For example, in temperature sensor data point,
one formula is for increase over the last value while the other is for increase over a normal
value, each time a new temperature data point is recorded.

10
IOT PHYSICAL DEVICES & ENDPOINTS

loT devices are connected to the Internet and send information about themselves or about their
surroundings (e g information sensed by the connected sensors) over a network (to other
devices or servers/storage) or allow actuation upon the physical entities/environment around
them remotely.

An loT device can consist of a number of modules based on functional attributes, such as
• Sensing : Sensors can be either on board the IoT device or attached to the device loT device
can collect various types of information from the on board or attached sensors such as
temperature, humidity, light intensity, etc The sensed information can be communicated either
to other devices or cloud based servers/storage
• Actuation : loT devices can have various types of actuators attached that allow taking actions
upon the physical entities in the vicinity of the device For example, a relay switch connected
to an loT device can turn an appliance on/off based on the commands sent to the device
• Communication : Communication modules are responsible for sending collected data to
other devices or cloud-based servers/storage and receiving data from other devices and
commands from remote applications
Analysis Processing : Analysis and processing modules are responsible for making sense of
the collected data

11
RASPBERRY PI
Raspberry Pi is a low-cost minicomputer with the physical size of a credit card Raspberry Pi
runs various flavors of Linux and can perform almost all tasks that a normal desktop computer
can do. In addition to this, Raspberry Pi also allows interfacing sensors and actuators through
the general purpose I/O pins Since Raspberry Pi runs Linux operating system, it supports
Python "out of the box"

12
Raspberry Pi has serial, SPI and 12 C interfaces for data transfer as shown in Figure
Serial
The serial interface on Raspberry Pi has receive (and transmit (pins for communication with
serial peripherals
SPI
Serial Peripheral Interface ( is a synchronous serial data protocol used for communicating with
one or more peripheral devices In an SPI connection, there is one master device and one or
more peripheral devices
There are five pins on Raspberry Pi for SPI interface
•MISO (Master In Slave Out) Master line for sending data to the peripherals
•MOSI (Master Out Slave In) Slave line for sending data to the master
•SCK (Serial Clock) Clock generated by master to synchronize data transmission
•CEO (Chip Enable 0) To enable or disable devices
•CEO (Chip Enable 1) To enable or disable devices
12C
The I2C interface pins on Raspberry Pi allow you to connect hardware modules I2C interface
Mows synchronous data transfer with just two pins SDA (data line) and SCL (clock line)

13