Big Data
Big Data
Big Data
Dr.V.Bhuvaneswari
Assistant Professor
Department of Computer Applications
Bharathiar University
Coimbatore
[email protected], [email protected]
visit at www.budca.in/faculty.php
Big Data Roadmap
Timeline – Big Data Predictions
Data Growth in Units
Data Landscape
Data Explosion
Big Data Myths
Big Data
5Vs of Big Data
Why Big Data
Data as Data Science
3
Dr.V.Bhuvaneswari, Asst.Professor, Dept. of Computer Applications, Bhararthiar University
Timeline – Big Data
Predictions
1944- Yale Library in 2040 will have “approximately
200,000,000 Volumes
1961- Scientific Journals will grow exponentially rather than
linearly, doubling every fifteen years and increasing
by a factor of ten during every half-century.
1975- Ministry of Posts and Telecommunications in Japan
introduced words as unifying unit of measurement
1997- First article published by Michael Cox and David
Ellsworth in in the ACM digital library to the term
“Big data.”
Dr.V.Bhuvaneswari, Asst.Professor,
Dept. of Computer Applications,
Bhararthiar University 6
BIG DATA FACTS
Every 2 days we create as much information
as we did from the beginning of time until
2003
Over 90% of all the data in the world was
created in the past 2 years.
It is expected that by 2020 the amount of
digital information in existence will have
grown from 3.2 zettabytes today to 40
zettabytes.
Every minute we send 204 million emails,
generate 1.8 million Facebook likes, send
278 thousand Tweets, and up-load 200,000
Dr.V.Bhuvaneswari, Asst.Professor, Dept. of Computer Applications, Bhararthiar University 7
Big Data Explosion
30 billion RFID 4.6
tags today billion
12+ TBs camera
(1.3B in 2005)
of tweet data phones
every day world
wide
100s of
millions
of GPS
data every
? TBs of
enabled
day
devices
sold
annually
25+ TBs 2+
of billion
log data people
on the
every day 76 million smart Web by
meters in 2009… end 2011
200M by 2014
Data Deluge
Big Data Market Size
Potential Talent Pool -Big
Data
India will require a minimum of 1 lakh data scientists in the next couple
of years in addition to data analysts and data managers to support the
Big Data space.
Dr.V.Bhuvaneswari, Asst.Professor, Dept. of Computer Applications, Bhararthiar University 11
BIG DATA MYTHS
Big Data
• New
• Only About Massive Data Volume
• Means Hadoop
• Need A Data Warehouse
• Means Unstructured Data
• for Social Media & Sentiment
Analysis
Dr.V.Bhuvaneswari, Asst.Professor,
Dept. of Computer Applications,
Bhararthiar University 12
Lets Us Clarify
Dr.V.Bhuvaneswari, Asst.Professor,
Dept. of Computer Applications,
Bhararthiar University 13
Big Data
Big Data is
A complete subject with tools, techniques
and frameworks.
Technology which deals with large and
complex dataset which are varied in data
format and structures, does not fit into
the memory.
Not about huge volume of data; provide
an opportunity to find new insight into the
existing data and guidelines to capture
and analyze future data
Dr.V.Bhuvaneswari, Asst.Professor,
Dept. of Computer Applications,
Bhararthiar University 14
Big Data : A Definition
Big data is the realization of greater
business intelligence by storing,
processing, and analyzing data that
was previously ignored due to the
limitations of traditional data
management technologies
:Source: Harness the Power of Big Data: The IBM Big Data Platform
Dr.V.Bhuvaneswari, Asst.Professor,
Dept. of Computer Applications,
Bhararthiar University 15
BIG DATA as Platform
Source: IBM
Dr.V.Bhuvaneswari, Asst.Professor, Dept. of Computer Applications, Bhararthiar University 16
4 V‘s of Big Data
19
The 5 Key Big Data Use Cases
Data Models
Linear
Regression,
Decision Tree, Pre-Processing
Dimensionality - ETL
Reduction
Dash Clustering
Boards Outlier
ChartsPie, Analysis
Bar Association
Histogram Analysis
33
Data Science Applications
Data Personalization - Logs, Tweets, Likes
Smart Pricing – Air Transportation
Financial Services – Fraud Detection
Insurance
Smart Grids – Energy Management
41
Big Data Applications