4 Ppt on YARN MapReduce 31 10 20

Download as pdf or txt
Download as pdf or txt
You are on page 1of 17

Program: B.

Tech, CS, 5th sem,3rdyear

CS503: Data analytics

Unit IV
Topic: working of YARN in Hadoop
July-Dec

Ms. Jyoti Kumari


Asst. Professor, Computer Science & Engineering
Outlines
• Prerequisite of Course
• Course objectives
• Hadoop
• MapReduce
• Working of YARN
• Architecture of YARN
• Core components of YARN
• Advantages of YARN
• Research scope
• References

2
Prerequisite of Course

• Basics of programming language in which language you want to learn Data


analytics.
• Improve your analytical and logical skill to approach towards the problem.

• The fundamental concept of mathematics are clear so you can easily


understand the terms in the data analytics.

3
Course Objective

• Understand and remember algorithms and its analysis procedure.

•To introduce various techniques for representation of the data in the real
world.

•To design and implement various analysis algorithms.

4
Architecture of MapReduce

5
Working of MapReduce:

6
Role of Job tracker:-

7
Role of task tracker:-

8
Architecture of YARN:-
4 core components of –
1. Resource manager- only responsible for resource allocation.
2. Node manager- responsible for launching the application
container .
3. Application Master-
When the container has errors ,the application master will apply for
another resource from the scheduler to continue the process.
4. Container- it is the source unit which is available node splitting the
organization resources.

9
YARN

YARN:-
It means yet
Another
Resource
Negotiator.

10
YARN
It is also called next generation MapReduce.
Pupose of YARN:-
1. Divide the tasks for the Jobtracker.
2. Resources are managed by Resource manager.
3. Jobs are traced by application master.
4. Task tracker has become the node manager.
5. So global resource manager and local ode manger compose the
data computing framework.
6. Resource manager will be the resources distributor while the
apllication master is response for the communication with resource
manger and cooprate the node manager to complete the tasks.

11
Advantages of YARN:-

1. Increases the scalability of cluster.

2. Users can write their own program based on the


programming model.

3.It supports the program to have a specific checkpoint.

12
Uses of MapReduce:-
1.At google –
• index building for google search
• Article clustering for google news
• Statistical machine translation
2. At yahoo-
• Spam detection for yahoo mail
• Index building for yahoo search
3.At facebook-
• Ad optimization
• Spam detection

13
Research Scope
• Role of Data analytics in multiple disciplines

• To Organize and Analysis of Data for many companies.


• To predict about various result
• Extensive use of Big data in india for employment
problem.
•To analyse various techniques of business intelligence.
•Research scope in Data Mining and Data Warehouse

14
References

1. Michael Minelli, Michehe Chambers, “Big Data, Big


Analytics: Emerging Business Intelligence and Analytic
Trends for Today’s Business”, 1st Edition, Ambiga Dhiraj,
Wiely
2. CIO Series, 2013. Arvind Sathi, “Big Data Analytics:
Disruptive Technologies for Changing the Game”, 1st
Edition, IBM Corporation, 2012.1. Rajaraman, A.,

15
References
3. Barlow, M., Real-Time Big Data Analytics: Emerging
Architecture, O Reilly, 2013
4. Schonberger, V.M. , Kenneth Cukier, K., Big Data, John
Murray Publishers, 2013,

16
17

You might also like