Subject Name Parallel and Distributed Computing

GUJARAT TECHNOLOGICAL UNIVERSITY
Bachelor of Engineering
Subject Code: 3170721
Semester – VII
Subject Name: Parallel and Distributed Computing
Type of course: Elective
Prerequisite: Fundamental of programming and operating system, computer networking, and computer
Organization
Rationale: To introduce the fundamentals of parallel and distributed programming models, design
principles, and application development in different parallel programming environments.
Teaching and Examination Scheme:
Teaching Scheme Credits Examination Marks Total

L T P C Theory Marks Practical Marks Marks
ESE (E) PA (M) ESE (V) PA (I)
3 0 2 4 70 30 30 20 150
Content:
Sr. No. Content Total

Hrs
1 Introduction to Parallel Computing: 05

The Idea of Parallelism, Power and potential of parallelism, Examining sequential and
parallel programs, Scope and issues of parallel and distributed computing, Goals of
parallelism, Parallelism and concurrency using multiple instructions streams.
2 Parallel Architecture: 10
Pipeline architecture, Array processor, Multi processor architecture, Systolic architecture,
Dataflow architecture, Architectural classification schemes, Memory access classification,
Memory Issues : Shared vs. distributed, Symmetric multiprocessing (SMP), SIMD, Vector
processing, GPU co-processing, Flynn’s Taxonomy, Instruction Level support for parallel
programming, Multiprocessor caches and Cache Coherence, Non-Uniform Memory
Access (NUMA).
3 Parallel Algorithm Design Principles and Programming: 10

Need for communication and coordination/synchronization, Scheduling and contention,
Independence and partitioning, Task- Based Decomposition, Data Parallel Decomposition,
Characteristics of task and interaction, Load balancing, Data Management, parallel
algorithm models, Sources of overhead in parallel programs, Performance metrics
for parallel algorithm implementations, Parallel algorithmic patterns like divide and
conquer, Map and Reduce, Specific algorithms like parallel Merge Sort, Parallel graph
Algorithms.
4 Introduction to Distributed Systems: 06
Goals of the Distributed Systems, Relation to parallel systems, synchronous
versus asynchronous execution, design issues and challenges, Types of Distributed
Page 1 of 3
w.e.f. AY 2018-19
Systems, Distributed System Models, Hardware and software concepts related to
distributed systems, middleware models.
5 Distributed Computing and Communication design principles: 08
A Model of distributed executions, Models of communication networks, Global state of
distributed system, Models of process communication. Communication and Coordination:
Shared Memory, Consistency, Atomicity, Message- Passing, Consensus, Conditional
Actions, Critical Paths, Scalability, and cache coherence in multiprocessor systems,
synchronization mechanism.
6 Parallel and Distributed Programming Frameworks 06
Overview of CUDA, OpenMP, POSIX Threads, Apache Hadoop (DFS), and current trends
in parallel and distributed computing.
Suggested Specification table with Marks (Theory):
Distribution of Theory Marks
R Level U Level A Level N Level E Level C Level

7 14 21 14 7 7
Legends: R: Remembrance; U: Understanding; A: Application, N: Analyze and E: Evaluate C:

Create and above Levels (Revised Bloom’s Taxonomy)
Reference Books:
1) Introduction to Parallel Computing (2nd Edition), Ananth Grama, Anshul Gupta, and George Karypis,
Vipin Kumar, Addition Wesley
2) Parallel and Distributed Systems 2nd Edition, Arun Kulkarni, Nupur Prasasd Giri, Nikhilesh Joshi,
Bhushan Jadhav, Wiley
3) Introduction To Parallel Programming, Steven Brawer, Academic Press
4) Introduction To Parallel Processing, M.Sasikumar, Dinesh Shikhare and P. Ravi Prakash, Randy Chow,
T. Johnson, Distributed Operating Systems and Algorithms, Addison Wesley
5) Distributed Operating Systems, A.S. Tanenbaum, Prentice Hall
6) Ian Foster: Designing and Building Parallel Programs – Concepts and tools for Parallel Software
Engineering, Pearson Publisher, 1st Edition, 2019.
7) Parallel Programming in C with MPI and OpenMP Michael J.Quinn, McGrawHill Higher Education
Page 2 of 3
w.e.f. AY 2018-19
Course Outcomes:
Sr. CO statement Marks %

No. weightage
CO-1 Apply the fundamentals of parallel and distributed computing including 30%
parallel architectures and paradigms.
CO-2 Analyze the various design principles of parallel algorithms. 20%
CO-3 Learn the intricacies of parallel and distributed programming 30%
CO-4 Develop and execute basic parallel and distributed applications using 20%
basic programming models and tools.
List of Experiments:
The laboratory will be based on the implementation of the parallel algorithms (on a PC-cluster under Linux
platform). The programs will be based on POSIX Thread, MPI programming, Hadoop, Apache Spark etc.
List of e-Learning Resources:
1. https://hpc.llnl.gov/training/tutorials/introduction-parallel-computing-tutorial
2. https://www.geeksforgeeks.org/introduction-to-parallel-computing/
3. https://nptel.ac.in/
4. https://www.coursera.org/
Page 3 of 3
w.e.f. AY 2018-19

Subject Name Parallel and Distributed Computing

Uploaded by

Copyright:

Available Formats

Subject Name Parallel and Distributed Computing

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Subject Name Parallel and Distributed Computing

Uploaded by

Copyright:

Available Formats

GUJARAT TECHNOLOGICAL UNIVERSITY

Type of course: Elective

Teaching and Examination Scheme:

Teaching Scheme Credits Examination Marks Total

Sr. No. Content Total

1 Introduction to Parallel Computing: 05

3 Parallel Algorithm Design Principles and Programming: 10

Suggested Specification table with Marks (Theory):

Distribution of Theory Marks

R Level U Level A Level N Level E Level C Level

Legends: R: Remembrance; U: Understanding; A: Application, N: Analyze and E: Evaluate C:

Sr. CO statement Marks %

List of e-Learning Resources:

You might also like