Algorithm analysis & design

Introduction to Algorithms

Presented By:

DR. Marwa Mamdouh Emam

Lec 1 FCI, Minia University

 Contents
2

 Introduction to Algorithms.
 Growth of Functions (Asymptotic Notations).
 Recurrences, solution of recurrences by Substitution.
 Recursion Tree Method
 Master Method
 Analysis of Insertion Sort.
 Divide and Conquer Algorithms
 Analysis of Quick Sort, Merge Sort, and Binary Search.
 Contents
3

 Heap Sort
 Dynamic Programming
 Huffman Codes
 Graph Algorithms
 Single Source Shortest Path
 Minimum Spanning Tree.
 Bellman Ford Algorithm
 Agenda
4

 Introduction
 Algorithm Design.
 Examples
 Algorithm Analysis
 Running Time
 Example
 Guiding Principles
 Running Time vs. Order
 How to calculate Order?
 Examples
 Introduction
5

 What’s algorithm…?!
 Why algorithm…?!
 Is It Important?!
 Goal
 What’s Algorithm?
6

 Set of finite steps to solve certainproblem.

 Any well-defined computational procedure that takes some value, or set of

values, as input and produces some value,or set of values, asoutput.

 Algorithms are step-by-step instructions or procedures

designed to solve specific problems. They provide a
systematic and structured approach to achieving a desired
outcome or performing a task efficiently.
 What’s Algorithm?
7

 Problem???.
 Task to be performed.
 Consists of Inputs and Outputs.
 Program
 Is an instance of algorithm in a certain programming
language.
 Algorithm analysis refers to the process of evaluating and
studying algorithms to understand their efficiency, behavior, and
performance characteristics.
 It involves the systematic examination of algorithms to determine
how they use computational resources such as time and space,
and how their efficiency scales with the size of input data.
 Why Algorithm?
8

 Saveresources
 Save time.
 High Capacity
 Efficiency.
 Choose high performance algorithm
 Is It Worth?
9

 Real examples…
• Fibonacci: recursive vs. loop vs. Dynamic Pro.
(N = 30, 40, 50)
• Median filter: quick sort vs. countingsort
(WinSize = 11 or 15)
• String similarity: recursive vs. dynamic prog.
(S1 ="plynomialgood" S2 = "exponentialbad")
 Is It Worth?
10

 It's Crucial CSCourse!

• 4 CSCrucial Courses (according to IEEE-ACM)
1. Theory of computation “What can becomputed?”
2. Algorithms and data structures “Compute it efficiently”
3. Programming methodology and languages “Code it! different
paradigms”
4. Computer elements and architecture “understand the
destination”
 Is It Worth?
11

 It's Core Interview Question!

• Ask your graduate colleagues!!
 Goal
12

Think…

 Design…

 Analyze…
 Before & After!
13

 Before algorithm: Write code to solveproblem

 After algorithm: Write EFFICIENT code to solveproblem

• It’s a course!!

• It’s a skill and attitude

 RESOURCES
11

 Textbook:-
1. Thomas Cormen, Charles Leiserson, Ronald Rivest, and Clifford Stein
Introduction to Algorithms. 3rd ed. MIT Press, 2009.
2. Anany Levitin, Introduction to the design and analysis of algorithms
2nd Edition, 2007.

 Online Courses:-
1. [Stanford] Algorithms: Design and Analysis: Videos, Join the course

2. [MIT] Introduction to Algorithms: Videos

 Problem Solving: MainSteps
15

• Programming is a process of problem solving

• Problem Solving: MainSteps
Problem definition
Algorithm design
Algorithm analysis
Implementation
Testing
Maintenance
 Problem Solving: MainSteps
16

• Problem Solving: MainSteps

Problem definition
Algorithm design
Algorithm analysis
Implementation
Testing
Maintenance
Algorithm design
 Algorithm design: How to describeAlgorithm?
18

 Algorithm can be described/ represented in three ways.

1. Natural language like English:
2. Graphic representation called flowchart:
3. Pseudo-code Method:
Pseudocode is a human-readable, high-level description of
an algorithm. It uses plain language and simple constructs to
outline the logic and steps of the algorithm without being
tied to the syntax of any particular programming language.
 From our Objectives is Design algorithms using Pseudo-code.
 Pseudo-code conventions:
19

1. Comments begin with // and continue until the end of line.

2. Blocks are indicated with matching braces {and}.
3. An identifier begins with a letter. The data types of variables
are not explicitly declared.
4. Assignment of values to variables is done using the
assignment statement.
<Variable>:= <expression>; Or <Variable> ← <expression>;
5. There are two Boolean values TRUE and FALSE.
 Logical Operators AND, OR, NOT
 Relational Operators <, <=,>,>=, =, !=
 Pseudo-code conventions:
20

6- The following looping statements are employed.

For, while and repeat-until

 While Loop:
While < condition > do
<statement-1>
<statement-n>
End While
 Pseudo-code conventions:
21

 For Loop:
For variable: = value-1 to value-2 do
<statement-1>
<statement-n>
End For
 repeat-until:
repeat
<statement-1>
.
<statement-n>
until<condition>
 Pseudo-code conventions:
22

7- A conditional statement has the following forms.

If <condition> then <statement>
If <condition> then <statement-1>

Else <statement-1>

Case statement:
Switch (expression)
case 1 : <statement-1>

case n : <statement-n>
default : <statement-n+1>
End switch
 Pseudo-code conventions:
23

8- Input and output are done using the instructions

read & write (print).
9- The heading of algorithm takes the form,
Algorithm Name (Parameter lists)
 Algorithm Design:Examples
24

Example 1:
• write a Pseudo Code for finding the maximum number of ‘n’
given numbers in array A.
 Algorithm Design: Examples
25

Example 2:
• write a Pseudo Code to calculate the factorial of a number
(N).
 Algorithm Design: Examples
26

Example 2:
• write a Pseudo Code with a natural number, N, as its input which
calculates the following formula and writes the result in the standard
output:
𝟏 𝟏 𝟏
𝑺= + + ⋯+
𝟐 𝟒 𝑵
Algorithm analysis
 AlgorithmAnalysis
28

 Algorithm analysis refers to the process of evaluating and studying

algorithms to understand their efficiency, behavior, and performance
characteristics.
 It involves the systematic examination of algorithms to determine
how they use computational resources such as time and space, and
how their efficiency scales with the size of input data.
 The main goal is to determine the cost of running an algorithm and
how to reduce that cost. Cost is expressed as Complexity
• Time Complexity
• Space Complexity
 AlgorithmAnalysis
29

 Time Complexity:
 This aspect focuses on measuring how the runtime of an
algorithm grows with respect to the size of the input.
 Time complexity is often expressed using big O notation,
and it provides an upper bound on the number of basic
operations (such as comparisons or assignments) the
algorithm performs in relation to the input size.
 AlgorithmAnalysis
30

 Space Complexity:
 Space complexity deals with the amount of memory or
storage space an algorithm requires as a function of the
input size.
 It helps analyze how an algorithm utilizes memory
resources during its execution.
 Space & Time Complexities
31

• Space complexity
• How much space is required

• Time complexity
• How much time does it take to run the algorithm
 Space & Time Complexities
32

 Time Complexity Depends on: -

 Size of Data
 Number of Operations needed (n)
 Machine Speed (In case of small input size)
 Space Complexity Depends on: -
 Size of Data
 Size of Program
 Algorithms running time is an importantissue
 Time Complexity
27

 Running Time : # steps that the algorithm takes to

solve the problem.
• # steps expressed as a function of input size T(n) .
• T(n) = number of operations required.

 (n) is the Problem Size:

n could be the number of specific operations, or the size of data (e.g. an
array) or both.
 Time Complexity
28

 Number of operations (# steps)

 Each "simple" operation (+, -, =, <, >=) is one operation (1 unit of
time).

 Loops and function calls are not simple operations, but

depend upon the size of the data and the contents of a
function.

 Each memory access is one operation.

We measure T(n) of an algorithm by counting the number of steps

(operations).
 Algorithm Analysis: Example
35

 Design an Algorithm that get the max element from an array, and then
analyze this algorithm.

Init 1 1
Increment n
Cond (comp) n+1 n
n 1

 Running Time = T(n) = 1+n+n+1+n+1+1=3n+4.

 Algorithm Analysis: Example
36

 Design an Algorithm that get the max element from an array, and then analyze
this algorithm.

 Running Time = T(n) = C1+C2+C3 * n + C4.

 = O(n)
 GuidingPrinciples
37

 Worst Case
 Asymptotic analysis (Large input Size)
 Constants are not important
• What is order?
• Running time vs Order.
 Running Time vs.Order
31

 Order :
is the dominant factor in running time without
constants.
 Example :
• T(n) = 6n + 4 O(n)
• T(n) = 10 n^2 + 7n + 10^6 O(n^2)

 Example:
• Algo1 T(n) = 10^6 Log n O(Log n)
• Algo2 T(n) = 20 n O(n)
 How to calculate Order?
39

 Analysis
1. Statements
2. Conditions
 If-statement
 Switch case
 Nested if
3. Loops
4. Recursion
 How to calculate Order?
40

 Analysis
1. Statements O(1)
Exception:-
M=GetMax(A,N) O(Fn)
2. Conditions
 If-statement
 For condition O(1)
 For body O(body)
 O(IF)=O(1)+O(Body)= O(Body)
 How to calculate Order?
41

 Analysis
2. Conditions
If Condition then
B1
Else if condition Then
B2
……
Else
Bk

 Order O(Max(B1,B2…..,BK))
 How to calculate Order?
42

 Analysis
1. Loops
Loop
body
End loop

Order=O(body) x # iteration
 How to calculate Order?
43

 Example 1:-
O(1)

O(1)
O(n)
O(1) O(1)

O(1)

 O(n)
 How to calculate Order?
44

 Example 2:

B1= O(1)

B2= O(N)

O(1)

O(N)
O(N) X N
O(N) O(1)
B3=O(N^2)

O(1)
 O(N^2)
 How to calculate Order?
45

 Example 3:
O(1) X #Iteration
O(1) X Log N= O(log N) O(1)

Step no 0 1 2 3 …. K-1 K
Value of N N 𝑁 𝑁 𝑁 ........ 𝑁 𝑁
10 102 103 10𝐾−1 10𝐾

 Termination occur:
𝑁
 𝐾 = 𝟏
10
10𝐾 =N
𝐾 𝑙𝑜𝑔10 10 = 𝑙𝑜𝑔10 𝑁
 𝑙𝑜𝑔10 10𝐾 = 𝑙𝑜𝑔10 𝑁
 K=𝑙𝑜𝑔10 𝑁
 O(log N)