DESIGN AND

ANALYSIS OF
ALGORITHMS
Course Code : AR212
Credit Hours :3
Prerequisite : AR211
Instructor Information

Name :Dr. Aryaf Abdallah Aladwan

Office No. --
Tel (Ext) None
E-mail [email protected]
Office Hours Sun, Tue, Thu 10:00-11:00, 15:00-16:00
Class Times Building Day Start Time End Time Room No.
AI Sun -Tue, :0014 :0015 AI LAB
Thu
Salt Technical Sun -Tue, 12:00 :0013 8
College Thu
This course is to design efficient computer algorithms, prove their
correctness, and analyze their running times. it includes mathematical
analysis of algorithms (summations and recurrences), advanced data
structures (balanced search trees), algorithm design techniques (divide-
and-conquer, dynamic programming, and greedy algorithms), graph
algorithms (breadth-first and depth-first search, minimum spanning
trees, shortest paths).
Course Title: DESIGN AND ANALYSIS OF ALGORITHMS
Credit Hour(3)

[Pre-req. AR211
Textbook: Introduction to Algorithms, Thomas Cormen, Charls Leiserson & Ronald Rivest. 3rd Edition,
MIT Press, 2009.

Image of the textbook Cover

COURSE OBJECTIVES

The main learning objectives of the course are to:

1. Learn about the use of several design techniques and the importance of studying the complexity of
a particular algorithm
2. Computational complexity analysis of algorithms and worst-case algorithm runtime analysis using
asymptotic analysis and mathematical analysis of algorithms (summations and recursion)
3. Writing and analyzing recursive relationships for recursive algorithms
4. Describe the divide and conquer approach as a principle for algorithm design
5. Learn about searching and sorting algorithms
6. Ability to design, analyze and validate graph algorithms and shortest path algorithms
7. Ability to design and analyze algorithms that follow the approach of greedy algorithms
8. Understand dynamic programming algorithms and optimization algorithms and know when the
algorithm design situation calls for it.
9. Learn about advanced data structures
COURSE SYLLABUS

Week Course Topic

Week 1 Introduction to Algorithms, The Efficiency of Algorithms, Attributes of Algorithms. A Choice of
Algorithms. Classifications of algorithms: by implementation, by design, by field of study and by
complexity.
Week 2 Mathematical Background (summations and recurrences)
Week 3 Measuring Efficiency, Measuring the asymptotic growth of functions. The basic techniques for
manipulating bounded summations, searching techniques (linear search, binary search).
Week 4 Techniques for problem-solving: Divide and Conquer, the General Method, Recurrence Equations,
Solving Recurrence Equations (Master and iteration Methods).
Week 5 and 6 Sorting techniques based on Divide and Conquer: Merge Sort.
Quick Sort, Strassen's Matrix Multiplications.
Week 7 Other sorting techniques: Insertion sort, Selection Sort, Heap Sort
Week 8 Midterm Exam
Week 9 Graph Algorithms, Formal Definition of Graphs. Graphs Types.
Graph Terminologies, Strongly Connected Graph, Representations of Graphs.
Week 10 Graph Traversal, Breadth-First Search Algorithm, Depth-First Search Algorithm. Topological Sort
Algorithm.
Week 11 Greedy Algorithms, Minimum spanning trees
Week 12 Shortest paths algorithms
Week 13 Dynamic Programming
Week 14 Optimization Algorithms
Week 15 Advanced data structures (balanced search trees)
Week 16 Final Exam
Week 4
 Chapter 4
Recursion and Recurrence

By:
Dr. Aryaf Al-Adwan
Autonomous Systems Dept.
Design and Analysis of
algorithms Course
8/28
Introduction
toRecursion
 Recursive procedures are functions that invoke themselves
directly (call themselves from within themselves)

 Recursion:
. An alternative to iteration
. Recursion can be very elegant at times,
. Not inexpensive to implement...
Classic examples of recursion
. Recursive calculation of a string length, factorials,
divide and conquer, towers of Hanoi, binary searches, and more

 Recursive functions are used in many applied areas.

. In artificial intelligence.
. In searching data structures that are themselves "recursive" in nature, such
as
trees.

9/28
Example: Add the numbers from 1 to 5

Base case
int sum (int n)
{
if (n <= 1)
return n;
else
return (n + sum(n-1)); Recursive Case
} // end sum()

Notice the recursion in the Else branch

Note the ‘return’ calls itself! (calls the method it resides
in!)
Note the ‘Base Case’
Note the ‘Recursive Case’

10/2
8
int public sum (int n)
{
if (n <= 1)
return n;
else
return (n + sum(n-1));
}// end sum()

START WITH N = 5:

SUM(5): 5 + SUM(4) - A FUNCTION CALL (SUM(4)) WHOSE VALUE WE AWAIT

SUM(4): 4 +SUM(3) -A FUNCTION CALL WHOSE VALUE WE AWAIT

SUM(3): 3 + SUM(2) -A FUNCTION CALL WHOSE VALUE WE AWAIT

SUM(2): 2 + SUM(1) - A FUNCTION CALL WHOSE VALUE WEAWAIT

SUM(1): 1

11/2
8
WHEN WE GET SUM(1)=1,
THEN WE REFER TO 2 + SUM(1) equals 2 + 1 = 3. (complete)

NEXT, SUM (3) = 3+SUM(2), which is 3 + 3 OR 6; (now complete)

We now have a value for SUM(3)

NEXT, SUM (4) = 4+SUM(3), WHICH IS 4 + 6 OR 10; (now complete)

FINALLY, SUM (5) = 5+SUM(4), WHICH IS 5 + 10 OR 15 (now complete)

12/2
8
Consider: FACTORIAL FUNCTION

Where n! = n* (n-1) * (n-2) * ... 1! * 0!

Where 1! = 1 & 0! = 1
by definition.

Thus 5! = 5 * 4 * 3 * 2 * 1 = 120

13/2
8
Factorial

RECURSIVE FUNCTION IS WRITTEN

AS:
int public factorial (int n)
{
if (n==1)
return (1) /* THIS IS THE BASE CASE */
else
return (n * factorial (n-1) ); /*
RECURSIVE CASE */
} end factorial ()

14/2
8
Factorial int public factorial (int n)
{
if (n==1)
return (1) /* base case */
else
main calls argument n = 7. return (n * factorial (n-1) );
factorial(7) /* recursive case
value of n at this node: returned }// end factorial()
value
n=7
return (7*factorial(6)) return(7*720) = 5040 =
answer!!
recursive
call n=6
return(6*fact return(6*120) =
orial(5)) 720
n=5
return(5*factorial(4)) return(5*24) =
120
n=4
return (4*factorial(3)) return(4*6) =
24
n=3
return return (3 *2) =
(3*factorial(2)) 6
n=2
return (2*factorial(1)) return( 2 * factorial(1)) 2 * 1 = 2 Thus factorial (2) = 2. return
= 2.
n=1 1 is substituted for the
return base case call
(1) reached.
15/2
8
Recursive Definitions:
FibonacciSeries
Definition of fibi, for integer i > 0:

• fib1 = 1

• fib2 = 1

• fibn = fibn-1 + fibn-2, for n >2

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, ...

Chapter 7: 16
Recursion
Fibonacci
SeriesCode
public static int fib (int n) {
if (n <= 2)
return 1;
else
return fib(n-1) + fib(n-2);
}

This is straightforward, but an inefficient

recursion ...

Chapter 7: 17
Recursion
11/28
Data Structure in Recursion
 Stacks are Used

In recursive returns we may do some / all of the following: we

pop the stack to get to:
Returned values
Returned addresses
Data areas
Branch to statement
invoking function
taken.

 Calling function regains control and continues for a particular

‘call’ or message invocation
12/28
 Efficiency
ofRecursion
• Recursive method often slower than iterative; why?
• Overhead for loop repetition smaller than
• Overhead for call and return
• If easier to develop algorithm using recursion,
• Then code it as a recursive method:
• Software engineering benefit probably outweighs ...
• Reduction in efficiency
• Don’t “optimize” prematurely!

Chapter 7: 13
Recursion
Recurrence
s

21/28
Definitio
n
• A recurrence is an equation or inequality that describes a function
in terms of itself by using smaller inputs.

22/28
Solving
arecurrence
There are 4 methods :
1) Iteration Method
2) Tree Method
3) Substitution Method
4) Master Theorem

23/28
IterationMetho
d Example1
• Assume we have the following recursive function

void Test( int n)

{
if (n>0)
{
cout<<n;
Test (n-1);
}
}

24/28
FindingtheTotaltimebythisalgorith
m

void Test( int n) 

{ T(n)
if (n>0) 1
{
cout<<n; 1
Test (n-  T(n-1)
1);
}
}
So, T(n) = T(n-1) +2
25/28
RecurrenceRelatio
n

26/28
Solving the recurrence
(usingiteration)
T(n) = T(n-1) + ………..
1 Eq1
Find T(n-1) ?
T(n-1)=T(n-2)+1
So, Substitute T(n-1) in Eq1
T(n)= [ T(n-2) +1 ] +1  T(n-2) ………
+2 Eq2
Find T(n-2) ?
T(n-2)=T(n-3)+1
So, Substitute T(n-2) in Eq2
T(n)= [ T(n-3)
Continues for+1 ] +2 
k times T(n-3)
T (n-k) +
+3
k

27/28
Cont
.
• T (n-k) + k
• This means that I will stay in the recursive state and stop after k steps
• So, when is the stop case?
when n=0
n-k = 0 , so n=k
• T(n)= T(n-k) + k , but n=k
• T(n)= T(n-n) +n
• T(n) = T(0) + n , but T(0)=1
• T(n)= 1+n
= n+1  O(n)

28/28
IterationMetho
d Example2
• Assume we have the following recursive function

void Test( int n)

{
if (n>0)
{
for(int i=0; i<n; i++)
cout<<n;
Test (n-1);
}
}

29/28
Finding the Total time by this
algorithm
void Test( int 
n) T(n)
{ 1
if (n>0)
{ for(int i=0; i<n; i++)  n+1
cout<<n; n
Test (n-  T(n-1)
} 1);
}
So, T(n) = T(n-1) +2n+2
T(n) = T (n-1) +n
30/28
RecurrenceRelatio
n

31/28
Solving the recurrence
(byiteration)
T(n) = T(n-1) + n ………..
Find T(n-1) ? Eq1
T(n-1)=T(n-2)+n-1
So, Substitute T(n-1) in Eq1
T(n)= [ T(n-2) +n-1 ] +n  T(n-2) +(n- ………
1)+n Eq2
Find T(n-2) ?
T(n-2)=T(n-3)+n-2
So, Substitute
T(n)= T(n-2)
[ T(n-3) +n-2 in Eq2  T(n-3) +(n-2)+(n-1)+n
] +(n-1)+n
Continues for k times  T (n-k) + (n-(k-1))+(n-(k-2))+ …..+(n-
1)+n

32/28
Cont
.
T (n-k) + (n-(k-1))+(n-(k-2))+ …..+(n-1)+n
• This means that I will stay in the recursive state and stop after k steps
• So, when is the stop case?
when n=0
n-k = 0 , so n=k
• T(n)= T (n-k) + (n-(k-1))+(n-(k-2))+ …..+(n-1)+n , but n=k
• T(n)= T(n-n) +(n-(n+1)+(n-n+2)+……+(n-1)+n
• T(n) = T(0) + 1+2+3+……+(n-1)+n , but T(0)=1
• T(n)= 1+n(n+1)/2
= n+1  O(n2)

33/28
Solving the recurrence
byiteration Example3

34/28
 Solving the recurrence
byiteration Example3
T(n) = 2T(n/2) + 4n ………..
Eq1
Find T(n/2) ? 2)+4(n/2)
T(n/2)=2T(n/2
So, Substitute T(n/2) in Eq1
T(n)=2 [2T(n/22)+4(n/2)] +4n  T(n)=22 T(n/ 22 )+4n+ 4n ………
Find T(n/22) ? Eq2
T(n/ 22)=2T(n/23)+4(n/ 22)
So, Substitute T(n/ 22) in Eq2
T(n)= 22 [2T(n/23)+4(n/ 22)]+4n+ 4n  T(n)=23 T(n/ 23 )+4n+ 4n+4n
Continues for k times  2k T(n/ 2k )+k (4n)

35/28
Cont
.
2k T(n/ 2k )+k (4n)
• This means that I will stay in the recursive state and stop after k steps
• So, when is the stop case?
when n=1
n/2k = 1 , so n=2k  k=log2n
•T(n)=
T(n)=2k2T(n/
log 2k )+k
T(n/(4n)2 , but
)+ k=log2n
(4n) n log n log n  but 2log2n = nlog 2 =n
2 2 2
• T(n) = n T(1)+ log n (4n) 2
2
• T(n)= 4n+ 4n log n
2
 O(n log n)

36/28
Solving the recurrence using
MasterTheorem
• In the analysis of algorithms, the master
theorem for divide-and-conquer recurrences
provides an asymptotic analysis (using Big O
notation) for recurrence relations of types that
occur in the analysis of many divide and
conquer algorithms.
• The name "master theorem" was popularized
by the widely used algorithms textbook
Introduction to Algorithms Cormen,
Leiserson,
by Rivest, and Stein.

37/28
 GenericFor
m

a>=1
b>1
f(n)=O(nk log pn)
where:
• n is the size of an input problem.
• a is the number of subproblems in the recursion
• n/b is the size of each subproblem
• f (n) is the cost of the work done outside the recursive calls, which
includes the cost of dividing the problem and the cost of merging the
solutions to the subproblems.

We have to find :
1) log ba
2) k 38/28
 Cont
.
• Based on these values we have 3 cases:

Case1 Case2 Case3

If (log b a) > k If (log ba) = k If (log ba) < k
Then, Then, Then,
T(n)= O( n log a ) If p>-1  T(n)= O( nk log p+1
b If p≥0 
n) If p=-1 T(n)= O( nk T(n)= O( nk logpn)
loglogn) If p<-1 T(n)= If p<0 
O( nk) T(n)= O( nk)

39/28
Case
1Example

40/28
Another
exampleonCase1

41/28
Case
2Example

42/28
Another
exampleonCase 2

43/28
Case
3Example

44/28
 En
d

45/28