Analysis and Design of Algorithms (ADA)

GTU # 3150703

Unit-3:
Divide and Conquer
Algorithms

Dr. Gopi Sanghani

Computer Engineering Department
Darshan Institute of Engineering & Technology, Rajkot
[email protected]
9825621471
 Outline
Looping
 Introduction to Recurrence Equation
 Different methods to solve recurrence
 Divide and Conquer Technique
 Multiplying large Integers Problem
 Problem Solving using divide and conquer
algorithm –
 Binary Search
 Sorting (Merge Sort, Quick Sort)
 Matrix Multiplication
 Exponential
Recurrence Equation
Introduction
 Many algorithms (divide and conquer) are recursive in nature.
 When we analyze them, we get a recurrence relation for time complexity.
 We get running time as a function of 𝒏 (input size) and we get the running time on inputs of
smaller sizes.
 A recurrence is a recursive description of a function, or a description of a function in terms of
itself.
 A recurrence relation recursively defines a sequence where the next term is a function of the
previous terms.

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 4
Methods to Solve Recurrence
 Substitution
 Homogeneous (characteristic equation)
 Inhomogeneous
 Master method
 Recurrence tree
 Intelligent guess work
 Change of variable
 Range transformations

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 5
Substitution Method – Example 1
  We make a guess for the solution and then we use mathematical induction to prove the guess
is correct or incorrect.
  Time to solve the
Example 1: instance of size

 
𝑻 ( 𝒏)=𝑻 ( 𝒏− 𝟏)+𝒏 1

 Replacing by
  Timeand , wethecan write following equations.
to solve
instance of size
 
𝑻 ( 𝒏 −𝟏 ) =𝑻 ( 𝒏 −𝟐 )+𝒏 −𝟏 2

𝑻 ( 𝒏 −𝟐 ) =𝑻 ( 𝒏 −𝟑 )+ 𝒏− 𝟐
 
3

 Substituting equation in and equation in we have now,

𝑻 (𝒏)=𝑻 (𝒏− 𝟑)+𝒏 − 𝟐+𝒏− 𝟏+𝒏

 
4
Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 6
Substitution Method – Example 1
 
𝑻 (𝒏)=𝑻 (𝒏− 𝟑)+𝒏 − 𝟐+𝒏− 𝟏+𝒏
 
4

 From above, we can write the general form as,

𝑻 ( 𝒏 ) =𝑻 ( 𝒏 − 𝒌 ) +(𝒏 − 𝒌 +𝟏)+(𝒏− 𝒌 +𝟐)+…+𝒏

 

 Suppose, if we take then,

𝑻 ( 𝒏 ) =𝑻 ( 𝒏 −𝒏 )+(𝒏 −𝒏+𝟏)+(𝒏 − 𝒏+ 𝟐)+…+𝒏

 

𝑻 ( 𝒏 ) =𝟎+𝟏+𝟐+…+𝒏
 

  𝒏 ( 𝒏+𝟏 )
𝑻 ( 𝒏) = =𝑶 ( 𝒏𝟐 )
𝟐
Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 7
Substitution Method – Example 2
 

 Rewrite the equation,

 Now, replace by and

∴𝑡
  ( 𝑛 −1 )=𝑐 2+𝑐 2+𝑡(𝑛 −3)
 Substitute the values of and

 In general,

 Suppose if we take then,

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 8
Substitution Method Exercises
  Solve the following recurrences using substitution method.

1.  and 

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 9
Homogeneous Recurrence
  Recurrence equation

 The equation of degree in is called the characteristic equation of the recurrence,

 Which can be factorized as,

 The solution of recurrence is given as,

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 10
Homogeneous Recurrence – Example 1 : Fibonacci Series
  Fibonacci series Iterative Algorithm

Function fibiter(n)
i ← 1; j ← 0;
for k ← 1 to n do
j ← i + j;
i ← j – i;
return j

 Analysis of Iterative Algorithm: If we count all arithmetic operations at unit cost; the
instructions inside for loop take constant time . The time taken by the for loop is bounded
above by , .,
Case 1
Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 11
Homogeneous Recurrence – Example 1 : Fibonacci Series
  If the value of is large, then time needed to execute addition operation increases linearly with
the length of operand.
 At the end of iteration, the value of and will be and .
 As per De Moivre’s formula the size of is in .
 So, iteration takes time in . let be some constant such that this time is bounded above by for
all .
 The time taken by fibiter algorithm is bounded above by,

Case 2

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 12
Homogeneous Recurrence – Example 1 : Fibonacci Series
  Recursive Algorithm for Fibonacci series,

Function fibrec(n)
if n < 2 then return n
else return fibrec (n – 1) + fibrec (n – 2)
 The recurrence equation of above algorithm is given as,

 The recurrence can be re-written as,

 The characteristic polynomial is,

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 13
Homogeneous Recurrence – Example 1 : Fibonacci Series
  Find the roots of characteristic polynomial,
 

 The roots are, Here, and

and

  𝒌
 The general solution is therefore of the form, 𝒏
𝑻 𝒏= ∑ 𝒄 𝒓 𝒊 𝒊
𝒊=𝟏
 Substituting initial values and

 Solving these equations, we obtain

and
Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 14
Homogeneous Recurrence – Example 1 : Fibonacci Series
  Substituting the values of roots and constants in general solution,

𝒏
𝑻 𝒏∈ 𝑶( ∅ )
 

 Time taken for recursive Fibonacci algorithm grows Exponentially.

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 15
Example 2 : Tower of Hanoi
tower 1 tower 2 tower 3

tower 1 tower 2 tower 3

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 16
Example 2 : Tower of Hanoi
  The number of movements of a ring required in the tower of Hanoi problem is given by,

 The equation can be written as,

Inhomogeneous equation

 To convert it into a homogeneous equation, multiply with and replace by ,

 Solving equations (1) and (2), we have now

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 17
Example 2 : Tower of Hanoi
  The characteristic polynomial is, 𝒕  ( 𝒎 ) −𝟑 𝒕 ( 𝒎 −𝟏 ) +𝟐𝒕 (𝒎 −𝟐)=𝟎

 Whose roots are,

and
 The general solution is therefore of the form,

 Substituting initial values and

 Solving these linear equations we get and

  of Hanoi𝒎
 Therefore, time complexity of tower problem is 𝒎 as,
given
𝒕 𝒎 =𝟐 − 𝟏=𝑶 ( 𝟐 )
( )
Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 18
Homogeneous Recurrence Exercises
  Solve the following recurrences

1.

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 19
Master Theorem
  The master theorem is a cookbook method for solving recurrences.
Time to divide &
 Suppose you have a recurrence of the form recombine

Number of sub- Time required to

problems solve a sub-problem

 This recurrence would arise in the analysis of a recursive algorithm.

 When input size is large, the problem is divided up into sub-problems each of size . Sub-
problems are solved recursively and results are recombined.
 The work to split the problem into sub-problems and recombine the results is .

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 20
Master Theorem – Example 1
 
𝑇 (𝑛)=𝑎 𝑇 (𝑛/𝑏)+𝑓 (𝑛)
 

 There are three cases:

1.
2.

 Example 1:
 Here . So,=
 Also, =
 Case 2 applies:

#3150703 (ADA)  Unit 1 – Basics of Algorithms and

Dr. Gopi Sanghani 21
Master Theorem – Example 2
 
𝑇 (𝑛)=𝑎 𝑇 (𝑛/𝑏)+𝑓 (𝑛)
 

 There are three cases:

1.
2.

 Example 2:
 Here . So,

 Case 2 applies: the solution is

#3150703 (ADA)  Unit 1 – Basics of Algorithms and

Dr. Gopi Sanghani 22
Master Theorem – Example 3
 
𝑇 (𝑛)=𝑎 𝑇 (𝑛/𝑏)+𝑓 (𝑛)
 

 There are three cases:

1.
2.

 Example 3:
 Here . So, and

 So, is in
 Case 1 applies:

#3150703 (ADA)  Unit 1 – Basics of Algorithms and

Dr. Gopi Sanghani 23
Master Theorem Exercises
  Example 4:
 Example 5:
 Example 6: (Summer 17, Summer 19)
 Example 7: (Summer 17)
 Example 8: (Winter 18)
 Example 9: (Winter 19)

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 24
Recurrence Tree Method
  In recurrence tree, each node represents the cost of a single sub-problem in the set of
recursive function invocations.
 We sum the costs within each level of the tree to obtain a set of per level costs.
 Then we sum the all the per level costs to determine the total cost of all levels of the recursion.
 Here while solving recurrences, we divide the problem into sub-problems of equal size.
 E.g., where and is a given function.
 is the cost of splitting or combining the sub problems.

𝒏 

𝒏/𝒃
 
𝒏/𝒃
 

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 25
   Example 1:
Recurrence Tree Method
The recursion tree for this recurrence is  When we add the values across the levels of
the recursion tree, we get a value of for every
level.
𝑛
 
 The bottom level has nodes, each
contributing the cost

𝒍𝒐𝒈𝟐 𝒏
 
𝑛/2  
𝑛/2
  𝒏
 
 We have

2 2 2 2 𝒏
 
𝑛/2 𝑛/2 𝑛/2 𝑛/2
       

1
 
1
 
1
 
1
 

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 26
  Example 2:
Recurrence Tree Method
The recursion tree for this recurrence is  When we add the values across the levels of
the recursion tree, we get a value of for every
level.

𝑛
 

𝒍𝒐𝒈𝟑 𝒏
  𝑛/3
 
2𝑛/3   𝒏
 
𝒍𝒐𝒈𝟑/𝟐𝒏
 

1  𝑛 2  𝑛 1  2 𝑛 2  2 𝑛 𝒏
 
33 33 3 3 3 3

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 27
  Example 3:
Recurrence Tree Method
The recursion tree for this recurrence is  Sub-problem size at levelis
 Cost of problem at level Is
 Total cost,

2
𝑛
 

2 2
( 𝑛/2 )
 
( 𝑛/2 )
   
1/2 𝑛2

2 2 2 2
( 𝑛/4) ( 𝑛/4 ) ( 𝑛/4) ( 𝑛/4)
         
1/4 𝑛2

  ( 𝑛2 )
𝑂
Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 28
Recurrence Tree Method - Exercises
  Example 1:
 Example 2:
 Example 3:

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 29
Divide & Conquer (D&C)
Technique
Introduction
 Many useful algorithms are recursive in structure: to solve a given problem, they call
themselves recursively one or more times.
 These algorithms typically follow a divide-and-conquer approach:
 The divide-and-conquer approach involves three steps at each level of the recursion:
1. Divide: Break the problem into several sub problems that are similar to the original problem but smaller in
size.
2. Conquer: Solve the sub problems recursively. If the sub problem sizes are small enough, just solve the sub
problems in a straightforward manner.
3. Combine: Combine these solutions to create a solution to the original problem.

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 31
D&C Running Time Analysis
  The running-time analysis of such divide-and-conquer (D&C) algorithms is almost automatic.
 Let be the time required by D&C on instances of size .
 The total time taken by this divide-and-conquer algorithm is given by recurrence equation,
 

 The solution of equation is given as,

where is the power of in

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 32
Binary Search
Introduction
  Binary Search is an extremely well-known instance of divide-and-conquer approach.
 Let be an array of increasing sorted order; that is whenever .
 Let be some number. The problem consists of finding in the array if it is there.
 If is not in the array, then we want to find the position where it might be inserted.

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 34
Binary Search Example
 Input: sorted array of integer values.

1 3 7 9 11 32 52 74 90

Step 1:
1 2 3 4 5 6 7 8 9

1 3 7 9 11 32 52 74 90

Find approximate midpoint

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 35
Binary Search Example
Step 2:
1 2 3 4 5 6 7 8 9 𝒙=𝟕
 

1 3 7 9 11 32 52 74 90

    IsIs< =
midpoint
midpointvalue?
value?YES.
No.
Step 3:
1 2 3 4 5 6 7 8 9

1 3 7 9 11 32 52 74 90

Search for the target in the area before midpoint.

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 36
Binary Search Example
Step 4:
1 2 3 4 5 6 7 8 9 𝒙=𝟕
 

1 3 7 9 11 32 52 74 90

Find approximate midpoint

Step 5:
1 2 3 4 5 6 7 8 9

1 3 7 9 11 32 52 74 90

  > value of midpoint? YES.

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 37
Binary Search Example
Step 6:
1 2 3 4 5 6 7 8 9 𝒙=𝟕
 

1 3 7 9 11 32 52 74 90

  Search for the in the area after midpoint.

Step 7:
1 2 3 4 5 6 7 8 9

1 3 7 9 11 32 52 74 90

  Find approximate midpoint.

Is = midpoint value? YES.
Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 38
Binary Search – Iterative Algorithm
Algorithm: Function biniter(T[1,…,n], x)
𝑛=7
  𝑥=33
𝑥=7
  
if x > T[n] then return n+1
i ← 1; i 3
j ← n; 6
while i < j do
7
k ← (i + j ) ÷ 2
k 11
if x ≤ T [k] then j ← k
else i ← k + 1 32
return i 33
j 53

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 39
Binary Search – Recursive Algorithm
Algorithm: Function binsearch(T[1,…,n], x)
if n = 0 or x > T[n] then return n + 1
else return binrec(T[1,…,n], x)
Function binrec(T[i,…,j], x)
if i = j then return i
k ← (i + j) ÷ 2
if x ≤ T[k] then
return binrec(T[i,…,k],x)
else return binrec(T[k + 1,…,j], x)

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 40
Binary Search - Analysis
  Let be the time required for a call on binrecwhere is the number of elements still under
consideration in the search.
 The recurrence equation is given as,
𝑻  (𝒏)=𝒂𝑻 (𝒏/𝒃)+𝒇 (𝒏)
 Comparing this to the general template for divide and conquer algorithm,

 The complexity of binary search is

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 41
Binary Search – Examples
1.   Demonstrate binary search algorithm and find the element in the following array. [3 / 4]


2. Explain binary search algorithm and find the element in the following array. [7]

3. Let be a sorted array of distinct integers. Give an algorithm that can find an index such that
and provided such an index exists. Prove that your algorithm takes time in in the worst case.

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 42
Multiplying Large Integers
Multiplying Large Integers – Introduction
  Multiplying two 𝑛 digit large integers using divide and conquer method.
 Example: Multiplication of by .
1. Convert both the numbers into same length nos. and split each operand into two parts:

𝟎𝟗𝟖𝟏
 
𝟏𝟐𝟑𝟒
 

𝒘=𝟎𝟗  𝒙=𝟖𝟏
   
𝒚=𝟏𝟐  𝒛=𝟑𝟒
  𝟐   𝟐
2.   We can write as,
𝟎𝟗𝟖𝟏=𝟏𝟎 𝒘+𝒙 𝟏𝟐𝟑𝟒=𝟏𝟎 𝒚+𝒛
=

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 44
Multiplying Large Integers – Example 1
  Now, the required product can be computed as,
 

 The above procedure still needs four half-size multiplications:

 The computation 𝒓= (𝒘+

of can be𝒙done
) × (as,
𝒚 + 𝒛 ) ¿ 𝒘 ∙ 𝒚 + ( 𝒘 ∙ 𝒛 + 𝒙 ∙ 𝒚 )+ 𝒙 ∙ 𝒛

 Only one multiplication is required instead of two. Additional terms

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 45
Multiplying Large Integers – Example 1
 𝟏𝟎𝟒 𝒘 ∙ 𝒚 +𝟏𝟎𝟐 ( 𝒘 ∙ 𝒛+ 𝒙 ∙ 𝒚 ) + 𝒙 ∙ 𝒛  

 Now we can compute the required product as follows:

 𝒑=𝒘 ∙ 𝒚= 𝟎𝟗∙ 𝟏𝟐= 𝟏𝟎𝟖
 𝒒= 𝒙 ∙ 𝒛= 𝟖𝟏∙ 𝟑𝟒=𝟐𝟕𝟓𝟒

𝒓= ( 𝒘+ 𝒙 ) × ( 𝒚 + 𝒛 )=𝟗𝟎∙ 𝟒𝟔=𝟒𝟏𝟒𝟎
 

𝒓= ( 𝒘 + 𝒙 ) × ( 𝒚 + 𝒛 )=𝒘 ∙ 𝒚+ ( 𝒘 ∙ 𝒛 + 𝒙 ∙ 𝒚 ) + 𝒙 ∙ 𝒛
 

 981 ×1234=10 4 𝑝+10 2 (𝑟 − 𝑝 −𝑞)+𝑞

 ¿ 1080000 +127800 +2754
  ¿ 1210554.

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 46
Multiplying Large Integers – Analysis
  981 × 1234 can be reduced to three multiplications of two-figure numbers (09∙12, 81∙34 𝑎𝑛𝑑
90∙46) together with a certain number of shifts, additions and subtractions.
 Reducing four multiplications to three will enable us to cut 25% of the computing time required
for large multiplications.
 We obtain an algorithm that can multiply two 𝑛-figure numbers in a time,
𝑻(𝒏)= 𝟑𝒕 (𝒏/𝟐) + 𝒈(𝒏), 𝑻  (𝒏)=𝒂𝑻 (𝒏/𝒃)+𝒇 (𝒏)
 Solving it gives,

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 47
Multiplying Large Integers – Example 2
  Example: Multiply with using divide & conquer method.
 Solution using D&C

Step 1: 𝒘=𝟖𝟏
    𝒙=𝟏𝟒   𝒚=𝟕𝟔  𝒛=𝟐𝟐
Step 2:  Calculate and

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 48
Merge Sort
Introduction
 Merge Sort is an example of divide and conquer algorithm.
 It is based on the idea of breaking down a list into several sub-lists until each sub list consists
of a single element.
 Merging those sub lists in a manner that results into a sorted list.
 Procedure
 Divide the unsorted list into N sub lists, each containing 1 element
 Take adjacent pairs of two singleton lists and merge them to form a list of 2 elements. N will now convert
into N/2 lists of size 2
 Repeat the process till a single sorted list of all the elements is obtained

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 50
Merge Sort – Example

Unsorted Array
724 521 2 98 529 31 189 451
1 2 3 4 5 6 7 8

Step 1: Split the selected array

1 2 3 4 5 6 7 8
724 521 2 98 529 31 189 451

724 521 2 98 529 31 189 451

1 2 3 4 1 2 3 4

#3150703 (ADA)  Unit 1 – Basics of Algorithms and

Dr. Gopi Sanghani 51
Merge Sort – Example
Select the left subarray and Split Select the right subarray and Split
1 2 3 4 1 2 3 4
724 521 2 98 529 31 189 451

1 2 1 2 Split 1 2 1 2
724 521 2 98 529 31 189 451

1 1 1 1 1 1 1 1
724 521 2 98 529 31 189 451

521 724 2 98 31 529 189 451

Merge
2 98 521 724 31 189 451 529

2 31 98 189 451 521 529 724

#3150703 (ADA)  Unit 1 – Basics of Algorithms and
Dr. Gopi Sanghani 52
Merge Sort – Algorithm
Procedure: mergesort(T[1,…,n]) Procedure: merge(U[1,…,m+1],V[1,
…,n+1],T[1,…,m+n])
if n is sufficiently small then
insert(T) i ← 1;
else j ← 1;
array U[1,…,1+n/2],V[1,…,1+n/2] U[m+1], V[n+1] ← ∞;
U[1,…,n/2] ← T[1,…,n/2] for k ← 1 to m + n do
V[1,…,n/2] ← T[n/2+1,…,n] if U[i] < V[j]
mergesort(U[1,…,n/2]) then T[k] ← U[i];
mergesort(V[1,…,n/2]) i ← i + 1;
merge(U, V, T) else T[k] ← V[j];
j ← j + 1;

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 53
Merge Sort - Analysis
  Let be the time taken by this algorithm to sort an array of elements.
 Separating into takes linear time; also takes linear time.

where .
𝒕  ( 𝒏 ) =𝒍𝒕 ( 𝒏/𝒃 )+ 𝐠 ( 𝒏 )
 Applying the general case,

 Since the second case applies so,

  𝒌 𝒌
 Time complexity of merge sort is 𝜽 ( 𝒏 ) 𝒊𝒇 𝒍 <𝒃

{
𝒕 ( 𝒏 ) = 𝜽 ( 𝒏𝒌 𝒍𝒐𝒈𝒏 ) 𝒊𝒇 𝒍=𝒃 𝒌
𝜽 𝒏
( 𝒍𝒐𝒈 𝒍
)𝒃
𝒊𝒇 𝒍 >𝒃 𝒌

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 54
Quick Sort
Introduction
 Quick sort chooses the first element as a pivot element, a lower bound is the first index and an
upper bound is the last index.
 The array is then partitioned on either side of the pivot.
 Elements are moved so that, those greater than the pivot are shifted to its right whereas the
others are shifted to its left.
 Each Partition is internally sorted recursively.

Pivot
Element

0 1 2 3 4 5 6 7 8 9
42 23 74 11 65 58 94 36 99 87

LB UB

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 56
Quick Sort - Example
Procedure pivot(T[i,…,j]; var l) 0 1 2 3 4 5 6 7 8 9
p ← T[i] 42 23 74 11 65 58 94 36 99 87
k ← i; l ← j+1 k l
Repeat
k ← k+1 until T[k] > p or k ≥ j Swap
Repeat 42 23 36
74 11 65 58 94 74
36 99 87
l ← l-1 until T[l] ≤ p k l
While k < l do
Swap
Swap T[k] and T[l]
Repeat k ← k+1 until T[k] > p 11
42 23 36 42
11 65 58 94 74 99 87
Repeat l ← l-1 until T[l] ≤ p k l
Swap T[i] and T[l]
LB = 0, UB = 9
p = 42
k = 0, l = 10

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 57
Quick Sort - Example
Procedure pivot(T[i,…,j]; var l) 0 1 2 3 4 5 6 7 8 9
p ← T[i] 11 23 36 42 65 58 94 74 99 87
k ← i; l ← j+1 LB UB
Repeat
k ← k+1 until T[k] > p or k ≥ j 11 23 36
Repeat
k l
l ← l-1 until T[l] ≤ p
While k < l do LB UB
Swap T[k] and T[l] 11 23 36 42 65 58 94 74 99 87
Repeat k ← k+1 until T[k] > p
Repeat l ← l-1 until T[l] ≤ p
Swap T[i] and T[l]
23 36
k l

11 23 36 42 65 58 94 74 99 87

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 58
Quick Sort - Example
LB UB
Procedure pivot(T[i,…,j]; var l) 0 1 2 3 4 5 6 7 8 9
p ← T[i] 11 23 36 42 65 58 94 74 99 87
k ← i; l ← j+1
Swap
Repeat
k ← k+1 until T[k] > p or k ≥ j 65 65
58 58 94 74 99 87
Repeat
k l
l ← l-1 until T[l] ≤ p
While k < l do
Swap T[k] and T[l] 58 65 94 74 99 87
Repeat k ← k+1 until T[k] > p
Repeat l ← l-1 until T[l] ≤ p
Swap T[i] and T[l] LB UB
11 23 36 42 58 65 94 74 99 87

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 59
Quick Sort - Example
LB UB
Procedure pivot(T[i,…,j]; var l) Swap

p ← T[i] 94 74 99
87 87
99
k ← i; l ← j+1 k l
Repeat
Swap
k ← k+1 until T[k] > p or k ≥ j
Repeat
87
94 74 94
87 99
l ← l-1 until T[l] ≤ p k l
While k < l do LB UB
Swap T[k] and T[l] Swap
Repeat k ← k+1 until T[k] > p 74 87
87 74 94 99
Repeat l ← l-1 until T[l] ≤ p
Swap T[i] and T[l] k l

11 23 36 42 58 65 74 87 94 99

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 60
Quick Sort - Algorithm
Procedure: quicksort(T[i,…,j]) Procedure: pivot(T[i,…,j]; var l)
{Sorts subarray T[i,…,j] into p ← T[i]
ascending order}
k ← i
if j – i is sufficiently small
then insert (T[i,…,j]) l ← j + 1
else repeat k ← k+1 until T[k] > p or k ≥ j
pivot(T[i,…,j],l) repeat l ← l-1 until T[l] ≤ p
quicksort(T[i,…,l - 1]) while k < l do
quicksort(T[l+1,…,j] Swap T[k] and T[l]
Repeat k ← k+1 until T[k] > p
Repeat l ← l-1 until T[l] ≤ p
Swap T[i] and T[l]

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 61
Quick Sort Algorithm – Analysis
1.   Worst Case

 Running time depends on which element is chosen as key or pivot element.
 The worst case behavior for quick sort occurs when the array is partitioned into one sub-array with elements
and the other with element.
 In this case, the recurrence will be,

2. Best Case
 Occurs when partition produces sub-problems each of size n/2.
 Recurrence equation:

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 62
Quick Sort Algorithm – Analysis
3.   Average Case

 Average case running time is much closer to the best case.
 If suppose the partitioning algorithm produces a 9:1 proportional split the recurrence will be,

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 63
Quick Sort - Examples
 Sort the following array in ascending order using quick sort algorithm.
1. 5, 3, 8, 9, 1, 7, 0, 2, 6, 4
2. 3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5, 8, 9
3. 9, 7, 5, 11, 12, 2, 14, 3, 10, 6

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 64
Strassen’s Algorithm for Matrix
Multiplication
Matrix Multiplication
  Multiply following two matrices. Count how many scalar multiplications are required.

 To multiply matrices, total scalar multiplications are required.

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 66
Matrix Multiplication
  In general, and are two matrices to be multiplied.
and

 Computing each entry in the product takes multiplications and there are entries for a total of .

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 67
Strassen’s Algorithm for Matrix Multiplication
  Consider the problem of multiplying two matrices.
 Strassen’s devised a better method which has the same basic method as the multiplication of
long integers.
 The main idea is to save one multiplication on a small problem and then use recursion.

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 68
Strassen’s Algorithm for Matrix Multiplication
  and
Step 1 Step 2 Step 3
     

Final Answer:

Where,

All above
operations No multiplication is
involve only one required here.
multiplication.
Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 69
Strassen’s Algorithm - Analysis
  It is therefore possible to multiply two matrices using only seven scalar multiplications.
 Let be the time needed to multiply two matrices by recursive use of equations.

𝒕  ( 𝒏 ) =𝒍𝒕 ( 𝒏/𝒃 )+ 𝐠 ( 𝒏 )
Where
 The general equation applies with and .
 Since the third case applies and
 Since , it is possible to multiply two matrices in a time .
  𝒌 𝒌
𝜽 ( 𝒏 ) 𝒊𝒇 𝒍 <𝒃

Dr. Gopi Sanghani

{
𝒕 ( 𝒏 ) = 𝜽 ( 𝒏𝒌 𝒍𝒐𝒈𝒏 ) 𝒊𝒇 𝒍=𝒃 𝒌
𝜽 (𝒏
𝒍𝒐𝒈 𝒃 𝒍
) 𝒊𝒇 𝒍 >𝒃 𝒌
#3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 70
Exponentiation
Exponentiation - Sequential
  Let and be two integers. We wish to compute the exponentiation .
 Algorithm using Sequential Approach:

function exposeq(a, n)
r ← a
for i ← 1 to n - 1 do
r ← a * r
return r

 This algorithm takes a time in since the instruction is executed exactly times, provided the
multiplications are counted as elementary operations.

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 72
Exponentiation - Sequential
  But to handle larger operands, we must consider the time required for each multiplication.
 Let is the size of operand .
 Therefore, the multiplication performed the time round the loop concerns an integer of size
and an integer whose size is between and , which takes a time between
 
and

 
and and suppose
The body of loop executes time as,

here times multiplication is already done so

The size of in the 10th iteration will be between i.e., between
10-10+1 10

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 73
Exponentiation - Sequential
  The total time spent multiplying when computing an with exposeq is therefore,

 If we use the divide-and-conquer multiplication algorithm,

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 74
Exponentiation – D & C
  Suppose, we want to compute
 We can write as,

 In general,

 Algorithm using Divide & Conquer Approach:

function expoDC(a, n)
if n = 1 then return a
if n is even then return [expoDC(a, n/2)] 2
return a * expoDC(a, n - 1)
Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 75
Exponentiation – D & C
  0 𝑖𝑓 𝑛=1
Number of operations performed by
the algorithm is given by, {
𝑁 ( 𝑛 ) = 𝑁 ( 𝑛 / 2 )+1 𝑖𝑓 𝑛 𝑖𝑠 𝑒𝑣𝑒𝑛
𝑁 ( 𝑛 −1 ) +1 𝑜𝑡h𝑒𝑟𝑤𝑖𝑠𝑒
 

Time taken by the

algorithm is given by, Solving it gives,

function expoDC(a, n)
if n = 1 then return a
if n is even then return [expoDC(a, n/2)] 2
return a * expoDC(a, n - 1)
Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 76
Exponentiation – Summary

Multiplication
Classic D&C
exposeq
expoDC

Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 77
Thank You