Divide-and-Conquer

Dr. Bibhudatta Sahoo

Communication & Computing Group
Department of CSE, NIT Rourkela
Email: [email protected], 9937324437, 2462358
 Divide and Conquer
• Divide and conquer was a successful military strategy long
before it became an algorithm design paradigm.
• The wise general would attack so as to divide the enemy army
into two forces and then mop up one after the other.
• To use divide and conquer as an algorithm design technique,
we must divide the problem into two smaller sub-problems,
solve each of them recursively, and then meld the two partial
solutions into one solution to the full problem.
• Whenever the merging takes less time than solving the two
sub-problems, we get an efficient algorithm

Divide-and-Conquer 2
 Divide-and-Conquer
• Divide-and conquer is a general
algorithm design paradigm:
– Divide: divide the input data S
in two or more disjoint subsets
S1, S2, …
– Recur: solve the sub-problems
recursively
– Conquer: combine the
solutions for S1, S2, …, into a
solution for S
• The base case for the recursion are
sub-problems of constant size
• Analysis can be done using
recurrence equations

Divide-and-Conquer 3
 Divide and Conquer – 3 steps
• First step is called Divide which is dividing the given
problems into smaller sub problems which are
identical to the original problem and also these sub
problems may or may not be of same size.
• Second step is called Conquer where we solve these
sub problems recursively.
• Third step is called Combine where we combine
solutions of the sub problems to get solution for the
original problem

Divide-and-Conquer 4
Divide and Conquer – 3 steps

Divide-and-Conquer 5
 Divide and Conquer – 3 steps

• Divide/Break :This step involves breaking the problem into smaller

sub-problems. Sub-problems should represent a part of the original
problem. This step generally takes a recursive approach to divide
the problem until no sub-problem is further divisible. At this stage,
sub-problems become atomic in nature but still represent some
part of the actual problem.
• Conquer/Solve: This step receives a lot of smaller sub-problems to
be solved. Generally, at this level, the problems are considered
'solved' on their own.
• Merge/Combine: When the smaller sub-problems are solved, this
stage recursively combines them until they formulate a solution of
the original problem. This algorithmic approach works recursively
and conquer & merge steps works so close that they appear as one.

Divide-and-Conquer 6
Fundamental of Divide & Conquer Strategy
1. Relational Formula: It is the formula that we
generate from the given technique. After generation of
Formula we apply D&C Strategy, i.e. we break the
problem recursively & solve the broken sub-problems.

2. Stopping Condition: When we break the problem

using Divide & Conquer Strategy, then we need to know
that for how much time, we need to apply divide &
Conquer. So the condition where the need to stop our
recursion steps of D&C is called as Stopping Condition.

Divide-and-Conquer 7
 Points to be taken care of
• Threshold lower limit, below which the problem
can’t be subdivided or where you want to handle the
problem without further divisions.
• The size of the sub instances into which an instance
is split.
• Number of such Instances The Algorithm used to
combine sub-solutions.

Divide-and-Conquer 8
 Control abstraction Divide & Conquer
1. Algorithm DAndC(P)
2. {
3. If Small(P) then return S(P);
4. else
5. {
6. divide P into smaller instances , P1 , P2 ,…, Pk , k ≥ 1
7. Apply DAndC to each of these sub-problems;
8. Return Combine (DAndC(P1 ), DAndC(P2) ,…, DAndC(Pk))
9. }
10. }

Divide-and-Conquer 9
Divide and Conquer

Divide-and-Conquer 10
 Why Divide and Conquer?
• Sometimes it’s the simplest approach
• Divide and Conquer is often more efficient than
“obvious” approaches
– E.g. Mergesort, Quicksort
• But, not necessarily efficient
– Might be the same or worse than another approach
• Must analyze cost
• Note: divide and conquer may or may not be
implemented recursively

Divide-and-Conquer 11
 Suitability of D&C
• Divide and conquer is not suitable where the
solution of size n depends upon n sub-solutions,
each of size (n-1).
• Overlapping Sub problems - Where sub problems
have a dependency on each other and there is no
easy way to merge those sub problems and merging
will take significant time as solving the original
problem.

Divide-and-Conquer 12
 Examples
• Mergesort is the classic example of a divide-and-
conquer algorithm.
• It takes only linear time to merge two sorted listsof
n/2 elements each of which was obtained in O(n log
n) time.
• Divide and conquer is a design technique with many
important algorithms to its credit, including
mergesort, the fast Fourier transform, and
Strassen’s matrix multiplication algorithm.

Divide-and-Conquer 13
 Merge-Sort
• Merge-sort on an input Algorithm mergeSort(S, C)
sequence S with n
elements consists of Input sequence S with n
three steps: elements, comparator C
– Divide: partition S into Output sequence S sorted
two sequences S1 and S2
according to C
of about n/2 elements
each if S.size() > 1
– Recur: recursively sort S1
(S1, S2)  partition(S, n/2)
and S2
– Conquer: merge S1 and S2 mergeSort(S1, C)
into a unique sorted mergeSort(S2, C)
sequence
S  merge(S1, S2)

Divide-and-Conquer 14
 Cost for a Divide and Conquer Algorithm

• Perhaps there is…

– A cost for dividing into sub problems
– A cost for solving each of several subproblems
– A cost to combine results
• So (for n > smallSize)
T(n) = D(n) + Sum[ T(size(Ii) ] + C(n)
– often rewritten as
T(n) = b T(n/c) + f(n)

• These formulas are recurrence relations

Divide-and-Conquer 15
 The concept: Recurrence relations
• Recurrence relations are recursive definitions of
mathematical functions or sequences.
• For example, the recurrence relation
g(n) = g(n-1) + 2n -1
g(0) = 0
defines the function g(n) = n^2
The recurrence relation
f(n) = f(n-1) + f(n-2)
f(1) = 1, f(0) = 1
defines the famous Fibonacci sequence 1,1,2,3,5,8,13,....

Divide-and-Conquer 16
 What is a Recurrence Relation?
• A recurrence is an equation or inequality that
describes a function in terms of its value on smaller
inputs.
• A recurrence relation for T(N) is simply a recursive
definition of T(N).
– This means T(N) is written as a function of T(k)
– where k < N.
• As exemplified by the two prior examples, two
common types are:
– T(N) = a T(N-1) + b
– T(N) = a T(N/2) + bN + c

Divide-and-Conquer 17
 Example Type 1: Divide-and-conquer recurrence

• Assume a divide-and-conquer algorithm divides a problem of

size n into a sub-problems.
• Assume each sub-problem is of size n/b.
• Assume f(n) extra operations are required to combine the
solutions of sub-problems into a solution of the original
problem.
• Let T(n) be the number of operations required to solve the
problem of size n.
T(n) = a T(n/b) + f(n)
• In order to make the recurrence well defined T(n/b) term will
actually be either T(n/b) or T( n/b ).
• The recurrence will also have to have initial conditions. (e.g.
T(1) or T(0))
Divide-and-Conquer 18
 Example: Recurrence Equations
• Merge sort: To sort an array of size n, we sort the left
half, sort the right half, and then merge the two
halves.
We can do the merge in linear time (i.e., cn for some
positive constant c).
So, if T(n) denotes the running time on an input of
size n, we end up with the recurrence
T(n) = 2T(n/2) + cn.
This can also be expressed as T(n) = 2T(n/2) + O(n).

Divide-and-Conquer 19
 Example: Recurrence Equations
• Selection Sort: In selection sort, we find the smallest element
in the input sequence and swap with the leftmost element
and then recursively sort the remainder (less the leftmost
element) of the sequence.
This leads to the recurrence T(n) = cn + T(n−1).

• Polynomial Multiplication: The straightforward divide-and-

conquer algorithm to multiply two polynomials of degree n
leads to T(n) = 4T(n/2)+ cn.
However, a clever rearrange of the terms improves this to
T(n) = 3T(n/2)+cn.

Divide-and-Conquer 20
 Example Type 2: Recurrence Equations
Linear recurrence relations:
T(n) = T(n-1) + n for n>0 and T(0) = 1
• These types of recurrence relations can be easily solved
using substitution method.

T(n) = T(n-1) + n
= T(n-2) + (n-1) + n
= T(n-k) + (n-(k-1))….. (n-1) + n
Substituting k = n, we get
T(n) = T(0) + 1 + 2+….. +n = n(n+1)/2 = O(n2)

Divide-and-Conquer 21
 Example Type 3: Value substitution before solving
• Sometimes, recurrence relations can’t be directly solved using
techniques like substitution, recurrence tree or master
method.
• Therefore, we need to convert the recurrence relation into
appropriate form before solving.
Example: T(n) = T(√n) + 1
To solve this type of recurrence, substitute n = 2m as:
T(2m) = T(2m /2) + 1
Let T(2m) = S(m), S(m) = S(m/2) + 1
Solving by master method, we get
S(m) = Θ(logm) As n = 2m or m = log2(n),
T(n) = T(2m) = S(m) = Θ(logm) = Θ(loglogn)

Divide-and-Conquer 22
 Recurrence Relation: Merge sort
1. Algorithm MergeSort(low, high)
2. / / a[low : high] is a global array to be sorted.
3. / / Small(P) is true if there is only one element
4. / / to sort. In this case the list is already sorted.
5. {
6. if (low < high) then / / If there are more than one element
7. {
8. / / Divide P into subproblems.
9. / / Find where to split the set.
10. mid:=  (low + high)/2;
11. / / Solve the subproblems.
12. MergeSort(low, mid);
13. MergeSort(mid + 1, high);  b if n  2
14. / / Combine the solutions.
T (n )  
15. Merge(low, mid, high); 2T (n / 2)  bn if n  2
16. }
17. }

Divide-and-Conquer 23
 Recurrence Equation Analysis
• The conquer step of merge-sort consists of merging two
sorted sequences, each with n/2 elements and implemented
by means of a doubly linked list, takes at most bn steps, for
some constant b.
• Likewise, the basis case (n < 2) will take at b most steps.
• Therefore, if we let T(n) denote the running time of merge-
sort:

 b if n  2
T (n )  
2T (n / 2)  bn if n  2
• We can therefore analyze the running time of merge-sort by
finding a closed form solution to the above equation.
– That is, a solution that has T(n) only on the left-hand side.

Divide-and-Conquer 24
 Methods for solving Recurrence Relation
1. Substitution method
2. Telescopic Method
3. Recursion Tree
4. Guess & Test
5. Master’s Theorem

• Time complexity of DAndC algorithm can be found

out by solving the recurrence relation that
represents the algorithm.

Divide-and-Conquer 25
 1: Iterative Substitution; example 001
• In the iterative substitution, or “plug-and-chug,” technique, we iteratively
apply the recurrence equation to itself and see if we can find a pattern:
T ( n )  2T ( n / 2)  bn
 2( 2T ( n / 2 2 ))  b( n / 2))  bn
 2 2 T ( n / 2 2 )  2bn
 23 T ( n / 23 )  3bn
 2 4 T ( n / 2 4 )  4bn
 ...
 2i T ( n / 2i )  ibn

• Note that base, T(n)=b, case occurs when 2i=n. That is, i = log n.
• So, T (n)  bn  bn log n
• Thus, T(n) is O(n log n).

Divide-and-Conquer 26
 1: Iterative Substitution; example 002
T ( 2)  1 (assume N  2 ) k

T ( N )  2T ( N / 2)  2

T ( N )  2[ 2T ( N / 4)  2]  2
T ( N )  2 T ( N / 2 )  ( 2  2)
2 2 2

r 1
T (N )  2 T (N / 2 )   2
r r i

i 1

Divide-and-Conquer 27
 1: Iterative Substitution; example 002
k 2
k 1
T (N )  2 T (N / 2 k 1
)  2 2 i

i 0
k 1 k 1
T (N )  2  2( 2  1)
N N
T ( N )   2(  1) N

2 2 Note :  2
i 0
i
 2 N 1
1

3N
T (N )  2
2
Note, although this is still O(N), it is better than the
T(N)=2N found earlier.
Divide-and-Conquer 28
 Exercises
Solve the recurrence relations with T(1) = 1, N = 2m
(a) T(N) = T(N/2) + 1
(b) T(N) = T(N/2) + N
(c) T(N) = 2T(N/2) + 1
(d) T(N) = 2T(N – 1) + 1
(e) T(N) = 2T(N – 1) + N

Divide-and-Conquer 29
 2. Telescopic Sum
• Re-write the recurrence relation for subsequent
smaller values of n so that the left side of the re-
written relation is equal to the first term of the right
side of the previous relation.
• Re-write the relation until a relation involving the
initial condition is obtained.
• Add the left sides and the right sides of the relations.
• Cancel the equal terms on the left and the right side.
• Add the remaining terms on the right side. The sum
will give the general formula of the sequence

Divide-and-Conquer 30
 2. Telescopic Sum; example 001
• Given recurrence relation: • Adding the left and the right
T(1) = 1 ; N=1 side and canceling equal
T(N) = T(N-1) + 1; N >1 terms, we obtain:
• T(N) = 1 + 1 + … + 1
Telescoping: • There are N lines in the
T(N) = T(N-1) + 1 telescoping, thus the sum of
T(N-1) = T(N-2) + 1 the 1s is N:
T(N-2) = T(N-3) + 1 • T(N) = N
…. • Hence T(N) = O( N )
T(3) = T(2) +1
T(2) = T(1) + 1
T(1) = 1

Divide-and-Conquer 31
 2. Telescopic Sum; example 002
• Merge sort: Telescoping:
T(1) = 1 ; N=1 T(N) / N = T(N/2) / (N/2) + 1
T(N) = 2T(N/2) + N; N > 1 T(N/2) / (N/2) = T(N/4) / (N/4) + 1
Preprocessing – T(N/4) / (N/4) = T(N/8) / (N/8) + 1
• divide both sides by N …
T(N) / N = T(N/2) / (N/2) + 1 T(8)/8 = T(4)/4 + 1
N = 2m T(4)/4 = T(2)/2 + 1
T(2)/2 = T(1) + 1
• Adding the left and the right
side and canceling equal terms,
we obtain:
• T(N)/N = T(1) + 1 + … + 1
Divide-and-Conquer 32
 2. Telescopic Sum; example 002
• T(N)/N = T(1) + 1 + …. + 1 • T(N) = O(NlogN )
• There are logN equations,
thus
1 + … + 1 = logN
T(N)/N = T(1) + logN
T(N)/N = LogN + 1
because T(1) = 1
T(N) = NlogN + N
Hence T(N) = O(NlogN )

Divide-and-Conquer 33
 2. Telescopic Sum; example 003
T (n) = T (n-1) +1 and T (1) = θ (1).
Solution:
T (n) = T (n-1) +1
= (T (n-2) +1) +1
= (T (n-3) +1) +1+1
= T (n-4) +4
= T (n-5) +1+4
= T (n-5) +5, … = T (n-k) + k ; Where k = n-1
T (n-k) = T (1) = θ (1)
T (n) = θ (1) + (n-1) = 1+n-1=n= θ (n).

Divide-and-Conquer 34
 2. Telescopic Sum; example 004
T (n) = 1 if n=1
T (n) = 2T (n-1) if n>1
T (n) = 2T (n-1)
= 2[2T (n-2)]
= 22T (n-2) = 4[2T (n-3)]
= 23T (n-3)
= 8[2T (n-4)] = 24T (n-4)] ................. EQ.1
Repeat the procedure for i times T (n) = 2i T (n-i)
Putting n-i=1 or i= n-1 in (Eq.1) 
T (n) = 2n-1 T (1) = 2n-1 .1 {T (1) =1 .....given} = 2n-1

Divide-and-Conquer 35
 3: The Recursion Tree
• A recursion tree is a tree whose each node represents the cost
of a certain recursive sub-problem. In order to get the cost of
the entire algorithm, we need to sum up the costs in each
node.
• One of the strengths of the recursion tree is that it is useful to
visualize what happens when a recurrence is iterated.
• Its diagram shows the structure of recursive calls, which
means that how the main-problem divided into sub-problems,
and the costs for combining those sub-tasks.
• The recursion tree of the merge sort with its recurrence relation T(n) =2T(n/2)+n

Divide-and-Conquer 36
Complete recursion tree of the merge sort: T(n) = 2T(n/2) + n

Divide-and-Conquer 37
Explaining the proof that T(n) = 𝚶(𝒏log𝒏)

Divide-and-Conquer 38
 The Recursion Tree
(1) The depth of the recursion tree is log base b of n when the
recurrence is in the form of T(n) = aT(n/b)+f(n)
(2) The depth of the recursion tree means the length of the
longest branch among its branches.

Divide-and-Conquer 39
 3: The Recursion Tree

• Draw the recursion tree for the recurrence relation and look for a
pattern:
 b if n  2
T (n )  
2T (n / 2)  bn if n  2
time
depth T’s size
0 1 n bn

1 2 n/2 bn

i 2i n/2i bn

… … … …

Total time = bn + bn log n

(last level plus all previous levels)
Divide-and-Conquer 40
 3: The Recursion Tree
• A recursion tree is useful for visualizing what happens when a
recurrence is iterated. It diagrams the tree of recursive calls
and the amount of work done at each call.
• Consider the recurrence : T(n) = 2T(n/2) + n2.

Divide-and-Conquer 41
 3: The Recursion Tree
• sum across each row of the tree to obtain the total work done
at a given level:

This a geometric series, thus in the limit the sum is O(n2). The depth of the tree
in this case does not really matter; the amount of work at each level is
decreasing so quickly that the total is only a constant factor more than the root.

Divide-and-Conquer 42
 3: The Recursion Tree
Considering T(n) = T(n/3) + T(2n/3) + n, expanding out the first
few levels, the recurrence tree is:

the tree here is not balanced: the longest path is the rightmost one, and its
length is log3/2 n. Hence our guess for the closed form of this recurrence
is O(n log n).

Divide-and-Conquer 43
Divide-and-Conquer 44
 4: Guess-and-Test Method
• In the guess-and-test method, we guess a closed form
solution and then try to prove it is true by induction:
 b if n  2
T (n)  
2T (n / 2)  bn log n if n  2
• Guess: T(n) < cn log n.
T ( n )  2T ( n / 2)  bn log n
 2( c ( n / 2 ) log( n / 2))  bn log n
 cn(log n  log 2)  bn log n
 cn log n  cn  bn log n

• Wrong: we cannot make this last line be less than cn log n

Divide-and-Conquer 45
 4: Guess-and-Test Method, Part 2
• Recall the recurrence equation:
 b if n  2
T (n)  
2T (n / 2)  bn log n if n  2
• Guess #2: T(n) < cn log2 n.
T (n)  2T (n / 2)  bn log n
 2(c( n / 2) log 2 (n / 2))  bn log n
 cn(log n  log 2) 2  bn log n
 cn log 2 n  2cn log n  cn  bn log n

– if c > b.
 cn log 2 n
• So, T(n) is O(n log2 n).
• In general, to use this method, you need to have a good
guess and you need to be good at induction proofs.
Divide-and-Conquer 46
 5: Master Method
• Many divide-and-conquer recurrence equations have the
form:

 c if n  d
T (n)  
aT (n / b)  f (n ) if n  d

• The Master Theorem:

1. if f (n) is O(n logb a  ), then T (n) is (n logb a )

2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.

Divide-and-Conquer 47
 5: The Master Theorem
• The solution to the recurrence T(n) = aT(n/b)+ Θ(nk); T(1)=
Θ(1), where a, b, and k are all constants, is given by:

T(n) = Θ(nk) if a < bk

T(n) = Θ(nk log n) if a = bk
T(n) = Θ(n logb a) if a > bk

• The theorem just given is a simplified version. The general

version does not restrict the on recursive term to be a
polynomial in terms of n.

Divide-and-Conquer 48
 Master Method, Example 1
• The form:  c if n  d
T (n)  
aT (n / b)  f (n ) if n  d
• The Master Theorem:
1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.
• Example:

T (n)  4T (n / 2)  n
Solution: logba=2, so case 1 says T(n) is O(n2).

Divide-and-Conquer 49
 Master Method, Example 2
• The form:  c if n  d
T (n)  
aT (n / b)  f (n ) if n  d
• The Master Theorem:
1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.

• Example: T (n)  2T (n / 2)  n log n

Solution: logba=1, so case 2 says T(n) is O(n log2 n).

Divide-and-Conquer 50
 Master Method, Example 3
• The form:  c if n  d
T (n)  
aT (n / b)  f (n ) if n  d
• The Master Theorem:
1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.
• Example:
T (n)  T (n / 3)  n log n
Solution: logba=0, so case 3 says T(n) is O(n log n).

Divide-and-Conquer 51
 Master Method, Example 4
• The form:  c if n  d
T (n)  
aT (n / b)  f (n ) if n  d
• The Master Theorem:
1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.
• Example:
T (n)  8T (n / 2)  n 2

Solution: logba=3, so case 1 says T(n) is O(n3).

Divide-and-Conquer 52
 Master Method, Example 5
• The form:  c if n  d
T (n)  
aT (n / b)  f (n ) if n  d
• The Master Theorem:
1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.
• Example:
T (n)  9T (n / 3)  n 3

Solution: logba=2, so case 3 says T(n) is O(n3).

Divide-and-Conquer 53
 Master Method, Example 6
 c if n  d
• The form: T (n)  
aT (n / b)  f (n ) if n  d

• The Master Theorem:

1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.
• Example:
T (n)  T (n / 2)  1 (binary search)

Solution: logba=0, so case 2 says T(n) is O(log n).

Divide-and-Conquer 54
 Master Method, Example 7
• The form:  c if n  d
T (n)  
aT (n / b)  f (n ) if n  d
• The Master Theorem:
1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.
• Example:
T (n)  2T (n / 2)  log n (heap construction)

Solution: logba=1, so case 1 says T(n) is O(n).

Divide-and-Conquer 55
 Iterative “Proof” of the Master Theorem
• Using iterative substitution, let us see if we can find a pattern:
T ( n)  aT ( n / b)  f ( n)
 a ( aT ( n / b 2 ))  f ( n / b))  bn
 a 2T ( n / b 2 )  af ( n / b)  f ( n)
 a 3T ( n / b 3 )  a 2 f ( n / b 2 )  af ( n / b)  f ( n)
 ...
(logb n ) 1
a logb n
T (1)   a
i 0
f ( ni
/ b i
)
(logb n ) 1
 n logb aT (1)  a
i 0
i
f (n / b i )
• We then distinguish the three cases as
– The first term is dominant
– Each part of the summation is equally dominant
– The summation is a geometric series

Divide-and-Conquer 56
Proof of Masters Theorem
Proof of Masters Theorem

Divide-and-Conquer 58
Proof of Masters Theorem

Divide-and-Conquer 59
Proof of Masters Theorem

Divide-and-Conquer 60
Proof of Masters Theorem

Divide-and-Conquer 61
Suggested Exercise

Divide-and-Conquer 62
Solving recurrence relation

Substitution
method

Divide-and-Conquer 63
Application of masters theorem

Divide-and-Conquer 64
Application of masters theorem

Divide-and-Conquer 65
Application of masters theorem

Divide-and-Conquer 66
Application of masters theorem

Divide-and-Conquer 67
Application of masters theorem

Divide-and-Conquer 68
Application of masters theorem

Divide-and-Conquer 69
Application of masters theorem

Divide-and-Conquer 70
Application of masters theorem

Divide-and-Conquer 71
Application of masters theorem

Divide-and-Conquer 72
 Summary: Master theorem
• Master theorem lets you go from the recurrence to the
asymptotic bound very quickly, so you’re more like a pro.
• It typically works well for divide-and-conquer algorithms
• But master theorem does not apply to all recurrences. When
it does not apply, you can:

do some upper/lower bounding and get a potentially

looser bound
use the substitution method

Divide-and-Conquer 73
 Exercise

Divide-and-Conquer 74
 Example: Iterative Factorial Algorithm
int factorial (N)
{
temp = 1; Time Units to Compute
-------------------------------
for i = 1 to N { N loops.
temp = c for the multiplication.
temp*i;
}
N
return(temp);
} T ( N )   c  cN
i 1

Thus this requires O(N) computation.

Divide-and-Conquer 75
 Example: Recursive Factorial Algorithm

Time Units to Compute

int factorial (N) -------------------------------
{ c for the conditional
if (N == 1) d for the multiplication and then T(N-1).

return 1;
else return(N * T (1)  c
factorial(N-1));
}
T ( N )  c  d  T ( N  1)

T(N) = c+d+T(N-1) = f+T(N-1) is a recurrence relation.

How do we solve it?

Divide-and-Conquer 76
 Example : Search Algorithm

int a[N]; // An array

int value; // Value to be found
Time Units to Compute
int search ( ) {
---------------------------
for (i = 0; i < N; i++) {
----
if (a[i] == value)
N loops
return(i);
c for comparison
}
return(-1);
}
N 1
T ( N )   c  cN
Thus this is an O(N) algorithm. i 0

Divide-and-Conquer 77
 Example : Binary Search Algorithm
int a[N]; // Sorted array
int x; // Value to be found
Time Units to Compute
int bin_search (int left, int right) { -------------------------------
if (left > right) return –1; c for comparison.

int mid = (left+right) / 2; d for computation of mid.

if (x < a[mid]) c for comparison.
bin_search(left, mid-1); T(N/2)
else if (x > a[mid]) c for comparison.
bin_search(mid+1,right); T(N/2)
else return mid;
}

Thus T(N) = T(N/2) + 3c+d = T(N/2)+f

Divide-and-Conquer 78
 Analysis

T (1)  c (assume N  2 ) k

T(N)  T(N/ 2 )  f

T (N )  T (N / 4 )  2 f
T (N )  T (N / 2 )  kf  c  kf
k

T ( N )  c  k log N  O (log N )

Divide-and-Conquer 79
 Multiplying Two n Bit Numbers
• The naive approach is to simply multiply the two numbers
which takes O(n2) time because we need to multiply each digit
in one number with those in the second number, put them in
the correct position and add them
• The divide-and-conquer technique to multiply two n bit
number is attributed to a 1962 paper by Karatsuba, and
indeed it is sometimes called Karatusba multiplication.
• With divide-and-conquer multiplication, we split each of the
numbers into two halves, each with n/2 digits.
• Let us call the two numbers we're trying to multiply a and b,
with the two halves of a being aL (the left or upper half)
and aR (the right or lower half) and the two halves
of b being bL and bR.

Divide-and-Conquer 80
 Multiplying Two n Bit Numbers

In order to multiply a pair of n-digit numbers, we can recursively

multiply four pairs of n/2-digit numbers. The rest of the operations
involved are all O(n) operations

Divide-and-Conquer 81
 Multiplying Two n Bit Numbers
• If we take a look at the above formula, there are four
multiplications of size n/2, so we basically divided the
problem of size n into four sub-problems of size n/2. But that
doesn’t help because solution of recurrence T(n) = 4T(n/2) +
O(n) is O(n2).
• Suppose we are given a = 123456 and b = 654321. We can
rewrite a as 123000+456 and b as 654000+321.
• As a result, a b = 123 654 106 + (123 321 + 456 654) 103 +
456 321.

Divide-and-Conquer 82
Multiplying Two n Bit Numbers

T(n) = 3T(n/2) + cn

Divide-and-Conquer 83
The divide-and-conquer algorithm in action

Divide-and-Conquer 84
Multiplying Two n Bit Numbers

Divide-and-Conquer 85
 Multiplying Two n Bit Numbers
• How long does it take to multiply two n-bit numbers? Grade
school algorithm: Runtime O(n2)
• Recursive algorithm: How to solve in terms of sub-problem
solutions?
(a 2n/2 + b)(c 2n/2 + d) = ac 2n + (ad + bc)2n/2 + bd ------(1)
T(n) = 4T(n/2) + c n = O(n2)
Improvement: substitute (a+b)(c+d) = ac + ad + bc + bd
in equation (1)
 ac 2n + (ad + bc)2n/2 + bd
= ac 2n + (ac + ad + bc + bd )2n/2 + bd
T(n) = 3T(n/2) + c n = O(nlg 3)

Divide-and-Conquer 86
 Infinite Wall Problem
• You have an infinite wall on both sides where you are
standing and it has a gate somewhere in one
direction, you have to find out the gate,

• No design-Infinite time
• Incremental design- you go one step in one direction , come
back go to other direction one step, come back and then go 2
steps in other direction.

Divide-and-Conquer 87
 Infinite Wall Problem: Quadratic time solution
• 1+2.1+1
• 2+2.2+2
• 3+2.3+3
• :::
• n-1+2.(n-1)+n-1
• n+2n
• 4∑i+3n
• 4n(n-1)/2+3n=2n2+n

Divide-and-Conquer 88
 Infinite Wall Problem: Linear time solution
• You go 20 step in one direction, come back go 1 step
in other then 21direction in one way then come back
and then up to 2k
• 20+2.20+20
• 21+2. 21+21
• …
• 2K-1+2.2K-1+2K-1
• 3.2K
• 4(2K-1+2K-2+……1)+ 3.2K
• 4(2K-1)+ 3.2K = 7.2K -4 = 7N-4

Divide-and-Conquer 89
 Questions, Comments and Suggestions
• Divide step is the dominating operation and Combine
step is the dominant operation respectively in
following
• A) Merge Sort, Quick Sort
• B) Quick Sort, Merge Sort
• C) Bubble Sort, Counting Sort
• D) Radix Sort, Selection Sort

Divide-and-Conquer 90
Advantages of Divide and Conquer
• Divide and Conquer tend to successfully solve one of the biggest
problems, such as the Tower of Hanoi, a mathematical puzzle. It is
challenging to solve complicated problems for which you have no basic
idea, but with the help of the divide and conquer approach, it has
lessened the effort as it works on dividing the main problem into two
halves and then solve them recursively. This algorithm is much faster than
other algorithms.
• It efficiently uses cache memory without occupying much space because
it solves simple sub-problems within the cache memory instead of
accessing the slower main memory.
• It is more proficient than that of its counterpart Brute Force technique.
• Divide & Conquer algorithms are adapted for execution in multi-processor
machines.
• Since these algorithms inhibit parallelism, it does not involve any
modification and is handled by systems incorporating parallel processing.

Divide-and-Conquer 91
 Multi-processor machines.
• A Multiprocessor is a computer system with two or more
central processing units (CPUs) share full access to a common
RAM.
• The main objective of using a multiprocessor is to boost the
system's execution speed, with other objectives being fault
tolerance and application matching

Divide-and-Conquer 92
 Multi-core processor
• A multi-core processor is a computer processor on a single
integrated circuit with two or more separate processing units,
called cores, each of which reads and executes program
instructions.

Divide-and-Conquer 93
 Multi-threading
• In computer architecture, multithreading is the ability of a
central processing unit (CPU) (or a single core in a multi-core
processor) to provide multiple threads of execution
concurrently, supported by the operating system.
• Multi-threading is way to improve parallelism by running the
threads simultaneously in different cores of your processor.
• When multiple parallel tasks are executed by a processor, it is
known as multitasking.
• A CPU scheduler, handles the tasks that execute in parallel.
• The CPU implements tasks using operating system threads. So
that tasks can execute independently but have some data
transfer between them, such as data transfer between a data
acquisition module and controller for the system. Data
transfer occurs when there is a data dependency.
Divide-and-Conquer 94
Conventional parallel merge sort with 8 processors.

Divide-and-Conquer 95
Conventional parallel merge sort with 8 processors.

Divide-and-Conquer 96
Multicore Programming

Professional Multicore Programming:

Design and Implementation
for C++ Developers by Hughes & Hughes,
Wrox Publication

Divide-and-Conquer 97
 Disadvantages of Divide and Conquer
• Recursion is slow. Since most of its algorithms are
designed by incorporating recursion, so it
necessitates high memory management.
• Very simple problem may be more complicated than
an iterative approach. Example: adding n numbers
etc.
• An explicit stack may overuse the space.
• It may even crash the system if the recursion is
performed rigorously greater than the stack present
in the CPU.

Divide-and-Conquer 98
 Conclusions
• Divide and conquer means that when you are faced
with a large problem, you recursively divide it into
smaller versions of the same problem. When you
break up a problem in this way, the complexity often
decreases exponentially, leading to O(n log n) instead
of O(n) time complexity.
• Divide and conquer is just one of several powerful
techniques for algorithm design.
• Divide-and-conquer algorithms can be analyzed using
recurrences and the master method.
• Can lead to more efficient algorithms.
Divide-and-Conquer 99
 Divide and Conquer Algorithms
Divide and conquer is a way to break complex problems into smaller
problems that are easier to solve, and then combine the answers to
solve the original problem.

Divide-and-Conquer 100
Thanks for Your Attention!

Divide-and-Conquer 101
 Home work
1. Given a sorted array of distinct integers A[1…n], you want to
find out whether there is an index i for which A[i] = i. Given a
divide-and-conquer algorithm that runs in time O(logn).
2. Why Selection Sort can’t be converted to Divide and
Conquer Mechanism ?
3. What are the deciding factors in determining the
termination of further divisions of the problem into sub
problems. (Give two at least) .
4. You are given two pan fair balance. You have 12 identically
looking coins out of which one coin may be lighter or
heavier. How can you find odd coin, if any, in minimum trials,
also determine whether defective coin is lighter or heavier,
in the worst case?

Divide-and-Conquer 102
 Home work
5. Finding the Non-Duplicate: You are given a sorted array of
numbers where every value except one appears exactly
twice; the remaining value appears only once. Design an
efficient algorithm for finding which value appears only
once.
6. The problem is to tile a board of size 2k × 2k with one single
tile and 22k - 1 L-shaped groups of 3 tiles. A divide-and-
conquer approach can recursively divide the board into four,
and place a L-grouped set of 3 tiles in the center at the parts
that have no extra tile.
7. Find the nth smallest value among 2n numbers?

Divide-and-Conquer 103
 Home work

In terms of n, how many additions and multiplications

(total) are performed when this code is executed with an
even input of n? Simplify your result to a polynomial.

Divide-and-Conquer 104
 References
• https://ocw.tudelft.nl/courses/algoritmiek/?view=lectures

Divide-and-Conquer 105