Design and Analysis of

Algorithms (CS205)

Lecture 3
Divide & Conquer:
Merge sort
Divide And Conquer: Control Abstraction

3 steps involved • Divide, Conquer, Combine

Divide • Divide the given problem into smaller size subproblems

Conquer • Conquer the subproblem to get solution by calling recursively

• If subproblem is small, then return solution directly

Combine • Combine the subproblem solution to get original problem solution

Example DAC
• Let there be a problem of size N and let us divide this problem into 4 sub-problems say n1, n2, n3, and n4.
• Let the time taken to complete the whole problem be T(N),
• time taken to complete n1, n2, n3, n4 respectively be T(n1), T(n2), T(n3) and T(n4),
• time taken in dividing the problem into sub-problems be D(N),
• time taken to combine the solutions of sub-problems into one final solution be C(N).
• Then, T(N) = T(n1) + T(n2) + T(n3) + T(n4) + D(N) + C(N) (If N is large)
T(N) = c (If N is small)
• T(N) = 4T(N/4) + f(N); where f(N) = D(N) + C(N)
• Recurrence form
T(n) = a * T(n/b) + f(n), where a > 0; b > 1
• T(n) represents the running time of the algorithm for a problem of size n.
• "a” represents the number of subproblems the problem is divided into.
• "n/b" represents the size of each subproblem.
• f(n) represents the time spent on dividing the problem and combining the solutions of subproblems, often
referred to as the "combine" step.
• Divide and conquer is a recursion based algorithm. So, it will have a recurrence relation that mathematically
defines its behaviour.
 Advantages of Divide and Conquer Algorithm

01 02 03 04 05
1. The divide and 3. The divide and 4. The divide and 6. Brute force 7. The divide and
conquer algorithm conquer divides the conquer strategy technique and divide conquer technique is
solves all the problem into sub- makes use of cache and conquer quite faster than
problems recursively, problems which can memory because of techniques are other algorithms.
so any problem, run parallelly at the the repeated use of similar but divide and
which requires same time. Thus, this variables in conquer is more
recursion can make algorithm works on recursion. proficient than the
use of Divide and parallelism. This brute force method.
conquer. property of divide
and conquer is
extensively used in
the operating
system.
 Disadvantages of Divide and Conquer Algorithm

01 02 03 04
1. The divide and conquer 2. The implementation of 3. Memory overuse is 4. The system may crash in
technique uses recursion. divide and conquer requires possible by an explicit stack. case the recursion is not
Recursion in turn leads to high memory management. performed properly.
lots of space complexity
because it makes use of the
stack.
Applications of Divide and Conquer

Defective
Quicksort Mergesort Binary search
chessboard

Finding the
maximum and Strassen’s matrix
minimum in an multiplication
array
Quick Sort Idea(DAC)

• Get pivot element, usually first element; find where will the pivot element be placed
• Two pointers P,Q
• P moves left to right until element greater than pivot is found or (ii) end of array is
found
• Q moves right to left until element less than or equal pivot is found

35 50 15 25 80 20 90
How does quick sort work?
• Pick: Select an element.
• Divide: Split the problem set, move smaller parts to
the left of the pivot and larger items to the right.
• Repeat and combine: Repeat the steps and
combine the arrays that have previously been
sorted.
• Partition algo:
Quicksort

function quickSort(arr, low, high) function partition(arr, low, high)

if low < high pivot = arr[low] // Choose the first element as the pivot
pivotIndex = partition(arr, low, high) left = low + 1 // Initialize left pointer just after the pivot
quickSort(arr, low, pivotIndex - 1) right = high // Initialize right pointer at the end of the array
quickSort(arr, pivotIndex + 1, high) while left <= right do
// Find an element larger than or equal to the pivot from the left
while left <= right and arr[left] <= pivot do
left = left + 1
// Find an element smaller than the pivot from the right
while arr[right] > pivot do
right = right - 1
if left < right then
// Swap the elements at left and right pointers
swap(arr[left], arr[right])
// Swap the pivot element with the element at the right pointer
swap(arr[low], arr[right])
return right // Return the index of the pivot's final position
Quicksort: Performance Analysis
• Best case, Average case: Partition element is near the middle
<n/2> p <n/2>

For QuickSort, the average-case recurrence relation is:

T(n) = 2 * T(n/2) + O(n) = O(nlogn)
• Worst case: pivot selection consistently results in unbalanced partitions, causing the recursion tree to be
skewed.
• If the pivot selection is always the smallest or largest element, and the array is already sorted in
ascending or descending order, each partition will result in one subarray of size n-1 and another of
size 0. This creates a skewed tree with n levels, leading to a time complexity of O(n2).
p <n-1> //ascending, smallest pivot; è T(n) = T(n-1) + O(n) = O(n2)
<n-1> p //descending, largest pivot; è T(n) = T(n-1) + O(n) = O(n2)
Quicksort features
• In-place sorting: No extra memory required. It takes constant space
• Not stable: Positions of elements with equal keys might change during
the sorting process.

Alice, 25 David, 22 David, 22

Bob, 30 Carol, 25 Alice, 25
Carol, 25 Alice, 25 Carol, 25
David, 22 Bob, 30 Bob, 30
Name, Not stable Stable
age

• Perform QuickSort to sort by age.

• As you can see, Carol's and Alice's positions have changed after sorting,
even though their ages are the same. This is what makes QuickSort not
stable.
• In a stable sorting algorithm, Carol would still come before Alice in the
sorted list because Carol appeared before Alice in the original list.
Sorting Algo Types

Internal: All
elements needs to • Bubble, insertion, merge, quick,
be in main memory selection
before sorting

External: Some • External merge sort, B-Trees, B+

elements may be in
secondary memory Trees, Radix sort, Bucket sort
 Quick Sort in Computer
Applications
1. Due to speed and effectiveness in average-case scenarios, QuickSort is employed to
efficiently sort and arrange data, resulting in better user experiences and more
optimized data management.
2. File System Directory Listings (Operating Systems): Arrange files and folders in
directory listings
3. Search Engines (Web Browsers): Sort search results based on search history, user
profile
4. Text Editors and Code Editors
5. Graphical User Interfaces (GUIs):To list of recently used files, or a list of available
Wi-Fi networks
6. Network Packet Routing: Sorting Networking protocols based on criteria such as
priority
7. Web Servers (Load Balancing): Server load balancing based on their current load
and capacity.
8. Version Control Systems: Used to sort and present the history of changes in a
chronological order.
 Merge Sort
• Divide: In this step, the array/list divides itself recursively into
sub-arrays until the base case is reached.

• Recursively solve: Here, the sub-arrays are sorted using

recursion.

• Combine: This step makes use of the merge( ) function to

combine the sub-arrays into the final sorted array.

• Arrange the letters of the word ‘TECHVIDVAV’ in ascending

order
• Algorithm for Merge Sort
Step 1: Find the middle index of the array.
Middle = 1 + (last – first)/2
Step 2: Divide the array from the middle.
Step 3: Call merge sort for the first half of the array
MergeSort(array, first, middle)
Step 4: Call merge sort for the second half of the array.
MergeSort(array, middle+1, last)
Step 5: Merge the two sorted halves into a single sorted
array.
 Merge Sort Algo

• Algorithm for Merge Sort

• Step 1: Find the middle index of the array.
Middle = 1 + (last – first)/2
Step 2: Divide the array from the middle.
Step 3: Call merge sort for the first half of the array
MergeSort(array, first, middle)
Step 4: Call merge sort for the second half of the array.
MergeSort(array, middle+1, last)
Step 5: Merge the two sorted halves into a single sorted array.
Merge Sort Performance Analysis
• Best, Average, Worst case
T(n) = c, when n = 1
= 2T(n/2) + bn, when n > 1
è O(nlogn)
• Space complexity: Not inplace
• C + n + n + logn
Characteristics of Merge Sort
• External sorting algorithm: Merge sort can be used
when the data size is more than the RAM size. In such a
case, whole data cannot come into RAM at once. Thus,
data is loaded into the RAM in small chunks and the
rest of the data resides in the secondary memory.
• Non-inplace sorting algorithm: Merge sort is not an
inplace sorting technique as its space complexity is
O(n). Inplace or space-efficient algorithms are those
whose space complexity is not more than O(logn).
Merge sort in computer
applications
• Merge Sort's stability and ability to handle large datasets
efficiently make it a valuable choice for sorting tasks, particularly
in scenarios involving external storage, distributed systems, and
collaborative environments.
• External Sorting (Disk-Based Sorting): Merge Sort is commonly
used for sorting large datasets that cannot fit entirely in memory.
• Collaborative Document Editing Tools: Applications that support
collaborative document editing, such as Google Docs, need to
merge changes made by multiple users. Merge Sort can be used
to merge contributions and ensure data integrity.
• Merge Sort in Parallel Computing: Merge Sort can be parallelized
to take advantage of multi-core processors. It's applied in parallel
computing scenarios where data needs to be sorted efficiently
across multiple threads or nodes.
Strassen's Matrix Multiplication
• *Strassen, V., 1969. Gaussian
elimination is not
optimal. Numerische
mathematik, 13(4), pp.354-356.
Basic Matrix Multiplication

Suppose we want to multiply two matrices of size N x

N: for example A x B = C.

C11 = a11b11 + a12b21

C12 = a11b12 + a12b22
C21 = a21b11 + a22b21
2x2 matrix multiplication can be
C22 = a21b12 + a22b22 accomplished in 8 multiplication.(2log28 =23)
Basic Matrix Multiplication

void matrix_mult (){

for (i = 1; i <= N; i++) { algorithm
for (j = 1; j <= N; j++) {
compute Ci,j;
}
}}

N
Ci , j = å ai ,k bk , j
Time analysis k =1
N N N
Thus T ( N ) = ååå c = cN 3 = O( N 3 )
i =1 j =1 k =1
 Strassens’s Matrix
Multiplication
• Strassen showed that 2x2 matrix multiplication can
be accomplished in 7 multiplication and 18
additions or subtractions. .(2 =2 )
log 7
2
2.807

• This reduce can be done by Divide and Conquer

Approach.
Divide-and-Conquer
• Divide-and conquer is a general algorithm design
paradigm:
• Divide: divide the input data S in two or more disjoint
subsets S1, S2, …
• Recur: solve the subproblems recursively
• Conquer: combine the solutions for S1, S2, …, into a
solution for S
• The base case for the recursion are subproblems of
constant size
• Analysis can be done using recurrence equations
Divide and Conquer Matrix
Multiply
A ´ B = R
A0 A1 B0 B1 A0´B0+A1´B2 A0´B1+A1´B3
´ =
A2 A3 B2 B3 A2´B0+A3´B2 A2´B1+A3´B3

•Divide matrices into sub-matrices: A0 , A1, A2 etc

•Use blocked matrix multiply equations
•Recursively multiply sub-matrices
Divide and Conquer Matrix
Multiply
A ´ B = R
a0 ´ b0 = a0 ´ b0

• Terminate recursion with a simple base case

Strassens’s Matrix
Multiplication

P1 = (A11+ A22)(B11+B22) C11 = P1 + P4 - P5 + P7

P2 = (A21 + A22) * B11 C12 = P3 + P5
P3 = A11 * (B12 - B22) C21 = P2 + P4
P4 = A22 * (B21 - B11) C22 = P1 + P3 - P2 + P6
P5 = (A11 + A12) * B22
P6 = (A21 - A11) * (B11 + B12)
P7 = (A12 - A22) * (B21 + B22)
Comparison

C11 = P1 + P4 - P5 + P7
= (A11+ A22)(B11+B22) + A22 * (B21 - B11) - (A11 + A12) * B22+
(A12 - A22) * (B21 + B22)
= A11 B11 + A11 B22 + A22 B11 + A22 B22 + A22 B21 – A22 B11 -
A11 B22 -A12 B22 + A12 B21 + A12 B22 – A22 B21 – A22 B22
= A11 B11 + A12 B21
Strassen Algorithm
void matmul(int *A, int *B, int *R, int n) {
if (n == 1) {
(*R) += (*A) * (*B);
} else {
matmul(A, B, R, n/4);
matmul(A, B+(n/4), R+(n/4), n/4);
matmul(A+2*(n/4), B, R+2*(n/4), n/4);
matmul(A+2*(n/4), B+(n/4), R+3*(n/4), n/4);
matmul(A+(n/4), B+2*(n/4), R, n/4);
matmul(A+(n/4), B+3*(n/4), R+(n/4), n/4);
matmul(A+3*(n/4), B+2*(n/4), R+2*(n/4), n/4); Divide matrices in
sub-matrices and
matmul(A+3*(n/4), B+3*(n/4), R+3*(n/4), n/4);
recursively multiply
} sub-matrices
Time Analysis
Basic Matrix Multiplication
• Compute multiplication of square matrices
Method 2: Divide and conquer
• Divide sq matrix of size N to 4 matrices of size n/2
Time Complexity
• n2
Method 3: Stressmen Approach
Time Complexity
Trick to memorize