Chapter 4 Algorthms 1 31

CHAPTER 4
ALGORITHMS
Syllabus: Algorithms: Analysis, Asymptotic notation, Notions of space and time complexity, Worst- and average-case
analysis; Design: Greedy approach, Dynamic programming, Divide-and-conquer; Tree and graph traversals, Connected
components, Spanning trees, Shortest paths; Hashing, Sorting, Searching. Asymptotic analysis (best, worst, average
cases) of time and space, Upper and lower bounds, Basic concepts of complexity classes — P, NP, NP-hard, NP-complete.
4.1 INTRODUCTION 4.2.1 Properties of Algorithm
There are certain properties algorithms have:

In computer science, an algorithm provides a set of step- 1. An algorithm has zero or more inputs.
by-step instructions for solving various problems. The 2. It should produce at least one output.
set of instructions should be unambiguous, finite and 3. It should terminate within the finite time.
provide a clear termination point. An algorithm accepts 4. It should be deterministic (ambiguous).
some input values and produces outputs. The algorithm 5. It is generic to all programming languages.
can be built in simple English language or in program-
ming language. This subject helps in solving and analys- 4.2.2 Steps to Solve a Problem
ing complex problems through logical thinking.
1. Define the problem, which means what is input
and what should be the output.
4.2 ALGORITHM 2. Identify the constraints which are to be satisfied.
3. Design the algorithm for a given problem.
4. Draw the flowchart.
An algorithm is a well-defined procedure which takes a 5. Verify the process.
set of values as an input and produces a set of values as 6. Implement the coding.
the output. 7. Perform time and space analysis.
Chapter 4.indd 177 4/9/2015 9:54:03 AM

178 Chapter 4: Algorithms
4.2.3 Algorithm Analysis Example 4.1

If a problem has more than one solution, then the tech- Let us consider an example of a search problem
nique used to decide which solution is best is known as (searching a given item) in a sorted array. One way to
algorithm analysis. search is linear search (the order of growth is linear)
and the other way is binary search. Suppose we have
4.2.3.1 A Posteriori and A Priori Analysis two machines with different processing speeds. We run
linear search on the fast machine and binary search on
Analysis is done on two parameters: time complexity and slow machine. For small input size, the linear search
space complexity. Time complexity means time taken will perform better. But as we increase the size of
by an algorithm to solve a problem. Space complexity input, binary search will perform better. The reason
means memory space required by an algorithm to solve behind this is,the order of growth of binary search is
a problem. There are two types of analysis: logarithmic and that of linear search is linear. So, by
1. A Posteriori Analysis: It is a software- and using this technique the issues of machine dependency
hardware-dependent analysis. It keeps on chang- are resolved.
ing from one system to other system. This type of
analysis tells the exact time and space complexity, 4.2.4 Asymptotic Notation
meaning how much an algorithm takes time and
space for solving a problem on the given system. Asymptotic notation is based on two assumptions, which
2. A Priori Analysis: It is a software- and hard- hold in most of the cases and have their importance.
ware-independent analysis. It is constant for all the It is very much important to understand the assump-
systems. This type of analysis tells the approximate tions and limitations of asymptotic notations:
time and space complexity. It is the determination 1. Input size: It is interesting to know how the run-
of the order of magnitude of a statement, meaning ning time of an algorithm grows for a large input
how many times a statement is executing. size n.
2. Constant: The running time of an algorithm also
4.2.3.2 Asymptotic Analysis depends on various constants. But for a large input
n, the constant factors would be ignored.
Suppose there are given two algorithms for a task, how
do we find out which one is better? For representing best-case, average-case, and worst-case
This can be done by actually implementing both the complexity, there are three different notations as given
algorithms and running them on the same computer with in the following sub-sections.
different input values. Then the comparison is made in
terms of time. Algorithm that takes less time is considered 4.2.4.1 Theta Notation (Q)
better. But there are so many problems with this approach.
1. It might be possible that for certain inputs first Theta notation shows the tightest upper bound and the
algorithms performs better than second and for tightest lower bound to the given function (Fig. 4.1). It is
other set of inputs second algorithm performs used to define the average-case running time. For a given
better. So,the decision of choosing best among function f(n), we denote Q(g(n)) as Q(g(n)) = {f(n):
them becomes difficult. there exists positive constants c1 > 0, c2 > 0 and n0 > 0
2. With the change in operating machine, performance such that 0 < c1(g(n)) ≤ f (n) ≤ c2 (g(n)) ∀ n ≥ n0}
of algorithm varies. On one machine first algorithm
can work better and on another machine, the second
works better. This is another problem associated c1g(n)
with this approach.
f(n)
Above issues can be handled using asymptotic analy-
f(n) c2g(n)
sis techniques for analysing algorithms. In asymptotic
analysis, analysis of algorithm in done in terms of input g(n)
size. Actual running time is not computed, variation of
time with input size is calculated.
Now, understand that how the issues with above n
n0
Figure 4.1 | Theta notation (Q)
technique are resolved by asymptotic analysis technique
using the below example:
Chapter 4.indd 178 4/9/2015 9:54:04 AM

4.2 ALGORITHM 179
Problem 4.1: Consider the following functions: Problem 4.2: Consider the following functions:
f (n) = n f (n) = n
n g(n) = n
g(n) =
8
For f(n) = O(g(n)), find the values of c and n0.
For f(n) = Q(g(n)), find the values of c1, c2 and n0.
Solution: From the definition of Big-O notation, we
Solution: From the definition of theta (Q) notation, have
f (n) ≤ c(g(n)) ∀n ≥ n0
we have
0 < c1(g(n)) ≤ f (n) ≤ c2 (g(n)) ∀n ≥ n0 By putting c = 1 and n = 1, we get

By putting c1 = 1, 2, 3 to satisfy the above condition, n ≤ c ⋅ n ∀ n ≥ n0
we get
c1(g(n)) ≤ f (n) The above condition is satisfied. So, c = 1 and n0 = 1.
n
1   ≤ n
 8 
4.2.4.3 Omega Notation (W)
c1 = 1
Omega notation shows the lower bound to the given
By putting c2 = 1, 2, 3 to satisfy the above condition, function (Fig. 4.3). It is used to define the best-case run-
we get ning time. For a given function f(n), we denote W(g(n))
f (n) ≤ c2 (g(n)) as W(g(n)) = {f(n): there exist positive constants c > 0
and n0 > 0 such that 0 < c(g(n)) ≤ f (n) ∀ n ≥ n0 }.
n
8   ≤ n
 8 
c2 = 8 f(n)
cg(n)
By putting n0 = 1, 2, 3, …, to satisfy the above condi-
tion, we get n0 = 1. f(n)
g(n)
4.2.4.2 Big-O Notation (O)
Big-O notation shows the upper bound to the given func- n0

tion (Fig. 4.2). It is used to define the worst-case running Figure 4.3 | Omega notation (W).
time. For a given function f(n), we denote O(g(n)) as
O(g(n)) = {f(n): there exist positive constants c > 0 and
n0 > 0 such that 0 < f (n) ≤ c(g(n))∀n ≥ n0} Problem 4.3: Consider the following functions:
f (n) = n2
c(g(n)
g(n) = n
f(n)
For f(n) = W(g(n)), find values of c and n0.
Solution: From the definition of Omega notation, we
f(n) have
g(n) c(g(n)) ≤ f (n) ∀ n ≥ n0
By putting c = 1 and n = 1, we get
n0 c ⋅ n ≤ n2 ∀ n ≥ n0
Figure 4.2 | Big-O notation (O). The above condition is satisfied. So, c = 1 and n0 = 1.
Chapter 4.indd 179 4/9/2015 9:54:09 AM

O(nn)
O(an)
O(nk)
O(n2)
O(n)
Time
O(√n)
Olog(n)
O(log(log n)
O(1)
Complexities
Figure 4.4 | Time complexities in increasing order.
4.2.4.4 Small-o Notation (o) 4.2.4.7 Types of Complexities

1. Constant time complexity: O(1)
Small-o represents an upper bound which is not a tight
2. Logarithmic time complexity: O(log(log n)),
upper bound. It is also used to define the worst-case run-
O( log n ) and O(log n)
ning time. For a given function f(n), we denote o(g(n))
as o(g(n)) = {f(n): there exist positive constants c > 0
3. Linear time complexity: O( n ) and O(n)
4. Polynomial time complexity: O(nk ), where k is a
and n0 > 0 such that 0 < f (n) < c(g(n)) ∀ n ≥ n0}.
constant and is >1
5. Exponential time complexity: O(an ), where a > 1
4.2.4.5 Small Omega Notation (w)
The complexities in increasing order are represented in
Small omega (w) represents a lower bound which is not a Fig. 4.4.
tight lower bound. It is used to define the best-case run-
ning time. For a given function f(n), we denote w(g(n)) 4.2.5 Recurrence Relations
as w(g(n)) = {f(n): there exist positive constants c > 0 Recurrence relations are recursive definitions of
and n0 > 0 such that 0 < c(g(n)) < f (n) ∀ n ≥ n0 }. mathematical functions or sequences. For example,
the recurrence relation defines the famous Fibonacci
4.2.4.6 Properties of Asymptotic Notations sequence 1, 1, 2, 3, 5, 8, 13, …:
f (n) = f (n - 1) + f (n - 2)
1. Transitivity: All asymptotic notations show
transitivity. f(1) = 1
If f(n) = Q(g(n)) and g(n) = Q(h(n)), then f(n) = f(0) = 1

Q(h(n)). Given a function defined by a recurrence relation, we
If f(n) = O(g(n)) and g(n) = O(h(n)), then f(n) = want to find a `closed form’ of the function. In other
O(h(n)). words, we would like to eliminate recursion from the
If f(n) = W(g(n)) and g(n) = W(h(n)), then f(n) = function definition. There are several techniques for solv-
W(h(n)). ing recurrence relations, and these are discussed in the
If f(n) = o(g(n)) and g(n) = o(h(n)), then f(n) = following sub-sections.
o(h(n)).
If f(n) = w(g(n)) and g(n) = w(h(n)), then f(n) = 4.2.5.1 Iteration Method
w(h(n)).
This method is accurate but involves a lot of algebraic
2. Reflexivity: Only Theta, Big-O, and Omega nota- expressions which are difficult to keep track of. It can get
tions show reflexivity. Small-o and small-omega do very challenging for further complicated recurrence
not show reflexive property. relations.
f (n) = Q(f (n)) f (n) = O(f (n)) f (n) = W(f (n))
Problem 4.4: We have T (n) = 1 if n = 1
3. Symmetric: Only Theta notation is symmetric.
n
f(n) = Q(g(n)) if and only if g(n) = Q(f(n)). T (n) = 4T   + n if n > 1
 2 
4. Transpose symmetry:
f(n) = O(g(n)) if and only if g(n) = W(f(n)) n  n n
T (n) = 4T   + n = 4 4T   +  + n
f(n) = o(g(n)) if and only if g(n) = w(f(n))  2    4  2 
Chapter 4.indd 180 4/9/2015 9:54:13 AM

4.2 ALGORITHM 181
How deep does the tree go?

  n n n
= 4 4 4T   +  +  + n We stop at the leaf, and we know we are at a leaf when
   8  4  2  we have a problem of size 1:
n  n 2
= 64T   + 4n + 2n + n 1 =  i  ⇒ n2 = 22i ⇒ n = 2i ⇒ i = log n
 8   2 
n
= n + 2n + 4n + 64T  
The amount of work done is given as
 8  log n
ni
n
= n + 2n + 4n + + 2 j n + + 4i T  i 
∑ = Qn2
 2  i=0 2
This is geometrically decreasing in size, so it would not
Find the time complexity when (n/2i) = 1. get any bigger than n2.
Solution:
We have i = log n . So, the time complexity of n can 4.2.5.3 Master Theorem
be represented as
Master theorem provides a method to solve various
T (n) = n + 2n + 4n + 8n + 2i n + 4log n O(1)
recurrence relations. Although all the relations can’t
log(n -1)  be solved using this method, but it is one of the useful

We get n  ∑ 2i  + 4log n O(1) method of solving complex relations of the form:
 i = 0 
n
T (n) = aT   + f (n)
 b 
m
xm +1 - 1
We Know that ∑ xk = x -1
where a and b are constants. f is a function of n. this
k =0 recursive algorithm breaks up the input into sub-problems
 log(n -1)+1 - 1 of size n/b each.
So, n  2  + 4log n O(1)
 2 -1  Three cases of master theorem are as follows:

1. Case 1: If f(n) is O(n logb a-e ) then T(n) is Q(n logb a ) .
Solving, we get
The rate of growth at leaves is faster than f, more
n2log n - n + 4log n O(1) ⇒ n2 - n + n log 4O(1) work is done on leaves.
2. Case 2: If f(n) is Q(n logb a ) then T(n) as
⇒ 2n2O(1) - n ⇒ O(n2 ) Q(n logb a log n) . The rate of growth of leaves is same
as f. So, the same amount of work is done at every
So, the running time complexity is given as: T(n) = O(n2). level of tree. The leaves grow at the same rate as
f, so the same order of work is done at every level
of the tree. The tree has O(log n) times the work
done on one level, yielding T(n) as Q(n logb a log n) .
4.2.5.2 Recursion Tree 3. Case 3: f(n) is W(n logb a+e ). Here f grows faster than
the number of leaves, which means that asymptoti-
A recursion tree is useful for visualizing what happens
cally the total amount of work is dominated by the
when a recurrence is iterated. It represents a tree of
work done at the root node. For the upper bound, we
recursive calls and the amount of work done at each call
also need an extra smoothness condition on f in this
case, namely, that af(n/b) ≤ cf(n) for some constant
is shown with the help of a diagram.
c < 1 and large n. In this case, T(n) is Q(f(n)).
Example 4.2
Example 4.3
Consider the recurrence T (n) = 2T (n/2) + n2.
Consider the following recurrence:
n2
n
T (n) = 4T   + n
 2 
T(n/2) T(n/2)
n2 In the given relation, a = 4 and b = 2.
n2
For this recurrence, there are a = 4 sub-problems, each
(n/2)2 (n/2)2 1/2n2 dividing the input by b = 2, and the work done on each
call is f(n) = n. Thus, n logb a is n2 and f(n) is O(n2-e) for
e = 1, and Case 1 applies. Thus, T(n) is Q(n2).
(n/4)2 (n/4)2 (n/4)2 (n/4)2 1/4n2
Chapter 4.indd 181 4/9/2015 9:54:19 AM

Example 4.4 is placed. It is important to note that hash function

returns the same index value every time for same key.
n
T (n) = 4T   + n2 4.3.3 Collisions
 2 
For this recurrence, there are again a = 4 sub-problems, Collision occurs when one key value stores more than one
each dividing the input by b = 2; but now the work done value. Hash function producesthe same position for two
on each call is f(n) = n2. Again, n logb a is n2 and f(n) is values. Hash table cannot ensure to have single value
thus Q(n2), so Case 2 applies. Thus, T(n) is Q(n2 log n). corresponding to a key. This is because a hash table has
Note that increasing the work on each recursive call from fixed number of keys. Any new entry in hash table, after
linear to quadratic has increased the overall asymptotic all the slots are filled, causes collision. However, good
running time by only a logarithmic factor. hash tables use some mechanism to avoid the situations
of collisions or to handle them efficiently.
Example 4.5 4.3.3.1 Separate Chaining

1. If more than one item is assigned the same hash
n
T (n) = 4T   + n3 index, “chain” them together.
 2  2. In hash table, each position act as bucket that can
For this recurrence, there are again a = 4 sub-problems, store multiple data items.
each dividing the input by b = 2, but now the work done 3. There are the following two implementations:
on each call is f(n) = n3. Again, n logb a is n2 and f(n) is • Each bucket is itself an array: The disadvan-
thus W(n2+e ) for e = 1. Moreover, 4(n/2)3 ≤ kn3 for k = tages of bucket as array are
1/2, so Case 3 applies. Thus, T(n) is Q(n3). (a) Memory wastage
(b) Problem of overflow occurs when we try to
add new item to a full bucket.
• Each bucket is a linked list: The disadvantage of
4.3 HASHING
bucket as linked list is
Pointers require more memory.
Hashing is technique that transforms a variable length 0 arrow “ape”
input to a fixed length output. This technique is used for
1 null null
indexing, which helps in constant time search for inser-
tion, deletion and information retrieval. 2 “cat”
... ... null
4.3.1 Hash Table
A hash table is a data structure used for hashing. It is 4.3.3.2 Open Addressing
an associative array that contains keys with values. In
an array, indices are required to be integer, but a hash 1. Collision occurs if the hash function produces the
table allows floating point numbers, strings, array or same position occupied by another item. In this
structures as keys. Hash tables provide efficient lookup case, another open position is searched.
operation, that is, whenever a key is given, hash table
Example: Hash code produced for `White’ is 22.
provides associated value in almost constant time com-
But position 22 is already occupied, so it is placed
plexity. This transformation is done using hash function.
at position 23.
2. There are three ways of finding an open position.
4.3.2 Hashing Functions
This process is known as probing. To search any
A hashing function is a transformation function, coded item in hash table, probing may be required.
to return a value based on key. If h is a hash function Example: When item “white” is searched, first we
that takes key k as parameter, will compute the index at look for it at position 22. If it is not found there, we
which value is placed. If h is a hash function that takes move to next location. The search will be stopped
key as a parameter, it will compute index where value only when an empty position is encountered.
Chapter 4.indd 182 4/9/2015 9:54:21 AM

4.3 HASHING 183
4.3.3.4 Quadratic Probing

0 Arrow
The probe sequence is
1
h(key), h(key) + 1, h(key) + 4, h(key) + 9, …,
2 Cat wrapping around as necessary
3 The offsets are perfect squares: h + 12, h + 22, h + 32, ….
Examples:
4 Eye
a. Àpe’ (h = 0): try 0, 0 + 1 − open!
5 b. `Ball’ (h = 1): try 1, 1 + 1, 1 + 4, 1 + 9 − open!
... ... ... Advantage: It reduces clustering.
22 When 0 Arrow
23 White 1 Ape
24 Yellow 2 Cat
25 Zero 3
4 Eye
4.3.3.3 Linear Probing 5 Ball
In case of linear probing the probe sequence is h(key), ... ... ...
h(key) + 1, h(key) + 2, …, uptill required. 22 When
Examples: The item Àpe’ (h = 0) will be placed at
23 White
position 1, because position 0 is already occupied. then
`Ball’ (h = 1): try to place it at position 1, 1+1, 1+2. 24 Yellow
Position 3 is open so it will be placed here.
25 Zero
Advantage: If open position is available, linear probing
will definitely find it.
Disadvantage: It may fail to find an existing open
Disadvantage: If there are clusters of filled positions,
position. For example, let table size = 10, and x denote
then length of subsequent probes increases. Probe length
the occupied blocks. Trying to insert a key with h(key) =
is the number of positions considered during a probe.
0 offsets the probe sequence shown in italic.
0 Arrow
0 x 5 x 25
1 Ape
2 Cat 1 x 1 81 6 x 16 36
3 Ball 2 7
4 Eye 3 8
5 x
4 x 4 64 9 9 49
... ... ...
22 When 4.3.3.5 Double Hashing

23 White
It uses the following two hash functions:
24 Yellow 1. h1 computes the hash code.
25 Zero 2. h2 computes the increment for probing.
The probe sequence is: h1, h1 + h2, h1 + 2 ×h2, ….
Chapter 4.indd 183 4/9/2015 9:54:22 AM

Examples:
0 Arrow
1. h1 = our previous h
2. h2 = number of characters in the string 1 1
3. Àpe’ (h1 = 0, h2 = 3): try 0, 0 + 3 − open!
`Ball’ (h1 = 1, h2 = 4): try 1 − open!
2 Cat
4.
3 Ball
Advantages:
4 Eye
1. It combines the good features of linear and qua-
dratic probing: 5
2. It reduces clustering
3. It will find an open position if there is one, pro- ... ... ...
vided the table size is a prime number 22 When
23 White
0 Arrow 24 Yellow
1 Ball 25 Zero
2 Cat
3 Ape
4.4 BINARY HEAP
4 Eye
5 A binary heap is a binary tree with two additional constraints:
... ... ... 1. Binary heap should satisfy the property of com-
22 When plete binary tree.
2. It should satisfy the heap ordering property, that
23 White is, all the nodes should be either greater than or
equal to or less than or equal to its children.
24 Yellow
According to the heap ordering property, binary heaps
25 Zero
are of two types:
1. Min-Heap Tree: In Min heap ordering, the

value of each node is less or equal to its child node.
4.3.3.6 Removing Items Under Open Minimum element of the tree will be the root ele-
Addressing ment. The largest element exists in the last level.
The tree should be an almost complete binary tree.
Consider the following scenario:
Using linear probing:
(a) Insert Àpe’ (h = 0): Try 0, 0 + 1 − open! 7
(b) Insert `Ball’ (h = 1): Try 1, 1 + 1, 1 + 2 − open!
(c) Remove Àpe’.
(d) Search for Àpe’: Try 0, 0 + 1 − no item. 10 17
(e) Search for `Ball’: Try 1 − no item, but `Ball’ is
further down in the table.
1. On removing an item from a position, leave a spe- 14 23 21 42
cial value in that position to indicate that an item
was removed.
2. There are three types of positions: occupied, empty 2. Max-Heap Tree: In max heap, each node in the
and removed. tree will be greater or equal to its child node. The
3. Stop probing on encountering an empty position, maximum element of tree will be the root element.
but not when encountering a removed position. The smallest element exists in the last level. The
4. Insert items in either empty or removed positions. tree should be an almost complete binary tree.
Chapter 4.indd 184 4/9/2015 9:54:23 AM

4.4 BINARY HEAP 185
4.4.1 Insertion in Min-Heap Tree

10
The following are the steps to follow for insertion in the
min-heap tree:
8 9 1. Add the new element to the last level of heap.
2. Compare the new element with its parent node.
3. If the order is correct then stop, otherwise swap the
4 1 2 6 element with its parent.
4. Perform steps 2 and 3 until a new element is at position.
Example 4.6
1. Insert the given value (5) in the given min-heap tree. 4. To satisfy the property of min-heap, swap (5, 7).
2. Element 5 is inserted at position of left child of node 5. Now, element 5 is in its right position. This is the
14. To satisfy the property of min-heap, swap (5, 14). final tree.
3. To satisfy the property of min-heap, swap (5, 10).
7 7
7
10 17 10 17
10 17
14 23 21 42 5 23 21 42
14 23 21 42
5 14
1 2 3
7 5
5 17 7 17
10 23 21 42 10 23 21 42
14 14
4 5
4.4.2 Deletion in Min-Heap Tree Example 4.7

The following are the steps to follow for deletion in the 1. Deletion always occurs at the root level, so element
min-heap tree: 7 will be deleted. Before deletion of element 7 at
the root, the element (42) is swapped to the root
1. Root of the heap is replaced with the element at node from the extreme right-hand side of the last
the last level. level and then 7 is deleted.
2. New root element is compared with its children; if 2. To maintain the min-heap property of the tree, swap
it maintains ordering then stop. 42 and min(left-child, right-child), which means
3. Otherwise, swap the root with one of its chil- swap 42 with 10.
dren and return to previous step. (Smaller ele- 3. To maintain the min-heap property of the tree, swap
ment is swapped for min-heap and larger child for 42 and min(left-child, right-child), which means
max-heap) swap 42 with 14.
Chapter 4.indd 185 4/9/2015 9:54:24 AM

4. This represents the final tree, which is in the min- a set of numbers in an ascending or descending
heap property. order.
3. In internal sorting, the data which is to be sorted fits
7 42 into the main memory. Internal sorting has an advan-
tage of the random access nature of the main memory.
10 17 10 17 4. In external sorting, large amount of data is
sorted. This large amount of data cannot be fit
14 23 21 42 14 23 21 in main memory, so the data is placed in auxil-
iary memory. When sorting has to perform, small
1 2
chunks of data are brought into main memory,
10 10 sorted and placed back to a temporary file. At last
all the sub-files are merged in one large file.
42 17 14 17 5. In stable sorting, the order of equal elements is
preserved in a sorted list. Some of the sorting algo-
rithms that show characteristics of stable sort are
14 23 21 42 23 21
insertion sort, bubble sort, merge sort, etc.; whereas
3 4 heap sort, quick sort, etc., are not stable sorts.
6. In-place sorting: In this, the input is transformed
using a data structure with a small, constant amount
4.4.3 Time Complexity of extra storage space. The input is usually overwrit-
ten by the output as the algorithm executes. Quicksort
1. The worst-case running time for insertion is is commonly described as an in-place algorithm.
O(log n) because at most (log n) swaps are required 7. Measurement of time complexity: The time
to stable the min-heap property of the tree. complexity of a sorting algorithm is measured
2. The worst-case running time for deletion is O(log n) by the number of critical operations performed.
because at most (log n) swaps are required to stable Examples of critical operations are as follows:
• Key comparisons
the min-heap property of the tree.
Some important points of binary heap are as follows: • Movements of records
1. Because of its structure, a heap with height k will • Interchanges of two records
have between 2k and 2k+1 - 1 elements. Therefore, 8. We use asymptotic analysis to denote the time
a heap with n elements will have height = log2n. efficiency of an algorithm. Efficiency of an algorithm
2. In heap data structure, element with highest or lowest also depends on the type of data for input, so we
priority is stored as root element. Heap is not consid- define best-, worst- and average-case efficiencies.
ered as sorted structure, rather it is partially ordered.
There is no relation among nodes at a given level. 4.5.1 Linear Search
Heap is very useful data structure for implement
ing priority queues. Element with highest priority Linear search is a sequential way of finding an element in
can be easily removed. a given list. Linear search is a special case of brute force
3. Since a binary heap is a complete binary tree, it search. Its worst case is proportional to the number of
can be easily represented as an array, and an array- elements in list.
based representation is space efficient. If the parent
node is stored at index I, the left child can be cal-
4.5.1.1 Pseudocode for Linear Search
culated by 2 ×I + 1 and right child by 2 ×I + 2.
LINEAR_SEARCH (A[ ], x)
4.5 SEARCHING AND SORTING 1. i = 1
2. while (A[i] = = x and i < n)
3. i = i + 1
4. if (A[i] = = x)
Searching and sorting are the most common activities
5. return i
performed by the computer. These operations are the 6. else return 0
most important part for managing the data.
1. Searching is the algorithmic process of finding an 4.5.1.2 Time Complexity
item in a collection of items. The search result is
either true or false depending on whether or not From the analysis of above pseudocode, it can be
the item is present or not in the list. observed that with the growth of input list, average and
2. Sorting is ordering a list of objects according to a worst case complexities also grow linearly. In general, if
given condition or property. For example, ordering there are n elements in the list, the worst-case requires n
Chapter 4.indd 186 4/9/2015 9:54:24 AM

4.5 SEARCHING AND SORTING 187
comparisons. So, the complexity of linear search can be 7, 5, 2, 4, 3, 9

expressed in terms of some linear function. 5, 7, 2, 4, 3, 9
So, running time T(n) = O(n).
5, 2, 7, 4, 3, 9
5, 2, 4, 7, 3, 9
4.5.2 Binary Search
5, 2, 4, 3, 7, 9
Binary search relies on a divide-and-conquer strategy to After the first pass, the first largest element (9) is on the
find an element within an already sorted list. It compares top of array. So we have
the element to be searched with the middle element of
array. If the element is equal to the middle element, the 5, 2, 4, 3, 7, 9
search stops here and the position of element is returned. After the second pass, we get
If it is less or greater than the middle element, then 2, 4, 3, 5, 7, 9
the values of sub-array are adjusted accordingly. The
algorithm returns a unique value, that is, position of the After the third pass, we get
element in the list, if the element is present. 2, 3, 4, 5, 7, 9
After the fourth pass, we get
4.5.2.1 Pseudocode for Binary Search
2, 3, 4, 5, 7, 9
BINARY_SEARCH (A, value, left, right) After the fifth pass, we get
1. if right < left
2. return ‘not found’ 2, 3, 4, 5, 7, 9
3. mid = floor((right-left)/2)+left The sorted list is 2, 3, 4, 5, 7, 9
4. if A[mid] = value
5. return mid
6. if value < A[mid] 4.5.3.2 Time Complexity
7. return BINARY_SEARCH(A, value, left, mid-1)
8. else
9. return BINARY_SEARCH(A, value, mid+1, right) It can be seen from the above example that in the bubble
sort, each element passes one at a time and is placed in
its correct position.
4.5.2.2 Time Complexity So, the number of passes = (number of elements − 1)
= (n − 1).
The running time of the binary search is Q(log n).
Cost per pass = O(n).
Total cost = O(n)(n − 1).
4.5.3 Bubble Sort
So, the worst-case runtime complexity is O(n2).
Bubble sort algorithm is a comparison-based algorithm. The average-case and best-case runtime complexity is
It compares each element in the list with the next ele- O(n2).
ment. Elements are swapped if required. In every pass,
one element comes to its position. This process repeats
until a pass is made without disturbing any of the ele- 4.5.4 Selection Sort
ments from their position.
The selection sort first selects the smallest element from
4.5.3.1 Pseudocode for Bubble sort the unsorted list. Then that element is swapped with the
first element in the list. In the next step, the size of the
BUBBLE_SORT(A) list is reduced by one because one element is present at
1. for i 1 to n-1 its position. Next, the smallest element is swapped with
2. for j 1 to n-1 the second element in list, and so on.
3. If(A(j) > A(J+1))
4. Temp = A(j)
5. A(j) = A(j+1) 4.5.4.1 Pseudocode for Selection Sort
6. A(j+1) = Temp
SELECTION_SORT(A)
1. for j 1 to n-1
2. smallest j
Example 4.8
3. for i j + 1 to n
Sort the given elements (7, 5, 2, 4, 3 and 9) by using 4. if A[ i ] < A[ smallest]
bubble sort. 5. smallest i
6. Exchange A[ j ] A[ smallest]
After the first pass, we get
Chapter 4.indd 187 4/9/2015 9:54:25 AM

Example 4.9 Example 4.10

Sort the given elements (29, 64, 73, 34 and 20) by using Sorted part is bold in text and unsorted part is non-bold.
selection sort. Suppose 30, 10, 75, 33, 65 needs to be sorted using inser-
We have tion sort.
29, 64, 73, 34, 20 After the first pass, we get
After the first pass, we get 10, 30, 75, 33, 65
20, 64, 73, 34, 29 After the second pass, we get
After the second pass, we get 10,30,75, 33, 65
20, 29, 73, 34, 64 After the third pass, we get
After the third pass, we get 10, 30, 35, 75, 65
20, 29, 34, 73, 64 After the fourth pass, we get
After the fourth pass, we get 10, 30, 35, 65, 75
20, 29, 34, 64, 73 The sorted list is 10, 30, 35, 65, 75.
After the fifth pass, we get
20, 29, 34, 64, 73 4.5.5.2 Recurrence Relation of Insertion Sort
The sorted list is 20, 29, 34, 64, 73.
The recurrence relation for insertion sort is
 Q(1), n = 1
T (n) =  
4.5.4.2 Time Complexity T (n - 1) + Q(n), n > 1
Number of passes = (Number of elements − 1)
4.5.5.3 Time Complexity
Cost per pass = Swap + Comparisons = 1 + O(n)
Total cost = (n − 1)[1 + O(n)] In sorting, the most expensive part is the comparison of
So, the worst-case runtime complexity is O(n2). two elements. Zero comparisons are required to insert
first element, one for second element, two comparisons
The average-case and best-case runtime complexity is O(n2). for third element, and so on. Atmost N − 1 comparisons
are required for the last element in the list.
Complexity is: 1 + 2 + 3 + 4 … + (N − 1) = O(n2)
4.5.5 Insertion Sort
The insertion sort inserts each element into its proper The worst-case running time is O(n2)
position. It chooses one element, inserts it to its posi- The best-case running time is O(n)
tion and shifts the rest of the list by one location. This
process is repeated until no input element remains. Step- 4.5.6 Heap Sort
by-step procedure is given as follows.
1. Take an element from unsorted list. Heap sort is one of the comparison based sort that uses
2. Compare that element with sorted list from right heap data structure. It is a special case of selection sort.
to left. Its typical implementation is not stable, but can be made
3. Shift the list. stable. Heap can be built by the following two types:
Here, it is assumed that first element is already sorted. 1. Min-Heap: The parent node will be less than or
equal to its children nodes. So, the minimum ele-
ment will be at root.
4.5.5.1 Pseudocode for Insertion Sort 2. Max-Heap: The parent node will be greater than
or equal to its children nodes. The maximum ele-
INSERTION_SORT(A)
1. for j = 2 to n ment will be at the root.
2. key A [j]
3. // Insert A[j] into the sorted sequence A[1..j-1] 4.5.6.1 Heap Sort Algorithm for Sorting in
4. j i– 1 Increasing Order
5. while i > 0 and A[i] > key
6. A[i+1] A[i] 1. First build a max-heap from the input List.
7. i i– 1 2. In max heap, the largest element is at the root. So,
8. A[j+1] key swap the root with last element. Re-heapify the
Chapter 4.indd 188 4/9/2015 9:54:26 AM

tree. Repeat the above steps until the size of the 4.5.6.3 Pseudocode for BuildHeap Function
heap is greater than 1.
•• Heapify function: Heapify is an important BUILD_HEAP(A)
subroutine for manipulating max-heaps. Its 1. heapsize <- length(A)
2. for i <- floor( length/2 ) down to 1
inputs are an array A and an index i into the
3. Heapify(A, i)
array. When Heapify is called, it is assumed
that the binary tree rooted at left(i) and right(i) 4.5.6.4 Pseudocode for Heap Sort
is max_heap, but that A[i] may be smaller
than its children, thus violating the max-heap HEAP_SORT(A)
property. 1. BuildHeap(A)
•• Build Heap function: Heapify procedure 2. for i <- length(A) down to 2
is used in a bottom-up to convert an array A 3. exchange A[1] <-> A[i]
4. heapsize <- heapsize -1
into a max-heap. The elements in the subarray
5. Heapify(A, 1)
A[(n/2 + 1)⋅⋅⋅n] are all leaves of the tree, and
so each is a one-element heap to begin with. The
procedure BuildHeap goes through the remain- 4.5.6.5 Recurrence Relation of Heapify
ing nodes of the tree and runs Heapify on each Function
node. The recurrence relation of Heapify() is
  2n  
4.5.6.2 Pseudocode for Heapify Function T (n) ≤ T   + Q(1)
  3  
HEAPIFY(A, i)
1. le <- left(i) 4.5.6.6 Time Complexity
2. ri <- right(i)
3. if (le<=heapsize) and (A[le]>A[i]) The time complexity of Heapify() is O(log n). The time
4. largest <- le
5. else
complexity of BuildHeap() is O(n log n), and the overall
6. largest <- i time complexity of the heap sort is O(n log n).
7. if (ri<=heapsize) and (A[ri]>A[largest])
8. largest <- ri 4.5.6.7 Applications of Heap Sort
9. if (largest != i)
10. exchange A[i] <-> A[largest] 1. It sorts a nearly sorted (or k-sorted) array.
11. HEAPIFY(A, largest) 2. It sorts k largest (or smallest) elements in an array.
Example 4.11
Let {6, 5, 3, 1, 8, 7, 2, 4} be the list that we want to sort recursively checked if another swap is needed, to keep
from the smallest to the largest. (Note: For `building larger numbers above smaller numbers on the heap
the heap’ step, larger nodes do not stay below smaller binary tree.)
node parents. They are swapped with parents, and then 1. Build the heap:
Heap Newly Added Element Swap Elements

Nil 16
16 20
20 16 16 20
20 16 11 11
20 16 11 8 8
20 16 11 8 18
20 16 11 8 18 16 18
20 18 11 8 16 14
20 18 11 8 16 14 11 14
20 18 14 8 16 11
Chapter 4.indd 189 4/9/2015 9:54:28 AM

2. Sort the list:

Heap Swap Elements Delete Element Sorted Array Details
20 18 14 8 16 11 20, 11 Swap 20 and 11 in order to
delete 20 from the heap.
11 18 14 8 16 20 20 Delete 20 from the heap and
add to the sorted array.
11, 18, 14, 8, 16, 20 11, 18 20 Swap 11 and 18 as they are
not in order in the heap.
18 11 14 8 16 20 11 16 20 Swap 11 and 16 as they are not
in order in the heap.
17, 15, 16, 7, 10, 19 18 11 20 Swap 18 and 11 in order to
18 16 14 8 11 20 delete 17 from the heap.
11 16 14 8 18 20 18 20 18 Delete 18 from the heap and
11 16 14 8 18 20 11 14 20 18 Swap 11 and 14 as they are not
11 14 16 8 18 20 14 16 20 18 Swap 14 and 16 in order to
16 14 11 8 18 20 20 18 16 Delete 16 from the heap and
8 14 11 16 18 20 8 14 20 18 16 Swap 14 and 8 as they are not
14 8 11 16 18 20 14 11 20 18 16 Swap 14 and 11 in order to
10, 7, 15, 16, 17, 19 14 20 18 16 14 Delete 14 from the heap and
11 8 14 16 18 20 add to the sorted array.
11 8 14 16 18 20 11 8 20 18 16 14 Swap 11 and 8 in order to
8, 11, 15, 16, 17, 11 20 18 16 14 11 Delete 11 from the heap and
19 add to the sorted array.
7, 10, 15, 16, 17, 19 8 20 18 16 14 11 8 Delete 8 from the heap and add
8 11 14 16 18 20 to the sorted array.
MERGE_SORT
20 18 16 (A,
14 11p, r)Completed
1. if p < r
2. Then q = floor[(p + r)/2]
4.5.7 Merge Sort
3. MERGE (A, p, q)
Merge sort is a divide-and-conquer algorithm. It divides 4. MERGE (A, q + 1, r)
the input array in two subarrays, calls itself for the two 5. MERGE (A, p, q, r)
sub-arrays and then merges the two sorted arrays. The Pseudocode for MERGE function is given as follows:
MERGE(A, p, q, r) function is used for merging the two
arrays. The MERGE(A, p, q, r) is a key process that MERGE (A, p, q, r)
assumes that array[l⋅⋅⋅m] and array[m + 1 ⋅⋅⋅ r] are sorted 1. n1 ¬ q − p + 1
and merges the two sorted sub-arrays into one. Merge sort 2. n2 ¬ r − q
is a stable sort. It is not an in-place sorting technique. 3. create arrays L[1…n1 + 1] and R[1…n2 + 1]
4.5.7.1 Pseudocode for Merge Sort 4. For i ¬ 1 to n1

5. doL[i] ¬ A[p + i − 1]
MERGE_SORT (A, p, r)
6. For j ¬ 1 to n2
1. if p < r
2. Then q = floor[(p + r)/2] 7. Do R[j] ¬ A[q + j]
3. MERGE (A, p, q) 8. L[n1 + 1] ¬ ∞
4. MERGE (A, q + 1, r) 9. R[n2 + 1] ¬ ∞
5. MERGE (A, p, q, r)
10. i ¬ 1
Chapter 4.indd 190 4/9/2015 9:54:29 AM
11. j ¬ 1
2. n2 ¬ r − q
3. create arrays L[1…n1 + 1] and R[1…n2 + 1]
4. For i ¬ 1 to n1
5. doL[i] ¬ A[p + i − 1]
6. For j ¬ 1 to n2
7. Do R[j] ¬ A[q + j]
8. L[n1 + 1] ¬ ∞ 4.5.7.3 Time Complexity
9. R[n2 + 1] ¬ ∞
The above recurrence can be solved using either recur-
10. i ¬ 1 rence tree method or master theorem. It falls under case
11. j ¬ 1 2 of the master theorem, and the solution of the recur-
rence is O(n log n). The time complexity of merge sort
12. For k ¬ p to r
in all three cases (worst, average and best) is the same,
13. Do if L[i] R[j] as merge sort always divides the array in two sub-arrays
14. then A[k] ¬ L[i] and takes linear time to merge the two arrays.
15. i ¬ i + 1
16. Else A[k] ¬ R[j] 4.5.7.4 Applications of Merge Sort
17. j ¬ j + 1
1. Merge sort is useful for sorting linked lists in O(n
log n) time. Other n log n algorithms such as heap
4.5.7.2 Recurrence Relation of Merge Sort
sort and quick sort (average case n log n) cannot
The recurrence relation for merge sort is be applied to linked lists.
2. It is used in inversion count problem.
 Q(1), n = 1
 3. It is used in external sorting.
T (n) =   n  
2T   + Q(n), n > 1 Note: Merge sort is not preferable for smaller-size
  2   element array.
Example 4.12
Sort the given list of elements: (54, 26, 93, 17, 77, 31, 44, Step 2: After dividing the array, merge sort calls the
55 and 20). MERGE function to put the subarray into a sorted order.
Step 1: A recursive merge sort is called on the array Array A:
of elements. Merge sort is a recursive function which
divides an array into two parts; and calls itself recur- 54 26 93 17 77 31 44 55 20
sively to divide the array into sub-parts until each part
is having one element.
p r
Merge sort (A, 0, 8) 54 26 93 17 77 31 44 55 20
0 1 2 3 4 5 6 7 8
54 26 93 17 77 31 44 55 20
54 26 93 17 77 31 44 55 20
54 26 93 17 77 31 44 55 20
54 26
Merge (A, p, q, r)
54 26 93 17 77 31 44 55 20
26 54 93 17 77 31 44 20 55
26 54 93 17 77 31 44 20 55
17 26 54 77 93 20 31 44 55
Completed 17 20 26 31 44 54 55 77 93
Chapter 4.indd 191 4/9/2015 9:54:30 AM

PARTITION(S, p, r)
1. x <- S[r]
2. i <- p-1
192 Chapter 4: Algorithms 3. for j <- p to r-1
4. do if S[j] <= x
4.5.8 Quick Sort 5. then i <- i+1
6. swap S[i] <-> S[j]
Quicksort is also divide-and-conquer strategy of sorting a list 7. swap S[i+1] <-> S[r]
of elements like merge sort. This divides array of elements 8. return i+1
into sub-arrayS[p…r]. The concept is described as follows:
4.5.8.3 Recurrence Relation of Quick Sort
1. Divide: Partition S[p…r] into two sub-arrays
S[p…q − 1] and S[q + 1…r] such that each element 1. For worst case: The worst case of quick sort
of S[p…q − 1] is less than or equal to S[q], which occurs when the pivot we picked turns out to be
is, in turn, less than or equal to each element of the least element of the array to be sorted, in every
S[q + 1…r]. Compute the index q as part of this step (i.e. in every recursive call). A similar situa-
partitioning procedure. tion will also occur if the pivot happens to be the
2. Conquer: Sort the two sub-arrays S[p…q − 1] and largest element of the array to be sorted. Then
S[q + 1…r] by recursive calls to quicksort. recurrence relation of quick sort is
3. Combine: Since the sub-arrays are sorted in T (n) = T (1) + T (n - 1) + Θ(n)
place, no work is needed to combine them; the 2. For best case: The best case of quicksort occurs
entire array S is now sorted. when the pivot we picked happens to divide the
array into two almost equal parts, in every step.
4.5.8.1 Pseudocode for Quick Sort Thus, we have k = n/2 and n − k = n/2 for the
QUICKSORT(S, P, r) original array of size n. Then the recurrence rela-
1. If p < r tion of quick sort is given as
2. then q <- PARTITION(S, p, r)  n
T (n) = 2T   + Θ(n)
3. QUICKSORT(S, p, q-1)  2
4. QUICKSORT(S, q+1, r)
4.5.8.2 Pseudocode for Partition
The best-case and average-case running time of quick
PARTITION(S, p, r) sort = O(n log n).
1. x <- S[r]
2. i <- p-1 The worst-case running time of quick sort = O(n2).
3. for j <- p to r-1 Note: The worst case of quick sort occurs when the
4. do if S[j] <= x given elements are already sorted or almost sorted.
5. then i <- i+1
6. swap S[i] <-> S[j]
Example
7. 4.13
swap S[i+1] <-> S[r]
8. return i+1
Sort the given list of elements: (38, 81, 22, 48, 13, 69, 93, (a) Elements at the left of the pivot are less than or
14, 45, 58, 79, 72) equal to that of pivot.
Step 1: Choose the pivot element which is at position (b) Elements at the right of pivot are greater than that
[(left + right)/2]. of pivot.
Step 2: Now in partitioning phase: During the parti- The tree of recursive call is given as follows:
tioning process:
Quicksort (A,1,12)
38 81 22 48 13 69 93 14 45 58 79 72
Pivot = 69
14 58 22 48 13 38 45 69 93 81 79 72
Quicksort (A,1,7) Quicksort (A,9,12)

14 58 12 48 13 38 45 Pivot = 48 93 81 79 72 Pivot = 81
38 45 22 14 13 45 58 79 72 81 93

14 45 22 48 13 Pivot = 22 79 72 Pivot = 79
14 13 22 38 45

14 13 38 45 Pivot = 38
Pivot = 14
13 14 38 45
Chapter 4.indd 192 4/9/2015 9:54:31 AM

COUNTING_SORT(A[], B[], k)
1. for i = 1 to k do
2. C[i] = 0
3. for j = 1 to length(A) do AND SORTING
4.5 SEARCHING 193
4. C[A[j]] = C[A[j]] + 1
4.5.9 Randomized Quick Sort 5. for 2 = 1 to k do
6. C[i] = C[i] + C[i-1]
In the randomized version of quick sort, we impose a 7. for j = 1 to length(A) do
distribution on the input. This does not improve the B[C[A[j]]] = A[j]
C[A[j]] = C[A[j]] - 1
worst-case running time, but it improves the probability
of getting the average-case running time. Although this may look complicated, it is actually a
In this version, we choose a random key for the pivot. very simple and clever algorithm.
Assume that the procedure Random(a, b) returns a 1. An array A[ ] stores the initial data to be sorted.
random integer in the range [a, b); there are b − a + 1 2. An array C[ ] is used to count the occurrences of
integers in the range, and the procedure is equally likely the data values.
to return one of them. The new partition procedure 3. An array B[ ] is used to store the final, sorted, list.
simply implements the swap before actually partitioning. 4. The first for loop initializes C[ ] to zero.
5. The second for loop increments the values in C[ ],
4.5.9.1 Pseudocode for Randomized according to their frequencies in the data.
Quick Sort 6. The third for loop adds all the previous values,
making C[ ] contain a cumulative total.
RANDOMIZED_PARTITION (A, p, r)
7. The fourth for loop writes out the sorted data
1. i RANDOM (p, r) into the array B[ ].
2. Exchange A[p] A[i]
3. return PARTITION (A, p, r)
The pseudocode for randomized quick sort has the O(n + k), where n is the number of elements in the input
same structure as quick sort, except that it calls the ran- array and k is the range of input.
domized version of the partition procedure.
Note:
RANDOMIZED_QUICKSORT (A, p, r) 1. Counting sort is efficient if the range of input data
is not significantly greater than the number of
1. If p<r then objects to be sorted.
2. Q RANDOMIZED_PARTITION (A, p, r) 2. It is not a comparison-based sorting. Its running
3. RANDOMIZED_QUICKSORT (A, p, q) time complexity is O(n) with space proportional to
the range of data.
4. RANDOMIZED_QUICKSORT (A, q+1, r) 3. This can be used as subroutine to some other algo-
rithm. It is often used as a subroutine to another
4.5.9.2 Time Complexity sorting algorithm, such as radix sort.
4. It uses a partial hashing to count the occurrence of
The best- and average-case running time of randomized the data object in O(1).
quick sort = O(n log n). 5. It can be extended to work for negative inputs also.
The worst-case running time of randomized quick
sort = O(n2).
Note: The worst case of randomized quick sort occurs Example 4.14
when all the given elements are equal. Consider the data in the range 0 to 9.
4.5.10 Counting Sort Input data: 1, 4, 1, 2, 7, 5, 2
The steps to sort elements through counting sort are as
Counting sort is a sorting technique based on keys follows:
within a specific range. It works by counting the number 1. Take a count array to store the count of each
of objects having distinct key values (kind of hashing), unique object:
and then doing some arithmetic to calculate the position
Index: 0 1 2 3 4 5 6 7 8 9
of each object in the output sequence.
Count: 0 2 2 0 1 1 0 1 0 0
4.5.10.1 Pseudocode for Counting Sort 2. Modify the count array such that each element at
each index stores the sum of the previous counts:
COUNTING_SORT(A[], B[], k)
Index: 0 1 2 3 4 5 6 7 8 9
1. for i = 1 to k do
2. C[i] = 0 Count: 0 2 4 4 5 6 6 7 7 7
3. for j = 1 to length(A) do The modified count array indicates the position of each
4. C[A[j]] = C[A[j]] + 1 object in the output sequence.
5. for 2 = 1 to k do
6. C[i] = C[i] + C[i-1]
7. for j = 1 to length(A) do
B[C[A[j]]] = A[j]
C[A[j]] = C[A[j]] - 1
Chapter 4.indd 193 4/9/2015 9:54:32 AM
Table 4.1 | Sorting techniques

Sorting Average Best Time Worst Time Space Stability In-Place
Algorithm Time Sorting
Bubble sort O(n2) O(n2) O(n2) Constant Stable Yes
Selection sort O(n2) O(n2) O(n2) Constant Stable Yes
2 2
Insertion sort O(n ) O(n) O(n ) Constant Stable Yes
Heap sort O(n × log(n)) O(n × log(n)) O(n × log(n)) Constant Instable Yes
Merge sort O(n × log(n)) O(n × log(n)) O(n × log(n)) Depends Stable No
Quick sort O(n ×log(n)) O(n × log(n)) O(n2) Constant Stable Yes
Count sort O(n) O(n) O(n) Constant Stable No
3. Output each object from the input sequence fol- • It does not have any parallel edge.
lowed by decreasing its count by 1.
(a) Process the input data: 1, 4, 1, 2, 7, 5, 2. Posi-
tion of 1 is 2.
(b) Put data 1 at index 2 in the output. Decrease
count by 1 to place the next data 1 at an index 1
smaller than this index.
(c) Put data 2 at index 4 in the output. Decrease Figure 4.5 | Simple graph.
count by 1 to place the next data 2 at an index 3
smaller than this index. 2. Multigraph: A graph with a self-loop and parallel
(d) Now, move to the next element 3 which has count 4, edges is called a multigraph (Fig. 4.6).
and index 4 is already filled by 2 which means ele-
ment 3 does not exist in the given input list.
(e) Follow this process for all the elements.
4.5.11 Comparison of Sorting Techniques
The comparison of the various sorting techniques is

Figure 4.6 | Multigraph.
given in Table 4.1.
4.6 GRAPH 4.6.2 Types of Simple Graph
4.6.2.1 Null Graph

A graph G = (V, E) is a set of vertices and edges where
each edge consist of pair of vertices.
If in the given graph there is no edge, then it is called a
1. Finite set of vertices are known as nodes of graph. null graph. In other words, a null graph does not have
2. (u, v) is a pair of vertices called edge. any edge between any two vertices (Fig. 4.7).
3. For directed graph (u, v) is an ordered pair that
represents an edge from node u to node v. A
D
4.6.1 Types of Graph
B
There are two types of graphs:
1. Simple graph: A graph is called a simple graph E
C
(Fig. 4.5) if
• It contains no self-loop. Figure 4.7 | Null graph.
Chapter 4.indd 194 4/9/2015 9:54:33 AM

4.7 GREEDY APPROACH 195
4.6.2.2 Regular Graph graph. If adj[i][j] = 1, this indicates there exists an edge
between vertices i and j; otherwise, the value will be 0.
A graph is called a regular graph if all of its vertices have The adjacency matrix for undirected graph is symmet-
the same degree (Fig. 4.8). ric. Weights can also be represented by this matrix. The
value adj[i][j] = w means weight of edge from i to j is w.
Vertex 1 2 3 4 5
1 2
1 0 1 1 1 0
5 2 1 0 1 1 1
3 1 1 0 1 0
1-Regular 2-Regular 3-Regular 4 1 1 1 0 1
Figure 4.8 | Regular graph.
3 4
5 0 1 0 1 0
If a graph has n number of vertices and it is k-regular,

This type of representation is very easy for implementa-
where k is a positive constant, then the number of edges
tion purpose. O(1) time is required to remove an edge.
= (n × k)/2.
Testing that an edge exists between two nodes or not
can be computed in O(1) time complexity.
4.6.2.3 Complete Graph
Space requirement is more for both dense and sparse
In the given graph, if every vertex is adjacent to all other matrices is O(V 2). Addition of a vertex takes O(V2) time.
remaining vertices, then that graph is called a complete
graph (Fig. 4.9). If a graph is a complete graph and is 4.6.3.2 Adjacency List
having n vertices, then it is called a Kn graph.
To implement adjacency list, an array of linked list is
used. Array size is kept equal to the total number of ver-
tices. For each vertex i, linked list of its adjacent nodes
is stored in array[i]. Weights of edges can also be stored
in linked list nodes. The representation of adjacency list
K4 graph K5 graph is as follows:
Figure 4.9 | Complete graphs.
A complete graph having n vertices has [n ×(n −1)]/ 1 2 3 4 Null
2 edges. 2 1 3 4 5 Null
Note: The number of simple graphs possible with n ver- 3 1 2 4 Null
tices = 2[n(n −1)]/2. 4 1 2 3 5 Null
5 2 4 Null
4.6.3 Graph Representation
The space required for adjacency list is: O(V+E),
Graph data structure is useful in many real-time applica- which is less than that of adjacency matrix. In worst-
tions like they are used to represent complex networks. case, space consumed is O(V2). Addition of vertex is
The network represented by graph can be of city defin- easy, but finding an edge between vertices, say u and v,
ing paths, of telephone or circuit network. is not efficient. It requires O(V) complexity.
The following two are the most commonly used repre-
sentations of a graph:
1. Adjacency matrix
4.7 GREEDY APPROACH
2. Adjacency list
Incidence matrix and incidence list are two other rep- A greedy algorithm is an algorithm for solving prob-
resentations for graph. Any representation can be chosen lems in an optimised way. In this approach,the algo-
depending on the situation. rithms execute repeatedly to maximize the outcome,
considering the local conditions for some global prob-
4.6.3.1 Adjacency Matrix lem. For some problems, it guarantees the optimal
solution while for some it does not. The main strat-
An adjacency matrix is a two-dimensional array of size egy is to implement the problem so that it requires
V ×V, where V represents the number of vertices in a minimal resources.
Chapter 4.indd 195 4/9/2015 9:54:35 AM

The important terms are given as follows: knapsack exactly; otherwise, we could add a fraction of
1. Solution space: The set of all possible solutions one of the remaining objects and increase the value of
for given n inputs is called solution space. the load. Thus, in an optimal solution, nSi = 1xiwi = W.
2. Feasible solution: The feasible solution is one of
the solutions from the solution space which satis- 4.7.2.1 Pseudocode for Fractional
fies the given condition.
Knapsack
3. Optimal solution: The optimal solution is one
of the feasible solutions which optimizes our goal. Greedy-fractional-knapsack (w, v, W)
1. for i =1 to n
4.7.1 Applications of Greedy Approach
2. Do x[i] =0
The following are the applications of the greedy approach: 3. weight = 0
4. while weight < W
1. Knapsack problem (fractional)
5. Do i = best remaining item
2. Job sequence with deadline
6. if ( weight + w[i] ≤ W)
3. Huffman coding
4. Minimum cost spanning tree: 7. Then x[i] = 1
a. Kruskal’s algorithm 8. weight = weight + w[i]
b. Prim’s algorithm 9. else
5. Singlesource shortest path:
10. x[i] = (w - weight) / w[i]
a. Dijkstra’s algorithm
b. Bellman—Ford algorithm 11. weight = W
12. Return x
4.7.2 Fractional Knapsack
In this problem, items are given along with its weight
and profit. The target is to maximize the profit consider- If the ratio of vi/wi is already sorted in decreasing order,
ing the weight constraint. Unlike 0-1 knapsack problem, then the time taken by the while loop will be O(n). So,
fraction of item can be considered. Fractional knapsack the total time required will be O(n log n). If heap data
problem can be solved by Greedy approach. structure is used, which keeps highest value/weight ratio
Greedy Algorithm: The solution can be obtained by at root. The construction of heap will take O(n) time
making different choices. At each stage, the best possible and while loop will take O(log n) time (heap property is
choice is made. In fractional knapsack, first of all profit restored for each root removal). This does not improve
value/weight ratios are calculated and then sorted in the worst case, but it is faster if small number of items
descending order. Item with highest value/weight ratio are to be filled in knapsack.
is chosen and added in the collection. The maximum
weight is checked after adding each item. If the entire Example 4.16
item cannot be included, then fraction of it is added to
Consider the following details:
the collection.
1. Knapsack size = 20
2. Number of objects = 3
Example 4.15
There are n items in a store. For i = 1, 2, …, n, item i Object Obj1 Obj2 Obj3
has weight wi > 0 and worth vi > 0. A thief can carry a Profit 25 24 15
maximum weight of W pounds in a knapsack. In this ver- Weight 18 15 10
sion of a problem, the items can be broken into smaller
pieces, so the thief may decide to carry only a fraction xi Solution:
of object i, where 0 ≤ xi ≤ 1. Item i contributes xiwi to
the total weight in the knapsack, and xivi to the value Step 1: Calculate ratio (profit/weight):
of the load. Object Obj1 Obj2 Obj3
In symbol, the fraction knapsack problem can be Profit 25 24 15
stated as follows.
Weight 18 15 10
Maximize nSi = 1xivi subject to constraint nSi = 1xiwi
≤ W. It is clear that an optimal solution must fill the
Ratio 1.38 1.6 1.5
Chapter 4.indd 196 4/9/2015 9:54:35 AM

4.7 GREEDY APPROACH 197
Step 2: Put according to decreasing ratio (profit/ 4.7.3.2 Pseudocode for Huffman Coding
weight):
HUFFMAN(f[1...n])
Object Obj2 Obj3 Obj1 1. T = empty binary tree
2. Q = priority queue of pairs (i, f[i]),
Profit 24 15 25
i = 1...n, with f as comparison key
Weight 15 10 18 3. for each k = 1...n − 1
Ratio 1.6 1.5 1.38 4. i = extractMin(Q)
5. j = extractMin(Q)
Step 3: Put items into the knapsack till the knapsack 6. f[n + k] = f[i] + f[j]
is full: 7. insertNode(T, n + k) with children i, j
8. insertRear(Q,(n + k, f[n + k]))
(a) Put object 2 into the knapsack; it will add (value = 9. return T
24 and weight = 15) to the knapsack. Knapsack has
5 units capacity remaining.
(b) Now object 3 cannot be put completely into the 4.7.3.3 Time Complexity
knapsack, so put a fraction of it. Add (value = 7.5
and weight = 5) to the knapsack. Knapsack is com- From the above pseudocode, we can see that with each
pletely full, which means no other object can be put iteration,the problem size is reduced by 1. Hence, there
into it. are exactly n iterations. The ith iteration consists of
locating the two minimum values in a list of length n −
Step 4: Total knapsack value = 31.5 (which is optimum).
i + 1. This is a linear operation, and so Huffman’s
algorithm clearly has a time complexityof O(n2).
4.7.3 Huffman Coding However, it would be faster to sort the weights ini-
tially, and then maintain two lists. The first list consists of
Huffman coding is a common technique of encoding, weights that have not been combined yet, and the second
includinglossless data compression. The algorithm’s list consists of trees that have been formed by combining
output can be viewed as a variable-length code table weights. This initial ordering is obtained at the cost of
for encoding a source symbol (such as a character in a O(n log n). Obtaining the minimum two trees at each step
file). Huffman coding is based on the frequency of occur- then consists of two comparisons (comparing the heads of
rence of a data item (pixel in images). The main aim the two lists, and then comparing the larger item with the
of Huffman coding is to encode the data that occurs item after the smaller). The ordering of the second list can
frequently using less number of bits. Code books store be maintained cheaply by using a binary search to insert
codes that are constructed for images. Both the code new elements. Since at step i there are i − 1 elements in
book and encoded data are transmitted to decode the the second list, O(log i) comparisons are needed for inser-
information. tion. Over the entire duration of the algorithm, the cost of
Huffman codes can be constructed using following two keeping this list sorted is O(n log n). Therefore, the overall
ideas: time complexity of Huffman’s algorithm is O(n log n).
1. For an optimum code, symbols that have high
probability should be assigned shorter code words. Example 4.17
2. To make optimum prefix code, two symbols that
In the following example five different symbols are given
occur least frequently should be assigned same
along with their frequency.
length.
Symbol Frequency
4.7.3.1 Principle of Huffman Codes
A 24
1. Starting with the two least probable symbols, g B 12
and d, of an alphabet A, if the code word for g is C 10
[m]0, the code word for d would be [m]1, where [m] D 8
is a string of 1s and 0s.
2. Now, the two symbols can be combined into a E 8
group, which represents a new symbol y in the
There are a total of 186 bits (with 3 bits per code word).
alphabet set.
3. The symbol y has the probability P(g ) + P(d ). Here, the least frequency symbols are E and D. First
Recursively, determine the bit pattern [m] using of all, we will connect them and frequency of node will
the new alphabet set. become 16. Then B and C are combined to make the
Chapter 4.indd 197 4/9/2015 9:54:36 AM

frequency 22. These will be two new nodes. In the next 3. At each step, an edge is added to the graph that
step, these two nodes are brought together and act as has minimum weight and is the neighbour to the
subordinate to the root node. A also acts as subordinate previous vertices.
to root node.
Steps of Prim’s Algorithm
1. Find the minimum weight edge from graph and
4.7.3.4 Code Tree According to Huffman
mark it.
2. Consider the neighboring edges of the selected
edges and mark the minimum weight edge among
those edges.
0 62 1 3. Continue this until we cover n-1 edges.
4. The resultant tree is minimum spanning tree and it
A
should always be connected.
24 0 38 1 Input: Undirected connected weighted graph G = (V, E )
in adjacency list representation, source vertex s in V and
w is an array representing weights of edges.
Output: p[1⋅⋅⋅|V |], representing the set of edges com-
0 22 1 16 1
0
posing an MST of G.
B C E
Note: The field p(v) names the parent of v in the tree.
D
12 10 8 8
Key(v) is the minimum weight of any edge connect-
ing v to a vertex in tree. Key(u)←∞ means there is no
Symbol Frequency Code Code Total such edge.
Length Length
PRIM (G, w, s)
A 24 0 1 24 1. for each u in V(G)
B 12 100 3 36 2. do key(u) ∞
3. π(v) NIL
C 10 101 3 30
4. key [r] 0
D 8 110 3 24 5. Q V [G]
E 8 111 3 24 6. while Q != empty
7. do u Extract-Min(Q)
8. for each v in Adjacent[u]
9. do if v ∈ Q and w(u, v) < key [u]
The total length is 138 bits.
4.7.4 Minimum Spanning Tree 10. then π[v] u

11. key [v] w(u, v)
Spanning tree is a sub-graph that connects all the verti-
ces together. A single graph can have multiple spanning Time Complexity
trees. Minimum spanning tree for a weighted undirected
graph is the spanning tree with minimum total weights. Prims algorithm use min heap data structure and its
The weight of spanning tree is addition of weights of all time complexity is O(VlogV+E logV ).
the edges in the spanning tree. An MST with N vertices, Maximum edges in tree, E = V − 1
has N − 1 edges for a given graph. Therefore, O(VlogV + (V − 1)logV) = O(2VlogV − logV)
4.7.4.1 Prim’s Algorithm Thus, complexity = O(VlogV )
Prim’s algorithm is a method to find minimum spanning Problem 4.5: Use Prim’s algorithm to find an MST in
tree. In this algorithm minimum spanning tree formed the following weighted graph. Use alphabetical order
should always be connected. Prim’s algorithm is an to break ties.
example of Greedy approach. b 5 d
Properties of Prim’s Algorithm 2 2
Prim’s algorithm has the following properties: a 6 3 1 z

1. Prime’s algorithm always results into a single tree. 3 4
2. Tree grows until it covers all the vertices of the
c 2 e
graph.
Chapter 4.indd 198 4/9/2015 9:54:37 AM

KRUSKAL(V, E, w)
1. A ¬ {φ} Set A will ultimately contain the
edges of the MST 4.7 GREEDY APPROACH 199
2. For each vertex v in V[G]
Solution: Prim’s algorithm will proceed as follows. 3. Do MAKE-SET(v)
4. Sort E into non-decreasing order by weight w
First we add edge {d, e} of weight 1.
5. For each edge (u, v)∈ E, taken in
1.
2. Next, we add edge {c, e} of weight 2.
non-decreasing order by weight w
3. Next, we add edge {d, z} of weight 2.
4. Next, we add edge {b, e} of weight 3. 6. do if FIND-SET(u) ≠ FIND-SET(v)
5. And finally, we add edge {a, b} of weight 2. 7. Then A ¬ A ∪ {(u, v)}
8. UNION(u, v)
This produces a MST of weight 10.
9. Return A
b d
2 2 Time Complexity
a 1 z The edge weight can be compared in constant time.
3
Initialization of priority queue takes O(E log E ) time
by repeated insertion. At each iteration of while loop,
minimum edge can be removed in O(log E ) time, which
c 2 e
is O(log V ), since the graph is simple. The total running
time is O((V + E ) log V ), which is O(E log V ) since the
4.7.4.2 Kruskal’s Algorithm graph is simple and connected.
One another algorithm to find the minimum spanning Example 4.18

tree is Kruskal’s. This method considers all minimum
Let us use the Kruskal’s algorithm to solve the following
edges of graph one by one in increasing order and gives
example.
a resultant minimum spanning tree. In Kruskal’s algo-
rithm, it is not necessary for a tree to be connected. b 8 c 7 d
Implementation steps: 4 2 9
1. Start from the source node. i e
a 4 14
2. Scan the weights and include the minimum weights
7 6
to the tree in increasing order.
8 10
3. Stop if N − 1 edges are included.
h g f
Uses a disjoint-set data structure to determine whether 1 2
an edge connects vertices in different components.
Step 1: Take only the vertices of graph.
Data Structure
b c d
Before formalizing the above idea, let us quickly under-
stand the disjoint-set data structure:
1. Make-SET(v): It creates a new set whose only i
a e
member is pointed to by v. Note that for this oper-
ation, v must already be in a set.
2. FIND-SET(v): It returns a pointer to the set
containing v. h g f
3. UNION(u, v): It unites the dynamic sets that
contain u and v into a new set that is a union of Step 2: Choose the minimum weight edge from the
these two sets. original graph and draw that edge in disconnected graph
containing n connected components as shown below:
Algorithm
Starting with an empty set A, select the shortest edge b c d
that has not been chosen or rejected at every step,
regardless of the position of this edge in the graph. The
pseudocode for Kruskal algorithm is given: a i e
KRUSKAL(V, E, w)
1. A ¬ {φ} Set A will ultimately contain the
edges of the MST h g f
2. For each vertex v in V[G] 1
3. Do MAKE-SET(v)
4. Sort E into non-decreasing order by weight w
5. For each edge (u, v)∈ E, taken in
non-decreasing order by weight w
Chapter 4.indd6.
199 do if FIND-SET(u) ≠ FIND-SET(v) 4/9/2015 9:54:39 AM
Step 3: Now select the next minimum weight edge and 2. Calculate the temporary distance of neighbours of
draw the corresponding edge in disconnected graph. active nodes. Distance is calculated by adding up
the weights associated with those edges.
b c d 3. Update the distance, and set the current node an
2 antecessor if such a calculated distance of a node is
smaller as the current one. This step is also called
i update and is Dijkstra’s central idea.
a e
4. Set the node as active node which has minimal tem-
porary distance. Mark the distance as permanent.
5. Repeat steps 2 to 4 until there are no nodes left
h g f with a permanent distance, whose neighbours still
1 2 have temporary distances.
Here, there are two edges containing weight 2, add all
possible edges of same weight until they form loop in the Dijkstra’s Pseudocode
tree (since tree doesnot contain any loop but a graph
DIJKSTRA (G , S):
may contain). That is why edge(g,f) and edge(i,c) both 1. for each vertex v in Graph:
are added. 2. distance[v] := infinity
Step 4: Continue this process and check for the next 3. previous[v] := undefined
smallest edge and add it into tree. Do this till we get 4. distance[ S ] := 0
(n − 1) edges in the tree. 5. Q := the set of all nodes in Graph
6. While Q is not empty:
The final minimum spanning tree would be: 7. u := node in Q with smallest distance[ ]
8. remove u from Q
b c d 9. for each neighbour v of u:
10. alt := distance[u] + distance_between(u, v)
11. If alt < distance[v]
i e 12. distance[v] := alt
a
13. previous[v] := u
14. Return previous[ ]
h g f
Time Complexity
Like Prim’s algorithm, Dijkstra’s algorithm runs in O(E
4.7.5 Single-Source Shortest Path log V) time.
1. Dijkstra’s algorithm with list: O(V2)
Given a weighted graph G, find the shortest path from 2. Dijkstra’s algorithm with modified binary heap:
a given vertex to every other vertex in G. Note that we O((E + V) log V )
can solve this problem quite easily with BFS traversal in 3. Dijkstra’s algorithm with Fibonacci heap: O(E +
the special case when all weights are equal. The Greedy V log V )
approach to this problem is repeatedly selecting the best
choice from those available at that time. Example
The following are the steps:
1. Start from the source node; find the distance of all
4.7.5.1 Dijkstra’s Algorithm
nodes reachable from the source.
Dijkstra’s algorithm is a single source shortest path algo- 2. Then pick the node which has the minimum dis-
rithm. It calculates the shortest distance starting from tance, from the pool of nodes. Source
the start node till another node in the graph. While cal- 3. Calculate again the distance from the selected A B C
culating the shortest distance, it excludes the longer dis- node, and follow the above two steps till the entire ∞ ∞ ∞ ∞
tance paths. Dijkstra’s algorithm does not operate on nodes are selected. Nil Nil N
negative weights. For calculating shortest distance with O 10 3 ∞
2 A Nil A A N
negative edges, Bellman-Ford algorithm is used. Steps to B D
10 O 4
perform Dijkstra’s algorithm are as follows:
1 4 8 7 9 AC Nil C
1. Assign all the nodes with distance infinity and 0 to A
3 O
initial node.Distance of start node s permanent and Nil
3 ACB
of all the other nodes is temporary.Set start node
C E O
as active node.
ACBD Nil
Chapter 4.indd 200 4/9/2015 9:54:40 AM

4.8 GRAPH TRAVERSAL 201
Source to know that the node has been explored before or not.
A B C D E The re-traversal becomes more for dense graph, which
∞ ∞ ∞ ∞ ∞ Cost increase computation time. Tree traversal is a special
Nil Nil Nil Nil Parent case of graph traversal. The following are two graph
O 10 3 ∞ ∞ traversal methods:
2 A Nil A A Nil Nil
B D 1. Breadth-first traversal (BFT)
Source O 4 11 6 2. Depth-first traversal (DFT)
A 4 B8 C 7 D 9 E AC Nil C C C
2
∞ ∞ ∞ ∞ ∞ Cost O 6 6 B D
4.8.1 Breadth-First Traversal
Nil Nil Nil Nil ParentNil
ACB B C A 1
CO 10 3 ∞E ∞
3
O 6 3 C The breath-first
E traversal starts from the root node and
Nil A A Nil Nil ACBD Nil 3 traverses all its neighbours. Then for each neigh-
C then
O 4 11 6 bour, its corresponding unvisited neighbour is processed.
Nil C C C
2 The pseudocode for BFT is given as follows:
O 6 6 B D
Nil B C A 1 BST (V)
1. visited(V) =1
O 6 3 C
3
E 2. add(V, Q)
Nil C 3. while(Q is not empty)
4. u = delete(Q)
4.7.5.2 Bellman—Ford Algorithm 5. for all x adjacent to u
6. if( x is not visited)
The Bellman—Ford algorithm is used to compute 7. visited(x) = 1
single source shortest path to all other vertices of 8. add(Q, x)
weighted directed graph. This algorithm also consid-
ers negative weights of graph edges. This algorithm 4.8.1.1 Time Complexity
is slower but is capable of even finding a negative
weight cycle in graph. The time complexity of BFT is O(V + E), where V is
the number of vertices in the graph and E is the number
Bellman-Ford Pseudocode of edges in the graph.
1. d[s] ← 0 4.8.1.2 Application of BFT

2. for each v V – {s}
3. do d[v] ← ∞ The following are the applications of BFT. It can be
4. for i ← 1 to |V| – 1 do used to find out
5. for each edge (u, v) E do
6. if d[v] > d[u] + w(u, v) then 1. whether the given graph is connected.
7. d[v] ← d[u] + w(u, v) 2. the number of connected components in a graph.
8. [v] ← u 3. whether the given graph contains a cycle.
9. for each edge (u, v) E 4. the shortest path if all the edges have the same weight.
10. do if d[v] > d[u] + w(u, v) 5. whether the given graph is a bipartite graph.
11. then report that a negative-weight
cycle exists
Problem 4.6: Consider the graph G = (V, E ) and
find the BFT starting node A.
Time Complexity
Graph (G) A
The time complexity of Bellman-Ford algorithm is
O(VE), where E = V2, so the worst-case running time of B C
Bellman—Ford algorithm is O(V3).
D E F G
4.8 GRAPH TRAVERSAL H I
J
Graph traversal means visiting all the nodes in a graph in
some order. In graph traversal, some nodes may be vis- Solution: A is the starting node and Q is a queue data
ited more than once. This is because it is not necessary structure to store the values.
Chapter 4.indd 201 4/9/2015 9:54:41 AM

BFT (A): 10. u = delete (Q) → I

1. Visited (A) = 1 For all adjacent nodes to I which are not already
Add (Q, A) added in the queue, Add (Q, X), where X is the
adjacent node to I.
A
J
2. u = delete (Q) → A
11. u = delete (Q) → J
For all adjacent nodes to A which are not already
For all adjacent nodes to J which are not already
added in the queue, Add (Q, X), where X = (B, C)
added in the queue, Add (Q, X), where X is the
is the adjacent node to A.
adjacent node to J.
B C
3. u = delete (Q) → B
12. The queue is empty, which means graph is tra-
For all adjacent nodes to B which are not already
versed completely.
added in the queue, Add (Q, X), where X = (D, E)
is the adjacent node to B.
C D E
4.8.2 Depth-First Traversal
4. u = delete (Q) → C
As already discussed, there are many ways of tree tra-
For all adjacent nodes to C which are not already
versal. Consider the tree given in Fig. 4.10:
added in the queue, Add (Q, X), where X = (F, G)
is the adjacent node to C.
Tree
D E F G
5. u = delete (Q) → D I Root
For all adjacent nodes to D which are not already
added in the queue, Add (Q, X), where X = (H) is
the adjacent node to D. F K
E F G H
6. u = delete (Q) → E A H Z
For all adjacent nodes to E which are not already
added in the queue, Add (Q, X), where X = (F) is
the adjacent node to E. D
F G H Figure 4.10 | DFT of a tree.
7. u = delete (Q) → F
For all adjacent nodes to F which are not already Preorder traversal for given tree: I F A D H K Z
added in the queue, Add (Q, X), where X = (I) is Preorder traversal also results in depth first traversal.
the adjacent node to F. This is because the traversal goes deeper before travers-
G H I ing its siblings. In this tree, the descendants of F, that
is, A, H, D are traversed first and after that sibling of F,
8. u = delete (Q) → G that is, K are traversed.
For all adjacent nodes to G which are not already
added in the queue, Add (Q, X), where X = (I) is
the adjacent node to G. 4.8.2.1 Pseudocode for DST
H I DFT (V)
1. visited (V) =1
9. u = delete (Q) → H 2. for all x adjacent to V
For all adjacent nodes to H which are not already 3. if (x is not visited)
added in the queue, Add (Q, X), where X = (J) is 4. visited (x) =1
the adjacent node to H. 5. DFT (x)
I J
The above is the recursive code for DFT.
Chapter 4.indd 202 4/9/2015 9:54:42 AM

4.9 DYNAMIC PROGRAMMING 203

4. POP (S) → D
The time complexity of DFT is O(V + E ), where V is For all adjacent nodes to D which are not already
the number of vertices in the graph and E is the number added in the stack, PUSH (S, X). Every neighbour
of edges in the graph. element will be stored in the stack from right to left.
Here, X = (H) is the adjacent node to D.
4.8.2.3 Application of DFT H E C
The following are the applications of BFT. It can be 5. POP (S) → H
used to find out For all adjacent nodes to H which are not already
added in the stack, PUSH (S, X). Every neighbour
1. whether the given graph is connected. element will be stored in the stack from right to left.
2. the number of connected components in a graph. Here, X = (F, G) is the adjacent node to H.
3. whether the given graph contains a cycle.
4. whether the given directed graph is strongly F G E C
6. POP (S) → F
connected.
For all adjacent nodes to F which are not already
Problem 4.7: Consider the graph G = (V, E) and find added in the stack, PUSH (S, X). Every neighbour
the DFT starting node A. element will be stored in the stack from right to left.
Here, X = (F) is the adjacent node to F.
Graph(G) A
G E C
7. POP (S) → G
B C For all adjacent nodes to G which are not already
D E F G element will be stored in the stack from right to left.
Here, X = (F) is the adjacent node to G.
H E C
8. POP (S) → E
Solution: A is the starting node and S is a stack data For all adjacent nodes to E which are not already
structure to store the values. Stack is internally used added in the stack, PUSH (S, X). Every neighbour
by the recursive call. element will be stored in the stack from right to left.
Here, X = (F) is the adjacent node to E.
DFT (A)
1. Visited (A) =1 C
PUSH(S, A) 9. POP (S) C
A For all adjacent nodes to C which are not already
2. POP(S) → A element will be stored in the stack from right to left.
For all adjacent nodes to A which are not already Here, X = (F) is the adjacent node to C.
element will be stored in the stack from right to left.
Here, X = (B, C) is the adjacent node to A. 10. The stack is empty, which means the graph is
traversed completely.
B C
3. POP(S) → B
For all adjacent nodes to B which are not already 4.9 DYNAMIC PROGRAMMING
element will be stored in the stack from right to left.
Here, X = (D, E) is the adjacent node to B. Dynamic programming is a method for solving complex
problems by breaking them down into simpler sub-prob-
D E C lems. It is applicable to problems exhibiting the proper-
ties of overlapping sub-problemsand optimal substructure.
Chapter 4.indd 203 4/9/2015 9:54:43 AM

Dynamic programming algorithms are used for optimiza- The total running time = O(2n).
tion (e.g. finding the shortest path between two points, or Note: Most of the recursive programs contain very less
the fastest way to multiply many matrices). A dynamic number of distinct calls and more number of repeated sub-
programming algorithm will examine all possible ways to problems or function calls. These problems are known as
solve the problem and will pick the best solution. overlapping sub-problems.
4.9.1 Applications of Dynamic Programming

4.9.2.2 Dynamic Solution of Fibonacci
The following are the applications of dynamic pro‑
gramming: In dynamic programing technique, we solve every problem
1. Fibonacci series exactly once and store the results in memory. That result
2. 0/1 knapsack is used later to get an optimum output (Fig. 4.11).
3. Largest common subsequence
Fib (6)
4. All-pair shortest path
4.9.2 Fibonacci Series Fib (5) 4
Fibonacci series is the sequence produced by adding the

previous two numbers in the sequence. First two num- Fib (4) 3
bers in sequences are 0 1 or 1 1. The recurrence relation
Array 0 1 1 2 3 5 8
to find the next term in the sequence is given below:
Fib (3) 0 1 2 3 4 5 6
Fn = Fn -1 + Fn -2 2
with initial values F1 = 0 and F2 = 1 or F1 = 1 and

F2 = 1. Fib (2) 1
Example: Fibonacci series 0, 1, 1, 2, 3, 5, 8, 13, 21, ….

Fib (1) Fib (0)
Figure 4.11 | Dynamic programming approach

4.9.2.1 Recurrence Relation of Fibonacci
0, if n < 0 for Fibonacci.

fib(n) = 1, if n = 1

fib(n - 1) + fib(n - 2), if n > 1
Pseudocode for Dynamic Solution of
Fibonacci Series
1. Algorithm Fast-Fibonacci(n)
Example 4.19 2. Let fib[0] and fib[1] be 1
3. for each i from 2 to n, do
The solution of Fibonacci through recursion tree is
4. Let fib[i] = fib[i - 2] + fib[i - 1]
shown as follows: 5. end of loop
Fib(6) 6. return fib[n]
Time Complexity
Fib(5) Fib(4)
1. Fibonacci series without dynamic programing
[space = O(n) and time = O(2n)]
Fib(4) Fib(3) Fib(3) Fib(2) 2. Fibonacci series with dynamic programing [space =
Level 6 O(n) and time = O(n)]
Fib(3) Fib(2) Fib(2) Fib(1) Fib(2) Fib(1)Fib(1) Fib(0) 4.9.3 0/1 Knapsack
One of the problems in combinatorial optimization is

k napsack problem. In this problem, the target is to
m aximize the profit by choosing items. Each item is
a ssociated with a weight and profit. The collection
Total number of nodes = (2n −1). of items is formed such that the chosen total weight
So, the total number of function calls = 2n, and one should be less than or equal to the given limit. The
function call takes O(1) time. profit can be as large as possible.
Chapter 4.indd 204 4/9/2015 9:54:44 AM

4.11 CONCEPTS OF COMPLEXITY CLASSES 205
4.9.3.1 Recurrence Relation 0/1 Knapsack 4.9.4.3 Time Complexity



The recurrence 0, relation of 0/1 Knapsack problem is = 0 or n =Complexity
if m 0 LCS with LCS with


KS(n, m) = KS(n - 1, m) if m < w[n ]
Recursive Dynamic
 Solution Programing
Max[{KS(n - 1, m - w[n ] + p[n ])} otherwise
  Time If (m = n) T(n) = O(mn)
  {KS(n - 1, m) + 0}], LCS (m, n) = O(2n)
0, if m = 0 or n = 0

KS(n, m) = KS(n - 1, m) if m < w[n ] Else
 LCS(m, n) = O(2m+n)
Max[{KS(n - 1, m - w[n ] + p[n ])} otherwise
 Space O(m + n) O(mn)
 {KS(n - 1, m) + 0}],
4.10 ALL-PAIR SHORTEST PATH

The time complexity of the knapsack problem is as follows: Given a directed graph G = (V, E ), where each edge (v, w)
1. Without dynamic programing = O(2n) has a non-negative cost C[v, w], for all pairs of vertices (v,
2. With dynamic programing = O(mn) w), find the cost of the lowest-cost path from v to w. It is
a generalization of the single-sourceshortest-path problem.
It is one of the NPC problems.
4.10.1 Floyd’s Algorithm
4.9.4 Longest Common Subsequence
Let Ak(i, j) be the minimum cost required to go from
A subsequence of a given sequence is just the given sub- vertex i to vertex j for which all intermediate vertices are
sequence in which zero or more symbols are left out. in the set (0, 1, 2, 3, …, k) (Fig. 4.12).
Let sequence (S) = {A, B, B, A, B, B}, subsequence
1
(S1) = {A, A}, subsequence (S2) = {A, B, A, B} and
subsequence (S3) = {B, A, B, A}. 2
i j
4.9.4.1 Common Subsequence
In the given two sequences X and Y, we say that a k
Figure 4.12 | Different paths from i to j.

sequence Z is a common subsequence of X and Y, if Z is
a subsequence of both X and Y.
4.10.1.1 Recurrence Relation of Floyd’s

Example 4.20 Algorithm
X = {A, B, B, A, B, B}
The recurrence relation of Floyd’s algorithm is
Y = {B, A, A, B, A, A}
Ak (i , j ) = Min{Ak -1(i , j ), Ak -1(i , k) + Ak -1(k, j)}
Z = {B, B, A} is a common subsequence. But Z = {A,
B, A, B} is not a common subsequence. 4.10.1.2 Time Complexity
The running time complexity: T(n) = O(V 3).

4.9.4.2 Recurrence Relation of LCS
4.11 CONCEPTS OF COMPLEXITY
The recurrence relation of longest common sub‑
sequence is
CLASSES
LCS(i, j)
4.11.1 P-Complexity Class
0, if i = 0 or j = 0

= 1 + LCS(i - 1, j - 1), if X[i] = Y [ j ] P in computational complexity theory is known as poly-
 nomial time or deterministic time. This is the most basic
Max[{LCS(i - 1, j)}{LCS(i, j - 1)}], if X[i] ≠ Y [ j ]
 complexity class. This consists of all the problems that
Chapter 4.indd 205 4/9/2015 9:54:46 AM

are solvable in polynomial time. These problems can be 5. Subgraph isomorphism problem
solved in O(nk) time, where k is some constant. In class 6. Subset sum problem
P, problems are efficiently solvable or traceable. 7. Clique problem
A language L is in class P iff there exists some Turing 8. Vertex cover problem
machine M such that: 9. Independent set problem
10. Dominating set problem
1. Turing machine runs in polynomial time for all the 11. Graph colouring problem
inputs.
2. For all y in L, M must return 1, otherwise 0. 4.11.4 NP-Hard
P includes many natural problems like linear programming,
finding maximum matching and calculating the greatest NP-hard is a class that includes problems which are “at
common divisor. Prime number is also a part of class P now. least as hard as hardest problem in NP”. A problem is
said to be NP hard if
4.11.2 NP Complexity Class 1. It is in NP

2. It is NP-complete
NP class consists of problems that are verifiable in poly- This means a problem H is NP-hard if any problem L
nomial time. This class contains those languages that has polynomial time reduction from L → H. The prob-
can be defined by second order logic. lem can be a decision problem, optimization problem or
Note: The most important open question in complexity a search problem.
theory is the P = NP problem (Fig. 4.13). Examples of NP-hard problem is the traveling sales-
man problem. It is an optimization problem of finding
NP-hard NP-hard the least-cost cyclic route through all nodes of a weighted
graph. There are decision problems that are NP-hard
NPC but not NP-complete, for example, the halting problem.
This is the problem which asks, `given a program and its
NP
P = NP = NPC
input, will it run forever?’ That is a yes/no question, so
this is a decision problem.
P
Note:
1. Let À’ be an NP-complete problem and `B’ be
P! = NP P = NP
an unknown problem; if À’ is a polynomial that
Figure 4.13 | Relation between P, NP, NP-complete reduces to `B’ (A ≤p B), then `B’ is NP-hard.
and NP-hard problems. 2. Let À’ be an NP-hard problem and `B’ be an
unknown problem; if À’ is a polynomial that
4.11.3 NP-Complete reduces to `B’ (A ≤p B), then `B’ is NP-hard.
3. Let À’ be a P-class problem and `B’ be an unknown
These are the hardest problems in NP. In the computa- problem; if (B ≤p A), then `B’ is a P-class problem.
tional complexity theory, the complexity class NP-complete 4. Let À’ be a P-class problem and `B’ be an unknown
(abbreviated NP-C or NPC) is a class of decision problems. problem; if (B ≤p A), then `B’ is an NP-class
A decision problem L is NP-complete if it is in the set of problem.
NP problems and also in the set of NP-hard problems.
NP-complete is a subset of NP, the set of all deci- Problem 4.8: Both P and NP are closed under the
sion problems whose solutions can be verified in polyno- operation of:
mial time; NP may be equivalently defined as the set of (A) Union (B) Intersection
decision problems that can be solved in polynomial time (C) Kleene’s closure (D) None of these
on a non-deterministic TM. A problem p in NP is also
Solution: (D) Both P and NP are closed under con-
NP-complete if every other problem in NP can be trans-
catenation and Kleene’s closure. The Kleene closure
formed into p in polynomial time.
of a language L, denoted by L* is defined as
The following list contains some well-known problems
{w1w2…wk−k∈N, wi ∈ L∀i = 1, 2, …, k}.
that are NP-complete when expressed as decision problems:
P is closed under Kleene’s closure as for all L ∈ P,
1. Boolean satisfiability problem (SAT)
L* ∈ P.
2. Knapsack problem
3. Hamiltonian path problem NP is closed under Kleene’s closure as for all L ∈ NP,
4. Travelling salesman problem L* ∈ NP.
Chapter 4.indd 206 4/9/2015 9:54:46 AM

SOLVED EXAMPLES 207
IMPORTANT FORMULAS
1. Theta notation (Q): 0 < c1(g(n)) ≤ f (n) ≤ c2 (g(n)) ∀n ≥ 11.

n0 . Insertion sort: O(n2).
(n)) ≤ f (n) ≤ c2 (g(n)) ∀n ≥ n0 . 12. Heap sort: O(n log n).
2. Big-O notation (O): 0 < f (n) ≤ c(g(n)) ∀n ≥ n0. 13. Merge sort: O(n log n).
3. Omega notation (W): 0 < c(g(n)) ≤ f (n) ∀n ≥ n0 .
14. Quick sort: O(n2) (worst case).
4. Small-o notation (o): 0 < f (n) < c(g(n)) ∀n ≥ n0 .
15. Quick sort: O(n log n) (average case).
5. Small omega notation (w): 0 < c(g(n)) < f (n) ∀n ≥ n0 . 16. Counting sort: O(n + k).
6. Master theorem: T (n) = aT (n/b) + f (n) . 17. If a graph has n number of vertices and it is
log b a -e k-regular, where k is a positive constant, then the
Case 1: f (n) = O(n ), then T (n) = Θ(n logb a ). number of edges = (n × k)/2.
Case 2: f (n) = Θ(n ), then T (n) = Θ(n logb a log n).

log b a
18. A complete graph having n vertices has
[n (n − 1)]/2 edges.
Case 3: f (n) = Ω(n logb a + e ).
19. Number of simple graphs possible with n vertices
= 2[n(n − 1)]/2.
af(n/b) ≤ cf(n) for some constant c < 1 and large n,
then T(n) = Q(f(n)). 20. Prim’s algorithm: O(E log V).
7. Linear search: O(n). 21. Kruskal’s algorithm: O((V + E) log V).
8. Binary search: Q(log n). 22. Dijkstra’s algorithm: O(E log V).
9. Bubble sort: O(n2). 23. Bellman—Ford algorithm: O(V3).
10. Selection sort: O(n2). 24. BFT and DFT: O (V + E).
SOLVED EXAMPLES
1. Consider T (n) = 6 T (n/2) + n3; then T(n) is equal to Solution:

3 T(n) = 4T(n/2) + n; By using Master Theorem.
(a) (n/2) (b) O(n log n)
(c) (n2 log n) (d) O(n3) a = 4, b = 2, f(n) = n
g(n) = (n logb a ) = n log2 4 = Θ(n2)
Solution: f (n) = O(n log2 4-e ), f(n) is smaller than g(n), but
T(n) = 6T(n/2) + n3; By using Master Theorem. is not a smaller polynomial.
a = 6, b = 2, f(n) = n3
log b a log 2 6
By case 1, T(n) = Θ(n2)
2.58
g(n) = (n ) = n = Θ(n )
Ans. (b)
log 2 6+ e
f(n) = W (n ) , f(n) is larger than g(n), but is
not a larger polynomial. 3. Consider the following statements:
3
By case 3, T(n) = Θ(f (n)) = n 1. An algorithm is the number of steps to be per-
Ans. (d)
formed to solve a problem.
2. An algorithm is the number of steps and their
2. Solve the following recurrence relation: T (n) = 4T (n/2) + n
implementation in any language to a given prob-
T (n) = 4T (n/2) + n.
lem to solve a problem.
(a) O(n2 log n) (b) O(n2) 3. To solve a given problem there may be more
(c) O(n/2) (d) O(log n) than one algorithm.
Chapter 4.indd 207 4/9/2015 9:54:51 AM

Chapter 4 Algorthms 1 31

Uploaded by

Copyright:

Available Formats

Chapter 4 Algorthms 1 31

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 4 Algorthms 1 31

Uploaded by

Copyright:

Available Formats

CHAPTER 4

4.1 INTRODUCTION 4.2.1 Properties of Algorithm

There are certain properties algorithms have:

Chapter 4.indd 177 4/9/2015 9:54:03 AM

4.2.3 Algorithm Analysis Example 4.1

Chapter 4.indd 178 4/9/2015 9:54:04 AM

0 < c1(g(n)) ≤ f (n) ≤ c2 (g(n)) ∀n ≥ n0 By putting c = 1 and n = 1, we get

4.2.4.2 Big-O Notation (O)

Big-O notation shows the upper bound to the given func- n0

By putting c = 1 and n = 1, we get

Chapter 4.indd 179 4/9/2015 9:54:09 AM

4.2.4.4 Small-o Notation (o) 4.2.4.7 Types of Complexities

If f(n) = Q(g(n)) and g(n) = Q(h(n)), then f(n) = f(0) = 1

Chapter 4.indd 180 4/9/2015 9:54:13 AM

How deep does the tree go?

Chapter 4.indd 181 4/9/2015 9:54:19 AM

Example 4.4 is placed. It is important to note that hash function

Example 4.5 4.3.3.1 Separate Chaining

Chapter 4.indd 182 4/9/2015 9:54:21 AM

4.3.3.4 Quadratic Probing

22 When 4.3.3.5 Double Hashing

Chapter 4.indd 183 4/9/2015 9:54:22 AM

1. Min-Heap Tree: In Min heap ordering, the

Chapter 4.indd 184 4/9/2015 9:54:23 AM

4.4.1 Insertion in Min-Heap Tree

4.4.2 Deletion in Min-Heap Tree Example 4.7

Chapter 4.indd 185 4/9/2015 9:54:24 AM

Chapter 4.indd 186 4/9/2015 9:54:24 AM

comparisons. So, the complexity of linear search can be 7, 5, 2, 4, 3, 9

Chapter 4.indd 187 4/9/2015 9:54:25 AM

Example 4.9 Example 4.10

Chapter 4.indd 188 4/9/2015 9:54:26 AM

Heap Newly Added Element Swap Elements

Chapter 4.indd 189 4/9/2015 9:54:28 AM

2. Sort the list:

4.5.7.1 Pseudocode for Merge Sort 4. For i ¬ 1 to n1

Chapter 4.indd 191 4/9/2015 9:54:30 AM

Quicksort (A,1,7) Quicksort (A,9,12)

Quicksort (A,1,5) Quicksort (A,9,10)

Quicksort (A,1,2) Quicksort (A,4,5)

Chapter 4.indd 192 4/9/2015 9:54:31 AM

Table 4.1 | Sorting techniques

4.5.11 Comparison of Sorting Techniques

The comparison of the various sorting techniques is

4.6 GRAPH 4.6.2 Types of Simple Graph

4.6.2.1 Null Graph

Chapter 4.indd 194 4/9/2015 9:54:33 AM

If a graph has n number of vertices and it is k-regular,

Chapter 4.indd 195 4/9/2015 9:54:35 AM

Chapter 4.indd 196 4/9/2015 9:54:35 AM

Chapter 4.indd 197 4/9/2015 9:54:36 AM

4.7.4 Minimum Spanning Tree 10. then π[v] u

Prim’s algorithm has the following properties: a 6 3 1 z

Chapter 4.indd 198 4/9/2015 9:54:37 AM

One another algorithm to find the minimum spanning Example 4.18

Chapter 4.indd 200 4/9/2015 9:54:40 AM

1. d[s] ← 0 4.8.1.2 Application of BFT

4.8 GRAPH TRAVERSAL H I

Chapter 4.indd 201 4/9/2015 9:54:41 AM