Solutions to

Introduction to Algorithms, 3rd edition

Jian Li
(yinyanghu)
June 9, 2014
ii
c 2014 Jian Li (yinyanghu)
All rights reserved.
This work is FREE and may be distributed and/or modied under the Cre-
ative Commons Attribution-NonCommercial-ShareAlike 4.0 International
License(cbna).
Acknowledgements
Contents
I Foundations 1
1 The Role of Algorithms in Computing 3
1.1 Comparison of running times . . . . . . . . . . . . . . . . . . 4
2 Getting Started 5
2.1 Insertion sort on small arrays in merge sort . . . . . . . . . . 6
2.1.1 a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1.2 b . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1.3 c . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1.4 d . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Correctness of bubblesort . . . . . . . . . . . . . . . . . . . . 8
2.2.1 a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.2 b . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.3 c . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.4 d . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 Correctness of Horners rule . . . . . . . . . . . . . . . . . . . 10
2.3.1 a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3.2 b . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3.3 c . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3.4 d . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4 Inversions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.4.1 a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.4.2 b . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.4.3 c . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.4.4 d . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
iii
iv CONTENTS
II Sorting and Order Statistics 15
III Data Structures 17
Part I
Foundations
1
Chapter 1
The Role of Algorithms in
Computing
3
4 CHAPTER 1. THE ROLE OF ALGORITHMS IN COMPUTING
1 second 1 minute 1 hour 1 day 1 month 1 year 1 century
log(n) 2
10
6
2
10
6
60
2
10
6
6060
2
10
6
606024
2
10
6
60602430
2
10
6
606024365
2
10
6
606024365100

N (10
6
)
2
(10
6
60)
2
(10
6
60 60)
2
(10
6
60 60 24)
2
(10
6
60 60 24 30)
2
(10
6
60 60 24 365)
2
(10
6
60 60 24 365 100)
2
n 10
6
10
6
60 10
6
60 60 10
6
60 60 24 10
6
60 60 24 30 10
6
60 60 24 365 10
6
60 60 24 365 100
nlog(n) 62, 746 2.8 10
6
1.33 10
8
2.75 10
9
7.18 10
10
7.97 10
11
6.86 10
13
n
2
1, 000 7, 746 60, 000 293, 939 1.6 10
6
5.6 10
6
5.6 10
7
n
3
100 391 1, 533 4, 421 13, 737 31, 594 146, 646
2
n
20 26 32 36 41 45 51
n! (9, 10) (11, 12) (12, 13) (13, 14) (15, 16) (16, 17) (17, 18)
Table 1.1: Solution to Problem 1.1
1.1 Comparison of running times
Table 1.1 shows the solution. We assume the base of log(n) is 2. And we
also assume that there are 30 days in a month and 365 days in a year.
Note Thanks to Valery Cherepanov(Qumeric) who reported an error in
the previous edition of solution.
Chapter 2
Getting Started
5
6 CHAPTER 2. GETTING STARTED
2.1 Insertion sort on small arrays in merge sort
2.1.1 a
The insertion sort can sort each sublist with length k in (k
2
) worst-case
time. So sorting all n/k sublists could be completed in (k
2
n/k) = (nk)
worst-case time.
2.1.2 b
Naive We could easily nd a naive method. Let us try to think n/k
sublists as n/k sorted queues. We scan all head elements of n/k queues, and
nd the smallest element, then pop it from the queue. The running time of
each scan is (n/k). And we need pop all n elements from n/k queues. So
this naive method costs n (n/k) = (n
2
/k) time.
Heap Sort If you do not know what the Heap Sort is, you could temporar-
ily skip this method before you read Chapter 6: Heapsort.
Similarly, we could use a min-heap to maintain all head elements. There
are at most n/k elements in the heap, so each INSERT and EXTRACT-MIN
operation takes O(log(n/k)) worst-case time. And every element enters and
leaves the heap just once. Therefore, the overall worst-case running time is
n O(log(n/k)) = O(nlog(n/k)).
Merge Sort We could use the same procedure in Merge Sort, except the
base case is a sublist with k elements instead. We get the recurrence
T(m) =

(1) if m k
2T(m/2) + (m) otherwise
Draw a recursion tree, and get the result
T(n) = 1/2 n/k 2k + 1/4 n/k 4k + + n
= nlog(n/k)
Therefore, the worst-case running time is (nlog(n/k)).
2.1. INSERTION SORT ON SMALL ARRAYS IN MERGE SORT 7
2.1.3 c
The largest value of k is (log(n)). The running time is (nk+nlog(n/k)) =
(nlog(n) + nlog(n/ log(n))) = (nlog(n)), which has the same running
time as standard merge sort.
2.1.4 d
Since k is the length of the sublist, we should choose the largest k that In-
sertion Sort can sort faster than Merge Sort on the list with length k.
In practice, Timsort, a hybrid sorting algorithm, use the exactly same
idea with some complicated techniques.
8 CHAPTER 2. GETTING STARTED
2.2 Correctness of bubblesort
2.2.1 a
We also need to prove that A

is a permutation of A.
2.2.2 b
Lines 2-4 maintain the following loop invariant:
At the start of each iteration of the for loop of lines 2-4, A[j] is
the smallest element of A[j..A.length]. Moreover, A[j..A.length]
is a permutation of the initial A[j..A.length].
Initialization Prior to the rst iteration of the loop, we have j = A.length,
so that the subarray A[j..A.length] have only one element, A[A.length].
Trivially, A[A.length] is the smallest element as well as a permutation of
itself.
Maintenance To see that each iteration maintains the loop invariant,
we assume that A[j] is the smallest element of A[j..A.length]. For next
iteration(decrementing j), if A[j 1] < A[j], i.e. A[j 1] is the smallest
element of A[j 1..A.length], we have done and skip lines 3-4. Otherwise,
lines 3-4 perform the exchange action to maintain the loop invariant. Also,
it is still a valid permuation, since we only exchange two adjacent elements.
Termination At termination, j = i. By the loop invariant, A[i] is the
smallest element of A[i..A.length] and A[i..A.length] is a permutation of
the initial A[i..A.length].
2.2.3 c
Lines 1-4 maintain the following loop invariant:
At the start of each iteration of the for loop lines 1-4, the sub-
array A[1..i 1] contains the smallest i 1 elements of the
initial array A[1..A.length]. And this subarray is sorted, i.e.
A[1] A[2] A[i 1].
Initialization Initially, i = 1, i.e. A[1..i 1] is empty. The loop invariant
trivially holds.
2.2. CORRECTNESS OF BUBBLESORT 9
Maintenance By loop invariant, A[1..i 1] contains the smallest i 1
elements and it is sorted. And lines 2-4 perform the action to move the
smallest element of the subarray A[i..A.length] into A[i]. So incrementing i
reestablishes the loop invariant for the next iteration.
Termination At termination, i = A.length. By the loop invariant, the
subarray A[1..A.length 1] contains the smallest A.length 1 elements.
Also, this subarray is sorted. So the element A[A.length] must be the largest
element and the array A[1..A.length] is sorted.
2.2.4 d
The worst-case running time of Bubble Sort is (n
2
), which is the same as
Insertion Sort.
10 CHAPTER 2. GETTING STARTED
2.3 Correctness of Horners rule
2.3.1 a
The running time is (n).
2.3.2 b
Naive-Polynomial-Evaluation shows the pseudocode of naive polynomial-
evaluation algorithm. The running time is (n
2
).
Naive-Polynomial-Evaluation(P(x), x)
1 y = 0
2 for i = 0 to n
3 t = 1
4 for j = 1 to i
5 t = t x
6 y = y + t a
i
7 return y
2.3.3 c
Initialization Prior to the rst iteration of the loop, we have i = n, so
that

n(i+1)
k=0
a
k+i+1
x
k
=

1
k=0
a
k+n+1
= 0 consistent with k = 0. So loop
invariant holds.
Maintenance By loop invariant, we have y =

n(i+1)
k=0
a
k+i+1
x
k
. Then
lines 2-3 perform that
y

= a
i
+ x y
= a
i
+ x (
n(i+1)

k=0
a
k+i+1
x
k
)
= a
i
+
n(i+1)

k=0
a
k+i+1
x
k+1
=
ni

k=0
a
k+i
x
k
So decrementing i reestablishes the loop invariant for the next iteration.
2.3. CORRECTNESS OF HORNERS RULE 11
Termination At termination, i = 1. By loop invariant, we get the result
y =

n
k=0
a
k
x
k
.
2.3.4 d
The given code fragment correctly evaluates a polynomial characterized by
the coecients a
0
, a
1
, , a
n
, i.e.
y =
n

k=0
a
k
x
k
= P(x)
12 CHAPTER 2. GETTING STARTED
2.4 Inversions
2.4.1 a
(1, 5), (2, 5), (3, 5), (4, 5), (3, 4)
2.4.2 b
Array n, n 1, n 2, , 1 has

n
2

= n(n 1)/2 inversions.

2.4.3 c
The running time of Insertion Sort and the number of inversions in the input
array are exactly same, since each move action in Insertion Sort eliminates
exact one inversion.
2.4.4 d
We could modiy the Merge Sort algorithm to count the number of inver-
sions in the array. The key point is that if we nd L[i] > R[j], then each
element of L[i..](represent the subarray from L[i]) would be as an inversion
with R[j], since array L is sorted.
COUNTING-INVERSIONS and INTER-INVERSIONS shows the pseu-
docode of this algorithm.
COUNTING-INVERSIONS(A, left, right)
1 inversions = 0
2 if left < right
3 mid = (left + right)/2
4 inversions = inversions +COUNTING-INVERSIONS(A, left, mid)
5 inversions = inversions +COUNTING-INVERSIONS(A, mid + 1, right)
6 inversions = inversions +INTER-INVERSIONS(A, left, mid, right)
7 return inversions
2.4. INVERSIONS 13
INTER-INVERSIONS(A, left, mid, right)
1 n
1
= mid left + 1
2 n
2
= right mid
3 let L[1 . . n
1
+ 1] and R[1 . . n
2
+ 1] be new arrays
4 for i = 1 to n
1
5 L[i] = A[left + i 1]
6 for i = 1 to n
2
7 R[i] = A[mid + i]
8 L[n
1
+ 1] = R[n
2
+ 1] =
9 i = j = 1
10 inversions = 0
11 counted = false
12 for k = left to right
13 if counted = false and L[i] > R[j]
14 inversions = inversions + n
1
i + 1
15 counted = true
16 if L[i] R[j]
17 A[k] = L[i]
18 i = i + 1
19 else A[k] = R[j]
20 j = j + 1
21 counted = false
22 return inversions
We can call COUNTING-INVERSIONS(A, 1, n) to get the number of
inversions in the array A. The worst-case running time is the same as Merge
Sort, i.e. (nlog(n)).
14 CHAPTER 2. GETTING STARTED
Part II
Sorting and Order Statistics
15
Part III
Data Structures
17
List of Figures
19
20 LIST OF FIGURES
List of Tables
1.1 Solution to Problem 1.1 . . . . . . . . . . . . . . . . . . . . . 4
21