CLRS Solutions

Osbert Ngok

2013-03-20
2
Contents

1 The Role of Algorithms in Computing 5

1.1 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 Algorithms as a technology . . . . . . . . . . . . . . . . . . . . . 5

2 Getting Started 7
2.1 Insertion sort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Analyzing algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3 Designing algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 9

3 Growth of Functions 15
3.1 Asymptotic notation . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 Standard notations and common functions . . . . . . . . . . . . . 16

3
4 CONTENTS
Chapter 1

The Role of Algorithms in

Computing

1.1 Algorithms
Exercise 1.1.1.
Answer 1. Calculating result of competition.
Exercise 1.1.2.
Answer 2. Productivity.
Exercise 1.1.3.
Answer 3. A set of 3NF tables of html, links and pdfs. It is stored in an Excel
file and does great in terms of minimizing data redundancy, with the expense
of difficulty to read the results without joining.
Exercise 1.1.4.
Answer 4. Both are looking for shorting path in a graph, but the known
solutions are different in terms of order of growth.
Exercise 1.1.5.
Answer 5. An algorithm to determine how much change should be returned
from buying a ticket with bank notes. Compose a piece of music using generic
algorithms.

1.2 Algorithms as a technology

Exercise 1.2.1.
Answer 6. Keygen. To calculate a valid series code.

5
6 CHAPTER 1. THE ROLE OF ALGORITHMS IN COMPUTING

Exercise 1.2.2.
Answer 7.

8n2 ≤ 64n lg n
n ≤ 8 lg n
n
≤8
lg n

Turning point is around 43.5. So when n ≤ 43, insertion sort rules.

Exercise 1.2.3.
Answer 8.

100n2 ≤ 2n
n ≥ 8 lg n
n
≥8
lg n

Turning point is around 14.325. So when n ≥ 15, polynomial wins.

Problem 1.1.
Answer 9. This problem is hilarious. For the size of n that can handle lg n
in 1 second i.e.1, 000, 000microseconds, the answer is obviously 21 000000, but
people might just want to count how many digits that number could have. I
typed the formula in python and python failed to respond because it tried to
caculate the exact value.
1 second 1 minute 1 hour 1 day 1 month 1 year 1 century
6 7 9 10 11
9.49×1012 15
lg n 103×10 101.8×10 101.08×10 102.6×10 107.8×10 10 109.5×10
√
n 1012 1015.6 1019.1 1021.9 1024.8 1027 1033
n 106 107.8 109.5 1010.9 1012.4 1013.5 1016.5
n lg n 104.8 106.45 108.1 109.44 1010.86 1011.9 1014.8
n2 103 103.9 104.7 105.9 106.2 106.7 108.2
n3 102 102.6 103.2 103.6 104.1 104.5 105.5
2n 19 25 31 36 41 44 54
n! 8 10 11 13 14 15 17
Chapter 2

Getting Started

2.1 Insertion sort

Exercise 2.1.1.

Answer 10.

31 41 59 26 41 58
31 41 59 26 41 58
31 41 59 26 41 58
26 31 41 59 41 58
26 31 41 41 59 58
26 31 41 41 58 59

Exercise 2.1.2.

Answer 11.

Insertion-Sort(A)
1 for j = 2 to A.length
2 key = A[j]
3 // Insert A[j] into the sorted sequence A[1 . . j − 1].
4 i = j−1
5 while i > 0 and A[i] < key
6 A[i + 1] = A[i]
7 i = i−1
8 A[i + 1] = key

Exercise 2.1.3.

Answer 12.

7
8 CHAPTER 2. GETTING STARTED

Linear-Search(A, v)
1 o = nil
2 for j = 1 to A. length
3 if A[j] = v
4 o=j
5 return o

Exercise 2.1.4.
Answer 13.
Input: Two sequences of n booleans ⟨a1 , a2 , . . . an ⟩ and ⟨b1 , b2 , . . . bn ⟩
Output: One sequence of n + 1 booleans ⟨c1 , c2 , . . . cn+1 ⟩ which represents the
sum of two binary sequences combined.

Binary-Addition(A, B, C)
1 for j = n to 1
2 // Calculate next digit
3 C[j] = (A[j] and B[j]) or (B[j] and C[j + 1]) or (C[j + 1] and A[j])
4 // Calculate current digit
5 C[j + 1] = A[j] xor B[j] xor C[j]
6 return C

2.2 Analyzing algorithms

Exercise 2.2.1.
( )
Answer 14. Θ n3
Exercise 2.2.2.
Answer 15.

Selection-Sort(A)
1 for j = 1 to A. length
2 smallest_index = j
3 for k = j + 1 to A. length
4 if A[k] < A[smallest_index]
5 smallest_index = k
6 // Swapping
7 temp = A[j]
8 A[j] = A[smallest_index]
9 A[smallest_index] = A[j]

Loop invariant is A[1 . . j], in ascending sorted order. When j = n, there is only
one variable in the remaining list and it is obviously( the
) smallest, so there is no
need to sort. The best and worst time are both Θ n2 .
2.3. DESIGNING ALGORITHMS 9

Exercise 2.2.3.

Answer 16. worse case: n

1 + ··· + n 1+n
average case: =
n 2
best case: 1
average case Θ: Θ (n)
worst case Θ: Θ (n)

Exercise 2.2.4.

Answer 17. Pre-calculate an output of a possible input; judge if the input is

the desired input first; if it is, output the prepared output; otherwise use the
normal algorithm.

2.3 Designing algorithms

Exercise 2.3.1.

Answer 18.

3 9 26 38 41 49 52 59
3 26 41 52 9 38 49 59
3 41 26 52 38 59 9 49
3 41 52 26 38 59 9 49

Exercise 2.3.2.

Answer 19.
10 CHAPTER 2. GETTING STARTED

Merge(A, p, q, r)
1 n1 = q − p + 1
2 n2 = r − q
3 let L[1 . . n1 + 1] and R[1 . . n2 + 2] be new arrays
4 for i = 1 to n1
5 L[i] = A[p + i − 1]
6 for j = 1 to n2
7 R[j] = A[q + j]
8 i=1
9 j =1
10 k = p
11 while k ≤ r and i ≤ n1 and j ≤ n2
12 if L[i] ≤ R[j]
13 A[k] = L[i]
14 i = i+1
15 else A[k] = R[j]
16 j = j+1
17 k = k+1
18 if k ≤ r
19 if i ≤ n1
20 for l = 1 to n1 − i + 1
21 A[k + l − 1] = L[i + l − 1]
22 else
23 for l = 1 to n2 − j + 1
24 A[k + l − 1] = R[j + l − 1]

Exercise 2.3.3.
Answer 20.
1. When k = 1, n = 2k = 2, T (n) = T (2) = 2 = 2 lg 2
( k) t
( k)
2. Suppose T
( k+1 ) 2 = 2 lg
( k ) 2 k+1is true, we have
T 2 = 2T 2 (+ 2)
= 2 · 2t lg 2k + 2k+1
= 2k+1 · k + 2k+1
= (k + 1) (2k+1 )
= 2k+1 lg 2k+1
so k + 1 is true.
3. Therefore ∀k ∈ Z≥1 , let n = 2k , T (n) = n lg n.
Exercise 2.3.4.
Answer 21.
{
Θ (1) if n ≤ c,
T (n) =
T (n − 1) + C (n) otherwise.
2.3. DESIGNING ALGORITHMS 11

Exercise 2.3.5.
Answer 22.

Binary-Search(A, v, f irst, last)

1 // Boundary cases
2 if f irst > last return nil
3 if f irst = = last
4 if A[f irst] = = v
5 return f irst
6 else
7 return nil
8 // Find element in the middle of the array
9 mid = ⌊(f irst + last) /2⌋
10 if A[mid] = = v return mid
11 if A[mid] < v return Binary-Search(A, v, mid + 1, last)
12 if A[mid] > v return Binary-Search(A, v, f irst, mid − 1)
13 print “What?!”

T (n) = T (n/2) + C
T (n) = Θ (lg n)

Exercise 2.3.6.
Answer 23. line 6 that moves the data may still take Θ (n) in worst case
even Binary-Search helps to(find ) insertion point in Θ (lg n), leading to the
worst-case running time still Θ n2 .
Exercise 2.3.7.
Answer 24. Θ (n lg n) gives a sorted array for free. We divide the array into
two halves, and there are 3 possibilities: either the pair exist within one of the
halves, or the pair reside into both halves each. The first two situations are no
more than a recursive call, so let’s take a look at the third (i.e. the Combine
step).
Suppose we have array A with n1 sorted elements, and B with n2 sorted
elements. To simplify the intimidating “whole sum is exactly x‘ statement, let’s
construct a new sorted array A′ from A, where element a′ ∈ A′ if and only
if (x − a) ∈ A. This will take Θ (n) which is OK for the Combine step of
a Θ (n lg n) divide-and-conquer algorithm. If there exists ap ∈ A and bq ∈ B
such that ap + bq = x, obviously we have bq = x − ap ∈ A′. Now the question
becomes:
“Given two sorted arrays A′ with n1 elements and B with n2 elements,
describe a Θ (n1 + n2 ) algorithm that, return if there is any common element
in these two arrays.”
This proves to be easy, for we can have two pointers pointing to the smallest
elements of both array and start the comparison. If they match, we got the job
12 CHAPTER 2. GETTING STARTED

done; if they don’t, we advance the smaller element pointer in its corresponding
array and repeat the process until a match or an out-of-boundary exception,
when we know there is no such a pair.
Problem 2.1.
Answer 25.

( 2)
a. With worst-case time of Insertion-Sort for a length ( k array
) is Θ k ,
there is no doubt that n/k sublists will make it Θ n/k · k = Θ (nk).
2

( )
b. It is easy to find a Θ n2 /k solution: for each iteration of putting the
smallest element from n/k sorted lists, and there are n interations in total.
But if we maintein a heap to store the smallest values (which initially
takes Θ (n/k) time), its insertion and deletion will only take Θ (lg (n/k))
in worst-case time, making each iteration Θ (lg (n/k)). Thus to merge the
sublists, it will take Θ (n lg (n/k)) worst-case time.
c. A good guess will be k = lg n, where

Θ (nk + n lg (n/k)) = Θ (n lg n + n lg n − n lg lg n)
= Θ (n lg n)

d. I don’t know, random pick from 2 to lg n?

Problem 2.2.
Answer 26. a. We need to prove that for each A′[i], there is a corresponding
A[j] in the original array, where i and j is one-to-one relationship.
b. loop invariant: for A[1 . . i], we have A[1] ≤ A[2] ≤ · · · ≤ A[i]. It holds
for Initialization when i = 1. line 3 ensures any value smaller than the
newer value will reside on the left of the newer value, while any value
bigger than the newer value will reside on the right of the new value. At
Termination, all values are sorted.
c. Similar to b.
( )
d. Both of them are Θ n2 .
Problem 2.3.
Answer 27.

a. Θ (n)
b. Code as follows:
2.3. DESIGNING ALGORITHMS 13

Evaluate-Polynomial(a0 , a1 , · · · , an , x)
1 sum = 0
2 for i = 0 to n
3 y = ai
4 for j = 1 to i
5 y = y·x
6 sum = sum + y
7 return sum

( )
Running time is Θ n2 , significantly higher than Horner’s Rule.

c. Can be proved by Mathematical Induction.

d. Omitted.

Problem 2.4.

Answer 28.

a. ⟨2, 1⟩, ⟨3, 1⟩, ⟨8, 6⟩, ⟨8, 1⟩, ⟨6, 1⟩

n(n−1)
b. When all elements are in reverse order, the number is 2 .

c. In Insertion-Sort line 6 - 7, every time these instructions are executed,

it corresponds to the inversion of A[i] and A[j], so the running time is
proportional.

d.

Inversion-Number(A, p, r)
1 if p ≥ r
2 return 0
3 else
4 q = ⌊(p + r) /2⌋
5 return Inversion-Number(A, p, q)
+Inversion-Number(A, q + 1, r)
+Inversion-Number-Merge(A, p, q, r)
14 CHAPTER 2. GETTING STARTED

Inversion-Number-Merge(A, p, q, r)
1 count = 0
2 n1 = q − p + 1
3 n2 = r − q
4 let L[1 . . n1 + 1] and R[1 . . n2 + 1] be new arrays
5 for i = 1 to n1
6 L[i] = A[p + i − 1]
7 for j = 1 to n2
8 R[j] = A[q + j]
9 L[n1 + 1] = ∞
10 R[n2 + 1] = ∞
11 i = 1
12 j = 1
13 for k = p to r
14 if L[i] ≤ R[j]
15 A[k] = L[i]
16 i = i+1
17 else
18 // Inverse exists
19 count = count + n1 − i + 1
20 A[k] = R[j]
21 j = j+1
22 return count
Chapter 3

Growth of Functions

3.1 Asymptotic notation

Exercise 3.1.1.
Answer 29.
Because both f (n) and g (n) are asymptotically nonnegative, there exist
n1 and n2 such that for n ≥ n1 , f (n) ≥ 0 and n ≥ n2 , g (n) ≥ 0. Take
n0 = max (n1 , n2 ), and when n ≥ n0 , f (n) ≥ 0 and g (n) ≥ 0.
Under the same condition we will have: max (f (n) , g (n)) ≥ 21 (f (n) + g (n)),
so max (f (n) , g (n)) = Ω (f (n) + g (n)).
On the other hand, max (f (n) , g (n)) ≤ f (n)+g (n), so max (f (n) , g (n)) =
O (f (n) + g (n))
Therefore, max (f (n) , g (n)) = Θ (f (n) + g (n)).
Exercise 3.1.2.
Answer 30.
∞ ( )
∑ ( ) ( )
b b b−k k
(n + a) = k n a = nb + o nb = Θ nb
k=0

Exercise 3.1.3.
( )
Answer 31. O n2 provides an upper bound; the “at least” waives the upper
bound, so it is meaningless.
Exercise 3.1.4.
Answer 32.
2n+1 = 2 · 2n , so 2n+1 = Θ (2n ) = O (2n ). Suppose there exist c0 , n0 when
n ≥ n0 , 22n ≤ c0 2n , which implies 2n ≤ c0 and won’t hold true when n is
sufficiently large.
Exercise 3.1.5.
Answer 33. Yes it is easy to prove.

15
16 CHAPTER 3. GROWTH OF FUNCTIONS

Exercise 3.1.6.

Answer 34. Omitted.

Exercise 3.1.7.

Answer 35. Suppose there exists function f (n) such that f (n) = o (g (n)) and
f (n) = ω (g (n)), let c = 2, we obtain n1 and n2 . Let n0 = max (n1 , n2 ), we
have:
For n ≥ n0 , f (n) < 2g (n) and f (n) > 2g (n), contradiction.

Exercise 3.1.8.

Answer 36.

Ω (g (n, m)) = {f (n, m) : there exist positive constants c, n0 , and m0

such that 0 ≤ cg (n, m) ≤ f (n, m) for all n ≥ n0 or m ≥ m0 }
Θ (g (n, m)) = {f (n, m) : there exist positive constants c1 , c2 , n0 , and m0
such that 0 ≤ c1 g (n, m) ≤ f (n, m) ≤ c2 g (n, m) for all n ≥ n0 or m ≥ m0 }

3.2 Standard notations and common functions

Exercise 3.2.1.

Answer 37. Proved by contradiction.

Exercise 3.2.2.

Answer 38.
loga c
alogb c = a loga b = aloga c·logb a = clogb a

Exercise 3.2.3.

Answer 39.
( ( n )n ( ( )))
√ 1
lg (n!) = lg 2πn 1+Θ
e n
(√ ) (( n )n ) ( ( ))
1
= lg 2πn + lg + lg 1 + Θ
e n
= Θ (n) + Θ (n lg n) + Θ (1)
= Θ (n lg n)

∀c > 0, ∃n0 = max (4, ⌈4c⌉) such that n0 ! ≥ 1 · 2n0 −2 · (4 · c) ≥ c2n0 .

∀c > 0, ∃n0 = ⌈c⌉ such that nn = n · nn−1 ≥ cn!.

Exercise 3.2.4.
3.2. STANDARD NOTATIONS AND COMMON FUNCTIONS 17

Answer 40. ⌈lg n⌉! = O (na ) is equal to n! = O (2an ), and can be proved
similarly by 3.2.3 that it is not true.
⌈lg lg n⌉! = (O (na ) is ⌈lg n⌉! =( O (2an )
) equal to an )
lg n
(lg n)! = O lg n = O (lg n ) = O lg nlg lg n·an = O (2an )

Exercise 3.2.5.

Answer 41. lg∗ (lg n) = lg∗ (n) − 1, asymtotically larger than lg (lg∗ n).

Exercise 3.2.6.

Answer 42. Through Vieta’s Formula, ϕ + ϕ̂ = 1, ϕ · ϕ̂ = −1, satisfy the

equation.

Exercise 3.2.7.

Answer 43. F1 , F2 are trivial. When n ≤ n0 (n0 ≥ 2) the equality holds,

Fn0 +1 = Fn0 −1 + Fn0

ϕn0 −1 − ϕ̂n0 −1 ϕn0 − ϕ̂n0
= √ + √
5 5
ϕn0 +1 ϕ̂2 − ϕ̂n0 +1 ϕ2 − ϕn0 +1 ϕ̂ + ϕ̂n0 +1 ϕ
= √
5
( ) ( 2 )
ϕ n0 +1
ϕ̂ − ϕ̂ − ϕ̂
2 n0 +1
ϕ −ϕ
= √
5
ϕ n0 +1
− ϕ̂ n0 +1
= √
5

Exercise 3.2.8.

Answer 44.

By definition, there exists c1 , c2 , n0 such that

0 ≤ c1 g (n) ≤ f (n) ≤ c2 g (n)

for n ≥ n0
WLOG let’s assume c1 > ln c2 .

0 ≤ c1 n ≤ k ln k ≤ c2 n
k ln k k ln k
≤n≤
c2 c1
18 CHAPTER 3. GROWTH OF FUNCTIONS
( n ′
) ( )( )
Because ln n = 1
ln n 1− 1
ln n > 0 when n is sufficiently large,
k ln k n k ln k
≤ ≤
c2 (ln (k ln k) − ln c2 ) ln n c1 (ln (k ln k) − ln c1 )
1 n 1
( ln ln k−ln c2
)k ≤ ≤ ( )k
c2 1 + ln k
ln n c1 1 + lnln
ln k−ln c1
k

Select n1 such that ln (k (n)) > c2 > c1 when n ≥ n1 ,

1 n 1
( )k ≤ ≤ ( )k
ln ln k
c2 1 + ln k ln n c1 1 + c1ln−ln
ln
k
c1

1 n 1
( )k ≤ ≤ k
c2 1 + ln k
ln k
ln n c1
1 n 1
k≤ ≤ k
2c2 ln n c1

Problem 3.1.
( )
Answer 45. Only need to prove p (n) = Θ nd . Let a′ = max (ai ), construct
∑d ( )
p′ (n) = a′ ni as an upperbound of p to compare.
i=0
( )
′ ′n
d+1
−1 ′n
d+1
−1 ′ 10 1 ( )
p (n) = a <a =a n −
d
= Θ nd
n−1 n − 10 n
1 9 n
( d)
So p (n) = O n . ( )
On the other hand, let a′′ = |min (0, min ai )| let n0 = max 10, 8a
10 ′′
d
a .

∑
d−1
p′′ (n) = ad nd − a′′ ni
i=0
nd − 1
= ad nd − a′′
n−1
nd − 1
> ad nd − a′′
n − 10
1
n
( 10 ′′ )
a
> ad − 9 nd
n
( )
10 ′′
9 a
> ad − 10 ′′ nd
8ad a
1 ( )
= ad nd = Θ nd
9
( d) ( d)
So p (n) = Ω n . So p (n) = Θ n .
3.2. STANDARD NOTATIONS AND COMMON FUNCTIONS 19

Problem 3.2.
Answer 46.

• lg (lg∗ n)

A B O o Ω ω Θ
lgk n nϵ yes yes no no no
nk cn yes yes no no no
√
n nsin n no no no no no
2n 2n/2 no no yes yes no
nlg c clg n yes no yes no yes
lg(n!) lg(nn ) yes no yes no yes
Problem 3.3.
Answer 47.

• n1/ lg n = Θ(1)
• lg (lg∗ n)
• lg ∗ (lg n) = lg∗ n
∗
• 2lg n

• ln ln n
√
• lg n
• ln n
• lg2 n
√
• 2 2 lg n
(√ )lg n √
• 2 = Θ ( n)
• 2lg n = Θ (n)
• lg (n!) = n lg n
• 4lg n = n2
• n3
• (lg n)!
lg n
• nlg lg n = (lg n)
( )n
• 32
• 2n
20 CHAPTER 3. GROWTH OF FUNCTIONS

• en
• n · 2n

• n!
• (n + 1)!
n
• 22
n+1
• 22
2 sin n
Let gi′ (n) = gi (n) .

Problem 3.4.
Answer 48.

1. Wrong. Let f (n) = n and g (n) = n2 .

2. Wrong. Let f (n) = n and g (n) = n2 .

3. Correct.
4. Correct.

5. Correct.
6. Correct.

7. Wrong. Let f (n) = 4n .

8. Correct.