Analysis of Algorithms

Recurrences

222301 FCI CS321 Algorithms Analysis and Design

https://thinqi.cu.edu.eg
2
3
4
5
6
 Recurrences and Running Time
• An equation or inequality that describes a function in
terms of its value on smaller inputs.
T(n) = T(n-1) + n
• Recurrences arise when an algorithm contains recursive
calls to itself
• What is the actual running time of the algorithm?
• Need to solve the recurrence
– Find an explicit formula of the expression
– Bound the recurrence by an expression that involves n

7
 Example Recurrences
• T(n) = T(n-1) + n Θ(n2)
– Recursive algorithm that loops through the input to
eliminate one item
• T(n) = T(n/2) + c Θ(lgn)
– Recursive algorithm that halves the input in one step
• T(n) = T(n/2) + n Θ(n)
– Recursive algorithm that halves the input but must
examine every item in the input
• T(n) = 2T(n/2) + 1 Θ(n)
– Recursive algorithm that splits the input into 2 halves
and does a constant amount of other work

8
 Analysis of BINARY-SEARCH
Alg.: BINARY-SEARCH (A, lo, hi, x)
if (lo > hi) constant time: c1
return FALSE
mid  (lo+hi)/2 constant time: c2
if x = A[mid]
constant time: c3
return TRUE
if ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x)same problem of size n/2
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi, x)same problem of size n/2

• T(n) = c + T(n/2)
– T(n) – running time for an array of size n
9
 Methods for Solving Recurrences

• Iteration method

• Substitution method

• Recursion tree method

• Master method

10
 The Iteration Method
• Convert the recurrence into a summation and try
to bound it using known series
– Iterate the recurrence until the initial condition is
reached.
– Use back-substitution to express the recurrence in
terms of n and the initial (boundary) condition.

11
 The Iteration Method
T(n) = c + T(n/2)
T(n) = c + T(n/2) T(n/2) = c + T(n/4)
= c + c + T(n/4) T(n/4) = c + T(n/8)
= c + c + c + T(n/8)
Assume n = 2k
T(n) = c + c + … + c + T(1)
k times
= clgn + T(1)
= Θ(lgn)

12
 Iteration Method – Example
T(n) = n + 2T(n/2) Assume: n = 2k
T(n) = n + 2T(n/2) T(n/2) = n/2 + 2T(n/4)
= n + 2(n/2 + 2T(n/4))
= n + n + 4T(n/4)
= n + n + 4(n/4 + 2T(n/8))
= n + n + n + 8T(n/8)
… = in + 2iT(n/2i)
= kn + 2kT(1)
= nlgn + nT(1) = Θ(nlgn)

13
The substitution method

1. Guess a solution

2. Use induction to prove that the

15
 Substitution method
• Guess a solution
– T(n) = O(g(n))
– Induction goal: apply the definition of the asymptotic notation

• T(n) ≤ d g(n), for some d > 0 and n ≥ n0

– Induction hypothesis: T(k) ≤ d g(k) for all k < n (strong induction)

• Prove the induction goal

16
 Example: Binary Search
T(n) = 1 + T(n/2)
• Guess: T(n) = O(lgn)
– Induction goal: T(n) ≤ c lgn, for some c and n ≥ n0
– Induction hypothesis: T(n/2) ≤ c lg(n/2)
• Proof of induction goal:
T(n) = T(n/2) + 1 ≤ c lg(n/2) + 1
= c lgn – c + 1 ≤ c lgn
if: – c + 1 ≤ 0, c ≥ 1
• Base case? 17
 Example 2
T(n) = T(n-1) + n
• Guess: T(n) = O(n2)
– Induction goal: T(n) ≤ c n2, for some c and n ≥ n0
– Induction hypothesis: T(n-1) ≤ c(n-1)2 for all k < n
• Proof of induction goal:
T(n) = T(n-1) + n ≤ c (n-1)2 + n
= cn2 – (2cn – c - n) ≤ cn2
if: 2cn – c – n ≥ 0  c ≥ n/(2n-1)
For n ≥ 1  1/(2*1-1) n ≥ 1  any c ≥ 1 will work

18
 Example 2 - Substitution
T(n) = 3T(n/4) + n2
• Guess: T(n) = O(n2)
– Induction goal: T(n) ≤ dn2, for some d and n ≥ n0
– Induction hypothesis: T(n/4) ≤ d (n/4)2
• Proof of induction goal:
T(n) = 3T(n/4) + n2
≤ 3d (n/4)2 + n2
= (3/16) d n2 + n2
≤ d n2 if: d ≥ (16/13)

• Therefore: T(n) = O(n2)

19
 Example 3
T(n) = 2T(n/2) + n
• Guess: T(n) = O(nlgn)
– Induction goal: T(n) ≤ cn lgn, for some c and n ≥ n0
– Induction hypothesis: T(n/2) ≤ cn/2 lg(n/2)
• Proof of induction goal:
T(n) = 2T(n/2) + n ≤ 2c (n/2)lg(n/2) + n
= cn lgn – cn + n ≤ cn lgn
if: - cn + n ≤ 0  c ≥ 1
• Base case? 20
The recursion-tree method

Convert the recurrence into a tree:

Used to “guess” a solution for the recurrence

21
 Master’s method
• “Cookbook” for solving recurrences of the form:
n
T (n)  aT    f (n)
b
where, a ≥ 1, b > 1, and f(n) > 0

Idea: compare f(n) with nlogba

• f(n) is asymptotically smaller or larger than nlogba by a

• f(n) is asymptotically equal with nlogba 24

n
T (n)  aT    f (n)
b
where, a ≥ 1, b > 1, and f(n) > 0

Case 1: if f(n) = O(nlogba -) for some  > 0, then: T(n) = (nlogba)

Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)

Case 3: if f(n) = (nlogba +) for some  > 0, and if

af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then:
T(n) = (f(n))
regularity condition
25
 Examples

T(n) = 2T(n/2) + n

a = 2, b = 2, log22 = 1

Compare nlog22 with f(n) = n

 f(n) = (n)  Case 2

 T(n) = (nlgn)
26
 Examples
T(n) = 2T(n/2) + n2
a = 2, b = 2, log22 = 1
Compare n with f(n) = n2
 f(n) = (n1+) Case 3  verify regularity cond.
a f(n/b) ≤ c f(n)
 2 n2/4 ≤ c n2  c = ½ is a solution (c<1)
 T(n) = (n2)

27
 Examples (cont.)

T(n) = 2T(n/2) + n

a = 2, b = 2, log22 = 1

Compare n with f(n) = n1/2

 f(n) = O(n1-) Case 1

 T(n) = (n)

28
 Examples

T(n) = 3T(n/4) + nlgn

a = 3, b = 4, log43 = 0.793

Compare n0.793 with f(n) = nlgn

f(n) = (nlog43+) Case 3

Check regularity condition:

T(n) = 2T(n/2) + nlgn

a = 2, b = 2, log22 = 1

• Compare n with f(n) = nlgn

• f(n) must be polynomially larger by a factor of n 

30