DataStructures

SolvingRecurrences

Tzachi(Isaac)Rosen 1

Outline
Recurrence
TheSubstitutionMethod
TheIterationMethod
TheMasterMethod

Tzachi(Isaac)Rosen 2

1
 Recurrence
Arecurrence isafunctiondefinedintermsof
oneormorebasecases,and
itself,withsmallerarguments.
Examples:

Tzachi(Isaac)Rosen 3

Outline
Recurrence
TheSubstitutionMethod
TheIterationMethod
TheMasterMethod

Tzachi(Isaac)Rosen 4

2
 TheSubstitutionMethod

Guess theformoftheanswer,thenuseinduction tofind

Tzachi(Isaac)Rosen 5

SubstitutionwithExactFunction
Claim:

T(n)=nlgn+n
Proof:
Guess T(n)=nlgn+n
Basis:
n=1 nlgn+n=1=T(n)
Inductivestep:
InductivehypothesisisthatT(k)=klg k+kforallk<n.

Tzachi(Isaac)Rosen 6

3
 SubstitutionwithOrderofGrowth
InductionParadigm:
Nametheconstantintheadditiveterm.
Showtheupper(O)andlower()boundsseparately.
Show the upper (O) and lower () bounds separately
Mightneedtousedifferentconstantsforeach.

Example:
Claim:

T(n)=(nlgn).

Proof:
Nametheconstantintheadditiveterm:T(n)2T(n/2)+cn,forsomeconstantc.
GuessandShowbyInduction:T(n)dn lg n,forsomepositiveconstantd.
Nametheconstantintheadditiveterm:T(n)2T(n/2)+cn,forsomeconstantc.
GuessandShowbyInduction:T(n)dn lg n,forsomepositiveconstantd.

Tzachi(Isaac)Rosen 7

SubstitutionwithOrderofGrowth
Upperbound:
Nametheconstantintheadditiveterm:T(n)2T(n/2)+cn,for
someconstantc.
t t
GuessandShowbyInduction:T(n)dn lg n,forsomepositive
constantd.

Therefore,T(n)=O(nlgn).

Tzachi(Isaac)Rosen 8

4
SubstitutionwithOrderofGrowth
Lowerbound:
Nametheconstantintheadditiveterm:T(n)2T(n/2)+cn,for
( ) ( / ) ,
someconstantc.
GuessandShowbyInduction:T(n)dn lg n,forsomepositive
constantd.

Therefore,T(n)=(nlgn).

Tzachi(Isaac)Rosen 9

SubstitutionwithOrderofGrowth
Conclusion:
T(n)=O(nlgn)and
T(n) O(n lg n) and
T(n)=(nlgn),
Hence,T(n)=(nlg n).

Tzachi(Isaac)Rosen 10

5
 BaseCase
Whataboutbasecases?
T(n)isalwaysconstantforanyconstantn.
Weareultimatelyinterestedinanasymptotic
solutiontoarecurrence.
Hence,itwillalwaysbepossibletochoosebase
casesthatwork.
Whenwewantanexactsolution,thenwehaveto
dealwithbasecases.

Tzachi(Isaac)Rosen 11

ExactForm
Theproofshouldbeexactformofourinductive
hypothesis.
Example:
GivenT(n)=8T(n/2)+(n2)
ItcanbeshownthatT(n)=O(n3).
Thatis,T(n)8T(n/2)+cn2 .
ButguessingT(n)dn3,willnotdo.
Si
Since,

Tzachi(Isaac)Rosen 12

6
 ExactForm
IntheotherhandGuessingT(n)dn3 dn2 will
do.
do
Since,

Tzachi(Isaac)Rosen 13

TheRecursionTreeMethod
Usetogenerateaguess.Thenverifybysubstitutionmethod
Example:
T(n)=T(n/3)+T(2n/3)+(n).

Therearelog3nfulllevels,andafter
log3/2nlevels,theproblemsizeisdown
to 1
to1.
Eachlevelcontributescn.
Goodguess:T(n)=(nlg n)

Tzachi(Isaac)Rosen 14

7
 TheRecursionTreeMethod
Upperbound:
Guess:T(n)dn
Guess: T (n) dn lg n.
n

Therefore,T(n)=O(nlgn).

Tzachi(Isaac)Rosen 15

TheRecursionTreeMethod
Lowerbound:
Guess:
Guess:T(n)dn
T(n) dn lg n.
n
Substitutionisthesameasfortheupperbound,but
replacingby.
Endupwith
Therefore,T(n)=(nlgn).
Conclusion
T(n)=O(nlgn)and
T(n)=(nlgn),
Hence,T(n)=(nlg n).

Tzachi(Isaac)Rosen 16

8
 ChangingVariables
Considertherecurrence
T(n)=2T(n)+lg n
Rename
m=lgn
yieldsT(2m)=2T(2m/2)+m
Rename
S(m)=T(2m)
yieldsS(m)=2S(m/2)+m
But,
But
S(m)=2S(m/2)+m=(mlg m)
Hence,
T(n)=T(2m)=S(m)=(mlgm)=(lgnlglgn)

Tzachi(Isaac)Rosen 17

Outline
Recurrence
TheSubstitutionMethod
TheIterationMethod
TheMasterMethod

Tzachi(Isaac)Rosen 18

9
 TheIterationMethod

Expand therecurrence
Work somealgebratoexpressasasummation
Evaluate thesummation

Tzachi(Isaac)Rosen 19

AnExample
Claim:
0 n=0
Let T (n) =
c + T (n 1) n > 0
ThanT(n)=(n)
Proof:
T(n) = c + T(n 1) = c + c + T(n 2) = = nc +T(0)=cn
T(n)=c+T(n1)=c+c+T(n2)==nc + T(0) = cn

Tzachi(Isaac)Rosen 20

10
 AnExample
Claim:
0 n=0
Let T (n) =
n + T (n 1) n > 0
ThanT(n)=(n2)
Proof:
T(n)=
( )
n+T(n1)=n+n 1+T(n2)==n+n1++n(n1)+T(0)=
==n+n1++1= n n + 1
2

Tzachi(Isaac)Rosen 21

Outline
Recurrence
TheSubstitutionMethod
TheIterationMethod
TheMasterMethod

Tzachi(Isaac)Rosen 22

11
 TheMasterMethod

Cookbook"methodforsolvingdivideand
kb k" h df l di id d
conquerrecurrencesoftheform

T(n)=aT(n/b)+f(n),

witha1,b>1,andf(n)asymptoticpositive.

Tzachi(Isaac)Rosen 23

TheMasterTheorem

Tzachi(Isaac)Rosen 24

12
 TheMasterTheorem
LetT(n)=aT(n/b)+f(n),suchthata1,b>1and
f( )
f(n)asymptoticpositive
t ti iti


n( )
logb a
(
f (n) = O n logb a )



> 0
(
T (n) = n logb a log n ) f (n) = n logb a( )
c <1

( f (n) ) f ( n) = n logb a +
(and )

af (n / b) < cf (n) for large n
Tzachi(Isaac)Rosen 25

Case1
T(n)=9T(n/3)+(n).
Wehavea=9,b=3,f(n)=cn,and
log a = log 9 = 2
b 3
Alltheconditionsofthefirstcasearesatisfied:
a=91,b=3>1,f(n)=cn asymptoticpositive
logg a
and cn = O ( n b ) = O ( n 2 ) for all 0< 1
andforall0<
Hence,T(n)=(n2).

Tzachi(Isaac)Rosen 26

13
 Case1
T(n)=5T(n/2)+(n2)
Wehavea=5,b=2,f(n)=n2,and
log b a = log 2 5
Alltheconditionsofthefirstcasearesatisfied:
a=51,b=2>1,f(n)=cn2 asymptoticpositive
logg 5
and cn 2 = O ( n 2 ) for some >0
andforsome >0
Hence, T (n) = n log 5
2

Tzachi(Isaac)Rosen 27

Case2
T(n)=T(2n/3)+1
Wehavea=1,b=3/2,f(n)=1,
log b a = log 3 / 2 1 = 0
Alltheconditionsofthesecondcaseare
satisfied:a=11,b=3/2>1,f(n)=1
asymptotic positive and 1 = (n logb a ) = (1)
asymptoticpositiveand
(
Hence, T (n) = n b log n = (log n )
log a
)
Tzachi(Isaac)Rosen 28

14
 GeneralizedCase2

When,forsomek0,

Tzachi(Isaac)Rosen 29

Case2
T(n)=27T(n/3)+(n3lgn)
Wehavea=27,b=3,f(n)=n3lgn,
log b a = log 3 27 = 3
Alltheconditionsofthesecondcaseare
satisfied:a=271,b=3>1,f(n)=n3lgn
asymptotic positive and cn lg n = (n lg n)
3 3
asymptoticpositiveand
Generalcase2withk=1, T (n) = (n log n )
3 2

Tzachi(Isaac)Rosen 30

15
 Case3
T(n)=3T(n/4)+(nlgn)
Wehavea=3,b=4,f(n)=cnlg
We have a = 3 b = 4 f(n) = cn lg n,and
n and
log b a = log 4 3
Alltheconditionsofthesecondcasearesatisfied:
a=31,b=4>1,f(n)=cn lg nasymptoticpositiveand
cn lg n = (n log 4 3+ e ) forsome >0,
andforsufficientlylargen,
y g ,
af(n/b)=c3(n/4)lg(n/4)c(3/4)nlgn=df(n)ford=3/4c
Hence,T(n)=(nlg n).

Tzachi(Isaac)Rosen 31

Case3

Theregularitycondition
af(n/b)cf(n)
forsomeconstantc<1andallsufficientlylargen
isalwaysholdswhenever

forconstant >0.

Tzachi(Isaac)Rosen 32

16
 Case3
T(n)=5T(n/2)+(n3)
Wehavea=5,b=2,f(n)=n3,
log b a = log 2 5
Conditionsofthesecondcasearesatisfied:
a=51,b=2>1,f(n)=cn3 asymptotic
positive and cn3 = (n log 5+ ) for some >0
positiveandforsome
2
>0
Hence,T(n)=(n3).

Tzachi(Isaac)Rosen 33

NonCase
NotimpliestoT(n)=2T(n/2)+nlgn,
Sincea=2,b=2,and n b = n
log a

Butnlgn isnotpolynomiallargerthann.
f (n)
log b a = (n lg n ) = lg n
n n

Tzachi(Isaac)Rosen 34

17