Akra, Bazzi - On The Solution of Linear Recurrence Equations PDF
Akra, Bazzi - On The Solution of Linear Recurrence Equations PDF
Akra, Bazzi - On The Solution of Linear Recurrence Equations PDF
°
c 1998 Kluwer Academic Publishers, Boston. Manufactured in The Netherlands.
Abstract. In this article, we present a general solution for linear divide-and-conquer recurrences of the form
X
k
un = ai ub n c + g(n).
bi
i=1
Our approach handles more cases than the Master method does[1]. We achieve this advantage by defining a new
transform - the Order transform - which has useful properties for providing asymptotic answers (compared to
other transforms which supply exact answers). This transform helps in mapping the sequence under consideration
to the two dimensional plane where the solution becomes easier to obtain. We demonstrate the power of the final
results by solving many “difficult” examples.
Keywords: Divide and conquer, Linear recurrence, Running time, Algorithm, Order transform, Order of growth.
1. Introduction
where:
Pk
• u0 , ai ∈ R∗+ , i=1 ai ≥ 1
• bi , k ∈ N, bi ≥ 2, k ≥ 1
• g(x) is defined for real values x, and is bounded, positive and nondecreasing function
∀x ≥ 0
Such equations arise when studying the running time of divide-and-conquer algorithms.
The Master method[1] addresses the problem for the case k = 1 only, with some restrictions
on g(n). The solution we present is valid ∀k ≥ 1 and with minor restrictions on g(n).
The main idea is to define a transform to help in mapping Equation 1 into a larger
dimensional space where the order solution can be obtained easily. Consequently, we prove
that if p0 is the real solution of the characteristic equation
196 AKRA AND BAZZI
X
k
ai b−p
i =1
i=1
2. Literature Survey
According to Cormen, Leiserson, and Rivest [2], recurrences were studied as early as 1202
by L. Fibonacci, for whom the Fibonacci numbers are named. A. De Moivre introduced the
method of generating functions for solving recurrences. The master method was provided
by Bentley, Haken, and Saxe [1]. Knuth [3] and Liu [4] showed how to solve linear
recurrences using the method of generating functions. Purdom and Brown [5] contains an
extended discussion of recurrence solving. However, we are not aware of any work in the
literature that solves the above divide-and-conquer linear recurrences, which are the subject
of this paper.
3. The Solution
• In Corollary 1 we summarize the results as they apply for the original recurrence
equation.
Then,
1. ∀x ≥ 0, f (x) = f (bxc).
2. ∀n ≥ 0, f (n) = un .
In other words, f (x) is a staircase function which matches with un at integer values of x.
In proving the above theorem, we will use the following lemma whose proof can be found
in [2].
X
k ³x´
f (x) = ai f + g(bxc). (3)
i=1
bi
x ³ n + 1´
∈ 0, since bi ≥ 2.
bi 2
But
³ n + 1´
0, ⊂ [0, n) for n ≥ 1.
2
Hence, we conclude that
x bxc j bxc k j x k
∈ [0, n), ∈ [0, n) and = ∈ [0, n) (using Lemma 1).
bi bi bi bi
198 AKRA AND BAZZI
Therefore,
³x´ ³j x k´
f = f (by assumption)
bi bi
³j bxc k´
= f (using Lemma 1)
bi
³ bxc ´
= f (by assumption).
bi
Replacing in Equation 3, we obtain
X
k ³ bxc ´
f (x) = ai f + g(bxc).
i=1
bi
However,
X
k ³ bxc ´
f (bxc) = ai f + g(bxc).
i=1
bi
Therefore,
f (x) = f (bxc)∀x ≥ 0,
X
k ³n´
f (n) = ai f + g(n) (4)
i=1
bi
Let n ≥ 1. We already proved in Part 1 that f (n/bi ) = f (bn/bi c). Now since bn/bi c ∈
[0, n) we conclude f (bn/bi c) = ubn/bi c . Replacing in Equation 4, we obtain
X
k
f (n) = ai ub bn c + g(n).
i
i=1
But,
X
k
un = ai ub bn c + g(n).
i
i=1
Definition 1. (Regularity Conditions) Let S be the set of all real-valued function f (x)
of the real variable x satisfying the following conditions:
1. ∀x ≥ 0, f (x) is bounded.
LINEAR RECURRENCE EQUATIONS 199
2. ∀x ≥ 0, f (x) is nondecreasing.
In Lemma 2 we will show that the functions f (x) and g(bxc) as defined in Theorem 1 both
belong to S.
Theorem 2 (The Order Transform) Let P {} be a mapping that assigns to each function
f (x) ∈ S a real-valued function F (s, p) of the real variables s ∈ R+ and p, defined by:
Z s
F (s, p) = P {f (x)} ≡ f (u)u−p−1 du.
1
1. P {} exists.
2. P {} is linear.
3. P {} is one-to-one.
n ³ x ´o ³ f (s) ´
P f = a−p F (s, p) − Θs + Θs (1),
a sp
where Θs (h(s, p)) is a function bounded between c1 (p)h(s, p) and c2 (p)h(s, p), -for
some positive functions c1 (p), c2 (p)-, ∀s > s0 , ∀p.
Proof:
1. Since f is bounded and the range of the integral is finite, then P {} exists.
3. Let f1 (x), f2 (x) ∈ S and let P {f1 (x)} = P {f2 (x)}. Then,
Z s Z s
−p−1
f1 (u)u du = f2 (u)u−p−1 du
1 1
Z s Z s
∂ −p−1 ∂
f1 (u)u du = f2 (u)u−p−1 du
∂s 1 ∂s 1
f1 (s)s−p−1 = f2 (s)s−p−1
f1 (s) = f2 (s)
Rs
Let us investigate the asymptotic behavior of a−p s/a
f (v)v −p−1 dv with respect to s.
Since f ∈ S, then
(A) If p 6= 0, we get
Z s
dv 1 h 1 is 1 ³ ap − 1 ´
p+1
= − = .
s/a v p v p s/a sp ap
Z ³ f (s) ´
s
f (v) ap − 1
i.e., p+1
dv = Θs ( > 0, since a > 1)
s/a v sp p
(B) If p = 0 then
Z s ³s´
dv log a
p+1
dv = log(s) − log = log a = p .
s/a v a s
Z s ³ f (s) ´
f (v)
i.e., dv = Θs (note that log a > 0). (8)
s/a v p+1 sp
where:
Pk
• u0 , ai ∈ R∗+ , i=1 ai ≥ 1
• bi , k ∈ N, bi ≥ 2, k ≥ 1
202 AKRA AND BAZZI
Proof:
1. Note that g(x) ∈ S by definition. Hence, g(bxc) is clearly bounded and non-decreasing.
Moreover, g(bxc) is positive. There remains to prove that
x
∀c > 1, ∃x2 , k2 , such that g(b c) ≥ k2 g(bxc) ∀x ≥ x2 .
c
2
c
Let c > 1 be given. Let x > max{x1 + c + 1, c−1 + 1}, then the following three useful
inequalities can be derived:
x > x1 + c + 1
bxc > x1 + c
bxc − c > x1 (10)
On the other hand,
c2
x > +1
c−1
c2
bxc >
c−1
bxc(c − 1) > c2
bxc
bxc − > c
c
bxc
bxc − c > (11)
c
Finally,
x > x1 + c + 1
bxc > x1 (12)
Using Inequalities 10, 11, and 12, we conclude
³j x k´ ³j bxc k´
g = g (using Lemma 1)
c c
³ bxc ´
≥ g − 1 (since g is non-decreasing)
c
LINEAR RECURRENCE EQUATIONS 203
³ bxc − c ´
= g
c
≥ k1 g(bxc − c) (using Inequality 10)
³ bxc ´
> k1 g (using Inequality 11)
c
> k1 2 g(bxc) (using Inequality 12).
2
Therefore, ∀c > 1, ∃x2 = max(x1 + c + 1, c−1 c
+ 1) > 0, k2 = k1 2 > 0, such that
g(b xc c) ≥ k2 g(bxc), which completes the proof that g(bxc) ∈ S.
2. To prove that f (x) ∈ S, note that f is defined in terms of a finite sum of positive
terms each of which is bounded and also positive. So, f is bounded and positive. Also
according to Theorem 1, f (x) = f (bxc). So, it is sufficient to prove that
(A) f (n) is non-decreasing for all n > 0,
(B) ∀c > 1, ∃n3 , k3 such that f ( nc ) ≥ k3 f (n) ∀n > n3 .
To prove that f (n) is non-decreasing, we use strong induction. Note that
f (0) = u0 ,
X
k
Pk
f (1) = ai u0 + g(1) ≥ u0 (since i=1 ai ≥ 1 and g(1) ≥ 0)
i=1
Assume that for all k < n we have f (k) ≥ f (k − 1), and let us prove that f (n) ≥
f (n − 1). Consider
X
k ³n´
f (n) = ai f + g(n)
i=1
bi
X
k ³j n k´
= ai f + g(n)
i=1
bi
X
k ³j n − 1 k´
≥ ai f + g(n − 1) (g(n) is nondecreasing)
i=1
bi
X
k ³n − 1´
= ai f + g(n − 1)
i=1
bi
= f (n − 1).
So, for all n ≥ 1 we have f (n) ≥ f (n − 1). Therefore, f is non-decreasing.
To prove the other regularity condition, we use strong induction again. Using the results
of the previous Part, ∃k2 , x2 , such that g(b xc c) ≥ k2 g(bxc) ∀x ≥ x2 , ∀c > 1. Consider
n0 = bx2 c,
n f (0) o
k3 = min , k2 .
f (n0 )
204 AKRA AND BAZZI
As a result,
n f (0) o ³n´
∀c > 1, ∃n3 = 0, k3 = min , k2 , such that f ≥ k3 f (n) ∀n ≥ n3 ,
f (n0 ) c
Theorem 3 Let f (x) be a function defined as in Theorem 1. Let p0 be the real solution
Pk
of the characteristic equation i=1 ai b−p
i = 1. Then p0 always exists and is unique and
positive. Furthermore,
³ Z x ´
g(u)
f (x) = Θ(xp0 ) + Θ xp0 p +1
du
x1 u
0
LINEAR RECURRENCE EQUATIONS 205
³ Xk ´ ³ f (s) ´ Z s g(buc)
−p
i.e., F (s, p) 1 − ai bi + Θs = du + Θs (1). (13)
i=1
sp 1 up+1
Pk
Let h(p) = 1 − i=1 ai b−p
i . Then,
X
k
h(0) = 1 − ai ≤ 0,
i=1
lim h(p) = 1 > 0,
p→∞
d Xk
h(p) = ai (log bi )b−p
i > 0, ∀p (bi ≥ 2, ai > 0).
dp i=1
So, h(p) = 0 has a unique positive solution p0 . Replacing p0 in Equation 13, we get
³ f (s) ´ Z s g(buc)
Θ s p0 = p0 +1
du + Θs (1)
s 1 u
³ Z x
g(buc) ´
i.e., f (x) = Θ xp0 du + Θ(xp0 ). (14)
1 up0 +1
³x´
∃k1 , x1 > 0 such that g ≥ k1 g(x)∀x > x1 .
2
206 AKRA AND BAZZI
Z x Z x Z x
g(u) g(buc) g(u)
i.e., k1 du ≤ du ≤ du
x1 up0 +1 x1 up0 +1 x1 up0 +1
Z x ³ Z x ´
p0 g(buc) p0 g(u)
i.e., x du = Θ x du .
x1 up0 +1 x1 u
p0 +1
³ Z x
g(u) ´
i.e., f (x) = Θ(xp0 ) + Θ xp0 du ,
x1 up0 +1
Theorem 4 Let f (x) be a function defined as in Theorem 1. Let p0 be the unique solution
of the characteristic equation. Then,
1. If ∃² > 0 such that g(x) = O(xp0 −² ), then f (x) = Θ(xp0 ).
2. If ∃² > 0 such that g(x) = Ω(xp0 +² ) and g(x)/xp0 +² is a non decreasing function,
then f (x) = Θ(g(x)).
3. If g(x) = Θ(xp0 ) then f (x) = Θ(xp0 log x).
Proof:
1. Suppose ∃² > 0 such that g(x) = O(xp0 −² ), i.e.,
g(x) = φ(x)xp0 +²
Z x Z x
g(u)
i.e., xp0 du = xp0 φ(u)u²−1 du
x1 up0 +1 x1
But g(x) ∈ S, so
Z x ³ g(x) ´
g(u)
p +1
du = Θ (from proof of scaling property)
x/a u
0 xp0
Therefore,
Z x Z x/a
p0 g(u) g(u)
x du = Θ(g(x)) + xp0 du
x1 up0 +1 x1 up0 +1
= Ω(g(x)) (18)
208 AKRA AND BAZZI
³ Z x
g(u) ´
f (x) = Θ(xp0 ) + Θ xp0 du
x1 up0 +1
Proof: The proof can be quickly obtained by combining the results of Theorems 1, 2, 3,
and 4.
LINEAR RECURRENCE EQUATIONS 209
4. Illustrative Examples
In this section we present five examples illustrating how to apply the above results.
4 16
Example: un = ub n c + 3ub n3 c + ub n4 c + Θ(n2 log log n)
3 2 3
Solving the characteristic equation, we get
4 −p0 16
2 + 3 × 3−p0 + 4−p0 = 1
3 3
p0 = 2.
210 AKRA AND BAZZI
So,
³ Z u2 log log u ´
n
un = Θ(n ) + Θ n2
2
du
1 u3
= Θ(n2 ) + Θ(n2 log n log log n)
= Θ(n2 log n log log n).
3
Example: un = ub n c + ub n3 c + ub n6 c + ub n8 c + Θ(n)
4 2
Solving the characteristic equation, we get
3 −p
2 + 3−p + 6−p + 8−p = 1
4
p0 = 1.
5. Conclusion
In this article we provided a general method for solving linear divide-and-conquer recur-
rences. The solution turned out to have an integral form. The solution includes a parameter
p0 which is the root of the recurrence characteristic equation. The root can be computed
by simple numerical algorithms.
Notes
1. The proof of this Lemma is quite lengthy and may be skipped at first reading.
References
1. J.L.Bently, D.Haken, and J.B.Saxe. A general method for solving divide-and-conquer recurrences. SIGACT
News, 12(3):6-44, 1980.
2. T.Cormen, C.Leiserson and R.Rivest. Introduction to Algorithms. McGraw-Hill, 1990, Chapter 4.
3. Donald E. Knuth. Fundamental Algorithms, volume 1 of The Art of Computer Programming. Addison-
Wesley, 1968. Second edition, 1973.
4. C. L. Liu. Introduction to Combinatorial Mathematics. McGraw-Hill, 1968.
5. Paul W. Purdom, Jr., and Cynthia A. Brown. The Analysis of Algorithms. Holt, Rinehart, and Winston,
1985.