Part1 1counting

TRƯỜNG ĐẠI HỌC BÁCH KHOA HÀ NỘI
VIỆN CÔNG NGHỆ THÔNG TIN VÀ TRUYỀN THÔNG
Discrete Mathematics
Nguyễn Khánh Phương
Department of Computer Science

School of Information and Communication Technology
E-mail: [email protected]
PART 1
COMBINATORIAL THEORY
(Lý thuyết tổ hợp)
PART 2
GRAPH THEORY
(Lý thuyết đồ thị) 2
Contents of Part 1: Combinatorial Theory
Chapter 1. Counting problem
• This is the problem aiming to answer the question: “How many ways are there that satisfy
given conditions?” The counting method is usually based on some basic principles and some
results to count simple configurations .
• Counting problems are effectively applied to evaluation tasks such as calculating the
probability of an event, calculating the complexity of an algorithm (how long the algorithm
will take to run), ....
Given N paintings in a row over a
distance of M centimeters.
Each painting i (1 ≤ i ≤ N) will be drawn
on a length of ti cm, so t1+t2+..+tn = M.
The K city's most famous artists have
been selected to do this work, each artist
will be assigned to draw at least one
painting. To facilitate the artist's work, if
someone is assigned to draw more than
one painting, the paintings must be
Street art adjacent to each other on the street art
3
Contents of Part 1: Combinatorial Theory
Chapter 1. Counting problem
• This is the problem aiming to answer the question: “How many ways are there that satisfy
given conditions?” The counting method is usually based on some basic principles and some
results to count simple configurations .
• Counting problems are effectively applied to evaluation tasks such as calculating the
probability of an event, calculating the complexity of an algorithm
Chapter 2. Existence problem
In the counting problem, configuration existence is obvious; in the existence problem, we need to
answer the question: "Is there a combinatorial configuration that satisfies given properties ?”
Chapter 3. Enumeration problem
This problem is interested in giving all the configurations that satisfy given conditions.
Chapter 4. Combinatorial optimization problem
• Unlike the enumeration problem, this problem only concerns the "best" configuration in a
certain sense.
• In the optimization problems, each configuration is assigned a numerical value (which is the
use value or the cost to construction the configuration), and the problem is that among the
configurations that satisfy the given conditions, find the configuration with the maximum or
minimum value assigned to it
Contents of Part 1
Chapter 0: Sets, Relations
Chapter 1: Counting problem
Chapter 2: Existence problem
Chapter 3: Enumeration problem
Chapter 4: Combinatorial optimization problem
NGUYỄN KHÁNH PHƯƠNG

Bộ môn KHMT – ĐHBK HN 5
TRƯỜNG ĐẠI HỌC BÁCH KHOA HÀ NỘI
VIỆN CÔNG NGHỆ THÔNG TIN VÀ TRUYỀN THÔNG
Chapter 1
counting problem

Bộ môn KHMT – ĐHBK HN
Contents
• Combinatorics is the branch of mathematics devoted to calculate the
number of ways in which a specified process can be carried out.
Problems of this type often occur in computing. A typical problem is
to determine how many times a particular sequence of steps in an
algorithm will be executed. You might need to know this in order to
estimate how long the algorithm will take to run.
• For example: determine the shortest path for a signal to travel through
a communication network
– Algorithm 1 (simplest): calculate the lengths of all the possible paths through the
network.
– Algorithm 2: just calculate the lengths of some selected paths
 In order to determine how efficient these algorithms are, and to be
able to compare them with each other and with other algorithms, we
must first answer the question: ‘How many such paths are there
altogether?’ 7
Contents
1. Basic counting principles
2. Elementary combinatorial configuration
3. The inclusion-exclusion principle
4. Recurrence relation
5. Generating function
8
1.1. The sum rule
1.2. The product rule
1.3. Tree diagram
9
1.1. The sum rule
Example:
• 40 textbooks on Mathematics; 50 textbooks on English
• to select 1 book: 40+50 choices
• What about selecting 2 books?
problem decompose combine
Task 1
Independent tasks
Let us consider two tasks:
m1 is the number of ways to do task 1
Tasks are independent of each other, i.e.,
Performing task 1 does not accomplish task 2 and vice versa. Task 2
Sum rule: the number of ways that “either task 1 or task 2 can be done, but not
both”, is m1 + m2.
Generalizes to multiple tasks ...

1.1. The sum rule
Let us consider two tasks: Task 1
Tasks are independent of each other, i.e.,
Performing task 1 does not accomplish task 2 and vice versa.
Task 2
Sum rule: the number of ways that “either task 1 or task 2 can be done, but not
both”, is m1 + m2.
Generalizes to multiple tasks ...
Things 1 2 3 … k
ways m1 m2 m3 … mk
select one of them: m1 +m2 +m3 +...+ mk ways

1.1. The sum rule
Example:
• 40 textbooks on Mathematics; 50 textbooks on English
• to select 1 book: 40+50 choices
• What about selecting 2 books?
Independent tasks
Generalized sum rule:

Tasks 1 2 3 … k

1.1. The sum rule
Example: How many 6-character strings are there, each of which is
taken from the set {A, B, C, D, E} containing at least 3 different
characters? For example, the string ABCCAC is the string that satisfies
the problem condition, while the string ABABAB does not.
Independent tasks
Generalized sum rule:
Tasks 1 2 3 … k

1.1. The sum rule
Generalized sum rule: If we have tasks T1, T2, …, Tk that can be
done in m1, m2, …, mk ways, respectively, and any two of these
tasks can not be done at the same time, then there are m1 + m2 +
… + mk ways to do one of these tasks.
The sum rule can also be phrased in terms of set theory: The
size of the union on k finite pair wise disjoint sets is the sum of
their sizes:
• Let A1, A2, …, Ak be disjoint sets. Then the number of ways
to choose any element from one of these sets is
|A1  A2  …  Ak| = |A1| + |A2| + … + |Ak|.
14
1.1. The sum rule
Example 1: A student can choose a computer project from one of three
lists. The three lists contain 23, 15, and 19 possible projects respectively.
How many possible projects are there to choose from?
23+15+19 
Example 2: How many strings of 4 decimal digits, have exactly three
digits that are 9s?
The string can have:
• The non-9 as the first digit (x999)
• OR the non-9 as the second digit (9x99)
• OR the non-9 as the third digit (99x9)
• OR the non-9 as the fourth digit (999x)
Thus, we use the sum rule:
• For each of those cases, there are 9 possibilities for the non-9 digit
(any number other than 9) 15
• Thus, the answer is 9+9+9+9 = 36
1.1. The sum rule
Example 3: What is the value of k once this program is executed?
n1=10; n2=20; n3=30;
k=0;
for (i1= 1; i1<=n1;i1++) k=k+1;
for (i2= 1; i2<=n2;i2++) k=k+1;
for (i3= 1; i3<=n3;i3++) k=k+1;
Answer: At the beginning, the value of k is assigned to 0. There are 3
independent “for” loops. After each for loop, the value of k is increased by one:
• The first “for” loop iterates 10 times,
• The second “for” loop iterates 20 times,
• The last “for” loop iterates 30 times
Therefore, at the end, the value of k = 10 + 20 + 30 = 60
16
1.1. The sum rule
1.3. Tree diagram
17
The sum rule
Object type 1
Consider 2 types of objects:
m1 is number of object type 1
m2 is number of object type 2 Object type 2
The decision to choose object type 1 or 2 is independent of each other, that is,
• the decision to choose object type 1 does not affect the decision to choose
object type 2, and vice versa.
Sum rule: The number of ways to choose an object type 1 or an object type 2 is m1
+ m2
Generalize to multiple types of objects:
Object type 1 2 3 … k
Quality m1 m2 m3 … mk
The number of ways to choose one object from these k types of objects
= m1 +m2 +m3 +...+ mk 18
The product rule
• Consider 2 types of objects:
– m1 is number of object type 1
– m2 is number of object type 2
The decision to choose object type 1 or 2 is independent of each other, that is,
• the decision to choose object type 1 does not affect the decision to choose
object type 2, and vice versa.
Product rule: The number of ways to choose an object type 1 and an object type 2
is m1 m2
Object type1 Object type2
• Generalize to multiple types of objects:

Object type 1 2 3 … k
Quality m1 m2 m3 … mk
The number of ways to select exact k objects, not any 2 objects of same type
19
= m1 m 2 m3 … mk
The product rule
• Consider two tasks:
– m1 is the number of ways to do task 1
– m2 is the number of ways to do task 2
– Tasks are independent of each other, i.e.,
• Performing task 1does not accomplish task 2 and vice versa.
Product rule: the number of ways that “both tasks 1 and 2 can be done”
in m1m2.
task 1 task 2
• Generalize to multiple tasks ...
20
Analysis
License Plate LL-DDDD 2 letters – 4 digits
# of possible plates = ?
Suppose a procedure can be constructed by a

series of steps
Step 1 Step 2 Step 3 Step k


m1 ways m2 ways m3 ways mk ways
Number of possible ways to complete the procedure is
m1 * m2 *  *mk
Analysis
License Plate LL-DDDD 2 letters – 4 digits
# of possible plates = ?
Procedure:
Step 1: Step 3:
Step 2: Step 4:
Step 5:
Step 6:
Generalized product rule: If we have a procedure consisting of
sequential tasks T1, T2, …, Tk that can be done in m1, m2, …, mk
ways, respectively, then there are m1 * m2 * … * mk ways to
carry out the procedure.
The product rule can also be phrased in terms of set theory:

Let A1, A2, …, Ak be finite sets. Then the number of ways to
choose one element from each set in the order of A1, A2, …,
Ak is
|A1 x A2 x … x Ak| = |A1| * |A2| * … * |Ak|.

24
The product rule
• If each element ai of k- tuple (a1, a2, ..., ak) has mi number of
ways to select (i = 1, 2, ..., k), then the number of tuples could be
generated is the product of these m1m2 ... mk
25
The product rule
In many counting problems, it is only after building the first element, we just know how
to build the second element; after building the first two elements we only know how to
build the third element,… In that case, we use the general product rules:
Suppose we construct a k-tuple (a1, a2, ..., ak) by building each element in turnand
• a1 can be chosen by m1 way;
• After a1 is selected, a2 can be chosen by m2 way;
• ...
• After a1, a2, ..., ak-1 are chosen, ak can be chosen by mk ways;
Then the number of k-tuples generated is m1m2 ... mk
Example: 6 persons are competing for 4 prizes. How many different outcomes are
A B C
possible? 1st prize 2nd prize 3rd prize 4th prize
D E F
A 4-tuple = (1st prize, 2nd prize, 3rd prize, 4th prize) 26

Example 1: There are 18 math majors and 325 CS majors. How many
ways are there to pick one math major and one CS major?
Total is 18 * 325 = 5850
Example 2: The license plate: 2 letters-4 digits
(a) no letter or digit can be repeated
26  25  10  9  8  7
(b) with repetitions allowed
26  26  10  10  10  10
(c) same as (b), but only vowels and even digits
52x54
27
Example 3: How many functions are there from set A to set B?
So, how many Boolean

A B
functions on n variables?
2n
2
To define each function we have to make 3 choices, one for each element of A.
Each has 4 options (to select an element from B).
How many ways can each choice
4 x 4 x 4 be made?
43= 64 = |B| |A|

Example 4: How many strings of 4 decimal digits
(a) do not contain the same digit twice?
We want to choose a digit, then another that is not the same, then
another…
• 1st digit: 10 possibilities (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
• 2nd digit: 9 possibilities (all but not first digit)
• 3rd digit: 8 possibilities
• 4th digit: 7 possibilities
Total = 10*9*8*7 = 5040
(b) end with an even digit?

First three digits have 10 possibilities (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
Last digit has 5 possibilities (0, 2, 4, 6, 8)
Total = 10*10*10*5 = 5000 29
Example 5: Consider the following nested loop:
for (i=1;i<=9;i++)
for (j=1;j<=7;j++) {
[ Statement 1;
Statement 2; ]
}
How many times the statements in the inner loop will be executed?
Solution: 9 × 7 = 63 times (based on the product rule)
30
More complex counting problems
• Combining the product rule and the sum rule.
• Thus we can solve more interesting and complex problems.
31
More complex counting problems
Example 1: How many BASIC variables: single letter
or single letter+single digit
26 + 26 * 10=286
rule of sum rule of product

Example 2: How many integers from 1 through 999 do not have any repeated
digits?
Let A = integers from 1 to 999 not having repeated digits.
Partition A into 3 sets:
• A1=one-digit integers not having repeated digits;
• A2=two-digit integers not having repeated digits;
• A3=three-digit integers not having repeated digits.
 |A| = |A1| + |A2| + |A3| (by the sum rule)
32
= 9 + 9×9 + 9×9×8 = 738 (by the product rule)
Count the number of ways to put things together into various combinations.
Example 3: Passwords consist of character strings of 6 to 8 characters. Each

character is an upper case letter or a digit.
26 (A-Z) 10 (0-9)
Each password must contain at least one digit. How many passwords are
possible? constraint
– Let P – total number of possible passwords
– Pi – total number of passwords of length i, i = 6,7,8
 P = P6 + P7 + P8 (the sum rule)
Pi – computing it directly is tricky (hmm…) How??
– “popular” counting trick: let’s calculate all of them, including those with no
digits and then subtract the ones with no digits.
P6 = (# possibilities without constraint) – (# passwords with no digits)
= 366 – 266
P7 = 367 - 267
P8 = 368 – 268
33
 P = P6 + P7 + P8 = 366-266 + 367 - 267 + 368 – 268 =2,684,483,063,360
Count the number of ways to put things together into various combinations.
Example 3: Passwords consist of character strings of 6 to 8 characters. Each

character is an upper case letter or a digit.
26 (A-Z) 10 (0-9)
Each password must contain at least one digit. How many passwords are
possible? constraint
– Let P – total number of possible passwords
 P = P6 + P7 + P8 = 366-266 + 367 - 267 + 368 – 268 =2,684,483,063,360
Comment: If your computer can try 200 million passwords per second, then how
long can the password be determined, so the intruder could penetrate this computer
system?
(2 684 483 063 360/200 000 000) / (60 * 60) hours
Nearly 4 hours!
34
IP Address Example (Internet Protocol v. 4)
Example 4: An address is a string of 32 bits – it begins with a network id

(netid), followed by a host number (hostid), which identifies a computer as a
member of a particular network.
• Main computer addresses are in one of 3 types:
– Class A (largest networks): address contains a 0 followed by 7-bit
“netid”, and a 24-bit “hostid”
– Class B (medium networks): address contains a 10 followed by a 14-bit
netid and a 16-bit hostid.
– Class C (smallest networks): address contains a 110 has 21-bit netid and
an 8-bit hostid.
Netids all 1s are not allowed. Hostids that are all 0s or all 1s are not allowed.
35
How many valid IP addresses are there?
Wedding picture
Example 5: Consider a wedding picture of 6 people
– There are 10 people, including the bride and groom
(a) How many possibilities are there if the bride must be in the picture?
• Product rule: place the bride AND then place the rest of the party
• First place the bride:
 She can be in one of 6 positions
• Next, place the other 5 people via the product rule:
 There are 9 people to choose for the second person, 8 for the
third, etc.  Total = 9*8*7*6*5 = 15120
 Product rule yields 6 * 15120 = 90,720 possibilities
Wedding picture
b) How many possibilities are there if the bride and the groom must be
in the picture?
• Product rule: place the bride/groom AND then place the rest of the party
• First place the bride and groom:
 He can be in one of 5 remaining positions
 Total of 6*5 = 30 possibilities
• Next, place the other 4 people via the product rule:
 There are 8 people to choose for the 3rd, 7 for the 4th, etc.
 Total = 8*7*6*5 = 1680
Wedding picture
c) How many possibilities are there if only one the bride or the groom must be in the
picture?
Sum rule: place only the bride
• Product rule: place the bride AND then place the rest of the party
– First place the bride
– Next, place the other 5 people via the product rule (We can’t choose the
groom!!!)
 There are 8 people to choose for the second person, 7 for the third, etc.
 Total = 8*7*6*5*4 = 6720
OR place only the groom (hmm… quickly, how many?)
• Same possibilities as for bride: 40,320
 Sum rule yields 40,320 + 40,320 = 80,640 possibilities
Alternative way to get the answer ???

Wedding picture
a) How many possibilities are there if the bride must be in the picture?
90,720 possibilities
b) How many possibilities are there if the bride and the groom must be in the
picture?
50,400 possibilities
c) How many possibilities are there if only one the bride or the groom must be in
the picture?
• Total ways to place the bride (with or without groom): 90,720
 See (a).
• Total ways for both the bride and groom: 50,400
 See (b).
• Total ways to place ONLY the bride:
90,720 – 50,400 = 40,320
• Same number for the groom
• Total = 40,320 + 40,320 = 80,640
1.1. The sum rule
1.3. Tree diagram
40
1.3. Tree diagram
In many cases, we can solve the counting problems by construction
of “Tree diagrams”.
Example 1: In a tennis match, the player wins the game if he is the first
player wins two sets. Question: What is the probability that player A
will win the game in 3 sets?
Solution: Construct tree diagram:
Thus, the answer is ???
2/6
1st set winner
2nd set winner
3rd set winner NGUYỄN KHÁNH PHƯƠNG

41
1.3. Tree diagram
Example 2:When buying a PC system, you have the choice of
 3 models of the basic unit: B1, B2, B3;
 2 models of keyboard: K1, K2;
 2 models of printer: P1, P2.
Question: How many distinct systems can be purchased?
The number of distinct systems is: 42

3×2×2 = 12
1.3. Tree diagram
Example 3: Suppose we flip a coin and then throw a single die. What
are the possible combined outcomes?
There are 12 possible combined outcomes: 2 ways in which the coin

can come up followed by 6 ways in which the die can come up 43
Contents
44
2.1. Permutation
2.2. Combination
45
2.1. Permutation
2.2. Combination
Fruit salad is a combination of apples, grapes and bananas. We don't

care what order the fruits are in.
The permutation that will open the lock is 942, we do care about the
order.
46
2.1. Permutation
2.1.1. Permutation
2.1.2. k-permutation
2.1.3. Circulation permutation
2.1.4. Permutation of multisets
2.2. Combination
47
2.1.1. Permutation
Example 1: A PIN is defined as a sequence of any 4 digits from the set {0, 1, …, 9}.
Question 1. How many different PINs are possible if repetitions are allowed?
Solution. Choosing a PIN is a 4-step operation:
• Step 1: Choose the 1st symbol (10 different ways).
• Step 2: Choose the 2nd symbol (10 different ways).
• Step 3: Choose the 3rd symbol (10 different ways).
• Step 4: Choose the 4th symbol (10 different ways).
 Based on the product rule:
10×10×10×10 = 10,000 PINs are possible.
Question 2. How many different PINs are possible if repetitions are NOT allowed?
Solution. Choosing a PIN is a 4-step operation:
• Step 1: Choose the 1st symbol (10 different ways).
• Step 2: Choose the 2nd symbol (9 different ways).
• Step 3: Choose the 3rd symbol (8 different ways).
• Step 4: Choose the 4th symbol (7 different ways).
 Based on the product rule:
48
10×9×8×7 = 5,040 PINs are possible.
2.1.1. Permutation
• A permutation of a set A of objects is an ordered arrangement of
the elements of A where each element appears only once
Example: If A = {a, b, c}, then the permutations of A are
1. abc
2. acb
3. bac
4. bca
5. cab
6. cba
• The number of permutations of any set with n elements is
P(n) = n! = n(n − 1) · · · 2 · 1
(Note that by definition 0! = 1)
Proof: 49
2.1.1. Permutation
• The number of permutations of any set with n elements is
P(n) = n! = n(n − 1) · · · 2 · 1
Proof: (by product rule)
Forming a permutation is an n-step operation:
• Step 1: Choose the 1st element ( n different ways).
• Step 2: Choose the 2nd element ( n-1 different ways).
• …
• Step n: Choose the nth element (1 way).
Based on the product rule, the number of permutations is
n  (n-1) … 2  1 = n!
50
• A permutation of a set A of n objects x1, x2, ..,xn is an ordered
arrangement of the elements of A where each element appears
only once.
• A k-permutation of a set A of n objects x1, x2, ..,xn is an ordered
arrangement of k distinct elements of A.
1st 2nd 3rd  k -th
x1 x2 x3  xn
Example: 3-permutation of a set A of 4 objects x1, x2, x3, x4
x1
1st
x2
2nd
x3
3rd
 x2
1st
x1
2nd
x3
3rd
51
• A permutation of a set A of n objects x1, x2, ..,xn is an ordered
arrangement of the elements of A where each element appears
only once.
 “permutations” means “n-permutations”.
52
 The number of k-permutations of a set A with n=|A| elements is
P(n, k) = n(n−1)…(n−k+1) = n! / (n−k)!
Example 1: 6 persons are competing for 4 prizes. How many different

outcomes are possible?
1st prize 2nd prize 3rd prize 4th prize
D F C E B A
6!
P (6, 4)  6*5* 4 *3 
2!
53
Example 2: Suppose you “have” time to listen to 10 songs on your daily
jog around campus. There are 6 A tunes, 8 B tunes, and 3 C tunes
to choose from.
Finally, suppose you still want 4 A, 4 B, and 2 C tunes, and the order
of the groups doesn’t matter, but you get dizzy and fall down if all the
songs by any one group aren’t played together.
How many playlists are there?
P(n, k): k unique choices out of n objects, order matters

54
Example 3. How many ways are there to arrange 4 students to sit on the
bench of 10 seats provided that sitting on the lap of each other is not
allowed.
Solution. The students are numbered 1 to 4, seats 1 through 10. Each
placement of the students can be represented by an ordered set (s1, s2, s3,
s4), where si  {1, 2, ..., 10} are the seat of student ith.
As si sj, i j, each placement is a 4-permutation of 10: P(10, 4) =
10*9*8*7=5040.
Note: We can also argue by directly using the Product rule:

We take turns putting the students into their seats.
• 1st student: there are 10 seats to be selected to sit
• 2nd student: put him one of the remaining 9 seats,…
According to the Product rule, there are 10*9*8*7 ways of placement 55
2.1. Permutation
2.1.1. Permutation
2.2. Combination
56
Circulation permutation n!/n=(n-1)!
Example 3: 6 people A, B, C, D, E, F are seated around a round table,
how many different circular arrangements are possible, if arrangements
are considered the same when one can be obtained from the other by
rotations?
A
ABCDEF, B
BCDEFA, F
CDEFAB,
DEFABC, E C
EFABCD,
FABCDE D
are the same arrangements circularly
 There are 6! ways to seat 6 people around the table
For each seating, there are 6 “rotations” of the seating
Thus, the final answer is 6!/6 = 5! = 120 57
Hoán vị vòng tròn
• A circulation permutation of a set A of n objects is an ordered of all n
objects of A arranged as a circle; there is not the beginning object and
the ending object.
"The number of circulation permutations of an n-set equals n!/n = (n-1)!”
Proof ????
1 circulation
permutation of set
4 objects A, B, C, D
4 permutations of
set 4 objects A, B,
C, D
• A circulation permutation of a set A of n objects is an ordered of all n
objects of A arranged as a circle; there is not the beginning object and
the ending object.
"The number of circulation permutations of an n-set equals n!/n = (n-1)!”
• A circular k-permutation of a set A of n objects is an ordered k objects

of A arranged as a circle; there is not the beginning object and the
ending object. Prove the following result:
"The number of circular k-permutations of an n-set equals P(n,k)/k".
59
Example 4: How many ways to arrange 3 couples in a round table with
alternating sex
60
2.1. Permutation
2.1.1. Permutation
2.2. Combination
61
2.1.4. Permutations of multisets
• A multiset M is a collection whose members need not be distinct.
Example: The collection
M = (a, a, a, b, b, c, d, d, d, 1, 2, 2, 2, 3, 3, 3, 3)
is a multiset; and sometimes it is convenient to write
M = (3a, 2b, c, 3d, 1, 32, 43).
• A multiset M over a set S can be viewed as a function v : S → ℕ from
S to the set ℕ of nonnegative integers; each element x S is repeated
t(x) times in M; we write M = (S; t).
• Let M be a multiset and |M| = n.
– A k-permutation of M is an ordered arrangement of k objects
selected from n objects of M.
– A n-permutation of M is called a permutation of M.
62
– A k-permutation of M is an ordered arrangement of k objects selected
from n objects of M.
Proposition 1. Let M be a multiset of r different types where each type has

infinitely elements. Then the number of k-permutations of M equals
Example 1. What is the number of binary numerals with at most 4 digits?

Solution: The question is to find the number of 4-permutations of the
multiset (0, 1). Thus the answer is 24 = 16.
Example 2. What is the number of ternary numerals with at most 4 digits?

• Solution: The question is to find the number of 4-permutations of the
63
multiset (0, 1, 2). Thus the answer is 34 = 81.
– A k-permutation of M is an ordered arrangement of k objects selected
from n objects of M.
Proposition 1. Let M be a multiset of r different types where each type has

infinitely elements. Then the number of k-permutations of M equals
1st position 2nd position 3rd position …. kth position
r choice r choice r choice
The total = rk
64
Proposition 2. Let M be a multiset of r different types with repetition numbers n1,
n2,…, nr respectively. Let n = n1+ n2 +… + nr . Then the number of permutations of M
equals
𝟏 𝟐 𝒓
Proof.
• List the elements of M as
nr
• Let S be the set consisting of the elements
Let X be the set of all permutations of S, and let Y be the set of all permutations of M.
There is a map f : X → Y, sending each permutation of S to a permutation of M by
removing the subscripts of the elements. Note that for each permutation  of M there
are n1!, n2!,…,nr! ways to put the subscripts of the first, the second, ..., and the rth type
elements back, respectively.
Thus there are n1!n2! … nr! elements of X sent to Y by f. Therefore
|X| n!
| Y |  65
n1 ! n2 !...nr ! n1 ! n2 !...nr !
Proposition 2. Let M be a multiset of r different types with
repetition numbers n1, n2,…, nr respectively. Let n = n1+ n2
+… + nr . Then the number of permutations of M equals
Example 3: The number of 0-1 words of length n with exactly

m ones and (n – m) zeros equals to
n! n 
 Cn    .
m
m !(n  m)! m

66
Example 4: Find the number of 8-permutations of the multiset
M = (a, a, a, b, b, c, c, c, c) = (3a, 2b, 4c).
Solution:
How to select 8 elements from (3+2+4) = 9 elements ???
(2a,2b,4c)
(3a, b, 4c)
(3a, 2b, 3c)
• The number of 8-permutations of (2a, 2b, 4c): 8!/(2!2!4!)
• The number of 8-permutations of (3a, b, 4c): 8!/(3!1!4!)
• The number of 8-permutations of (3a, 2b, 3c): 8!/(3!2!3!)
 Thus the answer is
8!/(2!2!4!) + 8!/(3!1!4!) + 8!/(3!2!3!) = 420 + 280 + 560 = 1260
67
2.1. Permutation
2.2. Combination
68
2.2. Combination
2.1.1. Definitions
2.2.2. Binomial coefficients
2.2.3. Combinations of Multisets
2.2.4. Multinomial coefficients

2.2.1. Definitions
• A k-combination of a set of n elements is a subset of size k of n
elements.
(Note: A permutation is a sequence while a combination is a set)
Example:
• The 2-permutation (sequence) of SOHN is:
SO, SH, SN, OH, ON, OS, HN, HS, HO, NS, NO, NH
• The 2-combination (set) of SOHN is:
{S,O},{S,H},{S,N},{O,H},{O,N},{H,N}
70
2.2.1. Definitions
• The number of all k-combinations of a set of n elements denoted
n k
  , Cn or C (n, k )
k
and read “n choose k”.
n n!
  Proof ???
 k  k ! (n  k )!
This number is also called a binomial coefficient because such
numbers occur as coefficients in the expansions of powers of
binomial expressions such as (a+b)n
71
k-combination
Example: X = {a, b, c}; n = 3, r = 2
k-permutation k-combination
select 2 from 3 elements of X: select 2 from 3 elements of X:
1. (a, b) 1. {a,b}
2. {a, c} Order is not the matter
2. (a, c)
3. {b, c}
3. (b, a)
Ordered sequence
4. (b, c)
5. (c, a)
6. (c, b) 2!
n! n!
P  n(n  1)...(n  k  1) 
n
k
C 
k
(n  k )! n
k !(n  k )! 72
k-combination
Example: X = {a, b, c}; n = 3, r = 2
k-permutation k-combination
Select 2 from 3 elements of X: Select 2 from 3 elements of X:
1. (a, b) 1. {a,b}
2. {a, c} Order is not the matter
2. (a, c)
3. {b, c}
3. (b, a)
Ordered sequence
4. (b, c) n!
Pnk  n(n  1)...( n  k  1) 
5. (c, a) (n  k )!
6. (c, b)
Consider the set A of all k-permutation of n elements. Divide them into subset so that the two permutations
belonging to the same subset differ only in order. Obviously, these subsets are a partition on the set A, and each such
subset corresponds to a k-combination of n elements. There are the same number of k-permutation in each subset,
and is equal to k! (number of permutations). The number of subsets is the number of k-combination of n. According
to the sum rule, product of k! and the number of subsets is equal to the number of k-permutation of n, that is, equal
to n (n-1) ... (n - k + 1). Therefore, the number of k-combination of n is:
n(n  1)(n  2)...(n  k  1) n! 73
or
k! k !(n  k )!
k-combination of a set
Example: Consider the following nested loop
for a:=1 to n
for b:=1 to a-1
for c:=1 to b-1
[Statements]
endfor
endfor
endfor
Question: How many times the statements in the innermost loop will be
executed?
Answer: Each iteration corresponds to a triple of integers (a, b, c) where a > b >
c . The set of all this kind of triples corresponds to all 3-combinations of {1, …,
n}. Thus, the total number of iterations is C(n,3).
Difference between permutation and combination
• A combination is a set while a permutation is a sequence.
Let’s consider the difference between unordered and ordered selections:
1. Two ordered selections are the same if
– the elements chosen are the same;
– the elements chosen are in the same order.
• Ordered selections correspond to k-permutations.
• The number of all k-permutations is P(n, k) = n! / (n−k)!
2. Two unordered selections are the same if
– the elements chosen are the same (regardless of the order in which the
elements are chosen)
• Unordered selections correspond to k-combinations
• The number of all k-combinations is C(n, k) = n! / k! (n−k)!
75
When dealing with any counting problem, we should ask
ourselves about the importance of order in the problem:
• When order is relevant, we think in terms of permutations
and arrangements and the rule of product.
• When order is not relevant, combinations could play a key
role in solving the problem.
76
Example: A club has 25 members.
a. How many ways are there to choose 4 members of the club
to serve on an executive committee?
b. How many ways are there to choose a president, vice

president, secretary, and treasurer of the club?
77
2.2. Combination
2.1.1. Definitions
78
Binomial Coefficients
(a + b)4 = (a + b)(a + b)(a + b)(a + b)

4  4  4  4  4 
=0a4 +1 a3b +2a2b2 +3 ab3 +4 b4
Binomial Theorem: Let x and y be variables, and let n be any nonnegative integer.
Then
n  n  j n
(x  y )    x n
y j
j
j  0  
79
n  n  j j
n
(x  y )   x y
n
j
j 0  
What is the coefficient of a8b9 in the expansion of (3a +2b)17?
What is n? 17
 17  17  8 9 8 9
What is j? 9   (3a ) 8
(2b ) 9
  3 2 a b
9 9
What is x? 3a
What is y? 2b
80
n n  n  j j
(x  y )   x y
n
j
j 0   What is coefficient
(a + b)2 = a2 + 2ab + b2 of a9b3 in (a + b)12?
(a + b)3 = a3 + 3a2b + 3ab2 + b3
(a + b)4 = a4 + 4a3b + 6a2b2 + 4ab3 + b4
A. 36
Pascal’s triangle B. 220
C. 15
D. 6
E. No clue

81
Sum each row of Pascal’s Triangle:  n  2 n
 
n
j
j 0  
Powers of 2
Proof ???
Suppose you have a set of size
n. How many subsets does 2n
it have?
How many subsets of size 0

does it have?

does it have?

does it have?
Add them up we have the result. 82

n  n n
Sum each row of Pascal’s Triangle:    2
j
j 0  
Proof ???
Alternative (clever) proof? Look at binomial theorem…
n n  n  j j
(x  y )   x y
n
j
j 0  
x and y are variables; can pick

any numbers… hmm…
Pick x=1 and y=1 !
n n  n  j j
j 1 1  (1  1)n
j 0  
n n  n
j  2
j 0  
Pascal Identity
A relationship between the entries in Pascal’s triangle: n n -1 n -1
     
j j -1  j 
Suppose T is a set, |T|=n. Let a be an

element in T, and let S = T - {a}. So,
|S| = n-1.
n 
Let’s count the  j  subsets of size j.
 
Note that some of these contain a, and
some don’t.
n -1 
 
How many contain a?  j-1
S T How many don’t? n - 1 

n-1 items n items  
 j 
a
2.2. Combination
2.1.1. Definitions
85
2.2.3. Combinations of multisets
• Let M be a multiset {a1, a2, …, an} (M has n distinct objects):
– A k-combination of M is an unordered collection of k objects selected from n
types of objects of M.
– A k-combination of M is also called an k-combination with repetition allowed.
– The number of k-combination of M is C(n+ k-1, k) = C(n+ k-1, n-1)
(the number of selections, with repetitions, of k objects from n distinct objects)
Need to divide k candies for n kids B1, B2, …,Bn. How many different ways to divide?
Let tj be the number of candies for kid Bj, j=1,…,n. At this point, the above problem
leads to the problem:
Let k and n be non-negative integers. How many non-negative integers in the following
equation have?
t1  t2  t3    tn  k
t1 , t2 , , tn  Z  86
The number of r-combination of M is C(n+ r-1, r)
Proof: When k objects are selected from the multiset M, we put them into the following
n boxes
so that the ith type objects are contained in the ith box, 1 ≤ i ≤ n.
Since the objects of the same type are identical, we may use the symbol "O" to denote
any object in the boxes, and the objects in different boxes are separated by a stick "|".
For example, for n = 4 and k = 7:
Convert the symbol "O" to zero 0 and the stick "|" to one 1, any such placement is
converted into a 0-1 sequence of length k+n–1 with exactly k zeros and n–1 ones
 Now the problem becomes counting the number of 0-1 words of length k+(n–1) with
exactly k zeros and n–1 ones
= C(n+ k-1, k) = C(n+ k-1, n-1) 87
Example 1: Example 1: How many ways to divide 10 candies for 4 kids
(there may be kids without any candies)
How many solutions are there to the equation x 1  x 2  x 3  x 4  10
When the variables are nonnegative integers (xi ≥ 0)?
Answer 1: 11 locations for bars. Pick 3 locations allowing repetitions

 k-combination with repetition allowed (n=11, k = 3). C(11+3-1,3)
1  3  6  0  10
Answer 2: (bài toán chia kẹo)

Select with repetition from x1, x2, x3, x4 10 times.
For example if x1 is selected twice then x1= 2 in the final solution.
Therefore: k-combination with repetition allowed (n = 4, k =10):
C(4+10-1,10) 88
Example 2: How many nonnegative integer solutions are there to the inequality
x1  x 2  x 6  10 ?
It is equivalent to
x1  x2    x6  x7  10, 0  xi , 1  i  6, 0  x7
which can be transformed to
y1  y2 y6  y7  9, y i  0 integer
where y i  x i for 1  i  6
and y7  x7  1
 Ans1: 10 locations for bars, pick 6 locations allowing repetitions [k = 6, n = 10]
 k-combination with repetition C(6+10-1,6)=C(15,6) =5005
 Ans2: 7 types of objects to select (y1,..,y7) and 9 times of selections [k = 9, n = 7]

 k-combination with repetition C(7+9-1,9)= C(15, 9) = 5005
The number of k-combination of M is C(n+ k-1, k)
Equivalence of the following: The number C(n+k–1, k) equals
• the number of selections, with repetition, of size k from a
collection of n types objects
• the number of nonnegative integer solutions of the equation
x1 + x2 + … + xn = k
• the number of ways to place k identical objects into n
distinct containers.
• the number of non decreasing sequences of length k whose
terms are taken from the set {1, 2, …, n}.
90
2.2. Combination
2.1.1. Definitions
91
The number of ordered arrangements of n objects, in which
there are k1 objects of type 1, k2 objects of type 2, ..., and km
objects of type m and where k1+k2+ ... +km = n, is
 n  n!
 
 k1 , k2 ,..., km  k1 !k2 !...km !
92
• The Multinomial Theorem:
 n  k1 k2 km
( x1  x2  ...xm )   
n
 x1 x2 ...xm
 k1 , k2 ,..., km 
where the summation is over all sequences of non-negative integers
(k1, k2, ..., km) such that k1 + k2 + ... + km = n.
93
Summary: select or order k objects from n distinct objects
Order matters Order does not matter
1 2
Repetition
not allowed
(k-permutation of n-set) (k-combination of n-set)
3 4
Repetition
allowed (k-permutation with repetition
allowed) (k-combination with repetition allowed)
The set A = {S,O,H,N}
The 2-permutation (sequence) of set A is: P(4,2) = 12 The 2-combination (set) of set A is: C(4, 2) = 6
SO, SH, SN, OH, ON, OS, HN, HS, HO, NS, NO, NH {S,O},{S,H},{S,N},{O,H},{O,N},{H,N}
1 2
The multi set B = {S, O, H, N}
The 2-permutation with repetition allowed of set B is: 42 = 16 The 2-combination with repetition allowed of
SS, OO, HH, NN, set B is: C(4+2-1,2)=10
SO, SH, SN, OH, ON, OS, HN, HS, HO, NS, NO, NH
{S, S}, {O, O}, {H, H}, {N, N},
{S,O},{S,H},{S,N},{O,H},{O,N},{H,N}
3 4
Contents
95
3. Inclusion-exclusion principle
3.1. Inclusion-exclusion principle
3.2. Derangement
96
The inclusion–exclusion principle is an equation relating the sizes of two sets
and and their union. It states that if A and B are two (finite) sets, then
|A B| = |A| + |B| - |A B|
The meaning of the statement is that

the number of elements in the union
of the two sets is the sum of the
elements in each set, respectively,
minus the number of elements that
are in both.
Similarly, for three sets A, B and C:

|A B C| = |A| + |B| + |C| - |A B| - |A C| - |B C| + |A B C|
This can be seen by counting how many times each region in the figure is 97
included in the right hand side.
More generally, for finite sets A1, A2,…,Am
where:
Nk  
1 i1  i2 ... ik  m
| Ai1  Ai2  ...  Aik |, k  1, 2,..., m
Note: Nk is the sum of the cardinalities of all intersections of k

from m given sets. For example:
N1 = |A1| + ... + |Am|
Nm = |A1  A2  ...  Am|
m
| A1  A2  ...  Am | | A
i 1
i | 
pairs ( ij )
| Ai  A j |  
triples ( ijk )
| Ai  A j  Ak | ...
 (  1) m 1 | Ai  A j  Ak  ...  Am | 98
Example 1: How many integers from 1 to 1000 are either multiples of 3
or multiples of 5?
• A = set of all integers from 1 to 1000 that are multiples of 3.
• B = set of all integers from 1 to 1000 that are multiples of 5.
• AB=?
The set of all integers from 1 to 1000 that are multiples of either 3 or 5.
• AB=?
The set of all integers that are both multiples of 3 and 5, which also is
the set of integers that are multiples of 15.
• To use the inclusion-exclusion principle to obtain |A  B|, we need
– |A|
– |B|
– |A  B|
99
• A = set of all integers from 1 to 1000 that are multiples of 3  |A| = ?
From 1 to 1000, every third integer is a multiple of 3, each of this
multiple can be represented as 3p, for any integer p from 1 through
[1000/3]=333, Hence |A| = 333.
• B = set of all integers from 1 to 1000 that are multiples of 5 |B| = ?
Similarly for multiples of 5, each multiple of 5 is of the form 5q for
some integer q from 1 through [1000/5]=200. Hence, we have |B| = 200.
• A  B: set of all integers from 1 to 1000 that are multiples of 15 |A  B|=?
To determine the number of multiples of 15 from 1 through 1000, each
multiple of 15 is of the form 15r for some integer r from 1 through
[1000/15]=66.  |A  B| = 66.
• From the principle, we have the number of integers either multiples of
3 or multiples of 5 from 1 to 1000 given by
|A  B| = |A| + |B| - |A B|
=333 + 200 – 66 = 467
100
Example 2: There are 350 applicants to a job, and
• (i) 220 with major in CS
• (ii) 147 with major in Business
• (iii) 51 with major in both CS and Business
How many applicants have major neither in CS nor Business ?
Answer:
• A: set of applicants with major in CS
• B: set of applicants with major in Bussiness
 AB:
Set of applicants with major both in CS and bussiness
 AB:
Set of applicants with major in CS or business or both
|A  B| = |A| + |B| - |A B| = 220 + 147 – 51 = 316
 The desired answer : 350 – 316 = 34 101
Example 3: In a class of students undergoing a computer course the following were
observed:
– Out of a total of 50 students: 30 know Java, 18 know C#, 26 know C, 9 know
both Java and C#, 16 know both Java and C, 8 know both C# and C, 47 know at
least one of the three languages.
From this we have to determine:
a. How many students know none of these languages?
b. How many students know all three languages?
a. 50 – 47 = 3
b.
A = All the students who know Java in class.
B = All the students who know C# in the class.
C = All the students who know C in class.
 Students know all three languages, so we need to find |A  B  C|
We have to derive the inclusion-exclusion formula for three sets
|A  B  C| = (|A| + |B| + |C|) –(|B  C| + |A  B| + |A  C|) + |A  B  C| 102
– Out of a total of 50 students: 30 know Java, 18 know C#, 26 know C, 9 know
both Java and C#, 16 know both Java and C, 8 know both C# and C, 47 know at
least one of the three languages.
A = All the students who know Java in class.
B = All the students who know C# in the class.
C = All the students who know C in class.
 Students know all three languages, so we need to find |A  B  C|.
|A  B  C| = (|A| + |B| + |C|) – (|B  C| + |A  B| + |A  C|) + |A  B  C|

47 = (30+18+26) – (8+9+16) + |A  B  C|
 |A  B  C| = 6
103
Example 4: How many binary strings of length 10 such that either start
with 00 or end with 11?
Answer:
A: set of binary strings of length 10 that start with 00 00xxxxxxxx
B: set of binary strings of length 10 that end with 11 8 positions
 Need to calculate |A  B| Each position: 0/1
The inclusion-exclusion formula for 2 sets:

|A B| = |A| + |B| - |A B|
104
3. Inclusion-exclusion principle
3.2. Derangement
105
3.2. Derangement
A derangement of {1, 2, …, n} is a permutation on the set such that none
of elements i is placed at position ith in the permutation.
(In other words, derangement is a permutation that has no fixed points)
Example: n = 5
• Permutation (2,3,5,4,1) is not a derangement
• Permutation (2,3,5,1,4) is a derangement
Denote Dn the number of derangements of {1, 2, …, n} Dn = ??

106
3.2. Derangement
• Denote Dn the number of derangements of {1, 2, …, n} Dn = ??
• Let S = set of all permutations of {1, 2, …, n}  |S| = n!
• Let Ai = subset of permutations of {1, 2, …, n} such that the ith element = i
in the permutation.
 ounts the number of permutations in which at least
one object i of the n objects appears in the ith position (its original position).
 Dn: number of permutations such that none of the n objects appears in their
original positions. Therefore:
Dn = n! -
Remember the inclusion-exclusion principle:
107
3.2. Derangement
Let Ai = subset of permutations of {1, 2, …, n} such that the ith element = i in
the permutation.
• |Ai| = (n-1)! since if exactly one of the n objects is placed at its original
position, that leaves the other (n-1) objects to be freely permuted in (n-1)!
possible ways. Hence,
since there are n terms in the sum.

• |AiAj| = (n-2)! since if exactly two of the n objects are placed at their
original positions, that leaves the other (n-2) objects to be freely permuted in
(n-2)! possible ways. Hence,
since there are C(n,2) terms in the sum

108
3.2. Derangement
the permutation.
1
• |Ai Aj Ak | = (n-3)! since if exactly three of the n objects are placed at their
original positions, that leaves the other (n-3) objects to be freely permuted in
(n-3)! possible ways. Hence,
since there are C(n,3) terms in the sum.

The pattern should be clear. When we reach the final term of 1 , we have
|A1 A2 …. An |=1
which corresponds to all n objects being placed at their original positions.
109
3.2. Derangement
the permutation.
We have:
Dn = n! -
110
3.2. Derangement
• An example of derangements arises in a very famous problem called
the hat-check problem:
“n people walk into a party and give their hats to hat-check girl.
Unfornately, she completely loses track of which of n hats belong to
which owners. Therefore, when the party finishes, their hats are returned
at random.”
Question: What is the probability that nobody receives their own hat
back?
This is equivalent to asking for the probability that a permutation of n
objects is a derangement.
 = What is the value in number????

since there are Dn possible derangements and n! possible permutations.
111
3.2. Derangement
It is amusing to note that as n → ∞, the probability that a permutation of n
values is a derangement is given by
 if n is large (in practical applications, any n greater than about 10 can be

considered to be large), then the probability that the permutation of the n values
is a derangement is approximately 1/e almost independently of the precise value
of n. n n! Probability #Derangements
0 1 1
1 1 0 0
2 2 0.5 1
3 6 0.333333333 2
4 24 0.375 9
5 120 0.3666666667 44
6 720 0.368055556 265
7 5040 0.367857143 1854
8 40320 0.367881944 14833
9 362880 0.367879189 133496
10 3628800 0.367879464 1334961 112
Note: 1/e  0.367879441
Contents
113
4. Recurrence relations
4.1. Recurrence relations
4.2. Solve recurrence relations
114
Definition:
A recurrence relation for the sequence {an} is the equation that expresses an
in terms of one or more of the previous terms in the sequence namely, a0, a1,
…, an-1, for all integers n with n  n0, where n0 is a nonnegative integer.
A sequence is called a solution to a recurrence relation if its terms satisfy the
recurrence relation.
Example: Consider the recurrence relation
an = 2an-1 – an-2 for n = 2, 3, 4, … The sequence an=n+1??
• The sequence an=3n is a solution of this recurrence relation?
For n  2 we see that: 2an-1 – an-2 = 2(3(n – 1)) – 3(n – 2) = 3n = an
Therefore, the sequence an=3n is a solution of the recurrence relation
• The sequence an=5 is also a solution of the this recurrence relation?
For n  2 we see that: 2an-1 – an-2 = 25 - 5 = 5 = an
Therefore, the sequence an=5 is also a solution of the recurrence relation
the same recurrence relation can have multiple solutions.  Why???
A recurrence without specifying any initial values (initial conditions).
 can have (and usually has) multiple solutions.
Example: an = 2an-1 – an-2 for n = 2, 3, 4. It has the following sequences an as solution:
• an=5
• an=3n
• an=n+1
If both the initial conditions and the recurrence relation are specified, then the
sequence is uniquely determined.
Example: an = 2an-1 – an-2 for n = 2, 3, 4
where a0=0; a1 = 3
 The sequence an=5 is not the solution
 The sequence an=3n is the unique solution
Some applications of recurrence relations:
1. To solve many counting problems we can try construct recurrence
relations.
2. Complexity analysis of the recursive algorithm is a recurrence
relation on the number of operations.

Modeling with Recurrence Relations
Example 1: Someone deposits $10,000 in a savings account at a bank yielding 5% per
year with interest compounded annually. How much money will be in the account
after 30 years?
Solution: P30 = (1.05)3010,000 = 43,219.42
• Let Pn denote the amount in the account after n years.
How can we determine Pn on the basis of Pn-1?
• We can derive the following recurrence relation:
Pn = Pn-1 + 0.05Pn-1 = 1.05Pn-1.
• The initial condition is P0 = 10,000.
Then we have:
• P1 = 1.05P0
• P2 = 1.05P1 = (1.05)2P0
• P3 = 1.05P2 = (1.05)3P0
•…
•Pn = 1.05Pn-1 = (1.05)nP0
 We now have a formula to calculate Pn for any natural number n and can avoid the
iteration. 118
Example 2: Fibonacci
A pair of rabbits is placed on an island. After 2 months old, a pair of rabbits
produces another pair each month. How many rabbits are there after n months?
 n = 1: f1 = 1
 n = 2: f2 = 1 Fibonacci sequence: 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233,etc.
 n > 2: fn = fn-1 + fn-2
The n > 2 formula is true since each new pair comes from a pair at least 2 months old.
New-born Month 1-month rabbits >=2-month rabbits Total fn
rabbits
1 1 0 1
2 0 1 1
3 1 1 2
4 1 2 3
5 2 3 5
6 3 5 8
7 5 8 13
8 8 13 21
Example 2: Fibonacci
A pair of rabbits is placed
on an island. After 2
months old, a pair of
rabbits produces another
pair each month. How
many rabbits are there after
n months?
 n = 1: f1 = 1
 n = 2: f2 = 1
 n > 2: fn = fn-1 + fn-2
The n > 2 formula is true since each new pair comes from a pair at least 2 months
old.
Example 3: Let an denote the number of bit strings of length n that do
not have two consecutive 0s (“valid strings”). Find a recurrence relation
and give initial conditions for the sequence {an}.
Solution:
Idea: The number of valid strings equals the number of valid strings
starting with a 0 plus the number of valid strings starting with a 1.
• The sum rule: an = |A| + |B|.
• We have:
– A = {1??????}  |A| = an-1
– B = {0??????}  2nd place is 1  |B| = an-2
• Therefore
an = an-1 +an-2, n > 2 fn = fn-1 + fn-2 , n > 2
a1 = 2; a2 = 3; f1=1, f2=1
This sequence satisfies the same recurrence relation as the Fibonacci sequence. 121
Since a1 = f3 and a2 = f4, we have an = fn+2
Example 4: The Tower of Hanoi, consists of three towers (a), (b), (c) together with
n disks of different sizes. Initially these disks are stacked on the tower (a) in an
ascending order, i.e. the smaller one sits over the larger one.
The objective of the game is to move all the disks from tower (a) to tower (c),
following 3 rules:
• Only one disk can be moved at a time.
• Only the top disk can be moved
• No large disk can be sit over a smaller disk.
Let hn denote the minimum number of moves needed to solve the Tower of Hanoi
problem with n disks. What is the recurrence relation for hn?
122
Tower a Tower c Tower b
Tower of Hanoi: n=5
The objective of the game is to move all the disks from tower (a) to tower (c), following
3 rules:
1. Only one disk can be moved at a time.
2. Only the top disk can be moved
3. No large disk can be sit over a smaller disk.
Tower a Tower c
Toán rời rạc
Tower b
Tower of Hanoi
• h1 = 1
• For n ≥ 2, we need to do 3 following steps to transfer all disks from tower (a) to tower (c):
(1) Move the top n - 1 disks (following the rules of the game) from tower (a) to tower (b)
(2) Move the largest disk to the tower (c)
(3) Move the n-1 disks (following the rules of the game) from tower (b) to tower (c), placing
them on top of the largest disk
Tower a Tower c
Toán rời rạc
Tower b
Tower of Hanoi
• h1 = 1
• For n ≥ 2, we need to do 3 following steps to transfer all disks from tower (a) to tower (c):
The problem of n-1 disks  #moves = hn-1
 #moves = 1
(3) Move the n-1 disks (following the rules of the game) from tower (b) to tower (c), placing
The problem of n-1 disks #moves = hn-1
hn = 2hn-1 + 1, n ≥ 2
h1 = 1
Tower a Tower c
Toán rời rạc
Tower b
Tower of Hanoi: n=5
 #moves = 1
(3) Move the n-1 disks (following the rules of the game) from tower (b) to tower (c) , placing
Tower a Tower c
Toán rời rạc
Tower b
Tower of Hanoi
Find the formula to calculate hn for any natural number n and can avoid
the iteration.
hn = 2 hn−1 + 1
= 2 (2 hn−2 + 1) + 1 = 22 hn−2 + 2 + 1
= 22(2 hn−3 + 1) + 2 + 1 = 23 hn−3 + 22 + 2 + 1
…
= 2n−1 h1 + 2n−2 + … + 2 + 1
= 2n−1 + 2n−2 + … + 2 + 1 (because h1 = 1)
= 2n − 1
History “Tower of Hanoi”
Legend: a group of Eastern monks are the keepers of three towers on
which sit 64 golden rings. Originally all 64 rings were stacked on one
tower with each ring smaller than the one above. The monks are to move
the rings from this first tower to the third tower one at a time but never
moving a larger ring on top of a smaller one. Once the 64 rings have all
been moved, the world will come to an end.
 Number of moves = 264-1 = 18 446 744 073 709 551 615
 which would require more than five billion centuries!
It was invented by the French mathematician Eduoard Lucas

in 1883

Some applications of recurrence relations:
1. To solve many counting problems we can try construct recurrence
relations.
2. Complexity analysis of the recursive algorithm is a recurrence
relation on the number of operations.

Example 5. Recall the factorial function
Consider the following (recursive) algorithm for computing n!

Factorial(n);
How many times is function Factorial called when we execute the statement Factorial(n); ?
• When n = 1, Factorial is called 1 T(1) = 1
• Otherwise:
– We perform 1. T(n) = 1 + T(n-1)
– Plus the number of calls in the recursive call in Factorial(n − 1)  T(n-1)
This can be expressed as a formula (similar to the definition of n!):
T(1) = 1 Find the formula to calculate T(n) for
T(n) = 1 + T(n − 1), n >1 any natural number n and can avoid the
This is known as a recurrence relation. iteration. explicit formula
Factorial(n);
T(n) = T(n − 1) + 1, n >1

= [T(n-2) +1] + 1 [has 2 ones]
=[[T(n-3)+1]+1]+1 [has 3 ones]
=….
= T(n – (n-1)) + (n-1) [has n-1 ones]
= T(1) + (n-1)
= 1 + (n-1)
=n
How many multiplications M(n) does Factorial perform?

• When n = 1 we don’t perform any  M(1) = 0
• Otherwise:
– We perform 1.
– Plus the number of multiplications we perform in the recursive call, Factorial(n − 1)  M(n-1)
This can be expressed as a formula (similar to the definition of n!):
M(0) = 0; M(1) = 0 Find the formula to calculate T(n) for
M(n) = 1 + M(n − 1), n >1 any natural number n and can avoid the
This is known as a recurrence relation. iteration.
Example 6. Performance of Recursive Binary Search
Input: An array A consists of n elements: A[0],…,A[n-1] in ascending
order; Value key with the same data type as array A.
Output: the index in array if key is found, -1 if key is not found
To solve this problem, the divide and conquer technique is built based on the following
argument: The given key is either
equal to the element at the middle of array A
or equal to the element at the left (L) half of array A
or equal to the element at the right (R) half of array A.
or not in array
134
Input: An array A consists of n elements: A[0],…,A[n-1] in ascending order;
Value key with the same data type as array A.
To solve this problem, the divide and conquer technique is built based on the following
argument: The given key is either
equal to the element at the middle of array A
or equal to the element at the left (L) half of array A
or equal to the element at the right (R) half of array A.
or not in array
int binsearch(int low, int high, int A[], int key)
{
if (low <= high)
{
mid = (low + high) / 2;
if (A[mid]==key) return mid;
else if (key < A[mid])
return binsearch(low, mid-1, A, key);
else
return binsearch(mid+1, high, A, key);
}
else return -1; 135
}
Example: Binary Search
{
if (low <= high)
{
if (A[mid]== key) return mid;
key=33
else
}
else return -1;
}
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo hi
binsearch(0, 14, A, 33);
136
{
if (low <= high)
{
key=33
else
}
else return -1;
}
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo mid hi

The section to be binsearch(0, 6, A, 33);
investigated is halved
137
after each iteration
{
if (low <= high)
{
key=33
else
}
else return -1;
}
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo hi

138
{
if (low <= high)
{
key=33
else
}
else return -1;
}
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo mid hi

investigated is halved binsearch(4, 6, A, 33);
139
after each iteration
{
if (low <= high)
{
key=33
else
}
else return -1;
}
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo hi

140
{
if (low <= high)
{
key=33
else
}
else return -1;
}
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo mid hi

investigated is halved binsearch(4, 6, A, 33);
141
after each iteration binsearch(4, 4, A, 33);
{
if (low <= high)
{
key=33
else
}
else return -1;
}
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo
hi
142
{
if (low <= high)
{
key=33
else
}
else return -1;
}
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo
hi
mid binsearch(0, 14, A, 33);
143
{
if (low <= high)
{
key=33
Value 33 is located @index=4
else
}
key=31??
else return -1;
}
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo
hi
mid binsearch(0, 14, A, 33);
144
Input: An array A consists of n elements: A[0],…,A[n-1] in ascending
order; Value key with the same data type as array A.
{
if (low <= high)
{
else if (key < S[mid])
else
}
else return -1;
}
 binsearch(0, n-1, A, key);
How many times is binsearch called in the worst case ?
145
{
if (low <= high)  T(0) = ?
{
 binsearch(0, -1, A, key);
if (A[mid]==key) return mid;  T(1) = ?
return binsearch(low, mid-1, A, key);  binsearch(0, 0, A, key);
else
 binsearch(0, -1, A, key);
}  binsearch(1, 0, A, key);
else return -1;
}
Let T(n): the number of times that binsearch is called in the worst case when array A has n elements
• T(0) = 1 Recurrence relation:
• T(1) = 2 T(n) = T(n/2) + 1
• T(2) = T(1) + 1 = 3 T(0) = 1
• T(4) = T(2) + 1 = 4 T(1) = 2
• T(8) = T(4) + 1 = 4 + 1 = 5 NGUYỄN KHÁNH PHƯƠNG
• T(n) = T(n/2) + 1
4. Recurrence relations
4.2. Solving recurrence relations
147
4.2. Solving Recurrence Relations
Solving the recurrence relation for a sequence a0, a1, a2, a3,….an,… is to give
the explicit formula to compute the value for the general term an, i.e., to find an
expression for an that does not involve any other ai
Example: Given the recurrence relation:
an = 2an-1 – an-2 where n = 2, 3, 4,…
a0=0; a1 = 3
 The explicit formula for the above recurrence relation is an=3n
 an=3n is the solution to the above recurrence relation
• Does not exist the method to solve all types of recurrence relation.
• Consider method to solve the recurrence relation with following types:
– A linear homogeneous recurrence relation of degree k with constant
coefficients
– A linear nonhomogeneous recurrence relation of degree k with constant
coefficients
148
Definition: A linear homogeneous recurrence relation of degree
k with constant coefficients is a recurrence relation of the form:
an = c1an-1 + c2an-2 + … + ckan-k
where c1, c2, …, ck are real number constants, and ck  0.
• A sequence satisfying such a recurrence relation is uniquely

determined by the recurrence relation if it also satisfying k initial
conditions:
a0 = C0, a1 = C1, a2 = C2, …, ak-1 = Ck-1
where C0, C1, ..., Ck-1 are constants.
149
Definition: A linear homogeneous recurrence relation of degree
k with constant coefficients is a recurrence relation of the form:
an = c1an-1 + c2an-2 + … + ckan-k
where c1, c2, …, ck are real number constants, and ck  0.
Explain:
• Linear (tuyến tính): the right-hand side is the sum of the terms
before the term an in the sequence where the coefficients (c1, c2,
..,ck) are constant (not a function dependent on n)
• Homogeneous (thuần nhất): the right-hand side has no additional
terms other than the terms ai of the sequence
• Degree k: the right hand side has the (n-k)th term of the sequence
150
A linear homogeneous recurrence relation of degree k with constant coefficients
Example 1: Which one is the linear homogeneous recurrence

relation of degree k with constant coefficients
1) an = 4an-1 +2nan-3
2) hn = 2hn-1 + 1
3) bn = 5bn-2 + 2(bn-3)2
4) qn = 3 qn-6 + qn-8
151
A linear homogeneous recurrence relation of degree k with
constant coefficients
• We try to find solution of the form an = rn, where r is a constant.
• The sequence {an = rn } is a solution of the recurrence relation
an = c1an−1 + … + ckan−k
if and only if r satisfying:
rn = c1rn−1 + … + ckrn−k, or (substract the right-hand side from the left
rk − c1rk−1 − … − ck = 0 and × by rk−n)
this is called the characteristic equation of the recurrence relation, and

its solution is called characteristic roots of the recurrence relation.
• We will use characteristic roots to obtain explicit formula for the
sequence.
152
A linear homogeneous recurrence relation of degree k with
constant coefficients
• Theorem 1. Let c1 and c2 be real numbers.
Suppose that r2 - c1 r - c2 = 0 has 2 distinct roots r1 and r2. Then the
sequence {an} is a solution of the recurrence relation
an = c1 an-1 + c2 an-2
if and only if
an = 1(r1)n + 2(r2)n (1)
n = 0, 1, ..., where 1 and 2 are constants.
153
Example 1
Fibonacci sequence is given by the following recurrence relation:
Fn = Fn-1 + Fn-2, n  2,
F0 = 0, F1 = 1.
Find the explicit formula for Fn.
Solution: Solve the characteristic equation:
r2 - r - 1 = 0,
its characteristic roots are:
1 5 1 5
r1  ; r2 
2 2
Leonardo Fibonacci
1170-1250
Recurrence relation: an = c1 an-1 + c2 an-2

Characteristic equation: r2 − c1r − c2 = 0
154
Example 1
• The explicit formula:
Fn = 1.(r1)n + 2.(r2)n
where 1,  2 are constants and could be determined by using
initial conditions F0, F1. F0= 1+ 2 = 0
F1= 1r1+ 2r2 = 1
Solving these two equations, we have: 1  1 ;  2   1

5 5
Muavre formula
Therefore
1 
    
n n
1 5 1 5  , n  0.
Fn  
5 2 2 
 
155
The case: one characteristic root with multiplicities 2
But what happens if the characteristic equation has only one root?
Theorem 2: Let c1 and c2 be real numbers with c2  0. Suppose that r2 -
c1 r - c2 = 0 has only one root r0. Then a sequence {an } is a solution of
the recurrence relation an = c1 an-1 + c2 an-2
if and only if
a n   1r0n   2 nr0n
n = 0, 1, ..., where 1 , 2 are constants.
156
Example 2
Solving the following recurrence relation
an = 6 an-1 - 9 an-2
with initial conditions a0 = 1 and a1 = 6.
Solution:
Characteristic equation:
r2 - 6 r + 9 = 0 has one root r = 3. The solution is:
an = 1 3n + 2 n 3n
To determine 1, 2 , using the initial conditions, we have:
a0 = 1 = 1 ,
a1 = 6 = 1 * 3 + 2 *1* 3
Solving these equations, we have 1 = 1 and 2 = 1.
Therefore, the solution of the recurrence relation is:
an = 3n + n 3n
Recurrence relation: an = c1an−1 + … + ckan−k
Characteristic equation: rk − c1rk−1 − … − ck = 0
157
General case
Theorem 3. Let c1, c2, ..., ck be real numbers. Assume the characteristic
equation
rk - c1 rk-1 - c2 rk-2 - . . . - ck = 0
has k distinct roots r1, r2, ..., rk . Then a sequence {an} is a solution of
the recurrence relation:
an = c1 an-1 + c2 an-2 +...+ ck an-k,
if and only if
an = 1 r1n + 2 r2n + . . . + k rkn
where n = 0, 1, 2,..., and 1, 2, ..., k are constants
158
Example 3
Solving the recurrence relation:
an = 6 an-1 - 11 an-2 + 6 an-3
where initial conditions
a0 = 2, a1 = 5, a2 = 15.
Solution: Characteristic equation
r3 - 6 r2 + 11 r - 6 = 0
has 3 distinct roots r1 = 1, r2 = 2, r3 = 3.
Therefore, the solution is
an = 1 1n + 2 2n + 3 3n for some constants 1, 2, 3
Recurrence relation: an = c1an−1 + … + ckan−k

Characteristic equation: rk − c1rk−1 − … − ck = 0
159
Example 3
Using the initial conditions, we have following equations to
determine the value for constants 1, 2, 3:
a 0 = 2 =  1 +  2 + 3
a1 = 5 = 1 + 2.2 + 3.3
a2 = 15 = 1 + 2.4 + 3.9.
Solving above equations, we have
1 = 1, 2 = -1 and 3 = 2.
Therefore, the solution of the recurrence relation is
an = 1 - 2n + 2. 3n
160
General case
Given linear homogeneous recurrence relation of degree k with constant
coefficients k
an  ci ani
Its characteristic equation is: i 1
k
r k  ci r k i  0
i 1
Theorem 4: If the characteristic equation has t distinct roots

r1,…,rt with multiplicities m1,…,mt (m1+…+mt = k). Then:
an = ( , + , n+…+ , ) +
( + n+…+ ) + ... t  mi 1  n
an   i, j n ri
, , , j
(t,0 +  , n + … +  )
i 1  j 0 
,
where n≥0, and αij are constants.
161
Example 4
Solve the following recurrence relation:
cn = – 4cn-1 + 3cn-2 + 18cn-3 , n  3,
c0 = 1; c1 = 2; c2 = 13.
Solution: Characteristic equation
r3 + 4r2 – 3r – 18 = (r – 2)(r + 3)2 = 0
Hence, the solution of the recurrence relation:
cn = α10 2n + (α20 + α21 n)(– 3)n
where α10, α20, α21 are constants
162
Example 4
These constants are determined by using initial conditions:
0 = c0 = α10 20 + (α20 + α21.0) (-3)0 = α10 + α20
2 = c1 = α10 21 + (α20 + α21.1) (-3)1 = 2α10 - 3α20 - 3α21
13 = c2 = α10 22 + (α20 + α21.2) (-3)2 = 4α10 + 9α20 +18α21
Solving three equations above, we have: α10 =1 ; α20 = -1; α21=1
The solution of recurrence relation is:

cn  2  (1  n)(3)
n n
163
Example 5
Solving the recurrence relation
an = 7an-1 -16an-2 +12an-3, n  3;
a0 = 1, a1 = 5, a2 = 17.
Linear nonhomogeneous recurrence relation with constant coefficients
• Linear nonhomogeneous recurrence relation with constant

coefficients) has the term F(n) dependent on n (and
independent on any value of ai ) :
an = c1an−1 + … + ckan−k + F(n)
Linear homogeneous recurrence relation Nonhomogeneous term

Solving Linear nonhomogeneous recurrence relation
The following result is the basis for solving the nonhomogeneous
recurrence relation:
– If an = p(n) is a particular solution of linear nonhomogeneous
recurrence relation
 k 
an   ci ani   F (n)
 i1 
Then the solution of linear nonhomogeneous recurrence relation is of
the form:
an = p(n) + h(n),
where an = h(n) is the solution of the linear homogeneous part:
k
an   ci an i
i 1
Then, we have the method for solving linear nonhomogeneous recurrence relation:
1) Find the solution h(n) of linear homogeneous part.
2) Find particular solution p(n) of linear nonhomogeneous recurrence relation.
3) The solution of linear nonhomogeneous recurrence relation is of the form:
an = h(n) + p(n)
4) Determine the constants from equations obtained by initial conditions
How to find particular solution p(n) of linear nonhomogeneous recurrence
relation?
There is no known general method for solving nonhomogeneous linear

recurrence relation. However, we can develop a method for solving the
special case:
F(n) = G(n)  sn = (btnt + bt-1nt-1 + ... + b0)sn
where s is a constant; G(n) is polynomial in n
• If s is not equal to characteristic root, then particular solution p(n) is of
the form F(n)
• If s is equal to characteristic root with multiplicities m, then particular
solution p(n) is the form of nmQ(n)sn where Q(n) is of the form G(n)
Example 1: Tower of Hanoi
Example 4: The Tower of Hanoi, consists of three towers (a), (b), (c) together with n
disks of different sizes. Initially these disks are stacked on the tower (a) in an
ascending order, i.e. the smaller one sits over the larger one.
The objective of the game is to move all the disks from tower (a) to tower (c),
following 3 rules:
• Only one disk can be moved at a time.
• Only the top disk can be moved
• No large disk can be sit over a smaller disk.
Let an denote the minimum number of moves needed to solve the Tower of Hanoi
problem with n disks. What is the recurrence relation for an?
169
Tower a Tower c Tower b
To solve the recurrence relation, we could use the substitution method:
an = 2 an−1 + 1
= 2 (2 an−2 + 1) + 1 = 22 an−2 + 2 + 1
= 22(2 an−3 + 1) + 2 + 1 = 23 an−3 + 22 + 2 + 1
…
= 2n−1 a1 + 2n−2 + … + 2 + 1
= 2n−1 + 2n−2 + … + 2 + 1 (as a1 = 1)
= 2n − 1
170
Solving:
an = 2an-1 + 1, n  1;
a1 = 1.
Solution:
(1) Linear homogeneous part: an = 2an-1
 Characteristic equation: r - 2 = 0 has one root r = 2
 Solution of linear homogeneous part is: h(n) = c12n
(2) Linear Nonhomogenenous part F(n) = 1, then the particular solution is
of the form
p(n) = C
Substitute to the nonhomogeneos recurrent relation: C = 2C+1  C = -1

Hence the particular solution is: p(n) = -1.
Solving:
an = 2an-1 + 1, n  1;
a1 = 1.
(3) The solution of linear nonhomogeneous recurrence relation:

an = h(n) + p(n) = c12n – 1
(4) Determnine c1 by using initial condition:
a1 = c121 -1 = 1
 c1 = 1
Hence, the solution of linear nonhomogeneous recurrence relation:
an = 2n -1, n  1.
Example 2
an=5an-1 - 6an-2+7n, n2,
a0 = 0; a1 = 1
Solution:
(1) Linear homogeneous part: an=5an-1 - 6an-2
Characteristic equation: r2 – 5r +6 = 0 has 2 distinct roots r1 = 3, r2 = 2
Thus the solution of linear homogeneous part is:
h(n) = c13n + c22n
(2) The nonhomogeneous part: F(n) = 7n and 7 is not characteristic root, thus
the particular solution is of the form:
p(n) = C.7n
Substitute to the recurrence relation:
C7n = 5C7n-1 – 6C7n-2 + 7n  C = 49/20
Hence the particular solution is p(n) = (49/20)7n
Example 2
an=5an-1 - 6an-2+7n, n2,
a0 = 0; a1 = 1
Solution:
(3) The solution of linear nonhomogeneous recurrence relation is
an = p(n) + h(n) = (49/20)7n + c13n + c22n
(4) The constants c1, c2 are determined by using initial conditions:
a0 = c1 + c2 + 49/20 = 0
a1 = 3c1 + 2c2 +(49/20).7 = 1
 c1 = ?, c2 = ?
Hence, the solution of linear nonhomogeneous recurrence relation is
an = …
Example 3
Solving recurrence relation:
an = an-1 + n, n  1; a1 = 2
Solution:
(1) Linear homogeneous part: an = an-1
Characteristic equation r - 1 = 0 has the root r = 1
 The solution of linear homogeneous part is: h(n) = c11n
(2) Nonhomogeneous part F(n) = n1n, and 1 is the root of multiplicities 1,
thus the particular solution is of the form
p(n) = n1(C2 + C3n)1n
Substitute to the recurrence relation, we have:

n (C2 + C3n) = (n-1) [C2 + C3(n-1)] + n
Thus, C2 = ½ and C3 = ½ . The particular solution is
p(n) = (n+1)n/2
Example 3
Solving recurrence relation:
an = an-1 + n, n  1; a1 = 2
(3) The solution of linear nonhomogeneous recurrence relation is

an = c1+ (n+1)n/2
(4) The constant c1 is determined by using initial condition:
a1 = c 1 + 1 = 2
 c1 = 1
The solution of linear nonhomogeneous recurrence relation is
an = 1+ (n+1)n/2, n  1
Example 4
Solving the recurrence relation: an = 6an-1 -9an-2 + F(n)
Characteristic equation: r2 – 6r + 9 = (r - 3)2=0
 Characteristic root with multiplicities 2: r = 3
Consider some cases F(n) = G(n)sn where G(n) is polynomial of n:
• (1) Assume F(n) = n22n
– As: F(n) = G(n)sn = n22n
 Degree of G(n) is t = 2, s = 2 is not the characteristic root
– Thus, the particular solution is of the form:
(ptnt + pt-1nt-1 + ... + p0)sn
=(p2n2 + p1n + p0)2n
– As F(n) = G(n)sn = n23n
 Degree of G(n) is t = 2, s = 3 is characteristic root with multiplicities 2
nm(ptnt + pt-1nt-1 + ... + p0)sn
=n2(p2n2 + p1n1 + p0)3n
Example 4
Solving the recurrence relation: an = 6an-1 -9an-2 + F(n)
Characteristic equation: r2 – 6r + 9 = (r - 3)2=0
 Characteristic root with multiplicities 2: r = 3
Consider some cases F(n) = G(n)sn where G(n) is polynomial of n:
• (3) Assume F(n) = 3n
– As: F(n) = G(n)sn = 13n
=n2(p0)3n
– As F(n) = G(n)sn = n3n
=n2(p1n1 + p0)3n
Example 5
Solving the recurrence relation:
an = an-1 +an-2 +2n n  2;
a0 = 1, a1 = 2, a2 = 7.
Recurrence tree method
In this method, we draw a recurrence tree :
• Each node of the tree corresponds to an input data function.
As going down the tree, the size of the input data will
decrease.
• The last level of the tree corresponds to the smallest input
data size.
We calculate the time taken by every level of tree, and finally,
we sum the work done at all levels.
Example 1: Solving the recurrence relation:
T(n) = 2T(n/2) + n, T(1) = 4
n*4
• The last level log2n has the value = 4n, thus we have:
Example 2: Performance of Recursive Binary Search
{
if (low <= high)
{
else
}
else return -1;
}

Let T(n): the number of times that binsearch is called in the worst case when array A has n elements
• T(0) = 1 Recurrence relation:
• T(1) = 2 T(n) = T(n/2) + 1
• T(2) = T(1) + 1 = 3 T(0) = 1
• T(4) = T(2) + 1 = 4 T(1) = 2
• T(8) = T(4) + 1 = 4 + 1 = 5 NGUYỄN KHÁNH PHƯƠNG
• T(n) = T(n/2) + 1
Example 3: Solving the recurrence relation
T(n) = T(n/) + f(n), T() = c
Example 3: Solving the recurrence relation
T(n) = T(n/) + f(n), T() = c
Iteration Cost
0 TT(n) f(n)
1 f(n/)
T(n/) T(n/)    T(n/)
2 T(n/2) 2f(n/2)
   T(n/2) T(n/2)    T(n/2)
. .
. .
i i f(n/i)
. .
log n
• The value of T(n) is the sum of the cost of all level on the tree:
2.5. Recurrence relations and generating functions
2.5.1. Recurrence relations

2.5.2. Generating functions
185
Generating functions are a tool to solve a wide variety of counting
problems and recurrence relations.
Example 1:
How many ways to give 12 oranges for three children: A, B and C such that:
A gets at least four, and B and C gets at least two, but C gets no more than five
a, b , c are the number of oranges

that A, B and C gets, respectively
Find the number of integer solutions to

a + b + c = 12 where a ≥ 4, b ≥ 2, 2 c  5
186
Example 1: Find the number of integer solutions to
a + b + c = 12 where a ≥ 4, b ≥ 2, 2 c  5
Let
a: a(x) = x4 + x5 + x6 + x7 + x8 (since b + c  4  a  8)
b : b(x) = x2 + x3 + x4 + x5 + x6 (since a + c  6  b  6)
c : c(x) = x2 + x3 + x4 + x5 (since 2  c  5)
The coefficient of x12 in g(x) = a(x) b(x) c(x)
n=12
=(x4+ x5+ x6+ x7+ x8) (x2+ x3+x4+ x5+ x6)(x2+ x3+x4+ x5)
=???
is the solution to the problem
g(x) is called a generating function.
Why???
Assume xa, xb, xc are terms derived from (x4+ x5+ x6+ x7+ x8) , (x2+ x3+x4+ x5+ x6), (x2+ x3+x4+ x5)
respectively when developing the right-hand side  4 a  8, 2 b  6, 2 c  5
When developing the right-hand side, these three terms give us the term xn where n = a +b + c:
g(x) = (x4+ x5+ x6+ x7+ x8) (x2+ x3+x4+ x5+ x6)(x2+ x3+x4+ x5)
= …. + Kxn +…..
 The coefficient of xn in f(x) is the number of positive integer solutions to
a + b + c = n where 4 a  8, 2 b  6, 2 c  5
187
Example 1: Find the number of integer solutions to
a + b + c = 12 where a ≥ 4, b ≥ 2, 2 c  5
Let
a: a(x) = x4 + x5 + x6 + x7 + x8 (since b + c  4  a  8)
b : b(x) = x2 + x3 + x4 + x5 + x6 (since a + c 6  b  6)
1. x4*x3*x5
c : c(x) = x2 + x3 + x4 + x5 (since 2  c  5) 2. x4*x4*x4
The coefficient of x12 in g(x) = a(x) b(x) c(x) 3. x4*x5*x3
=(x4+ x5+ x6+ x7+ x8) (x2+ x3+x4+ x5+ x6)(x2+ x3+x4+ x5) 4. x4*x6*x2
which is 14, is the solution 5. x5*x2*x5
g(x) is called a generating function. 6. x5*x3*x4
7. x5*x4*x3
8. x5*x5*x2
9. x6*x2*x4
10. x6*x3*x3
11. x6*x4*x2
12. x7*x2*x3
13. x7*x3*x2
14. x8*x2*x2 188
Example 2: Four kinds of jelly beans: Red, Green, White, Black. In how many
ways can we select 24 jelly beans so that we have an even number of white
beans and at least six black ones?
• White: 1+ x2+ x4+….+ x16 + x18
• Black: x6+ x7+….+ x16+ x18
• Red (green): 1+ x1+ x2+….+ x16+ x18
g(x)=(1+ x2+ x4+….+ x16+ x18)(x6+ x7+….+ x16+ x18) (1+ x1+ x2+….+ x16+ x18)2
The coefficient of x24 is the solution.
Example 3: How many nonnegative integer solutions are there for

c1+c2+c3+c4=25?
• g(x)=(1+ x1+ x2+….+ x24+ x25)4
• The coefficient of x25 is the solution
189
Definition: Let a0, a1, a2, … be a sequence of real numbers. The function

g ( x)  a0  a1 x  ...  ak x  ...   ai x i
k
i 0
is called the generating function for the given sequence.
• A finite sequence
a0, a1, a2, …, an
can be regarded as the infinite sequence
a0, a1, a2, …, an, 0, 0,… [set all terms higher than n to 0]
and its generating function
g(x) = a0 + a1x + a2x2 + … + anxn
is a polynomial.
Example 4: For the binomial coefficients we already know that:
n n
( x  y )   C ( n, k ) x y
n k nk
 ( x  1)   C ( n, k ) x k  g ( x)
n
k 0 k 0
(x+1)n is the generating function for the sequence

C(n, 0), C(n, 1),…,C(n, k),…, C(n, n), 0,0,0… 190

g ( x)  a0  a1 x  ...  ak x  ...   ai x i
k
i 0
Some useful generating function:
Ex1: (1  xn+1) = (1  x)(1 + x + x2 + x3 + ...+ xn).
So (1-xn+1)/(1-x) = 1 + x + x2 + x3 + ...+ xn,
 (1-xn+1)/(1-x) is the generating function for the sequence:
1, 1, 1, …, 1, 0, 0, 0, … . (where the first n+1 terms are 1)
Ex2: From Ex1, If n   and |x| < 1, then 1 = (1  x)(1 + x + x2 + x3 + ...).
So, 1/(1-x) = 1 + x + x2 + x3 + ...
 1/(1-x) where |x| < 1 is the generating function for the sequence:
1, 1, 1, …, 1, … .
Ex3: From Ex2, set x = ay  |ay| < 1, then
1/(1-x) = 1/(1-ay) = 1 + ay + (ay)2 + (ay)3 + ….
191
 1/(1-ay) where |ay| < 1 is the generating function for the sequence 1, a, a2, a3,…
• If we have two generating functions F(x) and G(x), we define the sum and
product as follows:
 
F ( x )   ak x k
G ( x )   bk x k
k 0 k 0

F ( x )  G ( x )    ak  bk  x k Match all terms with equal powers in x.
k 0
  k 
F ( x )G ( x)     a j bk  j  x k
k 0  j 0 
Ex4: Example why multiplying generating functions is useful:
=(1+x+x2+x3+...)(1+x+x2+x3+...) where |x| < 1
( )
  k  
   1 x k  *   1 x k     1 x   (k  1) x
   
 k k
 k 0   k 0  k 0  j 0  k 0
 1/(1-x)2 is the generating function for the sequence 1,2,3,4,….
192
Another way to show that 1/(1-x)2 is the generating function of the sequence 1,
2,3, 4…:


( )
 1/(1-x)2 is the generating function for the sequence 1,2,3,4,….
Ex5: Show that x/(1-x)2 is the generating function for the sequence 0,1,2,3,….
We have

( )
= 0x0 + x1 + 2x2 + 3x3+….
193
Ex5: Show that x/(1-x)2 is the generating function for the sequence 0,1,2,3,….
We have

( )
= 0x0 + x1 + 2x2 + 3x3+….
Ex6: Show that (x+1)/(1-x)3 is the generating function for the sequence 12,22,32,42,….
From Ex5 we have:

( )

( )
194
Ex6: Show that (x+1)/(1-x)3 is the generating function for the sequence 12,22,32,42,….
From Ex5 we have:

( )

( )
Ex7: Show that x(x+1)/(1-x)3 is the generating function for the sequence 02, 12, 22, 32,
42, ….
195
Generating functions are a tool to solve a wide variety of counting problems and
recurrence relations.

g ( x)  a0  a1 x  ...  ak x  ...   ai xi
k
i 0
Example 1: Solve the recurrence relation an-3an-1 = n, n1, a0=1
Let 𝑔 𝑥 = 𝑎 𝑥 be the generating function for a0, a1,…, an
Remember that (see Ex5) 196


g ( x)  a0  a1 x  ...  ak x k  ...   ai x i
i 0
xn
We find an by determining the coefficient of in g(x)
 determining the coefficient of xn in each of the three summands 197
We find an by determining the coefficient of xn in each of the three summands

1. (7/4)/(1-3x) = (7/4) [1/(1-3x)]
= (7/4) [1 + 3x + (3x)2 + (3x)3+….] See Ex3.
 The coefficient of xn is (7/4)3n
2. (-1/4)/(1-x) = (-1/4) [1/(1-x)]
= (-1/4) [1 + x + (x)2 + (x)3+….] See Ex2.
 The coefficient of xn is (-1/4)
3. (-1/2)/(1-x)2 = (-1/2) [1/(1-x)2]
= (-1/2) [1 + 2x + 3(x)2 + 4(x)3+….] See Ex4.
 The coefficient of xn is (-1/2)(n+1)
Therefore, an = (7/4) 3n – (1/2)n -3/4, n ≥ 0
198
Solve the recurrence relation an = 2 an−1 + 1, n1, a0=0
)-1
( )-1
Remember that (see Ex2)

( )
199
( )-1
Remember that (see Ex2)

( )
( -1=
( (
( (
200
( (
We find an by determining the coefficient of xn in each of the two summands

1. (1)/(1-2x) = (1) [1/(1-2x)] See Ex3.
2 3
= (1) [1 + 2x + (2x) + (2x) +….]
The coefficient of xn is (1)2n
2. (-1)/(1-x) = (-1) [1/(1-x)]
= (-1) [1 + x + (x)2 + (x)3+….] See Ex2.
The coefficient of xn is (-1)
Therefore, an = 2n -1, n ≥ 0
201
Example 3: Solve the recursive relation an+2-5an+1+6an=2, n0, a0=3, a1=7
202
Example 3: Solve the recursive relation an+2-5an+1+6an=2, n0, a0=3, a1=7
Therefore, an =2(3n)+1, n ≥ 0 203

Part1 1counting

Uploaded by

Copyright:

Available Formats

Part1 1counting

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Part1 1counting

Uploaded by

Copyright:

Available Formats

TRƯỜNG ĐẠI HỌC BÁCH KHOA HÀ NỘI

VIỆN CÔNG NGHỆ THÔNG TIN VÀ TRUYỀN THÔNG

Nguyễn Khánh Phương

Department of Computer Science

NGUYỄN KHÁNH PHƯƠNG

NGUYỄN KHÁNH PHƯƠNG

Generalizes to multiple tasks ...

Generalizes to multiple tasks ...

select one of them: m1 +m2 +m3 +...+ mk ways

Generalized sum rule:

select one of them: m1 +m2 +m3 +...+ mk ways

select one of them: m1 +m2 +m3 +...+ mk ways

Generalize to multiple types of objects:

• Generalize to multiple types of objects:

• Generalize to multiple tasks ...

License Plate LL-DDDD 2 letters – 4 digits

Suppose a procedure can be constructed by a

Step 1 Step 2 Step 3 Step k

Number of possible ways to complete the procedure is

License Plate LL-DDDD 2 letters – 4 digits

The product rule can also be phrased in terms of set theory:

NGUYỄN KHÁNH PHƯƠNG

A 4-tuple = (1st prize, 2nd prize, 3rd prize, 4th prize) 26

So, how many Boolean

43= 64 = |B| |A|

NGUYỄN KHÁNH PHƯƠNG

(b) end with an even digit?

• Combining the product rule and the sum rule.

• Thus we can solve more interesting and complex problems.

rule of sum rule of product

Example 3: Passwords consist of character strings of 6 to 8 characters. Each

Example 3: Passwords consist of character strings of 6 to 8 characters. Each

Example 4: An address is a string of 32 bits – it begins with a network id

Alternative way to get the answer ???

2nd set winner

3rd set winner NGUYỄN KHÁNH PHƯƠNG

The number of distinct systems is: 42

There are 12 possible combined outcomes: 2 ways in which the coin

Fruit salad is a combination of apples, grapes and bananas. We don't

Example: 3-permutation of a set A of 4 objects x1, x2, x3, x4

 “permutations” means “n-permutations”.

Example 1: 6 persons are competing for 4 prizes. How many different

1st prize 2nd prize 3rd prize 4th prize

P(n, k): k unique choices out of n objects, order matters

Note: We can also argue by directly using the Product rule:

• A circular k-permutation of a set A of n objects is an ordered k objects

Proposition 1. Let M be a multiset of r different types where each type has

Example 1. What is the number of binary numerals with at most 4 digits?

Example 2. What is the number of ternary numerals with at most 4 digits?

Proposition 1. Let M be a multiset of r different types where each type has

1st position 2nd position 3rd position …. kth position

r choice r choice r choice

• Let S be the set consisting of the elements

Example 3: The number of 0-1 words of length n with exactly

m !(n  m)! m

NGUYỄN KHÁNH PHƯƠNG

b. How many ways are there to choose a president, vice

(a + b)4 = (a + b)(a + b)(a + b)(a + b)