Probability & Statistics for Scientist and Engineers

Dr. M. M. Bhatti

College of Mathematics and Systems Science, Shandong University of Science and

Technology, Qingdao, China
[email protected], [email protected]

March 17, 2023

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 1 / 29

Outline

1 Introduction (continues)
Bayes’ Theorem or Rule
Combinatorial Analysis & Fundamental principle of counting
Combinational analysis, counting, and tree diagrams
Permutations

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 2 / 29

Bayes’ Theorem or Rule
Bayes’ Theorem was named after 18th-century mathematician Thomas Bayes.
Bayes’ Theorem is a mathematical formula used to calculate the probability
of an event occurring. It’s based on the fact that the probability of another
event has already occurred. The theorem was unused for two centuries
because of the high volume of calculation capacity required to execute its
transactions.
Bayes’ Theorem have been used in wide variety of problems in probability
theory.
Statistics
Machine learning
Artificial intelligence
Data mining
Decision analysis
Finance (calculating or updating risk evaluation)
And many other fields
Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 3 / 29
Bayes’ Theorem or Rule

What is meant by Bayes theorem in probability?

In Probability, Bayes theorem is a mathematical formula, which is used
to determine the conditional probability of the given event. Conditional
probability is defined as the likelihood that an event will occur, based on
the occurrence of a previous outcome.

How is Bayes theorem different from conditional probability?

Bayes theorem defines the probability of an event based on the prior knowl-
edge of the conditions related to the event. In case, if we know the condi-
tional probability, we can easily find the reverse probabilities using the Bayes
theorem.

When can we use Bayes theorem?

Bayes theorem is used to find the reverse probabilities if we know the con-
ditional probability of an event.
Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 4 / 29
Bayes’ Theorem or Rule

Suppose that A1 , A2 , . . . An , are mutually exclusive events whose union is

the sample space S, i.e., one of the events must occur. Then if A is any
event, we have the following important theorem
Theorem 1.11: Bayes’ Rule

P (Ar )P (A|Ar )
P (Ar |A) = Pn , r = 1, 2, · · · , n (1)
j=1 P (Aj )P (A|Aj )

This enables us to find the probabilities of the various events

A1 , A2 , . . . An , that can cause A to occur. For this reason Bayes’ theorem
is often referred to as a theorem on the probability of causes.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 5 / 29

Bayes’ Theorem or Rule
Problem 1.16
Prove Bayes’ theorem. (Theorem 1.11)

P (Ar )P (A|Ar )
P (Ar |A) = Pn , r = 1, 2, · · · , n (2)
j=1 P (Aj )P (A|Aj )

Since A results in one of the mutually exclusive events A1 , A2 , . . . , An , we

have by Theorem 1.10 (Problem 1.14),

P (A) = P (A1 )P (A|A1 ) + P (A2 )P (A|A2 ) + · · · + P (An )P (A|An )

Xn
= P (Aj )P (A|Aj ) (3)
j=1

P (Ar ∩ A) P (Ar )P (A|Ar )

∴ P (Ar |A) = = Pn (4)
P (A) j=1 P (Aj )P (A|Aj )

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 6 / 29

Bayes’ Theorem or Rule
Problem 1.17
Suppose in Problem 1.15a that the one who tosses the coin does not reveal whether it
has turned up heads or tails (so that the box from which a marble was chosen is not
revealed) but does reveal that a red marble was chosen. What is the probability that Box
I was chosen (i.e., the coin turned up heads)?
a
Box I contains 3 red and 2 blue marbles while Box II contains 2 red and
8 blue marbles. A fair coin is tossed. If the coin turns up heads, a marble is
chosen from Box I; if it turns up tails, a marble is chosen from Box II. Find the
probability that a red marble is chosen.

Let us use the same terminology as in Problem 1.15, i.e., A = R, A1 = I, A2 = II. We

seek the probability that Box I was chosen given that a red marble is known to have been
chosen. Using Bayes’ rule with n = 2, this probability is given by
  
1 3
P (I)P (R|I) 2 3+2 3
P (I|R) = =      =
P (I)P (R|I) + P (II)P (R|II) 1 3 1 2 4
+
2 3+2 2 2+8
(5)
Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 7 / 29
Bayes’ Theorem or Rule

Problem 1.18
A bag I contains 4 white and 6 black balls while another Bag II contains 4
white and 3 black balls. One ball is drawn at random from one of the bags,
and it is found to be black. Find the probability that it was drawn from Bag
I.
Let E1 be the event of choosing bag I, E2 the event of choosing bag II, and
A be the event of drawing a black ball.
Then,
1
P (E1 ) = P (E2 ) = (6)
2
Also, P (A|E1 ) = P (drawing a black ball from Bag I)= 6/10 = 3/5.
P (A|E2 ) = P (drawing a black ball from Bag II)= 3/7

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 8 / 29

Bayes’ Theorem or Rule

By using Bayes’ theorem, the probability of drawing a black ball from bag I
out of two bags,

P (A|E1 )P (E1 )
P (E1 |A) = (7)
P (A|E1 )P (E1 ) + P (A|E2 )P (E2 )

1 3
× 7
P (E1 |A) = 2 5 = (8)
1 3 1 3 12
× + ×
2 5 2 5

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 9 / 29

Bayes’ Theorem or Rule

Problem 1.19
A man is known to speak the truth 2 out of 3 times. He throws a die and
reports that the number obtained is a four. Find the probability that the
number obtained is actually a four.

Let A be the event that the man reports that number four is obtained.
Let E1 be the event that four is obtained and E2 be its complementary
event.
Then, P (E1 ) = Probability that four occurs= 1/6.
P (E2 ) = Probability that four does not occur = 1 − P (E1 ) = 1 − (1/6) =
5/6.
Also, P (A|E1 ) = Probability that man reports four and it is actually a four
= 2/3
P (A|E2 ) = Probability that man reports four and it is not a four = 1/3.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 10 / 29

Bayes’ Theorem or Rule

By using Bayes’ theorem, probability that number obtained is actually a

four, P (A|E1 ) =

P (A|E1 )P (E1 )
P (E1 |A) = (9)
P (A|E1 )P (E1 ) + P (A|E2 )P (E2 )

1 2
× 2
P (E1 |A) = 6 3 = (10)
1 2 5 1 7
× + ×
6 3 6 3

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 11 / 29

Combinatorial Analysis & Fundamental principle of
counting

In many cases the number of sample points in a sample space is not very
large, and so direct enumeration or counting of sample points needed to
obtain probabilities is not difficult. However, problems arise where direct
counting becomes a practical impossibility. In such cases use is made of
combinatorial analysis, which could also be called a sophisticated way
of counting.
Fundamental principle of counting: tree diagram
If one thing can be accomplished in n1 different ways and after this a second
thing can be accomplished in n2 different ways, . . . , and finally a kth
thing can be accomplished in nk different ways, then all k things can be
accomplished in the specified order in n1 , n2 , . . . , nk different ways.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 12 / 29

Combinatorial Analysis & Fundamental principle of
counting
Example 1.13
If a man has 2 shirts and 4 ties, then he has 2 · 4 = 8 ways of choosing a
shirt and then a tie.

Example 1.14
Letting the shirts be represented by S1 , S2 and the ties by T1 , T2 , T3 , T4 ,
the various ways of choosing a shirt and then a tie are indicated in the tree
diagram of Fig. 1-4.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 13 / 29

Combinational analysis, counting, and tree diagrams

Problem 1.20
A committee of 3 members is to be formed consisting of one representative
each from labor, management, and the public. If there are 3 possible repre-
sentatives from labor, 2 from management, and 4 from the public, determine
how many different committees can be formed using (a) the fundamental
principle of counting and (b) a tree diagram.

(a) We can choose a labor representative in 3 different ways, and after this a
management representative in 2 different ways. Then there are 3·2 = 6
different ways of choosing a labor and management representative.
With each of these ways we can choose a public representative in 4
different ways. Therefore, the number of different committees that
can be formed is 3 · 2 · 4 = 24.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 14 / 29

Combinational analysis, counting, and tree diagrams
(b) Denote the 3 labor representatives by L1 , L2 , L3 ; the management rep-
resentatives by M1 , M2 ; and the public representatives by P1 , P2 , P3 , P4 .
Then the tree diagram of Fig. 1-10 shows that there are 24 different
committees in all. From this tree diagram we can list all these different
committees, e.g., L1 M1 P1 , L1 M1 P2 , etc.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 15 / 29

Combinational analysis, counting, and tree diagrams

Problem 1.21
A coin is tossed 3 times. Use a tree diagram to determine the various
possibilities that can arise.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 16 / 29

Permutations

Suppose that we are given n distinct objects and wish to arrange r of these
objects in a line. Since there are n ways of choosing the 1st object, and after
this is done, n−1 ways of choosing the 2nd object, . . . , and finally n−r+1
ways of choosing the rth object, it follows by the fundamental principle of
counting that the number of different arrangements, or permutations as
they are often called, is given by

n Pr = n(n − 1)(n − 2) . . . (n − r + 1) (11)

where it is noted that the product has r factors. We call n Pr the number
of permutations of n objects taken r at a time.
In the particular case where r = n, (11) becomes

n Pn = n(n − 1)(n − 2) . . . 1 = n! (12)

which is called n factorial.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 17 / 29

Permutations

We can write (11) in terms of factorials as

n!
n Pr = (13)
(n − r)!

If r = n, we see that (13) and (12) agree only if we have 0! = 1, and we

shall actually take this as the definition of 0!.
To help you to remember, think “Permutation ... Position”
Example 1.15
The number of different arrangements, or permutations, consisting of 3
letters each that can be formed from the 7 letters A, B, C, D, E, F, G is
7! 7·6·5·4·3·2·1
7 P3 = = = 7 · 6 · 5 = 210 (14)
4! 4·3·2·1

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 18 / 29

Permutations

Suppose that a set consists of n objects of which n1 are of one type (i.e.,
indistinguishable from each other), n2 are of a second type, . . . , nk are of
a kth type. Here, of course, n = n1 + n2 + · · · + nk . Then the number of
different permutations of the objects is
n!
n Pn1 ,n2 ,...,nk = (15)
n1 !n2 ! . . . nk !

Example 1.16
Five red marbles, two white marbles, and three blue marbles are arranged
in a row. If all the marbles of the same color are not distinguishable from
each other, how many different arrangements are possible?

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 19 / 29

Permutations

Assume that there are N different arrangements. Multiplying N by the

numbers of ways of arranging:
(a) the five red marbles among themselves,
(b) the two white marbles among themselves,
(c) the three blue marbles among themselves (i.e., multiplying N by
5!2!3!),
We obtain the number of ways of arranging the 10 marbles if they were all
distinguishable, i.e., 10!. Then
10!
(5!2!3!)N = 10! or N = (16)
5!2!3!
In general, the number of different arrangements of n objects of which n1
n!
are alike, n2 are alike, . . . , nk are alike is where n1 + n2 +
n1 !n2 ! . . . nk !
· · · + nk = n.
Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 20 / 29
Permutations

Example 1.17
The number of different permutations of the 11 letters of the word
M ISSISSIP P I, which consists of 1M , 4I 0 s, 4S 0 s, and 2P 0 s, is
11!
= 34, 650 (17)
1!4!4!2!

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 21 / 29

Permutations

Problem 1.22
In how many ways can 5 differently colored marbles be arranged in a row?

We must arrange the 5 marbles in 5 positions thus: - - - - -. The first

position can be occupied by any one of 5 marbles, i.e., there are 5 ways of
filling the first position. When this has been done, there are 4 ways of filling
the second position. Then there are 3 ways of filling the third position, 2
ways of filling the fourth position, and finally only 1 way of filling the last
position. Therefore:
Number of arrangements of 5 marbles in a row = 5 · 4 · 3 · 2 · 1 = 120
In general
Number of arrangements of n different objects in a row
= n(n − 1)(n − 2) . . . 1 = n!
This is also called the number of permutations of n different objects
taken n at a time and is denoted by n Pn .
Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 22 / 29
Permutations

Problem 1.23
In how many ways can 10 people be seated on a bench if only 4 seats are
available?
The first seat can be filled in any one of 10 ways, and when this has been
done, there are 9 ways of filling the second seat, 8 ways of filling the third
seat, and 7 ways of filling the fourth seat. Therefore:
Number of arrangements of 10 people taken 4 at a time = 10 · 9 · 8 · 7 =
5040
In general
Number of arrangements of n different objects taken r at a time
= n(n − 1)(n − 2) . . . (n − r + 1)
This is also called the number of permutations of n different objects
taken r at a time and is denoted by n Pr . Note that when r = n, n Pn = n!
as in Problem 1.19.
Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 23 / 29
Thank you for your kind attention!

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 5 March 17, 2023 24 / 29