Lecture 4 - Basic Probabaility Theory - Full
Lecture 4 - Basic Probabaility Theory - Full
Lecture 4 - Basic Probabaility Theory - Full
By
Prof. Ruwan Jayathilaka
B.A, MEcon (Colombo), MSc (NUS), PhD (Griffith)
Objectives
• Explain the importance of probability theory to statistical inference
• Define the terms random experiment, sample space, elementary
event, event, union of events, intersection of events, complement of
an event and mutually exclusive events
• Explain the different approaches to assigning probabilities to events
define the terms joint, marginal and conditional probabilities and
independent events
• Calculate probabilities using the basic probability rules - complement
rule, addition rule and multiplicative rule of probabilities
• Understand the use of probability trees in calculating complicated
probabilities
• Understand Bayes‘ law as an alternative way of calculating
conditional probabilities.
Slide 2 of 51
1
2023 August 13
Thinking Challenge
• What’s the probability
of getting a head on
the toss of a single fair
coin? Use a scale from
0 (no way) to 1 (sure
thing).
• So toss a coin twice.
Do it! Did you get one
head & one tail?
What’s it all mean?
Slide 3 of 51
Many Repetitions!*
Total Heads
Number of Tosses
1.00
0.75
0.50
0.25
0.00
0 25 50 75 100 125
Number of Tosses
Slide 4 of 51
2
2023 August 13
Introduction
•Probability theory is an integral part of all statistics,
and in particular it is essential to the theory of
statistical inference.
•Statistical inference provides the decision maker -
perhaps a business person or an economist - with a
body of methods that aid in decision-making in
situations of uncertainty.
•Probability theory can be used to measure the
degree of uncertainty.
Slide 5 of 51
3
2023 August 13
Slide 7 of 51
Sample Space - S
• Determine the outcomes
–Build an exhaustive list of all possible outcomes.
–Make sure the listed outcomes are mutually exclusive.
• A list of outcomes {O1, O2,……. On} that meets the two conditions
above is called a sample space.
S={O1, O2,……. On}
• Example: Experimental 1, sample space would be
S = {head, tail}
Experimental 2, sample space would be
S = {1,2,3,4,5,6}
• The individual outcomes in a sample space are also called simple
event or elementary events.
Slide 8 of 51
4
2023 August 13
Probability of an outcome
• The probability of an outcome is simply the chance that an outcome
Oi will occur, denoted by
P(Oi), where i = 1, 2 ,….n.
Requirements of probabilities
• Given a sample space of S={O1, O2,……. On}, the probabilities assigned
to the outcome Oi must satisfy two basic requirements:
(1) The probability of an outcome must lie between 0 and 1.
0 ≤ P (Oi) ≤ 1 for each i, i = 1, 2,…., n
(2) The sum of the probabilities of all possible outcome in a sample
space must be equal to 1. n
Slide 10 of 51
10
5
2023 August 13
Classical Approach
• is used by mathematicians to help determine the probability
logically for outcomes associated with games of chance.
• If an experiment has n possible outcomes, this method would
assign a probability of 1/n to each outcome. It is necessary to
determine the number of possible outcomes.
Slide 11 of 51
11
Classical Approach
Experiment: Rolling two dice and observing the total
outcomes: {2, 3, …, 12}
Examples:
P(2)=1/36 1 2 3 4 5 6
1 2 3 4 5 6 7
2 3 4 5 6 7 8
3 4 5 6 7 8 9
P(6)=5/36
4 5 6 7 8 9 10
5 6 7 8 9 10 11
6 7 8 9 10 11 12
P(10)=3/36
Slide 12 of 51
12
6
2023 August 13
13
‘There is a 40% chance A & B will sell 3 desktops on any given day’.
Slide 14 of 51
14
7
2023 August 13
Subjective Approach
• ‘In the subjective approach we define probability as the degree of belief
that we hold in the occurrence of an event.’
E.g. weather forecasting’s ‘P.O.P.’
• ‘Probability of Precipitation’ (or P .O.P .) is defined in different ways by
different forecasters, but basically it’s a subjective probability based on past
observations combined with current weather conditions.
• POP 60% – based on current conditions, there is a 60% chance of rain (say).
E.g.
• What is the likelihood of a recession next year?
• The outcomes could be either a recession next year or no recession next
year. But there is no reason to believe that these two outcomes are equally
likely (the equally likely approach not possible).
• This experiment cannot be replicated to use the relative frequency
approach.
• In this situation we can only use our 'subjective judgment' to assign a
probability to the above event
Slide 15 of 51
15
Event
• An event is any collection of one or more possible outcomes from
an experiment. Events are denote by capital letters. For example, in
the tossing a die experiment the event ‘an even number is
observed’ can be described as A={2, 4, 6}, where { } is read ‘the set
of consisting of’
Probability of an Event
• The probability of an event E, denote by P(E), is equal to the sum of
the probabilities assigned to the outcomes contained in E.
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑖𝑛 𝐸
P(E) = 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑖𝑛 𝑆
• For example, in the tossing a die experiment, the probability of the
event E={an even number is observed} is
1 1 1 1
P(E)=P{2,4,6}=P(2) + P(4) + P(6) = 6 + 6 + 6 = 2
Slide 16 of 51
16
8
2023 August 13
Venn diagram
• A useful geometrical representation of a sample space S, called a
Venn diagram, is presented in following figure. In a Venn diagram,
the entire sample space S is represented by a rectangle; points
inside the rectangle represent the individual outcomes in S. An
event A in the sample space S is represented by a circle, and points
inside the circle represent the individual outcomes in event A.
Slide 17 of 51
17
Venn diagram
• Now, let’s look at how to combine events to form new
ones. In arithmetic, new numbers can be created from
existing ones by means of operations such as addition and
multiplication. Similarly, in probability, new events can be
created from events already defined by means of
operations called complement of events, intersection of
events, union of events, mutually exclusive events,
dependent events and independent events.
Slide 18 of 51
18
9
2023 August 13
Slide 19 of 51
19
Complement of an Event
• The complement of event A is defined to be the event consisting of
all sample points that are ‘not in A’.
• Complement of A is denoted by Ac, 𝐴ҧ , A’
• The Venn diagram below illustrates the concept of a complement.
• P(A) + P(Ac) = 1
A Ac
Slide 20 of 51
20
10
2023 August 13
Complement of an Event
• For example, the rectangle stores all the possible tosses of 2 dice
{(1,1), (1,2),… (6,6)}
• Let A = tosses totalling 7
= {(1,6), (2, 5), (3,4), (4,3), (5,2), (6,1)}
• P(Total = 7) + P(Total not equal to 7) = 1
A Ac
Slide 21 of 51
21
Slide 22 of 51
22
11
2023 August 13
Slide 23 of 51
23
• The probability of A B,
P(A B) = 11/36
Slide 24 of 51
24
12
2023 August 13
For example,
A = tosses totalling 9 = {(3,6), (6,3), (4,5), (5,4)}
and B = tosses totalling 11 = {(5,6), (6,5)}
Therefore, A B = { } = Empty set. P(A B) = 0
Slide 25 of 51
25
Example 1
• The number of spots turning up when a six-sided die is tossed is
observed. Consider the following events:
A: The number observed is an even number.
B: The number observed is greater than 4.
C: The number observed is less than 4.
D: The number observed is 4.
a) Define a sample space for this random experiment, and assign
probabilities to the outcomes.
b) Find P(A), P(B), P(C) and P(D).
ҧ
c) Find P(𝐴).
d) Find P(A B).
e) Find P(A B).
f) Are events B and C mutually exclusive?
Slide 26 of 51
26
13
2023 August 13
Example 1 - Solution
a)The Sample Space is S= {1, 2, 3, 4, 5, 6}. Since each of the six possible
outcomes is equally likely to occur:
P(1)= P(2)= P(3)= P(4)= P(6)= 1/6
b) The events A, B, C and D are as follows: A = {2, 4, 6}, B= {5, 6},
C = {1, 2, 3}, D = {4}. Since the probability of an event E is equal to the
sum of the probabilities assigned to the outcomes contained in E,
P(A)= P(2) + P(4)= P(6)= 1/6 + 1/6 + 1/6 = 0.5
Similarly,
P(B)= P(5) + P(6)= 1/6 + 1/6 = 1/3 = 0.333
P(C)= P(1) + P(2) + P(3)= 1/6 + 1/6 + 1/6 = 1/2 = 0.5
P(D)= P(3)= 1/6 = 0.167
c) The three outcomes in A are represented in Figure by the points
lying outside the region describing event A.
The complement of event A is 𝐴ҧ ={1, 3, 5}. Therefore,
P(𝐴ҧ ) : P(1) + P(3) + P(5) = 3/6 = 0.5
Slide 27 of 51
27
Example 1 - Solution
d) The intersection of the two events A and B is (A B) = {6}.
Therefore,
P(A B) = P(6) = 1/6= 0.167
e) The union of the two events A and B is (A B) = {2, 4, 5, 6}.
Therefore,
P ( A B ) = P(2) + P(4) + P(5) + P(6) = 4/6 = 0.67
f) Two events B and C are mutually exclusive if the occurrence of one
precludes the occurrence of the other — that is, if the event
(B C) is an empty set (i.e. it has no possible outcomes).
The events B = {5, 6} and C = {1, 2, 3} defined in this example are
mutually exclusive because B and C have no outcomes in common.
The event (B C) = { } is an empty set.
Slide 28 of 51
28
14
2023 August 13
Example 2
• A & B Pty Ltd manufactures window air-conditioners in both a
deluxe model and a standard model. An auditor undertaking a
compliance audit of the firm is validating the sales account for
April. She has collected 200 invoices for the month, some of which
were sent to wholesalers and the remainder to retailers. Of the
140 retail invoices, 28 are for the standard model. Of the
wholesale invoices, 24 are for the standard model. If the auditor
selects one invoice at random, find the probability that:
a) the invoice selected is for the deluxe model .
b) the invoice selected is wholesale invoice for the deluxe model
c) the invoice selected is either a wholesale invoice or an invoice
for the standard model.
Slide 29 of 51
29
Example 2 - Solution
• The sample space S here consists of the 200 invoices. Whenever
the outcomes can be classified according to two relevant
characteristics - in this case, according to the model sold and type
of purchaser - it is worthwhile displaying the pertinent
information in a cross-classification table, below. The numbers
given in this example have been circled in the Table; you should
check to confirm that you can fill in the remaining numbers
yourself from the given information.
Model Wholesale (W) ഥ
Retail (𝑊) Total
Delux (D) 36 112 148
ഥ)
Standard (𝐷 24 28 52
Total 60 140 200
Slide 30 of 51
30
15
2023 August 13
Example 2 - Solution
The events of interest are as follows:
W : wholesale invoice is selected
𝑊ഥ : retail invoice is selected
D : invoice for deluxe model is selected
𝐷ഥ : invoice for standard model is selected
31
Example 2 - Solution
b) Since there are 36 wholesale invoices for the deluxe model, the
event (D W) contains 36 outcomes. Hence, the probability that
the invoice selected was a wholesale invoice for the deluxe
model is
36
𝑃 𝐷∩𝑊 = = 0.18
200
Slide 32 of 51
32
16
2023 August 13
Conditional probabilities ?
• Conditional probability is used to determine how two events are
related; that is, we can determine the probability of one event
given the occurrence of another related event.
• Again, the probability of an event given that another event has
occurred is called a conditional probability
• Conditional probabilities are written as P(A|B) and read as
‘the probability of event A given that event B has occurred’, and is
calculated as:
𝑃(𝐴 𝑎𝑛𝑑 𝐵) 𝑃(𝐴 ∩ 𝐵)
𝑃 𝐴𝐵 = =
𝑃(𝐵) 𝑃(𝐵)
for P(B) > 0 and undefined otherwise
Similarly,
𝑃(𝐴 ∩ 𝐵)
𝑃 𝐵𝐴 =
𝑃(𝐴)
Slide 33 of 51
33
34
17
2023 August 13
where
P(AB) = probability of events A and B occurring together or in
succession; This is known as a joint probability
P(A) = marginal probability of event A occurring
P(B) = marginal probability of event B occurring
Slide 35 of 51
35
𝑃(𝐷 ∩ 𝑊 ) 36
𝑃 𝐷𝑊 = = = 0.6
𝑃(𝑊) 24 + 36
Slide 36 of 51
36
18
2023 August 13
37
Rules of probability
• Complement rule
Sum of the probability assigned to each possible outcome in a
sample space must equal one. Give any event A and its
complement 𝐴,ҧ each outcome in the sample space must
belong to either A or 𝐴.ҧ Thus, For any event A,
P(𝐴)ҧ = 1 − P(A)
• Addition rule
Probability of the union of two events from the probability of
other events. For any two events A and B,
P(AB) = P(A) + P(B) - P(AB)
if the A and B are mutually exclusive, P(AB) =0, Then addition
rule simplifies to
P(AB) = P(A) + P(B)
Slide 38 of 51
38
19
2023 August 13
Rules of probability
• Multiplication rule
Used to find the probability of a joint event. Simply
arrangement of the definition of conditional probability.
• Multiplication rule for dependent events
P(AB) = P(B|A).P(A) or
P(AB) = P(A|B).P(B)
• Multiplication rule for independent events
P(AB) = P(A).P(B)
Slide 39 of 51
39
Probability trees
• Various possible events of an experiment are represented by
lines or branches of the tree.
Example 3
The manager of a men’s clothing store has recorded the buying
behaviour of customers over a long period of time. He has
established that the probability that a customer will buy a shirt is
about 0.4. A customer buys a tie 50% of the time when a shirt is
purchased, but only 10% of the time when a shirt is not purchased.
Find the probability that a customer will buy the following:
a) a shirt and a tie
b) a tie
c) a shirt or a tie
d) a tie but not a shirt
Slide 40 of 51
40
20
2023 August 13
Example 3 - Solution
The random experiment consists of observing, for each customer,
whether a shirt is purchased and whether a tie is purchased. The two
basic events of interest are therefore:
41
Example 3 - Solution
Stage 1 Stage 2 Possible outcomes joint Probabilities
T (RT) P(RT)=0.20
R
ത
(R𝑇) ത
P(R𝑇)=0.20
𝑇ത
T ത
(𝑅T) ത
P(𝑅T)=0.06
𝑅ത ത 𝑇)
(𝑅 ത ത 𝑇)=0.54
ത
P(𝑅
𝑇ത
Calculating the outcomes at the end of the tree, we find that a sample space
for the experiment is
S = { (R 𝑇),
ത (R ഥ𝑇 ), (𝑅ത T), (𝑅ത 𝑇)
ത }
Calculating the required probabilities is now quite simple, amounting to little
more than reading off the joint (outcome) probabilities from tree.
Slide 42 of 51
42
21
2023 August 13
Example 3 - Solution
a) The probability that a customer will buy a shirt and a tie is
P(R T) = 0.20
Slide 43 of 51
43
Bayes’ law
• Bayes’ law named after the English clergyman and mathematician
Thomas Bayes, provide us with an alternative formula for calculating
conditional probabilities.
• The alternative formula simplifiers the calculation of P(A|B) when
P(AB) and P(B) are not given directly.
𝑃(𝐴 ∩ 𝐵 ) 𝑃 𝐴 . 𝑃(𝐵|𝐴)
𝑃 𝐴𝐵 = =
𝑃(𝐵) 𝑃 𝐴 . 𝑃 𝐵 𝐴 + [ 𝑃 𝐴ҧ . 𝑃 𝐵 𝐴ҧ
Slide 44 of 51
44
22
2023 August 13
P(A) = P(B) =
Whole space
A B
P(A|B)= P(B|A)=
P(A B)=
P(A).P(B|A) = X = = P(A B)
P(B).P(A|B)= X = = P(A B)
Slide 45 of 51
45
X
=
= X X =
Slide 46 of 51
46
23
2023 August 13
Example 4
Due to turnover and absenteeism at an assembly plant, 20% of the
items are assembled by inexperienced employees. Management
has determined that customers return 12% of the items assembled
by inexperienced employees, whereas only 3% of the items
assembled by experienced employees are returned. Given that an
item has been returned, what is the probability that it was
assembled by an inexperience employee?
Slide 47 of 51
47
Example 4 - Solution
Slide 48 of 51
48
24
2023 August 13
Summary of Formulas
𝑃(𝐴 ∩ 𝐵)
𝑃 𝐴𝐵 =
𝑃(𝐵)
𝑃 𝐴ҧ = 1 − 𝑃 𝐴
𝑃 𝐴∪𝐵 =𝑃 𝐴 +𝑃 𝐵 −𝑃 𝐴∩𝐵
𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐴 .𝑃 𝐵 𝐴
= 𝑃 𝐵 .𝑃 𝐴 𝐵
Slide 49 of 51
49
1. There was a statistics professor who driving his car, would always
accelerate hard before coming to any Intersection, whip straight through
it, then slow down again once he'd got past it. One day, he took a
passenger, who was understandably unnerved by his driving style, and
asked him why he went so fast over intersections. The statistics professor
replied, "Well, statistically speaking, you are far more likely to have an
accident at an intersection, so I just make sure that I spend less time
there”.
2.
Slide 50 of 51
50
25
2023 August 13
Slide 51 of 51
51
26