Lecture note Statistics and Probability for Engineering Stat 2010

CHAPTER 3
3. Basic Concepts in Probability
3.1. Introduction
- Probability theory is used to evaluate uncertainties to make decision or computing the chance
of occurrence of an event.
- Probability theory is the foundation upon which the logic of inference is built.
- In general, probability is the chance of an outcome to occur in an experiment or
- It is the measure of how likely an outcome is to occur.
3.2. Definitions of Some Basic Terms in Probability
 Random experiment:
- An experiment is the process from which an observation (measurement) is obtained.
- Results of an experiment may not be the same even through conditions which are identical.
- Such experiments are called random experiments.
 Outcome:
- The result of a single trial of a random experiment.
 Sample space:
- It is a set S which consists of all possible outcomes of a random experiment.
- Each outcome is called sample point.
 Events:
- Event is any subset of a sample space S.
 Mutually exclusive events:
- Mutually exclusive events are events which cannot happen at the same time.
- The intersection of two mutually exclusive sets is empty set.
 Complement of an Event:
Complement of an event A means non- occurrence of A and is denoted by A, or A , or A
c
-
- Contains those points of the sample space which doesn’t belong to A.
 Independent events
- Events in which the occurrence of one event doesn`t influence the occurrence of the other.
 Equally likely outcomes
- Equally likely outcomes are outcomes of an experiment which has equal chance (equally
probable) to appear.
- In most case it is commonly assumed finite or countable infinite sample space is equally likely.
- If we have n equally likely outcomes in the sample space then the probability of the ith sample
1
point xi is p (xi) = 𝑛, where xi can be the first, second, . . . or the nth outcome.
Example 3.1: a. tossing a six sided fair die.
b. tossing a coin.
c. Gender of a new born child
Could the above Statements refer to random experiment? What are the possible Outcomes? List
the Sample space? Define events? Define mutually exclusive events? Define complement of
events? Define independent events? Are the outcomes equally likely outcomes?

DEC April, 2024

1
Lecture note Statistics and Probability for Engineering Stat 2010
3.3. Counting Rules
- In many cases the numbers of sample points (elements) in a sample space are not very large,
hence direct enumeration or counting of sample points to obtain probabilities is not difficult.
- However, problems arise when direct counting become practically impossible.
- To avoid such difficulties we apply fundamental principles of counting (counting techniques).
3.3.1. Addition Rule
- If there are n1, n2, . . . , and nk options to accomplish a certain task.
- Then we would have n1 + n2 + . . . + nk options to accomplish the complete task.
Example 3.2: An engineering student undertaking a capstone project T has a choice of 4
professors A, B,C, D who can give him a list of projects from which he can choose one. A has 5
projects, B has 7 projects, C has 6 projects, and D has 2 projects. Find the number of ways can
the student be choose exactly one project.

3.3.2. Multiplication Rule (Fundamental Principle of counting or mn-Rule)

- Suppose a task is completed in K stages by carrying out a number of subtasks in each one of
these K stages.
- If in the first stage the task can be accomplished in n1 different ways and after this in the
second stage the task can be accomplished in n2 different ways, . . . , and finally in the Kth stage
the task can be accomplished in nk different ways.
- Then, the overall task can be done in n1 × n2 ×・ ・ ・× nk different ways.
Example 3.3: A professor assigns a capstone project T to senior engineering students consisting
of four parts A,B,C,D that must be done in sequence independent of the previous ones. Part A
can be done in 2 ways, Part B in 3 ways, Part C in 5 ways, and Part D in 4 ways. Find the
number of ways can the student be arranged those four parts.

3.3.3. Permutation
- It deals on arrangement of n objects in a specific order.
- In this case order is crucial.
- The following are three different cases of arrangements:
Case1: Having n distinct objects, then the number of permutations (arrangements) of these n
objects taken together will be n! (Where, n! = n*(n-1)*(n-2)* . . . * 3 * 2 * 1)
Case 2: - The arrangement of n objects in a specified order using r objects at a time is called the
permutation of n objects taken r objects at a time.
- It is written as nPr and the number of ways of arrangement of these r objects from n
objects is given by the formula:
n!
n pr 
(n  r )!
Case 3: - Having n objects, of which k1, k2 . . . kr are same.
- The number of permutations (arrangement) of these n objects is given by:
n!
n p k1k 2 ...k r 
k1!*k 2 !*... * k r !

DEC April, 2024

2
Lecture note Statistics and Probability for Engineering Stat 2010
Example 3.4. An engineer wants to visit three different sites in Dukem, Debrezeit and Adama.
How many different possible sequences or orders of visiting theses three sites are possible?
Example 3.5: In the Olympic 100-m dash, there are 9 runners. In how many ways can the gold,
silver, and bronze can be given to these runners?
Example 3.6: How many distinct permutations can be formed from all letters in the word “unusually”?
3.3.4. Combinations
- It is one of the counting techniques in which order of the objects is not important.
- The total number of combinations of r objects selected from n (also called the combinations of
𝑛 n
C
n things taken r at a time) is denoted by n r or ( ) or C r defined as
𝑟
𝑛 𝑛! 𝑛 (𝑛−1)(𝑛−2). . .( 𝑛−𝑟+1) n Pr
( )= = =
𝑟 𝑟! (𝑛−𝑟)! 𝑟! r!
Example 3.7: A printed circuit board has eight different locations in which a component can be placed.
If five identical components are to be placed on the board, how many different designs are possible?

3.4. Definitions /Assignments of probability

- In any random experiment there is always uncertainty whether a particular event will occur or not.
- As a measure of chance, or probability, with which we can expect the event to occur, it is
convenient to assign a number between 0 and 1.
- If we are sure or certain that the event will occur, we say that its probability is 1 or 100%,
- If we are sure that the event will not occur, we say that its probability is zero.
- There are different procedures by means of which we define or estimate probability of events.
- Some of the procedures are the following:
 CLASSICAL APPROACH:

Let S be a sample space, associated with a certain random experiment and consisting of finitely
many sample points m, say, each of which is equally likely to occur whenever the random
experiment is carried out. Then the probability of any event A, consisting of n sample points
(0 ≤ n ≤ m), is given by:
𝑛
P(A) = 𝑚
 RELATIVE FREQUENCY APPROACH:
Let N(A) be the number of times an event A occurs in N repetitions of a random experiment, and
𝑁(𝐴)
assume that the relative frequency of A, , converges to a limit as N →∞. This limit is
𝑁

denoted by P(A) and is called the probability of A.

DEC April, 2024

3
Lecture note Statistics and Probability for Engineering Stat 2010
Example 3.8: In an experiment of tossing a coin and a die together, find the probability of an
event A consisting head and even numbers?

Example 3.9: The national center for health statistics reported that of every 559 deaths in recent
years, 24 resulted that from automobile accident, 182 from cancer, and 353 from other disease.
What is the probability that particular death is due to an automobile accident?

 AXIOMATIC APPROACH:

Probability is a function, denoted by P, defined for each event of a sample space S, taking events
into the real line  , and satisfying the following three properties (or axioms of probability):

Axiom 1: P(A) ≥ 0 for every event A (probability of an event is nonnegative)

Axiom 2: For the sure or certain event S, P(S) = 1

Axiom 3: For countably infinite many pair wise disjoint (mutually exclusive) events Ai,

i =1, 2, 3, . . . , n, . . . Ai  Aj =  , i ≠ j, it holds

𝑷 ( 𝑨𝟏  𝑨𝟐  𝑨𝟑 . . . ) = 𝑷 ( 𝑨𝟏 ) + 𝑷(𝑨𝟐 ) + 𝑷(𝑨𝟑 ) . . . or

 
= P ( Ai ) =
i 1
P(A)
i 1
i

3.5. Some Important Theorems or Rules on Probability

Theorem 1: If A = A1  A2  …  An, where A1, A2 , . . . , An are mutually exclusive events,

then, P(A) = P(A1) + P(A2) + . . . + P(An)
In particular, if A = S, the sample space, then
P(A1) + P(A2) + … + P(An) = 1
Theorem 2: If A` is the complement of A, then P(A` ) = 1 - P(A)
Theorem 3: If A and B are any two events, then
P(A  B) = P(A) + P(B) - P(A  B)
More generally, if A1, A2, A3 are any three events, then
P(A1  A2  A3) = P(A1) + P(A2) + P(A3) - P(A1  A2) - P(A2  A3) - P(A3  A1)
+ P(A1  A2  A3)
Theorem 4: For any events A and B, P(A) = P(A  B) + P(A  B`)

DEC April, 2024

4
Lecture note Statistics and Probability for Engineering Stat 2010
3.6. Conditional Probability
 The conditional probability of an event A, given that event B has occurred with P(B) > 0, is
denoted by P(A|B) and is defined by:
𝑃 (𝐴 ∩ 𝐵 )
𝑃(𝐴|𝐵) = .
𝑃(𝐵)
 P (A ∩ B )= P(A|B)P(B)= P(B)P(A|B)
 P (A ∩ B )= P(B|A)P(A)= P(A)P(B|A) where P(A) > 0,
Example 3.10: A fair coin tossed three successive times. Find the conditional probability P(A|B)
where A and B are the events A = {more heads than tails come up}, B = {1st toss is a head}.
Example 3.11: Ms. Perez figures that there is a 30 percent chance that her company will set up a
branch office in Phoenix. If it does, she is 60 percent certain that she will be made manager of
this new operation. What is the probability that Perez will be a Phoenix branch office manager?
3.7. Total Probability Theorem or the Rule of Elimination
Theorem 6: Let B1, B2, . . ., Bn be a partition of the sample space S, if
(i) Bi ∩ Bj = ∅ for i ≠ j where i, j =1, 2, 3, . . .,n
n
(ii) B = S i
i 1

(iii) P(Bi) > 0 for all i.

Then, for any event A in S,
P(A) = P(A ∩ B1) + P(A ∩B2) + P(A ∩B2) + . . . + P(A ∩ Bn )
= P(B1).P(A\B1) + P(B2).P(A\B2) + . . . + P(Bn).P(A\Bn)

Example 3.12:
The member of a consulting firm rent cars from three rental agencies: 60% from agency 1, 30%
from agency 2, and 10% from agency 3. If 9% of cars from agency 1 needs tune-up, 20% of cars
from agency 2 needs tune-up, and 6% of cars from agency 3 needs tune-up, what is the
probability that a rental car delivered to the firm will need tune-up?
3.8. Baye`s Theorem or Rule

- Suppose that B1, B2, . . .,Bn are mutually exclusive events whose union is the sample space S.
- Then if A is any event, we have the following important theorem:

P ( Bi ) P ( A \ B)
P ( Bi \ A)  n
i

 P( B ) P ( A
j 1
j
\B ) j

Example 3.13: In reference to example 3.12 above, if rental car delivered to the consulting firm
needs a tune-up, what is the probability that it came from rental agency 2? It came from rental
agency 1? It came from rental agency 3?

DEC April, 2024

5
Lecture note Statistics and Probability for Engineering Stat 2010
3.9. Independent Event

 Two events A1 and A2 are said to be independent (statistically or stochastically or in the

probability sense), if P(A1 ∩ A2) = P(A1) P(A2)
 Two events A1 and A2 are said to be dependent when

P(A1 ∩ A2) ≠ P(A1) P(A2)

 Two events A1 and A2 are independent means the occurrence of one event A1 is not affected
by the occurrence or non-occurrence of A2 and vice versa.

Remark: If two events A and B are independent then

P(B\A) = P(B), for P(A) > 0 and P(A|B) = P(A) where P(B) > 0.

 The definition of independent event can be extended in to more than two event as follow:

The events A1, A2, . . . ,An are said to be independent (statistically or stochastically or in the
probability sense) if, for all possible choices of k out of n events (2 ≤ k ≤ n), the probability of
their intersection equals the product of their probabilities. More formally, for any k, 2 ≤ k ≤ n
and any integer j1, j2 . . . , jk with 1 ≤ j1 < . . . < jk ≤ n, we have:
k k
P ( A j i )   P ( A j i )
i 1 i 1

Example 3.14:

A fair die is tossed twice. Find the probability of getting a 4, 5, or 6 on the first toss and a 1, 2, 3,
or 4 on the second toss?

Example 3.15:

A system composed of n separate components is said to be a parallel system, if it functions when

at least one of the components functions. For such a system, if component i, independent of other
components, functions with probability Pi , i = 1, . . . , n, what is the probability that the system
functions?

DEC April, 2024

6