Chapter 5

Statistical Models in Simulations

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.1

 Contents
•  Basic Probability Theory Concepts
•  Discrete Distributions
•  Continuous Distributions
•  Poisson Process
•  Empirical Distributions
•  Useful Statistical Models

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.2

 Purpose & Overview
•  The world the model-builder sees is probabilistic rather than
deterministic.
•  Some statistical model might well describe the variations.

•  An appropriate model can be developed by sampling the

phenomenon of interest:
•  Select a known distribution through educated guesses
•  Make estimate of the parameters
•  Test for goodness of fit

•  In this chapter:
•  Review several important probability distributions
•  Present some typical application of these models

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.3

 Basic Probability Theory Concepts

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.4

 Review of Terminology and Concepts
•  In this section, we will review the following concepts:
• Discrete random variables
• Continuous random variables
• Cumulative distribution function
• Expected value
• Variance
• Standard deviation
• Covariance
• Correlation
• Quantile

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.5

 Discrete Random Variables
•  X is a discrete random variable if the number of possible values
of X is finite, or countable infinite.
•  Example: Consider packets arriving at a router.
•  Let X be the number of packets arriving each second at a router.
RX = possible values of X (range space of X) = {0,1,2,…}
p(xi) = probability the random variable X is xi , p(xi) = P(X = xi)
•  p(xi), i = 1,2, … must satisfy:

1. p(xi ) ≥ 0, for all i

∞
2. ∑ i=1
p(xi ) = 1

•  The collection of pairs (xi, p(xi)), i = 1,2,…, is called the probability

distribution of X, and
•  p(xi) is called the probability mass function (PMF) of X.

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.6

 Continuous Random Variables
•  X is a continuous random variable if its range space RX is an interval or
a collection of intervals.
•  The probability that X lies in the interval [a, b] is given by:
b
P(a ≤ X ≤ b) = ∫ f ( x)dx
a

•  f(x) is called the probability density function (PDF) of X, and

satisfies:
P(X∈[a,b])
f(x)
1. f ( x) ≥ 0 , for all x in R X
2. ∫ f ( x)dx = 1
RX

3. f ( x) = 0, if x is not in RX
x
•  Properties a b
x0
1. P( X = x0 ) = 0, because ∫ f ( x)dx = 0
x0

2. P(a ≤ X ≤ b) = P(a < X ≤ b) = P(a ≤ X < b) = P(a < X < b)

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.7

 Continuous Random Variables
•  Example: Life of an inspection device is given by X, a
continuous random variable with PDF:

⎧ 1 − x / 2
⎪ e ,x ≥0
f ( x) = ⎨ 2
⎪⎩0 , otherwise

•  X has exponential distribution with mean 2 years

•  Probability that the device’s life is between 2 and 3 years is:
1 3 −x / 2
P(2 ≤ x ≤ 3) = ∫ e dx = 0.145
2 2
Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.8
 Cumulative Distribution Function
•  Cumulative Distribution Function (CDF) is denoted by F(x), where
F(x) = P(X ≤ x) F(x)

•  If X is discrete, then F ( x) = ∑ p( xi ) 1
xi ≤ x

x
•  If X is continuous, then F ( x) = ∫ f (t )dt
−∞
x
•  Properties
1. F is nondecreasing function. If a ≤ b, then F (a) ≤ F (b)
2. lim F ( x) = 1
x →∞

3. lim F ( x) = 0
x →−∞

•  All probability questions about X can be answered in terms of the CDF:

P(a ≤ X ≤ b) = F (b) − F (a), for all a ≤ b

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.9

 Cumulative Distribution Function
•  Example: The inspection device has CDF:
1 x −t / 2
F ( x) = ∫ e dt = 1 − e − x / 2
2 0
•  The probability that the device lasts for less than 2 years:

P(0 ≤ X ≤ 2) = F (2) − F (0) = F (2) = 1 − e−1 = 0.632

•  The probability that it lasts between 2 and 3 years:

P(2 ≤ X ≤ 3) = F (3) − F (2) = 1 − e ( − 32

)− (1− e ) = 0.145
−1

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.10

 Expected value
•  The expected value of X is denoted by E(X)

•  If X is discrete E ( X ) = ∑ xi p( xi )
all i

•  If X is continuous
∞
E ( X ) = ∫ x ⋅ f ( x)dx
−∞

•  a.k.a the mean, m, µ, or the 1st moment of X

•  A measure of the central tendency

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.11

 Variance
•  The variance of X is denoted by V(X) or Var(X) or σ 2
•  Definition: V(X) = E( (X – E[X])2 )

•  Also V(X) = E(X2) – ( E(X) )2

•  A measure of the spread or variation of the possible values of X around the

mean

f(x) f(x)
σ2 σ2
large small

x x
µ µ

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.12

 Standard deviation
•  The standard deviation (SD) of X is denoted by σ

•  Definition: σ = V (x)

•  The standard deviation is expressed in the same units as the mean

•  Interprete σ always together with the mean

•  Attention:
•  The standard deviation of two different data sets may be difficult to compare

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.13

 Expected value and variance: Example
•  Example: The mean of life of the previous inspection
device is:
∞
1 ∞ −x / 2 −x / 2
E ( X ) = ∫ xe dx = − xe
∞
+ ∫ e − x / 2 dx = 2
2 0 0
0

•  To compute the variance of X, we first compute E(X2):

∞
1 −x / 2
E ( X ) = ∫ x e dx = − x e
2∞ ∞
2 2 −x / 2
+ ∫ e − x / 2 dx = 8
2 0 0
0

•  Hence, the variance and standard deviation of the

device’s life are: V ( X ) = 8 − 22 = 4
σ = V (X ) = 2
Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.14
 Expected value and variance: Example
∞
1 ∞ −x / 2 −x / 2
E ( X ) = ∫ xe dx = − xe
∞
+ ∫ e − x / 2 dx = 2
2 0 0
0

Partial Integration

∫ u ( x)v' ( x)dx = u ( x)v( x) − ∫ u ' ( x)v( x)dx

Set
u ( x) = x
v' ( x) = e − x / 2
⇒
u ' ( x) = 1
v( x) = −2e − x / 2
∞
1 ∞ −x / 2 1 −x / 2 ∞
E ( X ) = ∫ xe dx = ( x ⋅ (−2e ) − ∫ 1⋅(−2e − x / 2 )dx )
2 0 2 0
0

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.15

 Mean and variance of sums
•  If x1, x2, …, xk are k random variables and if a1, a2, …, ak are k
constants, then

E(a1x1+a2x2+…+akxk) = a1E(x1)+a2E(x2)+…+akE(xk)

•  For independent variables

Var(a1 x1 + a2 x2 +  + ak xk ) = a12 Var( x1 ) + a22 Var( x2 ) + … + ak2 Var( xk )

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.16

 Coefficient of variation
•  The ratio of the standard deviation to the mean is called
coefficient of variation (C.O.V.)
• Dimensionless
• Normalized measure of dispersion

standard deviation σ
C.O.V = = ,µ > 0
mean µ

• Can be used to compare different datasets, instead the

standard deviation.

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.17

 Covariance
•  Given two random variables x and y with µx and µy, their
covariance is defined as

Cov(x, y) = σ2xy = E[(x-µx)(y-µy)] = E(xy) - E(x) E(y)

•  Cov(x, y) measures the dependency of x and y, i.e., how x and y

vary together.

•  For independent variables, the covariance is zero, since

E(xy) = E(x)E(y)

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.18

 Correlation coefficient
•  The normalized value of covariance is called the
correlation coefficient or simply correlation

σ xy2
Correlation (x, y) = ρ x , y =
σ xσ y

•  The correlation lies between -1 and +1

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.19

 Quantile
•  The x value at which the CDF takes a value α is called the
α-quantile or 100α-percentile. It is denoted by xα.

P(X ≤ xα) = F(xα) = α , α∈[0,1]

F(x)

1
α

x

xα

•  Relationship:
• The median is the 50-percentile or 0.5-quantile

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.20

 Mean, median, and mode
•  Three different indices for the central tendency of a
distribution:

n
• Mean:
∞
E ( X ) = µ = ∑ pi xi = ∫ x ⋅ f ( x)dx
−∞
i =1

• Median: The 0.5-quantile, i.e., the xi for that half of the values
are smaller and the other half is larger.

• Mode: The most likely value, i.e., the xi that has the highest
probabiliy pi or the x at which the PDF is maximum.

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.21

 Mean, median, and mode
Median Modes
Mean
Mode No Mode
pdf, f(x)

pdf, f(x)

pdf, f(x)
Median Median
Mean Mean

x x x

Mode Mode
pdf, f(x)

Median
pdf, f(x)

Median

Mean Mean

x x

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.22

 Selecting among
mean, median, and mode
Select central
tendency

Is data Yes
categorial?
Use mode

No

Is total of Yes
interest? Use mean

No

Is
Yes
distribution Use median
skewed?

No

Use mean

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.23

 Relationship between simulation and
probability theory

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.24

 Central limit theorem
•  Let Zn be the random variable
X ( n) − µ
Zn =
σ2
n

•  and Fn(z) be the distribution function of Zn for a sample

size of n, i.e., Fn(z)=P(Zn ≤ z), then

Fn ( z ) ⎯n⎯
⎯→ Θ( z )
→∞

•  where Θ(z) is normal distribution with µ=0 and σ2=1

z
1 − y2
Θ( z ) = ∫e 2
dy for − ∞ < z < ∞
2π −∞

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.25

 Strong law of large numbers
•  Let X1, X2, …, Xn be IID random variables with mean µ.

X (n) ⎯n⎯
⎯→ µ with probability 1
→∞

Sample mean

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.26

 Strong law of large numbers

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.27

 Discrete Distributions

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.28

 Discrete Distributions
•  Discrete random variables are used to describe random
phenomena in which only integer values can occur.

•  In this section, we will learn about:

• Bernoulli trials and Bernoulli distribution
• Binomial distribution
• Geometric and negative binomial distribution
• Poisson distribution

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.29

 Bernoulli Trials and Bernoulli Distribution
•  Bernoulli trials:
•  Consider an experiment consisting of n trials, each can be a success
or a failure.
⎧1 if the j - th experiment is a success
X j = ⎨
⎩0 if the j - th experiment is a failure

failure success

•  The Bernoulli distribution (one trial):

⎧ p, xj =1
p j ( x j ) = p( x j ) = ⎨ , j = 1,2,..., n
⎩q := 1 − p, x j = 0
•  where E(Xj) = p and V(Xj) = p (1-p) = p q

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.30

 Bernoulli Trials and Bernoulli Distribution
•  Bernoulli process:
•  n Bernoulli trials where trials are independent:

p(x1,x2,…, xn) = p1(x1) p2(x2) … pn(xn)

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.31

 Binomial Distribution
•  The number of successes in n Bernoulli trials, X, has a binomial
distribution.

1 n

⎧⎛ n ⎞ x n − x
⎪⎜⎜ ⎟⎟ p q , x = 0,1,2,..., n
p( x) = ⎨⎝ x ⎠
⎪0, otherwise
⎩
The number of
Probability that
outcomes having the
there are
required number of
x successes and
successes and
(n-x) failures
failures

•  The mean, E(x) = p + p + … + p = n×p

•  The variance, V(X) = pq + pq + … + pq = n×pq

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.32

 Geometric Distribution
•  Geometric distribution
• The number of Bernoulli trials, X, to achieve the 1st success:

success

⎧ q x −1 p, x = 0,1,2,..., n
p( x) = ⎨
⎩0, otherwise

•  E(x) = 1/p, and V(X) = q/p2

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.33

 Negative Binomial Distribution
•  Negative binomial distribution
• The number of Bernoulli trials, X, until the k-th success

k-th success
• If X is a negative binomial distribution with parameters p and
k, then:
⎧⎛ x − 1⎞ x − k k
⎪⎜⎜ ⎟⎟ q p , x = k , k + 1, k + 2,...
p( x) = ⎨⎝ k − 1⎠
⎪0, otherwise
⎩
⎛ x − 1⎞ x − k k −1
p ( x) = ⎜⎜ ⎟⎟ q p ⋅ p
⎝ k − 1⎠
 k − th success
(k-1 ) successes

•  E(X) = k/p, and V(X) = kq/p2

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.34

 Poisson Distribution
•  Poisson distribution describes many random processes quite
well and is mathematically quite simple.
•  where α > 0, PDF and CDF are:
⎧α x −α x
αi
p( x) = ⎨ x! e , x = 0,1,...
⎪
F ( x) = ∑ e −α
⎪⎩0, otherwise i =0 i!
•  E(X) = α = V(X)

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.35

 Poisson Distribution
•  Example: A computer repair person is “beeped” each time
there is a call for service. The number of beeps per hour
~ Poisson(α = 2 per hour).

•  The probability of three beeps in the next hour:

p(3) = 23/3! e-2 = 0.18
also, p(3) = F(3) – F(2) = 0.857-0.677=0.18

•  The probability of two or more beeps in an 1-hour period:

p(2 or more) = 1 – ( p(0) + p(1) )
= 1 – F(1)
= 0.594

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.36

 Continuous Distributions

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.37

 Continuous Distributions
•  Continuous random variables can be used to describe
random phenomena in which the variable can take on any
value in some interval.

•  In this section, the distributions studied are:

• Uniform
• Exponential
• Weibull
• Normal
• Lognormal

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.38

 Uniform Distribution
•  A random variable X is uniformly distributed on the interval
(a, b), U(a, b), if its PDF and CDF are:

⎧ 1 ⎧0, x<a
⎪ , a≤ x≤b ⎪ x − a
f ( x) = ⎨ b − a F ( x) = ⎨ , a≤ x<b
⎪⎩0, otherwise ⎪ b − a
⎩1, x≥b
•  Properties
•  P(x1 < X < x2) is proportional to the length of the interval
[F(x2) – F(x1) = (x2-x1)/(b-a)]
•  E(X) = (a+b)/2 V(X) = (b-a)2/12

•  U(0,1) provides the means to

generate random numbers, from
which random variates can be
generated.

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.39

 Exponential Distribution
•  A random variable X is exponentially distributed with
parameter λ > 0 if its PDF and CDF are:
⎧λe − λx , x ≥ 0 ⎧⎪0, x<0
f ( x) = ⎨ F ( x) = ⎨ x
− λt − λx
⎩0, elsewhere ⎪⎩∫0 λe dt = 1 − e , x ≥ 0

•  E(X) = 1/λ V(X) = 1/λ2

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.40

 Exponential Distribution
• Used to model interarrival
times when arrivals are
completely random, and to
model service times that
are highly variable

• For several different

exponential PDF’s (see
figure), the value of
intercept on the vertical
axis is λ, and all PDF’s
eventually intersect.

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.41

 Exponential Distribution
•  Memoryless property
• For all s and t greater or equal to 0:

P(X > s+t | X > s) = P(X > t)

• Example: A lamp ~exp(λ = 1/3 per hour), hence, on average, 1

failure per 3 hours.

•  The probability that the lamp lasts longer than its mean life is:
P(X > 3) = 1 – P(X < 3) = 1 – (1 – e -3/3) = e -1 = 0.368

•  The probability that the lamp lasts between 2 to 3 hours is:

P(2 ≤ X ≤ 3) = F(3) – F(2) = 0.145

•  The probability that it lasts for another hour given it is operating

for 2.5 hours:
P(X > 3.5 | X > 2.5) = P(X > 1) = e -1/3 = 0.717

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.42

 Exponential Distribution
•  Memoryless property

P( X > s + t )
P( X > s + t | X > s) =
P( X > s)
−λ ( s +t )
e
= − λs
e
− λt
=e
= P( X > t )

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.43

 Weibull Distribution
•  A random variable X has a Weibull distribution if its PDF has the form:
⎧ β ⎛ x −ν ⎞ β −1 ⎡ ⎛ x −ν ⎞ β ⎤
⎪ exp⎢− ⎜ ⎟ ⎥, x ≥ ν
f ( x) = ⎨α ⎜⎝ α ⎟⎠
⎣⎢ ⎝ α ⎠ ⎦⎥
⎪0, otherwise
⎩
•  3 parameters:
•  Location parameter: υ, (−∞ < ν < ∞)
•  Scale parameter: β , (β > 0)
•  Shape parameter: α, (> 0)

•  Example: υ = 0 and α = 1:

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.44

 Weibull Distribution
•  Weibull Distribution
⎧ β ⎛ x −ν ⎞ β −1 ⎡ ⎛ x −ν ⎞ β ⎤
⎪ exp⎢− ⎜ ⎟ ⎥, x ≥ ν
f ( x) = ⎨α ⎜⎝ α ⎟⎠ ⎢⎣ ⎝ α ⎠ ⎥⎦
⎪0, otherwise
⎩

•  For β = 1, υ=0
⎧ 1 − αx
f ( x) = ⎨α e , x ≥ ν
⎪
⎪⎩0, otherwise

When β = 1,
X ~ exp(λ = 1/α)

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.45

 Normal Distribution
•  A random variable X is normally distributed if it has the PDF:
2
1 ⎛ x − µ ⎞
1 − ⎜
2 ⎝ σ ⎠
⎟
f ( x) = e , −∞ < x < ∞
σ 2π
•  Mean: − ∞ < µ < ∞
•  Variance: σ 2 > 0
•  Denoted as X ~ N(µ,σ2)

•  Properties:
•  lim f ( x) = 0, and lim f ( x) = 0
x→−∞ x→∞
•  f(µ - x) = f(µ + x); the PDF is symmetric about µ.
•  The maximum value of the PDF occurs at x = µ
Æ the mean and mode are equal

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.46

 Normal Distribution
•  Evaluating the distribution:
•  Use numerical methods (no closed form)
•  Independent of µ and σ, using the standard normal distribution:
Z ~ N(0,1)

X −µ
•  Transformation of variables: let Z = ,
σ

⎛ x − µ ⎞
F ( x) = P ( X ≤ x ) = P⎜ Z ≤ ⎟
⎝ σ ⎠
( x−µ ) /σ 1 −2z2
=∫ e dz
−∞
2π
( x−µ ) /σ
=∫ φ ( z )dz = Φ ( xσ− µ ) , where Φ( z ) = ∫
z 1 −t 2 / 2
e dt
−∞ −∞
2π

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.47

 Normal Distribution
•  Example: The time required to load an oceangoing vessel, X, is
distributed as N(12,4), µ=12, σ =2
•  The probability that the vessel is loaded in less than 10 hours:
⎛ 10 − 12 ⎞
F (10) = Φ⎜ ⎟ = Φ(−1) = 1 − Φ(1) = 0.1587
⎝ 2 ⎠

• Using the symmetry property, Φ(1) is the complement of

Φ(-1), i.e., Φ(-x) = 1-Φ(x)

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.48

 Normal Distribution
•  Why is the normal distribution important?
• The most commonly used distribution in data analysis
• The sum of n independent normal variates is a normal variate.
• The sum of a large number of independent observations from
any distribution has a normal distribution.

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.49

 Lognormal Distribution
•  A random variable X has a lognormal distribution if its pdf has
the form:
⎧ 1 ⎡ (ln x − µ ) 2 ⎤
⎪ exp⎢− ⎥, x > 0 µ=1,
f ( x) = ⎨ 2π σx 2σ 2
⎣ ⎦ σ2=0.5,1,2.
⎪0, otherwise
⎩

2
•  Mean E(X) = e µ+σ /2
2 2
•  Variance V(X) = e 2µ+σ /2 (eσ - 1)

•  Relationship with normal distribution

•  When Y ~ N(µ, σ2), then X = eY ~ lognormal(µ, σ2)
•  Parameters µ and σ2 are not the mean and variance of the lognormal
random variable X

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.50

 Poisson Process

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.51

 Poisson Process
•  Definition: N(t) is a counting function that represents the
number of events occurred in [0,t].

•  A counting process {N(t), t ≥ 0} is a Poisson process with mean

rate λ if:
•  Arrivals occur one at a time
•  {N(t), t ≥ 0} has stationary increments
•  Number of arrivals in [t, t+s] depends only on s, not on starting point t
•  Arrivals are completely random

•  {N(t), t ≥ 0} has independent increments

•  Number of arrivals during non-overlapping time intervals are independent
•  Future arrivals occur completely random

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.52

 Poisson Process
•  Properties

(λt ) n −λt
P(N (t ) = n ) = e , for t ≥ 0 and n = 0,1,2,...
n!

•  Equal mean and variance: E[N(t)] = V[N(t)] = λ t

•  Stationary increment:
•  The number of arrivals in time s to t, with s<t, is also Poisson-
distributed with mean λ(t-s)

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.53

 Poisson Process: Interarrival Times
•  Consider the interarrival times of a Poisson process (A1, A2, …), where Ai
is the elapsed time between arrival i and arrival i+1

•  The 1st arrival occurs after time t iff there are no arrivals in the interval [0, t],
hence:
P(A1 > t) = P(N(t) = 0) = e-λt
P(A1 ≤ t) = 1- P(A1 > t) = 1 – e-λt [CDF of exp(λ)]

•  Interarrival times, A1, A2, …, are exponentially distributed and independent

with mean 1/λ

Arrival counts Interarrival time

~ Poisson(λ) ~ exp(1/λ)

Stationary & Independent Memoryless

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.54

 Poisson Process: Splitting and Pooling
•  Splitting:
•  Suppose each event of a Poisson process can be classified as
Type I, with probability p and Type II, with probability 1-p.

•  N(t) = N1(t) + N2(t), where N1(t) and N2(t) are both Poisson processes
with rates λ p and λ (1-p)

λp N1(t) ~ Poisson(λp)
λ

Ν (t) ~ Poisson(λ)
N2(t) ~ Poisson(λ(1-p) )
λ(1-p)

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.55

 Poisson Process: Splitting and Pooling
•  Pooling:
n
P ( N1 + N 2 = n ) = ∑ P ( N1 = j ) P ( N 2 = n − j )
•  Suppose two Poisson j =0
processes are pooled together n
(λ1t ) j −λ1t (λ2t ) n − j −λ2t
=∑ e e
•  N1(t) + N2(t) = N(t), where N(t) is j =0 j! (n − j )!
a Poisson processes with rates
λ1 + λ2 − λ1t − λ2t
n
(λ1t ) j (λ2t ) n − j
=e e ∑
j =0 j! (n − j )!
− ( λ1 + λ2 ) t n
n
λ1j λn2− j
=e t ∑ j =0 j! (n − j )!
N1(t) ~ Poisson(λ1) N2(t) ~ Poisson(λ2)
− ( λ1 + λ2 ) t t n n λ1j λn2− j
=e ∑ n!
λ1 λ2 n! j =0 j! (n − j )!
− ( λ1 + λ2 ) t t n n ⎛ n ⎞ j n − j
λ1 + λ2
=e ∑ ⎜⎜ ⎟⎟λ1 λ2
n! j =0 ⎝ j ⎠
N(t) ~ Poisson(λ1 + λ2)
− ( λ1 + λ2 ) t tn
=e (λ1 + λ2 ) n
n!
Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.56
 Empirical Distributions

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.57

 Empirical Distributions
•  A distribution whose parameters are the observed values in a
sample of data.
•  May be used when it is impossible or unnecessary to establish that a
random variable has any particular parametric distribution.
•  Advantage: no assumption beyond the observed values in the
sample.
•  Disadvantage: sample might not cover the entire range of possible
values.

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.58

 Empirical Distributions: Example
•  Customers arrive in groups from 1 to 8 persons
•  Observation of the last 300 groups has been reported
•  Summary in the table below

Group Frequency Relative Cumulative Relative

Size Frequency Frequency
1 30 0.10 0.10
2 110 0.37 0.47
3 45 0.15 0.62
4 71 0.24 0.86
5 12 0.04 0.90
6 13 0.04 0.94
7 7 0.02 0.96
8 12 0.04 1.00

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.59

 Empirical Distributions: Example

Relative  Frequency Empirical  CDF  of  the  Group   Sizes

0,4 1,2
0,35 1
0,3
0,25 0,8
0,2 0,6
0,15 0,4
0,1
0,05 0,2
0 0
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.60

 Relationships among distributions

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.61

 Relationships among distributions
•  Distributions
• 10 discrete
• 25 continous
•  Relationships
• Dashed arrow shows
asymptotic relations
• Solid arrow shows
transformations or special
cases

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.62

 Relationships among distributions

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.63

 Useful Statistical Models

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.64

 Useful Statistical Models
•  In this section, statistical models appropriate to some
application areas are presented.

•  The areas include:

• Queueing systems
• Inventory and supply-chain systems
• Reliability and maintainability
• Limited data

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.65

 Useful models: Queueing Systems
•  In a queueing system, interarrival and service-time patterns
can be probabilistic.
•  Sample statistical models for interarrival or service time
distribution:
•  Exponential distribution: if service times are completely random
•  Normal distribution: fairly constant but with some random variability
(either positive or negative)
•  Truncated normal distribution: similar to normal distribution but with
restricted values.
•  Gamma and Weibull distributions: more general than exponential
(involving location of the modes of PDF’s and the shapes of tails.)

Calling population

Waiting line Server

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.66

 Useful models: Inventory and supply chain
●  In realistic inventory and
supply-chain systems, there
are at least three random
variables:
•  The number of units demanded
per order or per time period
•  The time between demands
•  The lead time = Time between
placing an order and the
receipt of that order

•  Sample statistical models for lead time distribution:

•  Gamma
•  Sample statistical models for demand distribution:
•  Poisson: simple and extensively tabulated.
•  Negative binomial distribution: longer tail than Poisson (more
large demands).
•  Geometric: special case of negative binomial given at least one
demand has occurred.

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.67

 Useful models: Reliability and maintainability
•  Time to failure (TTF)
• Exponential: failures are random
• Gamma: for standby redundancy where each component has
an exponential TTF
• Weibull: failure is due to the most serious of a large number
of defects in a system of components
• Normal: failures are due to wear

runtime downtime runtime

Time
Time 0 eor eod eor

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.68

 Useful models: Other areas
•  For cases with limited data, some useful distributions are:
• Uniform
f(x)
• Triangular
• Beta

•  Other distribution:
x
• Bernoulli
• Binomial
• Hyperexponential

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.69

 Summary
•  The world that the simulation analyst sees is probabilistic,
not deterministic.
•  In this chapter:
• Reviewed several important probability distributions.
• Showed applications of the probability distributions in a
simulation context.
•  Important task in simulation modeling is the collection
and analysis of input data, e.g., hypothesize a
distributional form for the input data.
•  Student should know:
• Difference between discrete, continuous, and empirical
distributions.
• Poisson process and its properties.

Prof. Dr. Mesut Güneş ▪ Ch. 5 Statistical Models in Simulations 5.70