Chapter 5

Statistical Models in
Simulation

Banks, Carson, Nelson & Nicol

Discrete-Event System Simulation
Purpose & Overview
 The world the model-builder sees is probabilistic rather
than deterministic.
 Some statistical model might well describe the variations.

 An appropriate model can be developed by sampling the

phenomenon of interest:
 Select a known distribution through educated guesses
 Make estimate of the parameter(s)
 Test for goodness of fit

 In this chapter:
 Review several important probability distributions
 Present some typical application of these models

2
Review of Terminology and Concepts
 In this section, we will review the following
concepts:
 Discreterandom variables
 Continuous random variables
 Cumulative distribution function
 Expectation

3
Discrete Random Variables [Probability Review]

 X is a discrete random variable if the number of possible

values of X is finite, or countably infinite.
 Example: Consider jobs arriving at a job shop.
 Let X be the number of jobs arriving each week at a job shop.
 Rx = possible values of X (range space of X) = {0,1,2,…}
 p(xi) = probability the random variable is xi = P(X = xi)
 p(xi), i = 1,2, … must satisfy:
1. p( xi )  0, for all i



2. i 1
p( xi )  1

 The collection of pairs [xi, p(xi)], i = 1,2,…, is called the probability

distribution of X, and p(xi) is called the probability mass function
(pmf) of X.
4
Continuous Random Variables [Probability Review]

 X is a continuous random variable if its range space Rx is an interval

or a collection of intervals.
 The probability that X lies in the interval [a,b] is given by:
b
P(a  X  b)   f ( x)dx
a

 f(x), denoted as the pdf of X, satisfies:

1. f ( x)  0 , for all x in R X
2.  f ( x)dx  1
RX

3. f ( x)  0, if x is not in RX

 Properties x
1. P( X  x0 )  0, because  f ( x)dx  0
0

x0

2. P(a  X  b)  P(a  X  b)  P(a  X b)  P(a  X b)

5
Continuous Random Variables [Probability Review]

 Example: Life of an inspection device is given by X, a

continuous random variable with pdf:

 1  x / 2
e , x0
f ( x)   2
0, otherwise

 1 3  x / 2with mean 2 years

X has an exponential distribution
2
 P(2that
Probability  xthe
 3device’s
) e is between
life dx  0.142 and 3 years is:
2

6
Cumulative Distribution Function [Probability Review]

 Cumulative Distribution Function (cdf) is denoted by F(x), where F(x)

= P(X <= x)
 If X is discrete, then F ( x)   p( xi )
all
xi  x
 If X is continuous, then x
F ( x)   f (t )dt

 Properties
1. F is nondecreasing function. If a b, then F (a)  F (b)
2. limx F ( x)  1
3. limx  F ( x)  0

 All probability questions about X can be answered in terms of the

cdf, e.g.:
P(a  X  b)  F (b)  F (a), for all a b

7
Cumulative Distribution Function [Probability Review]

 Example: An inspection device has cdf:

1 x t / 2
F ( x)   e dt  1  e  x / 2
2 0

 The probability that the device lasts for less than 2 years:
P(0  X  2)  F (2)  F (0)  F (2)  1  e 1  0.632

 The probability that it lasts between 2 and 3 years:

P(2  X  3)  F (3)  F (2)  (1  e  ( 3 / 2) )  (1  e 1 )  0.145

8
Expectation [Probability Review]

 The expected value of X is denoted by E(X)

 If X is discrete E ( x)   xi p( xi )
all i

 If X is continuous E ( x)   xf ( x)dx


 a.k.a the mean, m, or the 1st moment of X

 A measure of the central tendency
 The variance of X is denoted by V(X) or var(X) or s2
 Definition: V(X) = E[(X – E[X])2]
 Also, V(X) = E(X2) – [E(x)]2
 A measure of the spread or variation of the possible values of X around
the mean
 The standard deviation of X is denoted by s
 Definition: square root of V(X)
 Expressed in the same units as the mean

9
Expectations [Probability Review]

 Example: The mean of life of the previous inspection device

is: 
1  x/ 2 x / 2 
E ( X )   xe dx   xe   e  x / 2 dx  2
2 0 0
0

 To compute variance of X, we first compute E(X2):


1  x / 2 
E ( X )   x e dx   x e
2 2 x / 2 2  2 xe x / 2 dx  8
2 0 0
0

 Hence, the variance and standard deviation of the device’s life

are:
V ( X )  8  22  4
  V (X )  2
10
Useful Statistical Models
 In this section, statistical models appropriate to
some application areas are presented. The
areas include:
 Queueing systems
 Inventory and supply-chain systems
 Reliability and maintainability
 Limited data

11
Queueing Systems [Useful Models]

 In a queueing system, interarrival and service-time

patterns can be probablistic
 For more queueing examples, see Chapter 2; Chapter 6 is all queueing systems

 Sample statistical models for interarrival or service time

distribution:
 Exponential distribution: if service times are completely random
 Normal distribution: fairly constant but with some random
variability (either positive or negative)
 Truncated normal distribution: similar to normal distribution but
with restricted value.
 Gamma and Weibull distribution: more general than exponential
(involving location of the modes of pdf’s and the shapes of tails.)

12
Inventory and supply chain [Useful Models]

 In realistic inventory and supply-chain systems, there are

at least three random variables:
 The number of units demanded per order or per time period
 The time between demands
 The lead time

 Sample statistical models for lead time distribution:

 Gamma

 Sample statistical models for demand distribution:

 Poisson: simple and extensively tabulated.
 Negative binomial distribution: longer tail than Poisson (more
large demands).
 Geometric: special case of negative binomial given at least one
demand has occurred.

13
Reliability and maintainability [Useful Models]

 Time to failure (TTF)

 Exponential: failures are random
 Gamma: for standby redundancy where each
component has an exponential TTF
 Weibull: failure is due to the most serious of a large
number of defects in a system of components
 Normal: failures are due to wear

14
Other areas [Useful Models]

 For cases with limited data, some useful

distributions are:
 Uniform, triangular and beta
 Other distribution: Bernoulli, binomial and
hyperexponential.

15
Discrete Distributions
 Discrete random variables are used to describe
random phenomena in which only integer values
can occur.
 In this section, we will learn about:
 Bernoullitrials and Bernoulli distribution
 Binomial distribution
 Geometric and negative binomial distribution
 Poisson distribution

16
Bernoulli Trials
and Bernoulli Distribution [Discrete Dist’n]
 Bernoulli Trials:
 Consider an experiment consisting of n trials, each can be a
success or a failure.
 Let Xj = 1 if the jth experiment is a success
 and Xj = 0 if the jth experiment is a failure
 The Bernoulli distribution (one trial):
 p, x j  1, j  1,2,..., n

p j ( x j )  p( x j )  1  p  q, x j  0,j  1,2,...,n
0, otherwise

 where E(Xj) = p and V(Xj) = p (1-p) = p q
 Bernoulli process:
 The n Bernoulli trials where trails are independent:
p(x1,x2,…, xn) = p1(x1) p2(x2) … pn(xn)

17
Binomial Distribution [Discrete Dist’n]

 The number of successes in n Bernoulli trials, X, has a

binomial distribution.
 n  x n  x
  p q , x  0,1,2,..., n
p( x)   x 
0, otherwise


The number of outcomes having

Probability that there are
the required number of successes
x successes and (n-x) failures
and failures

 The mean, E(x) = p + p + … + p = n*p

 The variance, V(X) = pq + pq + … + pq = n*pq

18
Geometric & Negative
Binomial Distribution [Discrete Dist’n]

 Geometric distribution
 The number of Bernoulli trials, X, to achieve the 1st success:
 q x 1 p, x  0,1,2,..., n
p( x)  
0, otherwise
 E(x) = 1/p, and V(X) = q/p2

 Negative binomial distribution

 The number of Bernoulli trials, X, until the kth success
 If Y is a negative binomial
 y  1 distribution with parameters p and k,
then:  y k k
 q p , y  k , k  1, k  2,...
p( x)   k  1
0, otherwise


 E(Y) = k/p, and V(X) = kq/p2 19

Poisson Distribution [Discrete Dist’n]

 Poisson distribution describes many random processes

quite well and is mathematically quite simple.
 where a > 0, pdf and cdf are:
 e   x x
e   i

p ( x)   x! , x  0,1,... F ( x)  
i 0 i!
0, otherwise

 E(X) = a = V(X)

20
Poisson Distribution [Discrete Dist’n]

 Example: A computer repair person is “beeped” each

time there is a call for service. The number of beeps per
hour ~ Poisson(a = 2 per hour).

 The probability of three beeps in the next hour:

p(3) = e-223/3! = 0.18
also, p(3)= F(3) – F(2) = 0.857-0.677=0.18

 The probability of two or more beeps in a 1-hour period:

p(2 or more) = 1 – p(0) – p(1)
= 1 – F(1)
= 0.594

21
Continuous Distributions
 Continuous random variables can be used to
describe random phenomena in which the
variable can take on any value in some interval.
 In this section, the distributions studied are:
 Uniform
 Exponential
 Normal
 Weibull
 Lognormal

22
Uniform Distribution [Continuous Dist’n]

 A random variable X is uniformly distributed on the

interval (a,b), U(a,b), if its pdf and cdf are:
 1 0, x a
 , a xb x a
f ( x)   b  a F ( x)   , a  x b
0, otherwise b  a
1, xb

 Properties
 P(x1 < X < x2) is proportional to the length of the interval [F(x2) –
F(x1) = (x2-x1)/(b-a)]
 E(X) = (a+b)/2 V(X) = (b-a)2/12
 U(0,1) provides the means to generate random numbers,
from which random variates can be generated.

23
Exponential Distribution [Continuous Dist’n]

 A random variable X is exponentially distributed with

parameter l > 0 if its pdf and cdf are:
e  x , x  0 0, x 0
f ( x)   F ( x )   x  t
0 e dt  1  e , x  0
 x
0, elsewhere

 E(X) = 1/l V(X) = 1/l2

 Used to model interarrival times when arrivals are

completely random, and to model service times that are

highly variable

 For several different exponential pdf’s (see figure), the value

of intercept on the vertical axis is l, and all pdf’s eventually

intersect.

24
Exponential Distribution [Continuous Dist’n]

 Memoryless property
 For all s and t greater or equal to 0:
P(X > s+t | X > s) = P(X > t)

 Example: A lamp ~ exp(l = 1/3 per hour), hence, on

average, 1 failure per 3 hours.
 The probability that the lamp lasts longer than its mean life is:
P(X > 3) = 1-(1-e-3/3) = e-1 = 0.368
 The probability that the lamp lasts between 2 to 3 hours is:
P(2 <= X <= 3) = F(3) – F(2) = 0.145
 The probability that it lasts for another hour given it is
operating for 2.5 hours:
P(X > 3.5 | X > 2.5) = P(X > 1) = e-1/3 = 0.717

25
Normal Distribution [Continuous Dist’n]

 A random variable X is normally distributed has the pdf:

1  1  x   2 
f ( x)  exp   ,   x 
 2  2    

 Mean:     
 Variance:  2 0
 Denoted as X ~ N(m,s2)
 Special properties:
 lim x f ( x)  0, and lim x f ( x)  0.
 f(m-x)=f(m+x); the pdf is symmetric about m.
 The maximum value of the pdf occurs at x = m; the mean and
mode are equal.

26
Normal Distribution [Continuous Dist’n]

 Evaluating the distribution:

 Use numerical methods (no closed form)
 Independent of m and s, using the standard normal distribution:
Z ~ N(0,1)
 Transformation of variables: let Z = (X - m) / s,
 x 
F ( x )  P  X  x   P Z  
  
( x ) / 1 z2 / 2
 e dz

2
( x ) /
  ( z )dz   ( x  ) , where ( z )  
z 1 t 2 / 2
e dt
 
2

27
Normal Distribution [Continuous Dist’n]

 Example: The time required to load an oceangoing

vessel, X, is distributed as N(12,4)
 The probability that the vessel is loaded in less than 10 hours:
 10  12 
F (10)      (1)  0.1587
 2 

 Using the symmetry property, F(1) is the complement of F (-1)

28
Weibull Distribution [Continuous Dist’n]

 A random variable X has a Weibull distribution if its pdf has the form:
   x    1   x    

f ( x)     
exp   , x  
    
0, otherwise

 3 parameters:
 Location parameter: u, (  )
 Scale parameter: b , (b > 0)
 Shape parameter. a, (> 0)
 Example: u = 0 and a = 1:

When b = 1,

X ~ exp(l = 1/a)

29
Lognormal Distribution [Continuous Dist’n]

 A random variable X has a lognormal distribution if its pdf

has the form:
 1  ln x  μ  2  m=1, s2=0.5,1,2.
 exp , x  0
f ( x)   2π σx  2σ 2

0, otherwise

2
 Mean E(X) = em+s /2
2 2
 Variance V(X) = e2m+s /2 (es - 1)

 Relationship with normal distribution

 When Y ~ N(m, s2), then X = eY ~ lognormal(m, s2)
 Parameters m and s2 are not the mean and variance of the
lognormal

30
Poisson Process
 Definition: N(t) is a counting function that represents
the number of events occurred in [0,t].
 A counting process {N(t), t>=0} is a Poisson process
with mean rate l if:
 Arrivals occur one at a time
 {N(t), t>=0} has stationary increments
 {N(t), t>=0} has independent increments
 Properties
e  t ( t ) n
P[ N (t )  n]  , for t  0 and n  0,1,2,...
n!
 Equal mean and variance: E[N(t)] = V[N(t)] = lt
 Stationary increment: The number of arrivals in time s to t is
also Poisson-distributed with mean l(t-s)

31
Interarrival Times [Poisson Process]

 Consider the interarrival times of a Possion process (A1, A2, …),

where Ai is the elapsed time between arrival i and arrival i+1

 The 1st arrival occurs after time t iff there are no arrivals in the interval
[0,t], hence:
P{A1 > t} = P{N(t) = 0} = e-lt
P{A1 <= t} = 1 – e-lt [cdf of exp(l)]
 Interarrival times, A1, A2, …, are exponentially distributed and
independent with mean 1/l

Arrival counts ~ Poi(l) Interarrival time ~ Exp(1/l)

Stationary & Independent Memoryless

32
Splitting and Pooling [Poisson Dist’n]

 Splitting:
 Suppose each event of a Poisson process can be classified as
Type I, with probability p and Type II, with probability 1-p.
 N(t) = N1(t) + N2(t), where N1(t) and N2(t) are both Poisson
processes with rates l p and l (1-p)
lp N1(t) ~ Poi[lp]

l
N(t) ~ Poi(l)

N2(t) ~ Poi[l(1-p)]
l(1-p)

 Pooling:
 Suppose two Poisson processes are pooled together
 N1(t) + N2(t)N1(t)=~ N(t),
Poi[l ]
where N(t)
l
is a Poisson processes with
rates l1 + l2 1 1 l +l
1 2
N(t) ~ Poi(l + l )
1 2

N2(t) ~ Poi[l ] l
2 2
33
Nonstationary Poisson
Process (NSPP) [Poisson Dist’n]
 Poisson Process without the stationary increments, characterized by
l(t), the arrival rate at time t.
 The expected number of arrivals by time t, L(t):
t
Λ(t)   λ(s)ds
0

 Relating stationary Poisson process n(t) with rate l=1 and NSPP N(t)
with rate l(t):
 Let arrival times of a stationary process with rate l = 1 be t1, t2, …,
and arrival times of a NSPP with rate l(t) be T1, T2, …, we know:
ti = L(Ti)
Ti = L-1(ti)

34
Nonstationary Poisson
Process (NSPP) [Poisson Dist’n]
 Example: Suppose arrivals to a Post Office have rates 2 per minute
from 8 am until 12 pm, and then 0.5 per minute until 4 pm.
 Let t = 0 correspond to 8 am, NSPP N(t) has rate function:
2, 0  t  4
 (t )  
0.5, 4  t 8
Expected number of arrivals by time t:
2t , 0  t 4
(t )   4 t t
0 2 ds  4 0. 5ds 
2
 6, 4  t 8
 Hence, the probability distribution of the number of arrivals between
11 am and 2 pm.
P[N(6) – N(3) = k] = P[N(L(6)) – N(L(3)) = k]
= P[N(9) – N(6) = k]
= e(9-6)(9-6)k/k! = e3(3)k/k!

35
Empirical Distributions [Poisson Dist’n]

 A distribution whose parameters are the observed values

in a sample of data.
 May be used when it is impossible or unnecessary to establish that
a random variable has any particular parametric distribution.
 Advantage: no assumption beyond the observed values in the
sample.
 Disadvantage: sample might not cover the entire range of possible
values.

36
Summary
 The world that the simulation analyst sees is probabilistic,
not deterministic.
 In this chapter:
 Reviewed several important probability distributions.
 Showed applications of the probability distributions in a simulation
context.
 Important task in simulation modeling is the collection and
analysis of input data, e.g., hypothesize a distributional
form for the input data. Reader should know:
 Difference between discrete, continuous, and empirical
distributions.
 Poisson process and its properties.

37