Experimental Designs and ANOVA

Experimental Design
and Analysis of Variance

• An Introduction to Experimental Design and Analysis of Variance
• Analysis of Variance and the Completely Randomized Design
• Multiple Comparison Procedures
• Randomized Block Design
• Factorial
© 2021 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. 1
1. An Introduction to Experimental Design
and Analysis of Variance (1 of 3)
• Statistical studies can be classified as being either experimental or
observational.
• In an experimental study, one or more factors are controlled so that data can be
obtained about how the factors influence the variables of interest.
• In an observational study, no attempt is made to control the factors.
• Cause-and-effect relationships are easier to establish in experimental studies
than in observational studies.
• Analysis of variance (ANOVA) can be used to analyze the data obtained from
experimental or observational studies.
An Introduction to Experimental Design
In this chapter three types of experimental designs are introduced:
• a completely randomized design

• a randomized block design
• a factorial experiment
An Introduction to Experimental Design
• A factor is a variable that the experimenter has selected for investigation.
• A treatment is a level of a factor.
• Experimental units are the objects of interest in the experiment.
• A completely randomized design is an experimental design in which the
treatments are randomly assigned to the experimental units.
Analysis of Variance: A Conceptual Overview
(1 of 5)
• Analysis of Variance (ANOVA) can be used to test for the equality of three or
more population means.
• Data obtained from observational or experimental studies can be used for the
analysis.
• We want to use the sample results to test the following hypotheses:
H0 : 1  2  3  . . .  k
Ha : Not all population means are equal.
(2 of 5)
• If H0 is rejected, we cannot conclude that all population means are different.
• Rejecting H0 means that at least two population means have different values.
(3 of 5)
Assumptions for Analysis of Variance
• For each population, the response (dependent) variable is normally distributed.

• The variance of the response variable, denoted  2, is the same for all of the
populations.
• The observations must be independent.
(4 of 5)
Sampling distribution of x, given H0 is true.
(5 of 5)
Sampling distribution of x, given H0 is false.
2. Analysis of Variance and the Completely
Randomized Design
• Between-Treatments Estimate of Population Variance
• Within-Treatments Estimate of Population Variance
• Comparing the Variance Estimates: The F Test
• ANOVA Table
Between-Treatments Estimate of Population
Variance 2
• The estimate of  2 based on the variation of the sample means is called the
mean square due to treatments and is denoted by MSTR.
 j 1 j
k
n ( x  x )2
MSTR 
k 1
Numerator is called the sum of squares due to treatments (SSTR).

Denominator is the degrees of freedom associated with SSTR.
Within-Treatments Estimate of Population
Variance 2
• The estimate of  2 based on the variation of the sample observations within
each sample is called the mean square error and is denoted by MSE.
 j 1 j
k
( n  1)s j
2
MSE 
nT  k
Numerator is called the sum of squares due to error (SSE).

Denominator is the degrees of freedom associated with SSE.
Comparing the Variance Estimates:
The F Test (1 of 2)
If the null hypothesis is true and the ANOVA assumptions are valid, the
sampling distribution of MSTR/MSE is an F distribution with MSTR d.f.
equal to k  1 and MSE d.f. equal to nT  k.
If the means of the k populations are not equal, the value of MSTR/MSE
will be inflated because MSTR overestimates  2.
Hence, we will reject H0 if the resulting value of MSTR/MSE appears to be
too large to have been selected at random from the appropriate F
distribution.
Comparing the Variance Estimates:
The F Test (2 of 2)
• Sampling Distribution of MSTR/MSE
ANOVA Table for a Completely Randomized
Design (1 of 3)
SST is partitioned into SSTR and SSE.

SST’s degrees of freedom (d.f.) are partitioned into SSTR’s d.f. and SSE’s d.f.
Design (2 of 3)
• SST divided by its degrees of freedom nT  1 is the overall sample
that would be obtained if we treated the entire set of observations as one data
set.
With the entire data set as one sample, the formula for computing the total sum
of squares, SST, is:
2
SST   j 1  i 1( xij  x )  SSTR  SSE
k nj
Design (3 of 3)
• ANOVA can be viewed as the process of partitioning the total sum of squares
and the degrees of freedom into their corresponding sources: treatments and
error.
• Dividing the sum of squares by the appropriate degrees of freedom provides the
variance estimates, the F value and the p-value used to test the hypothesis of
equal population means.
Test for the Equality of k Population Means
(1 of 2)
• Hypotheses
H0 : 1  2  3  . . .  k
Ha : Not all population means are equal.
• Test Statistic
MSTR
F
MSE
Test for the Equality of k Population Means
(2 of 2)
Rejection Rule
p-value approach: Reject H0 if p-value  
Critical-value approach: Reject H0 if F  F
where the value of F is based on an F distribution with k  1 numerator d.f.

and nT  k denominator d.f.
Testing for the Equality of k Population Means:
A Completely Randomized Design (1 of 7)
Example: Chemitech experiment
Chemitech developed a new filtration system for municipal water supplies. There
are different methods that can be used to assemble the system. Chemitech has
narrowed down to three methods: A, B and C and wants to determine which
assembly method can produce the greatest number of filtration systems per
week.
For the purpose, it selected 15 workers and then randomly assigned each of the
three treatments to 5 of the workers.
Factor: Assembly method

Treatments: A, B and C
Experimental units: Employees
Response units: Number of units produced
Observation A B C
1 58 58 48
2 64 69 57
3 55 71 59
4 66 64 47
5 67 68 49
Sample mean 62 66 52
Sample variance 27.5 26.5 31.0
Sample standard 5.244 5.148 5.568
deviation
• Hypotheses
H0 : 1  2  3
Ha : Not all the means are equal
where:
1  mean number of units produced per week using method A
2  mean number of units produced per week using method B
3  mean number of units produced per week using method C
• Mean Square between Treatments
Because the sample sizes are all equal:
x  ( x1  x2  x3 ) 3  (62  66  52) 3  60
SSTR  5  62  60   5  66  60   5 52  60   520
2 2 2
MSTR  520 / 3  1  260

• Mean Square Error
SSE  4  27.5   4 26.5   4 31  340
MSE  340 / 15  3   28.33
• Rejection Rule
p-value approach: Reject H0 if p-value  .05
Critical-value approach: Reject H0 if F  3.89
where F.05  3.89 is based on an F distribution with 2 numerator degrees of

freedom and 12 denominator degrees of freedom.
• Test Statistic F  MSTR MSE  260 28.33  9.18
• Conclusion
The p-value is less than .01, where F = 9.18. (Excel provides a p-value of .004.)
Therefore, we reject H0.
There is sufficient evidence to conclude that the means of three populations are
not equal.
Design
Source of Sum of Degrees of Mean p-

Variation Squares Freedom Square F Value
Treatments 520 2 260.00 9.18 .004
Error 340 12 28.33
Total 860 14
Excel’s ANOVA : Single Factor Tool (1 of 3)
• Step 1: Click the Data tab on the Ribbon

• Step 2: In the Analysis group, click Data Analysis
• Step 3: Choose Anova: Single Factor from the list of Analysis Tools
• Step 4: When the Anova: Single Factor dialog box appears: (see details on
next slide)
• Summary and
output data
Testing for the Equality of k Population
Means: An Observational Study (1 of 11)
Example: National Computer Products, Inc. (NCP)
NCP manufactures printers and fax machines at plants located in Atlanta, Dallas
and Seattle. To measure how much employees at each plants know about quality
management, a random sample of 6 employees was selected from each plant
and the employees selected were given quality awareness examination.
Managers want to know if there is significant difference in the mean examination

score for the employees at three plants. An F test will be conducted using   .05.
A simple random sample of 6 employees from each of the three plants was taken
and their examination score was tabulated.
Factor: Quality awareness

Treatments: Atlanta, Dallas, Seattle
Experimental units: Employees
Response units: Examination score
Observation Atlanta Dallas Seattle
1 85 71 59
2 75 75 64
3 82 73 62
4 76 74 69
5 71 69 75
6 85 82 67
Sample mean 79 74 66
Sample variance 34 20 32
Sample standard 5.83 4.47 5.66
deviation
p-Value and Critical-Value Approaches
1. Develop the hypotheses.
H0 : 1  2  3
Ha : Not all the means are equal
where:
1  mean examination score for population 1
p-Value and Critical Value Approaches
2. Specify the level of significance.   .05
3. Compute the value of the test statistic.
Mean square due to treatments (sample sizes are all equal):
x  (79  74  66) 3  73
SSTR  6 79  73   6 74  73   6  66  73   516
2 2 2
MSTR  516 (3  1)  258
p-Value and Critical Value Approaches
4. Compute the value of the test statistic.
Mean square due to error:
SSE = 5 34.0   5 20.0   5 32.0   430

MSE  430 18  3   28.667
F  MSTR MSE  430 28.667  15
Source of Sum of Degrees of

Variation squares freedom Mean square F p-value
Treatments 516 2 258 15
Error 430 15 28.667
Total 846 17
Rejection Rule
Critical-value approach: Reject H0 if F  3.68
where F.05  3.68 is based on an F distribution with 2 numerator degrees of

freedom and 15 denominator degrees of freedom.
p-Value Approach
4. Compute the p-value.
With 2 numerator d.f. and 15 denominator d.f., the p-value is .01 for
F  6.36. Therefore, the p-value is less than .01 for F  15.
5. Determine whether to reject H0.
The p-value  .05,so we reject H0 .
Critical-Value Approach
4. Determine the critical value and rejection rule.
Based on an F distribution with 2 numerator d.f. and 15
denominator d.f., F.05  3.68, reject H0 if F  3.68.
5. Determine whether to reject H0.
Because F  15  3.68, we reject H0.
We can conclude that the mean examination scores of employees working at
different plants is not the same.
3. Multiple Comparison Procedures
• Suppose that analysis of variance has provided statistical evidence to reject the
null hypothesis of equal population means.
• Fisher’s least significant difference (LSD) procedure can be used to determine
where the differences occur.
Fisher’s LSD Procedure (1 of 2)
• Hypotheses
H0 : i   j
Ha : i   j
• Test Statistic
xi  x j
t
1 1
MSE   
n n 
 i j 
Fisher’s LSD Procedure (2 of 2)
Rejection Rule
p-value approach: Reject H0 if p-value  .
Critical-Value Approach: Reject H0 if t  t 2 or t  t 2 .
where the value of t /2 is based on a t distribution with nT  k

degrees of freedom.
Fisher’s LSD Procedure Based on the Test
Statistic (1 of 6)
• Hypotheses
H0 : i   j
Ha : i   j
• Test Statistic
xi  x j
• Rejection Rule
Reject H0 if xi  x j  LSD
1 1
Where LSD  t 2 MSE   
n n 
 i j 
Statistic (2 of 6)
Example: Chemitech Experiment
• Recall the Chemitech experiment where the managers wanted to know if there
is difference in mean number of units produced per week when different
assembly methods are used.
• Analysis of variance has provided statistical evidence to reject the null

hypothesis of equal population means. Fisher’s least significant difference (LSD)
procedure can be used to determine where the differences occur.
Statistic (3 of 6)
Example: Chemitech Experiment
For   .05 and nT  k  15 – 3  12 degrees of freedom, t.025  2.179.
1 1
LSD  t 2 MSE   
n n 
 i j 
 1 1
LSD  2.179 28.33     7.34
5 5
Statistic (4 of 6)
LSD for Method A and Method B
Hypothesis (A)
H0 : 1  2
Ha : 1  2
Rejection Rule: Reject H0 if x1  x2  7.34
Test Statistic: x1  x2  62  66  4
Conclusion: The mean number of units produced per week for method A is equal
to the population mean for method B.
Statistic (5 of 6)
LSD for Method B and Method C
Hypothesis (B)
H0 : 1  2
Ha : 1  2
Test Statistic: x1  x2  66  52  14
Conclusion: The mean number of units produced per week for method B is not
equal to the population mean for method C.
Statistic (6 of 6)
LSD for Method A and Method C
Hypothesis (C)
H0 : 1  2
Ha : 1  2
Test Statistics: x1  x2  62  52  10
Conclusion: The mean number of units produced per week for method A is not
equal to the population mean for method C. So A and B both differ from method
C.
Type I Error Rates
The comparisonwise Type I error rate  indicates the level of significance
associated with a single pairwise comparison.
The experimentwise Type I error rate  EW is the probability of making a Type I

error on at least one of the  k  1! pairwise comparisons.
 EW  1– (1–  )( k –1)!
The experiment-wise Type I error rate gets larger for problems with more
populations (larger k).
4. Randomized Block Design (1 of 10)
• Experimental units are the objects of interest in the experiment.

• A completely randomized design is an experimental design in which the
treatments are randomly assigned to the experimental units.
• If the experimental units are heterogeneous, blocking can be used to form
homogeneous groups, resulting in a randomized block design.
Randomized Block Design (2 of 10)
• ANOVA Procedure
For a randomized block design the sum of squares total (SST) is
partitioned into three groups:
sum of squares due to treatments,
sum of squares due to blocks, and
sum of squares due to error.
SST  SSTR  SSBL  SSE
The total degrees of freedom nT  1 are partitioned such that k  1 degrees

of freedom go to treatments, b  1 go to blocks, and  k  1 b  1 go to the
error term.
• ANOVA Table
Example: Air Traffic Controller Stress Test
A study measuring the fatigue and stress of air traffic controllers resulted in
proposals for modification and redesign of the controller’s workstation. Three
alternatives are selected for the purpose. A test needs to be conducted to
determine to what extent the three alternatives differ in terms of their effect on
controller stress.
• Example: Air Traffic Controller Stress Test
Factor . . . Workstation
Treatments . . . System A, B and C
Blocks . . . Controllers
Response variable . . . Miles per gallon
• Example: Air Traffic Controller Stress Test
Treatments Treatments Treatments

System A System B System C Block Means
Controller 1 15 15 18 16
Blocks Controller 2 14 14 14 14
Treatment 13.5 13 15.5
Means

• Mean Square Due to Treatments
The overall sample mean is 14. Thus,
SSTR  6[13.5  14   13  14   15.5  14  ]  21
2 2 2
MSTR  21 (3  1)  10.5
• Mean Square Due to Blocks
SSBL  3[16  14   . . .  13  14  ]  30
2 2
MSBL  30 (6  1)  6
• Mean Square Due to Error
SSE  70 – 21– 30  19
MSE  19 [(3  1)(6  1)]  1.9

ANOVA Table
Source of Sum of Degrees of

Variation Squares Freedom Mean Square F p- value
Treatments 21 2 10.5 10.5/1.9 = .024
5.53
Blocks 30 5 6.0
Errors 19 10 1.9
Total 70 17

• Rejection Rule

• Test Statistic
F  MSTR MSE  10.5 1.9  5.53
• Conclusion
The p-value is between .025 (where F  5.46) and .01 (where F  7.56).
(Excel provides a p-value of .024). Therefore, we reject H0.
and conclude that the population mean stress levels differ for the three
workstation alternatives.
Excel’s ANOVA – Two-Factor without
replication tool (1 of 5)
• Excel Data Worksheet
Step 1: Click the Data tab on the Ribbon
Step 2: In the Analyze group, click Data Analysis
Step 3: Choose Anova: Two-Factor Without Replication
from the list of Analysis Tools
Step 4: When the Anova: Two-Factor Without
Replication dialog box appears:
Enter A1:D7 in the Input Range Box
Select the check box for Labels
Enter .05 in the Alpha Box
Enter A9 in the Output Range box
Click OK
Example: Air Traffic Controller
Stress Test
• Anova: Two-Factor without
Replication Output (top)
Example: Air Traffic Controller
Stress Test
• Anova: Two-Factor
without Replication
Output (bottom)
5. Factorial Experiment
• In some experiments we want to draw conclusions about more than one

variable or factor.
• Factorial experiments and their corresponding ANOVA computations are
valuable designs when simultaneous conclusions about two or more factors are
required.
• The term factorial is used because the experimental conditions include all
possible combinations of the factors.
• For example, for a levels of factor A and b levels of factor B, the experiment will
involve collecting data on ab treatment combinations.
Two-Factor Factorial Experiment (1 of 9)
• ANOVA Procedure
• The ANOVA procedure for the two-factor factorial experiment is similar to the
completely randomized experiment and the randomized block experiment.
• We again partition the sum of squares total (SST) into its sources.
SST  SSA  SSB  SSAB  SSE
The total degrees of freedom, nT  1, are partitioned such that a  1 d.f. go

to Factor A,  b  1 d.f. go to Factor B, a  1 b  1 d.f. go to Interaction,
and ab  r  1 go to Error.
• ANOVA table with r replications
Step 1 Compute the total sum of squares

a b r
SST   ( xijk  x )2
i 1 j 1 k 1
Step 2 Compute the sum of squares for factor A

a
SSA  br  ( xi .  x )2
i 1
Step 3 Compute the sum of squares for factor B

b
SSB  ar  ( x j .  x )2
j 1
Step 4 Compute the sum of squares for interaction

a b
SSAB  r  ( xij  xi .  x. j  x )2
i 1 j 1
Step 5 Compute the sum of squares due to error
SSE  SST – SSA – SSB  SSAB
Example: GMAT test scores
A study was conducted to determine if the GMAT preparation programmes affect

GMAT scores. The GMAT exam is generally taken by students from College of
business, College of engineering and the College of Arts and Sciences. The test
also intended to check if the student’s undergraduate college affected the GMAT
scores.

Six students from each of the undergraduate schools are selected and are two of
them are randomly
Factor B Factor B
assigned to each College College Factor B College
preparation program, Business Engineering Arts and Sciences
resulting in 18 Factor A Three-hour review 500 540 480
students being 580 460 400
used in the study. Preparation

program
One-day program 460 560 420
540 620 480
10 – week course 560 600 480
600 580 410

Factors
Main effect (Factor A): Do the preparation programs differ in terms of effect GMAT
scores?
Main effect (Factor B): Do the undergraduate colleges differ in terms of effect on
GMAT scores?
Interaction effect: Do students in some colleges do better on one type of
preparation program whereas others do better on a different type of preparation
program?
Replications: 2
• ANOVA table with r replications
Source of Sum of Degrees of Mean p-

Variation Squares Freedom Square F Value
Factor A 6100 2 3050 1.3829 .2994
Factor B 45300 2 22,650 10.2695 .0048
Interaction 11,200 4 2,800 1.2695 .3503
Error 19,850 9 2,205.5556
Total 82,450 17
Conclusions Using the p-Value Approach
Programs: p-value  .2994    .05
There is no significant difference among the three preparation programs

Undergraduate college: p-value  .0048    .05
Undergraduate colleges do not provide same preparation for performance on the GMAT.
Interaction: p-value  .3503    .05
No significant interaction effect.
Excel’s ANOVA: Two Factor with replication
tool (1 of 2)
Example:
GMAT test scores
Excel’s ANOVA: Two Factor with replication
tool (2 of 2)
Example:
GMAT test scores

Experimental Designs and ANOVA

Uploaded by

Copyright:

Available Formats

Experimental Designs and ANOVA

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Experimental Designs and ANOVA

Uploaded by

Copyright:

Available Formats

Experimental Design

and Analysis of Variance

• a completely randomized design

• For each population, the response (dependent) variable is normally distributed.

Numerator is called the sum of squares due to treatments (SSTR).

Numerator is called the sum of squares due to error (SSE).

SST is partitioned into SSTR and SSE.

p-value approach: Reject H0 if p-value  

Critical-value approach: Reject H0 if F  F

where the value of F is based on an F distribution with k  1 numerator d.f.

Factor: Assembly method

MSTR  520 / 3  1  260

where F.05  3.89 is based on an F distribution with 2 numerator degrees of

Source of Sum of Degrees of Mean p-

• Step 1: Click the Data tab on the Ribbon

Managers want to know if there is significant difference in the mean examination

Factor: Quality awareness

MSTR  516 (3  1)  258

SSE = 5 34.0   5 20.0   5 32.0   430

F  MSTR MSE  430 28.667  15

Source of Sum of Degrees of

where F.05  3.68 is based on an F distribution with 2 numerator degrees of

5. Determine whether to reject H0.

The p-value  .05,so we reject H0 .

p-value approach: Reject H0 if p-value  .

Critical-Value Approach: Reject H0 if t  t 2 or t  t 2 .

where the value of t /2 is based on a t distribution with nT  k

• Analysis of variance has provided statistical evidence to reject the null

For   .05 and nT  k  15 – 3  12 degrees of freedom, t.025  2.179.

Rejection Rule: Reject H0 if x1  x2  7.34

Rejection Rule: Reject H0 if x1  x2  7.34

The experimentwise Type I error rate  EW is the probability of making a Type I

• Experimental units are the objects of interest in the experiment.

The total degrees of freedom nT  1 are partitioned such that k  1 degrees

Example: Air Traffic Controller Stress Test

• Example: Air Traffic Controller Stress Test

• Example: Air Traffic Controller Stress Test

Treatments Treatments Treatments

Example: Air Traffic Controller Stress Test

Example: Air Traffic Controller Stress Test

Source of Sum of Degrees of

Example: Air Traffic Controller Stress Test

p-value approach: Reject H0 if p-value  .05

Example: Air Traffic Controller Stress Test

• In some experiments we want to draw conclusions about more than one

SST  SSA  SSB  SSAB  SSE

The total degrees of freedom, nT  1, are partitioned such that a  1 d.f. go

Step 1 Compute the total sum of squares

Step 2 Compute the sum of squares for factor A

Step 3 Compute the sum of squares for factor B

Step 4 Compute the sum of squares for interaction

Step 5 Compute the sum of squares due to error

SSE  SST – SSA – SSB  SSAB

Example: GMAT test scores

A study was conducted to determine if the GMAT preparation programmes affect

Example: GMAT test scores

used in the study. Preparation

Example: GMAT test scores

Example: GMAT test scores