DESIGN OF EXPERIMENTS

How To Analyze
A Split-Plot
Experiment
by Kevin J. Potcner and Scott M. Kowalski

M
any quality improvement projects of the resulting product. A materials engineer may
require some form of experimentation run a plastic injection molding process using differ-
on a process. A chemical engineer may ent grades of raw material to determine which pro-
wish to determine the settings for certain process duces the least variability in breaking strength.
variables to optimize a critical quality characteristic The deliberate changing of input process vari-
ables with the intention of studying their effect on
output variables is referred to as a designed experi-
ment. Typically, statisticians identify a designed
In 50 Words experiment by describing two primary components:
Or Less 1. One component, referred to as the treatment
structure, details the different factors (input
• Experiments with simple design structures, variables) the experiment will incorporate and
the different settings (levels) for those factors.
such as complete randomization, are often not For example, a 25 full factorial treatment struc-
realistic in the real world. ture means five factors will be used in the
experiment, each studied at two levels, and all
• Typically an experiment will have some form of 2 x 2 x 2 x 2 x 2 = 32 treatment combinations
are to be run.
randomization restriction, and the split-plot 2. The other component is referred to as the
method is a solution. experimental or design structure of the experi-
ment. This component illustrates how the
• The analysis of a split-plot experiment involves experimental runs are to be carried out—for
example, defining the experimental and obser-
two error variances. vational units, selecting the experimental units
and assigning them to the treatment combina-
tions, choosing the randomization scheme and

QUALITY PROGRESS I DECEMBER 2004 I 67

DESIGN OF EXPERIMENTS

deciding how the treatment combinations will wood panel would be randomly assigned a partic-
be changed throughout the experiment. ular pretreatment and stain combination. But it
In a previous article in Quality Progress, we illus- turns out to be very difficult to apply the pretreat-
trated the features of the split-plot design, how ment to a small wood panel.
common the features are in industrial experimenta- The easiest way would be to apply each of the
tion and how the practitioner can recognize this sit- pretreatment types (one and two) to an entire board,
then cut each board into four smaller pieces and
apply the four stain types to the smaller pieces. This
is shown in Figure 2.
So how exactly will the experiment be conduct-
In many real experimental ed? For example, how many boards will be used
for each treatment combination? How many repli-
situations, a restriction is cates of each treatment combination will be run? In
what order will the experimental runs be conduct-
typically placed on the ed? How many measurements will be made on
each small piece? These decisions should be based
randomization of the runs. on both statistical and practical considerations.
Suppose the experimenter has decided to run three
replicates of the pretreatment factor. This results in six
boards and 3 x 2 x 4 = 24 total observations. To pro-
uation.1 We will now illustrate the proper analysis duce an experimental run for this process, you must
of this particular type of design structure. first pretreat a board. After one of the randomly
selected pretreatments has been applied, the board is
Example of a Split-Plot Design
Consider an experiment involving the water resis-
tant property of wood. Two types of wood pretreat-
ment (one and two) and four types of stain (one,
FIGURE 2 Treatment Application
two, three and four) have been selected as variables
of interest. A graphical representation of this type of
treatment design is shown in Figure 1. Pretreatment one Pretreatment two
Conducting this experiment in a completely
randomized fashion would require eight wood
panels for each full replicate of the design. Each

FIGURE 1 Factors That Affect Wood’s

Water Resistance
2
Pretreatment

Stain two Stain three Stain four Stain two

1
1 2 3 4
Stain one Stain four Stain three Stain one
Stain

68 I DECEMBER 2004 I www.asq.org

cut into four pieces and then stained using one of the TABLE 2 Whole-Plot Analysis Using
four stains selected at random. The Averages of Resistance
The reader should recognize this is a split-plot In Each Whole Plot
design for four reasons:
Analysis of variance for the average resistance
1. For the pretreatment factor, an experimental
unit is the entire board or a set of four pieces of Source DF SS MS F P
the board after they are cut. For the stain fac- Pretreat 1 195.51 195.51 4.03 0.115
Error 4 193.84 48.46
tor, an experimental unit is an individual piece
Total 5 389.35
of the board. Having unequal sized experi-
mental units for the different factors is one key DF = degrees of freedom
element of a split-plot design. SS = sums of squares
2. Each factor uses a different randomization MS = mean square
F = F-statistic
scheme. In contrast, a complete randomized P = p-value
design would use one randomization scheme
for all 24 experimental runs.
3. Note for a single run at one level of pretreat- ment, four separate runs are conducted for the
stains. As a result, pretreatment could be
thought of as a hard-to-change factor, while
TABLE 1 Data for Wood Example stain could be considered an easy to change
factor.
4. The number of experimental replicates is not
Pretreat Stain WP error Resistance the same for each factor. Pretreatment has only
2 2 4 53.5 three experimental replicates for each of the
2 4 4 32.5 two factor levels, while stain has six experi-
2 1 4 46.6 mental replicates for the stain factor levels.
2 3 4 35.4 Because of these features, we would say the
2 4 5 44.6 experimenter has run a 2 x 4 full factorial treatment
2 1 5 52.2 structure within a split-plot design structure. Each
2 3 5 45.9 of the six whole-plots (entire boards) has four sub-
plots (smaller pieces of board), resulting in three
2 2 5 48.3
replicates at the whole-plot level and six replicates
1 3 1 40.8
at the subplot level.
1 1 1 43.0
1 2 1 51.8 How To Analyze the Experiment
1 4 1 45.5 The simplest experiment from a statistical analy-
1 2 2 60.9 sis perspective is what’s called a completely ran-
1 4 2 55.3 domized design structure. This, however, would
1 3 2 51.1 require all 8 x 3 = 24 experimental runs to be con-
1 1 2 57.4 ducted in a completely random order. For the exper-
2 1 6 32.1 iment to be run in this way, each of the 24 runs
2 4 6 30.1 would need to be a “true” experimental run. This
2 2 6 34.4
would include a complete preparation and setup of
the experimental materials and equipment.
2 3 6 32.2
As you can imagine, this experimental approach
1 1 3 52.8
is not always efficient, practical or at times even
1 3 3 51.7
possible to run. In many real experimental situa-
1 4 3 55.3 tions, a restriction is typically placed on the ran-
1 2 3 59.2 domization of the runs. Such restriction, however,
WP = whole plot affects the statistical analysis.

QUALITY PROGRESS I DECEMBER 2004 I 69

DESIGN OF EXPERIMENTS

TABLE 3 Incorrect Completely TABLE 4 Correct Split-Plot Analysis

Randomized Design Analysis For Water Resistance of Wood
For Water Resistance of Wood Analysis of variance for resistance, using adjusted SS for tests

Source DF SS MS F P Source DF SS MS F P
Pretreat 1 782.04 782.04 13.49 0.002 Pretreat 1 782.04 782.04 4.03 0.115
Stain 3 266.00 88.67 1.53 0.245 WP (pretreat) 4 775.36 193.84 15.25 .*
Pretreat x Stain 3 62.79 20.93 0.36 0.782 Stain 3 266.01 88.67 6.98 0.006
Error 16 927.88 57.99 Pretreat x stain 3 62.79 20.93 1.65 0.231
Total 23 2038.72 Error 12 152.52 12.71
Total 23 2038.72
DF = degrees of freedom
SS = sums of squares WP = whole-plot errors
MS = mean square DF = degrees of freedom
F = F-statistic SS = sums of squares
P = p-value MS = mean square
F = F-statistic
P = p-value

An example illustrates the correct analysis of split- split-plot experiment as two separate experiments
plot experiments. Consider the previously described corresponding to the two levels of the split-plot
experiment involving the water resistant property of experiment: the whole-plot (WP) level and the sub-
wood. Two types of wood pretreatment (one and plot level.
two) and four types of stain (one, two, three and
four) have been selected as variables of interest. Whole-Plot Level Only
A graphical representation of the experiment is Again, suppose the experiment is carried out
shown in Figure 2 on p. 68 (for each pretreatment using three replicates of the pretreatment factor.
the stains have been randomly assigned to the four This involves six boards (three for pretreatment
panels). Table 1 (p. 69) gives the design as it was number one and three for pretreatment number
carried out: First a randomly selected pretreatment two). For now, let’s focus on only these six boards
is applied, then the wood is cut into four panels (before they are cut and the stains are applied) and
and the stains are applied in random order. break down the degrees of freedom (df).
The null hypothesis for all factors is H0: There is Because these six boards are randomly assigned
no effect due to the factor. A test statistic is neces- a pretreatment level, this part of the experiment is
sary to test this hypothesis. In this paper, the test essentially a completely randomized design with
statistics are all F-statistics, which are the ratio of one 2-level factor (pretreatment) and three repli-
the mean square (MS) for the factor of interest to cates. Therefore, there is 6 – 1 = 5 total df for this
the correct mean square error whole-plot level of the experiment.
Because the only factor has two levels, pretreat-
ment has 1 df. This leaves 4 df for the error term at
MS Factor the whole-plot level. Notice how thinking of the
F =
MS CorrectError experiment in this manner clearly shows the pre-
treatment variable has its own error term, “whole-
Once the F-statistic has been calculated, a p- plot error.” The split-plot design simply exploits
value can be computed and used to test the null the fact that each of the six pretreated boards can
hypothesis (we typically reject H0 if the p-value < be cut into four pieces and another factor (stain)
0.05). The p-value is the probability the test statistic can also be studied.
will take on a value at least as extreme as the ob- Once all the data are collected, we could write
served value of the statistic, assuming the null the model as:
hypothesis is true. Average response =
It is sometimes easier to think of the analysis of a pretreatment factor + WP error

70 I DECEMBER 2004 I www.asq.org

in which average response is the mean of the four Correct Split-Plot Analysis
different stain responses in each whole plot, and The split-plot model is:
WP error is the error term for the whole-plot factor Response = pretreatment + WP error + stain +
(pretreatment). pretreatment x stain interaction + SP error
The whole-plot experimental error is estimated in which SP error is the error for the subplot factor
by examining the variability that occurs between (stain) and the whole-plot by subplot interaction
the three whole plots within each of the two pre- (pretreatment x stain). To get the correct analysis of
treatment settings. Using these six averages will variance table with all sources of variation includ-
yield the correct F-test for pretreatment (pretreat- ing the two error terms involves removing the sums
ment is not significant with p = 0.115, as shown in of squares and df for the whole-plot error from the
Table 2, p. 69). However, the sums of squares will reported error term in the incorrect completely ran-
not be the same as the correct overall split-plot domized analysis. This can be done manually, but
analysis (they will be off by a factor of 4 = the num- then all F-tests and p-values will have to be gener-
ber of subplots in each whole-plot). ated manually as well. Fortunately, many software
packages can be tricked to do this for you automati-
Incorrect Completely Randomized
cally by using a nested model:
Design Analysis
Response = pretreatment + WP (pretreatment) +
If the 24 pieces involving the four stains are incor- stain + pretreatment x stain interaction + SP error,
rectly viewed as their own completely randomized
experiment, then there would be 24 – 1 = 23 total df.
This would involve 2 - 1= 1 df for pretreatment, 4 – 1
= 3 df for stain and (2 - 1)(4 - 1) = 3 df for the pre-
treatment by stain interaction. Therefore, there The limitations and
would be 23 – 7 = 16 df for error. The incorrect com-
pletely randomized model is: challenges of experimenting
Response = pretreatment + stain + pretreatment
x stain interaction + error. in the real world result in
Notice, however, this analysis is incorrect
because it does not remove the sums of squares and these simple experiments
4 df for whole-plot error discussed above (this is
viewing the experiment only at the subplot level). being the exception rather
The error term in this model is the sum of the
whole-plot error and the subplot error. than the norm.
When the whole-plot error is not removed from
the completely randomized analysis, the error
term used for testing the subplot factors is inflat- in which WP is a variable that goes from one to six
ed. Therefore, the F-test for all terms in the model indicating each whole-plot and must be declared as
would use the wrong error term. This can result a random factor.
in F-tests that are insignificant for some subplot Specifying the model in this way allows the cre-
factors while overstating significance for the ation of two separate estimates of experimental
whole-plot factor. error, an ingredient of the split-plot design. The
Table 3 shows the analysis. Notice the pretreat- nested term WP (pretreatment) comes from the fact
ment factor is incorrectly identified as significant the whole plots are nested within pretreatment.2
(p = 0.002), while the stain factor is insignificant (p This term will be the correct error term for the
= 0.245). We have seen in the earlier whole-plot pretreatment factor, and most software packages
analysis that pretreatment is not significant, and will correctly use this term for the F-test of pre-
we will see later in the correct split-plot analysis treatment. The df will also be correctly calculated
that stain is significant. as 2(3 - 1) = 4 in which 2 represents the number of

QUALITY PROGRESS I DECEMBER 2004 I 71

DESIGN OF EXPERIMENTS

levels for the pretreatment factor and 3 represents For example, the previous 3 df for stain can now
the number of replicates at the pretreatment level be broken down into 1 df for stain type, 1 df for
of the experiment. amount and 1 df for the stain type by amount inter-
The other estimate of experimental error, called action. This is also true for the previous pretreat-
the subplot error, is estimated by examining the vari- ment by stain interaction, which is now 1 df for
ation that occurs between the 12 pairs of experimen- pretreatment by stain type interaction, 1 df for pre-
tal runs that have the same pretreatment and stain treatment by amount interaction and 1 df for the
setting minus the whole-plot experimental error. pretreatment by stain type by amount interaction.
The whole-plot experimental error is used to test
Another Example
the significance of the whole-plot factor, pretreat-
ment. The subplot experimental error is used to Consider another example with one hard-to-
test the significance of the subplot factor, stain and change factor (Z), three easy-to-change factors (A, B,
pretreatment by stain interaction. Therefore, the C) and all factors at two levels. The hard-to-change
tests use a different mean square error in the de- factor is replicated so there are four whole plots,
nominator of the F-ratio. each with eight subplots.
Table 4 (p. 70) shows the F-statistic for the effect Table 5 gives the design as it was carried out:
of pretreatment, the whole-plot factor, is: First a level for Z is randomly selected, then the
eight combinations of A, B and C are carried out in
Mean squarepretreatment random order. The correct and incorrect analyses
782.04
F = = = 4.03. are shown in Table 6. Notice the incorrect analysis
Mean square 193.84
WP (pretreatment) indicates Z is significant, while the Z x A and A x B
interactions are shown as not significant.
Note the p-value of 0.115 indicates this factor is
not significant. The F-test for the effect of stain, the Extensions on the Split-Plot
subplot factor, is: An astute reader can probably now surmise the
split-plot framework can be expanded to even
Mean squarestain 88.67 more complicated experiments. Several extensions
F = = = 6.98. that can be made to the split-plot scenario are:
Mean squareerror 12.71
• It can have more than one hard-to-change fac-
tor. (Make sure the extra factor(s) is really hard
to change and not just inconvenient to change.)
Note the p-value of 0.006 indicates this factor is
• The whole-plot level design may involve blocks
significant. The F-test for the effect of the pretreat-
instead of being completely randomized.
ment by stain interaction is:
• There may be several easy-to-change factors,
which may necessitate using a fractional facto-
Mean squarepretreat x stain 20.93
F = = = 1.65. rial design at the subplot level (you must be
Mean squareerror 12.71 very careful because the alias structure is much
more complicated in split-plot designs).
• More factors could be added that are subplots
Note the p-value of 0.231 indicates the interac- for one factor while at the same time whole
tion effect is not significant. Notice for both pre- plots for other factors. This results in a split-
treatment and stain, these are different conclusions split-plot design.3
from the analysis assuming a completely random- The design and analysis of industrial experi-
ized design. ments involves understanding not only the treat-
Many experiments in industry involve two-level ment structure but also the three principles of the
factors. In the wood experiment, the four stains design structure: randomization, replication and
could actually be a 22 in stain type and amount. All controlling for known sources of variation (typical-
this does is add a little more structure to the experi- ly through blocking).
ment and the breakdown of the degrees of freedom. The experimenter should be made aware of an

72 I DECEMBER 2004 I www.asq.org

 TABLE 5 Data for the Second Example TABLE 6 Summary for Second Example

Z A B C WP Response Correct split-plot analysis

1 -1 1 1 1 108.4 Source DF SS MS F P
1 1 -1 1 1 131.6
Z 1 59.13 59.13 2.94 0.228
1 -1 -1 -1 1 124.0 WP (Z) 2 40.17 20.08 6.83 *
1 1 -1 -1 1 134.9 A 1 597.72 597.72 203.13 0.000
1 -1 1 -1 1 103.7 B 1 1226.36 1226.36 416.77 0.000
1 1 1 -1 1 112.9 C 1 1.49 1.49 0.51 0.486
1 1 1 1 1 113.4 ZxA 1 14.72 14.72 5.00 0.038
1 -1 -1 1 1 122.3 ZxB 1 285.01 285.01 96.86 0.000
ZxC 1 3.71 3.71 1.26 0.275
-1 -1 -1 -1 3 119.3
AxB 1 13.13 13.13 4.46 0.048
-1 1 1 -1 3 120.9
AxC 1 0.81 0.81 0.28 0.605
-1 1 1 1 3 123.0 BxC 1 1.16 1.16 0.40 0.537
-1 1 -1 1 3 127.9 Error 19 55.91 2.94
-1 -1 1 1 3 117.3 Total 31 2299.32
-1 -1 -1 1 3 120.9
-1 1 -1 -1 3 129.9 Incorrect completely randomized analysis
-1 -1 1 -1 3 115.4 Source DF SS MS F P
1 -1 1 1 2 100.8
Z 1 59.13 59.13 12.92 0.002
1 1 1 -1 2 114.4
A 1 597.72 597.72 130.65 0.000
1 1 -1 1 2 132.8 B 1 1226.36 1226.36 268.05 0.000
1 1 -1 -1 2 131.4 C 1 1.49 1.49 0.33 0.575
1 -1 -1 -1 2 118.4 ZxA 1 14.72 14.72 3.22 0.087
1 -1 1 -1 2 104.4 ZxB 1 285.01 285.01 62.30 0.000
1 1 1 1 2 111.7 ZxC 1 3.71 3.71 0.81 0.378
AxB 1 13.13 13.13 2.87 0.105
1 -1 -1 1 2 121.1
AxC 1 0.81 0.81 0.18 0.678
-1 1 1 -1 4 116.7
BxC 1 1.16 1.16 0.25 0.619
-1 -1 1 -1 4 112.8 Error 21 96.08 4.58
-1 -1 1 1 4 112.2 Total 31 2299.32
-1 1 -1 1 4 127.7
-1 -1 -1 -1 4 118.4
Z = hard-to-change factors
-1 1 1 1 4 120.9 A, B and C = easy-to-change factors
-1 1 -1 -1 4 127.0 WP = whole-plot errors
-1 -1 -1 1 4 119.4 DF = degrees of freedom
SS = sums of squares
Z = hard-to-change factors MS = mean square
A, B and C = easy-to-change factors F = F-statistic
WP = whole-plot errors P = p-value

important point about the experimental replication Getting Beyond Academics

in a split-plot design. The effect of the whole-plot Many practitioners of experimentation are
factor, which will have the least number of experi- beginning to incorporate the principles and
mental replicates, is estimated less precisely than methodology of designed experiments developed
the subplot factors, which will have more experi- in the statistical literature over the last 75 years.
mental replicates. Thus, if allowed a choice when The first experiments learned in typical statistical
planning a split-plot experiment, the experimenter and quality methodology training courses are
should try to put the most important factors at the those with simple design structures, such as the
subplot level. completely randomized design.

QUALITY PROGRESS I DECEMBER 2004 I 73

DESIGN OF EXPERIMENTS

In practice, however, the limitations and chal- some form of a restriction on the randomization. We
lenges of experimenting in the real world result in fear that more often than not, these features are not
these simple experiments being the exception rather being incorporated into the planning and analysis of
than the norm. Typically, an experiment will contain the experiment.
With the recent growth and
interest in the use of the statistical
sciences in today’s businesses,
however, we expect the sophistica-
tion and understanding of experi-
mentation will increase, and
designs such as the split plot will
become more readily recognized
and properly analyzed.

REFERENCES

1. S.M. Kowalski and K.J. Potcner,

“How To Recognize a Split-Plot Experi-
ment,” Quality Progress, Vol. 36, No. 11,
pp. 60-66.
2. D.C. Montgomery, Design and Analy-
sis of Experiments, fifth edition, Wiley, 2001.
3. Ibid.

BIBLIOGRAPHY

G.E.P. Box, “Split-Plot Experiments,”

Quality Engineering, Vol. 8, pp. 515-520.

KEVIN J. POTCNER is a technical trainer

at Minitab Inc. in State College, PA. He
earned a master’s degree in statistics from
the Rochester Institute of Technology in
New York and is a member of ASQ.

SCOTT M. KOWALSKI is a technical

trainer at Minitab Inc. in State College,
PA. He earned a doctorate in statistics
from the University of Florida, Gaines-
ville, and is a member of ASQ.

Please
comment
If you would like to comment
on this article, please post your
remarks on the Quality Progress
Discussion Board at www.asq.org,
or e-mail them to editor@asq.org.

74 I DECEMBER 2004 I www.asq.org