Chapter

6
Training Evaluation

6-1
 Training effectiveness refers to the benefits that the
company and the trainees receive from training.
 Training outcomes or criteria refer to measures that
the trainer and the company use to evaluate training
programs.
 Training evaluation refers to the process of collecting
the outcomes needed to determine if training is
effective.
 Evaluation design refers to from whom, what, when,
and how information needed for determining the
effectiveness of the training program will be collected.

6-2
 Companies are investing millions of dollars
in training programs to help gain a
competitive advantage.
 Training investment is increasing because
learning creates knowledge which
differentiates between those companies
and employees who are successful and
those who are not.
6-3
Because companies have made large dollar
investments in training and education and view
training as a strategy to be successful, they
expect the outcomes or benefits related to
training to be measurable.

6-4
 Formative evaluation – evaluation
conducted to improve the training process.

 Summative evaluation – evaluation

conducted to determine the extent to
which trainees have changed as a result of
participating in the training program.

6-5
 To identify the program’s strengths and
weaknesses.
 To assess whether content, organization,
and administration of the program
contribute to learning and the use of
training content on the job.
 To identify which trainees benefited most
or least from the program.
6-6
 To gather data to assist in marketing
training programs.
 To determine the financial benefits and
costs of the programs.
 To compare the costs and benefits of
training versus non-training investments.
 To compare the costs and benefits of
different training programs to choose the
best program.
6-7
 Conduct a Needs Analysis

Develop Measurable Learning Outcomes

Develop Outcome Measures

Choose an Evaluation Strategy

Plan and Execute the Evaluation

6-8
 6-9

Level Criteria Focus

1 Reactions Trainee satisfaction

2 Learning Acquisition of knowledge, skills, attitudes, behavior

3 Behavior Improvement of behavior on the job

4 Results Business results achieved by trainees

Cognitive
Outcomes Skill-Based
Outcomes

Affective Outcomes

Results
Return on
Investment
6 - 10
 Cognitive Outcomes
 Determine the degree to which trainees are
familiar with the principles, facts, techniques,
procedures, or processes emphasized in the
training program.
 Measure what knowledge trainees learned in the
program.
 Skill-Based Outcomes
 Assess the level of technical or motor skills.
 Include acquisition or learning of skills and use of
skills on the job.
6 - 11
 Affective Outcomes
 Include attitudes and motivation.
 Trainees’ perceptions of the program including
the facilities, trainers, and content.
 Results
 Determine the training program’s payoff for the
company.

6 - 12
 Return on Investment (ROI)
 Comparing the training’s monetary benefits with
the cost of the training.
▪ Direct costs
▪ Indirect costs
▪ Benefits

6 - 13
Good training outcomes need to be:
 Relevant
 Reliable
 Discriminate
 Practical

6 - 14
 Criteria relevance – the extent to which
training programs are related to learned
capabilities emphasized in the training
program.
 Criterion contamination – extent that training
outcomes measure inappropriate capabilities
or are affected by extraneous conditions.
 Criterion deficiency – failure to measure
training outcomes that were emphasized in
the training objectives.
6 - 15
 Outcomes Identified
Outcomes
by Needs
Measured in
Assessment and
Evaluation
Included in Training
Objectives

Contamination Relevance Deficiency

Outcomes Related to Training Objectives

6 - 16
 Reliability – degree to which outcomes can
be measured consistently over time.
 Discrimination – degree to which trainee’s
performances on the outcome actually
reflect true differences in performance.
 Practicality – refers to the ease with which
the outcomes measures can be collected.

6 - 17
 Threats to validity refer to a factor that will
lead one to question either:
 The believability of the study results (internal
validity),
validity) or
 The extent to which the evaluation results are
generalizable to other groups of trainees and
situations (external validity)

6 - 18
 Pre- and Posttests

Use of Comparison Groups

Random Assignment

6 - 19
 Posttest – only  Time series

 Pretest / posttest  Time series with

Comparison group and
 Posttest – only with Reversal
Comparison group
 Solomon Four – group
 Pretest / posttest with
Comparison group

6 - 20
 6 - 21

Factor How Factor Influences Type of Evaluation Design

Change potential Can program be modified?

Importance Does ineffective training affect customer service, product development, or

relationships between employees?
Scale How many trainees are involved?
Purpose of training Is training conducted for learning, results, or both?
Organization culture Is demonstrating results part of company norms and expectations?
Expertise Can a complex study be analyzed?
Cost Is evaluation too expensive?
Time frame When do we need the information?
 To understand total expenditures for training,
including direct and indirect costs.
 To compare costs of alternative training
programs.
 To evaluate the proportion of money spent on
training development, administration, and
evaluation as well as to compare monies spent
on training for different groups of employees.
 To control costs.

6 - 22
1. Identify outcome(s) (e.g., quality, accidents)
2. Place a value on the outcome(s)
3. Determine the change in performance after
eliminating other potential influences on
training results.
4. Obtain an annual amount of benefits
(operational results) from training by
comparing results after training to results
before training (in dollars)
6 - 23
5. Determine training costs (direct costs +
indirect costs + development costs +
overhead costs + compensation for trainees)
6. Calculate the total savings by subtracting the
training costs from benefits (operational
results)
7. Calculate the ROI by dividing benefits
(operational results) by costs.
 The ROI gives you an estimate of the dollar
return expected from each dollar invested in
training.
6 - 24