Douglas A.

Wiegmann
University of Illinois at Urbana-Champaign
Institute of Aviation
Savoy, IL 61874
Scott A. Shappell
FAA Civil Aeromedical Institute
P.O. Box 25082
Oklahoma City, OK 73125
February 2001
Final Report
This document is available to the public
through the National Technical Information
Service, Springfield, Virginia 22161.
Office of Aviation Medicine
Washington, D.C. 20591
A Human Error Analysis of
Commercial Aviation Accidents
Using the Human Factors
Analysis and Classification
System (HFACS)
U.S. Department
of Transpor tati on
Federal Avi ati on
Admi ni strati on
DOT/FAA/AM-01/3
NO NO NO NO NOTICE TICE TICE TICE TICE
This document is disseminated under the sponsorship of
the U.S. Department of Transportation in the interest of
information exchange. The United States Government
assumes no liability for the contents thereof.
Technical Report Documentation Page
1. Report No. 2. Government Accession No. 3. Recipient's Catalog No.
DOT/FAA/AM-01/3
4. Title and Subtitle 5. Report Date
A Human Error Analysis of Commercial Aviation Accidents February 2001
Using the Human Factors Analysis and Classification System (HFACS)
6. Performing Organization Code
7. Author(s) 8. Performing Organization Report No.
Wiegmann, D.A.
1
, and Shappell, S.A.
2
9. Performing Organization Name and Address 10. Work Unit No. (TRAIS)
11. Contract or Grant No.
1
University of Illinois at Urbana-Champaign, Institute of Aviation,
Savoy, IL 61874
2
FAA Civil Aeromedical Institute, P.O. Box 25082, Oklahoma City, OK 73125
99-G-006
12. Sponsoring Agency name and Address 13. Type of Report and Period Covered
Office of Aviation Medicine
Federal Aviation Administration
14. Sponsoring Agency Code
800 Independence Ave., S.W.
Washington, DC 20591
15. Supplemental Notes
Work was accomplished under task # AAM-A-00-HRR-520.
16. Abstract
The Human Factors Analysis and Classification System (HFACS) is a general human error framework
originally developed and tested within the U.S. military as a tool for investigating and analyzing the human
causes of aviation accidents. Based upon Reasons (1990) model of latent and active failures, HFACS
addresses human error at all levels of the system, including the condition of aircrew and organizational
factors. The purpose of the present study was to assess the utility of the HFACS framework as an error
analysis and classification tool outside the military. Specifically, HFACS was applied to commercial aviation
accident records maintained by the National Transportation Safety Board (NTSB). Using accidents that
occurred between January 1990 and December 1996, it was demonstrated that HFACS reliably
accommodated all human causal factors associated with the commercial accidents examined. In addition, the
classification of data using HFACS highlighted several critical safety issues in need of intervention research.
These results demonstrate that the HFACS framework can be a viable tool for use within the civil aviation
arena.
17. Key Words 18. Distribution Statement
Aviation, Human Error, Accident Investigation,
Database Analysis, Commercial Aviation
Document is available to the public through the
National Technical Information Service,
Springfield, Virginia 22161
19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price
Unclassified Unclassified 17
Form DOT F 1700.7 (8-72) Reproduction of completed page authorized
i
ACKNOWLEDGMENTS
The authors thank Frank Cristina and Anthony Pape for their assistance in gathering,
organizing and analyzing the accident reports used in this study.
iii
1
A HUMAN ERROR ANALYSIS OF COMMERCIAL AVIATION ACCIDENTS USING THE
HUMAN FACTORS ANALYSIS AND CLASSIFICATION SYSTEM (HFACS)
INTRODUCTION
Humans, by their very nature, make mistakes; there-
fore, it should come as no surprise that human error has
been implicated in a variety of occupational accidents,
including 70% to 80% of those in civil and military
aviation (OHare, Wiggins, Batt, & Morrison, 1994;
Wiegmann and Shappell, 1999; Yacavone, 1993). In
fact, while the number of aviation accidents attributable
solely to mechanical failure has decreased markedly over
the past 40 years, those attributable at least in part to
human error have declined at a much slower rate (Shappell
& Wiegmann, 1996). Given such findings, it would
appear that interventions aimed at reducing the occur-
rence or consequences of human error have not been as
effective as those directed at mechanical failures. Clearly,
if accidents are to be reduced further, more emphasis
must be placed on the genesis of human error as it relates
to accident causation.
The prevailing means of investigating human error in
aviation accidents remains the analysis of accident and
incident data. Unfortunately, most accident reporting
systems are not designed around any theoretical frame-
work of human error. Indeed, most accident reporting
systems are designed and employed by engineers and
front-line operators with only limited backgrounds in
human factors. As a result, these systems have been useful
for identifying engineering and mechanical failures but
are relatively ineffective and narrow in scope where
human error exists. Even when human factors are ad-
dressed, the terms and variables used are often ill-defined
and archival databases are poorly organized. The end
results are post-accident databases that typically are not
conducive to a traditional human error analysis, making
the identification of intervention strategies onerous
(Wiegmann & Shappell, 1997).
The Accident Investigation Process
To further illustrate this point, let us examine the
accident investigation and intervention process sepa-
rately for the mechanical and human components of an
accident. Consider first the occurrence of an aircraft
system or mechanical failure that results in an accident or
injury (Figure 1). A subsequent investigation takes place
that includes the examination of objective and quantifi-
able information, such as that derived from the wreckage
and flight data recorder, as well as that from the applica-
tion of sophisticated analytical techniques like metallur-
gical tests and computer modeling. This kind of
information is then used to determine the probable
mechanical cause(s) of the accident and to identify safety
recommendations.
Upon completion of the investigation, this objec-
tive information is typically entered into a highly-
structured and well-defined accident database. These
data can then be periodically analyzed to determine
system-wide safety issues and provide feedback to inves-
tigators, thereby improving investigative methods and
techniques. In addition, the data are often used to guide
organizations (e.g., the Federal Aviation Administration
[FAA], National Aeronautics and Space Administration
[NASA], Department of Defense [DoD], airplane manu-
facturers and airlines) in deciding which research or
safety programs to sponsor. As a result, these needs-
based, data-driven programs, in turn, have typically
produced effective intervention strategies that either
prevent mechanical failures from occurring altogether,
or mitigate their consequences when they do happen. In
either case, there has been a substantial reduction in the
rate of accidents due to mechanical or systems failures.
In stark contrast, Figure 2 illustrates the current
human factors accident investigation and prevention
process. This example begins with the occurrence of an
aircrew error during flight operations that leads to an
accident or incident. A human performance investiga-
tion then ensues to determine the nature and causes of
such errors. However, unlike the tangible and quantifi-
able evidence surrounding mechanical failures, the evi-
dence and causes of human error are generally qualitative
and elusive. Furthermore, human factors investigative
and analytical techniques are often less refined and
sophisticated than those used to analyze mechanical and
engineering concerns. As such, the determination of
human factors causal to the accident is a tenuous practice
at best; all of which makes the information entered in the
accident database sparse and ill-defined.
2
As a result, when traditional data analyses are per-
formed to determine common human factors problems
across accidents, the interpretation of the findings and
the subsequent identification of important safety issues
are of limited practical use. To make matters worse,
results from these analyses provide limited feedback to
investigators and are of limited use to airlines and govern-
ment agencies in determining the types of research or
safety programs to sponsor. As such, many research
programs tend to be intuitively-, or fad-driven, rather
than data-driven, and typically produce intervention
strategies that are only marginally effective at reducing
the occurrence and consequence of human error. The
overall rate of human-error related accidents, therefore,
has remained relatively high and constant over the last
several years (Shappell & Wiegmann, 1996).
Addressing the Problem
If the FAA and the aviation industry are to achieve
their goal of significantly reducing the aviation accident
rate over the next ten years, the primary causes of aviation
accidents (i.e., human factors) must be addressed (ICAO,
1993). However, as illustrated in Figure 2, simply
increasing the amount of money and resources spent on
human factors research is not the solution. Indeed, a
great deal of resources and efforts are currently being
expended. Rather, the solution is to redirect safety efforts
so that they address important human factors issues.
However, this assumes that we know what the important
human factors issues are. Therefore, before research
efforts can be systematically refocused, a comprehensive
analysis of existing databases needs to be conducted to
determine those specific human factors responsible for
aviation accidents and incidents. Furthermore, if these
efforts are to be sustained, new investigative methods and
techniques will need to be developed so that data gath-
ered during human factors accident investigations can be
improved and analysis of the underlying causes of human
error facilitated.
To accomplish this improvement, a general human
error framework is needed around which new investiga-
tive methods can be designed and existing postaccident
databases restructured. Previous attempts to do this have
met with encouraging, yet limited, success (OHare, et
al., 1994; Wiegmann & Shappell, 1997). This is prima-
rily because performance failures are influenced by a
Figure 1. General process of investigating and preventing aviation accidents involving mechanical or
systems failures.
Feedback
Mechanical
Failure
- Catastrophic failures
are infrequent
events
- When failures do
occur, they are often
less severe or
hazardous due to
effective
intervention
programs.
Data-Driven
Research
Research Sponsors
- FAA, DoD, NASA, & airplane
manufacturers provide
research funding.
- Research programs are needs-
based and data-driven.
Interventions are therefore
very effective.
Accident
Investigation
- Highly sophisticated
techniques and
procedures
- Information is
objective and
quantifiable
- Effective at
determining why the
failure occurred
Accident
Database
- Designed around
traditional
categories
- Variables are well-
defined and
causally related
- Organization and
structure facilitate
access and use
Database
Analysis
- Traditional
analyses are
clearly outlined
and readily
performed.
- Frequent analyses
help identify
common
mechanical and
engineering
safety issues.
M
i
t
i
g
a
t
i
o
n
P
r
e
v
e
n
t
i
o
n
Effective
Intervention
and Prevention
Programs
3
variety of human factors that are typically not addressed
by traditional error frameworks. For instance, with few
exceptions (e.g., Rasmussen, 1982), human error tax-
onomies do not consider the potential adverse mental
and physiological condition of the individual (e.g., fa-
tigue, illness, attitudes) when describing errors in the
cockpit. Likewise, latent errors committed by officials
within the management hierarchy such as line managers
and supervisors are often not addressed, even though it is
well known that these factors directly influence the
condition and decisions of pilots (Reason, 1990). There-
fore, if a comprehensive analysis of human error is to be
conducted, a taxonomy that takes into account the
multiple causes of human failure must be offered.
Recently, the Human Factors Analysis and Classifica-
tion System (HFACS) was developed to meet these needs
(Shappell & Wiegmann, 1997a, 2000a, and in press).
This system, which is based on Reasons (1990) model of
latent and active failures, was originally developed for the
U.S. Navy and Marine Corps as an accident investigation
and data analysis tool. Since its original development,
however, HFACS has been employed by other military
organizations (e.g., U.S. Army, Air Force, and Canadian
Defense Force) as an adjunct to preexisting accident
investigation and analysis systems. To date, the HFACS
framework has been applied to more than 1,000 military
aviation accidents, yielding objective, data-driven inter-
vention strategies while enhancing both the quantity and
quality of human factors information gathered during
accident investigations (Shappell & Wiegmann, in press).
Other organizations such as the FAA and NASA have
explored the use of HFACS as a complement to preexist-
ing systems within civil aviation in an attempt to capital-
ize on gains realized by the military (Ford, Jack, Crisp, &
Sandusky, 1999). Still, few systematic efforts have exam-
ined whether HFACS is indeed a viable tool within the
civil aviation arena, even though it can be argued that the
similarities between military and civilian aviation out-
weigh their differences. The purpose of the present study
was to empirically address this issue by applying the
HFACS framework, as originally designed for the mili-
tary, to the classification and analysis of civil aviation
accident data. Before beginning, however, a brief over-
view of the HFACS system will be presented for those
Feedback
Human
Error
- Errors occur
frequently and are
the major cause of
accidents.
- Few safety programs
are effective at
preventing the
occurrence or
consequences of
these errors.
Research Sponsors
- FAA, DoD, NASA, & Airlines
provide funding for safety
research programs.
- Lack of good data leads to
research programs based
primarily on interests and
intuitions. Interventions are
therefore less effective.
Fad-Driven
Research
M
i
t
i
g
a
t
i
o
n
P
r
e
v
e
n
t
i
o
n
Ineffective
Intervention
and Prevention
Programs
Accident
Investigation
- Less sophisticated
techniques and
procedures
- Information is
qualitative and
illusive
- Focus on what
happened but not
why it happened
Accident
Database
- Not designed
around any
particular human
error framework
- Variables often ill-
defined
- Organization and
structure difficult
to understand
Database
Analysis
- Traditional human
factors analyses
are onerous due
to ill-defined
variables and
database
structures.
- Few analyses have
been performed
to identify
underlying
human factors
safety issues.
Figure 2. General process of investigating and preventing aviation accidents involving human error.
4
readers who may not be familiar with the framework (for
a detailed description of HFACS, see Shappell and
Wiegmann, 2000a and 2001).
HFACS
Drawing upon Reasons (1990) concept of latent and
active failures, HFACS describes human error at each of
four levels of failure: 1) unsafe acts of operators (e.g.,
aircrew), 2) preconditions for unsafe acts, 3) unsafe
supervision, and 4) organizational influences. A brief
description of each causal category follows (Figure 3).
Unsafe Acts of Operators
The unsafe acts of operators (aircrew) can be loosely
classified into one of two categories: errors and violations
(Reason, 1990). While both are common within most
settings, they differ markedly when the rules and regula-
tion of an organization are considered. That is, errors can
be described as those legal activities that fail to achieve
their intended outcome, while violations are commonly
defined as behavior that represents the willful disregard
for the rules and regulations. It is within these two
overarching categories that HFACS describes three types
of errors (decision, skill-based, and perceptual) and two
types of violations (routine and exceptional).
Errors
One of the more common error forms, decision errors,
represents conscious, goal-intended behavior that pro-
ceeds as designed; yet, the plan proves inadequate or
inappropriate for the situation. Often referred to as
honest mistakes, these unsafe acts typically manifest as
poorly executed procedures, improper choices, or simply
the misinterpretation or misuse of relevant information.
In contrast to decision errors, the second error form,
skill-based errors, occurs with little or no conscious thought.
Just as little thought goes into turning ones steering
wheel or shifting gears in an automobile, basic flight
Figure 3. Overview of the Human Factors Analysis and Classification System (HFACS).
Errors
Perceptual
Errors
Skill-Based
Errors
UNSAFE
ACTS
Errors
Decision
Errors
Exceptional Routine
Violations
Inadequate
Supervision
Planned
Inappropriate
Operations
Failed to
Correct
Problem
Supervisory
Violations
UNSAFE
SUPERVISION
Substandard
Conditions of
Operators
PRECONDITIONS
FOR
UNSAFE ACTS
Substandard
Conditions of
Operators
PRECONDITIONS
FOR
UNSAFE ACTS
Adverse
Physiological States
Physical/
Mental
Limitations
Adverse Mental
States
Adverse Mental
States
Personal
Readiness
Crew Resource
Mismanagement
Substandard
Practices of
Operators
Resource
Management
Organizational
Climate
Organizational
Process
ORGANIZATIONAL
INFLUENCES
5
skills such as stick and rudder movements and visual
scanning often occur without thinking. The difficulty
with these highly practiced and seemingly automatic
behaviors is that they are particularly susceptible to
attention and/or memory failures. As a result, skill-based
errors such as the breakdown in visual scan patterns,
inadvertent activation/deactivation of switches, forgot-
ten intentions, and omitted items in checklists often
appear. Even the manner (or skill) with which one flies
an aircraft (aggressive, tentative, or controlled) can
affect safety.
While, decision and skill-based errors have domi-
nated most accident databases and therefore, have been
included in most error frameworks, the third and final
error form, perceptual errors, has received comparatively
less attention. No less important, perceptual errors occur
when sensory input is degraded, or unusual, as is often
the case when flying at night, in the weather, or in other
visually impoverished environments. Faced with acting
on imperfect or less information, aircrew run the risk of
misjudging distances, altitude, and decent rates, as well
as a responding incorrectly to a variety of visual/vestibu-
lar illusions.
Violations
Although there are many ways to distinguish among
types of violations, two distinct forms have been identi-
fied based on their etiology. The first, routine violations,
tend to be habitual by nature and are often enabled by a
system of supervision and management that tolerates
such departures from the rules (Reason, 1990). Often
referred to as bending the rules, the classic example is
that of the individual who drives his/her automobile
consistently 5-10 mph faster than allowed by law. While
clearly against the law, the behavior is, in effect, sanc-
tioned by local authorities (police) who often will not
enforce the law until speeds in excess of 10 mph over the
posted limit are observed.
Exceptional violations, on the other hand, are isolated
departures from authority, neither typical of the indi-
vidual nor condoned by management. For example,
while driving 65 in a 55 mph zone might be condoned by
authorities, driving 105 mph in a 55 mph zone certainly
would not. It is important to note, that while most
exceptional violations are appalling, they are not consid-
ered exceptional because of their extreme nature. Rather,
they are regarded as exceptional because they are neither
typical of the individual nor condoned by authority.
Preconditions for Unsafe Acts
Simply focusing on unsafe acts, however, is like focus-
ing on a patients symptoms without understanding the
underlying disease state that caused it. As such, investi-
gators must dig deeper into the preconditions for unsafe
acts. Within HFACS, two major subdivisions are de-
scribed: substandard conditions of operators and the
substandard practices they commit.
Substandard Conditions of the Operator
Being prepared mentally is critical in nearly every
endeavor; perhaps it is even more so in aviation. With
this in mind, the first of three categories, adverse mental
states, was created to account for those mental conditions
that adversely affect performance. Principal among these
are the loss of situational awareness, mental fatigue,
circadian dysrhythmia, and pernicious attitudes such as
overconfidence, complacency, and misplaced motiva-
tion that negatively impact decisions and contribute to
unsafe acts.
Equally important, however, are those adverse physi-
ological states that preclude the safe conduct of flight.
Particularly important to aviation are conditions such as
spatial disorientation, visual illusions, hypoxia, illness,
intoxication, and a whole host of pharmacological and
medical abnormalities known to affect performance. For
example, it is not surprising that, when aircrews become
spatially disoriented and fail to rely on flight instrumen-
tation, accidents can, and often do, occur.
Physical and/or mental limitations of the operator, the
third and final category of substandard condition, in-
cludes those instances when necessary sensory informa-
tion is either unavailable, or if available, individuals
simply do not have the aptitude, skill, or time to safely
deal with it. For aviation, the former often includes not
seeing other aircraft or obstacles due to the size and/or
contrast of the object in the visual field. However, there
are many times when a situation requires such rapid
mental processing or reaction time that the time allotted
to remedy the problem exceeds human limits (as is often
the case during nap-of-the-earth flight). Nevertheless,
even when favorable visual cues or an abundance of time
is available, there are instances when an individual simply
may not possess the necessary aptitude, physical ability,
or proficiency to operate safely.
6
Substandard Practices of the Operator
Often times, the substandard practices of aircrew will
lead to the conditions and unsafe acts described above.
For instance, the failure to ensure that all members of the
crew are acting in a coordinated manner can lead to
confusion (adverse mental state) and poor decisions in
the cockpit. Crew resource mismanagement, as it is re-
ferred to here, includes the failures of both inter- and
intra-cockpit communication, as well as communication
with ATC and other ground personnel. This category
also includes those instances when crewmembers do not
work together as a team, or when individuals directly
responsible for the conduct of operations fail to coordi-
nate activities before, during, and after a flight.
Equally important, however, individuals must ensure
that they are adequately prepared for flight. Conse-
quently, the category of personal readiness was created to
account for those instances when rules such as disregard-
ing crew rest requirements, violating alcohol restrictions,
or self-medicating, are not adhered to. However, even
behaviors that do not necessarily violate existing rules or
regulations (e.g., running ten miles before piloting an
aircraft or not observing good dietary practices) may
reduce the operating capabilities of the individual and
are, therefore, captured here.
Unsafe Supervision
Clearly, aircrews are responsible for their actions and,
as such, must be held accountable. However, in many
instances, they are the unwitting inheritors of latent
failures attributable to those who supervise them (Rea-
son, 1990). To account for these latent failures, the
overarching category of unsafe supervision was created
within which four categories (inadequate supervision,
planned inappropriate operations, failed to correct known
problems, and supervisory violations) are included.
The first category, inadequate supervision, refers to
failures within the supervisory chain of command, which
was a direct result of some supervisory action or inaction.
That is, at a minimum, supervisors must provide the
opportunity for individuals to succeed. It is expected,
therefore, that individuals will receive adequate training,
professional guidance, oversight, and operational leader-
ship, and that all will be managed appropriately. When
this is not the case, aircrews are often isolated, as the risk
associated with day-to-day operations invariably will
increase.
However, the risk associated with supervisory failures
can come in many forms. Occasionally, for example, the
operational tempo and/or schedule is planned such that
individuals are put at unacceptable risk and, ultimately,
performance is adversely affected. As such, the category
of planned inappropriate operations was created to ac-
count for all aspects of improper or inappropriate crew
scheduling and operational planning, which may focus
on such issues as crew pairing, crew rest, and managing
the risk associated with specific flights.
The remaining two categories of unsafe supervision,
the failure to correct known problems and supervisory
violations, are similar, yet considered separately within
HFACS. The failure to correct known problems refers to
those instances when deficiencies among individuals,
equipment, training, or other related safety areas are
known to the supervisor, yet are allowed to continue
uncorrected. For example, the failure to consistently
correct or discipline inappropriate behavior certainly
fosters an unsafe atmosphere but is not considered a
violation if no specific rules or regulations were broken.
Supervisory violations, on the other hand, are reserved
for those instances when existing rules and regulations
are willfully disregarded by supervisors when managing
assets. For instance, permitting aircrew to operate an
aircraft without current qualifications or license is a
flagrant violation that invariably sets the stage for the
tragic sequence of events that predictably follow.
Organizational Influences
Fallible decisions of upper-level management can
directly affect supervisory practices, as well as the condi-
tions and actions of operators. Unfortunately, these orga-
nizational influences often go unnoticed or unreported by
even the best-intentioned accident investigators.
Traditionally, these latent organizational failures gen-
erally revolve around three issues: 1) resource manage-
ment, 2) organizational climate, and 3) operational
processes. The first category, resource management, refers
to the management, allocation, and maintenance of
organizational resources, including human resource
management (selection, training, staffing), monetary
safety budgets, and equipment design (ergonomic speci-
fications). In general, corporate decisions about how
such resources should be managed center around two
distinct objectives the goal of safety and the goal of on-
time, cost-effective operations. In times of prosperity,
both objectives can be easily balanced and satisfied in
full. However, there may also be times of fiscal austerity
that demand some give and take between the two.
Unfortunately, history tells us that safety is often the loser in
such battles, as safety and training are often the first to be cut
in organizations experiencing financial difficulties.
7
Organizational climate refers to a broad class of orga-
nizational variables that influence worker performance
and is defined as the situationally based consistencies in
the organizations treatment of individuals (Jones, 1988).
One telltale sign of an organizations climate is its
structure, as reflected in the chain-of-command, delega-
tion of authority and responsibility, communication
channels, and formal accountability for actions. Just like
in the cockpit, communication and coordination are
vital within an organization. However, an organizations
policies and culture are also good indicators of its cli-
mate. Consequently, when policies are ill-defined,
adversarial, or conflicting, or when they are supplanted
by unofficial rules and values, confusion abounds, and
safety suffers within an organization.
Finally, operational process refers to formal processes
(operational tempo, time pressures, production quotas,
incentive systems, schedules, etc.), procedures (perfor-
mance standards, objectives, documentation, instruc-
tions about procedures, etc.), and oversight within the
organization (organizational self-study, risk manage-
ment, and the establishment and use of safety programs).
Poor upper-level management and decisions concerning
each of these organizational factors can also have a
negative, albeit indirect, effect on operator performance
and system safety.
Summary
The HFACS framework bridges the gap between
theory and practice by providing safety professionals
with a theoretically based tool for identifying and classi-
fying the human causes of aviation accidents. Because the
system focuses on both latent and active failures and their
interrelationships, it facilitates the identification of the
underlying causes of human error. To date, HFACS has
been shown to be useful within the context of military
aviation, as both a data analysis framework and an
accident investigation tool. However, HFACS has yet to
be applied systematically to the analysis and investigation
of civil aviation accidents. The purpose of the present
research project, therefore, was to assess the utility of the
HFACS framework as an error analysis and classification
tool within commercial aviation.
The specific objectives of this study were three-fold.
The first objective was to determine whether the HFACS
framework, in its current form, would be comprehensive
enough to accommodate all of the underlying human
causal-factors associated with commercial aviation acci-
dents, as contained in the accident databases maintained
by the FAA and NTSB. In other words, could the
framework capture all the relevant human error data or
would a portion of the database be lost because it was
unclassifiable? The second objective was to determine
whether the process of reclassifying the human causal
factors using HFACS was reliable. That is, would differ-
ent users of the system agree on how causal factors should
be coded using the framework? Finally, the third objec-
tive was to determine whether reclassifying the data using
HFACS yield a benefit beyond what is already known
about commercial aviation accident causation. Specifi-
cally, would HFACS highlight any heretofore unknown
safety issues in need of further intervention research?
METHOD
Data
A comprehensive review of all accidents involving
Code of Federal Air Regulations (FAR) Parts 121 and
135 Scheduled Air Carriers between January 1990 and
December 1996 was conducted using database records
maintained by the NTSB and the FAA. Of particular
interest to this study were those accidents attributable, at
least in part, to the aircrew. Consequently, not included
were accidents due solely to catastrophic failure, mainte-
nance error, and unavoidable weather conditions such as
turbulence and wind shear. Furthermore, only those
accidents in which the investigation was completed, and
the cause of the accident determined, were included in
this analysis. One hundred nineteen accidents met these
criteria, including 44 accidents involving FAR Part 121
operators and 75 accidents involving FAR Part 135
operators.
HFACS Classification
The 119 aircrew-related accidents yielded 319 causal
factors for further analyses. Each of these NTSB causal
factors was subsequently coded independently by both
an aviation psychologist and a commercially-rated pilot
using the HFACS framework. Only those causal factors
identified by the NTSB were analyzed. That is, no new
causal factors were created during the error-coding process.
RESULTS
HFACS Comprehensiveness
All 319 (100%) of the human causal factors associated
with aircrew-related accidents were accommodated us-
ing the HFACS framework. Instances of all but two
HFACS categories (i.e., organizational climate and
personal readiness) were observed as least once in the
8
accident database. Therefore, no new HFACS categories
were needed to capture the existing causal factors, and no
human factors data pertaining to the aircrew were left
unclassified during the coding process.
HFACS Reliability
Disagreements among raters were noted during the
coding process and ultimately resolved by discussion.
Using the record of agreement and disagreement be-
tween the raters, the reliability of the HFACS system was
assessed by calculating Cohens kappa an index of
agreement that has been corrected for chance. The ob-
tained kappa value was .71, which generally reflects a
good level of agreement according to criteria described
by Fleiss (1981).
HFACS Analyses
Unsafe Acts
Table 1 presents percentages of FAR Parts 121 and
135 aircrew-related accidents associated with each of the
HFACS categories. An examination of the table reveals
that at the unsafe acts level, skill-based errors were
associated with the largest percentage of accidents. Ap-
proximately 60% of all aircrew-related accidents were
associated with at least one skill-based error. This per-
centage was relatively similar for FAR Part 121 carriers
(63.6%) and FAR Part 135 carriers (58.7%). Figure 4,
panel A, illustrates that the proportion of accidents
associated with skill-based errors has remained relatively
unchanged over the seven-year period examined in the
study. Notably, however, the lowest proportion of acci-
dents associated with skill-based errors was observed in
the last two years of the study (1995 and 1996).
Among the remaining categories of unsafe acts, acci-
dents associated with decision errors constituted the next
highest proportion (i.e., roughly 29% of the accidents
examined, Table 1). Again, this percentage was roughly
equal across both FAR Part 121 (25.0%) and Part 135
(30.7%) accidents. With the exception of 1994, in which
the percentage of aircrew-related accidents associated
with decision errors reached a high of 60%, the propor-
tion of accidents associated with decision errors re-
mained relatively constant across the years of the study
(Figure 4, panel B).
Table 1. Percentage of Accidents Associated with each HFACS category.
HFACS Category FAR Part 121 FAR Part 135 Total
Organizational Influences
Resource Management
Organizational Climate
Organizational Process
4.5 (2)
0.0 (0)
15.9 (7)
1.3 (1)
0.0 (0)
4.0 (3)
2.5 (3)
0.0 (0)
8.4 (10)
Unsafe Supervision
Inadequate Supervision
Planned Inappropriate Operations
Failed to Correct Known Problem
Supervisory Violations
2.3 (1)
0.0 (0)
0.0 (0)
0.0 (0)
6.7 (5)
1.3 (1)
2.7 (2)
2.7 (2)
5.0 (6)
0.8 (1)
1.7 (2)
1.7 (2)
Preconditions of Unsafe Acts
Adverse Mental States
Adverse Physiological Sates
Physical/mental Limitations
Crew-resource Mismanagement
Personal Readiness
Unsafe Acts
Skill-based Errors
Decision Errors
Perceptual Errors
Violations
13.6 (6)
4.5 (2)
2.3 (1)
40.9 (18)
0.0 (0)
63.6 (28)
25.0 (11)
20.5 (9)
25.0 (11)
13.3 (10)
0.0 (0)
16.0 (12)
22.7 (17)
0.0 (0)
58.7 (44)
30.7 (23)
10.7 (8)
28.0 (21)
13.4 (16)
1.7 (2)
10.9 (13)
29.4 (35)
0.0 (0)
60.5 (72)
28.6 (34)
14.3 (17)
26.9 (32)
Note: Numbers in table are percentages of accidents that involved at least one instance of an HFACS category. Numbers in
parentheses indicate accident frequencies. Because more than one causal factor is generally associated with each accident,
the percentages in the table will not equal 100%.
9
Similar to accidents associated with decision errors,
those attributable at least in part to violations of rules and
regulations were associated with 26.9% of the accidents
examined. Again, no appreciable difference was evident
when comparing the relative percentages across FAR
Parts 121 (25.0%) and 135 (28.0%). However, an
examination of Figure 4, panel C, reveals that the relative
proportion of accidents associated with violations in-
creased appreciably from a low of 6% in 1990 to a high
of 46% in 1996.
Finally, the proportion of accidents associated with
perceptual errors was relatively low. In fact, only 17 of the
119 accidents (14.3%) involved some form of perceptual
error. While it appeared that the relative proportion of
Part 121 accidents associated with perceptual errors was
higher than Part 135 accidents, the low number of
occurrences precluded any meaningful comparisons across
either the type of operation or calendar year.
Preconditions for Unsafe Acts
Within the preconditions level, CRM failures were
associated with the largest percentage of accidents. Ap-
proximately 29% of all aircrew-related accidents were
associated with at least one CRM failure. A relatively
larger percentage of FAR Part 121 aircrew-accidents
involved CRM failures (40.9%) than did FAR Part 135
aircrew-related accidents (22.7%). However, the per-
centage of accidents associated with CRM failures re-
mained relatively constant over the seven-year period for
both FAR Part 121 and 135 carriers (Figure 4, panel d).
The next largest percentage of accidents was associ-
ated with adverse mental states (13.4%), followed by
physical/mental limitations (10.9%) and adverse physi-
ological states (1.7%). There were no accidents associ-
ated with personal readiness issues. The percentage of
accidents associated with physical/mental limitation was
higher for FAR Part 135 carriers (16%) compared with
FAR Part 121 carriers (2.3%), but accidents associated
with adverse mental or adverse physiological states were
relatively equal across carriers. Again, however, the low
number of occurrences in each of these accident cat-
egories precluded any meaningful comparisons across
calendar year.
Supervisory and Organizational Factors
Very few of the NTSB reports that implicated the
aircrew as contributing to an accident also cited some
form of supervisory or organizational failure (see Table
0
10
20
30
40
50
60
70
80
90 91 92 93 94 95 96
P
e
r
c
e
n
t
a
g
e
Year
A.
0
10
20
30
40
50
60
70
80
90 91 92 93 94 95 96
P
e
r
c
e
n
t
a
g
e
Year
D.
0
10
20
30
40
50
60
70
80
90 91 92 93 94 95 96
P
e
r
c
e
n
t
a
g
e
Year
C.
0
10
20
30
40
50
60
70
80
90 91 92 93 94 95 96
P
e
r
c
e
n
t
a
g
e
Year
B.
Figure 4. Percentage of aircrew related accidents associated with skill-based errors
(Panel A), decision errors (Panel B), violations (Panel C) and CRM failures (Panel D)
across calendar years. Lines represent seven year averages.
10
1). Indeed, only 16% of all aircrew-related accidents
involved some form of either supervisory or organiza-
tional involvement. Overall, however, a larger propor-
tion of aircrew-related accidents involving FAR Part 135
carriers involved supervisory failures (9.3%) than did
those accidents involving FAR Part 121 carriers (2.3%).
In contrast, a larger proportion of aircrew-related acci-
dents involving FAR Part 121 carriers involved organiza-
tional factors (20.5%) than did those accidents involving
FAR Part 135 carriers (4.0%).
DISCUSSION
HFACS Comprehensiveness
The HFACS framework was found to accommodate
all 319 causal factors associated with the 119 accidents
involving FAR Parts 121 and 135 scheduled carriers
across the seven-year period examined. This finding
suggests that the error categories within HFACS, origi-
nally developed for use in the military, are applicable
within commercial aviation as well. Still, some of the
error-factors within the HFACS framework were never
observed in this commercial aviation accident database.
For example, no instances of such factors as organiza-
tional climate or personal readiness were observed. In
fact, very few instances of supervisory factors were evi-
dent at all in the data.
One explanation for the scarcity of such factors could
be that, contrary to Reasons model of latent and active
failures upon which HFACS is based, such supervisory
and organizational factors simply do not play as large of
a role in the etiology of commercial aviation accidents as
once expected. Consequently, the HFACS framework
may need to be pared down or simplified for use with
commercial aviation. Another explanation, however, is
that these factors do contribute to most accidents, yet
they are rarely identified using existing accident investi-
gation processes. Nevertheless, the results of this study
indicate that the HFACS framework was able to capture
all existing causal factors and no new error-categories or
aircrew cause-factors were needed to analyze the com-
mercial accident data.
HFACS Reliability
The HFACS system was found to produce an accept-
able level of agreement among the investigators who
participated in this study. Furthermore, even after this
level of agreement between investigators was corrected
for chance, the obtained reliability index was considered
good by conventional standards. Still, this reliability
index was somewhat lower than those observed in studies
using military aviation accidents which, in some in-
stances, have resulted in nearly complete agreement
among investigators (Shappell & Wiegmann, 1997b).
One possible explanation for this discrepancy is the
difference in both the type and amount of information
available to investigators across these studies. Unlike the
present study, previous analysts using HFACS to analyze
military accident data often had access to privileged and
highly detailed information about the accidents, which
presumably allowed for a better understanding of the
underlying causal factors and, hence, produced higher
levels of reliabilities. Another possibility is that the
definitions and examples currently used to describe
HFACS are too closely tied to military aviation and are
therefore somewhat ambiguous to those within a com-
mercial setting. Indeed, the reliability of the HFACS
framework has been shown to improve within the com-
mercial aviation domain when efforts are taken to pro-
vide examples and checklists that are more compatible
with civil aviation accidents (Wiegmann, Shappell,
Cristina & Pape, 2000).
HFACS Analysis
Given the large number of accident causal factors
contained in the NTSB database, each accident ap-
peared, at least on the surface, to be relatively unique. As
such, commonalties or trends in specific error forms
across accidents were not readily evident in the data. Still,
the recoding of the data using HFACS did allow for
similar error-forms and causal factors across accidents to
be identified and the major human causes of accidents to
be discovered.
Specifically, the HFACS analysis revealed that the
highest percentage of all aircrew-related accidents as
associated with skill-based errors. Furthermore, this pro-
portion was lowest during the last two years of this study,
suggesting that accidents associated with skill-based er-
rors may be on the decline. To some, the finding that
skill-based errors were frequently observed among the
commercial aviation accidents examined is not surpris-
ing given the dynamic nature and complexity of piloting
commercial aircraft, particularly in the increasingly
congested U.S. airspace. The question remains, how-
ever, as to the driving force behind the possible reduc-
tion in such errors. Explanations could include
improved aircrew training practices or perhaps better
selection procedures. Another possibility might be the
recent transition within the regional commuter indus-
try from turboprop to jet aircraft. Such aircraft are
11
generally more reliable and contain advanced automa-
tion to help off-load the attention and memory de-
mands placed on pilots during flight.
Unfortunately, the industry-wide intervention pro-
grams and other changes that were made during the
1990s were neither systematically applied nor targeted at
preventing specific error types, such as skill-based errors.
Consequently, it is impossible to determine whether all
or only a few of these efforts are responsible for the
apparent decline in skill-based errors. Nevertheless, given
that an error analysis has now been conducted on the
accident data, future invention programs can be strategi-
cally targeted at reducing skill-based errors. Further-
more, the effectiveness of such efforts can be objectively
evaluated so that efforts can be either reinforced or
revamped to improve safety. Additionally, intervention
ideas can now also be shared across organizations that
have performed similar HFACS analyses. One example
is the U.S. Navy and Marine Corps, which have recently
initiated a systematic intervention program for address-
ing their growing problem with accidents associated with
skill-based errors in the fleet (Shappell & Wiegmann,
2000b). As a result, lessons learned in the military can
now be communicated and shared with the commercial
aviation industry, and vice versa.
The observation that both CRM failures and decision
errors are associated with a large percentage of aircrew-
related accidents is also not surprising, given that these
findings parallel the results of similar HFACS and hu-
man error analyses of both military and civil aviation
accidents (OHare et al., 1994; Wiegmann & Shappell,
1999). What is surprising, or at least somewhat discon-
certing, is the observation that both the percentage and
rate of aircrew-related accidents associated with both
CRM and decision errors have remained relatively stable.
Indeed, both the FAA and aviation industry have in-
vested a great deal of resources into intervention strate-
gies specifically targeted at improving CRM and
aeronautical decision making (ADM), with apparently
little overall effect.
The modest impact that CRM and ADM programs
have had on reducing accidents may be due to a variety
of factors, including the general lack of systematic
analyses of accidents associated with these problems.
Consequently, most CRM and ADM training pro-
grams use single case studies to educate aircrew, rather
then focus on the fundamental causes of these prob-
lems in the cockpit using a systematic analysis of the
accident data. Another possible explanation for the
general lack of CRM and ADM effectiveness is that
many established training programs involve class-
room exercises that are not followed up by simulator
training that requires CRM and ADM principles to be
applied. More recent programs, such as the Advanced
Qualification Program (AQP), have been developed
to take this next step of integrating ADM and CRM
principles into the cockpit. Given that the current
HFACS analyses has identified the accidents associ-
ated with these problems, at least across a seven-year
period, more fine-grained analyses can be conducted
to identify the specific problems areas in need of
training. Furthermore, the effectiveness of the AQP
program and other ADM training in reducing aircrew
accidents associated with CRM failures and decision
errors can be systematically tracked and evaluated.
The percentage of aircrew-related accidents associated
with violations (e.g., not following federal regulations or
a companys standard operating procedures) exhibited a
slight increase across the years examined in this study.
Some authors (e.g., Geller, 2000) have suggested that
violations, such as taking short-cuts in procedures or
breaking rules, are often induced by situational factors
that reinforce unsafe acts while punishing safe actions.
Not performing a thorough preflight inspection due to
the pressure to achieve an on-time departure would be
one example. However, according to Reasons (1990)
model of active and latent failures, such violation-induc-
ing situations are often set up by supervisory and man-
agement policies and practices.
Such theories suggest that the best strategy for reduc-
ing violations by aircrew is to enforce the rules and to
hold both the aircrew and their supervisors/organiza-
tions accountable. Indeed, this strategy has been effective
with the Navy and Marine Corps in reducing aviation
mishaps associate with violations (Shappell, et al., 1999).
Still, as mentioned earlier, very few of the commercial
accident reports examined in this study cited supervisory
or organizational factors as accident causes, suggesting
that more often than not, aircrews were the only ones
responsible for the violations. Again, more thorough
accident investigations may need to be performed to
identify possible supervisory and organizational issues
associated with these events.
Although pilots flying with FAR Part 135 scheduled
carriers had fewer annual flight hours during the years
covered in this study (NTSB, 2000), the overall number
of accidents associated with most error types was gener-
ally higher for FAR Part 135 scheduled carriers, com-
pared with FAR Part 121 scheduled carriers. This finding
is likely due, at least in part, to the fact that most pilots
12
flying aircraft operating under FAR Part 135 are younger
and much less experienced. Furthermore, such pilots
often fly less sophisticated and reliable aircraft into areas
that are less likely to be controlled by ATC. As a result,
they may frequently find themselves in situations that
exceed their training or abilities. Such a conclusion is
supported by the findings presented here, since a larger
percentage of FAR Part 135 aircrew-related accidents
were associated with the physical/mental limitations of
the pilot. However, a smaller percentage FAR Part 135
aircrew accidents were associated with CRM failures,
possibly because some FAR Part 135 aircraft are single-
piloted, which simply reduces the opportunity for
CRM failures.
These differences between FAR Parts 121 and 135
schedule carriers may be less evident in future aviation
accident data since the federal regulations were changed
in 1997. Such changes require FAR Part 135 carriers
operating aircraft that carry ten or more passengers to
now operate under more stringent FAR Part 121 rules.
Thus, the historical distinction in the database between
FAR Part 135 and 121 operators has become somewhat
blurred in the years extending beyond the current analy-
sis. Therefore, future human-error analyses and com-
parisons across these different types of commercial
operations will therefore need to consider these changes.
SUMMARY AND CONCLUSIONS
This investigation demonstrates that the HFACS
framework, originally developed for and proven in the
military, can be used to reliably identify the underlying
human factors problems associated with commercial
aviation accidents. Furthermore, the results of this study
highlight critical areas of human factors in need of
further safety research and provide the foundation upon
which to build a larger civil aviation safety program.
Ultimately, data analyses such as that presented here will
provide valuable insight aimed at the reduction of avia-
tion accidents through data-driven investment strategies
and objective evaluation of intervention programs. The
HFACS framework may also prove useful as a tool for
guiding future accident investigations in the field and
developing better accident databases, both of which
would improve the overall quality and accessibility of
human factors accident data.
Still, the HFACS framework is not the only possible
system upon which such programs might be developed.
Indeed, there often appears to be as many human error
frameworks as there are those interested in the topic
(Senders & Moray, 1991). Indeed, as the need for better
applied human error analysis methods has become more
apparent, an increasing number of researchers have pro-
posed other comprehensive frameworks similar to HFACS
(e.g., OHare, in press). Nevertheless, HFACS is, to date,
the only system that has been developed to meet a specific
set of design criteria, including comprehensiveness, reli-
ability, diagnosticity, and usability, all of which have
contributed to the frameworks validity as an accident
analysis tool (Shappell & Wiegmann, in press). Further-
more, HFACS has been shown to have utility as an error-
analysis tool in other aviation-related domains such as
ATC (HFACS-ATC; Pounds, Scarborough, & Shappell,
2000) and aviation maintenance (HFACS-ME; Schmidt,
Schmorrow, & Hardee, 1998), and is currently being
evaluated within other complex systems such as medi-
cine (currently referred to as HFACS-MD). Finally, it
is important to remember that neither HFACS nor
any other error-analysis tool can fix the problems
once they have been identified. Such fixes can only be
derived by those organizations, practitioners and hu-
man factors professionals who are dedicated to im-
proving aviation safety.
REFERENCES
Bird, F. (1974). Management guide to loss control. At-
lanta, GA: Institute Press.
Fleiss, J. (1981). Statistical Methods for Rates and Propor-
tions. New York: John Wiley.
Ford, C., Jack, T., Crisp, V. & Sandusky, R. (1999).
Aviation accident causal analysis. Advances in
Aviation Safety Conference Proceedings, (P-343).
Warrendale, PA: Society of Automotive Engi-
neers Inc.
Geller, E. (March, 2000). Behavioral safety analysis: A
necessary precursor to corrective action. Profes-
sional Safety, 29-32.
International Civil Aviation Organization (1993).
Investigation of human factors in accidents and
incidents (Human Factors Digest #7), Montreal:
Canada.
13
Jones, A. (1988). Climate and measurement of consen-
sus: A discussion of organizational climate. In S.
Cole, R.Demaree & W. Curtis, (Eds.), Applica-
tions of Interactionist Psychology: Essays in Honor of
Saul B. Sells (pp. 283-290). Hillsdale, NJ:
Earlbaum.
National Transportation Safety Board (2000). Aviation
accident statistics. [On-line]. Available:
www.ntsb.gov/aviation/Stats.htm
OHare, D. (in press). The Wheel of Misfortune. Ergo-
nomics.
OHare, D., Wiggins, M., Batt, R., and Morrison, D.
(1994). Cognitive failure analysis for aircraft acci-
dent investigation. Ergonomics, 37, 1855-69.
Pounds, J., Scarborough, A., & Shappell, S. (2000). A
human factors analysis of Air Traffic Control op-
erational errors (Abstract). Aviation, Space and
Environmental Medicine, 71, pp. 329
Rasmussen, J. (1982). Human errors: A taxonomy for
describing human malfunction in industrial in-
stallations. Journal of Occupational Accidents, 4,
pp. 311-33.
Reason, J. (1990). Human error. New York: Cambridge
University Press.
Schmidt, J., Schmorrow, D., & Hardee, M. (1998). A
preliminary human factors analysis of Naval Avia-
tion maintenance related mishaps. Proceedings of
the 1998 Airframe/Engine Maintenance and Repair
Conference (P329), Long Beach, CA.
Senders, J., & Moray, N. (1991). Human error: Cause,
prediction and reduction. Hillsdale, NJ: Earlbaum.
Shappell, S., & Wiegmann, D. (1996). U. S. Naval
Aviation mishaps 1977-92: Differences between
single- and dual-piloted aircraft. Aviation, Space,
and Environmental Medicine, 67, 65-9.
Shappell, S. & Wiegmann D. (1997a). A human error
approach to accident investigation: The taxonomy
of unsafe operations. The International Journal of
Aviation Psychology, 7, pp. 269-91.
Shappell, S. & Wiegmann, D. (1997b). A reliability
analysis of the Taxonomy of Unsafe Operations
(Abstract). Aviation, Space, and Environmental
Medicine, 69, pp. 620.
Shappell, S. & Wiegmann, D. (2000a). The Human
Factors Analysis and Classification System
(HFACS). (Report Number DOT/FAA/AM-00/7).
Washington DC: Federal Aviation Administration.
Shappell, S. & Wiegmann, D. (2000b). Is proficiency
eroding among U.S. Naval aircrews? A quantita-
tive analysis using the Human Factors Analysis
and Classification System (HFACS). Proceedings
of the 44
th
meeting of the Human Factors and Ergo-
nomics Society.
Shappell, S. & Wiegmann, D. (2001). Applying Reason:
The Human Factors Analysis and Classification
System (HFACS). Human Factors and Aerospace
Safety, 1, 59-86.
Shappell, S., Wiegmann, D., Fraser, J., Gregory, G.,
Kinsey, P., & Squier, H (1999). Beyond mishap
rates: A human factors analysis of U.S. Navy/
Marine Corps TACAIR and rotary wing mishaps
using HFACS (Abstract). Aviation, Space, and
Environmental Medicine, 70, pp. 416-7.
Wiegmann, D. & Shappell, S. (1997). Human factors
analysis of post-accident data: Applying theoreti-
cal taxonomies of human error. The International
Journal of Aviation Psychology, 7, pp. 67-81.
Wiegmann, D. & Shappell, S. (1999). Human error and
crew resource management failures in Naval avia-
tion mishaps: A review of U.S. Naval Safety Center
data, 1990-96. Aviation, Space, and Environmental
Medicine, 70, pp. 1147-51.
Wiegmann, D., Shappell, S., Cristina, F. & Pape, A.
(2000). A human factors analysis of aviation acci-
dent data: An empirical evaluation of the HFACS
framework (Abstract). Aviation, Space and Envi-
ronmental Medicine, 71, pp. 328.
Yacavone, D. W. (1993). Mishap trends and cause
factors in Naval aviation: A review of Naval Safety
Center data, 1986-90. Aviation, Space and Envi-
ronmental Medicine, 64, 392-5.