756 Full
756 Full
756 Full
BMJ Qual Saf: first published as 10.1136/bmjqs.2010.048694 on 1 March 2011. Downloaded from http://qualitysafety.bmj.com/ on March 3, 2023 by guest. Protected by copyright.
Using prospective clinical surveillance
to identify adverse events in hospital
< Additional appendices are ABSTRACT AEs (harm caused by errors) and potential
published online only. To Background: To improve patient safety, organisations AEs (errors with the potential for harm).
view these files please visit must systematically measure avoidable harms. Clinical
the journal online (http://
Numerous studies have demonstrated a high
qualitysafety.bmj.com). surveillancedconsisting of prospective case finding incidence of AEs and preventable AEs in
1 and peer reviewdcould improve identification of hospitalised patients.1e8 These studies have
The Ottawa Hospital,
Ottawa, Canada adverse events (AEs), preventable AEs and potential prompted significant investments to improve
2
Department of Medicine, AEs. The authors sought to describe and compare
patient safety.
University of Ottawa, Ottawa, findings of clinical surveillance on four clinical services
Canada Detection of AEs and preventable AEs is
3
in an academic hospital.
Clinical Epidemiology a primary step to achieving a safe healthcare
Methods: Clinical surveillance was performed by a nurse
Program, Ottawa Hospital system. By systematically measuring these
Research Institute, Ottawa, observer who monitored patients for prespecified
Canada clinical events and collected standard information about outcomes and analysing their causes, health-
4
Institute for Clinical each event. A multidisciplinary, peer-review committee care planners may design system changes to
Evaluative Sciences, Ottawa, rated causation for each event. Events were prevent them.9e14 Unfortunately, most
Canada
5 subsequently classified in terms of severity and type. hospitals and health systems have rudimen-
University of Ottawa Heart
Institute, Ottawa, Canada Results: The authors monitored 1406 patients during tary approaches to identify AEs. These
6
Department of Medicine, their admission to four hospital services: Cardiac approaches include the use of voluntary
Sunnybrook Health Sciences Surgery Intensive Care (n¼226), Intensive Care reporting, chart reviews and scanning of
Centre and University of (n¼211), General Internal Medicine (n¼453) and
Toronto Centre for Patient administrative data.15 Each of these methods
Obstetrics (n¼516). The authors detected 245 AEs
Safety, Toronto, Canada has important limitations including a failure
during 9300 patient days of observation (2.6 AEs per
to report, inconsistent peer review and poor
Correspondence to 100 patient days). 88 AEs (33%) were preventable. The
Dr Alan J Forster, Ottawa proportion of patients experiencing at least one AE,
specificity for patient safety problems.13 15e23
Hospital, Civic Campus, 1053 preventable AE or potential AE was 13.7%, 6.1% and Detecting AEs is especially challenging
Carling Avenue, because of the invisible nature of treatment-
5.3%, respectively. AE risk varied between services,
Administrative Services
ranging from 1.4% of Obstetrics to 11% of Internal related harm. When a patient experiences
Building 1, Ottawa, ON K1Y
4E9, Canada; Medicine and Intensive Care patients experiencing at a poor outcome, it is often assumed the
aforster@ohri.ca least one preventable AE. The proportion of patients outcome was caused by the underlying
experiencing AEs resulting in permanent disability or disease process. While this is often true,
Accepted 1 February 2011
death varied between services: ranging from 0.2% on careful peer review will often determine that
Published Online First
1 March 2011 Obstetrics to 4.9% on Cardiac Surgery Intensive Care. medical care was, at least, partially respon-
No services shared the most frequent AE type. sible. Furthermore, adverse outcomes and
Conclusions: Using clinical surveillance, the authors
the circumstances leading to them are often
identified a high risk of AE and significant variation in
not systematically documented. This will
AE risks and subtypes between services. These
often lead to disagreement between peers
findings suggest that institutions will need to evaluate
service-specific safety problems to set priorities and when chart review is used to identify AEs.24 25
design improvement strategies. Clinical surveillance is a promising method
of AE detection. In this method, patients and
providers are directly and indirectly moni-
tored by a trained observer prospec-
INTRODUCTION tively.26e29 As soon as clinical events are
identified, the observer records prespecified
This paper is freely available
online under the BMJ Improving patient safety requires the mini- information to facilitate subsequent peer
Journals unlocked scheme, misation of treatment-related harm. This is review. This avoids missing information that
see http://qualitysafety.bmj.
com/site/about/unlocked.
typically measured as adverse events (AEs) occurs from incomplete documentation in
xhtml (harms caused by medical care), preventable the patient record.
BMJ Qual Saf: first published as 10.1136/bmjqs.2010.048694 on 1 March 2011. Downloaded from http://qualitysafety.bmj.com/ on March 3, 2023 by guest. Protected by copyright.
Clinical surveillance has several advantages over Establishment of surveillance parameters
existing AE detection methods. It consists of active Before starting surveillance, we identified several service-
surveillance rather than relying solely on voluntary specific surveillance parameters describing surveillance
reporting; it ensures timely collection of information location, baseline patient information and triggers.
relevant to case classification rather than retrospectively We prespecified the surveillance location to ensure
relying on what is documented in charts, and it enhances consistent and representative case identification. On the
peer review through timely review of cases when Obstetrics service, for example, we monitored patients in
reviewers still remember the case. On the other hand, the case-room, the obstetrical operating room, and the
although it has been used successfully for years in postpartum ward. For Medicine, however, we monitored
tracking surgical complications and healthcare-associ- patients on the ward only. We also prespecified whether
ated infections, it has not been widely adopted for all AE we would include care preceding the observation period.
types in multiple, diverse settings. For example, for Intensive Care patients we included
In this study, we describe our findings from imple- events leading to unit admission.
menting clinical surveillance in our healthcare facility. We prespecified variables describing baseline patient
We describe a general process and the results obtained information to enable consistent case reviews and permit
during its application in four diverse services at an future risk adjustments. For example, when we imple-
academic hospital. The comparison of results between mented on the Medicine service, we collected informa-
services was of particular interest, as it would validate our tion regarding the presence of diagnoses associated with
ability to learn useful information from the programme poor outcomes.30
and would direct strategies for improvement. Most importantly, we prespecified triggers which
represented the factors we used to prompt case reviews.
METHODS Trigger tool methodology has been used successfully in
other institutions to address specific AE types such as
This study took place at a 1000-bed multicampus urban- adverse drug events,18 19 31e37 hospital-acquired infec-
based academic hospital. The hospital provides tertiary- tions38 and surgical complications,39e41 so we adapted
level patient care, including organ transplants, and is the concept to create a more generalised model for the
a Level 1 trauma centre. We performed our clinical identification of all types of AEs in various settings. For
surveillance activities as part of a research study, which each service, we reviewed the literature to identify
was approved by the Ottawa Hospital Research Ethics previously published risk factors for adverse outcomes. A
Board. multidisciplinary team specific to each service then
determined the face validity of these triggers and
Overview discussed additional triggers. Consensus was used to
We conducted surveillance sequentially on the following define the final set of triggers. The triggers represent
four services: General Internal Medicine, Obstetrics, clinical phenomenon, pharmacy orders and system
Intensive Care, and Cardiac Surgery Intensive Care. For events relevant to the patients treated. For example, ‘stat
each service, we monitored care for 12 weeks using Caesarean section’ was a system event on the Obstetrics
a common method, which was adapted for the service service, while ‘unplanned OR return’ was used on the
to ensure relevancy to the patients treated. The Cardiac Surgery Intensive Care unit. For all services, we
general method included establishment of surveillance had an ‘Other’ category to account for events that were
parameters, case finding, peer review and event classifi- not predicted a priori. (See online appendix 1) contains
cation (figure 1). the triggers we used for the four services.
BMJ Qual Saf: first published as 10.1136/bmjqs.2010.048694 on 1 March 2011. Downloaded from http://qualitysafety.bmj.com/ on March 3, 2023 by guest. Protected by copyright.
Case finding had no discernible change in status, then the event was
Case finding was performed daily Monday to Friday classified as a process problem. For outcomes, we first
between 08:00 and 16:00 by a trained nurse (observer). assessed whether they were caused by medical care (ie,
The observer training consisted of a patient safety an AE). If so, reviewers determined whether it was
presentation, relevant literature, practice case report preventable (ie, it was caused by error or was avoidable
generation and reviews, familiarisation with triggers and in some other way). We used six-point Likert scales with
service-specific integration. We integrated the observer a cut-point of 3 for both of these assessments. For
into the daily activities of the specific service to ensure processes, reviewers determined whether the procedural
maximal interaction with the providers and opportuni- or process problem could have caused harm (ie,
ties to observe care yet with minimal disruption to the a potential AE). Again, we used a six-point Likert scale
providers’ routine. This meant there were slight differ- with a cut-point of 3 for this assessment (ie, a score of 4,
ences in how the observations were performed. For 5 or 6 indicated a process problem).
example, in the Intensive Care Unit, observations
occurred at the bedside during morning team rounds, Event classification
while on the Medicine service they occurred at the Event classification occurred following categorisation of
nursing station. The observer was encouraged to ask cases as AEs, preventable AEs or potential AEs. For these
providers for details if they were not volunteered or cases, the observer defined the event’s severity and type.
documented in the chart. For severity, events caused one of the following levels of
All patients in the surveillance locations were eligible increasing harm: none; laboratory abnormalities only;
for case finding. The observer captured baseline infor- symptoms only; temporary disability; permanent
mation on all patients admitted to the service. Once data disability; or death. Events were also classified according
were entered, the observer monitored the patient until to one or more of the following types: hospital-acquired
discharge. The observer identified triggers using infection; adverse drug event; procedural complication;
a combination of three common AE detection methods: operative complication; diagnostic error; management
daily interactions with clinicians and administrators to error; anaesthetic complication; obstetrical injury; and
enquire about or discuss trigger events (prompted and system design flaw. As almost all errors could be argued
voluntary reporting), daily chart reviews and direct to be system errors, only those events that could not be
observation. If triggers occurred when our observer was assigned to another type were deemed to be system
not present, we identified them based on information design flaws.
volunteered by providers or from the chart.
Once a trigger was identified, we captured standard Analysis
information describing the case. The observer generated We described patient baseline characteristics using
a case summary using a standard format describing the median and IQR for continuous variables and frequency
patient and event in detail, the response to the event, distributions for categorical variables. We described
the impact of the response and the documented cause of events in terms of preventability, severity and type for
the response if there was one. This information was each service overall. We calculated the risk of experi-
compiled by the observer through discussions with the encing at least one event per hospital encounter. We also
relevant staff on the service. calculated the event rate in terms of events per patient
day of observation. We used SAS version 9.0 for all data
Peer review management and analyses.
Peer review took place weekly. For all reviews, we used
the same definitions and classification systems as those RESULTS
used in the Harvard Medical Practice Study and other
patient safety studies (See online appendix 2).1e7 We Table 1 describes the patient populations in terms of
used a majority opinion to consolidate the ratings of the their baseline characteristics. There were important
multiple reviewers.42 Cases were initially classified as differences between services. We observed many more
outcomes (in which a patient experienced some form of patients in Medicine (n¼453) and Obstetrics (n¼516)
harm, including pain, other distressing symptoms, such than Cardiac Surgery (n¼226) or Intensive Care
as dyspnoea, or reasonable markers for such symptoms (n¼211). Individual patients spent more days on Medi-
such as increases in respiratory rate) or process problems cine than Intensive Care, Obstetrics and Cardiac
(in which a task did not occur as planned). For example, Surgery, with median lengths of stay on the monitored
if a patient experienced a sudden drop in blood pres- unit of 8 days, 4.8 days, 2.4 days and 2.1 days, respec-
sure, the event was classified as an outcome. On the tively. The indications for admission varied extensively
other hand if a patient missed a dose of an antibiotic but between the four services. Medicine patients were older
759
BMJ Qual Saf: first published as 10.1136/bmjqs.2010.048694 on 1 March 2011. Downloaded from http://qualitysafety.bmj.com/ on March 3, 2023 by guest. Protected by copyright.
Original research
BMJ Qual Saf: first published as 10.1136/bmjqs.2010.048694 on 1 March 2011. Downloaded from http://qualitysafety.bmj.com/ on March 3, 2023 by guest. Protected by copyright.
(median age on Medicine, Cardiac Surgery, Intensive critical care areas (Cardiac Surgery Intensive Care and
Care and Obstetrics was 74, 70, 66 and 30 years, respec- Intensive Care). The risk of AEs causing permanent
tively) and frailer (proportion of patients with disability or death varied 24-fold across services:
a Charlson Index of 3 on Medicine, Cardiac Surgery and Cardiac Surgery Intensive Care (11/226¼4.9%), Inten-
Intensive Care was 24%, 3% and 7%, respectively). sive Care (8/211¼3.8%), Medicine (2/453¼0.4%),
Table 2 demonstrates the overall and service-specific Obstetrics (1/516¼0.2%).
AE risk and rate. 192 patients (13.7%) encountered 245 Table 3 also describes the types of AEs identified.
AEs (2.6 events per 100 patient days), 86 patients (6.1%) There were important proportions of all AE types, but
encountered 88 preventable AEs (0.9 events per 100 overall the most common types were therapeutic errors,
patient days), and 75 patients (5.3%) encountered 81 procedural complications, and adverse drug events.
potential AEs (0.9 events per 100 patient days). The Within each service, the top three AE types differed, and
greatest number of events was observed on the Medicine no service shared the most frequent AE type. For Cardiac
service, which accounted for 63% (75/119) of patients Surgery, the top three types were surgical complications
with AEs, 56% of (48/86) patients with preventable AEs (47%), hospital-acquired infections (19%) and proce-
and 56% (40/75) of patients with potential AEs. dural complications (11%). For Intensive Care, the top
The risk and rate of all AEs and preventable AEs were three types were hospital-acquired infections (25%),
highest for the Intensive Care unit. Of the 211 patients procedural complications (24%) and therapeutic errors
observed on Intensive Care, 52 (24.6%) experienced at (19%). For Medicine, the top three types were adverse
least one AE, and 23 (10.9%) experienced a preventable drug events (30%), therapeutic errors (24%) and
AE. The corresponding event rates were 4.5 and 1.4 per procedural complications (12%). For Obstetrics, the top
100 patient days for AEs and preventable AEs, respec- three types were therapeutic errors (36%), procedural
tively. The lowest risk and rate of events occurred on the complications (32%) and adverse drug events (27%).
Obstetrics service. Of the 516 patients observed on
Obstetrics, 20 (3.9%) experienced at least one AE, and
seven (1.4%) experienced a preventable AE. The corre- DISCUSSION
sponding event rates were 1.5 and 0.5 per 100 patient
days for AEs and preventable AEs, respectively. We used clinical surveillance to detect AEs on four
Table 3 describes the severity of all AEs detected. Of all hospital services. AE risk per encounter was 13% with
245 events detected, six (2.5%) resulted in/were associ- a range of 4% to 25% depending on service. Overall,
ated with/potentially led to death, and 16 (6.6%) led to 36% of AEs were preventable, though this was highly
permanent disability. The vast majority of events resulted variable between services, ranging from 13% to 56%.
in symptoms only (n¼75, 30.7%) or led to temporary General Internal Medicine had the greatest number of
disability (n¼112, 45.9%). The majority of events that AEs but also had the highest number of patients and
led to death or permanent disability occurred in the two comorbidities. After controlling for length of stay and
BMJ Qual Saf: first published as 10.1136/bmjqs.2010.048694 on 1 March 2011. Downloaded from http://qualitysafety.bmj.com/ on March 3, 2023 by guest. Protected by copyright.
Table 3 Adverse event severity and type
Overall Cardiac Intensive Medicine Obstetrics p Value
N (%) surgery N (%) care N (%) N (%) N (%) (c2 test)
Total 245 (100) 62 (100) 72 (100) 89 (100) 22 (100)
Severity <0.001
Lab/physiological 36 (14.7) 2 (3.2) 10 (13.9) 23 (25.8) 1 (4.6)
Symptoms 75 (30.7) 9 (14.5) 18 (25.0) 38 (42.7) 10 (45.5)
Temporary 112 (45.9) 40 (64.5) 36 (50.0) 26 (29.2) 10 (45.5)
Permanent 16 (6.6) 8 (12.9) 7 (9.7) 0 (0.0) 1 (4.6)
Death 6 (2.5) 3 (4.8) 1 (1.4) 2 (2.3) 0 (0.0)
Type <0.001
Procedural complication 42 (17.14) 7 (11.3) 17 (23.6) 11 (12.4) 7 (31.8)
Infection 37 (15.1) 12 (19.4) 18 (25.0) 7 (7.9) 0 (0.0)
Adverse drug event 46 (18.8) 4 (6.5) 9 (12.5) 27 (30.3) 6 (27.3)
Therapeutic error 49 (20.0) 5 (8.1) 14 (19.4) 22 (24.7) 8 (36.4)
Surgical complication 37 (15.1) 29 (46.8) 7 (9.7) 0 (0.0) 1 (4.6)
Diagnostic error 10 (4.1) 3 (4.8) 4 (5.6) 3 (3.4) 0 (0.0)
Falls 10 (4.1) 0 (0.0) 0 (0.0) 10 (11.2) 0 (0.0)
System problem 11 (4.5) 2 (3.2) 3 (4.2) 6 (6.7) 0 (0.0)
Pressure ulcer 3 (1.2) 0 (0.0) 0 (0.0) 3 (3.4) 0 (0.0)
the number of patients observed, the risk of AEs, follows that solutions will also necessarily vary. For
including those with the most severe consequences, was example, in the Cardiac Surgery Intensive Care unit,
greatest in the two critical care areas. There were interventions should be directed to preventing certain
differences in the types of AEs identified; however, types of surgical complications. On the Medicine service,
therapeutic errors were common in all subgroups interventions that focus specifically on reducing errors
studied. Finally, clinical surveillance identified an related to the medication administration process would
important number of potential AEs. be most beneficial. The conclusion that errors and their
These findings are important because they identify solutions vary by patient service is intuitively obvious but
ample opportunities for quality improvement. We found is often not incorporated in the ‘top down’ approaches
a greater AE risk than most previous studies of AEs which to improve quality and safety taken by accreditation
have tended to rely solely on chart review.1e7 We used organisations or other regulators.
a clinical observer to identify events within 24 h of their Our study also supports the conclusion that priority
occurrence, and reviewed them within 7 days of their setting, as it pertains to safety improvement, must occur
identification. The timely identification of problems in a highly strategic manner. The greatest number of AEs
would ultimately mean a more rapid response to rectify occurred on the service with the frailest patients. Effec-
them. The reviews were performed by clinicians who tive interventions to improve safety in these patients
knew the clinical setting and were well respected by their would have the largest reduction on the institutional
local peers. We believe that these factors resulted in number of events. However, after controlling for the
classifications with a much greater degree of face validity number of patients and length of observation, these
than existing methods to identify AEs (such as incident patients had a risk of events similar to the least frail
reports and chart reviews). These factors also led us to patients. The highest event risk and the most severe
identify a significantly greater proportion of events events occurred in services that had very invasive thera-
attributed to diagnostic and therapeutic errors, than pies and where patients were most acutely ill. Therefore,
prior studies. Furthermore, the systematic nature of our efforts to prevent injuries in these groups might be more
data collection ensured our methods were less biased efficient and associated with a greater overall impact in
than morbidity and mortality reviews or closed-claims terms of reducing the economic burden of AEs. For all
analysis.15 43 44 these reasons, organisations will have to be very clear
Our study supports the concept that improvement about their goals when targeting the safety problems
strategies must be locally developed and must target they hope to solve.
specific problems. We found that AE risk and type varied Finally, this research has highlighted the feasibility of
by service. The underlying explanation for this finding clinical surveillance. We used a comprehensive strategy
likely relates to the frequency of underlying processes involving the use of direct observation, voluntary and
occurring within these areas that in turn lead to more prompted reporting, as well as daily chart reviews. This
opportunities to cause harm. Regardless of the cause, it combination of AE detection methods has been
BMJ Qual Saf: first published as 10.1136/bmjqs.2010.048694 on 1 March 2011. Downloaded from http://qualitysafety.bmj.com/ on March 3, 2023 by guest. Protected by copyright.
recommended by others15 and serves to increase the safety events. Second, we recommend an economic
overall performance of the monitoring system. Our analysis to determine the most efficient method of
programme was well accepted in the four services adverse event detection. Third, we would recommend
studied with very little financial support (the clinical evaluations of this method in different hospitals. Finally,
observer was a funded position). However, all other and most importantly, we recommend studies to deter-
functions required to maintain the process were volun- mine if adverse event detection leads to improvement in
tary, and there was very little infrastructure support patient outcomes.
required. There was no evidence that providers who
were being observed were overtly or covertly averse to Acknowledgements We would like to thank A Jennings for her assistance
throughout the study and in the preparation of the manuscript. This research
the programme or changed their behaviours. In fact, the received financial support from the Canadian Patient Safety Institute, the
methodology was so well received that our institution Canadian Institute for Health Research, the Healthcare Insurance Reciprocal
wanted to and is currently taking steps to implement it of Canada, the University of Ottawa Heart Institute, and the Ottawa Hospital
Centre for Patient Safety.
on other services.
Our findings are important in that we used a repro- Funding This research received financial support from the Canadian Patient
Safety Institute, the Canadian Institute for Health Research, the Healthcare
ducible method of case detection and case review Insurance Reciprocal of Canada, the University of Ottawa Heart Institute and
applying a standard method of classification to facilitate the Ottawa Hospital Centre for Patient Safety.
comparison. However, there are three important limita- Competing interests None.
tions which we would like to highlight. First, we are Ethics approval Ethics approval was provided by the Ottawa Hospital Research
uncertain of the programme’s reliability. Threats to Ethics Board.
reliability include the observer and peer-review Provenance and peer review Not commissioned; externally peer reviewed.
processes. We attempted to mitigate these concerns
through standardisation of the process of observation REFERENCES
using triggers and case report forms; and, for the peer 1. Baker GR, Norton PG, Flintoft V, et al. The Canadian Adverse Events
Study: the incidence of adverse events among hospital patients in
review, the use of multiple reviewers. These approaches Canada. CMAJ 2004;170:1678e86.
have been successful in improving reliability in prior 2. Forster AJ, Asmis TR, Clark HD, et al. Ottawa Hospital Patient Safety
Study: incidence and timing of adverse events in patients admitted to
settings. However, we recommend further research a Canadian teaching hospital. CMAJ 2004;170:1235e40.
before our method be adopted by health systems to 3. Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events
and negligence in hospitalized patients. Results of the Harvard
compare institutions. Second, the generalisability of our Medical Practice Study I. N Engl J Med 1991;324:370e6.
surveillance feasibility is unknown, particularly because it 4. Thomas EJ, Studdert DM, Burstin HR, et al. Incidence and types of
adverse events and negligent care in Utah and Colorado. Med Care
was performed using a single observer. Even though we 2000;38:261e71.
successfully implemented our programme within 5. Vincent C, Neale G, Woloshynowych M. Adverse events in British
hospitals: preliminary retrospective record review. BMJ
multiple settings in different facilities, it is possible that 2001;322:517e19.
other settings may create challenges. Further work is 6. Wilson RM, Runciman WB, Gibberd RW, et al. The Quality in
Australian Health Care Study. Med J Aust 1995;163:458e71.
recommended to establish whether the programme can 7. Davis P, Lay-Yee R, Briant R, et al. Adverse events in New Zealand
be successfully adapted within non-teaching hospitals public hospitals I: occurrence and impact. New Zeal Med J
2002;115:U271.
and in different health systems. Third, the observer 8. O’Neil AC, Petersen LA, Cook EF, et al. Physician reporting
cannot be everywhere at once and may miss some events compared with medical-record review to identify adverse medical
events. Ann Intern Med 1993;119:370e6.
as they are happening. This means the programme 9. Cullen DJ, Bates DW, Small SD, et al. The incident reporting system
cannot be used to intervene systematically and directly in does not detect adverse drug events: a problem for quality
improvement. Jt Comm J Qual Improv 1995;21:541e8.
patient care when it does not meet the standard. 10. Beckmann U, Bohringer C, Carless R, et al. Evaluation of two
methods for quality improvement in intensive care: facilitated incident
However, this is not the stated purpose of the monitoring and retrospective medical chart review. Crit Care Med
programme. Furthermore, if cases are detected in which 2003;31:1006e11.
11. Flynn EA, Barker KN, Pepper GA, et al. Comparison of methods for
an immediate intervention is warranted, then this detecting medication errors in 36 hospitals and skilled-nursing
appropriate response should be easily performed. For facilities. Am J Health Syst Pharm 2002;59:436e46.
12. Berwick DM, Leape LL. Reducing errors in medicine. BMJ
example, we identified a few instances when a critical 1999;319:136e7.
laboratory abnormality had not been recognised and 13. Leape LL. Reporting of adverse events. N Engl J Med
2002;347:1633e8.
simply brought it to the attention of the treating team. 14. Leape LL. A systems analysis approach to medical error. J Eval Clin
In summary, our study supports the conclusion that Pract 1997;3:213e22.
15. Thomas EJ, Petersen LA. Measuring errors and adverse events in
clinical surveillance appears to be an effective means of health care. J Gen Intern Med 2003;18:61e7.
detecting patient safety issues. Before making a general 16. Kellogg VA, Havens DS. Adverse events in acute care: an integrative
literature review. Res Nurs Health 2003;26:398e408.
recommendation for widely adopting this method, we 17. Walshe K. Adverse events in health care: issues in measurement.
recommend further evaluations to address four issues. Qual Health Care 2000;9:47e52.
18. Murff HJ, Patel VL, Hripcsak G, et al. Detecting adverse events for
First, we recommend comparing the findings of clinical patient safety research: a review of current methodologies. J Biomed
surveillance with other methods of identifying patient Inform 2003;36:131e43.
BMJ Qual Saf: first published as 10.1136/bmjqs.2010.048694 on 1 March 2011. Downloaded from http://qualitysafety.bmj.com/ on March 3, 2023 by guest. Protected by copyright.
19. Karson AS, Bates DW. Screening for adverse events. J Eval Clin 34. Hope C, Overhage JM, Seger A, et al. A tiered approach is more cost
Pract 1999;5:23e32. effective than traditional pharmacist-based review for classifying
20. Allan EL, Barker KN. Fundamentals of medication error research. Am computer-detected signals as adverse drug events. J Biomed Inform
J Hosp Pharm 1990;47:555e71. 2003;36:92e8.
21. Iezzoni LI. Assessing quality using administrative data. Ann Intern 35. Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug
Med 1997;127:666e74. events: development of a computer-based monitor and comparison
22. Bates DW, Evans RS, Murff H, et al. Detecting adverse events using with chart review and stimulated voluntary report. JAMIA
information technology. J Am Med Inform Assoc 2003;10:115e28. 1998;5:305e14.
23. Weinger MB, Slagle J, Jain S, et al. Retrospective data collection and 36. Jha AK, Kuperman GJ, Rittenberg E, et al. Identifying
analytical techniques for patient safety studies. J Biomed Inform hospital admissions due to adverse drug events using
2003;36:106e19. a computer-based monitor. Pharmacoepidemiol Drug Saf
24. Thomas EJ, Lipsitz SR, Studdert DM, et al. The reliability of medical 2001;10:113e19.
record review for estimating adverse event rates. Ann Intern Med 37. Rothschild JM, Landrigan CP, Cronin JW, et al. The Critical Care
2002;136:812e16. Safety Study: The incidence and nature of adverse events and
25. Hayward RA, Hofer TP. Estimating hospital deaths due to medical serious medical errors in intensive care. Crit Care Med
errors: preventability is in the eye of the reviewer. JAMA 2005;33:1694e700.
2001;286:415e20. 38. Haley RW, Culver DH, White JW, et al. The efficacy of infection
26. Forster AJ, Halil RB, Tierney MG. Pharmacist surveillance of adverse surveillance and control programs in preventing nosocomial
drug events. Am J Health Syst Pharm 2004;61:1466e72. infections in US hospitals. Am J Epidemiol 1985;121:182e205.
27. Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug events 39. Neumayer L, Mastin M, Vanderhoof L, et al. Using the Veterans
and potential adverse drug events. Implications for prevention. ADE Administration National Surgical Quality Improvement Program to
Prevention Study Group. JAMA 1995;274:29e34. improve patient outcomes. J Surg Res 2000;88:58e61.
28. Andrews LB, Stocking C, Krizek T, et al. An alternative strategy for 40. Khuri SF, Daley J, Henderson WG. The comparative assessment and
studying adverse events in medical care. Lancet 1997;349:309e13. improvement of quality of surgical care in the Department of Veterans
29. Donchin Y, Gopher D, Olin M, et al. A look into the nature and causes Affairs. Arch Surg 2002;137:20e7.
of human errors in the intensive care unit. 1995. Qual Saf Health Care 41. Rowell KS, Turrentine FE, Hutter MM, et al. Use of national surgical
2003;12:143e8. quality improvement program data as a catalyst for quality
30. Charlson ME, Pompei P, Ales KL, et al. A new method of classifying improvement. J Am Coll Surgeons 2007;204:1293e300.
prognostic comorbidity in longitudinal studies: development and 42. Forster AJ, O’Rourke K, Shojania KG, et al. Combining ratings from
validation. J Chronic Dis 1987;40:373e83. multiple physician reviewers helped to overcome the uncertainty
31. Classen DC, Pestotnik SL, Evans RS, et al. Description of associated with adverse event classification. J Clin Epidemiol
a computerized adverse drug event monitor using a hospital 2007;60:892e901.
information system. Hosp Pharm 1992;27:774e9. 43. Vincent C, Taylor-Adams S, Chapman EJ, et al. How to
32. Rozich JD, Haraden CR, Resar RK. Adverse drug event trigger tool: investigate and analyse clinical incidents: clinical risk unit and
a practical methodology for measuring medication related harm. Qual association of litigation and risk management protocol. BMJ
Saf Health Care 2003;12:194e200. 2000;320:777e81.
33. Honigman B, Lee J, Rothschild J, et al. Using computerized 44. Vincent C, Stanhope N, Crowley-Murphy M. Reasons for not
data to identify adverse drug events in outpatients. JAMIA reporting adverse incidents: an empirical study. J Eval Clin Pract
2001;8:254e66. 1999;5:13e21.