1874-4346/23 Send Orders for Reprints to reprints@benthamscience.

net

The Open Nursing Journal

Content list available at: https://opennursingjournal.com

RESEARCH ARTICLE

Comparison of Perception Differences among Nurses based on Harm

Assessment Education in the Patient Safety Classification System
Kwangmi Lee1 and Eunhee Shin2,*
1
Department of Public Health Service Team, National Cancer Center, Goyang-si, South Korea
2
Department of Nursing Science, Sangji University, Wonju-si, Gangwon-do, South Korea

Abstract:
Background:
Accurate harm assessment is critical in the patient safety event management system, but few studies have been published to support the need for
training in harm assessment of patient safety events for nurses.

Objective:
The purpose of this study was to see if there was a difference in the degree of agreement between before and after nurses received training on a
patient safety event harm assessment guideline.

Methods:
After participating in online harm assessment education, 65 subjects completed the self-report questionnaire. Data for the general characteristics of
the study subjects and the frequency of respondents for each scenario were analyzed using descriptive statistics. Fleiss' kappa was calculated by
estimating the inter-rater agreement among respondents for each scenario.

Results:
The agreement value of subjects by Fleiss' Kappa value improved from k = 0.23 before education to k = 0.31 after education, according to the harm
assessment. There was no change at k = 0.30 before and after education for the harm period.

Implication for Nursing & Conclusion:

This study’s findings suggest that harm assessment agreement among nurses could be increased through harm assessment education. As a result,
case-based education on harm assessment must be expanded, as well as related programs for practical education via patient safety event casebooks.

Keywords: Nurses, Patient safety classification system, Harm assessment, Agreement value, Harm assessment guideline, Harm duration.

Article History Received: January 09, 2023 Revised: May 30, 2023 Accepted: July 21, 2023

1. INTRODUCTION assists each medical institution in collecting data on patient

safety events (PSEs) severity of the event, and the impact on
The Institute of Medicine emphasized the importance of
the patient [3], and health care professionals or patient safety
patient safety and patient safety reporting in its 1999 report “To
experts rely on information about the event that occurred, the
Err Is Human: Building a Safer Health System” and the World
degree of harm, and information on contributing factors [4].
Health Organization (WHO) Board of Directors announced in
2003 the need to develop standard terminology and However, while the patient safety reporting system provides
classification systems, harm assessment, and reporting-learning critical information for analyzing actual or potential PSEs, low-
systems as systematic factors for patient safety [1, 2]. quality data limits its utility. Accurate harm assessment is
According to reports on patient’s safety, the reporting system critical in the PSE management system because it is an
essential step in the study of PSEs and provides an opportunity
* Address correspondence to this author at the Department of Nursing Science, to lead patient risk management.
Sangji University, Wonju-si, Gangwon-do, South Korea;
E-mail: [email protected] The Patient Safety Organization developed the Common

DOI: 10.2174/18744346-v17-230927-2023-5, 2023, 17, e187443462308170

2 The Open Nursing Journal, 2023, Volume 17 Lee and Shin

Formats, a standardized PSE reporting format [5]. In addition, the consistency of harm assessment in nurse's PSE reports.
the WHO developed a 5-point scale for harm assessment [6],
the National Coordinating Council for Medication Error 2.2. Study Population
Reporting and Prevention (NCC MERP) developed a tool for The nurses who participated in the previous study’s survey
considering the duration and permanence of harm, with a kappa were evaluated using the developed “Harm assessment
value of 0.74 [7]. The Institute for Healthcare Improvement guideline” with the cooperation of the national cancer center
developed a Global Trigger Tools with a mean kappa value of specializing in cancer with 555 beds in Gyeonggi-do, which
0.65 [8, 9]. The reliability of the reporter's judgment in the type
conducted the harm and harm period evaluation among nurses
and severity of PSEs is the most important factor in the harm
in 2018. Following education on the harm assessment of PSEs,
score. If the reliability of the harm assessment is low,
a survey on nine scenarios of PSEs, was conducted, and the
comparing PSE data and determining direction will be difficult.
results before and after education were compared and analyzed.
The clinical backgrounds of clinicians, nurses, and
The study was conducted after receiving Institutional
pharmacists who report PSEs in the clinical setting differ,
Review Board (IRB) approval in February 2020, but due to the
which is the cause of deviation in harm assessment. Nurses
unexpected COVID-19 situation, collective education was not
were reported to be the most active in reporting events, owing
possible, so the research method was changed to online
to the fact that they witnessed PSEs the most frequently during
education and an online survey.
the nursing process, and reported lower harm assessments for
medication errors than clinicians [10]. Second, researchers Following the creation of an educational video by the
emphasize the importance of education and training in investigator, the investigator commissioned an online survey
understanding harm scores [11]. Finally, it has been specialist to have the subjects watch the educational video and
demonstrated that the reliability of PSEs can be affected by then participate in the survey. Many of the 133 subjects who
how they are reported and that bias can be reduced because participated in the 2018 survey no longer worked at the
web-based reporting can review integrated data and direction institution due to job changes or leave, and some refused to
rather than paper-type reporting [12]. Because nurses report participate in the study. As a result, the results of 65 research
more PSEs than clinicians [13], nurses' behaviors, attitudes, subjects who completed their education and participated in the
and perceptions are especially important when reporting PSEs survey were analyzed after they voluntarily agreed to
[3]. It has also been reported that nurses' relationships with participate in the study.
doctors, workload, and working environment, such as
colleagues, all have an impact on data quality [14]. 2.3. Guidelines for Harm Assessment and Educational
Videos
According to the study of inter-rater agreement in harm
assessment for nurses, the AHRQ Common Format Harm A survey of clinical nurses was conducted using the
Scale version 1.1 and 1.2 (AHRQ’s version 1.1 or 1.2) showed AHRQ’s version 1.2 to report PSEs [15], indicating that harm
a moderate degree of agreement, with k=0.45, but agreement and harm period evaluation were dispersed when harm such as
for each ward in the same institution was found to be very low “wrongly labeled samples” or “thoracic drainage” was not
[14]. Another study for nurses using the AHRQ’s version 1.2 obvious in the scenario. With these findings in mind, the
found low agreement between raters, with k = 0.21 [15]. “Harm assessment guideline” was created by referring to data
developed elsewhere to assist in assessing the harm and harm
The low consistency in nurses’ harm assessment can be
period of nurses.
interpreted as a subject in the interpretation of each patient
safety event scenario of the AHRQ’s version 1.2 used in the We developed “Harm assessment guidelines” by referring
study, but it also suggests the need for education so that harm to existing guidelines [16 - 18] related to harm assessment. The
assessment scores can be assigned accurately [3]. “Harm assessment guidelines” began with an overview of the
harm WHO’s harm classification system and scale, as well as
Similarly, according to the findings of a study that
the NCC MERP and AHRQ harm classification system and
examined PSEs reported in the emergency room, emergency scale. The AHRQ’s version 1.2 was then described. Since it is
room residents require education on the classification of patient necessary to confirm whether there has been a deviation from
safety events and harm assessment [16]. However, few studies standard treatment when classifying preventable harm
have been published to support the need for training in harm assessment, the considerations when assessing the patient’s
assessment of PSEs for nurses or clinicians who are major PSE outcome were explained. Finally, the harm assessment
reporters in the clinical setting. As a result, the goal of this guidelines in PSEs, as well as the PSEs classification and harm
study was to see if there was a difference in consistency before system were explained and the harm assessment was conducted
and after developing guidelines for PSE harm assessment using actual patient safety event examples. Fig. (1) depicts the
education for nurses in clinical settings. contents of the “patient safety event classification” presented in
this study.
2. MATERIALS AND METHODS
As for the expert review of the developed guidelines, it
2.1. Research Design was impossible to hold face-to-face meetings due to the
COVID-19 situation, so the contents were revised and
This is a single-group pre- and post-design experimental supplemented by listening to patient safety experts’ reviews
study comparing before and after education by developing a and opinions through non-face-to-face methods such as video
“Harm assessment guideline” educational material to improve conferencing and e-mail.
Comparison of Perception Differences among Nurses The Open Nursing Journal, 2023, Volume 17 3

No
Were there deviations from generally
accepted performance standards?

Yes -Not a patient safety event

-Consider complications, keep
monitoring
No
Did the deviation affect the patient?

Yes Near miss (A/B)

Did the deviation cause moderate or No

Moderate harm(E), Severe harm
severe harm or death?
(F) /No harm (C), Mild harm (D)
Yes

Sentinel event Has any harm occurred?

Yes No

Adverse event No harm event

(E,F) (C,D)

• Requires treatment or intervention

• Additional interventions needed
→Moderate harm (E)
for prevention→ Mild harm (D)
• Short or long-term hospitalization
• Otherwise →No harm (C)
required → Severe harm (F)

Fig. (1). Algorithm for classifying patient safety events.

2.4. Research Tool included in the questionnaire.

There were also 13 questions to determine patient safety
2.4.1. Patient Safety Event Scenarios are Chosen and
incident participation in the current department, patient safety
Reviewed for Harm Assessment
incident experience in the current department, patient safety
The patient safety scenario of the questionnaire was incident type, patient safety incident report, and experienced
developed in this study with reference to AHRQ’s version 1.2 patient safety incident type.
[4], with a focus on domestic patient safety cases. Several
emails were sent to the study manager for use approval prior to 2.4.2. Definition of Harm and Harm Duration
AHRQ’s version 1.2 in this study, but no response was
The level of harm and harm duration were assessed for
received, so the contents are described in the related table.
domestic patient safety incidents using the criteria presented by
The AHRQ’s version 1.2 is intended to evaluate harm and NCC MERP [20] and classified as follows; first, the level of
harm duration for each of the nine patient safety scenarios. harm is near miss (A): an environment that can cause a
Each scenario includes (1) incorrect administration route, (2) hazardous event, such as unorganized medical equipment, near
lacerations of body parts, (3) allergic reactions to contrast miss (B): an incident occurred but did not reach the patient, and
agents, (4) abdominal infections, (5) mislabeled samples, (6) no harm event (C): an incident occurred, but no harm to the
chest tube drainage, (8) overdose, and (9) incorrect time patient and no additional monitoring required; no harm event
administration; in this study, actual cases related to this were (D): if an incident has occurred and no harm will occur to the
selected by referring to the Korean Association of Hospital patient, or additional intervention is required to prevent harm,
Nurses “Patient Safety Incident Cases and Prevention” [19]. adverse event (E): when an incident occurs that causes
temporary harm to the patient and requires additional treatment
After selecting the scenario, three experts and one
or intervention, adverse event (F): An accident that causes
university professor working in relation to patient safety work
temporary harm to the patient and necessitates short or long-
at a medical institution reviewed each topic and whether the
term hospitalization, and a sentinel event: an incident that
selected scenario was appropriate, and then conducted a study.
causes near-death or permanent harm to the patient, or the
As the general characteristics of the subject, questions such patient died, or the event contributed to the patient’s death.
as gender, length of service at current workplace, current When an incident occurs and temporary harm to the patient
department, job satisfaction in current department, completion lasts more than one year, it is classified as permanent,
of patient safety education in current department, patient safety temporary harm to the patient and lasts less than one year, and
education method, and content of patient safety education were unknown.
4 The Open Nursing Journal, 2023, Volume 17 Lee and Shin

2.5. Statistical Analysis the online training and survey were completed.
SPSS 26.0 was used to analyze the collected data (SPSS 3. RESULTS
Inc, Chicago, IL, USA). First, the general characteristics of the
study subjects were calculated, as well as the frequency of 3.1. General Characteristics of Research Subjects
respondents for each scenario before and after education. The
Fleiss' kappa value was then calculated by estimating the All of the subjects in this study were female, and the
general ward was the most common work department,
interrater agreement for each scenario among respondents
accounting for 52.3% of the subjects. When asked how long
before and after education. The agreement rates among
they had been at their current job, 33.8% said 5 years or more
respondents were: 0.81–1.00 for perfect agreement, 0.61–0.8
and less than 10 years, 58.8% said more than 10 years, and the
for significant agreement, 0.41–0.60 for moderate agreement,
majority of subjects said more than 5 years.
0.21–0.40 for slight agreement, 0.01–0.20 for slight agreement
and a score of 0 or less for almost no agreement. In terms of job satisfaction, “satisfied” was the highest at
70.8%. It was discovered that all of the subjects received
2.6. Ethical Consideration patient safety education at their current workplace, with 86.3%
receiving “certificated brochure training” and 70.8% receiving
Before proceeding with the research, approval was
“theoretical lectures.” Patient safety education content, was
obtained from the IRB of the institution to which the research
90.8% “incident reporting procedures,” 84.6% “understanding
subjects belonged. Following that, the purpose of the study was
patient safety,” and 81.5% “time and method identification.”
explained to the hospital’s nursing headquarters, and the study
At 64.6%, patient safety education time was greater than one
was carried out with cooperation. hour but less than four hours (Table 1).
The researcher explained the purpose and method of the
study to the research subjects who participated in the 2018 3.2. Patient Safety Incident Experience of Research
study through an explanation for the online education and Subjects
survey. After collecting e-mail addresses for online education The majority of study subjects (92.3%) had experience
and re-survey, the relevant web address was sent if the subject with PSEs, with “no harm safety event” 83.3% and “near miss”
voluntarily expressed their intention to participate. Prior to the 66.7% having the highest duplicate responses to experienced
start of online education, participants could only participate in PSEs. When asked if they had any experience reporting PSEs,
the education and survey if they agreed voluntarily. 83.1% said they had, with the most common types of patient
Furthermore, to protect personal information, the collected e- safety incidents being “fall” 83.3% and “medication” 66.7%
mail addresses of the subjects were discarded immediately after (Table 2).

Table 1. Participants’ General Characteristics.

Variables Categories n (%)

Male 0(0.0)
Gender
Female 65(100.0)
General ward 34(52.3)
Special ward (ICU, ER, etc) 11(16.9)
Current department
Outpatient department 16(24.6)
Others+ 4(6.2)
<1 0(0.0)
≥1–<5 5(7.7)
Working period at current job (year)
≥5–<10 22(33.8)
≥10 38(58.8)
Very satisfied 2(3.1)
Satisfied 46(70.8)
Job satisfaction at current job
Unsatisfied 17(26.2)
Very unsatisfied 0(0.0)
Yes 65(100.0)
Whether or not getting a patient safety training at current job
No 0(0.0)
Theoretical lectures 46(70.8)
Case-based discussion training 5(7.7)
Method of a patient safety training (duplication check) Certificated brochure training 56(86.3)
Department conveying training 37(56.9)
Others++ 2(3.1)
Comparison of Perception Differences among Nurses The Open Nursing Journal, 2023, Volume 17 5

(Table ) contd.....
Variables Categories n (%)
Understanding patient safety 55(84.6)
Time and method of patient identification 53(81.5)
Grade and criteria of patient safety incident reporting 51(78.5)
Contents of a patient safety training (duplication check) Incident reporting procedures 59(90.8)
Inpatient care management 23(35.4)
Activating near miss reporting 24(36.9)
Others 0(0.0)
<1 19(29.2)
≥1–<4 42(64.6)
Time of a patient safety training (hour) ≥4–<8 3(4.6)
≥8 1(1.5)
Missing 2(0.7)
Yes 32(49.2)
Whether or not participating events§ of a patient safety at current job
No 33(50.8)
Note: +Operating room, ++ Cyber training, practical training, § Special lecture, seminar, Campaign, etc.

Table 2. Participants’ experience of patient safety accident.

Categories n (%)
Experience of patient safety accident at current job -
Yes 60(92.3)
No 5(7.7)
Type of patient safety accident (duplication check) -
Near miss 40(66.7)
No harm safety event 50(83.3)
Mild/Moderate/Severe safety event 28(46.7)
Sentinel event 13(21.7)
Reporting on patient safety accidents -
Yes 54(83.1)
No 6(9.2))
Missing 5(7.7)
Types of patient safety accidents experienced (duplication check) -
Surgery 4(6.7)
Delivery 0(0.0)
Treatment procedure 8(13.3)
Anesthesia 0(0.0)
Clinical examination 7(11.7)
Blood transfusion 0(0.0)
Medication 40(66.7)
Infection 1(1.7)
Computerized disorder 0(0.0)
Medical equipment/Medical device 4(6.7)
Hospital meal 3(5.0)
Fall 50(83.3)
Treatment material contamination /failure 3(5.0)
Suicide/Self-harm 7(11.7)
Other 1(1.7)

3.3. Agreement of Harm and Harm Duration after distribution of respondents by scenario were the 'Medication
Education given via wrong route' scenario. Before education, it was
Tables 3 and 4 show the results of the study subjects” 46.6% of “mild harm” and 24.8% of “moderate harm,” but
evaluations of harm and duration of harm for each scenario. after education, it was 52.3% of “mild harm” and 27.7% of “no
The items that showed changes before and after training in the harm.”
6 The Open Nursing Journal, 2023, Volume 17 Lee and Shin

Table 3. Breakdown of Harm scale Assigned to each Scenario (Before vs. After education).

Frequency of Respondents (%)

Near Miss Near Miss No Harm Mild Harm Moderate Harm Severe Harm
- Sentinel Event
Scenario (A) (B) (C) (D) (E) (F)
Before 3 (4.6) 2 (3.1) 4 (6.2) 31 (47.7) 15 (23.1) 5 (7.7) 5 (7.7)
1. Medication given via wrong route
After 1 (1.5) 0 (0.0) 18 (27.7) 34 (52.3) 11 (16.9) 1 (1.5) 0 (0.0)
Before 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.8) 19 (29.2) 31 (47.7) 15 (23.1)
2. Body part laceration during surgery
After 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 18 (27.7) 40 (61.5) 7 (10.8)
Before 0 (0.0) 0 (0.0) 1 (1.5) 2 (3.1) 25 (38.5) 33 (50.8) 4 (6.2)
3. Contrast allergy
After 1 (1.5) 0 (0.0) 0 (0.0) 6 (9.2) 30 (46.2) 28 (43.1) 0 (0.0)
Before 0 (0.0) 1 (1.5) 0 (0.0) 3 (4.6) 3 (4.6) 35 (53.8) 23 (35.4)
4. Abdominal site infection
After 2 (3.1) 0 (0.0) 0 (0.0) 0 (0.0) 3 (4.6) 41 (63.1) 19 (29.2)
Before 0 (0.0) 0 (0.0) 1 (1.5) 0 (0.0) 29 (44.6) 17 (26.2) 18 (27.7)
5. Mislabeled specimen
After 0 (0.0) 0 (0.0) 0 (0.0) 5 (7.7) 35 (53.8) 10 (15.4) 15 (23.1)
Before 0 (0.0) 1 (1.5) 0 (0.0) 0 (0.0) 1 (1.5) 14 (21.5) 49 (75.4)
6. Wrong site surgery
After 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 3 (4.6) 4 (6.2) 58 (89.2)
Before 0 (0.0) 3 (4.6) 11 (16.9) 31 (47.7) 17 (26.2) 1 (1.5) 2 (3.1)
7. Chest tube drain
After 1 (1.5) 3 (4.6) 21 (32.3) 27 (41.5) 13 (20.0) 0 (0.0) 0 (0.0)
Before 2 (3.1) 1 (1.5) 3 (4.6) 34 (52.3) 23 (35.4) 0 (0.0) 2 (3.1)
8. Medication overdose
After 0 (0.0) 1 (1.5) 1 (1.5) 41 (63.1) 21 (32.3) 1 (1.5) 0 (0.0)
Before 2 (3.1) 6 (9.2) 26 (40.0) 27 (41.5) 3 (4.6) 1 (1.5) 0 (0.0)
9. Medication given at the wrong time
After 2 (3.1) 11 (16.9) 33 (50.8) 18 (27.7) 1 (1.5) 0 (0.0) 0 (0.0)

Table 4. Breakdown of Harm duration Assigned to each Scenario (Before vs. After education).

Frequency of Respondents (%)

-
- Permanent Temporary Unknown
Before* 5 (7.7) 29 (44.6) 29 (44.6)
1. Medication given via wrong route
After 0 (0.0) 36 (55.4) 29 (44.6)
Before 30 (46.2) 15 (23.1) 20 (30.7)
2. Body part laceration during surgery
After 22 (33.8) 27 (41.5) 16 (24.6)
Before 1 (1.5) 53 (81.5) 11 (16.9)
3. Contrast allergy
After 0 (0.0) 54 (83.1) 11 (16.9)
Before 25 (38.5) 14 (21.5) 26 (40.0)
4. Abdominal site infection
After 25 (38.5) 10 (15.4) 30 (46.2)
Before 6 (9.2) 34 (52.3) 25 (38.5)
5. Mislabeled specimen
After 7 (10.8) 38 (58.8) 20 (30.8)
Before 61 (93.9) 1 (1.5) 3 (4.6)
6. Wrong site surgery
After 64 (98.5) 0 (0.0) 1 (1.5)
Before 1 (1.5) 58 (89.2) 6 (9.2)
7. Chest tube drain
After 0 (0.0) 57 (87.7) 8 (12.3)
Before 2 (3.1) 41 (63.1) 22 (33.9)
8. Medication overdose
After 0 (0.0) 41 (63.1) 24 (36.9)
Before 0 (0.0) 36 (55.4) 29 (44.6)
9. Medication given at the wrong time
After 0 (0.0) 33 (50.8) 32 (49.2)
Note: * missing = 2
** Adapted from the work of T. Williams et al. The reliability of AHRQ Common Format Harm Scales in rating patient safety events. J Patient Saf 2015 Mar;11(1):52-9.

Following that, in the “Wrong site surgery” scenario, evaluation before and after education (Tables 3 & 4).
71.4% of “Sentinel Event” and 27.1% of “Severe Harm” were
evaluated as 89.2% of “Sentinel Event” after education. The Fleiss' kappa value of respondents’ agreement showed
Finally, in the “Medication given at the wrong time” scenario, that the harm assessment improved from k = 0.23 before
the “Near Miss” rate before education was 9.8% but increased education to k = 0.31 after education. There was little change
to 16.9% after education. In contrast to the harm assessment, in the harm period from k=0.29 before education to k = 0.30
there were no significant differences in the harm period after education (Table 5).
Comparison of Perception Differences among Nurses The Open Nursing Journal, 2023, Volume 17 7

Table 5. Comparison of the harm scale and harm duration agreement values.

Overall Kappa Value*

-
Before Education After Education
Harm scale .23 .31
Harm duration .29 .30
Note: * Fleiss’ Kappa index of agreement.

4. DISCUSSION before and k=0.31 after education, which was lower than the
previous study [3, 4]. Even though the subject was a patient
Because patient safety is dependent on information about
safety manager of a medical institution, in the case of a study
PSEs reported in the patient safety incident reporting system,
conducted under the supervision of UHC in the United States,
the harm of the incident, and the factors that caused the
there was a significant difference in the perception of patient
occurrence, harm assessment of patient safety incident
safety events between domestic studies that targeted general
reporters standardization is critical. In particular, since the
nurses.
Patient Safety Act was enacted in Korea in 2016, patient safety
incidents are managed with a focus on reported incidents, and The reasons for perception among healthcare professionals,
in order to promote qualitative improvement, with voluntary including nurses, doctors, and pharmacists, are due to the
reporting of each patient safety incident, a standardized diversity of each healthcare professional's educational
evaluation of harm and factors is very necessary. As a result, background [13], differences in understanding of harm items
the accuracy of the classification system of PSEs reported in [10], and whether the format in which PSEs are reported is
the patient safety system is critical, but the findings of previous paper-based or web-based [12]. Finally, it was reported that
studies [3, 4, 15] show that there is a significant difference in unclear guidelines for the definition and knowledge structure of
nurses' perception of PSEs. each harm item in harm assessment may lead to a biased view
of harm assessment [3, 21].
The AHRQ Common Format Harm Scale evaluation tool
used in this study was created as part of a project to standardize Furthermore, when harm assessment is conducted using
the assessment of harm in PSEs in the United States. The standardized evaluation tools such as the AHRQ’S version 1.1,
assessment tool has created a scenario for determining the the low consistency value between inter-rater is inconsistent
extent of harm and the duration of the harm. In the harm with the understanding of harm items as in other studies, and
assessment criteria, a ‘near miss’ is a concept that defines the selection of risk scores among evaluators is very subjective,
situations and events that did not reach the patient but may the necessity of education was suggested as a way to reduce
cause 'harm' in the future, whereas a 'no harm event' and an this [3, 22].
'adverse event' are concepts that are divided by 'harm' among
Until now, no studies have reported the results of training
the events that did reach the patient. The harm period is a
medical personnel to increase the consistency of evaluators of
concept that divides whether the harm occurs for more than one
harm assessment in reporting actual PSEs, but assessment
year.
guidelines have been developed [16]. In this study, an
The inter-rater harm confidence value for the AHRQ’s educational video was created in order to improve the
version 1.1 PSE scenario for 921 institutional quality, risk, and consistency of nurses' harm assessment by referring to existing
safety managers at UHC PSN (Patient Safety Net®) was guidelines, and then the consistency value was compared after
k=0.51, and reliability measurements for version 1.2 with an education.
updated version 1.1 were conducted on 13,280 UHC PSN After explaining the developed harm classification system
managers, with a moderate reliability value of k=0.47 between and scale using the example of PSEs in the educational video,
inter-rater [4]. nurses who participated in the education were organized to
The same tool AHRQ’S versions 1.1 and 1.2 measured conduct harm assessment of PSEs based on the actual
inter-measurer reliability values for the risk of PSE scenarios educational contents. Due to the COVID-19 situation, online
for nurses working in cancer centers, with version 1.1 k=0.45 education was conducted, but nurses who participated in the
and version 1.2 k=0.48, indicating moderate reliability [3]. In education were asked to participate in a harm assessment
survey after watching educational videos.
another study, doctors, nurses, pharmacists, and other
healthcare professionals who are the primary reporters of PSEs The study found that the Fleiss' kappa value, which
in the clinical field were evaluated for nine PSE scenarios, and represents agreement among raters in harm assessment before
it was found that the nurse group had a higher reliability education, was 0.23 but improved to 0.31 after education.
difference in patient safety case assessment than the other According to the findings of this study, nurses who play a
healthcare professionals [10]. significant role in patient safety incident reporting can perform
well as nurses by providing feedback through continuous
In this study, general nurses working at advanced general monitoring of patient safety reports and practical education on
hospitals and the national cancer center in Korea were reported harm assessment. It stated that an educational program that
as having k=0.21 as a result of evaluating the harm of PSEs actively assists in achieving this goal is required. Furthermore,
between nurses using a questionnaire tool developed based on it is thought necessary to create a tool for standardizing harm
domestic PSEs [15]. This study also revealed that k=0.23 assessment at the level of a multidisciplinary society.
8 The Open Nursing Journal, 2023, Volume 17 Lee and Shin

There are several limitations in generalizing the results of PSEs = Patient Safety Events
this study. First, as the study was conducted only for nurses NCC MERP = National coordinating council for medication error
working in one medical institution, there are limitations in its reporting and prevention
application to other ethnicities or countries. Second, this study
was not able to conduct a control comparison due to the same ETHICS APPROVAL AND CONSENT TO
group comparison before and after education on harm PARTICIPATE
assessment, and all subjects were female nurses who were in Before proceeding with the research, the Institutional
charge of nursing cancer patients as the institution where this Review Board (IRB) of the institution to which the research
study was conducted was a National Cancer Center. subjects belonged obtained ethical approval (IRB No. NCC
Furthermore, when the research was divided into before and 2020-0049). Prior to the start of online education, participants
after education, the research subjects were asked to read and could only participate in the education and survey if they
evaluate each scenario immediately without separate education agreed voluntarily.
to explain the definition and standards of harm and harm
period, it is believed that differences in education and HUMAN AND ANIMAL RIGHTS
experience between individuals could have been greatly
This study did not involve any animals. All humans were
reflected. Depending on their work experience other than
used in accordance with the ethical standards of the committee
online video education in this study, the survey results of the
in charge of human research with the Bioethics and Safety Act
research subjects who participated in the education may have
and with the Helsinki Declaration of 1975, as revised in 2013.
resulted from the accumulation of experience related to
reporting related patient safety incidents, etc. Finally, it may
CONSENT FOR PUBLICATION
have influenced the results of this study due to the difference in
the survey method before and after education and the number After explaining the purpose of the study and assuring the
of subjects in the second survey decreased significantly due to subject about the privacy and confidentiality of the obtained
those who first participated in the survey who resigned from data, informed consent was obtained from the participants in
their jobs or refused to participate in the survey that conducts this study.
education, and future studies require to minimize the gap
between studies. AVAILABILITY OF DATA AND MATERIALS

5. IMPLICATION FOR NURSING The data sets used in the current study are available upon
reasonable request from the corresponding author [F.S].
The finding of this study has shown that harm assessment
education was effective among ways to increase the FUNDING
consistency of harm assessment of PSEs among nurses, which
is important information for patient risk management of This study was funded by the MOE (Ministry of
institutions. Based on this, it can be used as a basic material for Education) through the NRF (National Research Foundation of
developing standardized educational materials that can help Korea) (NRF2018R1D1A3B07049955).
nurses assess the harm of PSEs when reporting PSEs in the
future and educational methods using them. CONFLICT OF INTEREST
There are no conflicts of interest reported by the authors.
CONCLUSION AND RECOMMENDATIONS
This article’s content and writing are solely the responsibility
In this study, accurate harm assessment of medical staff is of the authors.
critical in the patient safety incident reporting system, but there
is a perception gap due to a lack of education. A study was ACKNOWLEDGEMENTS
conducted to develop guidelines for education and to
understand the difference in the effectiveness of education in The contributions of all participants in this study are
the degree of consistency in harm assessment before and after greatly appreciated. We would like to thank Editage
education using them in order to increase the consistency of the (www.editage.co.kr) for English language editing.
most active nurse's harm assessment in patient safety event REFERENCES
reporting in the clinical field.
[1] Institute of Medicine. Patient safety: Achieving a new standard of care.
According to the findings of this study, harm assessment Washington, DC: National Academy Press 2003.
education in the patient safety classification system can [2] World Health Organization. World Alliance for Patient Safety:
forward programme 2005. Geneva: World Health Organization 2004.
increase the consistency rate in harm assessment among nurses. [3] Abbasi T, Adornetto-Garcia D, Johnston PA, Segovia JH, Summers B.
As a result, it is suggested that a related education program be Accuracy of harm scores entered into an event reporting system. J
developed that can actively assist patients in performing their Nurs Adm 2015; 45(4): 218-25.
safety obligations as nurses through the expansion of offline [http://dx.doi.org/10.1097/NNA.0000000000000188] [PMID:
25803804]
field education using harm assessment guidelines and [4] Williams T, Szekendi M, Pavkovic S, Clevenger W, Cerese J. The
casebooks of patient safety incidents. reliability of AHRQ common format harm scales in rating patient
safety events. J Patient Saf 2015; 11(1): 52-9.
LIST OF ABBREVIATIONS [http://dx.doi.org/10.1097/PTS.0b013e3182948ef9] [PMID:
24080718]
[5] Clancy CM. Common formats allow uniform collection and reporting
WHO = World health organization
of patient safety data by patient safety organizations. Am J Med Qual
Comparison of Perception Differences among Nurses The Open Nursing Journal, 2023, Volume 17 9

2010; 25(1): 73-5. reporting: A collaborative hospital study. Qual Saf Health Care 2006;
[http://dx.doi.org/10.1177/1062860609352438] [PMID: 20042765] 15(1): 39-43.
[6] WHO. Conceptual framework for the international classification for [http://dx.doi.org/10.1136/qshc.2004.012559] [PMID: 16456208]
patient safety version 1.1: final technical report January. 2009. [14] Patrician PA, Brosch LR. Medication error reporting and the work
[7] Forrey RA, Pedersen CA, Schneider PJ. Interrater agreement with a environment in a military setting. J Nurs Care Qual 2009; 24(4):
standard scheme for classifying medication errors. Am J Health Syst 277-86.
Pharm 2007; 64(2): 175-81. [http://dx.doi.org/10.1097/NCQ.0b013e3181afa4cb] [PMID:
[http://dx.doi.org/10.2146/ajhp060109] [PMID: 17215468] 19584755]
[8] Classen DC, Resar R, Griffin F, et al. Global trigger tool shows that [15] Lee K, Yoon K, Yoon B, Shin E. Differences in the perception of
adverse events in hospitals may be ten times greater than previously harm assessment among nurses in the patient safety classification
measured. Health Aff 2011; 30(4): 581-9. system. PLoS One 2020; 15(12): e0243583.
[http://dx.doi.org/10.1377/hlthaff.2011.0190] [PMID: 21471476] [http://dx.doi.org/10.1371/journal.pone.0243583]
[9] Hanskamp-Sebregts M, Zegers M, Vincent C, van Gurp PJ, de Vet [16] AHRQ. Users guide version 1.2 AHRQ Common Formats for Patient
HCW, Wollersheim H. Measurement of patient safety: A systematic Safety Organization. 2013.
review of the reliability and validity of adverse event detection with [17] HPI. SEC & SSER Patient safety measurement system for healthcare.
record review. BMJ Open 2016; 6(8): e011078. 2011.
[http://dx.doi.org/10.1136/bmjopen-2016-011078] [PMID: 27550650] [18] ASHRM. Serious safety events: A focus on harm classification:
[10] Williams SD, Ashcroft DM. Medication errors: How reliable are the Deviation in care as link. 2014.
severity ratings reported to the national reporting and learning system? [19] Chae G, Noa W, Park M, Jung E, Cho E, Jung M. Prevention and risk
Int J Qual Health Care 2009; 21(5): 316-20. management for patient safety. In: Hospital Nurses Association. 2014;
[http://dx.doi.org/10.1093/intqhc/mzp034] [PMID: 19679598] pp. 26-105.
[11] Mayo AM, Duncan D. Nurse perceptions of medication errors: What [20] NCC MERP Index for categorizing medication errors algorithm
we need to know for patient safety. J Nurs Care Qual 2004; 19(3): [Internet]: NCC MERP; c1996-2001. Available from:
209-17. https://www.nccmerp.org/types-medication-errors (revised 2001 Feb
[http://dx.doi.org/10.1097/00001786-200407000-00007] [PMID: 20)
15326990] [21] Liang C, Gong Y. Knowledge representation in patient safety
[12] Liang C, Gong Y. Predicting harm scores from patient safety event reporting: An ontological approach. J Data Inf Sci 2017; 1(2): 75-91.
reports. Stud Health Technol Inform 2017; 245: 1075-9. [http://dx.doi.org/10.20309/jdis.201615]
[http://dx.doi.org/10.3233/978-1-61499-830-3-1075] [PMID: [22] Feeser VR, Jackson A, Senn R, et al. Patient safety event reporting and
29295267] opportunities for emergency medicine resident education. West J
[13] Evans SM, Berry JG, Smith BJ, et al. Attitudes and barriers to incident Emerg Med 2020; 21(4): 900-5.
[http://dx.doi.org/10.5811/westjem.2020.3.46018] [PMID: 32726262]

© 2023 The Author(s). Published by Bentham Science Publisher.

This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is
available at: https://creativecommons.org/licenses/by/4.0/legalcode. This license permits unrestricted use, distribution, and reproduction in any medium, provided the
original author and source are credited.