This is a repository copy of Strengths and limitations of Early Warning Scores: a

systematic review and narrative synthesis.

White Rose Research Online URL for this paper:

http://eprints.whiterose.ac.uk/121108/

Version: Accepted Version

Article:
Downey, CL orcid.org/0000-0001-9818-8002, Tahir, W, Randell, R
orcid.org/0000-0002-5856-4912 et al. (2 more authors) (2017) Strengths and limitations of
Early Warning Scores: a systematic review and narrative synthesis. International Journal
of Nursing Studies, 76. pp. 106-119. ISSN 0020-7489

https://doi.org/10.1016/j.ijnurstu.2017.09.003

(c) 2017, Elsevier Ltd. This manuscript version is made available under the CC BY-NC-ND
4.0 license https://creativecommons.org/licenses/by-nc-nd/4.0/

Reuse
Items deposited in White Rose Research Online are protected by copyright, with all rights reserved unless
indicated otherwise. They may be downloaded and/or printed for private study, or other acts as permitted by
national copyright laws. The publisher or other rights holders may allow further reproduction and re-use of
the full text version. This is indicated by the licence information on the White Rose Research Online record
for the item.

Takedown
If you consider content in White Rose Research Online to be in breach of UK law, please notify us by
emailing [email protected] including the URL of the record and the reason for the withdrawal request.

[email protected]
https://eprints.whiterose.ac.uk/
Strengths and limitations of Early Warning Scores: a
systematic review and narrative synthesis
CL Downey1, W Tahir1, R Randell2, JM Brown3, DG Jayne1

1. Leeds Institute of Biomedical & Clinical Sciences, Clinical Sciences Building,

St. James’s University Hospital, University of Leeds, Leeds, LS9 7TF
2. School of Healthcare, Baines Wing, University of Leeds, Leeds, LS2 9JT
3. Leeds Institute of Clinical Trials Research, Worsley Building, University of
Leeds, Leeds, LS2 9NL.

Correspondence to:
Miss Candice L Downey
Level 7, Clinical Sciences Building
St James’s University Hospital
Leeds
LS9 7TF
Email: [email protected]
Tel: 0113 2065281
Fax: 0113 2065281

Category: Systematic review

Originality:
This article is an original work, has not been published before, and is not being
considered for publication elsewhere in its final form, in either printed or
electronic media. It is not based on any previous communication to a society or
meeting.

Conflicts of Interest:
There are no known conflicts of interest associated with this review and there has
been no significant financial support for this work that could have influenced its
outcome.

Sources of support:
This review is independent research arising from a Doctoral Research Fellowship
(Candice Downey, DRF-2016-09-03) supported by the National Institute for Health
Research. The views expressed in this publication are those of the author and not
necessarily those of the NHS, the National Institute for Health Research, Health
Education England or the Department of Health. DGJ received funding support
through an NIHR Research Professorship.

1
Abstract

Background
Early warning scores are widely used to identify deteriorating patients. Whilst their
ability to predict clinical outcomes has been extensively reviewed, there has been no
attempt to summarise the overall strengths and limitations of these scores for
patients, staff and systems. This review aims to address this gap in the literature to
guide improvements for the optimization of patient safety.

Methods
A systematic review was conducted of MEDLINE®, PubMed, CINAHL and The
Cochrane Library in September 2016. The citations and reference lists of selected
studies were reviewed for completeness. Studies were included if they evaluated
vital signs monitoring in adult human subjects. Studies regarding the paediatric
population were excluded, as were studies describing the development or validation
of monitoring models. A narrative synthesis of qualitative, quantitative and mixed-
methods studies was undertaken.

Findings
232 studies met the inclusion criteria. Twelve themes were identified from synthesis
of the data: Strengths of early warning scores included their prediction value,
influence on clinical outcomes, cross-specialty application, international relevance,
interaction with other variables, impact on communication and opportunity for
automation. Limitations included their sensitivity, the need for practitioner
engagement, the need for reaction to escalation and the need for clinical judgment,
and the intermittent nature of recording.
Early warning scores are known to have good predictive value for patient
deterioration and have been shown to improve patient outcomes across a variety of
specialties and international settings. This is partly due to their facilitation of
communication between healthcare workers.
There is evidence that the prediction value of generic early warning scores suffers in
comparison to specialty-specific scores, and that their sensitivity can be improved by
the addition of other variables. They are also prone to inaccurate recording and user
error, which can be partly overcome by automation.

Conclusions
Early warning scores provide the right language and environment for the timely
escalation of patient care. They are limited by their intermittent and user-dependent
nature, which can be partially overcome by automation and new continuous
monitoring technologies, although clinical judgment remains paramount.

2
Contribution of Paper

What is already known about the topic?

 Early warning scores are widely used to identify deteriorating patients
 Early warning scores have strengths and limitations which influence their
effectiveness

What this paper adds

 Early warning scores can be used across a range of specialties and
international settings
 Early warning scores facilitate communication by providing a common
language
 They are limited by their intermittent and user-dependent nature, which must
be taken into account when interpreting them.

3
Introduction

The early warning score system is predicated on the idea that derangements in
simple physiological observations can identify hospital inpatients at high risk of
deterioration.1 Prodromal warning signs such as increased respiratory rate or
decreased blood pressure precede critical illness,2 and early recognition of these
events presents an opportunity for decreasing mortality.3 The early warning score
system allows the user to record and respond to multiple parameters simultaneously,
so that subtle changes in vital signs can be used to initiate early emergency
management of the patient to reverse the abnormal physiological decline or prompt
admission to a critical care area.2
Early warning scores have been widely adopted internationally, and different
versions exist. A number of reviews have examined the impact of early warning
scores on patient outcomes; however, there exists no formal literature review
regarding the overall strengths and limitations of early warning scores for patients,
staff and systems. This review aims to address this knowledge gap and provide an
overview of current systems, highlighting the benefits and identifying areas for future
improvement.

Methods

Study design
A systematic review methodology was adopted for the study, employing the
principles and methods provided by the Centre for Reviews and Dissemination
guidelines and following the PRISMA statement. A narrative synthesis approach was
chosen to synthesise the diverse range of selected studies in a structured manner,
following the European Social Research Council Guidance on the Conduct of
Narrative Synthesis in Systematic Reviews.4

Search strategy
A systematic review of the scientific literature was performed by CD. MEDLINE®,
PubMed, CINAHL and The Cochrane Library databases were searched for articles
published from the dates of inception of the databases (the earliest being 1947) to
September 2016. For on-going trials, Current Controlled Trials and ClinicalTrials.gov
were searched.
In order to extract all available data regarding early warning score systems, the
search strategy was kept necessarily broad. The search strategy was devised with
the help of a Research Support Advisor at the Leeds University Library, using both
MeSH and/or keyword search terms according to the database.
The search strategy for PubMed is detailed below and further details are provided in
Supplementary Material.
1. “Warning scor*”[tiab] including ("warning score" OR "warning score
calculation" OR "warning score levels" OR "warning score system" OR

4
 "warning score systems" OR "warning score value" OR "warning scores" OR
"warning scoring" OR "warning scoring system" OR "warning scoring
systems")
2. “Monitoring,Physiologic/”[Mesh]
3. #1 AND #2
4. #3 NOT ("Adolescent"[Mesh] OR "Child"[Mesh] OR "Infant"[Mesh])
In addition, citations and reference lists of selected studies were reviewed to identify
any missed papers.

Identifying relevant papers

Publications were selected in two phases: first by review of title and abstract and
then by full text review by CD. The articles were then independently verified by WT.
Studies were included if they evaluated vital signs monitoring in adult human
subjects. Study selection was not limited by the score used or the outcomes
measured. Selection was not limited to peer-reviewed publications and included
grey literature such as editorials and opinion pieces in order to provide insight into
stakeholders’ perspectives of early warning scores. Qualitative, quantitative and
mixed-methods studies were included. The search was not limited by year of
publication but papers had to be written in English due to lack of translation
resources.
Studies regarding the paediatric population were excluded, as were studies
describing the development or validation of monitoring models.

Data extraction and analysis

A narrative synthesis approach was chosen to synthesise the diverse range of
studies in a structured manner, following the European Social Research Council
Guidance on the Conduct of Narrative Synthesis in Systematic Reviews.4 Briefly,
studies were tabulated and grouped by outcome measure, setting and population.
Patterns were identified and translated to themes, which were further refined using
an iterative process. The evidence was synthesised to provide a meaningful
narrative, relevant to the research question.
Full details of selection, data extraction and analysis has been provided as
Supplementary Material.

Findings

The search identified 825 papers (285 Medline, 359 PubMed, 176 CINAHL and 5
Cochrane). Duplicates were eliminated. 232 papers met the inclusion criteria. A
flow diagram of the search process is shown in Figure 1.
There was 100% inter-rater agreement between CD and WT.

Themes

5
Themes identified included: prediction value; influence on clinical outcomes;
interaction with other variables; cross-specialty application; international relevance;
impact on communication; opportunity for automation; sensitivity; need for
practitioner engagement, reaction to escalation and clinical judgment; and
intermittent nature of recording. Tables 1 and 2 provide a summary of the most
relevant articles within each theme, grouped into ‘Strengths’ and ‘Limitations’. A full
list of selected articles is provided in Supplementary Materials.

Figure 1: PRISMA diagram summarising selection process

Summary of early warning score systems

Popular scores used internationally include the Modified Early Warning Score
(MEWS) and VitalPAC Early Warning Score (VIEWS), which have both been
validated as good predictors of mortality during hospitalization.3 However, individual
hospitals have introduced their own local early warning scores, such as the Chelsea
Early Warning Score (CEWS), introduced by the Chelsea and Westminster Hospital,
UK.5 This gave rise to variation in the reproducibility of different track-and-trigger
warning systems, leading to calls for the adoption of a national system.
In 2012, The Royal College of Physicians developed the National Early Warning
Score (NEWS) as a standardised approach to assessment and response to critical
illness.6 The NEWS was shown to be independently valid,7 and surveys showed that
staff found the NEWS was easy to use, did not increase workload and enhanced
their ability to identify deteriorating patients.8
NEWS has been widely adopted throughout the UK National Health Service.

Strengths of early warning scores

6
Prediction value
Early warning scores have consistently been found to accurately predict adverse
outcomes in a number of different populations. Despite being developed for general
medical hospital admissions, a recent retrospective study of 35 174 surgical
admissions found that NEWS discriminated deterioration in non-elective surgical
patients at least as well as in non-elective medical patients 9.

7
Theme (Strengths) Publications Setting Methodology Participants Early w arning Outcom e Findings
identified score m easure/s
w ithin theme

Prediction value Kovacs et al. Hospital Real-time Medical and surgical NEWS via Cardiac arrest, High prediction rates for death and ICU
(2016) 9 inpatients at observational admissions (n=87 VitalPAC death and admission; low er for cardiac arrest
a single study 399) unanticipated ICU
NHS Trust admission

Churpek et al. Hospital Nested case- Ward patients w ho MEWS Maximum MEWS, By 48 h prior to cardiac arrest, the MEWS w as
(2012) 10 inpatients at control study experienced cardiac individual higher in cases (P = 0.005) than controls
a single arrest (n=88) component vital
centre signs and other
and matched
predictors
controls (n=352)

Lee, Choi General Retrospective General w ard MEWS ICU transfer MEWS is an effective predictor of ICU transfer
(2014) 11 w ards at a observational patients w ith severe w ith optimum cutoff value 6
single study sepsis or septic
centre shock (n=100)
MEWS=>6 is an independent predictor of
Reini et al. A tertiary Prospective Patients admitted to MEWS Mortality, length of mortality and length of ICU stay, but not
(2012) 13 care observational ICU (n=518) stay, readmission to readmission.
general ICU study ICU
in a single
centre

Alraw i et al. Acute Real-time Acute medical MEWS In-patient mortality at Admission MEWS of 4-5 w as associated with
(2013) 14 Medical observational admissions from 7 days 12 times the odds of death; MEWS >6 had 21
Assessmen study nursing homes times the odds of death compared w ith those
t Unit at a (n=314) w ith a score of <1.
single
centre

Armagan et al. Emergency Prospective Patients presenting MEWS Death, hospital Patients w ith MEWS>4 w ere 35 times more
(2008) 15 Department observational to the Emergency admission, intensive likely to die in ED and 14 times more likely to
(ED) at a study Department (n=309) care unit (ICU) die in hospital than those presenting w ith a
single admission low -risk score. Those with MEWS =>5 w ere
centre 1.95 times more likely to be admitted to ICU

8
 Stark et al. Surgical Retrospective All surgical patients MEWS Death Maximum MEWS remained associated w ith
(2015) 17 w ards at a observational w ho experienced a death after multivariate analysis
single study “Code Blue” event
university (n=85)
hospital
All patients In-hospital mortality ,
Cei et al. 64-bedded Prospective, consecutively MEWS a combined outcome The risk of death w as incremental among all
(2009) 18 medical single centre, admitted over a of death and transfer the MEWS categories, as w ell as the risk of
w ard in a cohort study seven-month period to a higher level of the combined outcome of death and transfer.
public, non- (n=1107) care, length of stay The difference between length of stay was
teaching non-significant.
Hospital

Christensen et Emergency Retrospective A random sample of Bispebjerg Early Admission to ICU A BEWS > 5 is associated w ith a significantly
al. (2011) 19 Department observational emergency patients Warning Score and death w ithin 48 increased risk of ICU admission and death
(ED) at a study (n=300) (BEWS) hours of arrival at w ithin 48 hours of arrival.
single the ED
centre
Admission rates to
Peris et al. Surgical Retrospective Emergency surgical MEWS before and ICU and HDU After MEWS introduction, HDU admissions
(2012) 20 unit at a cohort study patients admitted after surgical (Patients w ith a significantly increased and ICU admissions
single before MEWS procedure MEWS 3 or 4 w ere significantly decreased. Mortality rate did not
centre introduction transferred to HDU, differ.
(controls, n=604) patients w ith MEWS
and after MEWS >= 5 w ere admitted
to ICU), mortality.
introduction
(intervention group,
n=478)

Influence on clinical Bokhari et al. ICU at a Retrospective Patients w ith Unspecified Survival to ICU Survival to ICU discharge increased from 44%
outcomes (2010) 23 single cohort study haematological discharge to 53% after the introduction of the
university malignancies intervention.
hospital admitted to ICU
before (n=27) and
after (n=105) use of
an early w arning
score and a critical
care outreach
nursing team

Moon et al. Single Retrospective Adult hospital MEWS Incidence of cardiac CCOS and MEWS w ere associated with
(2011) 24 tertiary audit admissions before arrest calls, in- significant reductions in the incidence of
referral (n=213 117) and hospital mortality, cardiac arrest calls (0.4% to 0.2%) and deaths
centre after (n=235 516) ICU admissions after per hospital admission (1.4% to 1.2%). Also
the introduction of cardiopulmonary reduced w ere the proportion of patients
MEWS charts and a resuscitation (CPR), admitted to ICU after CPR (3% to 2%) and
critical care outreach deaths after CPR their in-hospital mortality (52% to 42%).
service (CCOS)

9
 Adult patients
Bunkenborg et Medical and Prospective, non- admitted w ith >=24 MEWS Incidence of The adjusted unexpected patient mortality rate
al. (2014) 25 surgical randomised, hours length of unexpected patient w as significantly lower after the intervention
w ards at an before-and-after hospital stay before death (17 versus 61 per 100 adjusted patient years).
urban study (n=1870) and after
university (n=2234) the
hospital introduction of early
w arning scoring.
Adult patients
Drow er et al. Single Retrospective admitted before Adult Deterioration Incidence of in- The rate of cardiac arrests per 1000
(2013) 26 tertiary before-and-after (n=21 806) and after Detection System hospital adult admissions w as 4.67 before and 2.91 after the
teaching study (n=22 378) the (ADDS) cardiac arrests introduction of the intervention.
hospital introduction of an
early w arning score
system
Interaction w ith Heitz et al. Single Retrospective Patients admitted to MEWSMax All-cause mortality The inclusion of additional variables (mode of
other variables (2010) 63 tertiary observational hospital from the and higher care transport to ED, need for intravenous
hospital study Emergency utilization w ithin 24 antibiotics in ED, length of stay in the ED,
Department (n=280) hours gender) slightly improved the predictive ability
of MEWS.

Perera et al. Single Prospective Consecutive MEWS HDU/ICU admission, Combining MEWS w ith biochemical
(2011) 64 tertiary observational admissions to the cardio-respiratory parameters (C-reactive protein, albumin, w hite
hospital study Acute Medical Unit emergency/resuscita cell count, platelet count and haemoglobin)
(n=250) tion and death improved the sensitivity of prediction w hen
compared to MEWS alone.

Alraw i et al. Acute Real-time Acute medical MEWS In-patient mortality at Patients w ith a MEWS >6 had 21 times the
(2013) 14 Medical observational admissions from 7 days odds of death compared w ith those w ith a
Assessmen study nursing homes score of <1. An estimated glomerular filtration
t Unit at a (n=314) rate (eGFR) <30 ml/min/m2 w as associated
single w ith a 5-fold increase in the odds of death
centre w ithin 1 w eek2, compared w ith eGFR > 60
ml/min/m2. C-reactive protein (CRP) >100 mg/l
w as also associated with a 2.5 times higher
odds of death.

Cross specialty Silcock et al. Emergency Retrospective Unselected NEWS 48 hour and 30 day All three of the primary endpoints and the
application (2015) 48 ambulances cohort study prehospital patients mortality, ICU combined endpoint w ere associated with
transporting (n=1684) admission, higher NEWS scores.
patients to
a single
hospital
PMEWS scoring could have diverted 79
Challen and Single Retrospective, Patients presenting Physiological social Hospital admission patients safely from the emergency
Walter (2010) tertiary cohort study w ith ‘shortness of score (PMEWS) and need for department to alternative care providers.
49
hospital breath’ or ‘difficulty physiologically
breathing’ stabilizing treatment
transported to the

10
 ED by emergency in the emergency
medical services department
(n=215)

Essam et al. Ambulance Retrospective Patients attended by MEWS Percentage of MEWS had a minimal effect on transportation
(2015) 50 service at a before-and-after 19 volunteer patients transported or revisit rates. Scores w ere frequently not
single NHS study paramedics to hospital or treated calculated or recorded, or incorrectly
Trust (n=1932) at home and calculated.
revisited in 7 days
Single Acute medical
International Opio et al. resource- Retrospective patients admitted to VitalPAC Early Death w ithin 24 The discrimination of ViEWS in a resource
relevance (2013) 55 poor observational hospital (n=844) Warning Score hours of admission poor sub-Saharan Africa hospital is the same
hospital in study (ViEWS) as in the developed w orld
Uganda

Single
Asiimw e et al. resource- Retrospective Patients admitted to MEWS Probability of in- MEWS >=5 predicted mortality and
(2015) 56 poor cohort study hospital w ith sepsis hospital death complemented a novel prognostic index
hospital in (n=317)
Uganda

Baker et al. Single Prospective, Patients admitted to NEWS In-hospital mortality NEWS >=7 w as associated with 2.5 times the
(2015) 57 centre in observational ICU (n=269) odds of death.
Tanzania cohort study

There w as a positive correlation between EWS

Rylance et al. Tw o Prospective, Acute medical MEWS In-hospital mortality and risk of mortality
(2009) 58 resource- observational admissions (n=737)
limited study
hospitals in
Tanzania

Burch et al. Single Prospective, Medical patients MEWS Risks of hospital The proportion of patients admitted and those
(2008) 59 public observational presenting to the admission and in- w ho died in hospital increased significantly as
hospital in study emergency hospital death the MEWS score increased.
South department (n=790)
Africa

Kyriakos et al. Single Delphi study for Validation study MEWS Parameters and cut A MEWS for developing countries should
(2014) 60 hospital in the development points for the Cape record at least seven parameters.
South of the Cape Tow n Tow n MEWS chart Parameters and cut points differed from those
Africa MEWS chart in MEWS used in developed countries.

11
 Rosedale et Single Prospective Patients presenting South African Outcome in the ED SATS w as superior to the MEWS as a triage
al. (2011) 61 government cross-sectional to ED (n=589) Triage Score (death, hospital scoring system (4.4% vs 15.1% under-triage
hospital in study (SATS) vs MEWS admission or rate).
rural South discharge)
Africa

Wheeler et al. Single A prospective Adults admitted to MEWS vs HOTEL Mortality w ithin three MEWS and HOTEL lacked sensitivity and
(2013) 62 resource- cohort study medical w ards score days of admission specificity within the local population.
poor (n=302) (Hypotension,
hospital in Oxygen saturation,
Malaw i Temperature, ECG
abnormality, Loss
of independence)

Impact on Andrews and Single Interview s and 44 staff from one Generic early Staff opinion on the The Early Warning Score improves
communication Waterman university observations as surgical and one w arning score effectiveness of communication, empow ers nurses and
(2005) 65 hospital part of a general medical based on MEWS early w arning scores increases their confidence when reporting
grounded-theory w ard (30 nurses, 7 in detecting physiological deterioration to doctors.
approach doctors and 7 health physiological
care support deterioration
w orkers)

Neary et al. Single Questionnaire Convenience NEWS Staff opinion NEWS ‘empow ers nurses to more easily seek
(2015) 68 university study sample of 40 staff regarding the senior medical assistance’ and ‘avoids conflict’
hospital from general surgical strengths and pitfalls
w ards (27 doctors, after NEWS w as
13 nurses) introduced into
surgery

Opportunity for Prytherch et Single Classroom study 21 nurses w orking VitalPAC vs a Speed and accuracy Incorrect entries/omissions decreased from
automation al. (2006) 71 university on the medical paper-based of data entry, 29% to 10% using the VitalPAC method.
hospital assessment unit, generic early number of Few er incorrect clinical actions were indicated
inputting data from w arning score hypothetical clinical (14% to 5%) and mean time taken for
fictitious patients actions indicated participants to calculate and chart the early
either via pen and w arning score was 1.6-times faster with
paper (n=84), or into VitalPAC.
a handheld personal
digital assistant
(n=84).

Mohammed et Tw o NHS Classroom study 26 nurses from tw o VitalPAC vs a Accuracy and Accuracy improved with the use of the hand-
al. (2009) 72 Trusts surgical assessment paper-based efficiency of early held computers. The mean time to derive an
w ards, inputting data generic early w arning score early w arning score reduced from 37.9
from fictitious w arning score calculations seconds to 35.1 seconds.
patients via pen and
paper (n=260), or
into a handheld

12
 personal digital
assistant (n=260).

Schmidt et al. Tw o large Retrospective, Patients admitted to VitalPAC vs a Mortality During VitalPAC implementation, crude
(2015) 73 acute observational adult medical, paper-based mortality fell from 7.75% to 6.42% in one
teaching study surgical and generic early hospital, and from 7.57% to 6.15% at the
hospitals orthopaedic w ards w arning score second.
before (n=49 730)
and after (n=55 917)
the implementation
of an electronic
physiological
surveillance system

Bellomo et al. Ten Before-and-after Patients admitted to IntelliVue electronic Frequency of rapid The intervention w as associated with an
(2012) 74 hospitals in controlled trial 12 general w ards automated advisory response team calls, increased proportion of calls secondary to
the United before (n= 9617) vital signs monitors survival to hospital abnormal respiratory vital signs (from 21% to
States, and after (n=8688) discharge or to 90 31%). Survival increased from 86% to 92%.
Europe, after deployment of days for rapid Median length of stay and time to record
and electronic automated response team call observations were also significantly reduced.
Australia advisory vital signs patients; overall type
monitors and number of
serious adverse
events and length of
hospital stay
The Central Hospital length of Length of stay reduced from 9.7 days to 6.9
Jones et al. Single Historically- Consecutive patients Manchester stay, compliance days. Clinical attendance to patients w ith
(2011) 77 university controlled cohort admitted to the University w ith the early EWS 3, 4 or 5 increased from 29% at baseline
teaching study medical assessment Hospitals NHS w arning score to 78% w ith automated alerts. For patients w ith
hospital unit and one general Foundation Trust protocol, cardiac EWS >5, clinical attendance increased from
medical w ard before Early Warning arrest incidence, 67% at baseline to 96%.
(n=705) and after Score critical care
(CMFT EWS) utilisation and
(n=776) the
hospital mortality
implementation of an
automated alert
system

Table 1: Summary of relevant articles within each ‘Strength’ theme

13
Theme (Limitations) Publications Setting Methodology Participants Early Outcom e Findings
identified w arning m easure/s
w ithin theme score
Sensitivity and Shuk-Ngor et al. Single tertiary Prospective ED patients aw aiting MEWS Change in ED MEWS had a 100% sensitivity and a
specificity, including (2015) 79 university observational study admission to medicine, management, p 98.3% specificity in detecting patient
comparisons to teaching general surgery, adverse events deterioration, w hile there w as also a
specialty-specific hospital neurosurgery or clinical w ithin 24 hours high sensitivity and a high specificity
scores oncology. (active (100% and 97.8%) in the comparison
Intervention group: MEWS resuscitation, group.
monitoring (n= 269) ICU admission,
Control group: 4 hourly cardiac arrest
observations with no and death)
protocolised escalation
plan (n=275)
Barlow et al. Tw o hospitals: Retrospective Patients w ith community- Standardised Mortality The sensitivity and specificity of
(2007) 80 a 1000-bed analysis of data acquired pneumonia early w arning CURB65 w ere 71% and 69%
teaching prospectively (CAP) (n=419) score (SEWS) respectively. The sensitivity and
hospital and a collected for a vs CURB65 (a specificity of SEWS w ere 52% and 67%
500-bed district different study pneumonia- respectively.
general hospital specific risk
score)

Bayer et al. Single tertiary Retrospective Consecutive patients MEWS vs Sepsis The sensitivity and specificity of
(2015) 81 hospital observational admitted to the Prehospital PRESEP w ere 85% and 86%
analysis emergency department Early Sepsis respectively. The sensitivity and
(n=375) including those Detection specificity of MEWS w ere 74% and 75%
w ith sepsis (n=93), severe (PRESEP) respectively.
sepsis (n=60) and septic score
shock (n=12)
Bulut et al. Three university Prospective, General medical and MEWS vs In-hospital REMS w as a better predictor of both in-
(2014) 82 teaching observational cohort surgical patients admitted Rapid mortality and hospital mortality and escalation of care
hospitals study to the ED (n=2000) Emergency escalation of that MEWS.
Medicine care
Score (REMS)
Lobo et al. Single acute Cross-sectional audit Medical admissions w ith NEWS vs Change in NEWS had a low positive predictive
(2014) 86 hospital NEWS score >=7 (n=87) CREWS clinical value (35.4%) to detect clinical
(Chronic management deterioration. Application of the
Respiratory (indicating CREWS score in chronic hypoxaemic
Early Warning clinical patients safely low ered the trigger
Score) deterioration) threshold.
Need for practitioner Simmes et al. Single university Retrospective before- Surgical patients before Generic early Incidence of Introduction of an RRS resulted in a
engagement (2012) 41 medical centre after study (n= 1376) and after w arning score cardiac arrests 50% reduction in cardiac arrest rates
(n=2410) the introduction and unexpected and/or unexpected death. How ever, in
of a rapid response deaths 16% (15/91 events) activation was
system (RRS) w hich delayed for one or tw o days.
includes an early w arning
score

14
 Clifton et al. Single large Retrospective Post-surgical patients Generic Incidence of Missed alerts w ere common in
(2015) 102 university database analysis (n=200) paper-based errors in incomplete observation sets (15.1% vs
teaching early w arning observation sets 7.6%) and w hen a patient first became
hospital score unstable. Incorrect observation sets are
highly predictive of the next observation
set.
Niegsch et al. Single teaching Prospective, Hospitalised patients on MEWS Adherence to 58% of patients had been observed and
(2013) 105 hospital observational study 12 w ards (n=132) early w arning managed correctly. 77% had all MEWS
score guidelines elements recorded. 38% of patients
w ith abnormal MEWS w ere correctly
escalated by nursing staf f.
Peterson et al. Single large Observational study Inpatients w ho suffered an NEWS-based Compliance w ith The escalation protocol w as followed in
(2014) 107 tertiary centre of prospectively adverse event early w arning escalation 13% of ICU admissions, 31% of cardiac
collected data (unexpected death, score protocol in the arrests and 13% of unexpected deaths.
cardiac arrest or 24 hours Senior staff were involved in 53% and
unanticipated ICU preceding the 36% of cases of ICU admission and
admission) (n=144) event cardiac arrest, respectively.
Need for reaction to Cherry and Single acute Mixed methods study Acute Medical Unit MEWS Attitudes of The colour of the nurse’s uniform,
escalation Jones (2015) 103 Trust using questionnaires nursing staff (n=9) nursing staff show ing seniority, has an effect on a
and focus groups tow ards an early medic's attitude to review a patient w ith
w arning score a high MEWS score.
Day (2003) 112 Single acute Prospective audit Calls for medical Derby Response times The average response time to calls
general hospital assistance triggered by Modified Early to early w arning from SDU staff was 46·1 minutes
the early w arning score on Warning scores (guidelines suggest 30 minutes).
the surgical ‘step dow n System
unit’ (SDU) (n=45) (DMEWS)
Beckett et al. Single large Prospective Patients requiring medical Standardised Response times The median response time w as 5
(2009) 113 teaching observational study review overnight on 18 Early Warning to early w arning minutes for
hospital w ards (n=136) and 4 Score scores overnight SEWS>4 and 10 minutes if SEWS<4
critical care areas (n=159) (SEWS)
Need for clinical Martin (2015) 115 Single large Grounded theory Midw ives w orking on the Modified early Midw ives’ Midw ives experienced the tool as a
judgment tertiary teaching study using semi- labour w ard (n=6) obstetric experiences of threat to autonomy, undermining clinical
hospital structured interviews w arning score using the early judgement and w ere concerned about
(MEOWS) w arning score task orientation among junior
colleagues.
Neary et al. Single university Questionnaire study Convenience sample of NEWS Staff opinion Staff felt the NEWS did not correlate
(2015) 68 hospital 40 staff from general regarding the w ell clinically w ith patients w ithin the
surgical w ards (27 strengths and first 24 hours post-operatively.
doctors, 13 nurses) pitfalls after
NEWS w as
introduced into
surgery
Intermittent nature of Taenzer et al. Single large Comparative study Patients from 3 surgical Unspecified Accuracy of Manually recorded data w ere on
recording (2014) 117 tertiary hospital betw een oxygen units and 2 medical units manual vital intermittent, average 6.5% higher and did not reflect
saturation recordings w ho suffered an adverse signs chart manual SpO2 the high-risk patients’ physiological
using automatic evert (n=36) matched data collection state as w ell as continuous automated
continuous controls (n=176). 16 of sampling
monitoring and these patients w ere
intermittent manual classified as having
monitoring ‘prolonged desaturations.’
Table 2: Summary of relevant articles within each ‘Limitations’ theme

15
Early warning scores have been found to be excellent predictors of cardiac arrest 10,
ICU transfer11 and death on ICU12, as well as 30-day mortality and length of stay on
ICU.13
In nursing home residents admitted to hospital, MEWS was found to be an important
predictor of 7-day mortality.14 Patients with a MEWS of 4-5 on admission had 12
times the odds of death, and those with a score of >6 had 21 times the odds of
death, compared with those with a score of <1.
In the Emergency Department (ED), an early warning score can be used as a triage
instrument. One prospective study examined the MEWS of 309 patients presenting
to a Turkish ED and found that patients with a MEWS of 5 or more were 1.95 times
more likely to be admitted to ICU than those with a MEWS < 5, and 35 times more
likely to die in the ED and 14 times more likely to die in hospital.15
A group in Amsterdam retrospectively analysed the MEWS of 204 medical and
surgical patients who had experienced a ‘severe adverse event,’ including
cardiopulmonary arrest, unplanned ICU admission, emergency surgery or
unexpected death. Eighty one percent of patients had an MEWS value of 3 or more
at least once during the 48 hours before their event.16 Similarly, in an exclusively
surgical population, MEWS has been found to predict the risk of death after
cardiopulmonary arrest.17
An Italian prospective single-centre cohort study concluded that MEWS, even when
calculated once on admission, is a ‘simple but highly useful tool to predict in-hospital
outcome,’ in terms of mortality, critical care admission and length of hospital stay. 18
Similar results have been published from Denmark.19
The prediction value of early warning scores is important. They have been found to
prevent ICU admissions by aiding decision making for anaesthetists,20 and can be
used to help capacity planning by predicting the number of days a patient will spend
in hospital.21 However, they cannot be used in isolation nor can they replace clinical
judgment.22

Influence on clinical outcomes

The introduction of early warning scores has been found to improve patient
outcomes. However, the introduction of early warning scores is often accompanied
by that of a critical care outreach team, making the individual impact of the early
warning score difficult to assess.
A 2010 study reported that the introduction of an early warning score and a critical
care outreach team improved survival to ICU discharge in haematology patients.23
The introduction of an outreach service and MEWS at a tertiary referral centre was
associated with significant reductions in the incidence of cardiac arrest calls, the
proportion of patients admitted to ICU having undergone in-hospital CPR and their in-
hospital mortality.24
A large Danish prospective, non-randomized, controlled study investigated
unexpected in-hospital death before and after implementation of a clinical
intervention comprising systematic monitoring practice, early warning scoring, an

16
observation chart and an algorithm for bedside management.25 The adjusted
unexpected patient mortality rate was significantly lower after the intervention (17
versus 61 per 100 adjusted patient years). In a New Zealand tertiary hospital, the
introduction of an early warning score system in addition to an existing cardiac arrest
team decreased the incidence of in-hospital cardiac arrests from an average of 8.5
per month to 5.5 per month.26

Cross specialty application

Most of the studies evaluating early warning scores involve inpatients, and come
from a wide range of specialties. These include stroke,27 oncology, 28, 29 general
medicine 30-34 and surgery. 17, 20, 35-42 36-40 17, 41, 42
The use of early warning scores in the ED has been described above.
Early warning scores have also been implemented in the community. Ammitzboll
and Maarslet describe how an early warning score can be used to identify elderly
patients in need of medical assistance. At a score </= 2, 24% were visited by a
doctor or admitted directly. At a score >/= 5, this number increased to 45%. At a
score </= 5, 11% were admitted to hospital and at a score >/= 5, 31% were admitted
to hospital.43
Studies have described the use of early warning scores in private care homes.44 the
army 45 and dentists.46 The system has also proved valuable in the prehospital
setting.47, 48 Challen and Walter describe how early warning scores can help to
safely divert patients from the ED to alternative care providers.49 However, another
study amongst the ambulance service showed no effect of MEWS implementation on
transportation or revisit rates.50
Comprehensive implementation of the same early warning score allows a universal
language to be spoken across specialties. However, it is important to recognize the
limitations of early warning scores in certain patient groups. Vital signs have been
shown to be more accurate in detecting cardiac arrest in nonelderly patients
compared with elderly patients.51 Generic early warning scores cannot be used in
the maternity or paediatric populations;52,53 instead, specialized charts need to be
used for these groups.54

International relevance
A number of studies have shown that early warning scores can be used in countries
with limited healthcare resources, such as Uganda,55,56, Tanzania57, 58 and South
Africa.59,60,61 However, disease and population differences may strongly influence the
performance of early warning scores. A Malawi study showed that MEWS had only a
58.8% sensitivity and 56.2% specificity for mortality within three days. The authors
advised local validation and impact assessment before the adoption of early warning
scores adoption in resource-limited settings.62

Interaction with other variables

17
There is good correlation between early warning scores and other risk indicators. A
study of an early warning score in the ED63 found that adding specialty-specific
parameters (such as mode of transport to hospital) to the score provided more
accurate prediction of their risk. Alrawi has described how CRP and eGFR levels on
admission can be used in conjunction with MEWS to allow decision making on the
appropriate level of care at the point of hospital admission.14 A Sri Lankan study
showed that adding biochemical parameters to the early warning score improved the
sensitivity of predicted length of hospital stay and adverse outcomes.64

Impact on communication
A qualitative study in 2005 interviewed 30 nurses, 7 doctors and 7 health care
support workers with regard to the detection of physiological deterioration.65
Participants reported that quantifiable evidence is the most effective means of
referring patients to doctors, and that early warning scores achieve this by packaging
individual vital signs together, providing a ‘precise, concise and unambiguous means
of communicating deterioration, and confidence in using medical language.’ This
sentiment is echoed in other publications. Early warning scores help to facilitate
nurses’ communication with doctors 66 by providing ‘ammunition’ when referring
patients.67 A questionnaire study of surgical ward staff found that NEWS ‘empowers
nurses to more easily seek senior medical assistance’ and ‘avoids conflict.’68

Opportunity for automation

The detection of deteriorating patients is often later than it should be. One solution
could be the electronic charting of early warning scores to improve the accuracy,
reliability and availability of patients’ vital signs. There is a drive within healthcare
systems to improve the efficiency of information management in hospitals, through
integration and intelligent use of new technology.69
A number of software packages have become available to address this need. NHS
Education for Scotland has made the NEWS available as a smartphone app.70
Another such package, VitalPAC, was shown by Prytherch et al. to offer significant
advantages both in speed and accuracy of recording early warning scores.71 These
findings were echoed in a later study which found that a hand-held computer is
acceptable to nurses and helps to improve the accuracy and efficiency of early
warning scores in acute hospital care.72 Schmidt et al. associated the use of such
technology with reduced patient mortality.73 A before-and-after controlled trial of 18
305 patients investigated the effects of automated vital signs monitors 74 and found
their introduction to be associated with increased survival to discharge from 86% to
92% in patients receiving rapid response team calls. The same study noted that
there was also a decrease in the time required for vital signs measurement and
recording, from 4.1 minutes to 2.5 minutes.
However, the accuracy of electronic early warning score systems is still user-
dependent. One study of electronic observations found that levels of completeness
of observations differed between wards from 69% to 92%, with traditional gaps in
observations, such as recording of respiratory rate, still apparent. 75

18
Several technologies could provide the basis of a solution. Bonnici et al. suggest the
use of unobtrusive wearable monitors that track the patient's physiology
continuously.76
A number of studies have investigated the use of automated clinical alerts in an
attempt to deliver timely clinical responses to acutely deteriorating patients. An
historically-controlled study from 2011 found that automatic alerts significantly
improved clinical attendance to unstable general medical patients.77 However, the
potential of these technologies depends strongly on implementation, with poor-quality
deployment likely to worsen patient care.76

Limitations of early warning scores

Sensitivity, especially compared to specialty-specific scores

In 2003, Boyle reported that early warning scoring systems were largely unproven
and could prove to be over-sensitive and unspecific.78 However, the evidence base
has grown and a recent study from Hong Kong found that MEWS has a 100%
sensitivity and a 98.3% specificity in detecting patient deterioration.79
Nevertheless, early warning scores are generic tools which should be used to
complement, but not necessarily replace, existing prediction tools.80
A retrospective analysis of 419 patients with community-acquired pneumonia (CAP)
tested the performance of CURB65 (a mortality predictor in CAP) against generic
early warning scores. The study reported that CURB65 has a better sensitivity,
specificity, positive predictive value (PPV) and negative predictive value (NPV)
compared to the early warning score, and advised that it should not be supplanted
for the initial prognostic assessment in CAP.
Similarly, in the prehospital patient, the PRESEP score surpassed MEWS for
sensitivity, specificity, PPV and NPV.81 The same findings have been reported for
the REMS (Rapid Emergency Medicine Score),82 MEDS (modified mortality in
emergency department sepsis)83 and THERM (The Resuscitation Management
Score)84 scores in the ED, the PREEMPT-2 (PRE-critical Emergency Medical Patient
Triage) and PREAMBLE-2 (PRE-Admission Medical Blue-Light Emergency) scores
in acute medical admissions 85 and CREWS (Chronic Respiratory Early Warning
Score) for respiratory disease.86
Care must be taken when using early warning scores in certain subspecialties.
Teasdale notes the limitations of most early warning scores in patients with brain
injury.87 Similarly, patients prone to hypercapnia are at risk of inaccurate oxygen
monitoring when using early warning scores.88 Scores are also difficult to use in end-
of-life care.52,53,89

Need for practitioner engagement

The introduction of early warning scores has been shown to help improve the
monitoring of vital signs, especially respiratory rate90, 91,92. However, these systems

19
are highly user-dependent. A retrospective study of surgical patients before and
after the implementation of a rapid response system found that early warning score
recordings were ‘frequently incomplete.’41
Indeed, a number of studies into the implementation of early warning scores have
highlighted poor compliance as an issue.93,94, 95, 68, 96-100 101 User error can occur in
recording vital signs, calculating the score and escalating appropriately.
In the Amsterdam study, the authors retrospectively analysed the MEWS of patients
who had experienced a severe adverse event, and found that, even when the MEWS
was 3 or more, respiratory rate, diuresis, and oxygen saturation were documented in
only 30% to 66% of assessments.16 This is concerning, as missed alerts are
particularly common in incomplete observation sets.102
Even when observations are complete, the aggregate scores can be
miscalculated.103, 104 This is important, as Austen et al.5 found that calculation errors
were eleven times more likely to result in under-scoring than over-scoring, resulting
in the potential failure to recognise deteriorating patients. In addition, Clifton et al.
found that incorrect scores are highly predictive of the next observation set,
suggesting that clinical staff detect patient status in advance of the EWS system ‘by
using information not currently encoded within it.’102
Crucial to the success of early warning systems is the escalation of abnormal scores.
A Danish study found that only 38% of patients with abnormal MEWS were correctly
escalated by nursing staff,105 a finding echoed throughout the literature.106 Poor
compliance with the escalation protocol is commonly found when serious adverse
events occur.107
In particular, there have been concerns raised about compliance with early warning
scores overnight 108, 109 and at weekends 110 when, arguably, these scores could be of
most use. As such, a number of competency frameworks and audit systems have
been introduced, which show significant benefits in terms of patient safety.111

Need for reaction after escalation

A crucial step for the success of early warning scores is timely response to
escalation. Qualitative studies into the attitudes of nursing staff towards early
warning scores highlight concerns about difficulty in getting medical staff to review
the patient.103 In a UK NHS Trust, anecdotal evidence from nursing staff indicated
that response times by doctors were outside the established timescale, prompting an
audit which confirmed their concerns.112 A study by Beckett et al. showed a
significant inter-specialty variation in median response times and seniority of
responding staff, particularly within critical care, which recorded the slowest times .113

Need for clinical judgment

There is concern that early warning scores add to the de-skilling of practitioners.
Editorials stress that scores do not place importance on knowing individual patients,
nor the background to the observations being recorded. This may prevent nursing
students from ‘fully developing professional judgement as an aspect of decision

20
making when faced with a deteriorating patient.’114 Indeed, a study amongst
midwives found that this group experienced early warning scores as a ‘threat to
autonomy, undermining clinical judgement’ 115 and highlighted their concerns about
the delegation of vital signs monitoring to support staff, opposing holistic care.
To counteract this, some studies have suggested adding a measure of biological
capacity to the early warning score, such as mobility or frailty.116 However, it might
be simpler to acknowledge that early warning scores cannot replace clinical
judgement. 22 Neary et al.68 found that NEWS correlated poorly with the patient’s
clinical status within the first 24 hours post-operatively, and suggested that ‘nursing
acumen’ should dictate escalation parameters in certain scenarios.

Intermittent nature of recording

A significant limitation of early warning score systems is their intermittent nature. In a
cohort of patients with prolonged desaturations, manual recordings of SpO 2 did not
reflect the patient’s physiological state when compared with continuous automated
sampling.117
Ideally, every patient would receive ICU-style continuous monitoring, but this is
limited by bulky equipment, which would limit the patient’s mobility and potentially
hinder recovery. Indeed, when ICU-style monitoring was implemented on a general
ward, only 16% of patients remained connected in a 72-hour period.76 A number of
wearable wireless devices are in the early stages of evaluation to address this
problem.

Discussion

This systematic review and narrative synthesis was conducted to explore the
literature regarding the strengths and limitations of early warning score vital signs
monitoring systems, for both patients and clinical teams. To our knowledge, this is
the first literature review to systematically assess the extent of the evidence around
these tools.
Early warning scores have become ubiquitous with the recognition of the
deteriorating patient. This review confirms that early warning scores have excellent
predictive value and have been found to influence patient outcomes in the inpatient
setting. However, it is important to recognize that they are more effective in certain
patient groups, and care must be taken in the elderly, pregnant, paediatric, palliative
and head-injured populations. Specialist tools should continue to be used in these
groups.
Early warning scores are also used in a number of ways outside their original remit.
Studies investigating the use of early warning scores as a pre-hospital triage tool
show conflicting results. This can also be attributed to the mixed patient population
in pre-hospital care. Whilst some papers report that the universal language of early
warning scores improves communication between healthcare professionals, this is

21
not always reflected in the reaction to the escalation. Training may improve staff
engagement and the response to poor scores.
Limitations in the design of this review are acknowledged. The search criteria were
intentionally broad to capture a wide range of studies and optimize the
generalizability of the findings. This is a heterogeneous area of investigation and, by
including a range of early warning scores, settings and outcome measures, some of
the subtleties of individual systems may have been lost. In addition, the use of key
word searching can result in the omission of important papers. However, the search
strategy was checked for completeness by combining it with more specific term s
(such as EWS, MEWS) and this did not produce any additional references. Citations
and reference lists were also checked to optimise the search strategy.
The inclusion of a number of study types outside of randomised controlled trials
precluded traditional meta-analysis. Selected articles included qualitative,
quantitative and mixed-methods studies, alongside grey literature. The value of
other study designs in complex interventions is well recognised, but the wide range
sources necessitated the adoption of a narrative synthesis approach, which has
several limitations. Appraisal of quality is difficult with such variety of study design,
and data extraction relies heavily on the reviewers’ interpretation of the literature,
which may introduce bias. However, a narrative approach allows the synthesis of
diverse literature into common themes relevant to the research question.
Two interventions could improve the success of early warning scores to the benefit of
patients. Firstly, the introduction of automated early warning score systems can
minimize the risk of user error. Using a handheld computer device to document vital
signs can highlight erroneous data, improve accuracy of calculations and prompt
escalation. Scores can also be accessed remotely, which aids communication
between healthcare professionals. A number of UK NHS hospitals have begun to
adopt such systems.
In addition, new remote monitoring technologies, aided by wireless data
transmission, have the potential to overcome the intermittent nature of current early
warning score systems. A number of devices are emerging that promise to convey
the advantages of continuous vital signs monitoring to general ward patients. Whilst
it seems intuitive that continuous monitoring is safer than intermittent observations,
no large controlled trials have yet been conducted and this remains an exciting area
for future development.

Conclusion
This review has shown that early warning scores are successful in predicting and
improving patient outcomes across a range of settings and populations. The most
important advantage of early warning scores is that they are easy to use and
interpret, and so provide a common language across healthcare providers and
specialties. However, inaccurate recordings or inappropriate reactions to abnormal
scores can undermine the benefits of these systems.
Harnessing their strengths and recognizing their limitations can improve early
warning scores to the benefit of patients and healthcare professionals alike.

22
However, it is important to highlight the recurrent theme from the literature: whilst
early warning score systems are a useful tool, they can never replace clinical
judgment and experience in the management of the unwell patient.

References

1. Goldhill DR, McNarry AF. Physiological abnormalities in early warning

scores are related to mortality in adult inpatients. British Journal of
Anaesthesia. 2004;92(6):882-4.
2. Ridley S. The recognition and early management of critical illness.
Annals of the Royal College of Surgeons of England. 2005;87(5):315-22.
3. Bleyer AJ, Vidya S, Russell GB, Jones CM, Sujata L, Daeihagh P, et
al. Longitudinal analysis of one million vital signs in patients in an academic
medical center. Resuscitation. 2011;82(11):1387-92.
4. Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, et al.
Guidance on the conduct of narrative synthesis in systematic reviews. A
product from the ESRC methods programme Version. 2006;1:b92.
5. Austen C, Patterson C, Poots A, Green S, Weldring T, Bell D. Using a
local early warning scoring system as a model for the introduction of a
national system. Acute Medicine. 2012;11(2):66-73.
6. Day T, Oxton J. The National Early Warning Score in practice: a
reflection. British Journal of Nursing. 2014;23(19):1036-40.
7. Badriyah T, Briggs JS, Meredith P, Jarvis SW, Schmidt PE,
Featherstone PI, et al. Decision-tree early warning score (DTEWS) validates
the design of the National Early Warning Score (NEWS). Resuscitation.
2014;85(3):418-23.
8. Fox A, Elliott N. Early warning scores: a sign of deterioration in patients
and systems. Nursing management (London, England : 1994). 2015;22(1):26-
31.
9. Kovacs C, Jarvis SW, Prytherch DR, Meredith P, Schmidt PE, Briggs
JS, et al. Comparison of the National Early Warning Score in non-elective
medical and surgical patients. Br J Surg. 2016;103(10):1385-93.
10. Churpek MM, Yuen TC, Huber MT, Park SY, Hall JB, Edelson DP.
Predicting cardiac arrest on the wards: a nested case-control study. Chest.
2012;141(5):1170-6.
11. Lee JR, Choi HR. Validation of a modified early warning score to
predict ICU transfer for patients with severe sepsis or septic shock on general
wards. Journal of Korean Academy of Nursing. 2014;44(2):219-27.
12. Li YX, Ye HM. The prediction value of modified early warning score
grade for death of the patients in intensive care unit. Chinese Critical Care
Medicine. 2008;20(7):419-21.

23
13. Reini K, Fredrikson M, Oscarsson A. The prognostic value of the
Modified Early Warning Score in critically ill patients: a prospective,
observational study. European Journal of Anaesthesiology. 2012;29(3):152-7.
14. Alrawi YA, Parker RA, Harvey RC, Sultanzadeh SJ, Patel J, Mallinson
R, et al. Predictors of early mortality among hospitalized nursing home
residents. Qjm. 2013;106(1):51-7.
15. Armagan E, Yilmaz Y, Olmez OF, Simsek G, Gul CB. Predictive value
of the modified Early Warning Score in a Turkish emergency department.
European Journal of Emergency Medicine. 2008;15(6):338-40.
16. Ludikhuize J, Smorenburg SM, de Rooij SE, de Jonge E. Identification
of deteriorating patients on general wards; measurement of vital parameters
and potential effectiveness of the Modified Early Warning Score. Journal of
Critical Care. 2012;27(4):424.e7-13.
17. Stark AP, Maciel RC, Sheppard W, Sacks G, Hines OJ. An Early
Warning Score Predicts Risk of Death after In-hospital Cardiopulmonary
Arrest in Surgical Patients. Am Surg. 2015;81(10):916-21.
18. Cei M, Bartolomei C, Mumoli N. In-hospital mortality and morbidity of
elderly medical patients can be predicted at admission by the Modified Early
Warning Score: a prospective study. International Journal of Clinical Practice.
2009;63(4):591-5.
19. Christensen D, Jensen NM, Maaloe R, Rudolph SS, Belhage B, Perrild
H. Nurse-administered early warning score system can be used for
emergency department triage. Danish Medical Bulletin. 2011;58(6):A4221.
20. Peris A, Zagli G, Maccarrone N, Batacchi S, Cammelli R, Cecchi A, et
al. The use of Modified Early Warning Score may help anesthesists in
postoperative level of care selection in emergency abdominal surgery.
Minerva Anestesiologica. 2012;78(9):1034-8.
21. Subbe C, Falcus J, Rutherford P, Gemmell L. Capacity planning.
Knowing the score. Health Service Journal. 2003;113(5847):32-3.
22. Coulter Smith MA, Smith P, Crow R. A critical review: a combined
conceptual framework of severity of illness and clinical judgement for
analysing diagnostic judgements in critical illness. Journal of Clinical Nursing.
2014;23(5-6):784-98.
23. Bokhari SW, Munir T, Memon S, Byrne JL, Russell NH, Beed M.
Impact of critical care reconfiguration and track-and-trigger outreach team
intervention on outcomes of haematology patients requiring intensive care
admission. Annals of Hematology. 2010;89(5):505-12.
24. Moon A, Cosgrove JF, Lea D, Fairs A, Cressey DM. An eight year audit
before and after the introduction of modified early warning score (MEWS)
charts, of patients admitted to a tertiary referral intensive care unit after CPR.
Resuscitation. 2011;82(2):150-4.
25. Bunkenborg G, Samuelson K, Poulsen I, Ladelund S, Akeson J. Lower
incidence of unexpected in-hospital death after interprofessional
implementation of a bedside track-and-trigger system. Resuscitation.
2014;85(3):424-30.

24
26. Drower D, McKeany R, Jogia P, Jull A. Evaluating the impact of
implementing an early warning score system on incidence of in-hospital
cardiac arrest. New Zealand Medical Journal. 2013;126(1385):26-34.
27. Liljehult J, Christensen T. Early warning score predicts acute mortality
in stroke patients. 2015.
28. Cooksley T, Kitlowski E, Haji-Michael P. Effectiveness of Modified
Early Warning Score in predicting outcomes in oncology patients. Qjm.
2012;105(11):1083-8.
29. Young RS, Gobel BH, Schumacher M, Lee J, Weaver C, Weitzman S.
Use of the modified early warning score and serum lactate to prevent
cardiopulmonary arrest in hematology-oncology patients: a quality
improvement study. American journal of medical quality. 2014;29(6):530-7.
30. Cuthbertson BH, Boroujerdi M, Prescott G. The use of combined
physiological parameters in the early recognition of the deteriorating acute
medical patient.[Erratum appears in J R Coll Physicians Edinb. 2010
Jun;40(2):190]. Journal of the Royal College of Physicians of Edinburgh.
2010;40(1):19-25.
31. Huggan PJ, Akram F, Er BH, Christen LS, Weixian L, Lim V, et al.
Measures of acute physiology, comorbidity and functional status to
differentiate illness severity and length of stay among acute general medical
admissions: a prospective cohort study. Internal medicine journal.
2015;45(7):732-40.
32. Kellett J, Clifford M, Ridley A, Gleeson M. Validation of the VitalPACTM
early warning score (ViEWS) in acutely ill medical patients admitted. Irish
Medical Journal. 2013;106(10):318.
33. Kellett J, Kim A. Validation of an abbreviated VitalpacTM Early Warning
Score (ViEWS) in 75,419 consecutive admissions to a Canadian regional
hospital. Resuscitation. 2012;83(3):297-302.
34. Subbe CP, Davies RG, Williams E, Rutherford P, Gemmell L. Effect of
introducing the Modified Early Warning score on clinical outcomes, cardio-
pulmonary arrests and intensive care utilisation in acute medical admissions.
Anaesthesia. 2003;58(8):797-802.
35. Cuthbertson BH, Boroujerdi M, McKie L, Aucott L, Prescott G. Can
physiological variables and early warning scoring systems allow early
recognition of the deteriorating surgical patient? Critical Care Medicine.
2007;35(2):402-9.
36. Garcea G, Ganga R, Neal CP, Ong SL, Dennison AR, Berry DP.
Preoperative early warning scores can predict in-hospital mortality and critical
care admission following emergency surgery. Journal of Surgical Research.
2010;159(2):729-34.
37. Gardner-Thorpe J, Love N, Wrightson J, Walsh S, Keeling N. The
value of Modified Early Warning Score (MEWS) in surgical in-patients: a
prospective observational study. Annals of the Royal College of Surgeons of
England. 2006;88(6):571-5.

25
38. Goldhill DR, McNarry AF, Mandersloot G, McGinley A. A
physiologically-based early warning score for ward patients: the association
between score and outcome. Anaesthesia. 2005;60(6):547-53.
39. Kellett J, Wang F, Woodworth S, Huang W. Changes and their
prognostic implications in the abbreviated VitalPACTM Early Warning Score
(ViEWS) after admission to hospital of 18,827 surgical patients. Resuscitation.
2013;84(4):471-6.
40. Smith T, Den Hartog D, Moerman T, Patka P, Van Lieshout EM, Schep
NW. Accuracy of an expanded early warning score for patients in general and
trauma surgery wards. British Journal of Surgery. 2012;99(2):192-7.
41. Simmes FM, Schoonhoven L, Mintjes J, Fikkers BG, van der Hoeven
JG. Incidence of cardiac arrests and unexpected deaths in surgical patients
before and after implementation of a rapid response system. Annals of
intensive care. 2012;2(1):20.
42. Neary PM, Regan M, Joyce MJ, McAnena OJ, Callanan I. National
early warning score (NEWS) - evaluation in surgery. Int J Health Care Qual
Assur. 2015;28(3):245-52.
43. Ammitzboll O, Maarslet L. Early Warning Score in primary care in
Denmark. Ugeskrift for laeger. 2014;176(41).
44. Private homes should adopt NHS system. Nursing Standard.
2013;27(51):9-.
45. Carter C. Physiological observations and Early Warning Scoring tools
within the deployed field hospital. Journal of the Royal Army Medical Corps.
2013;159(4):283-6.
46. Omar Y. Avoiding medical emergencies. British Dental Journal.
2013;214(5):255-9.
47. Fullerton JN, Price CL, Silvey NE, Brace SJ, Perkins GD. Is the
Modified Early Warning Score (MEWS) superior to clinician judgement in
detecting critical illness in the pre-hospital environment? Resuscitation.
2012;83(5):557-62.
48. Silcock DJ, Corfield AR, Gowens PA, Rooney KD. Validation of the
National Early Warning Score in the prehospital setting. Resuscitation.
2015;89:31-5.
49. Challen K, Walter D. Physiological scoring: an aid to emergency
medical services transport decisions? Prehospital & Disaster Medicine.
2010;25(4):320-3.
50. Essam N, Windle K, Mullineaux D, Knowles S, Gray J, Siriwardena N.
Modified early warning scores (mews) to support ambulance clinicians'
decisions to transport or treat at home. Emergency Medicine Journal.
2015;32(5):e1-e.
51. Churpek MM, Yuen TC, Winslow C, Hall J, Edelson DP. Differences in
vital signs between elderly and nonelderly patients prior to ward cardiac
arrest. Critical Care Medicine. 2015;43(4):816-22.

26
52. Cook CA. Implementing the Modified Early Obstetric Warning Score
(MEOWS) to Detect Early Signs of Clinical Deterioration and Decrease
Maternal Mortality. Journal of Obstetric, Gynecologic & Neonatal Nursing.
2014;43(Supp 1):S22-S.
53. Lappen JR, Keene M, Lore M, Grobman WA, Gossett DR. Existing
models fail to predict sepsis in an obstetric population with intrauterine
infection.[Erratum appears in Am J Obstet Gynecol. 2011 Apr;204(4):359].
American Journal of Obstetrics & Gynecology. 2010;203(6):573.e1-5.
54. Edwards SE, Grobman WA, Lappen JR, Winter C, Fox R, Lenguerrand
E, et al. Modified obstetric early warning scoring systems (MOEWS):
validating the diagnostic performance for severe sepsis in women with
chorioamnionitis. American Journal of Obstetrics & Gynecology.
2015;212(4):536.e1-8.
55. Opio MO, Nansubuga G, Kellett J. Validation of the VitalPAC Early
Warning Score (ViEWS) in acutely ill medical patients attending a resource-
poor hospital in sub-Saharan Africa. Resuscitation. 2013;84(6):743-6.
56. Asiimwe SB, Abdallah A, Ssekitoleko R. A simple prognostic index
based on admission vital signs data among patients with sepsis in a resource-
limited setting. Crit Care. 2015;19:86.
57. Baker T, Blixt J, Lugazia E, Schell CO, Mulungu M, Milton A, et al.
Single Deranged Physiologic Parameters Are Associated With Mortality in a
Low-Income Country. Crit Care Med. 2015;43(10):2171-9.
58. Rylance J, Baker T, Mushi E, Mashaga D. Use of an early warning
score and ability to walk predicts mortality in medical patients admitted to
hospitals in Tanzania. Transactions of the Royal Society of Tropical Medicine
& Hygiene. 2009;103(8):790-4.
59. Burch VC, Tarr G, Morroni C. Modified early warning score predicts the
need for hospital admission and inhospital mortality. Emergency Medicine
Journal. 2008;25(10):674-8.
60. Kyriacos U, Jelsma J, James M, Jordan S. Monitoring vital signs:
development of a modified early warning scoring (MEWS) system for general
wards in a developing country. PLoS ONE [Electronic Resource].
2014;9(1):e87073.
61. Rosedale K, Smith ZA, Davies H, Wood D. The effectiveness of the
South African Triage Score (SATS) in a rural emergency department. South
African Medical Journal. 2011;101(8):537-40.
62. Wheeler I, Price C, Sitch A, Banda P, Kellett J, Nyirenda M, et al. Early
warning scores generated in developed healthcare settings are not sufficient
at predicting early mortality in Blantyre, Malawi: a prospective cohort study.
PloS one. 2013;8(3):e59830.
63. Expanded MEWS is more predictive: updates seen as more
appropriate for ED. ED Management. 2010;22(4):46-7.
64. Perera YS, Ranasinghe P, Adikari AM, Welivita WD, Perera WM,
Wijesundara WM, et al. The value of the Modified Early Warning Score and

27
biochemical parameters as predictors of patient outcome in acute medical
admissions a prospective study. Acute Medicine. 2011;10(3):126-32.
65. Andrews T, Waterman H. Packaging: a grounded theory of how to
report physiological deterioration effectively. Journal of Advanced Nursing.
2005;52(5):473-81.
66. Maiocco G. "Packaging" information about patient deterioration in
terms of vital signs and the Early Warning Score facilitated nurses'
communication with doctors. Evidence-based nursing. 2006;9(4):128.
67. SCORES GIVE US AMMUNITION WHEN REFERRING PATIENTS.
Nursing Standard. 2013;27(21):7-.
68. Neary PM, Regan M, Joyce MJ, McAnena OJ, Callanan I. National
early warning score (NEWS) - evaluation in surgery. International Journal of
Health Care Quality Assurance. 2015;28(3):245-52.
69. Gordo F, Abella A. Intensive care unit without walls: seeking patient
safety by improving the efficiency of the system. Medicina intensiva.
2014;38(7):438-43.
70. Sepsis screening. Nursing management (London, England : 1994).
2014;21(7):15.
71. Prytherch DR, Smith GB, Schmidt P, Featherstone PI, Stewart K,
Knight D, et al. Calculating early warning scores--a classroom comparison of
pen and paper and hand-held computer methods. Resuscitation.
2006;70(2):173-8.
72. Mohammed M, Hayton R, Clements G, Smith G, Prytherch D.
Improving accuracy and efficiency of early warning scores in acute care.
British Journal of Nursing. 2009;18(1):18-24.
73. Schmidt PE, Meredith P, Prytherch DR, Watson D, Watson V, Killen
RM, et al. Impact of introducing an electronic physiological surveillance
system on hospital mortality. BMJ Qual Saf. 2015;24(1):10-20.
74. Bellomo R, Ackerman M, Bailey M, Beale R, Clancy G, Danesh V, et
al. A controlled trial of electronic automated advisory vital signs monitoring in
general hospital wards. Crit Care Med. 2012;40(8):2349-61.
75. Nwulu U, Westwood D, Edwards D, Kelliher F, Coleman JJ. Adoption
of an electronic observation chart with an integrated early warning scoring
system on pilot wards: a descriptive report. CIN: Computers, Informatics,
Nursing. 2012;30(7):371-9.
76. Bonnici T, Tarassenko L, Clifton DA, Watkinson P. The digital patient.
Clinical Medicine. 2013;13(3):252-7.
77. Jones S, Mullally M, Ingleby S, Buist M, Bailey M, Eddleston JM.
Bedside electronic capture of clinical observations and automated clinical
alerts to improve compliance with an Early Warning Score protocol. Critical
Care & Resuscitation. 2011;13(2):83-8.
78. Boyle S. Critical care outreach service. N2N: Nurse2Nurse.
2003;3(10):10-4.

28
79. Shuk-Ngor S, Chi-Wai O, Lai-Yee W, Chung JYM, Graham CA. Is the
Modified Early Warning Score able to enhance clinical observation to detect
deteriorating patients earlier in an Accident & Emergency Department?
Australasian Emergency Nursing Journal. 2015;18(1):24-32.
80. Barlow G, Nathwani D, Davey P. The CURB65 pneumonia severity
score outperforms generic sepsis and early warning scores in predicting
mortality in community-acquired pneumonia. Thorax. 2007;62(3):253-9.
81. Bayer O, Schwarzkopf D, Stumme C, Stacke A, Hartog CS,
Hohenstein C, et al. An Early Warning Scoring System to Identify Septic
Patients in the Prehospital Setting: The PRESEP Score. Academic
emergency medicine. 2015;22(7):868-71.
82. Bulut M, Cebicci H, Sigirli D, Sak A, Durmus O, Top AA, et al. The
comparison of modified early warning score with rapid emergency medicine
score: a prospective multicentre observational cohort study on medical and
surgical patients presenting to emergency department. Emergency Medicine
Journal. 2014;31(6):476-81.
83. Cildir E, Bulut M, Akalin H, Kocabas E, Ocakoglu G, Aydin SA.
Evaluation of the modified MEDS, MEWS score and Charlson comorbidity
index in patients with community acquired sepsis in the emergency
department. Internal & Emergency Medicine. 2013;8(3):255-60.
84. Cattermole GN, Liow ECH, Graham CA, Rainer TH. THERM: the
Resuscitation Management score. A prognostic tool to identify critically ill
patients in the emergency department. Emergency Medicine Journal.
2014;31(10):803-7.
85. Carmichael HA, Robertson E, Austin J, McCruden D, Messow CM,
Belcher PR. A new approach to scoring systems to improve identification of
acute medical admissions that will require critical care. Scottish Medical
Journal. 2011;56(4):195-202.
86. Lobo R, Lynch K, Casserly LF. Cross-sectional audit on the relevance
of Elevated National Early Warning Score in medical patients at a Model 2
hospital in Ireland. Irish journal of medical science. 2014.
87. M. Teasdale G. NEWS is not suitable for all patients... McGinlay A,
Pearse RM. A national early warning score for acutely ill patients. BMJ
2012;345:e5310. (8 August.). BMJ: British Medical Journal (Overseas &
Retired Doctors Edition). 2012;345(7873):29-.
88. O'Driscoll BR, Grant K, Green D, Edeghere S, Bakerly ND, Murphy P,
et al. Clinical and scientific letters: The national early warning score gives
misleading scores for oxygen saturation in patients at risk of hypercapnia.
Clinical medicine (London, England). 2015;15(1):98.
89. Taylor P, Crouch S, Howell DA, Dowding DW, Johnson MJ. Change in
physiological variables in the last 2 weeks of life: an observational study of
hospital in-patients with cancer. Palliative medicine. 2015;29(2):120-7.
90. McBride J, Knight D, Piper J, Smith GB. Long-term effect of introducing
an early warning score on respiratory rate charting on general wards.
Resuscitation. 2005;65(1):41-4.

29
91. Odell M, Rechner IJ, Kapila A, Even T, Oliver D, Davies CW, et al. The
effect of a critical care outreach service and an early warning scoring system
on respiratory rate recording on the general wards. Resuscitation.
2007;74(3):470-5.
92. Paterson R, MacLeod DC, Thetford D, Beattie A, Graham C, Lam S, et
al. Prediction of in-hospital mortality and length of stay using an early warning
scoring system: clinical audit. Clinical Medicine. 2006;6(3):281-4.
93. Christensen D, Jensen NM, Maaloe R, Rudolph SS, Belhage B, Perrild
H. Low compliance with a validated system for emergency department triage.
Danish Medical Bulletin. 2011;58(6):A4294.
94. Hands C, Reid E, Meredith P, Smith GB, Prytherch DR, Schmidt PE, et
al. Patterns in the recording of vital signs and early warning scores:
compliance with a clinical escalation protocol. BMJ Quality & Safety.
2013;22(9):719-26.
95. Jonsson T, Jonsdottir H, Möller AD, Baldursdottir L. Nursing
documentation prior to emergency admissions to the intensive care unit.
Nursing in Critical Care. 2011;16(4):164-9.
96. Odell M. Detection and management of the deteriorating ward patient:
an evaluation of nursing practice. J Clin Nurs. 2015;24(1-2):173-82.
97. Smith S. Early warning scores: effective use. Nursing Times.
2011;107(3):16.
98. Stevenson JE, Israelsson J, Nilsson GC, Petersson GI, Bath PA.
Recording signs of deterioration in acute patients: The documentation of vital
signs within electronic health records in patients who suffered in-hospital
cardiac arrest. Health informatics journal. 2014.
99. Thorpe R. Planning a change project in mental health nursing. Nursing
standard (Royal College of Nursing (Great Britain) : 1987). 2015;30(1):38-44.
100. Watson A, Skipper C, Steury R, Walsh H, Levin A. Inpatient nursing
care and early warning scores: a workflow mismatch. Journal of nursing care
quality. 2014;29(3):215-22.
101. Le Jeune I, Masterton-Smith C, Subbe CP, Ward D. "State of the
Nation"--the Society for Acute Medicine's Benchmarking Audit 2013 (SAMBA
'13). Acute Medicine. 2013;12(4):214-9.
102. Clifton DA, Clifton L, Sandu DM, Smith GB, Tarassenko L, Vollam SA,
et al. 'Errors' and omissions in paper-based early warning scores: the
association with changes in vital signs--a database analysis. BMJ open.
2015;5(7):e007376.
103. Cherry PG, Jones CP. Attitudes of nursing staff towards a Modified
Early Warning System. British Journal of Nursing. 2015;24(16):812-8.
104. Smith AF, Oakey RJ. Incidence and significance of errors in a patient
'track and trigger' system during an epidemic of Legionnaires' disease:
retrospective casenote analysis. Anaesthesia. 2006;61(3):222-8.

30
105. Niegsch M, Fabritius ML, Anhoj J. Imperfect implementation of an early
warning scoring system in a Danish teaching hospital: a cross-sectional study.
PLoS ONE [Electronic Resource]. 2013;8(7):e70068.
106. Early warning systems: scorecards that save lives. Missouri Nurse.
2007(2):19-21.
107. Petersen JA, Mackel R, Antonsen K, Rasmussen LS. Serious adverse
events in a hospital using early warning score - what went wrong?
Resuscitation. 2014;85(12):1699-703.
108. Gordon CF, Beckett DJ. Significant deficiencies in the overnight use of
a Standardised Early Warning Scoring system in a teaching hospital. Scottish
Medical Journal. 2011;56(1):15-8.
109. Yiu CJ, Khan SU, Subbe CP, Tofeec K, Madge RA. Into the night:
factors affecting response to abnormal Early Warning Scores out-of-hours and
implications for service improvement. Acute Medicine. 2014;13(2):56-60.
110. Kolic I, Crane S, McCartney S, Perkins Z, Taylor A. Factors affecting
response to national early warning score (NEWS). Resuscitation. 2015;90:85-
90.
111. Higgins Y, Maries-Tillott C, Quinton S, Richmond J. Promoting patient
safety using an early warning scoring system. Nursing Standard.
2008;22(44):35-40.
112. Day BA. Early warning system scores and response times: an audit.
Nursing in Critical Care. 2003;8(4):156-64.
113. Beckett D, Gordon C, Paterson R, Chalkley S, Macleod D, Bell D.
Assessment of clinical risk in the out of hours hospital prior to the introduction
of Hospital at Night. Acute Med. 2009;8(1):33-8.
114. McCallum J, Duffy K, Hastie E, Ness V, Price L. Developing nursing
students' decision making skills: are early warning scoring systems helpful?
Nurse Education in Practice. 2013;13(1):1-3.
115. Martin RL. Midwives' experiences of using a modified early obstetric
warning score (MEOWS): a grounded theory study. Evidence Based
Midwifery. 2015;13(2):59-65.
116. Brabrand M, Kellett J. Mobility measures should be added to the
National Early Warning Score (NEWS). Resuscitation. 2014;85(9):e151.
117. Taenzer AH, Pyke J, Herrick MD, Dodds TM, McGrath SP. A
comparison of oxygen saturation data in inpatients with low oxygen saturation
using automated continuous monitoring and intermittent manual data charting.
Anesthesia & Analgesia. 2014;118(2):326-31.

31
32