Joint External Evaluation

IHR 2005

WHO Indonesia
JEE History
The Joint External Evaluation is the first of its kind and has a short history.

IHR Monitoring
and Evaluation 2016: The
Framework first JEE
mission is
conducted
IHR Annual Reporting
Revie After Action Review
IHR
WHA w Joint External
(2005) Com JEE
Evaluation
mittee Exercises

The IHR WHA61.2 is IHR Review JEE Tool is

(2005) is adopted. WHO Committee developed
adopted by develops a makes
the 58th framework for recommendation
World Health annual reporting s
Assembly to the WHA
(WHA)

2005 2008 2014 2015 2016

2
 Purpose of JEE
to assess country capacity to prevent, detect, and rapidly respond to
public health threats

to measure country specific status and progress in achieving the

targets

to identify progress made and ensure any improvements in capacity

are sustained

allows countries to identify the most urgent needs within their health
security system, to prioritize opportunities for enhanced preparedness,
response and action, and

to engage with current and prospective donors and partners to target

resources effectively

3| IHR MEF Joint External Evaluation

 Process of JEE
A country completed its baseline survey by using self-
reported data for the various indicators on the joint external
evaluation tool

This information is then given to the joint external

evaluation team

Evaluation Visit to the country by the team and participates

in in-depth discussion, site visits, meetings (team also use
other evaluation findings if available)

The evaluation team will draft a report to identify status

levels for each indicator, as well as an analysis of the
countrys capabilities, gaps, opportunities and challenges

Report finalize with host country and share with other

stakeholders to facilitate international support and share
best practices

4| IHR MEF Joint External Evaluation

JEE Tool Purpose

The JEE Tool is a data gathering instrument designed to evaluate

capacities required for the implementation of IHR (2005)

The Tool was developed through international collaboration with

Member States, subject matter experts, and international
organizations.

The Tool has 19 technical areas that are identified as important for
the implementation of IHR (2005) for Global Health Security.

Most indicators in the Tool combine qualitative and quantitative

measures.

5
 JEE Tool
All the indicators of the Tool are around PREVENT, DETECT and RESPOND plus
other IHR related hazards and PoEs
PREVENT DETECT RESPOND
1. National Legislation, 1. National Laboratory 1. Preparedness
Policy and Financing System
2. Emergency Operation Centres
2. IHR Coordination, 2. Real Time Surveillance
3. Linking Public Health and Security Authorities
Communication and 3. Reporting 4. Medical Countermeasures and Personnel
Advocacy
4. Workforce Deployment
3. Anti-microbial Resistance Development 5. Risk Communication
(AMR)
4. Zoonotic Disease Other IHR related HAZARDs and PoEs
5. Food Safety 1. Point of Entries (PoEs)
6. Biosafety and Biosecurity 2. Chemical Events
7. Immunization 3. Radiation Emergencies

6| IHR MEF Joint External Evaluation

Technical Areas that REQUIRE joint scoring (according to the JEE Tool)

PREVENT RESPOND

1. National Legislation, Policy and Financing 12. Preparedness

2. IHR Coordination, Communication and 13. Emergency Operations Centres
Advocacy 14. Linking Public Health and Security
3. Anti-microbial Resistance (AMR) Authorities
4. Zoonotic Disease 15. Medical Countermeasures and Personnel
5. Food Safety Deployment
6. Biosafety and Biosecurity 16. Risk Communication
7. Immunization

DETECT OTHER HAZARDS

8. National Laboratory System 17. Points of Entry (PoEs)

9. Real-Time Surveillance 18. Chemical Events
10. Reporting 19. Radiation Emergencies
11. Workforce Development

7
Indicators - PREVENT

Capacities Indicators

P.1.1 Legislation, laws, regulations, administrative requirements, policies or other government instruments in place are
sufficient for implementation of IHR.
National Legislation,
Policy and Financing
P.1.2 The state can demonstrate that it has adjusted and aligned its domestic legislation, policies and administrative
arrangements to enable compliance with the IHR (2005)

IHR Coordination,
P.2.1 A functional mechanism is established for the coordination and integration of relevant sectors in the implementation of
Communication and
IHR.
Advocacy
P.3.1 Antimicrobial resistance (AMR) detection
Antimicrobial P.3.2 Surveillance of infections caused by AMR pathogens
Resistance
P.3.3 Healthcare associated infection (HCAI) prevention and control programs
P.3.4 Antimicrobial stewardship activities
P.4.1 Surveillance systems in place for priority zoonotic diseases/pathogens
Zoonotic Disease P.4.2 Veterinary or Animal Health Workforce
P.4.3 Mechanisms for responding to zoonoses and potential zoonoses are established and functional
P.5.1 Mechanisms are established and functioning for detecting and responding to foodborne disease and food
Food Safety
contamination.

Biosafety and P.6.1 Whole-of-Government biosafety and biosecurity system is in place for human, animal, and agriculture facilities
Biosecurity
P.6.2 Biosafety and biosecurity training and practices

P.7.1 Vaccine coverage (measles) as part of national program

Immunization
P.7.2 National vaccine access and delivery
 Format and Colour Scoring System
Every indicator has attributes that reflect various levels of capacity with
scores of 1-5
No Capacity 1: Attributes of a capacity are not in place Red -
Limited Capacity -2: Attributes of a capacity are in development stage (some are
achieved and some are undergoing; however, the implementation has started).
Yellow -
Developed Capacity 3: Attributes of a capacity are in place; however, there is the
issue of sustainability and measured by lack of inclusion in the operational plan in
National Health Sector Planning (NHSP) and/or secure funding. Yellow -
Demonstrated Capacity 4: Attributes are in place, sustainable for a few more years
and can be measured by the inclusion of attributes or IHR (2005) core capacities in the
national health sector plan. Green -
Sustainable Capacity 5: Attributes are functional, sustainable and the country is
supporting other countries in its implementation. This is the highest level of the
achievement of implementation of IHR (2005) core capacities. Green -

9| IHR MEF Joint External Evaluation

Sample Indicator Table - Zoonosis
 Sample questions Zoonosis
Scores for each indicator are determined based on the Host Countrys presentation of each
technical area, as well as responses to technical area questions and supporting documentation.

Technical area questions for Zoonotic Disease indicator P.4.1

P.4.1 Surveillance systems in place for priority zoonotic diseases/pathogens

1. Does the country have a mechanism in place to identify priority zoonotic diseases that pose a national health risk?
2. Does the country have a surveillance system in place for relevant animal populations?
3. Please describe partnerships between ministry of health, ministry of agriculture and wildlife specialists, as they relate to zoonotic disease
detection and response.
a) Are situational awareness reports or reports of potential disease outbreaks shared between the agencies?
4. Are public health laboratories and animal health laboratories linked?
a) Is there a process for sharing specimens between public health and animal health laboratories?
b) Is there a process for sharing laboratory reports between public health and animal health laboratories?
c) Are these reports shared on a regular basis, or only when zoonoses are discovered or suspected?
5. Describe reports produced from animal surveillance systems for zoonotic disease.
a) What ministries receive reports produced by the animal surveillance systems on zoonotic diseases?
b) How is animal surveillance systems linked to surveillance systems used for human pathogens?
c) Is there a mechanism or mechanisms for establishing interagency response teams in the event of a suspected zoonotic outbreak?
d) Is there a process for sharing surveillance reports between public health and animal health laboratories? How do these systems
pick up emerging diseases versus endemic diseases?

Sample JEE Technical Area 11

questions
Sample Questions - Zoonosis
Challenges Surrounding Scoring
In many instances, situational and contextual difficulties arise in assigning scores.

Scoring Disparity in the JEE Tool

Sensitivities in Incomplete
Scoring Documentation

Scoring
Process
?
Comprehension of Other Situation-specific
Indicators Events

Maintaining the Integrity of the

Tool
13
Inputs into Priority Action Development

Priority Actions and Scores

JEE External Team Expert Input

Host Country Expert Input

Other Assessments (PVS, laboratory

assessments, etc.)

Host Country JEE Self Evaluation

Priority Actions

A good priority action statement should specify

the actions to be taken, the type of personnel
Tip 1
and other resources needed to take the action,
the unit or organization that should be
responsible for the action, and the outcomes to
occur if the actions are taken successfully.
Tip2

Although standard procedure recommends

gaining consensus on scoring before
Tip 2 developing priority actions. It is helpful to
review priority actions prior to the scoring
discussion. Identifying the countrys priority
actions helps to mitigate conflicts in scoring.

15
 Overview
JEE coordinated and central level

Evaluation of implementation of IHR capacities can be

done at sub-national level

Duration 5-10 days (depending size and complexity of

the country)

Countries are recommended to conduct JEE every 4-5

years

16 | IHR MEF Joint External Evaluation

 Country Requests an External Evaluation

Country Request
(can be requested to through partners or alliance, who will inform
WHO IHR Monitoring Team)

WHO Request

Request will reach to WHO IHR Monitoring Team

WHO will communicate with the Country through the

WHO regional and country offices requesting to conduct
the JEE in collaboration with partners.

17 | IHR MEF Joint External Evaluation

 Planning Begins
Country will identify Point of Contact (PoC)

Scheduling the evaluation

External Evaluation team will have subject matter

experts as per the technical areas of JEE tool.

Engagement of relevant government sectors in the

evaluation process

18 | IHR MEF Joint External Evaluation

 The Country Receives Joint External Evaluation Materials &
Sets Date for the Joint External Evaluation

WHO IHR Monitoring Team provides the Country POCs with

the resources and materials necessary for preparing for and
conducting the Self-Evaluation
Country Implementation Guide;
Joint External Evaluation Tool;
Final Report Template; and
Sample Agenda for the Joint External Evaluation
JEE Process Premises;
Country Overview Presentation Template; and
Technical Area Presentation Template

The country confirms dates for the Joint External Evaluation to

take place in coordination with the WHO IHR Monitoring Team.

19 | IHR MEF Joint External Evaluation

 Country Preparatory Phase
Evaluation Planning and Preparation Phase: 6-8 Weeks Prior to Evaluation Mission

There are two key elements in this phase:

Completion by the Country of the self-evaluation; and

Use the JEE tool for self evaluation
Use the final report template to complete self evaluation
Ideally takes 4-5 weeks after the receipt of JEE materials

Development of the agenda for the JEE visit ensuring

multisectoral representation from all relevant country
ministries.
Used JEE agenda template (described later in the presenation)

20 | IHR MEF Joint External Evaluation

Country Initiates Self-Evaluation Process
Joint External Evaluation Planning and Preparation Phase: 6 Weeks Prior to Evaluation Visit

Begin working on the Self-Evaluation in collaboration

with relevant in-country representatives and
stakeholders

Provides internal reflection

Country should also provide supplemental

documentation

Send the Self-evaluation report to WHO IHR Monitoring

team no later than 3 weeks prior to the JEE Mission

21 | IHR MEF Joint External Evaluation

 Scoring
Countries are asked NOT to provide a written score for any of
the 19 technical sections or the individual indicators, but instead
to focus on providing the critical information on their capabilities
based on the indicators and technical questions listed in the JEE
Tool.

The score is determined by consensus among the members of

the Joint External Evaluation Team.

Through the course of the discussions with the External

Evaluation Team, however, the country will be asked to propose
a score and provide rationale to inform the final decision by the
External Evaluation Team.
22 | IHR MEF Joint External Evaluation
Country Initiates Self-Evaluation Process
Joint External Evaluation Planning and Preparation Phase: 6 Weeks Prior to Evaluation Visit

The Country POCs decide which stakeholders to engage

based on the particular structure of their Member States health
system.

Imperative to include a multisectoral team with inputs from

representatives of all relevant sectors of government and other
stakeholders

WHO IHR Monitoring Team is also available to provide

guidance to the country on self evaluation

Once the evaluation is completed it is sent to WHO IHR

Monitoring Team (no later than 3 weeks prior to JEE mission)

23 | IHR MEF Joint External Evaluation

Draft Agenda for the Joint External Evaluation
Evaluation Planning and Preparation Phase: 3 Weeks Prior to Evaluation Visit

The Country prepares an agenda and potential site visits

for the external evaluation in coordination with the
external evaluation team lead

Critical to ensure
All of the IHR capacities are covered
There is strong multi-sectoral representation from across all
relevant government sectors like human health, agriculture,
animal health, security, others

The Country is provided with Sample Agenda

Consider for site visit

24 | IHR MEF Joint External Evaluation

 Country Prepares Logistics and Confirms Support Needs
Joint External Evaluation Planning and Preparation Phase: 4-5 weeks Prior to Evaluation Visit

The country in collaboration with WHO Country office is

expected to make all logistical preparations
Recommending a hotel for joint external evaluation team meetings;
Arranging site visits, briefings, and meetings with key stakeholders
relevant to the session;
Ensuring that meeting spaces are available for the Joint External
Evaluation Team;
Arranging for ground transportation for the Joint External Evaluation
Team;
Arranging for in-country travel for any site-visits that take place;
Arranging for translation services, if applicable; and
Arranging for refreshments and lunch during the week of the
evaluation
25 | IHR MEF Joint External Evaluation
 Country Prepares Logistics and Confirms Support Needs
Joint External Evaluation Planning and Preparation Phase: 4-5 weeks Prior to Evaluation Visit

The country should also communicate to the WHO IHR

Monitoring Team any special requirements for security
clearance, building access, visas or other official approvals

Country is responsible for assuming the cost of all in country

logistical preparation and can request WHO support through
WHO country office

26 | IHR MEF Joint External Evaluation

 Country Facilitates Joint External Evaluation Team Visit
Week(s) of Evaluation Visit

The Country will lead the Joint External Evaluation Team

through the agenda and the site visits over the course of
the evaluation visit.

The Joint External Evaluation Team evaluates the

Countrys capacities during meeting sessions and site
visits through presentations, document(s) review, and
active discussions.

Within tool there are series of technical area questions

under each of the 19 IHR core capacities (these are
intended to guide the Evaluation Team)
27 | IHR MEF Joint External Evaluation
The Country Reviews Joint External Evaluation Team Final Report
Post Evaluation Visit

The Final Report contains an analysis of the countrys

capabilities including
the scores for each IHR capacity,
best practices, and
Priority actions for improvement

JEE team shares the report with the Country for

feedback within 14 weeks of last day of the mission

It is posted online including in WHO websites and

countries are encouraged to share the final report
directly with current and potential partners
28 | IHR MEF Joint External Evaluation
The Country Reviews Joint External Evaluation Team Final Report
Post Evaluation Visit

Following the Final Report:

Country should develop a plan of action to address the gaps
Country can request WHO and/or other partners to support and
facilitate development of plan of action
Action plan should identify the countrys own commitments to
implementation
Identifying needs for external support

29 | IHR MEF Joint External Evaluation

 Thank You

30 | IHR MEF Joint External Evaluation