DraftWorkingPaper_RevDec10_0

STDF/Coord/293/Working Paper Draft Rev.3 (Dec.
2010)
Indicators to Measure the Performance of a National SPS System
STDF / OECD Working Paper (DRAFT)1
1
This draft working paper has been prepared by the STDF Secretariat in collaboration with the OECD.
It does not necessarily reflect the views of STDF's partners, donors, developing country representatives and
observers, or the OECD. This is the third revision of this paper and reflects discussions at the STDF / OECD
Technical Working Meeting on SPS Indicators, which took place in Geneva on 1 July 2010, as well as other
comments received following the meeting.
1
STDF/Coord/293/Working Paper Draft Rev.3 (Dec. 2010)
Executive Summary
1. Endorsement of the Paris Declaration on Aid Effectiveness in 2005 committed donors and
developing countries to change the way technical cooperation is delivered and managed as a means to
improve the effectiveness of assistance and advance progress towards the Millennium Development
Goals (MDGs). As such, donors and partner countries committed to put in place results-based
management frameworks to ensure that their activities achieve the desired objectives and targets.
Such frameworks are based on the articulation of a chain of results (logic model) and inclusion of
indicators for tracking results at each step in the chain.
2. This draft working paper has been prepared by the Standards and Trade Development Facility
(STDF) Secretariat, in collaboration with the Organisation for Economic Co-operation and
Development (OECD), to provide a framework to identify indicators to measure the performance of a
national SPS system, based on the logframe's intervention logic. This draft paper – and the
provisional set of indicators proposed – is a "work-in-progress". This version takes into account
discussions and comments made by participants at the STDF/OECD technical working meeting on
SPS indicators in Geneva on 1 July 2010. This draft will be further revised and improved through
further discussions with the STDF Working Group and other concerned stakeholders, as well as pilot
testing activities, which are planned to take place in selected countries in 2011.
3. The STDF work on SPS indicators, of which this working paper is one part, is designed to
support the identification and application of indicators to measure the performance of national SPS
systems. In particular, this work has three main objectives: (i) to sensitize the SPS community about
the importance of managing for results and, more specifically, about the value and role of indicators;
(ii) to identify, pilot test and refine a representative set of indicators to measure the performance of a
national SPS system; and (iii) to develop guidance materials to promote the use of indicators within
results-based management frameworks for national SPS systems. In addition, this work will
contribute to and support other STDF work (including on SPS action planning), as well as activities to
enhance the use of results-based management within SPS-related projects and programmes, and
efforts to monitor the impact of Aid for Trade by focusing on monitoring and evaluation of assistance
at an operational, issue-specific level.
4. The STDF work on SPS indicators should be distinguished from other work by international
and regional organizations to develop and/or apply sector-specific indicators for food safety, animal
and plant health systems, including as part of capacity evaluation tools. The SPS indicators proposed
here seek to "go beyond" existing sectoral indicators in an effort to develop comprehensive, cross-
cutting indicators for a national SPS system as a whole.
5. This working paper focuses on the identification of a set of provisional indicators to track and
measure performance of a national SPS system as a whole (macro level), based on the logical
framework's results chain and OECD terminology, i.e. inputs → activities → outputs → outcomes →
impacts. This paper does not undertake to develop indicators for particular SPS projects or
programmes (micro level), which will obviously depend on the specific objectives of the intervention
in question. The identification and application of macro-level indicators is a long-term and iterative
activity that is likely to require substantial time and resources from a range of stakeholders. The
intention is to substantially refine and improve the provisional indicators proposed here through
further discussions with STDF's partners and experts in other concerned organizations, as well as pilot
testing activities at the country-level. Activities to pilot test the use of these provisional indicators at
the country-level will provide useful feedback on the appropriateness and value of these indicators, as
well as on the complementary process of applying results-based management within the SPS area.
6. In essence, the purpose (medium-term) of a viable, functioning, resourced and transparent

national SPS system is to enhance food safety, animal and plant health (including the ability to meet
international SPS requirements). This will contribute towards the longer-term goal of meeting
2
national development objectives, which may include increased employment, income generation,
increased market access, poverty reduction, improved public and animal health, etc.
7. A viable and functioning national SPS system relies on the public sector, the private sector,
research and academia, and consumers and their organizations to achieve its purpose. For this system
to be functional and effective, each of the components must have the capacity to carry out their
particular roles and responsibilities. Effective linkages and synergies (including information
exchange, dialogue and coordination) between the various national stakeholders involved is essential.
Furthermore, this system operates within the context of: (i) other enabling (or disabling) factors (e.g.
rule of law, governance, the investment climate, logistics and transportation infrastructure) at the
country level; (ii) a regional framework (e.g. including regional trade agreements, SPS-related
strategies or priorities, etc. defined by governments in that region); and (iii) an international
framework (comprising international standard setting bodies, the WTO SPS Committee, bilateral
agreements with trading partners, etc.).
8. The identification and use of indicators for a national SPS system has several advantages.
Firstly, they are useful to aggregate the estimated impacts of multiple projects and interventions.
Secondly, in an environment where joint programmes and inter-organizational collaboration are
encouraged, developing and tracking key macro-level indicators provides a means to achieve
synergies and enhance effectiveness in reporting, monitoring and evaluation. Thirdly, macro-level
indicators can have considerable potential as policy tools to support SPS policy and decision-making
in a systematic way, particularly given the number of stakeholders involved and the often fragmented
state of SPS-related information at the national level. This is of particular relevance given efforts in
some countries to develop and/or apply SPS actions plans to provide a framework for SPS capacity
building and the mobilization of resources.
9. This paper also discusses common challenges that are – or are likely to be – faced in the
design and use of SPS indicators in practice. Quantifying long-term impacts is complex due to:
(i) the number of interventions (with and without donor support), as well as the linkages and
interdependencies between them and resulting problems of attribution; (ii) the time required to
observe results; (iii) the importance of other factors outside the scope of SPS (e.g. transportation or
financial infrastructure); and (iv) availability and reliability of data, including data fragmentation and
a lack of baseline data2. Inadequate financial resources for monitoring and evaluation, combined with
difficulties in establishing the counterfactual (i.e. testing the opposite hypothesis), compound these
challenges.
10. Finally, the paper makes some initial recommendations to support the identification and use
of SPS indicators. These focus on the need to: (i) adapt the provisional indicators proposed for use in
individual countries; (ii) strengthen data collection, reporting and management; (iii) pay attention to
widely-recognized guiding principles (i.e. relevance, limited number, clarity in design, feasibility,
identification of causal links, data quality and reliability, scale, etc.); (iv) identify targets and
baselines; and (v) enhance capacity in results-based management in particular and management
capacity in general.
2
Baseline data is collected at one point in time and is used as a point of reference against which results
will be measured or assessed. A baseline is needed for each performance indicator that will be used to measure
results during the investment. Results-Based Management Tools at CIDA: A How-to Guide Available at:
http://www.acdi-cida.gc.ca/acdi-cida/ACDI-CIDA.nsf/eng/NAT-92213444-N2H
3
I. Introduction
1. Demands for more rigorous monitoring and evaluation of Official Development Assistance
(ODA) have moved to the forefront of the international development agenda in recent years. Donors
and their national governments want to see evidence that resources are well spent, and that they
contribute towards meaningful results including achievement of the Millennium Development Goals
(MDGs). Implementing agencies and beneficiaries of technical cooperation have been asked to do
more to provide definitive measures of the effects of various types of assistance provided (e.g. in
agriculture development, private sector development, Aid For Trade, health, education), and to
demonstrate that it produces tangible results and impacts for people in developing countries, in terms
of poverty reduction, improved food security, etc.
2. Increasing flows of assistance have been allocated to enhance capacity in the sanitary and
phytosanitary (SPS) area in developing countries in the past decade. In general, the results of these
activities have been reported in terms of the outputs achieved (e.g. the number of officials trained,
new legislation developed), with limited information available about the medium to long-term impacts
on market access, poverty reduction, etc.. A number of factors contribute to this including issues
related to attribution and timing, as well as the challenges (including methodological difficulties, time
and financial resources required) inherent in quantifying these impacts. The STDF/OECD work on
good practice in SPS-related technical cooperation acknowledged this situation. However, it also
emphasized that without the systematic use of indicators to measure the results and sustainability of
SPS technical cooperation, the real effect of such assistance on trade is little understood.3
3. This draft working paper has been prepared by the Standards and Trade Development Facility
(STDF) Secretariat, in collaboration with the Organisation for Economic Co-operation and
Development (OECD). It is a direct response to demands for more rigorous monitoring and
evaluation of technical cooperation in general, and in the SPS area in particular. The focus is on the
identification of indicators for a national SPS system as a whole, based on the logframe's intervention
logic.
4. It is important to clarify at the outset that this paper does not undertake to develop indicators
for particular SPS projects or programmes, which will obviously depend on the specific objectives of
the intervention in question. Given the significant differences characterizing SPS-related projects and
programmes, it is unrealistic to try to develop project/programme level indicators here. Rather, the
intention is to provide a framework to identify indicators that are capable of measuring the
performance of an SPS system in a country over a period of time. As such, the indicators proposed in
this paper reflect the broad outcomes and results of relevant projects or programmes, as well as
complementary initiatives and actions by both public and private sector and other concerned
stakeholders in the country.
5. This draft working paper – including the preliminary set of indicators proposed – is a "work-
in-progress". It reflects discussions and comments made by participants at the STDF/OECD technical
working meeting in Geneva on 1 July 2010, which was attended by some 75 participants, including 16
experts from developing countries and regional economic communities in Africa whose participation
was funded by the STDF.
6. The technical working meeting, which was co-facilitated by an SPS consultant and a results-
based management specialist4, allowed an in-depth examination and discussion of a previous version
of this working paper and, in particular, the set of indicators proposed. Participants at this meeting
agreed on the value of a results-based management approach to identify indicators to measure the
3
STDF work on Good Practice G/SPS/GEN/875 and G/SPS/R/52:
http://www.standardsfacility.org/Good_Practice.htm
4
Kees van der Meer and Sheelagh O'Reilly, respectively.
4
performance of a national SPS system, and the use of the logical framework approach in that context.
There was also consensus on the need to ensure synergies with FAO, OIE, IICA and other
organizations that are applying capacity evaluation tools, since these tools generate information and
data that provide a useful measurement of capacity in key areas. It was further noted that the findings
of these evaluations generate useful baseline data, and some of these could be combined to form
composite indicators for particular aspects of SPS capacity. More in-depth discussions with the
organizations responsible for these capacity evaluation tools will be undertaken as a next step to
further explore these opportunities and the linkages with the STDF/OECD work.
7. The technical working meeting (and subsequent STDF Working Group meeting on 2 July)
also endorsed the need for pilot testing work on SPS indicators to ensure that country-level processes,
practicalities and experiences are reflected, and thereby further improve and advance the
STDF/OECD work. There was agreement that one size does not fit all and that any set of provisional
indicators would need to be adapted by countries depending on their particular circumstances. There
was also consensus that, wherever possible, pilot testing activities should be linked to other STDF
work (e.g. individual STDF projects, STDF economic analysis pilot testing, STDF/EIF training on
project design, SPS action planning, etc.) as well as capacity evaluation work led by other
organizations, notably OIE, FAO, IICA (see section IV below). It is intended that this draft working
paper will be improved and finalized based on this pilot testing work in selected countries, and
published as a guide for the development and application of SPS indicators.
Purpose
8. The STDF/OECD work on SPS indicators, of which this working paper is one part, is
designed to support the identification and application of indicators to measure the performance of
national SPS systems. In particular, this work has three main objectives:
i. to sensitize the SPS community at large about the importance of managing for results and,
more specifically, about the usefulness of indicators as a tool to better monitor and
measure outcomes, to improve project design and management and, where possible, to
evaluate the long-term impact of SPS capacity building;
ii. to identify, pilot test and refine a representative set of provisional indicators to measure
the performance of a national SPS system; and
iii. based on the pilot testing exercise, to develop guidance materials – targeted at national
authorities responsible for SPS management – to promote the use of indicators to measure
the performance of national SPS systems and strengthen the development of SPS action
plans.
9. Various stakeholders are expected to benefit from this work including:
• donors and development agencies responsible for financing and/or implementing SPS
capacity building initiatives;
• international organizations responsible for developing standards and encouraging
members to implement them, including through participation in SPS capacity building
initiatives; and
• other stakeholders in developing countries who are involved in SPS-related projects and
programmes, as well as national-led initiatives to strengthen SPS capacity including the
development and/or implementation of SPS policies, strategies, action plans, etc.
10. This work on SPS indicators will feed into and support other work by the STDF on SPS
action planning and the use of economic analysis to inform SPS decision-making. It will support
5
ongoing efforts to enhance results-based management and improve the development and application
of indicators as part of individual SPS-related projects and programmes. It also contributes to efforts
by the OECD's DAC Secretariat and the WTO to monitor the impact of Aid for Trade by focusing on
monitoring and evaluation of assistance at an operational, issue-specific level.
Methodology
11. This working paper has been prepared on the basis of: (i) desk research; (ii) inputs received
from STDF partners, donors and other collaborating organizations; and (iii) key informant interviews
with selected STDF partners and donors, as well as practitioners in the SPS field and the wider
"managing for results" community.
12. A number of challenges were encountered during the desk research. Despite requests for
information, it was difficult to obtain documents related to SPS-technical cooperation projects
including indicator sets. This partly reflects the fact that limited work seems to have taken place to
apply results-based management in the SPS area within individual organizations or to develop and use
SPS specific indicators. Where SPS-related documents were available, questions sometimes arose
regarding the terminology used. In some cases, little if any distinction was made between immediate
(output), medium-term (outcome) and long-term (impact) objectives and indicators. Other challenges
related to the use of aggregate-level indicators (e.g. increased food and agricultural exports) that were
difficult to link to project or programme interventions, the scarcity of baseline data and, in some
cases, confidentiality requirements.
Structure of this paper

13. Following this introductory section, the working paper is structured as follows:
• Section II provides a brief overview of the increased focus on indicators in the context of
results-based management and the aid effectiveness agenda, and introduces the different
types (output, outcome, impact) of indicators.
• Section III addresses the scope of a national SPS system and SPS capacity, as well as the
rationale for indicators to measure the results and performance of the system as a whole.
• Section IV sums up efforts to date to develop and apply indicators that are relevant for
SPS, notably indicators for food safety, animal and/or plant health.
• Section V considers technical issues related to the design and measurement of SPS
indicators.
• Section VI sets out a preliminary set of possible indicators for a national SPS system
based on the logframe's output-outcome-impact model.
• Section VII discusses some common challenges faced in the identification and/or
application of SPS indicators.
• Section VIII outlines some preliminary recommendations to strengthen the identification
and use of SPS indicators.
II. The aid effectiveness agenda and mounting attention to indicators
6
14. Endorsement of the Paris Declaration on Aid Effectiveness5 in 2005 committed donors and
developing countries to change the way technical cooperation is delivered and managed, as a means to
improve the effectiveness of available assistance and advance progress towards achievement of the
Millennium Development Goals (MDGs). One of the five core principles in the Paris Declaration
focused on "managing for results" (Box 1). By calling on donors and partner countries to direct
resources to achieving results, and use information on results to improve decision-making and
programme performance, it drew attention to the importance of indicators as a tool for this purpose.
Box 1. The Paris Principles on Aid Effectiveness, 2005

Ownership Partner countries exercise effective leadership over their development policies, and
strategies and coordinate development actions.
Alignment Donors base their overall support on partner countries' national development
strategies, institutions and procedures.
Harmonization Donors' actions are more harmonized, transparent and collectively effective.
Managing for Managing resources and improving decision-making for results.
results
Mutual Donors and partners are accountable for development results.
accountability
Source: Paris Declaration on Aid Effectiveness. Available at: http://www.oecd.org/dataoecd/11/41/34428351.pdf.
15. Reviewing the implementation of the 2005 Paris Declaration in 2008, the Accra Agenda for
Action6 concluded that greater efforts were needed to turn the Paris Principles into action. One of the
concerns raised focused on accountability, emphasizing that "developing and donor countries alike
must be accountable to each other and to their parliaments and citizens, demonstrating how their
policies and programmes translate into real impact on people's lives".7
Indicators as a key tool within results-based management
16. As part of this international dialogue on how to improve the effectiveness of technical
cooperation, donors and partner countries have committed to put in place results-based management
frameworks to ensure that their activities achieve the desired objectives and targets. Managing for
results implies "articulating a chain of results from project inputs, to activities, outputs, outcomes and
long-term impacts, which provides a framework within which to monitor and measure expected
changes that will result from project activities. Key changes described in the results chain are
translated into targets and associated baseline value, and indicators are identified for tracking results
at each step in a programme’s logic. Therefore, indicators are a critical component of the results-based
5
The Paris Declaration on Aid Effectiveness (2005) has been endorsed by some 114 countries and 25
international organizations. It sets out a roadmap of practical commitments to promote ownership, alignment,
harmonization, managing for results and mutual accountability, organized around five key principles, each of
which has a set of indicators of achievement.
6
The Accra Agenda for Action was adopted in September 2008, in Accra, Ghana by Ministers, Heads
of Agencies and other Senior Officials in an effort to accelerate and deepen implementation of the Paris
Declaration and help ensure achievement of the MDGs by 2015. See:
http://www.oecd.org/document/18/0,3343,en_2649_3236398_35401554_1_1_1_1,00.html
7
Gearing up for Accra: Setting a new agenda for action. OECD. DAC News. July 2008. See:
http://www.oecd.org/dataoecd/47/42/41018694.htm
7
management systems, enabling donors to integrate measurement of results into all phases of the
project or programme implementation."8
17. Results-based management is designed to improve programme delivery and strengthen

management effectiveness, efficiency and accountability through a focus on the achievement of
defined and measurable results and impact. It seeks to overcome what has been called the "activity
trap", that is, getting so involved in the detail of day-to-day activities that the ultimate purpose or
objectives are forgotten.9 As such, it aims to move the emphasis away from the type of activities
undertaken (e.g. training, provision of advice) to try to identify and measure the ultimate changes that
these activities are expected to induce, for instance on poverty reduction, economic growth and other
higher-level objectives.
18. Use of the logical framework (logframe10) to identify and, in turn, monitor and report on
outputs, outcomes and impact is a key part of the results-based management approach. Indicators are
also essential. Indicators give a precise definition of the intervention logic, providing an operational
description of the overall objective, purpose and results in terms of the variable (what will change?),
target value (how much?), target groups/beneficiaries (who whom?) and time (by when?). Indicators
quantify, describe and simplify information in a manner that facilitates understanding by policy
makers and practitioners. They also contribute towards accountability, transparency, continuous
feedback and ongoing learning.
Distinguishing between output, outcome and impact indicators

19. The results-based management literature categorizes indicators according to outputs,
outcomes and impacts (Figure 1).11 Output indicators measure goods and services provided.
Outcome indicators measure immediate or short-term results. Impact indicators monitor longer-term
results.
20. Efforts to measure SPS performance and results require output, outcome and impact
indicators, as well as data for each. Indicators of outputs alone are insufficient because the link
between a given output and the consequent outcome and/or impact may be ambiguous or of unknown
magnitude. For instance, the SPS dimension of a market access problem may be relatively small so
even a successfully implemented project (which satisfactorily meets the designated output indicators)
may not lead to significant improvements in market access (i.e. outcome and impact indicators) in the
absence of attention to other non-SPS constraints (see Figure 2).
8
OECD. 2010. How to manage for results: Some reflections on the use of common indicators.
paper prepared for the Joint Meeting of the Development Assistance Committee and The Working Party of the
Trade Committee on Aid for Trade, 7 June 2010. COM/DCD/TAD(2010)1. Available at:
http://www.olis.oecd.org/olis/2010doc.nsf/ENGDATCORPLOOK/NT00002B0E/$FILE/JT03283440.PDF
9
UNESCO. 2008. Results-based Programming, Management and Monitoring at UNESCO. Guiding
Principles. See: http://unesdoc.unesco.org/images/0017/001775/177568E.pdf
10
The logframe is an important and widely-used tool for managing the complete project cycle from
design to implementation, monitoring and evaluation. It provides a framework for conceptualizing project
objectives and linking them back to project interventions (World Bank. The LogFrame Handbook.
http://www.wau.boku.ac.at/fileadmin/_/H81/H811/Skripten/811332/811332_G3_log-framehandbook.pdf).
11
It is worth noting that the terminology used by bilateral donors and international organizations in the
context of results-based management differs. This paper will seek to use the OECD definitions.
8
Figure 1. OECD definitions of output, outcome and impact12
Output Outcome Impact
The products, capital goods and The likely or achieved short-term and Positive and negative, primary and
services which result from a medium-term effects of an secondary long-term effects produced
development intervention; may also intervention’s outputs. by a development intervention,
include changes resulting from the directly or indirectly, intended or
intervention, which are relevant to the unintended.
achievement of outcomes.
Figure 2. Impact of two different SPS projects on a market access problem13
Market access Market access

problem problem
SPS project impact SPS project impact
21. Similarly, the use of impact indicators alone is often insufficient because most development
objectives are achieved as a result of a number of different interrelated interventions. Measuring the
extent to which the objective has been achieved and identifying the contribution made by each
intervention or project is extremely complex and challenging. For instance, changes in market access
depend on the effects of multiple activities and interventions, as well as on other external factors (e.g.
competitiveness, exchange rate stability, transportation and financial infrastructure, ability to meet
requirements of individual buyers, absence of extreme weather events, appropriate ecological
conditions). Unless the contribution of a particular intervention to changes in market access
conditions is measured, that intervention may be credited for improvements it did not help bring about
or incorrectly blamed for problems it did not cause. Challenges related to attribution are also
discussed in sections V and VII of this document.
22. General characteristics of output, outcome and impact indicators are presented in Table 1.
12
OECD Glossary of Key Terms in Evaluation and Results Based Management (2002).
13
Adapted from World Bank. 1999. Environmental Performance Indicators.
9
Table 1. Characteristics of output, outcome and impact indicators14
Performance (efficiency of the project Results (changes resulting from the project
or programme) or programme)
Log frame level Inputs Outputs Outcomes Impact
Monitoring and Monitor resources Outputs are Assess early results Monitor and evaluate
evaluation and activities generated by the longer-term results of
activity project / the project /
programme, and programme.
track delivery of
goods and services
Characteristics Related to physical, Output indicators Outcome indicators Impact indicators
of indicator human and may include should respond quickly may move slowly and
financial resources physical outputs, and be easy to measure. be difficult to
provided for the services, training, They should measure measure. They must
project. advice, etc. the extent to which show evidence of
beneficiaries changed change and analysis
behaviour due to the must establish the
project. Typical extent to which
indicators include change is attributable
access, use and to the project /
satisfaction with programme being
respect to projects evaluated.
services.
Sources of Project documents, Project reporting, Surveys of Ongoing monitoring
verification administrative administrative beneficiaries, service and evaluation
records, etc. records, etc. providers, project activities, dedicated
reporting, etc. evaluation studies,
etc.
Frequency of 3-12 months 6-18 months 1-5 years 5 years and over
reporting
III. Measuring performance and results of national SPS systems
Components and capacities of a national SPS system
23. Following discussions at the technical working meeting, this revised paper seeks to clarify the
purpose of a national SPS system. In essence, the purpose (medium-term) of a viable, functioning,
resourced and transparent national SPS system is to protect human (food safety), animal or plant life
and health, as reflected in the preamble to the WTO Agreement on the Application of Sanitary and
Phytosanitary Measures (SPS Agreement). The SPS Agreement focuses specifically on the
relationship between international trade and measures related to food safety, animal and plant health.
It aims to strike a balance between the rights of governments to protect health of consumers by
ensuring food is safe and protecting plant health and animal health, while ensuring that such measures
are not disguised restrictions on trade. The ability to comply with international requirements –
notably the standards, guidelines and recommendations standards of the Codex Alimentarius
14
Adapted from FAO / World Bank / Global Donor Platform for Rural Development. 2009. Tracking
results in agriculture and rural development in less-than-ideal conditions. A source book of indicators for
monitoring and evaluation. 2009. See:
www.donorplatform.org/component/option,com_docman/task,doc_view/gid,863
10
Commission, the World Organisation for Animal Health (OIE) and the International Plant Protection
Convention (IPPC) – is fundamental to implementation of the SPS Agreement.
24. The existence of a viable, functioning, resourced and transparent national SPS system
therefore helps trade in agricultural and food products to flow as smoothly as possible. However, as
discussed above, its purpose goes beyond trade. Similarly, while the purpose of a national SPS
system is not to achieve higher-order goals, the existence of a national SPS system will contribute
towards the achievement of higher-order, longer-term national development goals and objectives,
which may include poverty reduction, increased employment, income generation, increased market
access, improved public health, etc. In this way, a national SPS system therefore also contributes
towards the attainment of MDG 1 (Eradicate extreme poverty and hunger) and MDG 8 (Develop a
global partnership for development).
Box 2: Definition of SPS Measures

SPS measures are measures intended to protect human, animal or plant life or health against risks arising from
the entry, establishment or spread of pests, diseases, disease-carrying organisms or disease-causing organisms;
or to protect human or animal health against risks arising from additives, contaminants, toxins or disease-
causing organisms in foods, beverages or feedstuffs; or otherwise to prevent or limit damage from the entry,
establishment or spread of pests.
Source: Annex A, SPS Agreement
25. There are synergies between a viable, functioning, resourced and transparent national SPS
system and success in accessing export markets. For instance, if a national SPS system has adequate
capacity for domestic inspection and enforcement, rates of rejections of food and agricultural exports
should be low. However, SPS capacity is important not only to meet requirements in export markets
but also to facilitate controls on imported agricultural and food products and on domestic production.
In countries with significant imports of food and agricultural products, these controls may sometimes
be of greater importance than export controls.
26. SPS capacity refers to a country's ability to design, disseminate and implement SPS measures
so as to achieve the appropriate level of protection (ALOP) against the risks faced (Box 2), and to
meet the SPS requirements of trading partner countries. As discussed below, these capacities exist in
both the public and private sector. Indeed, given the private sector's crucial role in food and
agricultural trade, producers, processors and traders must have a defined role and capacity to control
SPS risks (particularly in the area of food safety) that complements public sector resources, systems
and programmes of government.15
27. As illustrated in Figure 3, a national SPS system relies on the public sector (relevant
competent authorities for food safety, veterinary services, plant health and/or trade, SPS Enquiry
Points and National Notification Authorities), the private sector (including producers, processors,
traders, enterprises and their workers, industry associations, etc.), research and academia, and
consumers and their organizations. For this system to be functional and successful, each of the
components must have the capacity to carry out their particular roles and responsibilities. In addition,
it is essential to have effective linkages and synergies – including information exchange, dialogue and
coordination – between the various national stakeholders involved.
28. Figure 3 illustrates that the existence of an SPS system is, on its own, not sufficient for trade.
The results and performance that this system can achieve is influenced by other factors – such as rule
15
Gascoigne. 2007. Identification of Parameters for Good Practice and Benchmarks for Judging the
Impact of SPS-Related Technical Assistance. Report prepared for the Standards and Trade Development
Facility (STDF), Geneva.
11
of law, governance, the investment climate, logistics and transportation infrastructure, etc. – within
the country. These factors can be enabling or disabling. In addition, a national SPS system operates
within: (i) a regional framework that may include regional trade agreements, SPS-related strategies or
priorities, etc. defined by governments in that region; and (ii) an international framework comprising
the international standard setting bodies (Codex, OIE, IPPC), the WTO SPS Committee, bilateral
agreements with trading partners, etc.
12
Figure 3. National SPS System
Enabling environment (influencing Regional and international framework

factors in the country, e.g.)
Rule of law
Regional bodies /
economic
communities
Good NATIONAL SPS SYSTEM
governance
Domestic
National competent authorities (including consumers and
SPS Enquiry Point & Notification Authority) their organizations
Investment
ISSBs
climate Food Animal (Codex, OIE,
Safety Health IPPC)
Private sector
and industry
associations
Other market
requirements Plant Trade
Health
Research
and
academia WTO
Trade
facilitation Technical cooperation /
External support
Logistics,
infrastructure Trading 13
Partners
29. A national SPS system also relies on sectoral capacity in food safety, animal and plant health,
and trade to comply with SPS requirements and demonstrate compliance. This will normally include
policies, laws and regulations, institutions and infrastructure to carry out diagnostic analysis,
inspection, certification, monitoring and surveillance, enforcement, risk analysis, information
exchange, etc. across food safety, animal and plant health.
30. The level of SPS capacity required by any one country will reflect the type, number and
severity of SPS risks faced and the economic opportunities that are available if SPS risks are
controlled. This will differ significantly for different product-market combinations16 and particular
market segments depending on whether they have high, medium or low demands for quality and
safety.17
31. The hierarchy of trade-related SPS management functions, developed by the World Bank,
captures six key dimensions of SPS capacity (Figure 4 and Box 3).
Figure 4. Hierarchy of trade-related SPS management functions18
SPS
diplomacy
Technically-demanding risk
management functions
Institutional structures
and role clarity
Suitable and applied regulation
Application of basic "good practices"

for hygiene and safety
Awareness and recognition
32. While the hierarchy moves from lower levels of capacity (i.e. SPS awareness and recognition)
towards higher levels (i.e. SPS diplomacy), in practice, many of the functions in the hierarchy will
need to be carried out in parallel, with different thresholds for different product market combinations.
16
Personal communication. Kees Van der Meer.
17
Often a two or three-tier system develops in agricultural production, with some farmers producing on
contract to supply to tightly controlled standards for export; other, typically smaller farmers, producing
independently for the traditional local market; with perhaps an intermediate group supplying local supermarkets.
Kees van der Meer.
18
World Bank. 2005. Tanzania’s Agro-Food Trade and Emerging Sanitary and Phytosanitary (SPS)
Standards: Toward a Strategic Approach and Action Plan. Washington DC. See:
http://www.integratedframework.org/files/english/Tanzania_DTIS_Vol1_Nov05.pdf
14
Box 3. Key dimensions of SPS capacity*

Awareness and recognition of SPS requirements and controls: The extent to which
stakeholders – from the level of decision-makers to implementers and operators – in both the
public and private sector are: (i) aware of the importance of effective SPS controls to export
competitiveness; and (ii) recognize their own role and contribution within a functional SPS
system.
Application of basic "good practices" for hygiene and safety: The ability of actors within
export-oriented supply chains to apply established risk and quality management practices –
including good agricultural practices (GAP), good manufacturing practices (GMPs) and Hazard
Analysis and Critical Control Point (HACCP)-based quality management systems – from
production to distribution.
Suitable and applied regulation: The existence of a suitable legal regulatory framework for SPS
management as well as capacity (including competent staff, standard operating procedures,
financial resources, etc.) to effectively implement and enforce regulations.
Institutional structures and role clarity: Transparent institutional structures including clarity of
roles and mandates, and effective information exchange and coordination between the public and
private sector stakeholders involved in SPS management is a key dimension of SPS management
capacity.
Technically-demanding risk management functions: The ability to control SPS risks –

including several plant and animal diseases – that are more systemic in nature and not confined to
particular production or processing operations. Such risks cannot be fully controlled on a
decentralized basis and require broader oversight or collective action. Ability to control or manage
these risks normally requires more technically-demanding functions – including research and
analysis, effective systems for surveillance, quarantine or emergency management, which often
require sophisticated skills, specialized equipment and well-defined organizational structures,
supported by recurrent funding.
SPS Diplomacy: The extent to which countries can engage bilaterally with their trading partners
(both developed and developing and representing different market segments), as well as with
international standard-setting bodies (Codex Alimentarius, OIE and IPPC) and the WTO SPS
Committee.
* Source: Based on the World Bank's Hierarchy of trade-related SPS management functions
Therefore, achieving meaningful and sustainable impacts will depend on effective capacity in each of
these layers.
33. This hierarchy also recognizes the different and complementary roles of the public and private
sector within an effective SPS management system. While government authorities are responsible for
providing an effective legal and regulatory framework for SPS management, as well as provision of
technically-demanding risk management functions and engaging in SPS diplomacy with international
bodies and trading partners, the private sector has a fundamental role to play in the development and
sustainability of SPS capacity, particularly within the food safety sub-sector and in the least-
developed countries. In many cases it is through the specific actions of individual producers and
processors that compliance with SPS measures is achieved. One example is the application of
HACCP-based quality management systems and other elements of hygienic practices in the
production, processing and handling of agricultural and food products. In some cases, capacity in the
private sector (e.g. a laboratory established and operated by individual enterprises or an industry
organization) can complement, and even substitute for, the public sector capacity. In low-income and,
15
particularly, least-developed countries (LDCs), SPS management capacity is unevenly distributed

across the public and private sectors. For instance, in some cases where overall SPS management
capacity is very limited, considerable capacity may exist in key strategic areas such as established
export sectors or supply chains with a small number of dominant enterprises.19
Rationale for indicators for the national SPS system
34. Given the diversity of SPS problems, the variety of contexts in which they arise and the range
of possible solutions to address these problems, it is neither feasible nor practical to identify an
"exhaustive" set of indicators applicable to all situations. Nevertheless, it is possible to define some
representative indicators capable of measuring the overall SPS outcomes and results in a country over
a period of time. Such indicators are not directly linked to individual projects but rather reflect the
broad outcomes of multiple projects, national initiatives, policy changes, etc. This approach
acknowledges that the creation of an effective SPS system requires more than one project or
programme, as well as complementary initiatives and actions by both public and private sector
stakeholders in that country.
35. This working paper focuses on the identification and application of a set of provisional
indicators for a national SPS system (macro level), based on the logical framework model. The set of
indicators proposed here is a "work in progress" in that it is possible to consider adding or removing
indicators from the list in the logical framework matrix at any point, based on experience with their
use, including during the proposed pilot testing. This paper, including the provisional indicators
proposed, is intended to provide a framework to help countries set their own targets and measure their
SPS performance against these targets.
36. The identification and use of indicators for a national SPS system has several advantages.
Firstly, they are useful to aggregate the estimated impacts of multiple projects and interventions.
Secondly, in an environment where joint programmes and inter-organizational collaboration are
encouraged, developing and tracking key macro-level indicators provides a means to achieve
synergies and enhance effectiveness in reporting, monitoring and evaluation. Thirdly, macro-level
indicators can have considerable potential as policy tools to support SPS policy and decision-making
in a systematic way (Box 4), particularly given the number of stakeholders involved and the often
fragmented state of SPS-related information at the national level. This is of particular relevance given
Box 4. Use of SPS indicators to support national decision-making

The process of developing and applying macro-level SPS indicators at the national level, including the
collection of appropriate data, provides a means to:
• bring important SPS issues to the political agenda;
• promote national dialogue on SPS capacity and priority needs;
• facilitate reporting on the SPS situation to decision-makers and the general public (domestic and
international);
• assess the achievement of national goals and targets, and revise as required;
• facilitate the preparation and monitoring of SPS action plans;
• contribute to and support consideration of the costs and benefits of different types of interventions; and
• assess the performance of implementation of SPS action plans.
efforts in some countries to develop and/or apply SPS actions plans to provide a framework for SPS
capacity building and the mobilization of resources. Indicators to monitor the performance of these
19
Spencer Henson, Stenven Jaffee, Cees de Haan and Kees van der Meer. 2002. Sanitary and
Phytosanitary Requirements and Developing Country Agro-Food Exports: Methodological Guidelines for
Country and Product Assessments. World Bank. August 2002.
http://siteresources.worldbank.org/INTRANETTRADE/Resources/Topics/Standards/standards_challenges_met
hodologypaper.pdf
16
plans are useful to assess and evaluate the overall effects of SPS capacity building interventions, or
non-intervention, over the medium to long-term, and make any necessary adjustments.
37. While this working paper is expected to contribute to, and support, the process of developing
indicators for particular SPS projects and programmes, the focus here, as stated above, is not on
indicators for specific SPS interventions, projects or programmes. This reflects the need to define and
tailor indicators at the project/programme (micro) level to the specific objectives and components of
the project/programme in question.
38. Finally, the STDF/OECD work to identify provisional indicators to measure the performance
of a national SPS system should be distinguished from other ongoing work to develop and/or apply
sector-specific indicators for food safety, animal and plant health systems (see below). The SPS
indicators proposed here seek to "go beyond" existing sectoral indicators in an effort to develop
comprehensive, cross-cutting indicators for the national SPS system as a whole. The development
and application of these SPS indicators will obviously depend on the involvement of all the concerned
sectors (including food safety, animal and plant health) in the country.
IV. Efforts to date to identify and apply indicators for food safety, animal and/or plant health
39. The Food and Agricultural Organization of the United Nations (FAO), the World
Organisation for Animal Health (OIE) and other international/regional organizations have developed
– or are in the process of developing – sector-specific indicators for food safety, animal and/or plant
health. These sector-specific indicators – both quantitative and qualitative – seek to measure capacity
and performance within food safety, animal and plant health, and some focus directly on trade. As
such, they are relevant to the identification and application of indicators focusing on SPS as a whole.
However, SPS indicators should go beyond sectoral indicators for food safety, animal and/or plant
health to provide a measure of capacity in the SPS system as a whole.
40. Some of the existing sectoral indicators have been developed as part of SPS-related capacity
evaluation tools to provide a measure of the capacity of national food safety systems, veterinary
services, phytosanitary services, etc.20 Others have been developed in an effort to measure the
performance of specific interventions to enhance capacity in food safety, animal and/or plant health,
and trade.
41. The Inter-American Institute for Cooperation on Agriculture (IICA) has developed a tool to
assess the performance, vision and strategy for sanitary and phytosanitary services measures (IICA
PVS SPS Tool).21 This tool is of particular interest to the STDF/OECD work because "it approaches
the issue of sanitary and phytosanitary measures from an institutional, international, and horizontal
perspective. Institutional and international because it focuses on the responsibility of national public
and private entities of maximizing the benefits from and compliance with commitments made by the
country to international standardization organizations and the World Trade Organization (WTO). In
addition, the instrument is horizontal, because it analyzes all sectors rather than just one in
particular."22 The three components and variables included in the IICA PVS SPS Tool are presented
in Annex 1.
42. The IICA PVS SPS Tool has been applied in 28 countries in Latin America and the
Caribbean. Through this process, stakeholders at the country level have produced national
20
STDF. 2008. SPS-related Capacity Evaluation Tools: An overview of tools developed by
international organizations:
www.standardsfacility.org/files/various/STDF_Capacity_Evaluation_Tools_Eng_.pdf
21
http://www.iica.int/Esp/organizacion/LTGC/Sanidad/Publicaciones%20de%20SAIA/B0744I.pdf
22
IICA. 2008. performance, vision and strategy (PVS) for sanitary and phytosanitary measures: An
institutional vision. Written by Eric Bolanos Ledezma and Ana Marisa Cordero Pena for the Inter-American
Institute on Agriculture (IICA).
17
implementation agendas (or road maps) on SPS issues based on information collection and national
consultations involving public and private sector stakeholders (known as "common vision" sessions).
43. The Tool for the Evaluation of Performance of Veterinary Services (OIE-PVS Tool23),
developed by the OIE, includes a series of critical competencies to measure the performance of
veterinary services for animal health in four main areas (fundamental components of the OIE-PVS
Tool): (i) human, physical and financial resources; (ii) technical authority and capability; (iii)
interaction with stakeholders; and (iv) market access. Each critical competency includes qualitative
levels of advancement based on critical competencies in the OIE Terrestrial Code on Veterinary
Services (Chapter 3.1) and on the Evaluation of Veterinary Services (Chapter 3.2). To establish the
current level of performance, critical competencies with five possible levels of advancement are
identified for each of the four fundamental components. A higher level of advancement assumes that
the services are complying with the preceding (non 1) levels (e.g. level 3 assumes compliance with
level 2 criteria). For each critical competency PVS assessors use a list of suggested indicators that the
OIE has developed on the basis of extensive experience with the conduct of evaluations within the
PVS framework. In addition, the OIE has provided a Manual for Assessors as well as Guidelines for
countries requesting or considering a PVS Evaluation. The OIE has extensive experience in applying
these indicators as part of capacity evaluations carried out with the support of accredited experts
within countries. However, it should be noted that these are technical indicators for veterinary
services with limited attention to development outcomes or impact.24
44. The FAO has developed a generic set of indicators for food safety projects as part of a
guidance document on evaluating the impact of capacity building activities in the field of food quality
and safety.25 These indicators are intended to assist those involved in the development of specific
indicators for individual projects. In the plant health area, indicators have been developed as part of
the strategic framework for building national phytosanitary capacity proposed by the Commission on
Phytosanitary Measures (CPM) of the International Plant Protection Convention (IPPC).26
45. One challenge raised regarding some of the above tools is that they contain large numbers of
indicators to describe a system from a technical perspective, however, the indicators are not
prioritized.
46. The World Bank has developed a set of indicators of participation in international institutions
relating to SPS and other technical measures, as part of methodological guidelines for country and
product assessments of SPS requirements.27 The latter were developed to support the development of
action plans but have not been applied widely.
23
http://www.oie.int/eng/oie/organisation/A_2010_PVSToolexcludingindicators.pdf
24
The OIE emphasizes the need to recognize the challenges in "extracting" – out of context – particular
sector-specific indicators and cautions that indicators from the OIE-PVS Tool should be considered only within
the framework of the specific PVS evaluation, given that the number (and complexity) of indicators for each
critical competency varies and that these indicators have been determined by accredited experts for the purpose
of the PVS.
25
More information is available here: http://www.fao.org/ag/agn/agns/files/CBIndicatorPaper.pdf
26
IPPC. Building National Phytosanitary Capacity (Strategic Framework). February 2010.
https://www.ippc.int/file_uploaded/1267093551_2010-DRAFT-IPPC_bnpc_strategy.doc
27
Spencer Henson, Stenven Jaffee, Cees de Haan and Kees van der Meer. August 2002. Sanitary and
Phytosanitary Requirements and Developing Country Agro-Food Exports: Methodological Guidelines for
Country and Product Assessments. World Bank. August 2002.
http://siteresources.worldbank.org/INTRANETTRADE/Resources/Topics/Standards/standards_challen
ges_methodologypaper.pdf
18
47. The European Commission is currently developing indicators in the field of trade-related
assistance and private sector development in which SPS indicators are also being explored.28 These
indicators, when available, will also be of use to refine and improve the indicators proposed in this
working paper.
48. The Aid for Trade dialogue is examining the possibility of "developing a core list of
commonly agreed indicators for cross-country comparability, and to link them to subsets of country
specific indicators for Aid for Trade".29 In this context, work is ongoing to review existing indicators
to measure aid for trade results at the country level and consider their policy relevance and analytical
soundness. This work is taking place in the run-up to the Third Global Review of Aid for Trade in
2011, which will focus on monitoring and evaluation. An expert meeting at the OECD in October
2010 discussed the potential and value of selecting and integrating a small number of "universal"
indicators across all types of aid-for trade projects and programmes, as a means to facilitate the
aggregation of results at the country level. One approach being considered under this work is to
"identify and 'tag' certain aspects of results and promote common results measurement and reporting
practices. Indicators would be set at the outcome level where measurements of the results (or the
degree to which the goals were achieved) could be bundled up".30.
49. Bilateral donors, development agencies and other organizations involved in capacity building
activities related to food safety, animal and/or plant health, and SPS more broadly, are also making
increasing use of indicators to monitor and evaluate the performance and impact of their interventions.
Many of these organizations – including the Canadian International Development Agency (CIDA)31,
the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ)32, the Danish International
Development Agency (Danida), the United Kingdom Department for International Development
(DFID), the Economic Cooperation and Development Division at the Swiss State Secretariat for
Economic Affairs (SECO)33, the Swedish International Development Cooperation Agency (SIDA)34,
the United States Agency for International Development (USAID) – have developed guidance to
promote the routine use of logical frameworks (including the inclusion of indicators), as part of
results-based management for their SPS-related projects and programmes.
50. Similarly, most bilateral donors and multilateral agencies have developed separate indicators
to measure the outcomes of their trade-related assistance. For instance, indicators for "trade
development" activities include: export growth rates; changes in the export structure; changes in the
share of value-addition of exports; and trade's share in the country's GDP.35 However, an OECD
review of the efforts of donors and multilateral agencies to measure performance and impact of trade-
related assistance notes the challenges in determining the "development" effectiveness and longer-
term impact of trade-related assistance. These are often due to the lack of clear and measurable
programme objectives and performance indicators (particularly impact indicators) in programme
28
EC Internal Working Paper on Indicators in the Field of Trade Related Assistance and Private Sector
Development. Not dated.
29
OECD, 2009. COM/DCD/TAD(2009)4/REV1.
30
OECD. 2010. Experts Meeting on Indicators (22 October 2010): Measuring Aid-For-Trade Results
at the Country Level. Background Note for Discussion.
31
http://www.acdi-cida.gc.ca/acdi-cida/acdi-cida.nsf/eng/NIC-31595014-KEF
32
GIZ was established in January 2011 following a merger of the Deutsche Gesellschaft für
Technische Zusammenarbeit (GTZ), Internationale Weiterbildung und Entwicklung gGmbH (InWEnt) and
Deutsche Entwicklungsdienst gGmbH (DED).
33
http://www.seco-cooperation.admin.ch/dienstleistungen/00602/index.html?lang=en
34
http://www2.sida.se/shared/jsp/download.jsp?f=SIDA1489en_web.pdf&a=2379
35
OECD. 2007. Trade-Related Assistance. What do recent evaluations tell us? See:
http://www.oecd.org/dataoecd/19/3/37326353.pdf
19
documents, and difficulties to attribute changes in a country's export performance or overall policy
making to specific projects due to attribution and time lag problems.36
51. Some regional economic communities (including COMESA) have initiated work to identify
indicators to measure performance as part of SPS action planning. UNIDO intends to develop SPS
compliance indicators. The STDF will continue to make every effort to share information on its work
on SPS indicators with other concerned organizations and to identify and pursue concrete areas for
collaboration with concerned stakeholders, including during the planned pilot testing activities at the
country level.
V. Technical issues related to the design and measurement of SPS indicators37
52. Clarifying the scope of SPS is important since there are diverging implicit definitions that ask
for somewhat different indicators. Some definitions tend to include all animal health, plant health and
food safety measures in SPS, regardless of whether there is a relation to trade flows. Others narrowly
look at measures to promote exports from developing countries to premium markets. Most support
for SPS capacity building targets this latter segment. A third approach would be to consider only
trade-related measures and capacities needed thereof. Since many capacities can be used for trade-
related and domestic tasks there are areas of overlap, which is fine.
53. Country-specificity is important. Countries can differ much in the SPS capacities they need
and can afford. One size does not fit all. A number of factors affect the need for SPS capacities
including:
• Size of country (e.g. measured by area, population, size of the economy, size of the agriculture
and food sector, volume of trade in agriculture and food products). The demand for SPS services
increases with most measures of the size of a country. Small countries have relatively small
volumes of trade to protect and to certify. Therefore, affordability of SPS capacities is challenging
for small countries. Since many SPS capacities can be used for a variety of products and large
volumes, and involve significant minimum fixed costs, bigger countries can afford a broader
range of capacities and specialist services. Certain lumpy capacities which are considered basic
and unavoidable, will require small countries to spend relatively more than bigger countries
because of diseconomies of scale.
• Urbanization: Since urbanization results in more transport of food and agricultural products over
long distances, often between areas with different pest and disease situations, and with producers
and final users who do not know each other, health risks tend to increase and a stronger public
role is needed.
• Product-market combinations: Import restrictions and buyer requirements differ much between
products, destinations, market segments and by origin of production, because of inherent risks of
health hazards and preference of buyers. Hence, products can be called low, medium and high
SPS-sensitive. Some countries apply high biosecurity standards over a broad range of products.
Others are lenient. Supermarkets in OECD countries require high quality and food safety
standards, which function as a threshold for large market segments. Small countries may have
comparative advantage in a limited number of export products and their range of risky imports
may be limited. Their need for SPS capacity depends on the SPS-sensitivity of their product-
market relations.
36
Ibid.
37
Section provided by Kees van der Meer.
20
• Domestic income levels: High-income societies tend to be more sensitive about health hazards.
Consequently, their SPS capacities in the public and private sector need to be more
comprehensive and advanced. The adoption of food safety standards is highly related to the level
of income.
• Geo-political location: Membership of regional economic cooperation agreements or common

markets can highly affect what SPS capacities should be in place and what measures should be
taken by member countries. In particular the EU, ASEAN and CIS offer examples of such
requirements.
54. A particular benefit of country-level indicators in many areas, such as in cost of doing
business, investment climate, governance, human health, is international comparison. It helps
policy makers understand their relative position and gaps in performance. Such use could also be a
main benefit of SPS indicators. However, country-specific factors complicate the design of macro-
level SPS indicators that can be meaningfully compared internationally. There are two basic
directions to solve this. One is designing indicators that are corrected for scale, and the other is to
compare within groups of countries with similar characteristics (least developed countries, low
income countries, small countries, middle income countries, etc.).
55. Aggregation: Many macro-level indicators cannot be measured directly. They have to be
based on often large numbers of measurable sub-indicators and estimated through aggregation. This
is not unique to SPS, it is common in many areas of measurement of national capacities and
performance, such as cost of doing business, investment climate, governance and human health.
56. Estimating outcome and impact: Since outcome and impact will materialize in the future
their estimation will depend on availability of models that assume causality. However, attribution of
SPS measures to outcome and impact can be problematic. In many cases SPS capacities and
performance are not the main constraint to achieving more production, trade and income, and hence
estimation can be full of uncertainty.
57. There are three ways to collect information to estimate the level of an indicator. The first is
to collect data from statistics and administrations. Second, use can be made of data surveillance
among stakeholders and specialists. Third, specialist judgment can be used to assess information and
to give scores to questions on a point scale (often yes-no, three-point or five-point scale, but also
seven- or ten-point scales). The assessments for veterinary services, phytosanitary services and food
safety are largely based on expert assessments. Indicators in some other areas such as investment
climate and governance are mainly based on surveys with questionnaires.
58. Although having indicators for the SPS system as a whole is an important aim, it is also
important to have indicators for food safety and animal and plant health, because the issues are
significantly different and, in most countries, policy priorities for the sectors differ.
VI. Preliminary indicators to measure the performance of a national SPS system
59. This section begins to identify a preliminary set of indicators for a national SPS system
based on the logframe's output-outcome-impact model discussed above and guided by the hierarchy of
trade-related SPS management functions. The indicators in Table 2 are illustrative in that they are
indicative of the types of indicators that a country might utilize to measure their SPS performance.
Based on particular circumstances and needs at the country level, the indicators outlined below should
be adapted and modified for use at the national level. In some cases, they might also be adapted for
use at the sub-national and/or regional level.
60. It is important to acknowledge the limitations of these indicators. Previous efforts to identify
indicators in other cross-cutting areas, such as governance, stressed the need to accept that any
21
Box 5. Examples of quantitative and qualitative indicators
Quantitative indicators are objectively or independently verifiable numbers or ratios such as volume of
exports; output/cost ratios. For instance:
Number • number of accredited laboratories

• number of equivalency agreements with trading partners
• number of new jobs created in food and agricultural export sector
Percentage • percent of government budget devoted to SPS management

• percent of staff of SPS competent authorities with risk analysis knowledge and
capacity
Ratio • ratio of food safety / animal / plant health inspectors to population
Qualitative indicators are subjective descriptions or categories such as whether or not a law has been
passed or an institution has been established; beneficiaries’ assessment of whether a project’s services are
excellent, satisfactory or poor; or simply a narrative describing change. For instance:
Existence • SPS coordination mechanism established

(yes/no) • SPS action plan developed / not developed
• Amendments to relevant legislation passed / not passed
Category • Private sector awareness about SPS issues is "high", "medium" or "low"
(e.g., x or y or z) • Satisfaction of exporters / traders with respect to export controls is "high",
"medium" or "low"
indicators will be imperfect. The development and application of indicators is a long-term and
iterative activity that is likely to require substantial time and resources from a range of stakeholders.
The indicators outlined below represent an initial, working set of indicators, to be substantially refined
and improved based on pilot testing activities in countries.
61. The indicators proposed include a mix of quantitative, as well as qualitative or descriptive
indicators (Box 5). Quantitative and qualitative indicators can be combined and typically complement
each other.
62. Some of these indicators measure "commitment". While this is useful, there are inherent
challenges. For instance, having an SPS strategy or SPS coordination mechanism is only the first
step. The strategy needs to be implemented and the mechanism needs to be operational. Finally, the
strategy and mechanism need to have a positive effect on SPS management capacity.
63. Wherever possible, the indicators proposed are related to existing data sets and available
sources of verification.
64. A logical framework for a national SPS system – with indicators at the different levels – is
presented below (Table 2). This is intended to provide a framework to help countries set their own
targets and measure their SPS performance against these targets. As stated above, these indicators
should be regarded as a representative set of provisional "working" indicators for a national SPS
system, which may be modified and adapted on the basis of national circumstances and needs. In
other words, it would be possible to consider adding or removing indicators from the list in the logical
framework matrix at any point, based on experience with their use, including during the proposed
pilot testing. As this work advances, one option to explore could be to refine and adapt the
intervention logic in Table 2 to generate a series of logframes (supported by indicators) for national
SPS systems in different types of countries (e.g. least developed, middle-income, food importing
versus food exporting, etc.).
22
Table 2: Logical Framework for a National SPS System38
Intervention logic Objectively verifiable indicators Sources and means of verification Assumptions and risks
(examples – to be completed) (examples - to complete)
Impact / Goal To contribute to the • GDP growth rate • Poverty Statistics • Enabling external economic
achievement of national • Employment in food and agricultural sector • GDP and trade statistics from environment
development objectives (e.g. • Poverty rate Government and multilateral • Political stability
increased employment, income • Foreign exchange earnings from food and organizations • Absence of natural disasters
generation, increased market agricultural exports • Studies prepared for Global • Exchange rate stability
access, poverty reduction, • Share of smallholders and small and medium Burden of Foodborne Disease • Absence of extreme
improved public health, etc.) sized enterprises (SMEs) in overall exports Initiative weather events
• Incidence of food-borne diseases
Purpose Creation of a viable, • Record of compliance or non-compliance • Rejection databases of trading • Government commitment
(medium- functioning, resourced and • Increase in agri-food exports to new and partners (e.g. EU RASFF) to improve SPS capacity
term) transparent SPS system with existing markets • Reports on animal and plant • Government commitment
capacity to ensure food safety, • Reduction in rejections of agri-food exports health and food safety to promote agri-food trade
animal and plant health due to SPS issues outbreaks/situation submitted to • Transportation and
(including the ability to meet • Reduction in incidence of food-borne disease international organizations and financial infrastructure
international SPS and improvement in food hygiene trading partners • Good governance and rule
requirements). • Notifiable animal/plant diseases are controlled • Websites of SPS Enquiry Points of law
• Reduction in no. of pest and disease incursions • PRSPs
• Increase in (equitable) employment • National development plans
• Increased sourcing of agri-food products from • Notifications to WTO
domestic producers • Trade statistics
• Increased agricultural productivity • Employment statistics
• Information dissemination tools for national, • Reports from sector associations
regional and international stakeholders39 • Studies on Disability Adjusted
• Government funding for the SPS system Life Years (DALY)
• SPS is integrated into national economic • etc.
development plans and processes
• SPS issues are considered by in-country donor
coordination mechanisms
38
This draft logframe includes an indicative "working" set of indicators, which will be further developed and improved during the pilot testing phase. It is also
intended that the logical framework and indicators to emerge from the pilot testing work would be modified and adapted by stakeholders in countries based on their own
particular national circumstances and needs.
23
Outcomes 1. Government agencies, the • National SPS committee established in a •
private sector (including strategic ministry
small producers, traders, • Information dissemination tools for different
industry, and their stakeholders (e.g. database reports, video, other
associations), consumers media)
and donors understand the • SPS issues are integrated into national
relationship between SPS development plans
capacity and national • SPS is integrated into national discussions on
development, improved trade
health, etc. and are • SPS requirements are integrated into value
committed to strengthening chain development
SPS capacity • SPS awareness among government agencies
• SPS awareness among private sector
• SPS awareness among consumers
• Percentage of products that meet SPS
requirements
• Existence of research programmes on SPS
issues and dissemination of findings
39
National stakeholders may include relevant public sector organizations (including government ministries and departments responsible for food safety, animal and
plant health, competent authorities, relevant national committees), the private sector (producers, traders, suppliers, enterprises including SMEs, etc.) and their associations,
civil society 8e.g. consumer associations). Regional stakeholders may include trading partners and government organizations in the region, regional economic communities
(RECs), regional private sector or civil society organizations. International stakeholders may include trading partners and government organizations, international
organizations, donors, international private sector or civil society organizations, etc.
24
2. Development and • Stakeholder engagement in development of
implementation of a policy, laws and regulations
legal and regulatory • Existence of SPS Policy, Strategy and/or
framework [dependent on Action Plan that takes into account SPS risks
country legal system] for faced and market opportunities
SPS management [in • Roles and responsibilities of stakeholders (i.e.
accordance with the SPS Enquiry Point, Notification Authority,
appropriate level of food safety agencies, NPPO, veterinary
protection] authority, etc.) are clearly defined, understood
and budgeted for
• Accountability and management system
established
• National standards harmonized with
international standards (Codex, IPPC, OIE)
• Publication of laws and regulations
• Stakeholder engagement in development and
review of implementation mechanisms
25
3. SPS decision-making is • Existence of SPS coordination mechanism • Meeting reports
coordinated, collaborative • Active participation of SPS stakeholders • TORs for coordination
and transparent (public and private) in coordination mechanism mechanisms
• Decisions on allocation of SPS resources are • Documents analysing costs and
prioritized based on evidence of risks and benefits of different options
market opportunities • Tools for dissemination of SPS
• Mechanism to receive, analyse and act on SPS notifications to national
notifications of trading partners stakeholders
• etc.
4. Stakeholders (including • Work programmes

government, enterprises, • Standard operating procedures or procedural
SMEs, small-scale documents
producers, suppliers, • Continuity of operational funding
traders, etc.) have capacity • Available qualified technical and managerial
to carry out their roles and staff
responsibilities40 (as • Information management systems for food
articulated in legislation) safety, animal diseases and plant pests
including international • Up-to-date lists of animal diseases / plant pests
obligations where relevant for quarantine (as well as regulated non-
quarantine pests)
• No. of samples collected
• No. of diagnostic tests
• No. of disease / pest survey carried out
• Accountability, management and budgetary
monitoring systems in place
• Private sector application of good practices,
guidelines, etc.
• No. of producers / farms certified in good
practices
40
Including diagnostic capacity, inspection, monitoring and surveillance, certification, etc. for food safety, animal and plant health
26
5. Relevant national agencies • Participation in international standard-setting
are able to engage with bodies and SPS Committee
trading partners, • Knowledge and confidence to contribute to
international standard- development of international standards (Codex,
setting bodies, relevant OIE, IPPC)
regional bodies, WTO, etc. • Existence of national stakeholder consultations
and reporting mechanisms
• Knowledge and confidence to comment during
SPS Committee meetings
• Knowledge and confidence to submit and
comment on SPS Notifications
• Publication of SPS measures
• Market access agreements for new products
and in new countries
27
Impact indicators
65. As indicated in the logical framework above, impact indicators attempt to measure the
ultimate goal of a national SPS system in achieving progress towards shared higher-order objectives
such as economic growth, poverty reduction or sustainable development. Much more work needs to
be done on measuring SPS performance and on use of indicators to understand the impact of SPS
capacity on market access and development. However, there are obvious challenges, not least since
both development and the measurement of SPS capacity are multidimensional, highly complex
processes and it is often difficult to identify causal relationships.
66. Some initiatives – including the SPS Action Plan for the Greater Mekong Sub-Region,
financed by the Asian Development Bank – have taken steps to identify the possible benefits of
addressing deficiencies in SPS capacities and performance for producers and consumers in the
country and its trading partners. These are presented in Table 3.
Table 3. A possible set of SPS impact indicators41

Criteria Possible indicators Discussion
(1) Higher income from agriculture
and related enterprises through
lower losses of production,
specifically:
¾ reduced incidence of plant Amount of income gains; Estimation of gains may be difficult,
pests will lower losses of income gains per dollar of given the limited readily available
production product agronomic and phyto-pathological
information
¾ reduced incidence of Amount of income gains; Estimation of gains may be difficult
contagious animal diseases income gains per dollar of given the limited readily available
will result in lower mortality product animal husbandry and veterinary
and morbidity of animals and information
reduced loss of animal
products
(2) Higher income and wellbeing for
consumers through reduced
incidence of food-borne diseases
and related reduced morbidity
and mortality, specifically:
¾ reduced loss of productive Amount of income gains; Previous WHO work on DALYs or
time and improved healthy income gains per dollar of similar studies could be used for
life product estimation of incidence and reduction
of incidence. A valuation for healthy
life could be used.
¾ lower cost of medical Amount of income gains; A survey study is needed to assess
treatment and drugs income gains per dollar of the treatment cost per case of
product disease.
41
Provided by Kees van der Meer and based on Action Plan 2010-2015 for Improved SPS Handling in
GMS Trade. ADB. 2010.
28
Criteria Possible indicators Discussion

(3) Higher income for producers, Amount of income gains; This will require baseline studies
trading enterprises and income gains per dollar of estimating handling costs invoked by
consumers through lower product. SPS measures throughout the supply
transaction and mark-up cost, chains, and reductions in cost that
(Gains are the sum of cost
especially: can be achieved:
reduction of sub-indicators
below.)
¾ reduction of unnecessary Amount of cost; cost Assessment of unnecessary measures
SPS measures reduction per dollar of and their costs
product
¾ more efficient Amount of cost; cost Assessment of costs involved in
implementation of SPS reduction per dollar of implementation of measures and
measures product achievable savings
¾ reduction of informal Amount of cost; cost Survey among stakeholders to assess
payments reduction per dollar of costs incurred.
product
¾ reduced risks for private Amount of cost; cost A survey can touch upon some
investors reduction per dollar of perceived risks, but quantification of
product related costs (“insurance premium”)
will be difficult
(4) Increased economic growth, Net value added (=increased
employment and income through sales-incurred cost)
improved competitiveness, lower
(Gains are sum of gains from
cost of doing business, improved
sub-indicators below)
market access and import
substitution, specifically
¾ more investment because of Net value added (=increased Measurement of increases in
increased competitiveness sales-incurred cost) competitiveness, investment and
and lower cost of doing supply response will be necessary
business but difficult.
¾ improved market access and Net value added (=increased Predictability of increased market
market opportunities because sales-incurred cost) access varies between product
of compliance with market combinations (countries and
importing country products), but specialists should be
requirements able to design output and outcome
targets in terms of market access.
However, supply response and
attribution will remain difficult, in
part since factors outside the SPS
domain may sometimes play
dominant roles, such as volume,
quality and consistency of supply.
¾ import substitution of tier 1 Net value added (=increased In particular in food safety a larger
and tier 2 products. sales-incurred cost) part of purchases in tier1 (if any) and
tier 2 markets may be procured from
domestic sources. Supply response
will be difficult to assess in part
since volume, quality and
consistency of supply may in some
cases be of dominant concern to
buyers.
29
67. While indicators to measure the "economic" impact of SPS performance are clearly
important, there may be scope to further develop social, environmental and governance indicators
linked to SPS performance, particularly in terms of linkages to the Millennium Development Goals,
and this should be further explored during the pilot testing work.
VI. Common challenges in the development and/or application of SPS indicators
68. A number of common challenges are faced in the design and use of SPS indicators in practice.
Attribution and causality are key challenges. It is extremely difficult, as discussed above, to clearly
identify the links between activities to strengthen SPS capacity and their long-term impact on
economic growth or poverty reduction. Similar challenges exist in trade-related technical cooperation
more generally; there is increasing recognition that trade-poverty linkages require more
comprehensive analysis and conceptual underpinning.42
69. Quantifying long-term impacts is complex for a number of reasons including: (i) the number
of interventions (with and without donor support), as well as the linkages and interdependencies
between them and resulting problems of attribution; (ii) the time required to observe results; and (iii)
the importance of other factors outside the scope of SPS (e.g. transportation or financial
infrastructure). Reflecting these challenges, the European Commission, one of the largest donors in
the area of SPS and Aid For Trade, has decided to focus its monitoring and evaluation activities on
measuring outputs and possibly outcomes, noting that it is not "realistic to monitor trade impacts of
specific aid programmes because of the important number of external factors influencing trade".43
70. Availability and reliability of data, including the frequent lack of baseline data, is another
important challenge. Many countries lack capacity to produce and report the data necessary to track
and measure progress in the SPS area, as well as to adequately interpret available data. Limited
knowledge about the theory and practice of results-based management, including the logframe's
output-outcome-impact model and the terminology used, poses another difficulty.
71. Inadequate financial resources for monitoring and evaluation of individual SPS projects – as
well as the combined effects of SPS programmes and projects at the macro level – often exacerbates
the challenges in applying results-based management. Results-based management requires technical
capacity and financial resources to establish baseline data, monitor implementation (for instance
through data collection, reporting and/or surveys), interpret and analyse data, and make
recommendations to adapt activities accordingly. The STDF/OECD research on good practice
acknowledged that managing for results requires a minimum level of capacity to formulate and
implement SPS-focused policies and manage public resources to achieve goals. However, it also
highlighted that qualified and experienced managers are scarce in the SPS services of many
countries.4445
72. Many of the challenges faced in the SPS area in implementing results-based management also
apply in agricultural development and trade-related assistance more broadly. This has been
highlighted by an OECD review of the key findings, lessons learned and recommendations emerging
from evaluations of trade-related assistance undertaken by several bilateral donors and multilateral
agencies (Box 6), as well as in recent work by the OECD/WTO on Aid For Trade indicators.
42
Federal Ministry for Economic Cooperation and Development, Germany. Shaping German Aid for
Trade – Past Experience, Lessons Learnt, and the Way Forward. Discourse 013. June 2009.
43
EC response to OECD/WTO Donor Questionnaire on Aid for Trade 2008.
http://www.oecd.org/dataoecd/2/52/43039136.pdf
44
STDF/OECD. 2008. Good Practice in SPS-Related Technical Assistance. An Overview and
Synthesis of the Findings of STDF/OECD Research. G/SPS/GEN/875.
45
Gascoigne. 2007.
30
Box 6. Challenges with results-based management from the perspective of trade-related assistance
• Designing realistic and measurable objectives for trade-related assistance agreed upon by donors and the
partner country and in line with, or linked to, national development and poverty reduction objectives.
• Developing adequate quantitative and qualitative indicators to measure performance at output, outcome
and impact levels.
• Difficulties in evaluating cost-efficiency due to lack or inaccessibility of financial information.
• Complexities in isolating the contribution of one activity from other possible contributing factors (e.g.
other donors activities, national reform, external change, etc.).
Source: OECD 2007. Trade-Related Assistance. What do Recent Evaluations Tell Us?
73. Lastly but not least, as with Aid For Trade more generally,46 there is recognition that
measuring the impact of SPS capacity building interventions will never be easy given the difficulty in
establishing the counterfactual (i.e. testing the opposite hypothesis). The key question is: "Would the
change have occurred anyway or is it due to the (set of) capacity building intervention(s)?" This
question may be answered by identifying and estimating causal effects through counterfactual
methods. However, this is complex. The challenge for quantifying effect is finding a credible
approximation to what would have occurred in the absence of the intervention, and to compare it with
what actually happened. The difference is the estimated effect, or impact, of the intervention, on the
particular outcome of interest (e.g. per capita GDP, export volumes or incomes).47
74. Defining the counterfactual against which impacts are estimated is difficult, especially where
other influencing factors vary unsystematically with the state of SPS capacity. For example, how to
separate out the impact of enhancements in SPS capacity on exports flows from other plausible
influencing factors, for example transport costs or shifts in world market prices. Further, how to
estimate wider spill-over effects that may represent a significant part of the impact, for example
effects on small-scale producers or the environment.48
VIII. Some initial recommendations to support the identification and use of SPS indicators
Adapt these provisional indicators for use at the country level
75. Some degree of adaptation will be required in order to apply this set of indicators at the
national level. It may also be possible to adapt this set of indicators to generate different sets of
indicators for particular types of countries (e.g. least developed, middle-income, higher-income, etc.).
Choosing indicators is ultimately a political process in that it reflects priorities and induces
accountability. On the technical side, the choice of indicators depends considerably on the types of
data available. The availability of baseline data will be of critical importance to effectively use and
track these indicators. However, in several countries, data availability is likely to be limited and
efforts to use SPS indicators will need to be accompanied by work to gather and/or manage relevant
data.
76. Additional modifications may also be needed to adapt the logframe and indicators proposed
here so that they are contextually appropriate and fit with the approach to results-based management
that is being used by government agencies and donors within a country.
46
OECD/WTO. 2009. Aid for Trade at a Glance 2009: Maintaining Momentum.
47
http://ec.europa.eu/regional_policy/sources/docgener/evaluation/evalsed/sourcebooks/
method_techniques/counterfactual_impact_evaluation/index_en.htm
48
Henson, Spencer. Guidelines on the Use of Economic Analysis to Inform SPS-related Decision-
Making. Prepared for the STDF. November 2009. See:
http://www.standardsfacility.org/files/EconomicAnalysis/STDF_Coord_291_Guidelines_22Jan10.pdf
31
77. The proposed STDF work to pilot test SPS indicators will explore these issues in more detail.
The idea would be to take the indicators proposed here as a starting point and to work with selected
national authorities responsible for SPS management to tailor these indicators to national
circumstances and SPS objectives, ideally in such a way that comparison is still possible. In that way,
concerned stakeholders in the country could discuss indicators for the SPS system in their country,
possibly within the context of activities to develop SPS action plans, select indicators of most
relevance, and identify baselines and targets for these indicators.
78. One recommendation from the technical working meeting focused on paying greater attention
to the regional dimensions of indicators, particularly for countries that are working towards regional
integration. This could be promoted by encouraging countries to share and exchange information on
the SPS indicators they select in an effort to reach some degree of regional harmonization of SPS
indicators, wherever relevant and practical.
Strengthen data collection, reporting and management

79. Improving the collection, reporting and management of SPS-related data and information is a
prerequisite to strengthen results-based management and the use of indicators. Competent authorities
responsible for SPS management in many developing countries need better capacity to collect and
analyse data that can be used to measure SPS performance and results based on the hierarchy of trade-
related management functions, and to support their work more generally (e.g. setting risk-based
priorities for inspection), as has been recognized in the STDF work on the use of economic analysis to
inform SPS-related decision-making.49
80. Improving data collection, reporting and management may require additional financial and
human resources. In some cases, in order to ensure effectiveness of SPS data and information
management systems and their use, it may also be necessary to review roles and responsibilities of the
various stakeholders involved in data collection, reporting and management in the SPS area to avoid
overlaps and duplication. This will also be important to ensure that any different information systems
that are used are compatible with each other. Linking SPS data collection, reporting and
management, if possible, to data collection for national development programmes and strategies
(including Poverty Reduction and Strategy Papers) would further ensure that SPS systems are fully
integrated into other relevant national reporting mechanisms.
Use some fundamental guiding principles

81. The World Bank50 and OECD51 have identified a number of principles to guide the selection
of environmental and agro-biodiversity indicators, respectively. These principles are highly relevant
for SPS indicators and should be used to guide their development and application.
• Relevance: The selection of SPS indicators should start from a precise understanding of
national SPS objectives (or project objectives in the case of micro-level indicators) and the
overall SPS situation. Use of the logframe approach provides a practical means to link
output, outcome and impact indicators to objectives.
• Limitation in number: It is most effective to be selective and use smaller sets of well-
chosen indicators. Using too many indicators risks diluting their usefulness.
49
STDF. 2009. Using economic analysis to inform SPS decision-making. STDF Briefings. No. 3.
See: http://www.standardsfacility.org/files/EconomicAnalysis/STDF_BRIEFINGNo3_10389_09_LR.pdf
50
World Bank. 1999. Environmental Performance Indicators. A Second Edition Note by Lisa
Segnestam. Environmental Economic Series. Paper No. 71.
51
OECD Agri-Biodiversity Indicators: Background Paper. Prepared by Kevin Parris, Policies and
Environment Division, Agriculture Directorate, OECD. http://www.oecd.org/dataoecd/9/13/40350839.pdf
32
• Clarity in design: It is important to define indicators clearly in order to avoid confusion in

their development and use, and maintain the distinction between output, outcome and impact
indicators as far as possible.
• Feasibility: SPS indicators should be practical and realistic, in terms of current or planned
data availability. They should also be cost effective in terms of data collection, processing
and dissemination. This may lead to trade-offs between the information content of various
indicators and the costs of collecting them.
• Clear identification of causation: Causal links must be clearly identified. Where causal
links are not clear, it may be necessary to rely on indicators which are more general in nature,
for instance, describing the state of SPS capacity and not the impact.
• Quality and reliability: Indicators, and the information they provide, are only as good as the
data from which they are derived. Ideally an indicator should represent a reliable measure
and should have a sound statistical and scientific basis. If the "ideal" indicator to measure an
SPS problem is based on unreliable data, it may be best to use a proxy or alternative instead.
• Appropriate scale: SPS capacity building activities may have impacts that go beyond the
area in which they are implemented. For instance, a programme to control fruit fly in one
country may benefit producers in border areas of a neighbouring country. SPS indicators
could therefore also focus on impacts beyond the national level and, in some cases, it may be
beneficial to develop and use regional indicators.
• Timeframe: In many cases, there are substantial time lags before the effects of SPS capacity
building activities become clear. For instance, it often takes several years for countries to
establish surveillance and eradication programmes for plant pests and for their trading
partners to recognize these systems.
• Targets and baselines: The purpose of SPS macro-level indicators is to monitor and
evaluate the medium and long-term effects arising from SPS projects and programmes, as
well as actions or initiatives (e.g. development and implementation of an SPS policy or
strategy) led by national stakeholders. Baseline data providing a measure of particular SPS
problems or capacity is required. For micro-level indicators, data is needed before the project
or programme begins, during implementation and after the project or programme has ended
(to compare baseline values to targets).
• Easily interpreted: Variations in the direction of change of the indicators over time should
be clearly understood by concerned stakeholders in terms of an improvement or deterioration
in SPS performance at the macro or micro level as appropriate.
Enhance capacity in the area of results-based management in particular, and management

capacity in general
82. Increasing awareness and knowledge about results-based management, and equipping
officials in developing countries with the skills to apply this approach, is also essential. Having
knowledge about results-based management is important for officials responsible for SPS
management, as well as those involved in the design and formulation of particular projects. The EIF
and STDF are planning to pilot test joint training workshops on project design and formulation,
including use of the logical framework approach, in selected countries in 2011. The training materials
to be developed through this work will be of wider use to SPS experts more broadly.
83. Strengthen management capacity is also important. In the SPS services of most developing
countries, qualified and experienced managers are scarce. Yet, management capacity is crucial for the
33
authorities responsible for operating the SPS system as well as those involved in SPS capacity
building projects.52
52
STDF. 2008. Good Practice in SPS-related Technical Cooperation in the Greater Mekong Sub-
region. Prepared for the STDF by Kees van der Meer and Laura L. Ignacio. G/SPS/GEN/872. Sep. 2008. See:
http://www.standardsfacility.org/files/GEN/GEN872.pdf
34
Annex 1
Components and variables used in the IICA PVS SPS Tool
1. Interaction among public sector institutions, and between the latter and the private sector
Capacity of the National Services to collaborate with other entities of the public sector and obtain the
private sector’s collaboration and active participation in the implementation of programs and
activities.
Critical competencies:
1. Information
2. Communication
3. Official representation
4. Coordination mechanisms
5. Coordination between the Capital and the
Mission in Geneva
6. Priority of the issue
2. Capacity to promote access to international markets
Capacity and level of authority of the National Services to support access to and retention of markets.
1. Compliance with regulatory norms
2. Setting of regulatory norms
3. Harmonization
4. Transparency
5. Technical cooperation and special and differential treatment
3. Human and financial capital
Institutional and financial sustainability based on available human talent and financial resources.
1. Updating (this refers to the capacity of the public sector to keep its personnel updated regarding
information and knowledge on the application of the SPS Agreement. This is measured through
performance and the existence and implementation of an annual training plan for the staff)
2. Technical independence
3. Financial and technical resources
4. Human resources assigned to the matter
35

DraftWorkingPaper_RevDec10_0

Uploaded by

Copyright:

Available Formats

DraftWorkingPaper_RevDec10_0

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DraftWorkingPaper_RevDec10_0

Uploaded by

Copyright:

Available Formats

STDF/Coord/293/Working Paper Draft Rev.3 (Dec.

Indicators to Measure the Performance of a National SPS System

STDF / OECD Working Paper (DRAFT)1

6. In essence, the purpose (medium-term) of a viable, functioning, resourced and transparent

9. Various stakeholders are expected to benefit from this work including:

Structure of this paper

II. The aid effectiveness agenda and mounting attention to indicators

Box 1. The Paris Principles on Aid Effectiveness, 2005

Indicators as a key tool within results-based management

17. Results-based management is designed to improve programme delivery and strengthen

Distinguishing between output, outcome and impact indicators

Figure 1. OECD definitions of output, outcome and impact12

Output Outcome Impact

Figure 2. Impact of two different SPS projects on a market access problem13

Market access Market access

SPS project impact SPS project impact

Table 1. Characteristics of output, outcome and impact indicators14

III. Measuring performance and results of national SPS systems

Components and capacities of a national SPS system

Box 2: Definition of SPS Measures

Figure 3. National SPS System

Enabling environment (influencing Regional and international framework

Figure 4. Hierarchy of trade-related SPS management functions18

Suitable and applied regulation

Application of basic "good practices"

Awareness and recognition

Box 3. Key dimensions of SPS capacity*

Technically-demanding risk management functions: The ability to control SPS risks –

particularly, least-developed countries (LDCs), SPS management capacity is unevenly distributed

Rationale for indicators for the national SPS system

Box 4. Use of SPS indicators to support national decision-making

V. Technical issues related to the design and measurement of SPS indicators37

• Geo-political location: Membership of regional economic cooperation agreements or common

VI. Preliminary indicators to measure the performance of a national SPS system

Box 5. Examples of quantitative and qualitative indicators

Number • number of accredited laboratories

Percentage • percent of government budget devoted to SPS management

Ratio • ratio of food safety / animal / plant health inspectors to population

Existence • SPS coordination mechanism established

Table 2: Logical Framework for a National SPS System38

4. Stakeholders (including • Work programmes

Table 3. A possible set of SPS impact indicators41

Criteria Possible indicators Discussion

VI. Common challenges in the development and/or application of SPS indicators

Adapt these provisional indicators for use at the country level

Strengthen data collection, reporting and management

Use some fundamental guiding principles

• Clarity in design: It is important to define indicators clearly in order to avoid confusion in

Enhance capacity in the area of results-based management in particular, and management

Components and variables used in the IICA PVS SPS Tool

2. Capacity to promote access to international markets

3. Human and financial capital

You might also like