35 - Mann - IJTLD - 2010
35 - Mann - IJTLD - 2010
35 - Mann - IJTLD - 2010
THE ABOVE PRINCIPLE is key to attaining the vision of the Stop TB Partnership of seeing a tuberculosis (TB) free world by 2050.1 Access to an effective
diagnosis has long been a concern. Smear microscopy is not sufficiently sensitive to detect tuberculosis
disease in many cases, and particularly not in children and those who are co-infected with the human
immunodeficiency virus. Multi- and extensively drugresistant TB present new diagnostic and treatment
challenges. Thankfully, new diagnostic approaches
using existing tools have been recommended and new
tools are in the pipeline.2
Any new approach or tool must be evaluated before being adopted by national tuberculosis programmes (NTPs); huge sums of money are already
spent on TB diagnosticsestimated at more than
$1 billion per year globally3and resource-poor countries cannot afford to invest in interventions that are
Correspondence to: Gillian Hazel Mann, Liverpool School of Tropical MedicineCRESTHA, Pembroke Place Liverpool
L3 5QA, UK. Tel: (+44) 151 705 3139. Fax: (+44) 151 705 3743. e-mail: [email protected]
[A version in French of this article is available from the Editorial Office in Paris and from the Union website www.theunion.org]
mainly on test accuracy.10,11 While such data are necessary, they are not sufficient to assess the contribution
new diagnostics can make to the universal access requirements outlined in the Global Plan to Stop TB. A
number of stakeholders, the Subgroup for Introducing New Approaches and Tools (INAT) and the New
Diagnostics Working Group (NDWG) of the Stop TB
Partnership, among others,1113 have called for a strong
and comprehensive evidence base to support decision
making with regard to implementation of new and
improved diagnostic tools.
The NDWG has published Pathways to better diagnostics for tuberculosis: a blueprint for the development of TB diagnostics,2 which outlines the required
phases from needs assessment through test development to assessment of epidemiological impact, and
all stages in between (Figure). The Stop TB Partnerships Retooling Taskforce, a precursor to INAT, stipulated the need for evidence that captures not only
the benefits of new tools but also the risks and health
systems implications associated with them.14 This
range of evidence is encapsulated in the Organisation
for Economic Cooperation and Development definition of impact subscribed to by multi- and bilateral
donors who have signed the Paris Declaration on Aid
Effectiveness (2005), which states that impact consists of:
[The] positive and negative long-term effects on
identifiable population groups produced by a development intervention, directly or indirectly, intended or unintended. These effects can be economic, socio-cultural, institutional, environmental,
technological or of other types.15
1519
conditions in terms of diagnosing patients with various TB presentations, especially for the most infectious, and ensuring that they start appropriate treatment, its affordability, ease of implementation and
potential for scale-up (for the health system) and accessibility (especially to poor and vulnerable TB suspects). Articulating and communicating this overall
impact succinctly and with sufficient evidence is essential. It is required for national policy makers to make
rational decisions about which new diagnostic tests
and approaches to adopt, when and how to implement them, how to manage and finance them and how
to ensure sustainable access and appropriate use.14
To compile such evidence in a systematic manner,
we have developed an impact assessment framework
(IAF) that links evidence on inputs to outcomes. This
framework has been included in the NWDGs blueprint and has been adopted by the TREAT TB (Technology, Research, Education and Technical Assistance
for TB) initiative for use in its operational research
and field evaluations of new tools and approaches
that are at late stages of development or have recently
achieved international approval for use in TB diagnosis and treatment.
The present study describes the IAF, provides
examples of how it can be used and suggests
means of overcoming the remaining challenges in its
implementation.
Figure Pathway for the development of tuberculosis diagnostics, from needs assessment to delivery [reproduced from reference 2
with permission from the World Health Organization].
1520
framework (the IAF) to indicate how sufficient information for policy decisions could be collected in a
systematic manner for all new diagnostic tools and
approaches. The sufficiency of information has been
considered in line with the international targets of the
Global Plan to Stop TB and the Millennium Development Goals (MDGs).26 The IAF, with references relating to different types of evidence, is shown in Table 1.
The IAF comprises five interconnected layers:
1
2
3
4
5
Table 1
Layer of
assessment
Kinds of question(s)
being addressed
References
to studies
addressing
these
questions
Layer 1
How well does the new tool work in
Effectiveness
terms of accuracy?
analysis
How many additional cases will be
identied who would otherwise not
have been identied?
How many additional cases will actually
start (and complete) treatment as a
result of using the new tool?
16
Layer 2
Equity
analysis
27
20
21
22
Layer 3
What are the human resource impliHealth system cations of introducing the new tool
analysis
(training, number and cadre of staff)?
What are the infrastructure implications
(equipment, laboratory layout, safety
installations)?
What are the procurement implications
(reagents, consumables,
documentation)?
What are the implications for quality
assurance (internal and external)?
19
Layer 4
Scale-up
analysis
18
Layer 5
Policy
analysis
29
23
28
17
25
perspectives are all important here. Modelling techniques can provide information concerning the epidemiological benefits of scaling up and, when combined
with patient costs from Layer 2, total additional costs
or savings for patients. At the same time mathematical systems analysis techniques can outline the potential constraints to and resources required for scale-up.
When combined with cost analyses from Layer 3, these
can give an indication of total resources required as
well as identify and quantify likely resource gaps.
Layer 5: Policy analysis
This layer critically appraises the new intervention
studied in Layers 14 against other interventions that
are available or may become available for uptake in
the short to medium term. An important part of this
layer is a scoping of the risk that a given new diagnostic test may be supplanted by newer technology
within a short period of time. It requires a rapid assessment of data on other pipeline diagnostics from
the previous four layers and a review of whether
changes made for one diagnostic may provide a better platform for the next technology or, alternatively,
whether the new technology is disruptive,31 or market transformational,32 both terms used to describe a
technology that could radically alter the way in which
TB diagnosis is achieved.
1521
Layer 1
Effectiveness
analysis
1522
as health service use.30 Analysis of socio-economic status may use asset-based measures to define different
socio-economic groups.35 Demographic and health
surveys and more recent TB prevalence surveys are
increasingly using these methods.36,37
For Layer 3, a comparison of the health system inputs is required. Data for this may be gained through
economic analyses of standard vs. new diagnostic interventions, focusing on the health system and not
just the tool, and through interviews with health systems personnel.
Data for these comparisons can be obtained across
all study participants in both intervention and control arms, or through nested sub-studies on more limited numbers. For example, in-depth qualitative and
quantitative research on patient costs incurred during a diagnostic process (either control or intervention) is time consuming, and data are thus only collected for a subgroup of study participants. Data
from Layers 13 can then be fed into the modelling
and other methodologies required in Layers 4 and 5.
We recognise that the type of randomised trial
employed will depend on the stage of diagnostic development to which the IAF is being applied. During demonstration studies (which may be conducted
prior to STAG-TB approval), an explanatory RCT
with well-controlled study conditions and data collection instruments is appropriate. During subsequent
implementation or operational research, a pragmatic
RCT (PRCT) approach using existing health system
data will be more suitable (for a fuller description of
the difference between explanatory and pragmatic
RCTs see Zwarenstein et al.38) There are concerns
that RCT designs deny some patients (those in the control arm) the assumed benefits of a new technology
especially in the implementation research of STAGapproved technologies. Such ethical concerns need
to be addressed, for example by ensuring that the
PRCT includes a scale-up plan, such as through a
step wedged approach in which all sites access the
technology, but in a phased manner, to allow for
comparisons between those with and those without
the technology.
toring and partnership between service delivery programmes, academic organisations and patient organisations will be required. The increased focus on
patient-centred outcomes in particular will provide
opportunities for patient representatives and organisations to become more engaged in the research process. If patient groups are empowered to collect and
analyse relevant dataparticularly in Layer 2it
will give them a greater voice in policy decisionmaking at the national and international levels.
When it was originally developed, the IAF was envisaged as being applicable to single new tests. However, as we move further into implementation research it is clear that it will also need to be applied to
packages of tests, or combinations of existing and
novel tests, along with all the additional inputs required to introduce such packages and combinations
in different algorithms; this challenge is currently being addressed under the TREAT TB initiative.
The questions in the different layers of the IAF do
not all necessarily carry equal weight in any given circumstance. For example, a new test for detecting
drug resistance that is best suited for deployment in a
central reference laboratory may be more important
in monitoring drug resistance patterns than in directly
improving patient access. The questions about which
patient group or type of patient benefits (Layer 2) may
then assume lesser importance, whereas these may be
key research questions in a diagnostic approach or
test that is aimed at point of care.
We also recognise that while the IAF provides a
body of evidence for policy makers, evidence alone is
often not the only driver of policy change; process,
context and sometimes subjective factors, for example expert or political opinion, can also play a substantial role. These factors need clearer and more systematic documentation and analysis in the process
of implementation research. This is the subject of a
forthcoming study by Bissell et al.
There are concerns that accumulating a comprehensive evidence base such as the one we advocate
here will take too long and be too costly; rather than
promoting the rational uptake of new technologies, it
will instead impede the introduction of much needed
innovations. Such concerns are valid, but they must
be balanced against the dangers of the premature
introduction of tools into unprepared and underresourced health systems, often as a result of lobbying or forceful marketing. To counter both sides of
this argument, research and implementation partners
need to come together and collaborate on an unprecedented scale, and with a renewed sense of urgency.
By directly addressing the concerns of policy makers
through the research process, the adoption and implementation of new tools should be achieved more
rapidly, sustainably and with beneficial effects for affected populations.
Acknowledgements
The authors thank the New Diagnostics Working Group (NDWG)
for funding the initial development of this framework and the
United States Agency for International Development for funding
the TREAT TB (Technology, Research, Education and Technical
Assistance for TB) initiative, which has enabled its evolution. They
also thank all those people who have contributed to the process of
developing the impact assessment framework outlined in this article; these include R Thomson, R Beddell, M Van Lettow, A Harries,
R Dacombe, members of the NDWG and members of the TREAT
TB management team.
References
1 Stop TB Partnership, World Health Organization. The global
plan to stop TB 20062015. Geneva, Switzerland: WHO, 2006.
2 World Health Organization Stop TB Partnership New Diagnostics Working Group. Pathways to better diagnostics for tuberculosis: a blueprint for the development of TB diagnostics.
Geneva, Switzerland: WHO, 2009. http://whqlibdoc.who.int/
publications/2009/9789241598811_eng.pdf Accessed September 2010.
3 World Health Organization Special Programme for Research
and Training in Tropical Diseases/Foundation for Innovative
New Diagnostics. Diagnostics for tuberculosis: global demand
and market potential. Geneva, Switzerland: WHO, 2006.
4 World Health Organization. Moving research findings into new
WHO policies. Geneva, Switzerland: WHO, 2008. http://www.
who.int/tb/dots/laboratory/policy/en/index4.html Accessed September 2010.
5 Guyatt G H, Oxman A D, Vist G, et al. Rating quality of evidence and strength of recommendations GRADE: an emerging
consensus on rating quality of evidence and strength of recommendations. BMJ 2008; 336: 924926.
6 Guyatt G H, Oxman A D, Kunz R, et al. Rating quality of evidence and strength of recommendations: what is quality of
evidence and why is it important to clinicians? BMJ 2008; 336:
995998.
7 Schunemann H J, Oxman A D, Brozek J, et al. Grading quality
of evidence and strength of recommendations for diagnostic
tests and strategies. BMJ 2008; 336: 11061110.
8 Guyatt G H, Oxman A D, Kunz R, et al. Rating quality of evidence and strength of recommendations: incorporating considerations of resources use into grading recommendations. BMJ
2008; 336: 11701173.
9 Guyatt G H, Oxman A D, Kunz R, et al. Rating quality of evidence and strength of recommendations: going from evidence
to recommendations. BMJ 2008; 336: 10491051.
10 Pai M, Ramsay A, OBrien R. Evidence-based tuberculosis
diagnosis. PLoS Med 2008; 5: 10431049.
11 Pai M, Minion J, Steingart K, Ramsay A. New and improved
tuberculosis diagnostics: evidence, policy, practice, and impact.
Curr Opin Pulm Med 2010; 16: 271284.
12 Pai M, Ramsay A, OBrien R J. Comprehensive new resource
for evidence-based TB diagnosis. Expert Rev Mol Diagn 2009;
9: 637639.
13 Marais B J, Raviglione M C, Donald P R, et al. Scale-up of
services and research priorities for diagnosis, management, and
control of tuberculosis: a call to action. Lancet 2010; 375:
21792191.
14 Stop TB Partnership, World Health Organization Retooling
Task Force. New technologies for tuberculosis control: a framework for their adoption, introduction and implementation.
WHO/HTM/STB/2007.40. Geneva, Switzerland: WHO, 2007.
15 Organisation for Economic Cooperation and Development
Development Assistance Committee (OECD-DAC). Glossary of
key terms in evaluation and results-based management proposed harmonized terminology. 2002. Paris, France: OECD-
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
1523
1524
37 Muniyandi M, Rajeswari R. Socio-economic inequalities of tuberculosis in India. Expert Opin Pharmacother 2008; 9: 1623
1628.
38 Zwarenstein M, Treweek S, Gagnier J J, et al. Improving the
reporting of pragmatic trials: an extension of the CONSORT
statement. BMJ 2008; 337: a2390.
RSUM
cherche intresses rassembler autant de donnes possibles en rapport avec la politique suivre avant, pendant et aprs la phase de dmonstration de llaboration
de loutil. Les vidences rassembles peuvent tre utilises par les dcideurs politiques internationaux et nationaux pour soutenir des dcisions dadoption, de mise
en uvre et dextension. Linitiative TREAT TB (Technologie, Recherche, Education et Assistance Technique)
utilise lIAF dans sa recherche oprationnelle et dans ses
valuations sur terrain des nouveaux outils et des nouvelles approches du diagnostic de la tuberculose ; lIAF a
t incorpor dans la publication rcente du NDWG :
Pathways to better diagnostics for tuberculosis: a blueprint for the development of TB diagnostics . Cet article dcrit lIAF et les processus employs pour son amlioration et suggre les tapes ultrieures pour surmonter
les dfis que comporte sa mise en uvre.
RESUMEN
Varios interesados directos, entre ellos el Grupo de Trabajo sobre Nuevos Diagnsticos (NDWG) y el subgrupo
de introduccin de nuevos enfoques e instrumentos de
la Alianza Alto a la Tuberculosis, han destacado la necesidad de contar con una base de datos cientficos slida
y exhaustiva, a fin de respaldar la toma de decisiones
relacionadas con la introduccin de nuevos enfoques e
instrumentos perfeccionados de diagnstico. Con el
objeto de recoger estos datos de manera sistemtica, se
ha diseado un marco de evaluacin del impacto (IAF),
que vincula los datos aportados con los resultados
obtenidos.
El marco de evaluacin del impacto comporta cinco
estratos interconectados: el anlisis de eficacia, el anlisis de equidad, el anlisis de los sistemas de salud, el
anlisis de la ampliacin de escala y el anlisis de las
polticas. Este marco pueden usarlo los creadores de
nuevos mtodos diagnsticos y otros grupos cientficos
interesados, durante el desarrollo de un nuevo instrumento con el fin de recoger la mxima cantidad de datos
pertinentes a las polticas, antes de la fase de demostracin, durante la misma o despus de ella. Los datos
cientficos recogidos pueden ser tiles a los encargados
de definir las polticas a escala nacional o internacional,
a fin de respaldar las decisiones de adopcin, ejecucin
o ampliacin de escala. En la iniciativa TREAT TB (Tecnologa, Investigacin, Educacin y Asistencia Tcnica
para la Tuberculosis) se aplica el IAF en las evaluaciones
de la investigacin operativa y de terreno de los nuevos
instrumentos y estrategias utilizados en el diagnstico
de la tuberculosis. Tambin se ha incorporado en la
publicacin reciente del NDWG: Pathways to better diagnostics for tuberculosis: a blueprint for the development of TB diagnostics (Estrategias encaminadas a mejorar los mtodos diagnsticos de la tuberculosis: un
plan de accin para el desarrollo de medios diagnsticos). En el presente artculo se describe el IAF y los mecanismos que permiten mejorarlo y se sugieren nuevos
pasos que contribuyan a superar las dificultades que
plantea su ejecucin.