Health Policy and Systems Research: A Methodology Reader
Health Policy and Systems Research: A Methodology Reader
Systems Research
A Methodology Reader
Edited by Lucy Gilson
Health Policy and
Systems Research
A Methodology Reader
Edited by Lucy Gilson
WHO Library Cataloguing-in-Publication Data
Health policy and systems research: a methodology reader / edited by Lucy Gilson.
1.Health policy. 2.Health services research. 3.Delivery of health care. 4.Research. 5.Review literature. I.Gilson, Lucy.
II.Alliance for Health Policy and Systems Research. III.World Health Organization.
All rights reserved. Publications of the World Health Organization are available on the WHO web site (www.who.int) or can be purchased
from WHO Press, World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland (tel.: +41 22 791 3264; fax: +41 22 791 4857;
e-mail: [email protected]).
Requests for permission to reproduce or translate WHO publications – whether for sale or for noncommercial distribution – should be addressed to
WHO Press through the WHO web site (http://www.who.int/about/licensing/copyright_form/en/index.html).
The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part
of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of
its frontiers or boundaries. Dotted lines on maps represent approximate border lines for which there may not yet be full agreement.
The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by the World Health
Organization in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are
distinguished by initial capital letters.
All reasonable precautions have been taken by the World Health Organization to verify the information contained in this publication. However, the
published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of
the material lies with the reader. In no event shall the World Health Organization be liable for damages arising from its use.
The named authors alone are responsible for the views expressed in this publication.
Part 1
Introduction to Health Policy and Systems Research ....................................................................... 19
1. What is Health Policy and Systems Research? ..................................................................................... 21
Key characteristics ............................................................................................................................................... 21
Key areas of HPSR .............................................................................................................................................. 22
2. Health systems ........................................................................................................................................... 23
Goals .................................................................................................................................................................. 23
Elements and characteristics ............................................................................................................................... 23
Multi-levels of operation ..................................................................................................................................... 24
Interactions and interrelationships ....................................................................................................................... 25
3. Health system development or strengthening ..................................................................................... 26
4. Health policy ............................................................................................................................................... 28
5. Health policy analysis ................................................................................................................................ 28
Policy actors ........................................................................................................................................................ 29
The focus and forms of policy analysis ................................................................................................................ 29
6. The boundaries of HPSR ........................................................................................................................... 30
What HPSR is ...................................................................................................................................................... 31
What HPSR is not ............................................................................................................................................... 32
The distinction between HPSR and service delivery/disease programme research ............................................... 32
Fuzzy boundaries ................................................................................................................................................ 33
7. Understanding the nature of social and political reality ................................................................... 34
Positivism ............................................................................................................................................................ 35
Relativism ........................................................................................................................................................... 35
Critical realism .................................................................................................................................................... 36
HPSR perspectives on causality, generalizability and learning ............................................................................. 36
References ..........................................................................................................................................................38
Contents 5
Part 2
Doing HPSR: Key steps in the process ...................................................................................................... 41
Step 1: Identify the research focus and questions .................................................................................. 43
Networking and creative thinking ....................................................................................................................... 44
Literature search ................................................................................................................................................. 45
Key challenges .................................................................................................................................................... 46
Identifying the purpose of the research .............................................................................................................. 47
Taking account of multidisciplinarity ................................................................................................................... 51
Finalizing research questions ............................................................................................................................... 51
Step 2: Design the study ............................................................................................................................... 52
Using theory and conceptual frameworks to inform the study ............................................................................ 54
Step 3: Ensure research quality and rigour ............................................................................................... 55
Step 4: Apply ethical principles ................................................................................................................... 58
References ..........................................................................................................................................................59
Part 3
Understanding Health Policy and Systems ............................................................................................. 61
Health system frameworks ........................................................................................................................... 63
Bloom G, Standing H, Lloyd R (2008). Markets, information asymmetry and health care: Towards new
social contracts. ................................................................................................................................................... 63
de Savigny D et al. (2009). Systems thinking: What it is and what it means for health systems.
In: de Savigny D, Adam T, eds. Systems thinking for health systems strengthening. ............................................ 63
Conceptual frameworks for HPSR ............................................................................................................... 64
Atun R et al. (2010). Integration of targeted health interventions into health systems: a conceptual
framework for analysis. ....................................................................................................................................... 64
Bossert T (1998). Analyzing the decentralization of health systems in developing countries: decision
space, innovation and performance. .................................................................................................................... 64
Brinkerhoff D (2004). Accountability and health systems: toward conceptual clarity and policy relevance. ......... 64
Franco LM, Bennett S, Kanfer R (2002). Health sector reform and public sector health worker
motivation: a conceptual framework. .................................................................................................................. 64
Gilson L (2003). Trust and health care as a social institution. ............................................................................. 64
Kutzin J (2001). A descriptive framework for country-level analysis of health care financing arrangements. ....... 64
Vian T (2007). Review of corruption in the health sector: theory, methods and interventions. ............................ 64
Walt G, Gilson L (1994). Reforming the health sector in developing countries: the central role
of policy analysis. ................................................................................................................................................ 64
6 Gilson L, ed. (2012). Health Policy and Systems Research: A Methodology Reader
Alliance for Health Policy and Systems Research, World Health Organization
Part 4
Empirical papers .................................................................................................................................................. 65
Overview: research strategies and papers ................................................................................................. 67
1. Cross-sectional perspectives .................................................................................................................... 72
Blaauw D et al. (2010). Policy interventions that attract nurses to rural areas: a multicountry
discrete choice experiment. ................................................................................................................................. 75
Glassman A et al. (1999). Political analysis of health reform in the Dominican Republic. ................................... 83
Morrow M et al. (2009) Pathways to malaria persistence in remote central Vietnam: a mixed-method
study of health care and the community. ............................................................................................................ 95
Ramanadhan et al. (2010). Network-based social capital and capacity-building programs: an example
from Ethiopia ...................................................................................................................................................... 105
Ranson MK, Jayaswal R, Mills AJ (2011). Strategies for coping with the costs of inpatient care: a mixed
methods study of urban and rural poor in Vadodara District, Gujarat, India. ...................................................... 116
Riewpaiboon et al. (2005). Private obstetric practice in a public hospital: mythical trust in obstetric care. ......... 129
Rwashana AS, Williams DW, Neema S (2009). System dynamics approach to immunization healthcare
issues in developing countries: a case study of Uganda. .................................................................................... 139
Sheikh K, Porter J (2010). Discursive gaps in the implementation of public health policy guidelines
in India: The case of HIV testing. ........................................................................................................................ 152
Contents 7
Part 4 (continued)
4. Advances in impact evaluation ............................................................................................................... 253
Björkman M, Svensson J (2009). Power to the people: evidence from a randomized field experiment
on community-based monitoring in Uganda. ...................................................................................................... 257
Macinko J et al. (2007). Going to scale with community-based primary care: an analysis of the family
health programme and infant mortality in Brazil. ................................................................................................ 292
Marchal B, Dedzo M, Kegels G (2010). A realist evaluation of the management of a well-performing
regional hospital in Ghana. ................................................................................................................................. 303
Wang H et al. (2009). The impact of rural mutual health care on health status: evaluation of a social
experiment in rural China. (Reproduced in the print version only)
8 Gilson L, ed. (2012). Health Policy and Systems Research: A Methodology Reader
Alliance for Health Policy and Systems Research, World Health Organization
Part 5
Reflections on Health Policy and Systems Research ......................................................................... 469
de Savigny D et al. (2009). Systems thinking: Applying a systems perspective to design and evaluate
health systems interventions. In: de Savigny D, Adam T, eds. Systems thinking for health systems
strengthening. ..................................................................................................................................................... 471
English M et al. (2008). Health systems research in a low-income country: easier said than done. .................... 471
Erasmus E, Gilson L (2008). How to start thinking about investigating power in the organizational
settings of policy implementation. ....................................................................................................................... 471
Hanson K et al. (2008). Vouchers for scaling up insecticide-treated nets in Tanzania: methods for
monitoring and evaluation of a national health system intervention. ................................................................. 471
Hyder A et al. (2010). Stakeholder analysis for health research: case studies from low- and
middle-income countries. .................................................................................................................................... 472
Molyneux CS et al. (2009). Conducting health-related social science research in low income settings:
ethical dilemmas faced in Kenya and South Africa. ............................................................................................. 472
Ridde V (2008). Equity and heath policy in Africa: using concept mapping in Moore (Burkina Faso). ................ 472
Sakyi EK (2008). A retrospective content analysis of studies of factors constraining the implementation
of health sector reform in Ghana. ....................................................................................................................... 472
Schneider H, Palmer N (2002). Getting to the truth? Researching user views of primary health care. ................ 472
van der Geest S, Sarkodie S (1998). The fake patient: a research experiment in a Ghanaian hospital. ............... 472
Walt G et al. (2008). ‘Doing’ health policy analysis: methodological and conceptual reflections
and challenges. ................................................................................................................................................... 472
Contents 9
List of Figures
Figure 1 The different levels of health systems ..................................................................................................... 24
Figure 2 The interconnections among the health system building blocks ............................................................. 25
Figure 3 The terrain of HPSR ................................................................................................................................ 30
Figure 4 Multiple research purposes ..................................................................................................................... 50
List of Tables
Table 1 Typical system constraints and possible disease-specific and health-system responses ........................... 27
Table 2 Key elements of knowledge paradigms as applied in HPSR .................................................................... 35
Table 3 Priority research questions in three health policy and systems areas,
results of international priority-setting processes .................................................................................... 44
Table 4 Examples of HPSR questions .................................................................................................................. 48
Table 5 A summary of broad study designs ........................................................................................................ 52
Table 6 Key features of fixed and flexible research strategies ............................................................................. 53
Table 7 Criteria and questions for assessing research quality .............................................................................. 56
Table 8 Processes for ensuring rigour in case study and qualitative data collection and analysis ....................... 56
Table 9 Overview of papers presented ................................................................................................................ 68
Table 10 Procedures to ensure trustworthiness in case study research ................................................................. 162
List of Boxes
Box 1 Suggested topics for health systems research ......................................................................................... 31
Box 2 Topics addressed by existing empirical HPSR studies .............................................................................. 32
Box 3 Broad research questions of interest to national policy-makers .............................................................. 45
Box 4 The HPSR questions of different health policy and systems actors ......................................................... 46
Box 5 The purpose of different types of research ............................................................................................. 49
Box 6 Links between purpose and broad forms of research questions ............................................................. 50
Box 7 Eight ethical principles for clinical research in low- and middle-income countries .................................. 59
10 Gilson L, ed. (2012). Health Policy and Systems Research: A Methodology Reader
Alliance for Health Policy and Systems Research, World Health Organization
About this Reader
About this Reader
What does this Reader offer?
Health Policy and Systems Research (HPSR) is often criticized for lacking rigour, providing a weak basis for generalization of
its findings and, therefore, offering limited value for policy-makers. This Reader aims to address these concerns through
supporting action to strengthen the quality of HPSR.
The Reader is primarily for researchers and research users, teachers and students, particularly those working in low- and
middle-income countries (LMICs). It provides guidance on the defining features of HPSR and the critical steps in conducting
research in this field. It showcases the diverse range of research strategies and methods encompassed by HPSR, and it
provides examples of good quality and innovative HPSR papers.
The production of the Reader was commissioned by the Alliance for Health Policy and Systems Research (the Alliance) and
it will complement its other investments in methodology development and postgraduate training.
However, the knowledge base to support health system strengthening and policy change in LMICs is surprisingly weak
(World Health Organization, 2009). The body of available work is quite limited compared to other areas of health research
and suffers from various weaknesses. Thus, HPSR is criticized as being unclear in its scope and nature, lacking rigour in the
methods it employs and presenting difficulties in generalizing conclusions from one country context to another (Mills,
2012). Review of health policy analysis work, in particular, also shows that research in this area is often weakly
contextualized and quite descriptive, and offers relatively limited insights into its core questions of how and why policies
are developed and implemented effectively over time (Gilson & Raphaely, 2008). As HPSR remains a ‘cinderella’, or
marginal, field in health research these weaknesses are not particularly surprising. Within LMICs there are very few
national researchers working on health policy and systems issues, and there is a lack of relevant training courses (Bennett
et al., 2011). Yet the need is clear – as Julio Frenk, Dean of the Harvard School of Public Health, stated at the First Global
Symposium on Health Systems Research held in Montreux, Switzerland, in 2010:
we need to mobilise the power of ideas in order to influence the ideas of power, that is to say, the ideas of those
with the power to make decisions.
More specifically, the Reader aims to support the practice of, and training in, HPSR by:
n
encouraging researchers to value a multidisciplinary approach, recognizing its importance in addressing
the complexity of health policy and systems challenges;
n
stimulating wider discussion about the field and relevant research questions;
n
demonstrating the breadth of the field in terms of study approaches, disciplinary perspectives, analytical
approaches and methods;
n
highlighting newer or relatively little-used methods and approaches that could be further developed.
Part 1 provides an overview of the field of HPSR in LMICs and some of the key challenges of this kind of research.
Part 3 presents some key references of papers which provide overarching conceptual frameworks for understanding health
policy and health systems.
Part 4 is the main body of the Reader and presents a set of empirical papers drawn exclusively from LMICs. The papers
were selected because they:
n
together demonstrate the breadth and scope of HPSR work
n
provide good examples of different forms of research strategy relevant to HPSR
n
are high quality and innovative.
Part 5 presents a set of references for papers that reflect on specific concepts or methods relevant to HPSR as well as
some of the particular challenges of working in this field.
14 Gilson L, ed. (2012). Health Policy and Systems Research: A Methodology Reader
Alliance for Health Policy and Systems Research, World Health Organization
Doing HPSR: from research questions to reseach strategy
The defining feature of primary HPSR is that it is problem- or question-driven, rather than, as with epidemiology, method-
driven. Therefore, as outlined in Part 2, the first step in doing rigorous and good quality research is to clarify the purpose of
the research, what the study is trying to achieve, and to identify and develop relevant and well-framed research questions.
Good quality work then demands an understanding of the research strategy that is appropriate to the questions of focus.
The strategy is neither primarily a study design nor a method, but instead represents an overarching approach to
conducting the research; it considers the most appropriate methods of data collection and sampling procedure in terms of
the research purpose and questions. The art of study design in HPSR, as with all ‘real world research’, is about turning
research questions into valid, feasible and useful projects (Robson, 2002).
The papers in Part 4 are grouped by research strategy in order to encourage critical and creative thinking about the nature
and approach of HPSR, and to stimulate research that goes beyond the often quite descriptive cross-sectional analyses that
form the bulk of currently published work in the field. The research strategies were chosen to demonstrate the breadth of
HPSR work, covering both dominant and emerging approaches in the field.
They are:
1. Cross-sectional perspectives
2. The case study approach
3. The ethnographic lens
4. Advances in impact evaluation
5. Investigating policy and system change over time
6. Cross-national analysis
7. Action research
Each of the sections in Part 4 includes: a brief overview of the relevance of the research strategy to HPSR; critical elements
of the strategy that must be considered in conducting rigorous work; and an introduction to the selected papers.
We note that secondary research or synthesis methods are not addressed here, and readers interested in that particular
research area are encouraged to use relevant supporting materials. These include, for example, a Handbook developed
with the Alliance support and downloadable from:
http://www.who.int/alliance-hpsr/projects/alliancehpsr_handbook systematicreviewschile.pdf
Robson C (2011). Real world research: a resource for social scientists and practitioner-researchers, 3rd ed. Oxford,
Blackwell Publishing.
Thomas A, Chataway J, Wuyts M, eds (1998). Finding out fast: investigative skills for policy and development. London, Sage
Publications.
The team
A multidisciplinary group of researchers, with a range of relevant experience and organizational bases, supported the
Reader’s development process. The team was led by:
n
Lucy Gilson (health policy/health economics, South Africa/United Kingdom of Great Britain and Northern Ireland)
and included:
n
Sara Bennett (health policy/health economics, United States of America)
n
Kara Hanson (health economics, United Kingdom of Great Britain and Northern Ireland)
n
Karina Kielmann (medical anthropology, United Kingdom of Great Britain and Northern Ireland)
n
Marsha Orgill (health policy/health systems, South Africa)
n
Helen Schneider (public health/health policy, South Africa).
Irene Agyepong (public health manager/health policy, Ghana), Kabir Sheikh (health policy/public health, India) and Freddie
Ssengooba (health systems/health policy, Uganda), also contributed greatly to conceptualizing Part 2, in part through
their collaboration with Sara Bennett, Lucy Gilson and Kara Hanson in a set of parallel papers published in PLoS Medicine
(Bennett et al., 2011; Gilson et al., 2011; Sheikh et al., 2011).
A range of inputs or comments on the Reader’s development were also received from a broader group of colleagues who
deserve a special note of thanks (see below).
Ultimately, however, the selection of papers in this Reader reflects the particular perspectives of those most closely involved
in its development – both on the nature of the field and on what constitutes a good quality or unusual study and paper.
The Reader is, therefore, a starting point for reflection on HPSR, not an end point. It must be seen as a living document
that will develop over time.
Please note that this Reader is mostly available online at: http://www.who.int/alliance-hpsr/resources/reader/en.
16 Gilson L, ed. (2012). Health Policy and Systems Research: A Methodology Reader
Alliance for Health Policy and Systems Research, World Health Organization
For their comments and ideas, particular thanks to:
Virginia Berridge Prasanta Mahapatra Steven Russell
Thomas Bossert Martin Mckee William Savedoff
Susan Cleary Barbara McPake Jeremy Shiffman
Don de Savigny Anne Mills Sameen Siddiqi
Mike English Sassy Molyneux Barbara Starfield
Abdul Ghaffar Susan Fairley Murray Viroj Tangcharoensathien
Miguel Angel González Block Piroska Östlin Sally Theobold
Felicia Knaul Weerasak Putthasri Stephen Tollman
Mylene Lagarde Kent Ranson Wim Van Damme
Asa Cristina Laurell Michael Reich Frank Wafula
John Lavis Valéry Ridde Gill Walt
Bruno Marchal John-Arne Röttingen
Thanks also to all those who participated in the special session to discuss ideas about the Reader that was held as part of
the First Global Symposium on Health Systems Research, Montreux, November 2010.
Publishers acknowledgement
Grateful acknowledgement is made to the various publishers for permission to reproduce in this Reader, the full text articles
included in Part 4.
Commission on the Social Determinants of Health (CSDH) (2008). Closing the gap in a generation: health equity through
action on the social determinants of health. Geneva, World Health Organization.
Gilson L, Raphaely N (2008). The terrain of health policy analysis in low and middle income countries: a review of
published literature 1994–2007. Health Policy and Planning, 23(5):294–307.
Gilson L et al. (2011). Building the field of health policy and systems research: social science matters. PLoS Medicine
8(8):e1001079.
Mills A (2012). Health policy and systems research: defining the terrain; identifying the methods. Health Policy and
Planning 27(1):1-7.
Robson C (2002). Real world research: a resource for social scientists and practitioner-researchers, 2nd ed. Oxford,
Blackwell Publishing.
Sheikh K et al. (2011). Building the field of health policy and systems research: framing the questions. PLoS Medicine
8(8):e1001073.
Travis P et al. (2004). Overcoming health-systems constraints to achieve the Millennium Development Goals. Lancet,
364:900–906.
World Health Organization (2009). Scaling up research and learning for health systems: now is the time. Report of a High
Level Task Force, presented and endorsed at the Global Ministerial Forum on Research for Health 2008, Bamako, Mali.
Geneva, World Health Organization.
18 Gilson L, ed. (2012). Health Policy and Systems Research: A Methodology Reader
Alliance for Health Policy and Systems Research, World Health Organization