Policy Program and Project Evaluation A Toolkit For Economic Analysis in A Changing World 1St Ed Edition Anwar Shah Full Chapter PDF

Policy, Program and Project Evaluation:
A Toolkit for Economic Analysis in a

Changing World 1st ed. Edition Anwar
Shah
Visit to download the full and correct content document:
https://ebookmass.com/product/policy-program-and-project-evaluation-a-toolkit-for-ec
onomic-analysis-in-a-changing-world-1st-ed-edition-anwar-shah/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...
Policy, Program and Project Evaluation: A Toolkit for

Economic Analysis in a Changing World 1st ed. Edition
Anwar Shah
https://ebookmass.com/product/policy-program-and-project-
evaluation-a-toolkit-for-economic-analysis-in-a-changing-
world-1st-ed-edition-anwar-shah/
eTextbook 978-1305117204 Systems Analysis and Design in

a Changing World
https://ebookmass.com/product/etextbook-978-1305117204-systems-
analysis-and-design-in-a-changing-world/
Law and Economic Development: Behavioral and Moral

Foundations of a Changing World Kaushik Basu
https://ebookmass.com/product/law-and-economic-development-
behavioral-and-moral-foundations-of-a-changing-world-kaushik-
basu/
Evaluation Practice for Collaborative Growth: A Guide

to Program Evaluation with Stakeholders and Communities
Lori L Bakken
https://ebookmass.com/product/evaluation-practice-for-
collaborative-growth-a-guide-to-program-evaluation-with-
stakeholders-and-communities-lori-l-bakken/
Coping with Stress in a Changing World, 5th Edition
https://ebookmass.com/product/coping-with-stress-in-a-changing-
world-5th-edition/
Sociology in a Changing World 9th Edition, (Ebook PDF)
https://ebookmass.com/product/sociology-in-a-changing-world-9th-
edition-ebook-pdf/
Scientific American Environmental Science for a

Changing World Third Edition
https://ebookmass.com/product/scientific-american-environmental-
science-for-a-changing-world-third-edition/
Taxing Choices for Managing Natural Resources, the

Environment, and Global Climate Change: Fiscal Systems
Reform Perspectives Anwar Shah (Ed.)
https://ebookmass.com/product/taxing-choices-for-managing-
natural-resources-the-environment-and-global-climate-change-
fiscal-systems-reform-perspectives-anwar-shah-ed/
Abnormal Psychology in a Changing World 10th Edition,

(Ebook PDF)
https://ebookmass.com/product/abnormal-psychology-in-a-changing-
world-10th-edition-ebook-pdf/
Edited by
Anwar Shah
Policy, Program and

Project Evaluation
A Toolkit for Economic
Analysis in a Changing
World
Policy, Program and Project Evaluation
Anwar Shah
Editor
Policy, Program and

Project Evaluation
A Toolkit for Economic Analysis in a Changing
World
Editor
Anwar Shah
Governance Studies
Brookings Institution
Washington, DC, USA
ISBN 978-3-030-48566-5 ISBN 978-3-030-48567-2 (eBook)

https://doi.org/10.1007/978-3-030-48567-2
© The Editor(s) (if applicable) and The Author(s), under exclusive licence to Springer
Nature Switzerland AG 2020
This work is subject to copyright. All rights are solely and exclusively licensed by the
Publisher, whether the whole or part of the material is concerned, specifically the rights of
translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on
microfilms or in any other physical way, and transmission or information storage and retrieval,
electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are
exempt from the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information
in this book are believed to be true and accurate at the date of publication. Neither the
publisher nor the authors or the editors give a warranty, expressed or implied, with respect to
the material contained herein or for any errors or omissions that may have been made.
The publisher remains neutral with regard to jurisdictional claims in published maps and
institutional affiliations.
This Palgrave Macmillan imprint is published by the registered company Springer Nature
Switzerland AG.
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface
In recent decades, evaluation as a discipline has progressed from a tool for

post-evaluation to current operational management. Because of this, the
evaluation techniques have come into widespread use both in public and
private sector operations. The practice of this discipline has leapfrogged
the available guidance from the existing literature creating a wide vacuum
especially in evaluating the design and impact of policies, policy analysis,
and advice. This book attempts to fill this void by providing a primer on
both traditional and newer evaluation techniques. The book presents eas-
ily comprehensible and comprehensive tools of economic analysis that are
currently used in the evaluation literature to evaluate public projects, pro-
grams, policies, and policy analysis and advice. It is hoped that the book
would appeal to a wide range of readers interested in this subject such as
scholars, researchers, students, evaluation professionals and practitioners,
policymakers and public managers.
Washington, DC, USA Anwar Shah
v
Acknowledgements
This project took a long time from inception to fruition. The editor is
grateful to the authors for their patience and perseverance. Hopefully,
their patience will be rewarded by the potential impact of this book in
fostering the better design of public policies and programs globally to
advance the public interest.
vii
Contents
1 Introduction 1
Anwar Shah
2 A Primer on Public Sector Evaluations 11

Saubhik Deb and Anwar Shah
3 Economic Evaluation of Projects 59

Robin Boadway
4 The Marginal Cost of Public Funds: Concept,

Measurement, and Applications107
Bev Dahlby
5 Theory-Based Evaluations: Guidelines for Case

Studies in Program and Project Evaluation131
Ewa Tomaszewska
6 A Framework for Evaluating Anti-Corruption Policies

and Programs177
Jeff Huther and Anwar Shah
7 Evaluating External Analytical Advice on Budgetary

Institutions and Allocations203
Jeff Huther and Anwar Shah
ix
x Contents
8 Inter-Sectoral Allocation Choices221

Stuart Landon
9 Evaluation of Decentralization Programs283

Melville McMillan
10 Evaluating the Conditionality of External Development

Assistance Programs333
Anwar Shah
Author Index401
Subject Index405
List of Figures
Fig. 2.1 Results chain in ROME. (Source: Shah 2005) 39

Fig. 2.2 Road map for ROME. (Source: Andrews and Shah 2005) 40
Fig. 2.3 Involving citizens in results-oriented evaluations. (Source:
Andrews and Shah 2001) 41
Fig. 2.4 Result-oriented evaluations as an integral part of result-
oriented management. (Source: Andrews and Shah 2001) 42
Fig. 3.1 Shadow price of a project input. (Source: Author) 75
Fig. 3.2 Effect of an input subsidy. (Source: Author) 85
Fig. 3.3 Indirect effect on a distorted market. (Source: Author) 88
Fig. 3.4 Market for composite consumption good. (Source: Author) 92
Fig. 3.5 Effect of borrowing. (Source: Author) 94
Fig. 3.6 Market for a commodity. (Source: Author) 103
Fig. 4.1 The marginal cost of funds for an excise tax. (Source: Author) 109
Fig. 4.2 The marginal cost of public funds and the Laffer curve.
(Source: Author) 111
Fig. 5.1 Causal assumptions in impact of privatization programs on
corruption. (Source: Author) 136
Fig. 5.2 Causal assumptions in impact of judicial and legal reforms on
corruption (part 1). (Source: Author) 143
Fig. 5.3 Causal assumptions in impact of judicial and legal reforms on
corruption (part 2). (Source: Author) 144
Fig. 5.4 Causal assumptions in impact of civil service reforms on
corruption: improved compensation example. (Source: Author) 150
Fig. 5.5 Causal assumptions in impact of civil service reforms on
corruption: improved management of service delivery
example. (Source: Author) 152
xi
xii List of Figures
Fig. 5.6 Causal assumptions in impact of trade liberalization on

Fig. 5.7 Causal assumptions in impact of tax administration reforms on
Fig. 5.8 Causal assumptions in impact of direct anticorruption activities
on corruption: example of anticorruption agency. (Source:
Author)167
on corruption: example of office of the ombudsman. (Source:
Author)168
on corruption: example of transparency rules. (Source: Author) 169
on corruption: example of decentralization. (Source: Author) 171
Fig. 10.1 Decision tree for country administrators. Note: Outcome in
bold is the (Nash) equilibrium outcome. (Source: Huther and
Shah 1996) 344
Fig. 10.2 Donor loan conditions for a country with conflicting
administrative goals. Note: Both compliance and non-
compliance with loan maintained are stable outcomes
highlighted in bold (actual outcomes depend on domestic
political battles). (Source: Huther and Shah 1996) 346
List of Tables
Table 4.1 Distributional weights, distributional characteristics, and the

SMCF118
Table 4.2 The MCFs for labor taxes and green taxes in the EU 121
Table 4.3 The marginal cost of public funds for major provincial
income taxes in 2018 122
Table 6.1 The influence of anti-corruption programs on officials’
cost-benefit analysis 179
Table 6.2 Empirical evidence on selected anti-corruption programs 184
Table 6.3 Ratings on relevance of a menu of anti-corruption programs 188
Table 6.4 Summary of proposed rating factors for anti-corruption
programs191
Table 6.5 Effective anti-corruption programs based on governance
quality192
Table 7.1 Characteristics of a typical formal PER 204
Table 7.2 Quality of analysis evaluation (a): Clarity, rigor, and
consistency206
Table 7.3 Quality of analysis evaluation (b): Internalization of previous
work208
Table 7.4 Quality of analysis evaluation (c & d): Relevant
collaboration and consultations 209
Table 7.5 Quality of analysis evaluation (e): Summary of inputs and
assessments of output potential of PERs 211
Table 7.6 Evaluation of output potential of PERs: Criteria for
evaluation of output potential 213
Table 7.7 Quality of analysis evaluation (f): Evaluation of PERs’
treatment of borrower constraints 213
xiii
xiv List of Tables
Table 7.8 Overall rating of PER quality and timeliness 214

Table 7.9 Evaluation of PER impact on donor assistance strategies 215
Table 7.10 Evaluation of the impact: client expenditure policies 216
Table 7.11 Evaluation of the impact: cost-efficiency 217
Table 7.12 Overall impact rating 218
Table 7.13 Cumulative matrix of PER quality and impacts 218
Table 9.1 Decentralization components 286
Table 10.1 Responses to donor conditions for reductions in public
sector employment 340
Table 10.2 Payoff matrix in a two-player game 345
Table 10.3 Examples of approaches to improve incentives 347
Table 10.4 Taxonomy of grants and their potential impacts: a stylised view352
Table 10.5 Traditional and output-based (performance-oriented)
conditional grants 354
Table 10.6 Principles and better practices in grant design 356
Table 10.7 Sources of capital financing would differ by type of
investment and fiscal capacity of the recipient government 358
Table 10.8 On making the dog wag its tail: the NPM perspectives 359
Table 10.9 Comparison of two alternate results-based accountability
approaches360
Table 10.10 Implications of conceptual perspectives for loan/grant
conditions: a synthesis 371
Table 10.11 Good practice principles for development policy lending by
the World Bank 377
List of Boxes
Box 4.1 A Numerical Example Comparing the MCF in a Competitive

Market and Under Monopoly 113
Box 4.2 A Numerical Example of the MCF for a Tax on a Pollutant 115
Box 4.3 Incorporating Distributional Concerns in the Calculation of
the MCF 117
Box 10.1 An Output-Based Transfer for School Finance: An Illustrative
Example355
Box 10.2 Selected Examples of Development Assistance That Did Not
Work as Intended 380
xv
Notes on Contributors
Robin Boadway is Emeritus Professor of Economics at Queen’s

University, Canada. He is past Editor of the Journal of Public Economics
and Canadian Journal of Economics, and past President of the International
Institute of Public Finance and Canadian Economics Association. He was
Distinguished Fellow of the Centre for Economic Studies at the
University of Munich. His research interests are in public economics
and policy, and fiscal federalism. He works on fiscal federalism, cost–
benefit analysis and tax policy. His recent books include Fiscal
Federalism: Principles and Practice of Multiorder Governance (2009) with
Anwar Shah, and From Optimal Tax Theory to Tax Policy: Retrospective
and Prospective Views, The Munich Lectures (2012).
Bev Dahlby is Research Director and Distinguished Fellow at The School
of Public Policy at University of Calgary, Canada. Dahlby attended St.
Peter’s College, the University of Saskatchewan, Queen’s University,
and the London School of Economics. Dahlby has published exten-
sively on tax policy and fiscal federalism, including his book The
Marginal Cost of Public Funds: Theory and Applications (2008). He has
served as a policy advisor to the Canadian federal and provincial gov-
ernments. He was a member of Statistics Canada’s advisory council
from 2005 to 2012. Dahlby was a member of the Expert Panel on
Federal Support to Research and Development in 2010–2011 and
the Ecofiscal Commission from 2014 to 2019. In 2016, he chaired
the British Columbia Commission on Tax Competitiveness. In 2019,
Dahlby served on the Blue Ribbon Panel appointed by the Government
xvii
xviii NOTES ON CONTRIBUTORS
of Alberta to review the province’s finances. His international experi-

ence includes advisory work on tax reform for the IMF in Malawi, for
the Thailand Development Research Institute, and for the World
Bank in Brazil and Mexico.
Saubhik Deb is Economist and Independent Consultant with over
20 years of experience in policy research and evaluation. He has worked
with organizations such as the World Bank, Indian Council for Research
on International Economic Relations (ICRIER), and PwC on vari-
ous projects relating to evaluation of public sector programs. He spe-
cializes in impact evaluations using experimental and
quasi-experimental designs. His recent areas of interest include rural
water supply and sanitation, child health, nutrition, and education
and he has several books and journal publications to his credit. Deb
holds a PhD in Economics from Rutgers University, USA and an MA
and MPhil in Economics from Jawaharlal Nehru University, India.
Jeff Huther is Senior Project Manager for the LIBOR transition at the
Federal Reserve Board, USA. His past work at the Fed includes co-leading
the section that assesses money market conditions and the Fed’s bal-
ance sheet projections, overseeing the section that provides analyses
of money and reserves, and providing analyses of money market con-
ditions to the FOMC. He has also served as Assistant Vice President
within the Markets Group at the Federal Reserve Bank of New York.
Prior to his work at the Federal Reserve, Huther was Vice President
for Financial Engineering at Freddie Mac. He also worked for six
years at the US Treasury’s Office of Debt Management including
three years as Director. Earlier postgraduate work included two years
as Senior Analyst at the New Zealand Treasury. He holds a PhD from
Georgetown University, an MA from Boston University, and a BS in
Chemistry from St. Lawrence University.
Stuart Landon is Professor Emeritus of Economics at the University of
Alberta, Canada. He holds degrees in economics from Queen’s University
(PhD), the University of British Columbia (MA) and McGill
University (BA Honors), and has been a visiting academic at the
Australian National University and Victoria University of Wellington.
He has published papers on a wide variety of topics in Canadian, US,
European and Australian economics and finance journals and was
awarded the Canadian Economics Association’s Vanderkamp Prize
NOTES ON CONTRIBUTORS xix
(along with three coauthors) and the University of Alberta’s

Rutherford Award for Excellence in Undergraduate Teaching. He
served in numerous administrative positions at the University of
Alberta and co-chaired the working group that designed the new
revenue allocation model for the University.
Melville McMillan is Professor Emeritus in the Department of
Economics and a Fellow of the Institute of Public Economics at the
University of Alberta, Canada. McMillan’s research and teaching interests
are in public economics, particularly urban and local economics, fiscal fed-
eralism, and the demand/supply of public goods and services. He has
published in these areas and has advised governments and organizations
nationally and internationally (e.g., the World Bank). Although “retired,”
McMillan remains actively involved in academic and policy matters.
Anwar Shah is Non-Resident Senior Fellow at the Brookings Institution,
USA. He is also an advisor/consultant to the World Bank and Distinguished
Visiting Professor of Economics at Southwestern University of
Finance and Economics, China. He has previously served the World
Bank, Government of Canada (Ministry of Finance), the Government
of Alberta, and the UN Intergovernmental Panel on Climate Change.
He has published more than two dozen books in English, Spanish,
Chinese, Vietnamese and Russian languages and numerous articles in
leading economic journals on governance, public management
reforms, budget reform, federalism, local governance, fiscal reforms
and global climate change issues.
Ewa Tomaszewska is Principal Economist at HDR, Canada. She has
over 20 years of consulting experience in economic analysis of infrastruc-
ture projects, regulatory proposals, policies and programs for private and
public sector clients in the United States and Canada. She has extensive
experience across a wide range of economic assessments supporting
client needs with a major focus on cost–benefit analysis studies to
support applications for funding from higher-order governments,
strategic planning and decisions, or regulatory submissions; eco-
nomic and social impacts studies to support stakeholder and commu-
nity relations; and economic policy research and analysis to inform
and support strategic planning, policy decisions, and best practices
development. Tomaszewska holds a PhD in Economics from the
University of Alberta.
CHAPTER 1
Introduction
Anwar Shah
Evaluation as a discipline, in recent decades, has progressed from provid-

ing answers to questions like economic viability, cost-effectiveness, and
efficiency to questions that are critical for effective planning, financing,
design, implementation, and success of a program. Today evaluation can
provide valuable assistance in defining a problem, identifying program
targets, designing of interventions, identifying winners and losers and
organizational strengths and weaknesses, assessing the quality of interven-
tions and performance of program delivery and its impact, suggesting
modifications and alterations, and ultimately guiding a program or project
to its successful end. Evaluation is no longer just a tool for program or
project appraisals but also an important tool in operational management.
It is not just a snapshot of the end; it is also the means to the end (see also
Williams and Giardina 1993). This book provides an easily comprehensi-
ble and comprehensive survey of tools of analysis that are used in the
evaluation literature to evaluate public projects, programs, policies, and
policy analysis and advice. The following paragraphs provide an overview
of the book.
Chapter 2 by Deb and Shah provides a brief survey of the program
evaluation methods, their objectives, strengths, and weaknesses. The
A. Shah (*)
Governance Studies, Brookings Institution, Washington, DC, USA
© The Author(s) 2020 1

A. Shah (ed.), Policy, Program and Project Evaluation,
https://doi.org/10.1007/978-3-030-48567-2_1
2 A. SHAH
methods have been presented in a manner that reflects the changes in

outlook towards public programs and the changing role of evaluation.
Accordingly, we have first presented methods like cost-benefit analysis,
cost-effectiveness analysis, the social marginal cost of funds analysis, and
data envelopment analysis that are appropriate for guiding efficient alloca-
tion and utilization of resources. This is followed by a discussion on
multiple-objective evaluation, which is much more holistic. Apart from
efficiency, it addresses issues like the relevance of a program, effectiveness
of a program in achieving its objectives and sustainability of the program
benefits. Newer multi-criteria approaches such as the Iron Triangle,
Alternate Service Delivery Framework (ASDF), and the Results-Oriented
Management and Evaluation (ROME) are briefly sketched. The Iron
Triangle notes three important constraints faced by public managers—
time, cost, and quality. The ASDF brings into sharper focus the role of
government as a catalyst in managing and coordinating service delivery by
government and beyond government providers. ROME is particularly
noteworthy for recognizing the role of citizens as the principals and vari-
ous orders of governments as their agents. It provides an integrated
approach to managing for results, citizens-based evaluation of those
results, and the processes to hold the government to account by citizens.
Finally, the authors outline the theory-based evaluation approach. It is a
relatively new concept in evaluation literature where the focus is not just
on whether a program succeeds or fails but also on how and why a pro-
gram succeeds or fails.
The chapter concludes that the methods discussed serve different pur-
poses. Methods like cost-benefit analysis, cost-effectiveness analysis, and
data envelopment analysis address the question of efficiency in the alloca-
tion and utilization of funds. In multiple-objective evaluation, the empha-
sis is more on accountability in public sector programs, the effectiveness of
programs, and the sustainability of program benefits. The evaluation is
concerned with issues like identification of program beneficiaries, assessing
their requirements, tailoring of interventions to meet those requirements,
monitoring of interventions to ensure that the appropriate interventions
are being delivered to the participants, and finally the overall effectiveness
of the interventions in achieving the program objectives. So, a multiple-
objectives evaluation plays a big role in program planning, design, and
implementation. A theory-based evaluation assigns an even bigger role to
an evaluation in public programs. It goes deeper into the mechanism
through which the interventions bring about the desired effects. It
1 INTRODUCTION 3
analyzes the causal links between interventions and outcomes. So, instead
of passing a summative judgment on whether a program succeeded or
failed in achieving its objectives, the theory-based evaluation shows why it
succeeded or failed. Thus, it contributes to the development of more
effective programs in the future.
The authors argue that evaluation is different from other social research
in the sense that it derives its questions from policymakers, program spon-
sors, program managers, and stakeholders. So, the applicability of any spe-
cific evaluation method depends on the questions that the evaluator has
been asked to address. When the evaluation question is deciding upon
alternative interventions aimed at producing similar effects, cost-
effectiveness analysis might be more suitable. But it is not useful at all
when the problem is prioritizing among different programs addressing
different problems. A cost-benefit analysis will be more appropriate in that
case. Similarly, data envelopment analysis might not have the valuation
problems associated with the cost-benefit or cost-effectiveness analysis.
But its applicability is limited to comparing efficiencies of similar programs
only. For programs whose efficacies have already been established, multi-
ple objective evaluations might be enough for performance evaluations.
However, for pilot studies or for programs that have not been tested
before, a theory-based evaluation is much more desirable. But it is also
more time consuming and more expensive than any other evaluation
methods. So, the choice of the evaluation method would also depend
upon the availability of time and resources.
Chapter 3 by Robin Boadway summarizes the principles used to evalu-
ate projects (such as individual investment projects, general expenditure
programs, and the implementation of government policies) from an eco-
nomic point of view. Following the principles of welfare economics, the
objective of project evaluation is to measure the costs and benefits to indi-
viduals in society. The chapter begins with some theoretical perspectives
on cost-benefit analysis and then proceeds to discuss the difficulties of
implementing such an analysis and practical ways of dealing with these
problems. He describes project evaluation as “an art, though one with
scientific underpinnings.”
Broadly, the measurement of costs and benefits amounts to a measure-
ment of individuals’ “willingness to pay.” Two methods that have been
developed in the theoretical literature are “compensating variation” and
“equivalent variation,” depending on whether one wishes to use final or
initial prices (respectively) for goods affected by the project. In addition,
4 A. SHAH
one must consider that projects that have costs and benefits spread over
time must use a common set of prices to adjust for inflation and the time
value of money. Policymakers may also wish to add distributive weights—
in other words weighting more heavily improvements in the incomes of
the poorest. Once the discounted stream of costs and benefits is summed
(yielding the Net Present Value, or NPV), the decision rule for project
approval is simply determined by a (positive) sign of the NPV. Finally, the
risk and uncertainty of outcomes should also be included. Other, similar
techniques for project evaluation include the benefit-cost ratio and the
internal rate of return methods. However, these alternative measures have
some problems and may rank projects differently than the NPV criterion.
While the NPV method is in principle the same as is used by the private
sector to guide the choice of investment decisions, the implementation of
this procedure differs in some important ways when applied to the public
sector. This is because the public sector must take into consideration: the
marginal social values or shadow prices (rather than just market prices,
since markets may be distorted) of inputs including labor, capital, and
foreign exchange, and the impact of externalities such as pollution, gen-
eral equilibrium effects of the project, valuation of intangible benefits and
costs (such as time saved due to public transport), excess burden of public
financing (due to distortions of the tax system—the so-called marginal
cost of public funds, or MCF), the social discount rate, and social consid-
erations (such as equity or protection of special groups). Each of these
issues is considered in turn in this chapter.
Chapter 4 by Bev Dahlby probes more deeply into an evaluation tool
that received brief treatment in Chaps. 1 and 2—the Marginal Cost of
Public Funds (MCF), or the loss to consumers and producers caused by
raising an additional dollar of tax revenue. Taxes impose a cost on the
economy if they alter taxpayers’ consumption, production, and asset allo-
cation decisions, leading to a less-efficient allocation of resources. Raising
an additional dollar of tax revenue costs the private sector more than a
dollar if the allocation of resources in the economy becomes more dis-
torted. The marginal cost of public funds, MCF, is a measure of the cost
imposed on the private sector in raising an additional dollar of tax revenue.
Dahlby argues that the marginal cost of public funds should be used in
evaluating the opportunity cost of financing public sector expenditures. It
also provides a guide for tax reform by revealing which taxes impose the
greatest welfare losses in generating additional revenues. The MCF can
1 INTRODUCTION 5
also be used to measure the gains from tax reforms that shift the burden
from the high-cost tax bases to ones with lower costs.
In this chapter, Dahlby introduces the concept of the marginal cost of
public funds (MCF), examples of how the MCFs can be measured, and
examples of how they can be applied to guide tax reform and public
expenditure policies. In keeping with the theme of this volume, the
emphasis is on using the MCF as a tool for public policy analysis. Two
practical examples demonstrate the use of this evaluation tool in public
policy analysis and evaluation. The first is an assessment of the gains from
a tax reform that shifts some of the tax burdens from corporate to the
personal income tax base in Alberta, Canada. The second example illus-
trates how the MCF can be used to derive optimal matching rates for
intergovernmental infrastructure grants in a federation. In presenting
these applications, the author makes a convincing case of the critical
importance of the MCF as a tool for the evaluation of tax and expenditure
policies.
Theory-based evaluation traces the factors that contributed to a specific
outcome for the project. This a challenging task for project evaluation as
it requires establishing a causal chain based upon theoretical consider-
ations and analyzing data on various links in this causal chain. Ewa
Tomaszewska in Chap. 5 provides a guidebook for those interested in
conducting case studies in project evaluation determining the impact of
various projects in combating corruption. The author provides a concep-
tual framework and data requirements for such evaluation case studies for
privation programs, judicial and legal reforms, civil service reforms, trade
liberalization, tax administration reforms, and direct anticorruption activi-
ties (anti-corruption agencies), the Office of Ombudsman, transparency
rules and decentralization.
The impact of corruption on public service delivery performance and
poverty alleviation is widely recognized. A wide consensus has also recently
emerged that corruption is a symptom of failed governance and hence
curtailing corruption requires addressing the causes of misgovernance.
Nevertheless, the menu of potential actions to curtail corruption is very
large so a framework is needed that provides guidance on ordering poten-
tial actions. Prioritization of various actions depends on both the concep-
tual and empirical views of what works and what does not work in the
context of particular countries. Such a framework is also needed for evalu-
ating country anti-corruption programs and policies. Chapter 6 by Huther
and Shah proposes a framework for such evaluations. The chapter
6 A. SHAH
concludes that path dependency is critical in determining the relative effi-

cacy of various anti-corruption programs. For example, in a largely
corruption-free environment, anti-corruption agencies, ethics offices, and
ombudsmen serve to enhance the standards of accountability. In countries
with endemic corruption, the same institutions serve a function in form
only and not in substance. Under a best-case scenario, these institutions
might be helpful, but the more likely outcome is that they help to preserve
the existing system of social injustice. Successful anti-corruption programs
are those which address the underlying governance failures, resulting in
lower opportunities for gain and a greater likelihood of sanctions. Thus,
programs must be targeted to a country’s existing quality of governance.
Past experiences of the industrialized world confirm these conclusions
since, without exception, these countries did not achieve a reduction in
corruption by introducing technocratic solutions but, rather, by encour-
aging a sense of public duty among officials through accountability for
results. Such an accountability culture came about by empowering people
and by decentralizing decision making. These conclusions suggest the fol-
lowing stylized presentation of anti-corruption measures based on the
existing quality of governance. Addressing the governance failures which
distort officials’ cost-benefit assessment is likely to be the only route to
success in countries with high levels of corruption and poor governance
since direct dialogue on corruption is likely to be counter-productive
(resulting in simply another level of corrupt officials under the name of
anti-corruption offices). In countries with poor governance quality, exter-
nal advice can promote economic liberalization, judicial reform, and
greater public participation in public expenditure decisions without explic-
itly raising contentious issues of corruption and, one hopes, without
threatening their existing relationships. In countries with modest levels of
corruption and governance quality, where the existing governance struc-
ture has the capacity to reform, it is an important focus on improvements
in readily identifiable output indicators rather than uncertain measures of
corruption as measures of success. In countries with high governance
quality, explicit efforts to reduce corruption are likely to be successful—
commissions on corruption, ombudsmen, ethics offices, and the like can
rely on an infrastructure of public accountability and transparency to
ensure that their findings result in lower incentives to commit corrupt acts.
Public Expenditure Review (PER) is a widely used tool by the develop-
ment assistance community to develop advice on budgetary institutions
and allocations. This tool has also been used by both industrial and
1 INTRODUCTION 7
developing countries as an aid to public sector reforms. The most com-

mon format of PERs begins with a presentation of an overall picture of the
country’s fiscal situation. This picture typically focuses on the country’s
expenditure trends. This presentation provides the background, and fre-
quently the justification, for the specific issues addressed by the review.
The picture of the fiscal situation is frequently followed by an analysis of
the budget process which typically provides the foil for recommendations
made in the PER. In some cases, providing a picture of the country’s fiscal
situation and outlining the budget process may be the only tasks under-
taken in the PER. In other cases, PERs review selected inter- and intra-
sectoral issues. Almost all formal, and many informal, reviews also include
extensive data on a country’s expenditures. Chapter 7 by Huther and Shah
presents a framework for evaluating the quality and timeliness of PERs as
well as conducting a review of their impact. They highlight specific ele-
ments to consider and to rate in such an analysis and how to develop final
cost-efficiency and benefit-cost ratings.
Given that the empirical evidence on the relationship between govern-
ment expenditures and economic growth is inconclusive, Stuart Landon
in Chap. 8 asks if the composition of expenditures and design of programs
is a better determinant of the effectiveness of government expenditures
than the size of the public sector. Although there are no universal rules, he
suggests that a review of the empirical evidence drawn from a broad num-
ber of countries can help to identify the sectors that should generally
receive the highest priority in government budgets.
In theory, government intervention can improve welfare in the pres-
ence of market failure (such as insufficient competition and incomplete
markets arising from public goods and asymmetric information). However,
the nature and magnitude of the market failure must be known when
designing the program to ensure that the costs of intervention are justified
and that the government has the capacity to successfully carry it out. In
states with weak capacity, government intervention may be more harmful
than the market failure that the program was intended to address. Given
that state capacity is a critical input for the successful implementation of
government programs (and therefore the effectiveness of public expendi-
tures generally), it is worthwhile to invest in the capacity of public admin-
istration and reduce the size while improving the quality of the civil service
(though this may be politically difficult) and developing a system to moni-
tor expenditure effectiveness. Incentives for improved public sector per-
formance may be created by increasing wages and reducing wage
8 A. SHAH
compression when coupled with mechanisms that create accountability for

performance. Other tools for improving the public sector include subject-
ing the public sector to greater competition, involving the private sector
for provision, implementing user fees.
Landon also deals with sector-specific issues, including law and security,
military spending, infrastructure, transportation, operation and mainte-
nance expenditures, education, health, redistribution, regulation, financial
markets, state-owned enterprises, and industrial subsidies. For each of
these, he discusses whether the sector should be a priority in the public
budget as well as ways to make expenditures in each area more effective.
He also considers the efficiency and equity effects of expenditures in addi-
tion to the likely re-distributional consequences. Based on a review of the
literature, Landon assigns a low priority to subsidies to many state-owned
enterprises and private industry, poorly targeted consumption subsidies,
infrastructure that could be undertaken by the private sector, social secu-
rity programs, tertiary education and hospital care, military spending, and
extensive regulatory regimes. High priority should be assigned to develop
an effective legal system to protect and enforce property rights, effective
financial regulation, maintenance of existing infrastructure, and transpor-
tation, communications, and electricity infrastructure. Spending priority
should also be given to improvements in the quality and quantity of pri-
mary education and basic health care, water and sanitation, and well-
targeted consumption subsidy programs for the very poor.
Governments are becoming more decentralized. Political power and
public decision making in many countries around the world have, to vary-
ing degrees, shifted away from central governments, particularly over the
past quarter-century. This movement has been attributed to various forces;
for example, the growing number of democracies, urbanization, increas-
ing literacy, rising incomes, a growing middle class, and the failures of
central governments. The World Bank has been involved in this transition
in developing countries. Given the extent of the movement occurring and
the Bank’s initiatives, there is a natural wish to assess the Bank’s activities
regarding decentralization. A comprehensive evaluation of the Bank’s
undertakings in this area creates the opportunity to understand better the
potential for and limitations of decentralization, to identify the strengths
and weaknesses of Bank activities and practices relating to decentraliza-
tion, and to assist in refining Bank policies concerning decentralization.
Essentially, an evaluation is to generate information that will help the
Bank’s decentralization policies, programs, and practices to be more
1 INTRODUCTION 9
successful. Chapter 9 by McMillan outlines an approach in evaluating the

Bank’s decentralization initiatives. The basic methodology for evaluating
individual decentralization projects is outlined with elaboration compo-
nents of decentralization and steps to be taken in the evaluation. The
method for extending the evaluation across many projects follows. The
problem of selecting or sampling the projects to be evaluated is discussed.
Special treatment is given to projects on community-driven development.
The chapter also presents thoughts on bringing the various analyses and
the analysis of a rather diverse set of projects together and putting the
results into perspective.
Chapter 10 by Shah evaluates the conditionality of development assis-
tance in terms of its intended and unintended consequences. Development
assistance is motivated by altruistic, economic, political, military, and
humanitarian considerations. It is used to advance wide-ranging objectives
such as minimizing risks for loan repayment, efficiency, equity of the pub-
lic sector, overcoming infrastructure deficiencies, promoting growth, facil-
itating poverty alleviation and good governance, combating terrorism,
support for a specific ideology, influence peddling, and economic and
political imperialism. The provision of such assistance is often conditional
as even unconditional assistance almost always carries some explicit pre-
conditions and implicit conditions. Conditions are imposed as part of
lending or grant assistance unilaterally or by mutual agreement of the
donor and the recipient. These conditions form the contractual terms of
such assistance which bind the recipient to expected actions or results as a
quid pro quo for receiving such financial assistance. These conditions can
vary from being very vague to extremely clear and precise. They may
impose formal binding requirements or simply indicate informal non-
binding expectations. The chapter provides conceptual perspectives from
game theory, public choice, fiscal federalism, political economy, new insti-
tutional economics, and New Public Management literature on the design
of external assistance and its potential impacts. It shows how the neglect
of these conceptual considerations can result in a lack of effectiveness of
aid conditionality and waste of such assistance. It provides an overview of
the historical evolution of perspectives on donor-recipient relations and on
the conditionality of external assistance. It highlights the developing con-
sensus by the development assistance community on both the instruments
of development finance and associated conditions. It also briefly notes
progress, or lack thereof, for practice to conform to emerging consensus.
It cites examples where the inappropriate design of conditionality led to
10 A. SHAH
adverse consequences for project and program outcomes. The chapter

provides lessons on major issues in the conditionality of development
assistance.
Reference
Williams, Alan, and Emilio Giardina. 1993. Efficiency in the Public Sector: The
Theory and Practice of Cost-Benefit Analysis. Brookfield: Edward Elgar.
CHAPTER 2
A Primer on Public Sector Evaluations
Saubhik Deb and Anwar Shah
Introduction
Program evaluation has undergone major changes over the years at both a
conceptual and a functional level. It has transformed from a tool designed
mainly to assess the efficient allocation of resources to a full-grown disci-
pline concerned with the all-round development of a program. The mul-
titude of social and economic problems and the constraint on available
resources require prioritization among both problems and programs
directed at addressing those problems. As a result, assessment of the cost
efficiency of public interventions still occupies a prominent role in evalua-
tion research. However, a heightened awareness of the potentials and per-
ils of public problems both at the national and international arena and the
need for proactive measures have brought forth issues like efficacy and
accountability in the parlance of program evaluation. Consequently, evalu-
ation as a discipline has transgressed from providing answers to questions
like economic viability and efficiency to questions that are critical for effec-
tive planning, design, implementation, and success of a program. Today
evaluation can provide valuable assistance in defining a problem,
S. Deb (*)
Kolkata, India
A. Shah
Governance Studies, Brookings Institution, Washington, DC, USA
© The Author(s) 2020 11

A. Shah (ed.), Policy, Program and Project Evaluation,
https://doi.org/10.1007/978-3-030-48567-2_2
12 S. DEB AND A. SHAH
identifying program targets, designing interventions, identifying organi-

zational strengths and weaknesses, assessing the quality of interventions
and performance of program delivery, suggesting modifications and alter-
ations and ultimately guiding a program to its successful end. Evaluation
is no longer just a tool for program appraisals or a snapshot of the end of
a project or program. It also serves as a means to an end.
This chapter provides a brief survey of the program evaluation meth-
ods, their objectives, strengths, and weaknesses. The methods have been
presented in a manner that reflects the changes in outlook towards public
programs and the changing role of evaluation. Accordingly, we have first
presented methods like cost-benefit analysis, cost-effectiveness analysis,
the social marginal cost of funds analysis, and data envelopment analysis
that are appropriate for guiding efficient allocation and utilization of
resources. This is followed by a discussion on multi-criteria or multiple-
objectives evaluation (MCE/MOE), which is much more holistic in
nature. Apart from efficiency, it addresses issues like the relevance of a
program, effectiveness of a program in achieving its objectives and sustain-
ability of the program benefits. Newer approaches such as the Iron
Triangle, Alternate Service Delivery Framework, and the Results-Oriented
Management and Evaluation (ROME) are briefly sketched. Finally, we
discuss theory-based evaluation. It is a relatively new concept in evaluation
literature where the focus is not just on whether or not a program suc-
ceeds or fails but also on how and why a program succeeds or fails. This is
followed by the conclusion. An annex to this chapter provides a brief over-
view of the evaluation approach used by the World Bank.
Leading Approaches to Evaluation

In the following sub-sections, leading approaches to public sector evalua-
tions are highlighted.
Cost-Benefit Analysis
Cost-benefit analysis is one of the earliest methods of program evaluation.
It analyzes the economic viability of programs by comparing their total
benefits with the total costs. When resources are limited and different
programs need to be pitted against one another for allocation of funds,
cost-benefit analysis can be used for prioritization of programs based on
their net worth.
2 A PRIMER ON PUBLIC SECTOR EVALUATIONS 13
Cost-benefit analysis estimates the net present value of a program by

comparing the benefits of the program with the associated costs. The
notion of benefits and costs typically depends on the evaluation perspec-
tive. Costs and benefits calculated from program sponsors’ perspective
would be very different from those calculated from the perspective of pro-
gram beneficiaries and would lead to very different conclusions regarding
feasibility of programs. For public sector projects, cost-benefit analysis
should be conducted from a social perspective. In other words, all costs
and benefits for the community or the society as a whole should be taken
into consideration rather than restricting attention to program beneficia-
ries only. Such benefits and costs can be both direct and indirect. For
example, in an irrigation project, the direct costs are the capital and opera-
tional costs of the project and the compensation costs for the loss of land
due to construction or inundation. The direct benefits are regeneration of
degraded lands, enhanced crop yields, higher agricultural income and
employment, a decline in the variability of agricultural production and
increased sense of livelihood security. The indirect benefits include employ-
ment generation for the irrigation project and improvement in the quality
of life, whereas indirect costs might include the loss in bio-diversity and
the sufferings of people displaced from their lands.
The analysis requires determination of the length of the program—the
number of years over which benefits and costs of the program are to be
evaluated. Monetary values are then assigned to all benefits and costs.
Since benefits and costs materialize over different periods, for comparison
purpose, all current and future streams of benefits and costs are converted
to their present values by using an appropriate discount factor. Net present
value of a program is calculated as the difference between the present
value of its benefits and the present value of its costs. A program is feasible
only when its net present value is positive. Net present value can be used
as a criterion for allocating funds among competing programs. Alternatively,
benefit-cost ratios can also be used. Benefit-cost ratio is the ratio of pres-
ent value of benefits to present value of costs. However, for mutually
exclusive programs where implementation of one program precludes the
possibility of others, the comparison of programs based on benefit-cost
ratios can be misleading. In those cases, net present value should be used
instead.
One of the important aspects of social cost-benefit analysis is the choice
of the appropriate discount rate (Department of Finance 1987). The dis-
count rate reflects the opportunity cost of capital (the rate of return that
could have been earned through alternative investment of the funds). For
public programs, the interest rate on government borrowing can be used
for discounting future benefits and costs. However, if public investment
displaces private investment, such an interest rate will not reflect the true
opportunity cost of capital. In that case, the appropriate discount rate
should be the social opportunity cost rate. The social opportunity cost rate
is the return on private sector investment that has been displaced by the
public project. Another alternative is to use the social time preference rate.
It represents society’s preference between current and future consump-
tion. Social time preference rate is the required additional future con-
sumption that is necessary to compensate for the loss of one unit of present
consumption. Unfortunately, there is no authoritative way of choosing a
discount rate. But the outcome of the cost-benefit analysis crucially
depends on the choice of the discount rate. A lower discount rate puts
relatively more emphasis on future costs and benefits whereas a higher
discount rate puts more emphasis on short-term benefits and costs.
Accordingly, as the discount rate increases, the net present value of a proj-
ect decreases. So, depending on the choice of the discount rate, the net
present value of a project can be positive or negative, thereby making or
breaking the project. A sensitivity analysis (i.e., repeating the same cost-
benefit analysis for different discount rates) is required for checking the
robustness of results. Alternatively, the internal rate of return (IRR) of a
project can be reported. IRR is the discount rate at which the net present
value of a program is zero. So, it gives the decision makers a measure of
risk associated with the project. However, IRR is not a criterion for pro-
gram selection.
Another important aspect of cost-benefit analysis is the estimation and
valuation of social benefits and costs. The analysis involves the estimation
of incremental benefits and incremental costs that can be assigned solely to
the project. So it is important to construct the counterfactual, i.e., what
would happen or would have happened without the project. Incremental
values can then be calculated by comparing the benefits and costs with and
without the project. In ex post analysis, the problem is constructing the
counterfactual. For ex ante analysis, the problem is not only constructing
the counterfactual but also forecasting the expected benefits and costs of
the project. So, there is a certain amount of uncertainty involved in the
estimation of benefits and costs. Once the benefits and costs have been
estimated, the next step is to assign monetary values by using prices that
would reflect their true opportunity costs. When markets are perfectly
competitive, the market prices represent the opportunity costs of goods

and services. However, a perfectly competitive market seldom exists.
There are always market distortions in the form of monopolies, taxes, and
subsidies. In such cases shadow prices are used. Shadow prices are derived
prices that would mimic the prices under perfect competition (Squire and
van der Tak 1975). For non-marketed goods, marginal costs are used as
shadow prices. For marketed goods, shadow prices are calculated by mul-
tiplying the market prices with appropriate conversion factors. However,
the problem arises in the valuation of intangibles that cannot be quantified
and for which no market exists. In an irrigation project, increased sense of
livelihood security, loss in bio-diversity, sufferings of the displaced people
are examples of intangibles. In the absence of market prices, there are
indirect ways of assigning monetary values to these benefits and costs
(Pearce et al. 2006). For example, the costs of pollution can be measured
using depreciation of housing prices (hedonic pricing method).
Alternatively, the project beneficiaries and/or people affected by the proj-
ect can be asked directly to assign monetary values to these intangibles
(contingent valuation method). However, such valuations can always be
subject to criticisms.
Cost-benefit analysis can be useful for evaluation when it is relatively
easy to assign monetary values to benefits and costs. For industrial and
technical projects, cost-benefit analysis resembles a private profitability
analysis (Rossi and Freeman 1993) and gives valuable information on eco-
nomic viability of projects. Moreover, when resources are limited, it pro-
vides a framework for allocation of resources among different programs.
However, the problem arises when cost-benefit analysis is extended to
evaluation of public sector programs. Many of these programs like con-
struction of dams for irrigation purposes have negative spillover effects.
Cost-benefit analysis evaluates programs on the basis of their overall gains
and losses. So it fails to take into account such adverse distributional con-
sequences. A possible remedy is to undertake separate cost-benefit analysis
for different subgroups of the population. This would facilitate a better
understanding of the distributional effects of the program. Moreover,
assigning monetary values to benefits and costs of some programs can be
very complicated and controversial. For example, when the programs
involved are health campaigns and family planning, the problem boils
down to assigning monetary values to human life—a life saved or a life
prevented. So for projects where intangibles are important, the outcomes
of cost-benefit analyses should be used with caution.
Cost-Effectiveness Analysis
Cost-benefit analyses of many public programs are often viewed with skep-
ticism due to the inherent difficulty and subjectivity involved in monetary
valuation of outcomes for which market prices seldom exist. But such
valuations of program outcomes are redundant and can be avoided when
the decision problem is to choose among alternative programs to attain
similar goals. Cost-effectiveness analysis (CEA) can provide useful infor-
mation on cost efficiency of those alternatives. In programs where it is
difficult, if not impossible, to assign a monetary value to the program
objectives, like the benefits of saving a life or raising the educational
achievement levels of students, cost-effectiveness analysis can help the
decision makers in making judicious allocation of resources. As a result, it
has been used extensively for evaluations of health, educational, and envi-
ronmental programs.
The basic purpose of cost-effectiveness analysis is to identify programs
that can attain specific objectives at the lowest cost (Levin 1983). The
analysis involves identification of the alternatives, an estimation of their
costs, and an assessment of their effectiveness. Cost estimation takes into
account both the direct and indirect costs of the intervention. While direct
costs are the costs of inputs, like personnel, equipment, materials, utilities,
etc., indirect costs are the externalities or ‘spillover’ effects associated with
the intervention and should be included in the cost calculation. Many
programs use volunteers and other donated resources. The cost of these
resources would typically be omitted in normal cost accounting. But they
do represent a cost from the society’s perspective. So the appropriate
notion of cost to use is the opportunity cost, which would reflect the true
cost of a resource to the society. When markets are functioning efficiently,
market prices reflect the opportunity cost and, hence, should be used for
cost calculation. If market prices are not available, appropriate proxies
need to be used (Levin 1983).
The cost-effectiveness of a program is measured using either the cost-
effectiveness ratio or the incremental cost-effectiveness ratio (ICER).
Cost-effectiveness ratio is the ratio of the cost of a program to the effects
produced. For independent programs, programs that can be implemented
simultaneously, cost-effectiveness ratios for each of the alternatives are cal-
culated and ranked in ascending order. The project with the lowest cost-
effectiveness ratio is the most cost efficient and gets the highest priority.
For example, if the program objective is to increase the life expectancy of
a targeted population, the possible program alternatives can be provision

of drinking water, provision of health care, or a health awareness cam-
paign. A cost-effectiveness analysis would involve estimation and ranking
of costs per life-year gained for each of these alternatives. The programs
can then be implemented based on their priority rankings and the avail-
ability of funds.
However, for mutually exclusive programs where the implementation
of one program precludes the implementation of another, incremental
cost-effectiveness ratios are used for comparison. ICER is the ratio of the
difference in costs to the difference in effects between two alternative pro-
grams. For cost-effectiveness comparison, the programs are ranked based
on their effectiveness. If a program is more expensive and less effective
than the alternative, the program is dominated by the alternative, and
hence not considered for implementation. This is called the principle of
strong dominance. Once the dominated programs are eliminated, ICERs
are calculated for the remaining alternatives. The principle of extended
dominance rules out any program that has a higher ICER compared to a
more effective intervention. Finally, the programs are selected based on
the availability of funds or the society’s willingness to pay. For example, it
has been observed that the US healthcare system rejects any intervention
with an ICER (compared to the existing system) higher than $50,000 per
quality-adjusted life-year (Owens 1998).
The appeal of cost-effectiveness analysis arises from its operational sim-
plicity. Since cost estimations are very often straightforward and require
much less value judgment, cost-effectiveness analyses are likely to be more
accurate or less inaccurate than their cost-benefit counterparts. Apart from
measuring cost-effectiveness of outcomes, it can also be used for measur-
ing cost economy of program activities and cost efficiency of program
outputs. So, cost-effectiveness analysis can serve as a very useful tool for
program appraisal, planning, and implementation.
The problems of cost-effectiveness analysis are very similar to the ones
associated with the cost-benefit analysis. If the costs and effects of a pro-
gram are distributed over the years, the future costs and effects need to be
discounted for comparison. The choice of the discount rate might affect
the outcome of the analysis. Moreover, if there are qualitative differences
in the outcomes of alternative programs, the differences in quality are
added as costs to the deficient program. A cost-effectiveness analysis, in
such cases, would require a valuation of benefits. So, it would encounter
the same types of problems that it tries to avoid. However, the most
serious problem with this type of ex ante evaluation is the uncertainty

about the future. As Quade aptly pointed out (Quade 1967, p. 11):
“While one may be able to forecast coming events in the sense of mapping
out possible futures, there is no satisfactory way to predict a single future
in terms of which to work out the best system or determine the optimum
policy. Consequently, it becomes necessary to consider a range of possible
futures or contingencies. In any one of these futures, it may be possible to
designate a preferred course of action, but there is no way to determine
one for the entire range of possibilities.” To capture the uncertainty sur-
rounding such measures, in simulation-based studies like health interven-
tions, the focus of research in recent years has been to attach a probabilistic
measure to the incremental cost-effectiveness ratio (Van Hout et al. 1994;
Briggs and Fenn 1998; O’Brien and Briggs 2002). The cost-effectiveness
acceptability curve (CEAC) is one such measure that indicates the proba-
bility that a program is cost-effective in comparison to the alternative for a
given value of the maximum socially acceptable cost-effectiveness ratio.
However, it is difficult to incorporate such measures in survey-based eval-
uation studies. In such cases, it is important to carry out sensitivity analy-
sis, especially when the measures of cost-effectiveness of the alternative
programs are not very different from each other.
Social Marginal Cost of Funds Analysis

This approach argues that to evaluate public expenditure programs, the
social benefits from the program should be compared with the social costs
of the program and at the margin, marginal social benefits (MSB) should
equal marginal social costs (MSC). Marginal social cost is the marginal
cost of funds (MCF) interpreted as the loss of consumer and producer
surplus per dollar of additional tax revenue raised to finance public goods
(see Chap. 4 by Dahlby, this volume). Note that depending upon the tax
instrument used, additional financing may have implications for govern-
ment revenues from other sources or the revenues of other orders of gov-
ernments and these impacts must be quantified to determine the MCF. Also
there are special considerations that need to be taken into account when
attempting to measure the MCF when non-tax distortions such as monop-
oly and externalities also exist. Under monopoly, the degree of tax shifting
(how the tax affects the final good price) is a major determinant of the
MCF. In the case of externalities, taxing goods with positive externalities
raises the MCF beyond what it would be in the absence of spillovers. On
the other hand, taxing negative-externality-producing goods (such as pol-

lutants) instead of non-externality-producing goods creates a ‘double
dividend.’
Data Envelopment Analysis

Data Envelopment Analysis (DEA) is a non-parametric, linear-
programming-based approach to performance evaluation within organiza-
tional units. It is a technique to determine the relative efficiency of a set of
homogeneous units like schools, hospitals, banks etc. DEA was originally
developed as a tool for better evaluation and management of public sector
activities (Charnes et al. 1978, 1979). But over the years, it has been
applied in a wide range of areas like agriculture, mining, fisheries, energy,
banking, health care, education, housing, transportation, market research,
retail outlets, organizational effectiveness, and benchmarking (Charnes
et al. 1994).
For an organizational unit, efficiency can be measured as a ratio of out-
put to input. But in the presence of multiple inputs or multiple outputs,
such a measure can be inadequate unless the inputs and outputs can be
aggregated to form a weighted sum of inputs and outputs. However, such
aggregation requires the inputs and outputs to be expressed in a common
unit of measurement and also a choice of weights for each input and out-
put indicating their relative importance. While a priori choice of weights is
at best subjective, the inputs and outputs are very often measured in dif-
ferent units. They can be non-marketable, intangible, or qualitative in
nature for which prices might not exist. In such cases, DEA provides a
feasible alternative.
In DEA, an organizational unit is efficient if it is not possible to increase
any output without increasing any input and without decreasing any other
output, or if it is not possible to decrease any input without increasing any
other input and without decreasing any output (Charnes et al. 1978,
1981). So, the efficient units are Pareto optimal. The DEA identifies the
efficient organizational units that are the best-practice units and they form
the extreme points or the efficiency frontier. Similarly, a combination of
the efficient organizational units can be calculated to form an efficient
composite unit with composite inputs and composite outputs. These effi-
cient composite units (also called virtual producers) along with the best-
practice units constitute the entire efficient frontier and receive a relative
efficiency score of one. The inefficient units that are inside the frontier are
compared with their respective reference efficient units or the virtual units
to determine their relative efficiency.
So for each organizational unit, the central problem in DEA is to iden-
tify the best-practicing unit or the best ‘virtual producer’ with which it is
to be compared. This can be formulated as a linear programming prob-
lem. The solution to the linear programming problem determines the
weights, and the relative technical efficiency is calculated as a ratio of the
weighted sum of outputs to the weighted sum of inputs. The DEA assigns
the weight in such a manner so that the maximum weight is placed on the
favorable variables and the minimum weight is placed on the variable
where the unit compares unfavorably. In other words, the weights are
chosen in a way that gives the evaluation unit the maximum relative effi-
ciency score.
Apart from identifying inefficient units, DEA can also identify the
sources and level of inefficiency for each of the inputs and outputs and can
assign targets so as to maximize output. So it can be used for identification
of improvement priorities.
One of the main advantages of DEA is the non-parametric nature of the
analysis. It doesn’t require the specification of any functional form. In
parametric approach like regression analysis, a pre-specified production
function relating inputs to outputs is estimated using the data. The esti-
mated regression equation applies to all the organizational units. DEA in
contrast optimizes on each organizational unit. So, in DEA the focus is on
each evaluation unit and how it compares to a reference unit rather than
the entire population average. DEA also avoids the usual valuation prob-
lems associated with cost-benefit and cost-effectiveness analysis. It can
handle multiple inputs and multiple outputs and it does not require the
inputs and outputs to have the same unit of measurement. DEA over-
comes another shortcoming of cost-benefit and cost-effectiveness analysis.
It can incorporate exogenous factors in evaluation (Banker and Morey
1986a). This is done by treating these variables either as inputs or outputs.
It can also handle categorical variables (Banker and Morey 1986b).
As an evaluation method, DEA can be used for identification of ineffi-
cient units, reallocation of resources for efficient utilization, and also for
setting targets for inefficient units to improve performance (Kittelsen and
Førsund 1992; Tulkens 1993). For multi-criteria evaluation, i.e., when
the observational units are evaluated based on multiple criteria, DEA can
also be used to aggregate these criteria to construct a single index of
performance.
However, the main problem with DEA is that it relies on the authentic-
ity of the data. It uses the extreme point method to estimate the efficiency
frontier. So, any error in the data can create significant problems. For
example, the presence of outliers in the sample can have a significant effect
on forming the frontier and can jeopardize the relative efficiency estima-
tions. Moreover, DEA estimates the relative efficiency, efficiency relative
to the best-practice units. But the best-practice units can themselves be
inefficient. Since the efficiency estimates are bounded within zero and
one, DEA cannot capture the differences among organizational units with
an efficiency score of one.
Multiple-Criteria Evaluation (MCE)/Multiple-Objectives

Evaluation (MOE)
Public sector programs emerge out of society’s awareness of the deficien-
cies in the existing social structure and its attempt to rectify these prob-
lems through mitigating interventions. Since social problems are
innumerous and resources are scarce, cost efficiency is an important crite-
rion in program evaluation. However, it is equally important to ensure
that the program interventions are translated into positive outcomes, con-
tributing to the well-being of the program targets. So, apart from cost
efficiency, issues like relevance, the consistency of the program objectives
with the requirements of its beneficiaries, efficacy, the extent to which the
program is successful in achieving its objectives, and sustainability, the
continuation of program benefits even after completion of the program,
are crucial from an evaluation perspective. A multiple-objective evaluation
is an all-round assessment of a program in terms of its relevance, efficacy,
efficiency, sustainability, and its contribution towards development of
institutions necessary for design, delivery, and implementation of success-
ful programs in future. The World Bank uses a multiple-criteria evaluation
approach in evaluating the impact of its programs and projects (see the
Annex for details).
A program, from its inception to its end can be divided into the follow-
ing stages: conceptualization, design, implementation, and outcome. At
the conceptualization stage, the problem is specified and defined as pre-
cisely as possible to facilitate selection of the target group, design of appro-
priate interventions, and later evaluation of the program. Designing a
program is developing a set of interventions or treatments that would be
delivered to the target group and would eventually help in achieving the
program objectives. At the implementation stage, the inputs are used to

generate activities that are delivered to the program participants. The
activities produce a set of outputs. The outputs in turn are translated into
outcomes that are the ultimate goals or objectives of the program.
Multiple-objectives evaluation can be used at different stages of a pro-
gram. Depending on its nature and purpose, it can be broadly categorized
as formative or summative (Scriven 1967). Formative evaluation plays a
very critical role in the overall development of a program. It has two com-
ponents—a diagnostic evaluation and a process evaluation. Diagnostic
evaluations are carried out at the planning and design stage of a program.
It involves specification of program objectives, identification of target
groups, and an assessment of their needs. It also contributes to the design
of program activities by examining the extent to which the interventions
are consistent with the needs. A process evaluation, on the other hand, is
concerned with the monitoring of program activities and operations—
examining program deliveries, organizational strengths and weaknesses,
quality of the interventions, and consistency of the implementations with
the original design of the program. It provides the program staff continu-
ous feedback on the performance of program activities, uncovering any
obstacle that might emerge and suggesting modifications to ensure the
success of the program. So, a formative evaluation assists in the develop-
ment of a program. Summative evaluation, in contrast, assesses the efficacy
of a program in terms of achievement or non-achievement of program
objectives. Summative evaluation includes both outcome evaluation and
impact assessment. An outcome evaluation studies the usefulness of a pro-
gram in delivering its stated objectives. Impact assessment, in contrast, not
only evaluates a program on the basis of its stated objectives but also looks
into the indirect and unintended effects of the program.
The difference between formative and summative evaluation is the dif-
ference in purpose: “When the cook tastes the soup, that’s formative;
when the guests taste the soup, that’s summative” (Robert Stake, quoted
in Scriven 1991, p. 169). However, in spite of their differences, it is impor-
tant to recognize the complementarities between the two. A summative
evaluation is unwarranted and a waste of time, effort, and resources unless
it is possible to ensure that a program has measurable goals, has been well
implemented, and the activities have been delivered to the appropriate
targets. Moreover, with a growing emphasis on result-based monitoring,
formative evaluation includes assessment of early outcome measures that
are linked to the final objectives of the program.
Diagnostic Evaluation
A program conceived at a political, community, or administrative level
contains in it a broad definition of the problem and a list of goals that are
equally broad in nature. However, the planning, implementation, and
evaluation of a program require much more precise and operational defini-
tions that are clear, measurable, economical, and adequate. One of the
main purposes of diagnostic evaluation is to identify and define the prob-
lem. A usable definition is important for understanding the prevalence and
the magnitude of the problem, identification of the target groups and
their specific needs, and the design of interventions to meet the require-
ments of the program targets. For example, in a poverty alleviation pro-
gram, the identification of people who are in need of assistance requires a
definition of poverty. Poverty can be defined either in terms of annual
household income or daily calorie intake. Once the thresholds on income
or calorie intakes are established, only then is it possible to identify the
target population. It is the group that falls below the threshold. So thresh-
olds or boundaries are rules determining the eligibility to participate in a
program. If the rules are too stringent, the target population might shrink
to the level of non-existence. If the rules are too lax, the target population
might be too large and too diverse to design any effective intervention
strategy, and the program might become prohibitively expensive. Target
definitions should also be feasible to apply based on observable and mea-
surable characteristics for which data are readily available (Rossi and
Freeman 1993).
It is also important to collect information on the magnitude of the
problem and the geographic distribution of the target population.
Understanding the magnitude of the problem is necessary because target
populations are hardly homogenous in terms of their degree of depriva-
tions. A family that goes without food for several days at a stretch and a
family that eats one meal a day might be both eligible in a poverty allevia-
tion program. But their needs are very different and accordingly they
might require very different types of assistance. Moreover, in programs
where funds are insufficient to cover the entire target population, such
characterization based on need helps to identify the groups who are the
worst affected. Information on geographical distribution is important for
feasibility of the program and the subsequent outcome evaluations. If the
target population is sparsely located across a geographic region, it might
be difficult and expensive to administer an effective program. And if the
target group becomes too small, it might be impossible to do any outcome
evaluation. For example, an evaluation study of a tobacco use prevention

and cessation project in Southern California found no effect of the pro-
gram in changing the attitude of the students towards smoking (Flay et al.
1995). The researchers concluded that the target group of the seventh
graders had strong anti-tobacco feelings and very low rates of smoking,
making it impossible to find any effect of the program. The problem with
the program was that it failed to define an appropriate target group.
For appropriate identification of target population, concepts like preva-
lence rate, incidence rate, and population at risk can be very useful. The
prevalence rate defined for a given area over a specified period of time is
the ratio of the number of existing cases to the total population at risk.
The population at risk refers to the people who are most likely to develop
a particular problem. For health programs, the characterization might
include the incidence rate. Incidence rate is the ratio of number of new
cases of a particular problem in a given area over a specified period of time
to the total population at risk. These rates can also be estimated by age
group, sex, language, religion, ethnicity, and economic status. A detailed
characterization of the target population is essential for identification of
the appropriate target groups and tailoring of interventions to the particu-
lar characteristics of the groups.
Another important aspect of diagnostic evaluation is the specification of
program objectives in terms of measurable outcomes. In defining the out-
come measure, it is often best to take a collaborative approach (Weiss
1998). Discussions with the program sponsors, staff, and stakeholders can
reveal important evaluation questions. Moreover, in the formulation of
outcome measures, it is important to take into account the opinions of the
sponsors and other stakeholders to ensure that the outcome measures are
close approximations of the program objectives and reflect the views,
needs, and concerns of the major stakeholders.
In recent years, there is a growing emphasis on greater accountability in
public sector programs. Consequently, there is a shift in paradigm from
monitoring of activities and implementations to result-based monitoring.
Under result-based monitoring, the emphasis is more on whether the pro-
gram inputs, activities, and outputs are contributing towards the achieve-
ment of the program objectives (Kusek and Rist 2004). So, it is imperative
to develop indicators not only for performance measurement of program
activities, outputs, and outcomes but also for interim markers of progress
towards achievement of these outcomes.
The fruitfulness of an evaluation, be it a process evaluation or an out-

come evaluation, depends crucially on the development of appropriate
performance indicators. The indicators might be either quantitative or
qualitative. The outputs of a program might be quantifiable for which
quantitative indicators would suffice. But the outcomes are very often less
tangible and might require qualitative indicators. In developing the indi-
cators the C.R.E.A.M criteria can be used. The C.R.E.A.M criteria require
the indicators to be clear, relevant, economic, adequate, and monitorable
(Kusek and Rist 2004). In other words, a good indictor should be precise
and unambiguous, adequate and reliable in measuring the relevant vari-
able, available readily at reasonable cost, and can be validated
independently.
Process Evaluation
The main objective of process evaluation is to provide a systematic assess-
ment of program performance to facilitate progress towards achievement
of program objectives. The focus of traditional monitoring of programs is
concerned with the assessment of inputs, activities, and outputs. A process
evaluation, in contrast, is an evaluation strategy that tries to ensure that
the activities and outputs contribute to the achievement of program objec-
tives. So, a process evaluation is an integral part of a result-based monitor-
ing system. It provides crucial information on program coverage, quality,
and propriety of interventions and effectiveness of the interventions in
producing the desired effects. So it constitutes a feedback mechanism that
promotes informed decision making, better decision making, improved
performance, and greater accountability.
The success of a program depends on its ability to reach the appropriate
targets and the degree to which the targets actually participate in a pro-
gram. Program coverage measures the extent of actual target participation
in comparison to the desired level. Since the effectiveness and approval of
a program depends on its coverage, the objective of any program is to
maximize coverage given its budget constraint. Two related concepts in
this regard are under-coverage and over-coverage. Under-coverage of a
program can be measured as the ratio of the number served who are in
need of the program to total number in need. Over-coverage is the ratio
of the number served who are not in need of the program to the total
number served. The difference between these two ratios indicates a pro-
gram’s coverage efficiency. The higher a program’s coverage efficiency, the
more successful the program is in reaching its desired targets. So, coverage
efficiency is a very useful indicator of target participation that can be used

for modifying target definitions. Higher coverage efficiency can also help
program managers to gain the approval and support of program sponsors
and other stakeholders.
Another problem in public programs is the bias in target participation.
The bias refers to the degree to which different subgroups of target popu-
lation participate differently in the program (Rossi and Freeman 1993).
Such biases might occur due to implementation failure. All participants
might not have equal access to the program. Infrequent participation
might also reflect participants’ dissatisfaction with the program. Some par-
ticipants might be more motivated and find the program more useful. The
program staff can also contribute to the bias by treating participants dif-
ferently. The participants who are more likely to succeed might be treated
favorably whereas the participants who are likely to fail might be encour-
aged to drop out. As discussed later, the presence of such biases in cover-
age can seriously jeopardize the validity of subsequent outcome evaluations.
So, regular evaluation of participants’ satisfaction with a program is neces-
sary not only to reduce such biases but also to modify interventions for
better participation.
It is also important to monitor the delivery of the program and the
quantity and quality of interventions. A delivery is the procedure used to
provide treatments to the program participants. A failure in the delivery
system occurs when the interventions fail to meet the requirements of the
program beneficiaries. This can be either due to a failure to reach the pro-
gram beneficiaries or due to a failure to provide them with the appropriate
interventions. Different participants might receive different levels of inter-
ventions, wrong interventions, or no interventions due to lack of commit-
ment on the part of program staff or failure of the program to standardize
the interventions or both. Such deficiencies in program delivery can also
result in participation biases. Evaluation of the actual delivery of interven-
tions can identify these deficiencies in implementation and suggest correc-
tive measures. Qualitative methods like observing the program in operation
and informal interviews with the program participants can provide valu-
able insights into actual program delivery (Weiss 1998).
Another aspect of process evaluation is to identify the interim markers
of progress and evaluate the effectiveness of the interventions in achieving
them. Interim markers are the short-term objectives of a program that are
causally linked to the overall objectives of the program. A failure to achieve
the interim objectives would also imply an eventual failure in achieving the
goals of the program. An interim evaluation is essential for the perfor-

mance evaluation of the program interventions. Based on the findings of
such evaluations, interventions can be modified or replaced altogether.
A process evaluation is important because it provides information that
is crucial for the success of a program. An outcome evaluation is meaning-
less unless a program successfully delivers the appropriate interventions to
the appropriate targets. A process evaluation provides program managers
timely feedback on performance of different elements of the program and
identifies elements and areas that require improvement or modification.
So it helps program managers to make informed decisions. Process evalu-
ation can also be used for measuring cost economy of inputs in providing
the interventions and the cost efficiency of interventions to produce the
program outputs. It provides information on program coverage and the
effectiveness of the program in achieving its short-term objectives. All
these factors are essential elements of accountability, which is an important
issue in public sector programs.
Outcome Evaluation
Outcome evaluations study the effectiveness of programs in delivering the
program objectives, which can be both short term and long term. Since
process evaluation includes evaluations of program based on short-term
objectives (the interim markers of progress), outcome evaluations can be
used for both formative and summative evaluations.
The basic idea of an outcome evaluation is to study the net effect of an
intervention on participants or beneficiaries in terms of the outcome vari-
ables. The net effect refers to the changes in the outcome measures that
can be attributed solely to the intervention. In other words, it is the dif-
ference between the outcome measures after participation in the program
and the counterfactual, the outcome measures that would have been
observed had they not participated in the program. This is not the same as
differences in outcome measures before and after participation in the pro-
gram because there might be many other factors or events apart from the
program treatments which might directly or indirectly influence the out-
comes. The objective of an outcome evaluation is to establish causality
between the treatments offered and the outcomes produced by filtering
out the effects produced by all these external factors.
The prerequisite for any outcome assessment is to establish a set of
well-defined outcome measures and to recognize the factors apart from
the treatments that might influence these outcomes. The credibility of an
outcome evaluation crucially depends on the reliability and validity of the

outcome measures. Reliability of measures refers to the authenticity of the
data and the validity implies the extent to which the measures represent
the stated objectives of the program. It is also important to recognize and
control for the external factors and processes that might influence the
outcomes. Such external factors include endogenous factors specific to the
participants; short-term and long-term trends at the local, regional, or
national level; maturational factors; biases in the selection of the partici-
pants; and the program design effects. Endogenous factors refer to the
naturally occurring events. For example, in clinical trials, the participants
might recover from an illness naturally due to the inherent defense mecha-
nism of the body. The purpose of outcome assessments in such cases is to
study whether the medicines fasten the recovery process. Similarly, overall
economic conditions like expansions or recessions will affect the outcomes
of a job-training program or an income enhancement program.
Maturational factors related to naturally occurring changes due to the pas-
sage of time might also influence the effects of a program. With age, peo-
ple might get smarter, wiser, and more experienced. This will cause
behavioral changes or changes in receptive capability that cannot be attrib-
uted to the program. Maturational effects are very relevant in the evalua-
tions of educational programs for the young and also in juvenile justice
programs.
The presence of the program itself can influence the behavior of the
participants and in turn affect the program outcomes. This is known as the
design effect and it is an integral part of any program. A classic example is
the evaluation study of the effect of intensity of illumination on workers’
productivity conducted at the Hawthorne Works of the Western Electric
Company in the 1930s (Weiss 1998). It was observed that the productiv-
ity of the workers in the study group increased irrespective of the intensity
of illumination. The fact that they were getting more attention resulted in
higher efforts on the part of the workers. This is known as the ‘Hawthorne
Effect’ and it can be part of any program involving human subjects.
Another major problem associated with outcome evaluations is the
presence of selection biases (Barnow et al. 1980). Selection biases arise
when the program participants are characteristically different from the
non-participants in a way that might affect the program outcomes. The
differences might be due to differences in observable characteristics and/
or due to differences in unobservable characteristics. Very often in public
sector programs there is ‘creaming’ in the selection of targets. The
participants might be selected based on observable characteristics that also

make them more likely to succeed. There is also the possibility of self-
selection. In voluntary programs, the participants might be individuals
who are more likely to gain from the program and are more motivated, a
factor that can itself contribute to the outcome. For example, in evaluation
studies of the effects of job training programs on employment, the partici-
pants might be more motivated to find a job than the non-participants,
and hence they are more likely to find a job even in the absence of the
program. Similarly, in the evaluations of the effects of job training pro-
gram on wage rate, the participants might be individuals with lower skills
than the non-participants and accordingly their salary gains might be rela-
tively smaller. De-selection or attrition bias is also a common phenomenon
in public programs (Heckman et al. 2000). Participants drop out in the
middle of the programs. The dropout process is seldom random. It might
happen that the participants who decided to leave found the program
undesirable and benefited the least from the program while participants
who completed the program did so because they found the program
rewarding. So an outcome assessment based on the participants who com-
pleted the program would overestimate the impact of the program.
Outcome Evaluation Designs

An appropriate outcome evaluation design should control for all possible
external factors and biases that might influence outcomes. The available
evaluation designs in social research can be broadly classified into two
categories—randomized experimental designs, and non-experimental
designs.
Randomized Experimental Designs

Causal impact of a program on its beneficiaries is the change in the desired
outcome of the program that can be solely attributed to the program
interventions. So, theoretically, it can be easily estimated if we can observe
the outcome among the beneficiaries with and without the program.
However, the counterfactual does not exist. An alternative would be to
construct a control group—a group of individuals who would be similar
to the program beneficiaries in all possible aspects but would not receive
the program interventions. We would then be able to estimate the impact
of the program by comparing the post-intervention outcomes among the
program beneficiaries (the treatment group) and the non-beneficiaries
(the control group). A problem with this approach is that the presence of
Another random document with
no related content on Scribd:
we had better only have the kitchen fire in the mornings; but I don't like to
think of you going out to work cold."
"I don't feel the cold," said Luke. "We certainly must make a difference
somehow. Discuss it with mother and see if she can't help. She knows we are
rather in low water."
"I don't see how we can give away so much as you do Luke," said Rachel.
"We really could not afford that £5 that you gave for the heating of the Church
for instance."
Luke looked worried.
"I have always somehow managed to do my part in that way," he said. "I can't
bear not setting an example in giving."
"No, it's horrid," said Rachel. And yet she felt strongly, that if by giving away
money he was deprived of his much needed yearly holiday the work itself
would suffer.
After he had gone, the subject that had caused their talk and had brought to
light their poverty forced itself again upon her.
No holiday! All the summer in this tiny stuffy little house away from the flowers
and the breezy wind. How could she bear it herself, and still more how could
Luke go on working all day and the greater part of the evening in the terrible
heat, which was making her feel giddy and faint already.
She hurried into the drawing-room which was without sun and threw the
window open. Then she looked for her weekly bill books and sat down to
examine them. She saw they were higher than she supposed they ought to
be, but she did not see how she could economise with a man in the house. If it
were only herself and Polly they could do on less; but Luke, though he might
not notice that he was having less would soon reap the consequence and feel
limp.
Rachel leant her arms on the bureau and her chin on the palms of her hands
and gazed out Of the window. How could they manage to get away for a week
if no longer?
Unfortunately Heatherland was impossible.

Her mother found herself so impoverished that she was selling her house and
was on the point of going into a much smaller one in the village. Rachel had
felt very sad when Sybil had written to tell her the news. That her mother, at
her age, should have to uproot again would be a real trial; evidently her family
were in financial difficulties too.
Rachel began to think over her belongings and wondered if she could not sell
some of her wedding presents. There was the pearl necklace that an uncle
who was dead had given her. It was of little use to her now, and in her present
mood she felt that a breath of sea air would compensate for the loss of any
number of pearls.
Yes, she would certainly sell her pearls. She wondered if the day would come
when she would be reduced to selling many of her possessions. It looked like
it. It was a terrible shame that livings should be so small that the very
necessities of life should have to be done without. Well anyhow she would sell
her pearls and not tell Luke till it was done. She would get a cousin of hers to
do the transaction for her. She knew she might be cheated and it would be no
good for Luke to try and sell them. He was no business man and would
without doubt be contented with half their value. No, she would write to her
cousin. They simply must go away somewhere this summer.
She wrote the letter to her cousin and got Polly to run round to the post with it.
Then she began to wonder if she ought not to do what Luke had suggested;
ask the advice of Mrs. Greville. But she was saved the trouble, for late in the
morning her mother-in-law came round to see her.
"Luke tells me you are rather worried about the expenses," she said, "and I
am wondering if I can help you. Shall I look through your books and see what
you could do without?" and as she saw Rachel flush she added, "It is not at all
surprising my dear. Of course you have never been used to economise. I
hope you don't me an interfering old woman," she added kindly, as she saw
signs of distress on Rachel's face.
"It's very good of you," said Rachel; but she bit her lip feeling humiliated in the
extreme.
Mrs. Greville was not long in discovering things which would have to be done
without. For instance, she explained to Rachel she could make quite nice
puddings without eggs. Considering the expense of eggs, 4d each, it was
ruinous to follow the cookery books which prescribed more than one in quite
simple puddings. A great deal of money had been thrown away on
unnecessary eggs and they mounted up at once. Then it was much better in
these days to have margarine rather than butter. She never used anything but
margarine herself, and really you would not know the difference.
Rachel sat by her side smiling. Not being a housekeeper all these economies
seemed so paltry to her, and yet she knew they were necessary. She had of
her own will married a poor clergyman, and must bear the consequences. And
Mrs. Greville was being very kind; and giving as little pain as possible. Her
feelings towards Rachel had somewhat changed since that Sunday on which
she had gathered the girls' choir together. It had struck her mother-in-law as a
sporting action on her part and had pleased her. And now, for the sake of her
son, she was very anxious not to hurt his wife's sensitive spirit more than
necessary, but she had no idea how galling the whole thing was to her pride.
And Rachel was bent on her not guessing it. So she sat by her side smiling,
and watching her mother-in-law making notes for her as to the things that
were really necessary to have and those which were mere luxuries.
"I fear you must give up all luxuries, I hope Luke told you how poor he is,
when he asked you to share his poverty with him?" she said laughing. Rachel
joined in the laugh.
"We had much more interesting things to talk about," she answered. "Luke's
mind does not run on such matters as eggs and margarine."
"That's true," said Mrs. Greville. "The fact is, that some of the poorest people
are the clergy. It ought not to be so. They should not have to worry about eggs
and margarine as you say, they have so much more important things to think
of and they should be spared that. Besides they are expected to help in every
bit of work that goes on in the parish. Unless they have property of their own
the worry of pounds, shillings, and pence, weighs them down. Happily, as you
say, Luke does not worry himself about those kind of things, but then he has a
wife and mother to worry for him. If he had not he would have less time and
strength to think of his people. I don't suppose many realise how the clergy
suffer from poverty, for they suffer in silence."
"Well I hope Luke won't ever be reduced to wearing a coat green from age, as
a poor man in our part of the world at home has to do. Not that I suppose
Luke would notice if his coat were all colours of the rainbow."
Mrs. Greville laughed, and said goodbye while at the door she turned back to
say:
"Remember, not so many eggs, and margarine instead of butter. You'll find
that makes a difference very soon."
CHAPTER XV.
RACHEL'S PEARLS.
Rachel took out her pearls from the jewel box and looked at them. They were
certainly very beautiful. She had not worn them since her wedding day; and
she did not see any chance of wearing them again.
In case the parting with them should grow a little hard she packed them up
quickly and went to the post to register them.
It seemed to her as if she was parting with another link of the old life. But after
all what did that matter! She had Luke; and it was true what Gwen had said,
that Luke compensated for the loss of all else. Besides which, she knew that
they both needed a change and rest, and certainly sea breezes were of more
value just then than pearls locked up in her jewel case could be. Nevertheless
it cost her something to part with their beauty. It was not so much their value
that she had thought of as their beauty; and more than once she had taken
them out simply to have the pleasure of seeing something very lovely. There
was so little beauty surrounding her that she revelled in the sight of her pearls.
RACHEL TOOK OUT HER PEARLS—AND LOOKED AT THEM.
It was some time before the cousin, to whom she had sent them, wrote and
told her that he believed they would fetch a very good price; and one morning
at breakfast she opened a letter that was lying on the table beside her, and a
cheque, much larger than she had hoped for, fell out.
Her exclamation of surprise and pleasure caused Luke to look up from the
paper he was reading.
"What is it?" he asked.
"Something delightful," she answered. "We can now go for our holiday."
"What do you mean? Have you come in for a fortune?"

"I feel as if I had in our present strait," said Rachel still looking down at the
cheque in her hand. "Would you like to see what I have?" She held it out to
him.
Money meant little to Luke, except that it enabled him to carry out plans on
which he had set his heart. And there was a plan of his that sprang at once
into his mind as he looked at the cheque. His face glowed.
"Where did it come from?" he asked, astonished and delighted.
"I sold my pearls."
The smile on his face faded for a moment.
"I did not know you had any, I have never seen them."
"I have never shown them to you as I know that that kind of thing is not in your
line, you don't care for jewels; but I wore them on my wedding day. I hope you
saw them then."
"No, I only saw you. But why didn't you tell me what you were doing?"
"Because I was afraid you would not let me part with them. Uncle Joe gave
them to me, and I was fond of him."
"I shouldn't have prevented you from parting with them. What is the good of
pearls?"
"The good! Why their beauty is their good. They are gifts from God just as
everything else is that is lovely, and of good report. Don't despise them.
Besides," she said, feeling a little sore, "Uncle Joe gave them to me, and I
loved him." Then she added, determined not to give way to any feeling of
disappointment, "and now we can think about our holiday. Where shall we
go?"
She was folding the cheque up and putting it again in the envelope of her
letter. But on noticing that Luke did not answer her question, she glanced up
and found him looking out of the window with a dreamy happy smile on his
face. He was evidently thinking of the holiday. Perhaps his thoughts had flown
to Southwold and the moon's silver pathway on the sea. The happiness
displayed in his expression of face made her feel that the small self-denial that
she had exercised was well worth while.
"I do believe," he said, still looking out of the window, "that at last my dream
will be fulfilled."
"What is your dream?" asked Rachel. She had been right. He was evidently
dreaming of Southwold.
"Why, to put electric light in St. Marks. Think how attractive and bright it would
make the place. I never thought I should be able to do it. How much do you
think it would cost?"
Rachel was silent from astonishment and disappointment.
Then she said slowly:
"You don't suppose, do you Luke, that I have sold my pearls to be able to do
for the Church what the people are far better able to afford to do than we are?
You seem to forget that we are really poor, much poorer than many of the
congregation. No, the first thing we must do," she said decidedly, "is to pay
our bills and to start afresh, and then to go for a holiday."
Rachel's tone of voice was so decided that he turned and looked at her in
surprise.
"But," he said, "it is God's work that you are refusing."
"No," she answered, "I don't think it is. To provide the luxury of electric light in
a place where the gas is quite good and sufficient, seems to me to be not so
much doing the Will of God as paying our debts, and going for a holiday,
which will give you strength to do His work better."
Luke was silent.
"Besides," she added in a lower voice and smiling, "you forget next January."
Luke looked mystified. Then noticing the expression of his wife's eyes, he
remembered.
"I forgot for the moment," he said. "You are quite right."
At the thought of January Rachel's face had lost all the surprise and
disappointment as she looked down again at the envelope containing the
cheque. Then she suddenly drew it out with a gay laugh and flourished it in his
face.
"Happily it's mine," she said, "and you can't get at it."
CHAPTER XVI.
THE CHURCH COUNCIL.
"I know you won't give way," said Rachel, as she helped her husband into his
coat.
Luke did not answer at once.
"If you remain firm this time you will find it easier the next."
"I don't intend to give way," said Luke gravely. "It would be fatal."
"Yes, and you would never forgive yourself." Luke buttoned up his coat and
looked at his wife.
"We should hear the same excuse that that old woman gave for not attending
her Parish Church," he said smiling.
"What was that?"
"That she had been too ill for the last half year to come to the services but she
was proud to say she had not missed a single Whist Drive."
"Oh you mustn't give way," repeated Rachel earnestly. Then she added, laying
a hand on his arm, "You know what I shall be doing."
When Luke smiled his face was transfigured.

He smiled now.
"You will be doing the part of Moses," he said. "I am not sure that I should not
fail if it were not for that." Then he opened the door and was gone.
Rachel stood where he had left her looking absently at the door which he had
shut after him.
That the Church Councils were not always happy meetings she knew. Luke
said the right people were not on it. There were some really in earnest, but
these often seemed afraid of speaking. Those who spoke the most often and
the loudest were those who wanted to make their Church the most popular in
the town by way of parish dances, whist drives, etc. Luke, Rachel knew, had
always stood out against such methods of work, but people were growing
persistent, and the subject was to come up again this evening. His last words
had surprised her and made her anxious, for they showed her that his
resistance was growing weaker and that he felt himself in real danger of giving
way.
She knew what giving way would mean to him. It would lie on his heart like
lead. He would not look for blessing so expectantly and hopefully in his parish
if once his church began to cater for the amusement of his people instead of
putting its full strength into the spiritual work. By this time Rachel knew her
husband so well, that she felt sure that he would grow melancholy and
depressed, and his work would be robbed of zeal and happiness in
consequence.
It was not as if he had any doubts as to the wrongfulness of such methods for
Church work. He absolutely disapproved of them and had made his opinions
known. If he gave way or countenanced such proceedings, in the least,
people would cease to believe in him.
Well there was one thing she could do to help him to be strong; she would go
and do it. And while Luke was wrestling with his Church Council, Rachel was
wrestling in prayer.
Then she went down stairs to listen for his footstep.
When she heard it, it did not inspire her with hope. Luke came in quietly and
made his way slowly into the drawing-room where he knew he would find her.
"Well?" she said.

He took a seat on the sofa near her.
"I have gained my point."
Rachel's eyes shone.
"What good news," she said greatly relieved. "Why do you look so
melancholy?"
"Because though I have gained my point I have lost four of the most regular
members of the congregation. They walked out of the room."
"Oh well, that is not half so bad as if they had gained the victory," said Rachel
cheerfully. Then Luke looked lovingly down at his wife.
"I doubt if I should have taken such a strong attitude if it had not been for you.
I knew you were praying."
"Yes, I was praying."
"The knowledge of that helped me enormously. The four who resigned have
been my most loyal supporters and I can tell you it was hard to stand out
against them. They have been so exceedingly kind to me ever since I have
had the church. It was this fact that made it so difficult. Besides I love peace."
"Peace with honour, but not without."
"That's just it. I felt that my Master's honour was at stake."
"I can't tell you how thankful I am," said Rachel. And under his wife's influence
Luke regained something of his usual spirits. But Rachel had only heard
Luke's side of the question. The next day she was inundated with callers.
"I suppose you know," said Mrs. Moscombe, the wife of the owner of the
principal shop in the parish, "that the Vicar got his way by one vote only. I own
when I heard all the arguments in favour of opening the Hall for such
purposes I began to wonder if we ought not to do what the neighbouring
churches have done to attract the young people."
"I don't think that kind of thing attracts people to Church," said Rachel.
"But as my husband says," continued Mrs. Moscombe, "it saves the boys and
girls from going to worse places. Surely that is the work of the Church."
"It may save them for two or three days, possibly; but it really leads them to go
to unwise places of amusement in the long run, and I know my husband feels
very strongly that the Church loses its spiritual power if it goes in for catering
for amusements."
"But then, dear Mrs. Greville, your husband, forgive me for saying so, is rather
peculiar in his views. He scarcely moves with the times and isn't up to date as
they say."
Rachel flushed.
"The times are not so particularly good that one should wish to move with
them," she said. "I am very thankful that my husband does what he considers
right without swerving or moving with the times."
Rachel was glad to see the last of her visitor, but had hardly said goodbye
when the door opened to admit Mrs. Stone.
"Of course," she said as she took a seat, "your husband has told you all about
last night. I admired him immensely. He didn't give way an inch though the
majority were really against him."
"But anyhow he had a majority of one."
Mrs. Stone laughed.
"And he would not have had that if it had not been for me. I didn't agree with
him in the least! I must tell you, but I voted for his views as I always feel he is
such a good man that he probably knows what is for the good of his people
better than I do. If it had not been for my vote, he would himself have had to
give the casting vote."
"I am very disappointed that you don't agree with us," said Rachel.
"Well I do believe in people keeping up with the times, and girls and boys are
crazy now for dancing and cards. You can't get them if you don't give way.
Things have changed so much since our fathers' times."
Rachel was silent. She felt depressed. She quite expected people like Mrs.
Moscombe and others who had called to see her, and who did not profess to
be religious, to misunderstand Luke's action, but it was a blow to find that her
friend Mrs. Stone also disagreed with him.
But the last caller was the most trying of all.

Rachel heard her mother-in-law open the front door and walk heavily across
the little hall.
"What has Luke been doing?" she exclaimed almost before she was in the
room. "I hear he has quite estranged the four best supporters of the Church."
She looked at Rachel as if she were to blame.
"It was about the amusement question," said Rachel. "He put down his foot at
the proposal to introduce them into the work of the parish."
"Well I call it remarkably silly of him. It is a matter of very little importance and
certainly not worth wrangling over. I am quite thankful I am not a member of
the Council. I could not have voted against my son, but I should have felt very
vexed at being a party to such a loss to the Church."
"You mean?"
"I mean losing his four best financial supporters, and those who give the most
to the Easter offerings. Who is he to look to now, I should like to know? And if
he only waited to consider the state of his own finances and the expense of
food, (eggs are still fourpence a piece), he would not have made such a fatal
mistake."
Rachel was silent, but she disagreed with every word her mother-in-law had
spoken. Then after a pause during which Mrs. Greville tied and untied her
bonnet strings in her agitation, she said:
"I don't suppose any consideration respecting finance would weigh with Luke
against doing what he thinks right."
"My dear, young men often make a fatal mistake in going their own way,
thinking that youth must know better than age. Think of those four gray haired
men who know more of the world than Luke, being set at nought like that. I
have never known Luke to make such a mistake. If he had only consulted me
before he had acted."
Then Rachel spoke.
"But don't you see how noble it was of him to keep to what he felt right even
though he must have known what the result would be. I own am proud of him,
and should have been bitterly disappointed if he had given way. I am sure he
did the right thing."
Mrs. Greville looked at her son's wife and could not but admire the way she
stood up for her son's folly, (as she considered it). There was an expression
on her face that any mother-in-law would have been pleased to see on the
face of her son's wife. But for all that she felt it incumbent on her to give her a
snub.
"I daresay," she said, "that you admire him. So would most young girls who
only look for actions without weighing their cost. We all admire a man who is
not afraid to speak out. But when it comes to flouting those who have been
kind and considerate, and who never hesitate to give money for the work, it is
a different matter. Luke has done a bad thing for the parish by his action of
last night."
"People would never have believed in his convictions again if he had given
way," said Rachel.
"Well now, don't you go and encourage him in that kind of thing," said Mrs.
Greville. "I hope that you recognise the fact that Luke is not a paragon of
wisdom, neither can any one turn him from what he imagines his duty. But he
must remember that he now has a wife to support. He not only will stubbornly
stick to his point even when it means losing money for the work of the parish,
but will give away every penny he possesses without a thought of the
consequences. I daresay you have found that out."
Rachel laughed, thinking of the pearls.
"Well, am I not right?"
"Luke is the most generously minded man that I have ever come across," said
Rachel.
And then Mrs. Greville gave her a kiss. She could not resist it; though she
knew that her action would startle her daughter-in-law.
"My dear," she said, "I do believe that you love that boy of mine as much as I
do."
Rachel was tempted to answer "a great deal more," but forbore, only returning
the kiss with warmth. She was getting almost fond of Mrs. Greville.
"Well, you see," she said with a smile, "he is my husband."

"But that does not always follow I am sorry to say. Wives are generally very
quick in seeing and resenting faults in their husbands. And much as I love my
dear boy I see a great many in him."
"But there are more virtues after all," was Rachel's answer, "and to return to
the subject of whist drives, it is perfectly true what Luke quoted to me from
some speech yesterday. 'The Church has so little power with the world
because the world has so much power over the Church.' Don't you agree?"
"Well perhaps it is so. But when a man's bread and butter is concerned and
when the Church funds are low, I own I feel it is not the time to be too
particular."
"I am afraid I don't agree with you a bit."
"I don't suppose you do; that is because Luke has imbued you with his ideas
of right and wrong."
"Luke has a very high ideal," said Rachel, "and I am trying to live up to it."
And Mrs. Greville went away thinking to herself, "I only hope that Luke
realises what a devoted wife he has. I don't believe he does."
CHAPTER XVII.
LUKE IS DISCOURAGED.
The four members who had left the Church Council when the vote went
against Whist Drives for Church purposes, did not leave the Church. They
valued their Vicar too much to do so suddenly; but they were thoroughly vexed
at the decision arrived at.
It had been a blow to Luke to find that he only had a majority of one. He had
hoped that his congregation had felt with him in the matter, and finding how
strong the stream was towards such means of increasing the popularity of the
Church, depressed him not a little.
Moreover, he felt bound, greatly against his will, to preach against such
methods and to give his reasons for so doing; and though some respected
him for his courage, there were others who resented it. To Rachel, the Sunday
on which he mentioned the matter was a most painful day; though she was
glad that Luke had spoken out on what was much on his mind.
"I am beginning to think," he said, on sitting down to dinner after the service,
"that my time in the parish is about over. It seems to me my influence for good
is not strong enough. It wants a stronger man than I am here."
"You are tired," said Rachel. "That's what is making you downhearted. Last
week was such a very heavy one for you. In a day or two you will see things
differently."
Luke smiled unbelievingly.
"What I should like, and in fact what I have always longed for," he said, "is a
Church in London. Though I doubt if such an honour will ever come my way. I
am not a big enough man to be trusted with a London parish."
"London!" cried Rachel. "Oh Luke, I should hate it of all things. Besides you
must be a man strong in body as well as in soul to work a London parish
satisfactorily. I should be very sorry to see you undertake such a work."
"I'm quite strong enough," said Luke. "The only things that try me are the petty
quarrels and vexations of such a parish as this. I heard this morning that Went
and Ethers have fallen out, and on a ridiculously small matter. I fancy
everything would be larger and more important in London. It is just the petty
matters that worry me."
"Human nature is the same everywhere. I expect you would find small souls in
a London parish just as you do here."
"Would you very much object to London?" asked Luke. "Not that there is the
slightest chance of me being offered a Church there. But it is the dream of my
life. Fancy working in the very hub of the Universe. I should revel in it."
"The work would be enormous, unless you had several curates. And you know
how difficult they are to find now-a-days."
"I shouldn't mind the work. The more the better, so long as it is not spoilt by
bickerings and quarrellings. Should you very much dislike it?"
"Intensely. I don't feel in my present mood, as if I could endure it." Then seeing
a look of disappointment on her husband's face, she added, "But where thou
goest I will go, you know that."
"Yes, I have no doubt of that," he answered.
And his longing for London increased during the next few months. It was a
time of great disappointment for him. When he had first come to Trowsby, he
had had the warmest of welcomes, and the largest congregation in the place.
His preaching was arresting and people congratulated themselves on having
such a Vicar. He had come straight from France where he had been acting as
Chaplain, and had there shown great bravery under fire. Many came to hear
him just because of this. But when the rage for amusements began to show
itself, and it was found that the Vicar had no sympathy with it, and had no new
Gospel to preach, but preached the same Gospel as they had heard before
the war, untouched with modernism and the various other new religious
theories, the congregation that had increased out of curiosity gradually
dwindled, for they said, "He's not up to date." It was disheartening for Luke,
specially as he heard that a Church not very far off was crowded to
overflowing on account of all the social questions that were discussed during
the sermons, and well-known lecturers on the various religions came down
from London, Sunday after Sunday, to preach.
"Nevertheless," he said one day to his wife, "I shall continue to preach the
Gospel; and by-the-bye Rachel, I must somehow get three days of quiet at
least, for some of the men's Bible Class want to discuss those questions
which have been raised by the Modern Churchmen's Conference; and I must
prepare for the discussion. But I really don't see how I can manage it. I am
late as it is with the Parish Magazine."
Rachel was laying the table for dinner at the time and looked up quickly at her
husband.
"Pass that over to me," she said.
Luke looked at her a little doubtfully.
"Do you think you can manage it?" he asked.
Rachel laughed.
"Certainly I could. I am a little more intelligent than you give me credit for.
Have all the people sent in their accounts?"
"No, that's just it. Sargent has never sent in his description of the Temperance
Meeting, nor has Mrs. Lent of the Scripture Union Meeting. They are so often
late. It means a good long walk as there is no time to send them cards to
remind them of their duty. The manuscript ought to go in early to-morrow
morning to the printer or the magazine will not be out in time. I should be
thankful to have at least two quiet uninterrupted days; but then there are sick
people to visit. I don't see how I can."
"You can quite well if you will only trust me," said Rachel, smoothing the table
cloth. "You have never tried me."
"I wonder what my mother would say."
"What does it matter? I don't belong to your mother I belong to you. You must
take the responsibility of me," she added laughing.
And so it was settled and Luke had two whole days of quiet. He did not move
out of his study except for meals, and then he hurried over them and ate them
without speaking. Rachel, knowing what he was going through and in what
dead earnest he was, in his longing to rid his men of the terrible doubts that
had been sown in their hearts, kept silence. It might have been a quiet day
arranged by the Bishop!
Luke had given her a long list of people to visit, and had told her what
information was needed for the magazine, and Rachel set to work asking no
further questions. She dispensed money where she thought it was needed
(not always wisely alas!) and tended to the best of her knowledge the sick
people, singing to many of them; and though it was work just after her own
heart, being utterly unused to it having had no training whatever, it took a
great deal out of her, particularly as she was of a sympathetic nature. But she
felt it was well worth while when, after the two days were over, the strained
tired look on Luke's face had disappeared giving way to one full of peace and
happiness.
He had felt it his duty to face over again, all the arguments and difficulties that
his people might come across; and he realised that he was at war with the
Devil. The words of St. Paul often ran in his mind; "For we wrestle not against
flesh and blood, but against principalities, against powers, against the rulers of
darkness of this world, against spiritual wickedness in high places."
Not even Rachel was conscious of the spiritual warfare that was raging in her
husband's little study, nor how often he threw himself on his knees crying to
God for help in the conflict, nor had Rachel the faintest suspicion of the
victories that were gained within its walls.
She had often wished that her husband was not so preoccupied, and had
more thought for the small things of life which make all the difference to its
comfort. But though she felt that the things of which she complained in her
heart were so insignificant compared to the great matters about which Luke
was engaged, she had no idea that his preoccupation and absentness of mind
were often caused by the fact that he had either lost or won a spiritual battle.
He was thankful now that he had spent so much time in facing the doubts and
difficulties that he had met with in the course of his reading, so that the two
quiet days enabled him to prepare the subject in such a way as to make the
truth plain to his men.
On the night of the meeting Rachel found it rather hard to occupy her thoughts
with anything but the great strain which she knew Luke was passing through.
He was late home and to turn her mind from that which was making her
anxious she sat down to the piano and sang.
She was just finishing her song when she heard the front door open. She sat
still in her suspense, expecting Luke to come at once into the drawing-room.
But instead, she heard him going upstairs to his study, and walking heavily as
if he was tired.
For some moments she sat still where she was, then she followed him. But at
the study door she stopped.
Was that Luke groaning? Was he ill?
She very softly opened the door and looked in.

Luke was on his knees, his arms on the writing table and his face buried in
them. He was praying out loud.
Rachel closed the door and went downstairs again. Her heart was heavy, and
anxious. She knew that his habit was to pray out loud; but his prayer to-night
was mingled with groans and probably tears. What had happened? Rachel
moved restlessly about the room. Her impulse was to go to the piano, and
soothe her anxiety by playing. But she was afraid of disturbing Luke. Then she
took up her work and sat waiting.
It was late before she heard him coming down the stairs.
His face bore no trace of the anguish he had apparently been going through.
He came and sat down by Rachel's side without speaking.
"Well?" she said.
He was silent for a moment, then he said:
"I have never had such a fight with evil as to-night. I feel sure that the Devil is
working with all his might to destroy any good that may have been done."
"What happened?"
"We have been discussing for nearly three hours the articles of our faith. The
men had primed themselves with all the arguments they could lay hold of
against them. The Divinity of our Lord, the Virgin Birth, the Resurrection; and
the very men who I had hoped were on the eve of making the great decision
have been thrown back."
"But there were surely some who were helped by your words?"
"Yes, thank God. There is a small band of faithful Christians as firm in their
faith as I am. They know the Christ; and believing Him to be God take His
Word as truth. But the greatest number have been shaken by these views that
have been scattered broadcast since the Modern Churchmen's Conference at
Cambridge, and there are some who are weak in the faith and just tottering,
as it were." Luke rose up and began to walk about the room.
"I feel," he said, "that a stronger man than I am is wanted for this place. It
seems to me to be the stronghold of Satan."
"When I am weak, then am I strong," said Rachel.

Luke stood and looked at Rachel for a moment. Then his face broke into a
smile.
"Thank God for my wife," he said.
CHAPTER XVIII.
GAS STOVES VERSUS MOUNTAINS.
"Next year," said Rachel one day, as she and Luke were on their way to
Church, "next year we may not be able to afford a holiday. So I am resolved
that we shall have a good one this August. We will go to the Lakes."
Rachel wrote and secured rooms at Rydal and a month after the men's
meeting mentioned in the last chapter, they started off for the North.
What the sight of the beauty that now surrounded them was to Rachel can be
imagined. She told Luke that she had seen nothing that could be called
beautiful ever since coming to Trowsby, with the exception, she took pains to
add, of her primroses and hyacinths in the little garden of which she was
inordinately proud.
To sit by the Lake in the cool of the evening and watch the lights and shadows
on the mountains, was positive bliss to Rachel. She tried to make Luke revel
in it as much as she did, but alas, his thoughts were still engrossed with his
parish, not withstanding all Rachel's efforts to make him forget it.
"It will be so much better for the parish as well as for you if you will only put it
away from your mind," she said.

Policy Program and Project Evaluation A Toolkit For Economic Analysis in A Changing World 1St Ed Edition Anwar Shah Full Chapter PDF

Uploaded by

Copyright:

Available Formats

Policy Program and Project Evaluation A Toolkit For Economic Analysis in A Changing World 1St Ed Edition Anwar Shah Full Chapter PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Policy Program and Project Evaluation A Toolkit For Economic Analysis in A Changing World 1St Ed Edition Anwar Shah Full Chapter PDF

Uploaded by

Copyright:

Available Formats

Policy, Program and Project Evaluation:

A Toolkit for Economic Analysis in a

Policy, Program and Project Evaluation: A Toolkit for

eTextbook 978-1305117204 Systems Analysis and Design in

Law and Economic Development: Behavioral and Moral

Evaluation Practice for Collaborative Growth: A Guide

Sociology in a Changing World 9th Edition, (Ebook PDF)

Scientific American Environmental Science for a

Taxing Choices for Managing Natural Resources, the

Abnormal Psychology in a Changing World 10th Edition,

Policy, Program and

Policy, Program and

ISBN 978-3-030-48566-5 ISBN 978-3-030-48567-2 (eBook)

In recent decades, evaluation as a discipline has progressed from a tool for

Washington, DC, USA Anwar Shah

2 A Primer on Public Sector Evaluations 11

3 Economic Evaluation of Projects 59

4 The Marginal Cost of Public Funds: Concept,

5 Theory-Based Evaluations: Guidelines for Case

6 A Framework for Evaluating Anti-Corruption Policies

7 Evaluating External Analytical Advice on Budgetary

8 Inter-Sectoral Allocation Choices221

9 Evaluation of Decentralization Programs283

10 Evaluating the Conditionality of External Development

Fig. 2.1 Results chain in ROME. (Source: Shah 2005) 39

Fig. 5.6 Causal assumptions in impact of trade liberalization on

Table 4.1 Distributional weights, distributional characteristics, and the

Table 7.8 Overall rating of PER quality and timeliness 214

Box 4.1 A Numerical Example Comparing the MCF in a Competitive

Robin Boadway is Emeritus Professor of Economics at Queen’s

of Alberta to review the province’s finances. His international experi-

(along with three coauthors) and the University of Alberta’s

Evaluation as a discipline, in recent decades, has progressed from provid-

© The Author(s) 2020 1

methods have been presented in a manner that reflects the changes in

concludes that path dependency is critical in determining the relative effi-

developing countries as an aid to public sector reforms. The most com-

compression when coupled with mechanisms that create accountability for

successful. Chapter 9 by McMillan outlines an approach in evaluating the

adverse consequences for project and program outcomes. The chapter

A Primer on Public Sector Evaluations

Saubhik Deb and Anwar Shah

© The Author(s) 2020 11

identifying program targets, designing interventions, identifying organi-

Leading Approaches to Evaluation

Cost-benefit analysis estimates the net present value of a program by

competitive, the market prices represent the opportunity costs of goods

a targeted population, the possible program alternatives can be provision

serious problem with this type of ex ante evaluation is the uncertainty

Social Marginal Cost of Funds Analysis

the other hand, taxing negative-externality-producing goods (such as pol-

Data Envelopment Analysis

Multiple-Criteria Evaluation (MCE)/Multiple-Objectives

program objectives. At the implementation stage, the inputs are used to

evaluation. For example, an evaluation study of a tobacco use prevention

The fruitfulness of an evaluation, be it a process evaluation or an out-

efficiency is a very useful indicator of target participation that can be used

goals of the program. An interim evaluation is essential for the perfor-

outcome evaluation crucially depends on the reliability and validity of the

Washington, DC, USA Anwar Shah

2 A Primer on Public Sector Evaluations 11

3 Economic Evaluation of Projects 59

4 The Marginal Cost of Public Funds: Concept,

5 Theory-Based Evaluations: Guidelines for Case

6 A Framework for Evaluating Anti-Corruption Policies

7 Evaluating External Analytical Advice on Budgetary

8 Inter-Sectoral Allocation Choices221

9 Evaluation of Decentralization Programs283

10 Evaluating the Conditionality of External Development

Fig. 2.1 Results chain in ROME. (Source: Shah 2005) 39

Table 7.8 Overall rating of PER quality and timeliness 214