KPI 15 Extent ICF Intervention Lead Transformational Change

Climate Change Compass helps the UK Government
monitor, evaluate and learn from UK International Climate Finance
Extent to which ICF intervention is likely to

lead to Transformational Change
KPI 15 Methodology Note
November 2018
Join the conversation at climatechangecompass.org 1

Acronyms ........................................................................................................................................ 3
Rationale ......................................................................................................................................... 4
Summary table ............................................................................................................................... 4
Technical Definition ....................................................................................................................... 5
Methodological Summary.............................................................................................................. 6
Methodology ................................................................................................................................... 7
Worked Example ......................................................................................................................... 10
Data Management ........................................................................................................................ 13
Data disaggregation ..................................................................................................................... 14
Annex 1: Why report on KPI 15 ................................................................................................. 14
Annex 2: Examples of indicators against each of the TC criteria ............................................ 15
Annex 3: Comparability and synergies with other external indicators ................................... 17
Annex 4: Definitions of key methodological terms used across Methodology Notes ............ 18
Annex 5: Optional KPI 15 Reporting Template ........................................................................ 18
*Click on page numbers above to navigate to specific sections
About Climate Change Compass

The UK government has committed to provide at least £5.8 billion of International Climate Finance between 2016 and
2020 to help developing countries respond to the challenges and opportunities of climate change.
Visit www.gov.uk/guidance/international-climate-finance to learn more about UK International Climate Finance, its

results and read case studies. Visit www.climatechangecompass.org to learn more about how Climate Change
Compass is supporting the UK Government to monitor, evaluate, and learn from the UK International Climate Finance
portfolio.

Acronyms
BAU Business as Usual

CIF Clean Investment Fund
EERESH Energy Efficiency and Renewable Energy for SMEs and Households
FiT Feed-In Tariff
GCF Green Climate Fund
GHG Greenhouse Gas
HMG Her Majesty’s Government
ICF International Climate Finance
KPI Key Performance Indicator
M&E Monitoring and Evaluation
MEL Monitoring, Evaluation & Learning
MSMEs Micro, Small and Medium Enterprises
NA Not Applicable
SMEs Small and Medium Sized Enterprises
TA Technical Assistance
TC Transformational Change
ToC Theory of Change
UN United Nations
UK United Kingdom

Extent to which ICF intervention is likely to lead to

Transformational Change
Rationale
International Climate Finance (ICF) resources dedicated to climate change actions only form a very small
part of the financing required to help developing countries build resilience to climate impacts and shift to
lower carbon patterns of development.
The ICF will have greater impact if it can be ‘transformational’ by, for example, encouraging others to
replicate and scale-up successful activities and facilitating substantive institutional and policy change
toward a low carbon and climate resilient future. A challenge for this indicator is to capture these
different, often country-specific, dimensions of Transformational Change (TC), while remaining
sufficiently simple, to be unambiguous and pragmatic.
This indicator recognises that “transformation” is multi-dimensional, and that the indicator will not be
able to capture everything that, in time, may contribute to TC. Rather, the objective is to capture enough
evidence to form a reasonable qualitative picture of ICF effectiveness in this area.
Further information and a broader explanation of why one should report on KPI 15 is in Annex 1.
Summary table
Table 1: KPI 15 summary table
Units Box marking (i.e. 0, 1, 2, 3 or 4)
Disaggregation NA
summary
Headline data The self-assessment box marking (for each relevant criterion and an overall
to be reported marking) with explanatory text presenting evidence of transformation against
relevant criteria, both to justify the assessment and evaluate the reliability of the
evidence. See Annex 5 for an optional KPI 15 reporting template.
Latest revision September 2018.
The main revisions to this Methodology Note are:

• Step-by-step methodological guidance
• Narrative of why to report on KPI 15 (Annex 1)
• Inclusion of an optional KPI 15 reporting template (Annex 5)
Timing issues When to report: ICF programmes will be required to report ICF results once each
year in March. Please bear in mind how much time is needed to collect data
required to report ICF results and plan accordingly.
Reporting lags: Your programme may have produced results estimates earlier in
the year, for example during your programme’s Annual Review. It is acceptable to
provide these results as long as they were produced in the 12 months preceding
the March results commission. In some cases, data required for producing results
estimates will be available after the results were achieved – if because of this,
results estimates are only available more than a year away from when results are
delivered, this should be noted in the results return.

Links across KPI 15 supplements other IFC KPIs, which report on more tangible results (e.g.
KPI portfolio avoided GHG emissions or hectares of deforestation avoided) and gives an
indication of the likelihood that programme results will continue beyond the
funding period of the ICF programmes. For example, it could be deduced that
emissions reductions will continue beyond the end date of a GHG emissions
reduction programme, if the programme is deemed to be highly likely to be
transformational. This could be the result of a policy change or encouraging
others to replicate and scale up successful activities because of the ICF
programme.
Technical Definition
Transformational Change is ‘change which catalyses further changes’, enabling either, a shift from one
state to another (e.g. from conventional to lower carbon or more climate-resilient patterns of
development), or faster change (e.g. speeding up progress on cutting the rate of deforestation). However,
it can entail a range of simultaneous transformations to political power, social relations, decision-making
processes, equitable markets and technology.
Many of the transformations the ICF is seeking to bring about will only be evident after a period of time,
and most are unlikely to materialise within the period of ICF support. This indicator therefore tracks
early signs of transformation, or the extent to which ICF activities are being, or have a good likelihood of
being, transformational. It does so by using proxies for drivers of transformation, to assess the extent to
which ICF support can be linked, if not attributed, to likely Transformational Change.
These proxies (henceforth called the ‘criteria’, as set out in the ‘Methodology’ section) are based on the
Theory of Change (ToC) for Transformational Change (see ‘Rationale’ section).
The ICF is likely to be more transformational in developing countries if several of the following criteria
prevail, and if at least one criterion exists for each level of the Theory of Change (see Figure 1 below for
details):
• Political will and local ownership: Where the need for change is agreed locally, and the
process is locally owned. Where high-level political buy-in and broad support from across
societies, cultures, and interest groups enable widespread changes to patterns of development;
• Capacity and capability can be increased: Where a target country and target communities
have the capacities and capabilities necessary to bring about the change;
• Innovation: Where wider and sustained change comes from innovative new technologies with
the potential to demonstrate new ways of doing things;
• Evidence of effectiveness is shared: Where approaches which have proven successful in one
location are disseminated widely, and lessons on their usefulness are credible;
• Leverage / create incentives for others to act: Where the costs of climate action are
reduced to the point that acting on climate change risks and challenges is a sensible decision for
public agencies, commercial firms, and private individuals. These cost reductions may need to be
steep enough to overcome behavioural inertia;
• Replicable: Where good ideas piloted by the ICF are replicated by others in the same country,
and more widely;
• At scale: Where interventions (such as national, sectoral or regional programmes) have
sufficient reach to achieve progressive institutional and policy reform, or drive down the costs of
technology deployment;
• Sustainable: Where activities are likely to be sustained once ICF support ends.

Ultimately, many truly transformational changes will require a critical mass, to overcome political, market
and other sources of inertia. Many of the points above relate to achieving this critical mass and the more
of the above an intervention can promote, the greater the likelihood that it will lead to transformational
change.
The Theory of Change for Transformational Change below groups criteria at three different levels
(drivers, mechanism and enablers).
Figure 1: Theory of Change for Transformational Change:
Further information and a broader explanation of why one should report on Key Performance Indicator
(KPI) 15 is in Annex 1.
Methodological Summary
This KPI is a qualitative process indicator. The expectation is that it will normally be assessed at the
level of a significant ICF programme, or a country/thematic portfolio, rather than for individual projects.
This KPI will be assessed at programme or portfolio level as follows:
Expected results
A qualitative assessment of the type and nature of expected Transformational Change should be provided
at the start of the programme (or portfolio of programmes). This assessment should be guided by the
criteria included in the ‘Methodology’ section. It is not necessary to provide a box marking for the
expected result at this stage, the assumption being that this would be ‘4 – transformation judged very
likely’, since all ICF programmes should be designed to be transformational. A baseline assessment should
also be conducted, based on qualitative judgement assessment.
Actual results
ICF programme/portfolio managers should annually report:
• An overall assessment score of the likelihood that transformation linked to the ICF support
(programme, country, region or sector portfolio) as follows:
o 0: Transformational Change judged unlikely
o 1: No evidence yet available – too early to assess, but Transformational Change
expected
o 2: Some early evidence suggests Transformational Change judged likely

o 3: Tentative evidence of change – Transformational Change judged likely

o 4: Clear evidence of change – Transformational Change judged very likely
• A qualitative/narrative report against the relevant criteria of Transformational Change (see

‘Methodology’ section below), with supporting evidence of transformation in those criteria, using
programme (or portfolio)-specific sub-indicators.
Transformational Change evidence will not be aggregated at the overall ICF level in the same way as
other ICF KPIs. Scores might be synthesised to identify patterns and trends to assess overall progress
and to tease out lessons, rather than to form a view on the ICF’s expected future global transformational
impact. This KPI therefore adopts a qualitative approach to monitoring (not measuring) likelihood of
transformation, relative to expected change.
A summary of key methodological steps, which are expanded in the methodology section further below,
is demonstrated in the following diagrams:
Figure 2: Expected results
Figure 3: Actual results:
Where there is more than one ICF project programme, in a country, region or sector portfolio, the
assessment should be scored at the aggregate level, to reflect expected synergies (and reduce the risk of
double-counting).
Methodology
Expected Results
At the start of the programme or project, define:

• what successful transformation looks like for the programme/portfolio (including its Theory of
Change);
• the key stakeholders involved;
• which of the TC criteria are relevant to report against; and
• the programme/portfolio-specific indicators.
1. Describe the interventions that comprise the programme or country / thematic

portfolio
This step should list and very briefly describe – at impact and outcome levels and noting £ values – the
projects or programmes comprising the portfolio. This may be wider than just ICF programmes and
include other influencing activities.
2. Determine the baseline that Transformational Change is being assessed from
This should not require extra analysis beyond the Strategic Cases of the main interventions but may need
amending if new projects are added to the portfolio, which address new issues. The baseline is based on a
qualitative judgement assessment.
3. Describe the Theory of Change that links the programme / portfolio activities and
the expected Transformation
Though this step will draw heavily on the ToC of the main interventions, it may require additional work
given that Transformational Change should sit above those ToC interventions. The project ToCs should
be nested within the overall Transformational Change ToC.
4. Determine who else is crucial for ensuring this Transformational Change
This step contextualises the UK support and allows a political economy analysis of the change to be
summarised. Other stakeholders could be considered: a) those whose engagement is a necessary pre-
condition for change; b) those who have been (or need to be) engaged during implementation; c) those
who are not essential, but whose engagement presents opportunities which can / have been made use of.
This may need amending as additional key players are identified during programme / portfolio
implementation.
5. Describe what successful Transformational Change looks like; when is it expected to

occur; and how will it be assessed
This step has two purposes: (i) to set out what eventual impact is expected, and when it may be realised
(drawing on impact statements of the portfolio interventions); (ii) to set out the criteria and indicators
used to assess the likelihood of TC, drawing on relevant indicators and KPIs from project / programme
logframes.
Actual Results
At each reporting round, provide a narrative and assessment (score) of progress towards transformation.
1. Decide on which Transformational Change proxy criteria is relevant
Transformational Change proxy criteria can include:

• Political will and local ownership
• Capacity and capability increased
• Innovation

• Evidence of effectiveness is shared

• Leverage/incentives for others to act is created
• Replicability
• Scalability
• Sustainability
• Critical Mass
At least one criterion should be included from each level of the Theory of Change for Transformational
Change (see Rationale section).
The categories are not intended to be of equal importance, and may not all be relevant in every case.
However, an absence of some (notably ‘political will’ and ‘capability and capacity’) are likely to be major
constraints on Transformational Change. ‘Replication’, though clearly important, is likely to be a later
stage indicator. In turn, ‘sustainability’ is likely to rely on changes to many of the other criteria to be a
truly Transformational Change.
2. Determine if the criteria are equally weighted or if different weightings apply
Weightings should be applied to the individual criteria depending on how many are present at each level
of the Theory of Change (i.e. apply a lower weighting if there are multiple criteria from the same level of
the Theory of Change). For example, a programme with four criteria relevant for KPI 15 might have one
criteria weighted at 30% from the TOC level: Enabler (e.g. Sustainable), one criteria weighted at 30%
from the TOC level: Mechanism (e.g. At Scale) and two criteria weighted at 20% from the TOC level:
Drivers (e.g. Innovation and Evidence of Effectiveness).
3. Assign metrics to each of the selected criteria
This should draw on the programmes/projects logframe and other relevant ICF KPIs if appropriate, for
each of the selected criteria. If not, then they should be formulated at the time a baseline is set for the
intervention’s expected Transformational Change.
Examples of useful metrics for each criterion can be found in the table in Annex 2 and in the “KPI 15
Discussion Paper on How Programme Managers Can Communicate Case Studies and Evidence of
Progress Towards Transformational Change”, which is a supplementary document to this KPI
methodology note.
Please be aware that though these two sources suggest the types of evidence that could be used to
assess each criterion, programme managers should treat these as a guide and think carefully about what
types of evidence are most relevant to their particular programme and local circumstances. This is
important given that the barriers to systemic change are often hierarchical or local, or specific to
particular sectors.
4. Collect data against each of the metrics
Using the programmes/projects’ logframes, and other relevant ICF KPIs if appropriate, collect data against
each of the metrics.
5. Based on the data collected, score each criteria individually
Examples can be found in the “KPI 15 Discussion Paper on How Programme Managers Can
Communicate Case Studies and Evidence of Progress Towards Transformational Change”, which is a
supplementary document to this indicator.

6. Fill in KPI 15 reporting template (*optional)
The reporting template for KPI 15 (found in Annex 5) might be helpful to guide reporting. This step is
optional, and alternate reporting approaches may be used.
7. Assign overall KPI 15 score
Assign an overall KPI 15 score, based on the weighted mean of the respective individual criteria.
Note that where there is more than one ICF project in a country, region or sector portfolio, the
assessment should be scored at the aggregate level, to reflect expected synergies (and reduce the risk of
double-counting):
Consideration of contribution / attribution

While it may be possible to attribute change in some of the TC criteria to ICF activities, it is expected
that in many cases it will only be possible to track contribution to a wider effort.
As far as possible, reporting should be at the level of a significant programme or country (or similar)
portfolio, to help ensure that the links between different activities are understood, and an assessment
made of the likelihood that a critical mass of support for change is emerging.
This indicator seeks to track the transformational impact of HMG climate change “activities”. Though the
bulk of these will involve bilateral funding through the ICF, it will be important to recognise the role of
wider influencing and policy support provided by HMG. The contributions of others to the likely
Transformational Change – notably national and decentralized governments, but also other domestic and
international donors and organisations – should also be recorded as part of expected and actual results.
The methodology acknowledges that some ICF activities may inadvertently have an adverse effect on
Transformational Change (pilots might go wrong and undermine the case/support for change;
interventions may build capacity in one area by denuding it in another, etc.). It will be important that the
evidence presented is balanced, and also that any such negative influences are reported on.
Worked Example
Worked example 1
Based on a fictitious programme that provides financing for energy efficiency and small-scale renewable
energy projects in developing countries.
1. Decide on which Transformational Change proxy criteria is relevant
Relevant proxy criteria included: evidence of effectiveness shared; capacity & capability increased;
replicability; scale; sustainability.
2. Determine if the criteria are equally scored, or if different scores apply
The selected TC criteria received equal scoring metrics.
3. Assign metrics to each of the selected criteria; 4. Collect criteria data; 5. Score each
criteria individually, based on data; 6. Fill in KPI 15 reporting template; and 7. Assign
overall KPI 15 score – these results of these steps are shown in the optional reporting
template below

The following metrics were assigned against each TC criteria based on field evidence, and data was
reported and scored as below. All scores were based on evidence against the metrics:
Table 2: Worked example using reporting template
“Energy Efficiency and Renewable Energy for SMEs and Households” (EERESH)
Programme/Project summary:
Implementing Years: 2012 - 2015

Donors: DFID
Reporting level: Programme
Total Programme/Portfolio/Fund Size: £100m
UK Contribution to Programme/Portfolio/Fund: £10m
KPI 15 score: 3 (Tentative evidence of change – Transformational Change judged likely)
Transformational Change: Effectiveness shared, replicable, scalable, sustainable
Based on a fictitious programme that provides financing for energy efficiency and small-scale renewable
energy projects in developing countries. The programme aims to increase the flow of finance to small
and medium enterprises (SMEs) and households by creating a new partnership with the private sector,
specifically investing into a fund to leverage greater amounts of private finance. The fund tackles the
common barrier of institutional finance not being readily available to SMEs and home-based micro-
enterprise for low carbon projects in developing economies.
Criteria relevant to KPI 15:
The following criteria are deemed relevant to KPI 15:
Evidence of effectiveness is shared [TOC Level: Drivers] 20%
Metrics:
Number of new website visitors deemed an appropriate measure of whether evidence of effectiveness
is shared. Sign-up / log-in system helps track whether relevant users have accessed the website.
Number of activities (e.g. workshops, key publications) delivered to disseminate programme
information is a standard measure of whether effectiveness is shared.
Metric Data
Number of new website visitors 2,000
Number of activities (e.g. workshops, key 20
publications) delivered to disseminate
programme information
Justification of score:
A score of 3 (Tentative evidence of change – Transformational Change judged likely) was awarded, as
based on higher than average number of visitors to a programme’s website and a slightly higher than
average number of activities to disseminate programme information for a programme of this size.
Capacity and capability increased [TOC Level: Drivers] 20%

Metrics:
Metrics relating to Technical Assistance (TA) selected as most relevant to capacity building.
Metric Data
Total number of TA programmes approved 100
since inception
Total amount of TA funding approved $3.82m
Total number of people trained through the Over 500 per participating country (mean
TA Fund average)
A score of 2 (Some early evidence suggests Transformational Change judged likely) was awarded, as a
good level of TA programmes were approved but the total funding for each TA programme was
relatively low and more people could be trained via the TA fund.
Replicable [TOC Level: Mechanism] 20%
Metrics:
The potential for replicability assessed on whether low carbon loans developed beyond the
programme and whether the programme led to increased institutional knowledge of low carbon
investments, which has significant potential to instigate replicability.
Metric Data
Have the Financial Institutions developed Yes
low carbon loans beyond the programme
Did the programme lead to increased Yes, to some extent
institutional knowledge of low carbon
investments
A score of 3 (Tentative evidence of change – Transformational Change judged likely) was awarded. A
maximum score of 4 would have been awarded if it was reported that the programme had
categorically led to increased institutional knowledge of low carbon investments.
At Scale [TOC Level: Mechanism] 20%
Metrics:
The potential for scalability assessed not only via total number of individual sub-loans disbursed but
also by geographic spread and total number of financial institutions receiving investment.
Metric Data
Total number of financial institutions 30
receiving investment (loans and direct
investments)
Total number of countries receiving 25
investment
Total number of individual sub-loans 20,000
disbursed

A score of 4 (Clear evidence of change) was awarded, as many developing countries and financial
institutions were reached.
Sustainable [TOC Level: Enablers] 20%
Metrics:
Total number of financial institutions expected to extend their programme to new MSME recipients
was deemed the most appropriate proxy of EERESH’s sustainability.
Metric Data
Total number of financial institutions 79%
expected to extend their programme to
new MSME recipients
A score of 4 (Clear evidence of change – Transformational Change judged very likely) was awarded, as
over three quarters of the financial institutions expected to extend their programme to new MSME
recipients.
Overall score
Overall EERESH scored a weighted mean score of 3 based on equal weightings of the
above criteria.
Real-life examples can be found in the “KPI 15 Discussion Paper on How Programme Managers Can
Communicate Case Studies and Evidence of Progress Towards Transformational Change”, which is an
internal HMG document, and supplements this indicator.
Data Management
Data Sources
Some data will be available directly from programmes, for example from project-level M&E. Ideally, the
duty to collect data should be the responsibility of recipients of ICF funding, or a third-party auditing
entity. This information will need to be kept up to date by liaising with programme managers.
There will be multiple in-country sources for the self-assessment:

• personal contacts, e.g. with central and sub-regional government officials, other donors seeking
to replicate ICF-supported activities and with private investors
• partner Government policy statements and budgets to track changes in political will and capacity
to act
• analyses of the reports of others, for example World Bank reports on government policy and on
the business environment
• project monitoring reports may contain relevant information on capacity development, policy
implementation, etc.
Most Recent Baseline

The baseline should reflect the situation before the ICF support began. An assessment against the
relevant criteria should ideally be included in the Business Case or, if not, one should be made at the
start of the project/programme. It is acceptable to produce retrospective baseline scores if there is
documentation to support these. The baseline score is based on a qualitative judgement assessment,
rather than defined metrics.

Data Issues / Risks and Challenges

To minimise the risk of subjectivity in programmes’ self-assessments, more weight should be given to
examples of transformation where there are multiple sources of evidence to support the ranking and
where the evidence for this is as far as possible factual rather than based on the opinions of a few people
or on speculation.
Quality Assurance
All results estimates should be quality assured before they are submitted during the annual ICF results
return, ideally at each stage data is received or manipulated. For example, if data is provided by partners,
this data should be interrogated by the ICF programme team for accuracy, or are the very least data
should be sense checked for plausibility. When converting any provided data into KPI results data, quality
assurance should be undertaken by someone suitable and not directly involved in the reporting
programme. Suitable persons vary by department; this could be an analyst, a results / stats / climate and
environment adviser / economist.
Central ICF analysts will quality assure results that are submitted and this may lead to follow up requests
during this stage.
To avoid inherent reporting biases, it is strongly recommended that, where possible, data collection is
undertaken by a third party that is not directly involved with implementing the project. Where not
possible, consider using independent evaluations or alternative means to periodically check the validity of
results claims.
Any concerns about data quality or other concerns should be raised with your departmental ICF analysts
and recorded in documentation related to your results return.
Data disaggregation
NA
Annex 1: Why report on KPI 15

It is difficult to overstate the importance of Transformational Change, and therefore the strategic
importance of KPI 15 to the International Climate Finance (ICF) and its development partners and host
governments. Without such change, it will be unlikely that we can limit global warming of the planet to
two degrees Celsius, adapt to the impacts of climate change, or protect and expand our development
gains. Therefore, all UK ICF programmes should be transformational and bring Transformational Changes
in multiple sectors.
Transformations are hard to predict, multi-faceted, multi-causative, and take time to unfold. For these
reasons, monitoring transformation is inherently difficult, and this KPI is complex. It is implicitly linked to
other KPIs and embedded within programme design. It is therefore less important for accountability as to
the purpose and ‘big-picture’ thinking of projects. However, it is not easily aggregated across the
portfolio, or connected to more quantitative metrics such as Value for Money (VfM).
The purpose of monitoring Transformational Change via KPI 15 is not so that a programme can be
completed and report “transformation achieved”. Rather, the purpose is to encourage thinking and
programme design toward the sorts of transformations needed – to ‘design in’ learning from UK ICF
programmes as they are implemented.

Transformational Change “entails a range of simultaneous transformations to political power, social

relations, decision-making processes, equitable markets and technology.” KPI 15 is a qualitative indicator
that assesses the extent to which ICF programmes can be deemed likely to deliver ‘transformational’
change. That is, “change which catalyses further changes, enabling either a shift from one state to another
(e.g. from conventional to lower carbon or more climate-resilient patterns of development) or faster
change (e.g. speeding up progress on cutting the rate of deforestation).
The transformations the ICF seeks to bring about will only be evident after a period of time. Though it
will be necessary to monitor these longer-term changes, they are unlikely to materialise within the ICF
programme, typically up to 5 years. This indicator therefore tracks early signs of transformation, or the
extent to which key ICF activities either are being, or have a good likelihood of being, “transformational”.
It does so by using proxies for drivers of transformation, to assess the extent to which ICF support can
be linked, if not attributed, to likely Transformational Change. Proxies include political will and local
ownership; capacity and capability; innovation; sharing of effectiveness; leverage / incentives for others to
act; replicability; scalability and sustainability.
Applied appropriately, KPI 15 can provide more than a simple accounting function. KPI 15 can help ICF
programme developers and managers think through what is needed for a programme to catalyse deeper,
faster and/or wider changes in a particular sector, area or institutional context. Within these contexts, it
is essential to establish the set of conditions that should catalyse or at least contribute to the particular
Transformational Change sought.
Within this context, KPI 15 as it is formulated and presented in the Methodology Note, above, provides
insights into progress made against Transformational Change.
Annex 2: Examples of indicators against each of the TC

criteria
Criteria Approach and examples of indicators to assess by:
Political will Partner government is acting on climate change, as evidenced by:
and local • the tracking of influencing activities by HMG staff;
ownership • the quality of any national climate change strategy or similar, including whether
Fostering this has been costed and included in the national budget, whether any
political will to proposals it contains for regulatory changes are being or likely to be
act on climate implemented, whether the Ministry of Finance and key line ministries are
change actively tracking indicators of national change (via nationally formulated KPIs or
similar), etc.
• research provided through ICF activities informing debates on climate change
in national parliament or similar
• stakeholder engagement events organised by national or sub-regional
governments on climate change issues
• civil society efforts to foster informed debate on climate change [as measured
by mobile phone campaigns, newspaper column inches, twitter tweets, etc.]
• other [defined by programme or project]
Capacity and Evidence from HMG ICF country offices and spending units of one or more of the
capability following:
increased • Number of Government Departments or Agencies undertaking own analysis of
ICF-supported climate action following HMG support
activities


enhance local • number of sector, national, and decentralized government plans under
capacity to act implementation that mitigate risks and ensure adaptation to climate change by
on climate disadvantaged people and climate vulnerable communities
change • Key institutions addressing the new challenges climate change will pose are
supported by HMG, either to evolve or emerge
• HMG support makes developing country negotiators more influential in
international negotiations
• Relevant capacities developed in the private sector [e.g. creation of/support for
effective trade associations supporting low carbon firms, building the capacity
of financial intermediaries better to understand/assess the risk-reward profile
of new technologies or energy efficiency, etc.]
• Increase in number of peer reviewed climate change publications by UK-
supported local research bodies
Innovative Could include:
HMG- • Number of domestic low carbon technologies supported [where evidence can
supported be taken from the low carbon KPI9]
activities are
• Number of domestic adaptation measures/technologies supported
encouraging
innovation and • Number and potential scope of new policy approaches tested, fostering climate
testing new risk management or low-carbon technologies
approaches • Number and potential scope of new business models being tested and adopted,
supporting climate resilience or clean energy technologies/low carbon practices
• Number of new market mechanisms promoting vulnerability assessments or
achieving emissions reductions piloted
Evidence of • Number of activities (e.g. workshops, key publications) delivered to
effectiveness disseminate climate resilience measures and low carbon pathways programme
Ideas and experience, with evidence of take-up
lessons shared • other [defined by programme or project]
widely
Leverage / Could include:
create • Policy and regulatory reforms initiated through HMG-supported activities cut
incentives for costs for private investors [e.g. introduction of low-cost subsidized adaptive
others to act flood and drought-resistant crop strains; or where HMG has supported the
removal of regulations that hindered investment such as import tariffs on
HMG- essential components for renewable energy]
supported • Legislative changes that enable and encourage new market players, such as
activities are support to allow independent power providers to operate and sell electricity
creating the to the national grid
incentives for • Development and introduction of policies and regulations supported which
others to act on provide positive incentives for new approaches [e.g. where HMG have
climate change supported public tenders highlighting climate adaptive redesign protocols for
infrastructure; or the development and implementation of a Feed-In-Tariff
(FiT)]
• Evidence that public goods provision supported by ICF encourages investment
by others (e.g. new investments behind strengthened flood defences, private


investment decisions informed by publicly available UK-supported climate
projections, etc.)
Replicable • Number and value of UK-developed approaches being copied by others
HMG- [tracked in initiating country or region]
supported • Value of co-financing attracted into UK-initiated interventions
activities are • Volume of public finance leveraged [public finance leveraged indicator]*
being replicated
• Volume of private finance leveraged [use private finance leveraged indicator]*
by others
* These measures could equally fit under the ‘leverage/ incentives for others to
act’ criterion. Which one the programme manager chooses to put them under
will depend on what elements of the generic theory of change are most relevant
to the portfolio in question.
At Scale Ideally this will be a quantitative assessment of resources mobilised relative to the
assessed funding amount necessary to effect the desired change. It will be
location and context-specific.
Such measures may well draw on other criteria and could include:
• Proportion of population at risk whose climate adaptive resilience is judged to
have been markedly improved [drawing on other relevant KPIs]
• X% of infrastructure at risk built to higher standard [e.g. X% of roads
constructed or up-graded to cope with a climate-induced 1 in 5-10 year rain
storm]
• A particular renewable technology accounts for X% of market share
• X% of potential farmers are able to access a particular improved seed variety,
or Y% of farmers have been trained in new flood or drought-adaptive or lower
carbon practices
Sustainable A view on the likely sustainability of ICF-funded activities could comprise a
[Activities are synthesis of the evidence presented on each of the indicators listed above (and
likely to be should certainly draw on the other criteria).
sustained once Where relevant, other evidence should be included in this assessment [defined by
HMG funding programme or project].
ends] Such measures could include:
• Local government representatives paid to continue promotion of climate
risk management measures in farming practices/watershed management
• Target community assumes responsibility for management of solar/wind
projects e.g. within a decentralised 5-year plan
Annex 3: Comparability and synergies with other external

indicators
Although there appears to be no major players with a KPI similar to ICF’s KPI 15, there is considerable
appetite from ICF, ICF partners, donors, and community agency for learning around Transformational
Change. Most international climate financing mechanisms (e.g. ICF, GCF, Adaptation Fund, etc.) require a

comprehensive assessment of Transformational Change and theory of change as an eligibility requirement

for funding and MEL practices.
Annex 4: Definitions of key methodological terms used

across Methodology Notes
As different HMG departments may use the same terminology to refer to different concepts, this section
sets out definitions for key terms used across Methodology Notes for ICF KPIs. The terms used in these
notes refer to the concepts as defined below, rather than to alternative, department-specific usages of
these terms.
Counterfactual: The situation one might expect to have prevailed at the point in time in which a
programme is providing results, under different conditions. Commonly, this is used to refer to a ‘business
as usual’ (BAU) counterfactual case that would have been observed if the ICF-supported intervention had
not taken place.
Additionality: Impacts or results are additional if they are beyond the results that would have occurred
in the absence of the ICF-supported intervention. That is, results are additional if they go beyond what
would have been expected under a BAU counterfactual.
Causality: Causality refers to the assessment that one or more actors bear responsibility for additional
results or impacts, because of funding provided though the ICF or actions taken under an ICF programme.
Multiple development partners may be assessed to have played a causal role in delivering results.
Attribution: Attribution refers to allocating responsibility for impacts or results among all actors that
have played a causal role in programmes that deliver additional results. Results are commonly attributed
to causal actors based on their financial contributions to programmes (though there may be cases where
greater nuance is needed, as with KPI 11 and KPI 12).
Annex 5: Optional KPI 15 Reporting Template

PROGRAMME/PROJECT NAME
Programme/Project summary:
Implementing Years: [insert project years]

Donors: [insert donors]
Reporting level: [e.g. Programme/Country/Region/Fund]
Total Programme/Portfolio/Fund Size: [insert financial size]
UK Contribution to Programme/Fund/Country or regional level intervention: [insert
financial size]
KPI 15 score: [insert “likelihood that transformation linked to the ICF support will occur” score]
Transformational Change: [insert Transformational Change criteria as found in Methodology
Note (e.g. innovation, evidence of effectiveness, etc). Multiple criteria can be mentioned.]
[insert 1-2 paragraphs describing the ICF Programme]
Criteria relevant to KPI 15:
The following criteria are deemed relevant to KPI 15:
1. [insert relevant Transformational Change criteria] [insert TOC level1]
Metrics:
[insert short narrative introducing relevant metrics below (delete / add extra rows as appropriate –
note that at least one criteria should be selected from each level of the KPI 15 Theory of Change,
which can be found above in the Methodology Note). Narrative on metrics should provide an
explanation as to why the metrics have been selected within the context of the
programme/fund/country level intervention]:
Metric [*can be both quantitative and qualitative] Data

[insert metric] [insert data]
A score of [insert score] was awarded, as [insert narrative justifying score and a qualitative judgement
on the strength of evidence].
2. [insert relevant Transformational Change criteria] [insert TOC level2]
Metrics:
[insert short narrative introducing relevant metrics below (delete / add extra rows as appropriate –
note that at least one criteria should be selected from each level of the KPI 15 TOC, which can be
found in the Methodology Note). Narrative on metrics should provide an explanation as to why the
metrics have been selected within the context of the programme/fund]:
Metric [can be both quantitative and qualitative] Data



A score of [insert score] was awarded, as [insert narrative justifying score and a qualitative judgement
on the strength of evidence].
*Repeat step for as many Transformational Change criteria that are relevant
Overall score
Overall [insert programme name] scored a weighted mean score of [insert score and note on
criteria weightings].
1
E.g. Drivers, Mechanism, Enablers
2
E.g. Drivers, Mechanism, Enablers

Photo credits
Photos used in this KPI guidance note series were sourced from two websites, Climate Visuals and Unsplash (except the
photo for KPI 14 which belongs to IMC Worldwide). They are available for use under a Creative Commons license, which
enables organisations provided that photographers are credited. Photographers for this KPI guidance note series are
credited below.
KPI 1, Kenya. Georgina Smith / CIAT

KPI 2, Bhutan. Asian Development Bank.
KPI 4, Indonesia. Sigit Deni Sasmito/CIFOR.
KPI 5, Afghanistan. Asian Development Bank.
KPI 6 Roshni Sidapara.
KPI 7, Sri Lanka. Sansoni / World Bank. Photo ID: DSA0020SLA World Bank.
KPI 9, Abbie Trayler-Smith / Panos Pictures / Department for International Development.
KPI 11, Sri Lanka. Dominic Sansoni / World Bank. Photo ID: DSA0233SLA World Bank
KPI 12, Samuel Zeller. Solar panels.
KPI 13, Bangladesh. Sajid Chowdhury / Big Blue Communications. 2013
KPI 14, Rawpixel
KPI 15, Ryan Searle.
KPI 16 Alessandro Bianchi

KPI 15 Extent ICF Intervention Lead Transformational Change

Uploaded by

Copyright:

Available Formats

KPI 15 Extent ICF Intervention Lead Transformational Change

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

KPI 15 Extent ICF Intervention Lead Transformational Change

Uploaded by

Copyright:

Available Formats

Climate Change Compass helps the UK Government

monitor, evaluate and learn from UK International Climate Finance

Extent to which ICF intervention is likely to

Join the conversation at climatechangecompass.org 1

*Click on page numbers above to navigate to specific sections

About Climate Change Compass

Visit www.gov.uk/guidance/international-climate-finance to learn more about UK International Climate Finance, its

Join the conversation at climatechangecompass.org 2

BAU Business as Usual

Join the conversation at climatechangecompass.org 3

Extent to which ICF intervention is likely to lead to

The main revisions to this Methodology Note are:

Join the conversation at climatechangecompass.org 4

Join the conversation at climatechangecompass.org 5

Figure 1: Theory of Change for Transformational Change:

This KPI will be assessed at programme or portfolio level as follows:

Join the conversation at climatechangecompass.org 6

o 3: Tentative evidence of change – Transformational Change judged likely

• A qualitative/narrative report against the relevant criteria of Transformational Change (see

Figure 2: Expected results

Figure 3: Actual results:

Join the conversation at climatechangecompass.org 7

1. Describe the interventions that comprise the programme or country / thematic

2. Determine the baseline that Transformational Change is being assessed from

4. Determine who else is crucial for ensuring this Transformational Change

5. Describe what successful Transformational Change looks like; when is it expected to

1. Decide on which Transformational Change proxy criteria is relevant

Transformational Change proxy criteria can include:

Join the conversation at climatechangecompass.org 8

• Evidence of effectiveness is shared

2. Determine if the criteria are equally weighted or if different weightings apply

3. Assign metrics to each of the selected criteria

4. Collect data against each of the metrics

5. Based on the data collected, score each criteria individually

Join the conversation at climatechangecompass.org 9

6. Fill in KPI 15 reporting template (*optional)

7. Assign overall KPI 15 score

Consideration of contribution / attribution

1. Decide on which Transformational Change proxy criteria is relevant

2. Determine if the criteria are equally scored, or if different scores apply

The selected TC criteria received equal scoring metrics.

Join the conversation at climatechangecompass.org 10

Table 2: Worked example using reporting template

Implementing Years: 2012 - 2015

Criteria relevant to KPI 15:

The following criteria are deemed relevant to KPI 15:

Evidence of effectiveness is shared [TOC Level: Drivers] 20%

Capacity and capability increased [TOC Level: Drivers] 20%

Join the conversation at climatechangecompass.org 11

Replicable [TOC Level: Mechanism] 20%

At Scale [TOC Level: Mechanism] 20%

Join the conversation at climatechangecompass.org 12

Sustainable [TOC Level: Enablers] 20%

There will be multiple in-country sources for the self-assessment:

Most Recent Baseline

Join the conversation at climatechangecompass.org 13

Data Issues / Risks and Challenges

Annex 1: Why report on KPI 15