PROPOSED M&E STRATEGY FOR THE DFID FUNDED SCRP PROGRAM

1.0 Introduction
This section outlines the key features of the monitoring and evaluation strategy and
framework for the DFID funded School reconstruction - Humqadam program. The section will
describe what is expected to be measured and why, which agencies or units will be responsible
for the measurement and how, the frequency of measurement and how results of the M&E
system will be used and communicated.

The Monitoring and Evaluation system is expected to serve a number of purposes. It will
enable a comparative analysis of targets versus achievements in terms of outputs to be
achieved. These relate mainly to numbers of schools being rehabilitated or upgraded,
classrooms being built, facilities being provided or trainings being undertaken. It will also have
to supply information on how the learning environment has improved and the extent to which
PTCs and the community are actively involved in the functioning of the school. Getting more
children (especially girls) into school, retaining them and supporting them to be more literate
is the purpose of the KESP. The M&E System will have to yield information on these impacts
also. Merely reporting on progress is not sufficient. The M&E system should allow Humqadam
to modify and improve its course based on the findings, it should serve as feedback to the
communities as well as providing lesson learning and ideas for replication within SCRP. Finally,
it should be able to reinforce the policy advocacy component of the programme with robust
evidence from the field.

2.0 Strategy Analysis

Through a consultative process, a problem tree analysis was undertaken in early June 2014.
The main problem which SCRP is concerned with was agreed, and the causes and sub-causes
of this main problem were elaborated. This was also linked to the problem that KESP overall
is trying to address. Next, an objective tree was developed by rephrasing each of the problems
into positive desirable statements. Lastly a strategy analysis was undertaken. The objective
tree is depicted in Annex XX. Those statements which are coloured green are ones which fall
within Humqadam’s mandate. Those left white are ones which lie outside it control and which
need to be included in the project implementation risks and assumptions. Or they could be
issues which Humqadam can support the other components of KESP to address. A separate
risk mitigation matrix has been developed and is included in this document as Annex XX

3.0 Logical Framework

The strategy analysis precedes the development of the logical framework. The Humqadam
logical framework for KESP is shown in Annex XX. This is an initial draft, and as is good practice,
it will be constantly revisited and improved throughout the inception phase and regularly
during the implementation phase. This logical framework, which has been developed using
current DFID standards, shows the various hierarchical levels that need to be measured.

4.0 Hierarchy of Objectives: Inputs, outputs, purpose and goal

The M&E system has to be robust enough to measure the various hierarchies that are
expected to be influenced by the project. As part of one of three components of KESP (or
PESP), Humqadam’s activities related to the construction and rehabilitation of school
infrastructure, contribute to the overall goal and outcomes of KESP. Thus the overall goal and

1
project purpose of Humqadam remain the same as the goal and purpose of the Education
Sector Programme in its log frame and are restated here:

Goal: Human Capital Increased (Better educated and skilled citizens) in Khyber Pakhtunkhwa
Purpose: Sustained increase in the number of girls and boys participating in better quality
primary and secondary education in Khyber Pakhtunkhwa.

Humqadam’s contribution to the above mentioned goal will be through the following three
outputs:
 Rehabilitation and construction of schools i.e. classrooms and other facilities
 Capacitating the School Councils to engage in the planning, construction and post-
construction operation and maintenance of school rehabilitation and construction
 Awareness raising and mobilisation of the communities to provide oversight and
support to the School Councils to enable them to function effectively.

5.0 Input and Outputs Indicators

The monitoring and evaluation system will be linked with the financial management system
within the project. Financial disbursements will be linked to the quantity and quality of
outputs achieved by implementing partners, contractors and the like. The physical verification
of outputs will precede any approval of disbursements. This will not only improve the financial
transparency and accountability of Humqadam’s operations, it will ensure that reporting by
the various tiers and partners is done on time and properly.

On a monthly basis, the following outputs (amongst others) will be measured against targets
in annual work plans:
 Number of schools rehabilitated
 Number of primary schools upgraded to middle and secondary levels
 Number of classrooms rehabilitated
 Number of additional basic facilities (boundary walls, girls and boys toilets, water
supply and electricity) provided
 Number of schools provided with furniture and equipment
 Number of PTCs trained
 Number of PTC members trained
 Number of PTCs undergoing refresher courses (by type)
 Number of PTC members undergoing refresher courses (by type)

All the above indicators will be disaggregated by locality, (rural/gender), gender and school
levels.

5.1 Process Indicators

These indicators will measure changes in PTCs and communities’ knowledge and behaviour.
Such changes cannot be observed immediately and thus there has to be a lag between the
time that support was provided (in terms of mentoring or training) by Humqadam, and when
that training translates into better supervisory and management practices.

2
  Change in capacities and effectiveness of PTCs/SCs including ability to supervise
operation & maintenance,
 Change in of communities including increased ability to engage with their
schools, resulting in greater accountability from service providers

Baselines will need to be carried out to capture any changes. They will combine a mix of
qualitative and quantitative tools (see later).

5.2 Outcome indicators

In order to measure, outcomes, it is necessary to establish baselines. Baseline surveys would
be carried out at the beginning of engagement with any particular school.

 Functionality of school infrastructure

 Levels of satisfaction of teachers with learning environment
 Levels of satisfaction of students with learning environment
 Change in enrolment
 Change in retention
 Change in transition
 Change in CSR

The baseline surveys would be followed by impact assessment surveys at end of the
rehabilitation of the schools and then annually or biennially to measure changes in the above
indicators.

6.0 Data Collection Methods

6.1 Goal level data

At the goal level, the project will rely on secondary data. The goal is a higher level objective
encompassing beneficiaries beyond the immediate ones that the project targets. It is an
objective towards which the project is contributing, and is often expected to be achieved after
the project has concluded. In the case of Humqadam, the goal level objectives include lower
unemployment and higher literacy in the entire province. The project contributes towards
these objectives but is not responsible for their attainment or measurement. The
government’s Federal Bureau of Statistics carries out large scale surveys such as the labour
market surveys and the Pakistan Social and Living Standards Measurement (annually) and the
national census which has not been carried out since 1998. Then there are the provincial
information gathering and consolidating mechanisms such as the Elementary and Secondary
Education Department’s (E&SED) Education Management Information System. (EMIS). Such
secondary data will be relied upon to report against the indicators at goal level as these
pertain to the whole of KPK. The problems that exist there are that government statistics are
often unreliable or only available a considerable time after the data collection was
undertaken. This reduces their current relevance.

However, in the absence of any other system or organisation which collects province-wide
data, Humqadam will have to rely on these sources with expectations that the reliability of
data will be satisfactorily high and that it will be regularly collected and available throughout
the life of the project. The project will endeavour to work with EMIS to improve the quality of

3
data collected and to share the data which will be collected from within Humqadam’s own
M&E system.

6.2 Purpose level data

The project is directly contributing to increased enrolment, increased retention and transition
and a better learning environment. The purpose level indicators pertain to the schools, the
PTCs and the communities that the project will directly be involved with. Measuring those falls
within the responsibility of the project. In this regard, baseline surveys will be conducted to
measure the extent of enrolment, retention and transition. These will be done in a phased
manner for each school at the time that school has been targeted and the planning process
(for rehabilitation) begins. Structured questionnaires will capture student numbers (gender-
disaggregated) and the state of infrastructure (for example, the number of toilets, both
functional and those in disrepair, the condition of the classrooms both in terms of sufficiency
and condition of furniture e.g., fractures in walls).

Supplementing this will be more qualitative methods of evaluation exploration looking at

satisfaction of students, teachers and PTCs with the learning environment within the schools.
For this, the project will deploy tools such as focus group discussion and perception surveys
though given the volume of schools which are being targeted, this will be done on a sample
basis. Similarly, with regards to community mobilisation efforts, PAIMAN will conduct
Knowledge, Attitude, Practice and Behaviour (KAPB) surveys on engagement of a community
and again on exit. In the interim, the third party monitoring will undertake monitoring to
examine the processes that are being followed, to flag any issues and identify good practice.

The baselines will be repeated for impact assessment once the schools have been
rehabilitated or upgraded. These assessments will continue annually or, depending upon
resources, biennially. The resources required for effort will be considerable. Thus one option
being considered is to avail of the existing structure of information collection in the province.

In KP, an Independent Monitoring Unit has been established and is currently being
strengthened. Under the IMU, there will District Monitoring Officers aided by assistants at the
union council levels. Humqadam will dialogue with the Monitoring Unit to determine where
they can build mutually beneficial arrangements for collecting and sharing information. The
Monitoring Units currently collect a subset of the indicators that will be needed by Humqadam
to report on its progress. The School Report Card tool currently being rolled out in the province
will be assessed for its usefulness and adoption by Humqadam either in its current state or in
a developed form. The Report Cards in their current state yield useful information such as
student to classroom ratio (SCR), student to toilet ratio, and the functionality of various
facilities.

(This paragraph is for PESP: In the Punjab, District Monitoring Officers (DMO) based at districts
report to the Project Monitoring and Implementation Unit (PMIU). The PMIU oversees the
implementation of the education sector reform programmes. The DMOs in turn supervise a
field staff of Monitoring and Evaluation Assistants (MEAs), who are responsible for
undertaking regular field visits and collection of data on facilities. During the Inception Phase,
we will explore the extent to which we can both a) collaborate with this structure for
information and labour-sharing and b) use the data generated from this system).

4
6.3 Output level Indicators
The first of the three outputs relates to physical infrastructure and this data will be supplied
by NESPAK and validated by the monitoring officers based at the district level.

The second of the three indicators pertains to the effectiveness of the PTCs. For this, it is
expected that both quantitative and qualitative methods of data collection will be used.
Regarding whether the PTCs are regularly meetings and attendance, record-keeping,
undertaking good financial management etc., these issues can be captured using quantitative
checklists and rated using methods such as institutional indexes or other rating scales. These
will be undertaken for all PTCs targeted for training before and after the main capacity building
training which will last around 3 days.

Issues as to whether the PTCs had established innovative mechanisms for school management
after their capacity building or undertaken initiatives to seek resources from other sources
would have to be captured using more qualitative tools such as case studies, focus group
discussions or outcome mapping. Currently, it appears that given the magnitude of PTCs to be
assessed qualitatively, Humqadam will only be able to evaluate the performance of a sample
of them. For some of these, it can also pursue longitudinal assessments.

Built into the M&E system and its information gathering tools will be indicators to collect
information on essential cross-cutting issues such as gender, the environment and WASH
(water, sanitation and hygiene). Capacities of PTCs will be built to monitor social and
mitigation issues and their mitigation issues. PTCs will have to monitor these issues during
planning, construction and post-construction operation of the schools. It is expected that
formats such as those reproduced in Annex XX will be used to by PTCs during this monitoring
and will be consolidated on a quarterly basis by the district teams. Similarly in WASH, topics
will be included in the PTC training modules so that they are in a position to promote good
hygiene by ensuring that there are facilities and conditions to practice hygiene in school and
at home. The change in knowledge, attitude and practice of PTCs, teachers and students will
be captured on a regular basis through the various suggested tools. More quantitative
elements will be captured through the School Report Cards and Humqadam a district
monitoring surveys. The qualitative aspects will be captured through the envisaged I-SAPS
surveys, focus group discussions, observation checklists and third party process monitoring.

7.0 Structure
The first level of monitoring will be the Parent Teachers Councils and School Committees.
Their capacities to undertake planning, construction supervision and post construction
operation and maintenance will be built both through formal and informal trainings. This
capacity building will be undertaken by I-SAPS in a staggered manner across the province. The
Parents Teachers Councils will be expected to undertake, amongst other responsibilities, the
following:

- Monitor rehabilitation of the school infrastructure including environmental and social

monitoring
- Keep registers and records of meetings and minutes
- Monitor teacher and student attendance

5
 - Monitor operation and maintenance of the school infrastructure and facilities
including WASH, sanitation
- Financial monitoring

At the district level, Humqadam will build its monitoring and evaluation capacity. In each
district, there will be dedicated monitoring and evaluation staff within the School Support
Team who will be responsible for the following:
 Work with NESPAK to record the current state of school buildings and
associated infrastructure
 Compile monthly progress reports
 Work with project partners such as I-SAPS and Paiman in conducting their M&E
activities
 Ensuring all information is fed into the M&E MIS
 Facilitate the work of the third party monitoring
 Facilitate the holding of focus group discussions
 Developing case studies for lesson learning and stories of change
 Undertaking the physical verification of outputs and activities as a pre-
condition for approval of financial disbursements

8.0 Project Partners

In terms of monitoring of physical rehabilitation of schools, which will form the major
component of Humqadam’s portfolio in terms of financial outlay, Nespak will supervise the
planning and implementation of physical works and will report on all information regarding
rehabilitation and construction. Such information will be gathered on a continuous basis and
reported upon monthly. The information will be disaggregated by gender, locality and level of
school.

The number of trainings and refreshers being provided to PTCs and SCs and their type will be
collected on a monthly basis. This will be done by the partner I-SAPS which will be undertaking
the trainings. All information will be gender disaggregated. Similarly, PAIMAN the partner
responsible for community mobilisation will be reporting on a monthly basis upon the number
of village assemblies conducted and both the type and numbers of trainings given to the
Agents of Change (AoC).

The Monitoring & Evaluation Section in Islamabad will be overall responsible for the project’s
M&E activities and outputs. It will ensure that the project is sufficiency equipped in terms of
M&E capacity which will be routinely reinforced. It will be the repository of all information
being collected in the M&E system. It will undertake analysis of the data to aid senior
management in supervision and decision-making. In addition, it will constantly track
achievements against targets for all programme components i.e. infrastructure development,
PTC development and community mobilisation. Each month information on inputs and
outputs would be compiled and linked up with the financial management system. Every
quarter, an environmental monitoring report will additionally be generated.

In order to enhance the transparency and authenticity of data, a third party will be contracted
for process monitoring. The third party will continuously track and verify progress at the
implementation level, using both quantitative and qualitative methods, for indicators and sub-

6
indicators at the purpose and output levels. It will be regularly deployed in the field, have an
independent structure and collaborate closely with the monitoring and evaluation section of
Humqadam to ensure relevance of its outputs for all stakeholders. Apart from looking at the
effectiveness of the capacity building activities, it will assess increases in enrolment and
retention, and improvement in the quality of the educational environment in targeted
schools.

9.0 Knowledge management

An information database will be developed which links the district, provincial and national
tiers of the project. Information will be stored electronically and quantitative data gathered
through the use of tablets and/or smartphones where possible. The use of tablets and similar
devices will enhance efficiency, transparency and accuracy. The information will be collected
in written and visual (photographic and video) forms and provision will be made in the
information database to store all three types.

The data emanating from the M&E system would be used to communicate the progress of the
project to external stakeholders as well as for monitoring efficiency, identify what is working
and what is not, and for course correction. The periodic or regular reporting will be
complemented with documentation of case studies highlighting lessons being learnt and best
practices. Regular progress review and planning meetings of project staff at the district,
provincial and national levels will support management in its monitoring responsibilities as
well as information sharing and learning.

M&E data will also be used to support policy advocacy both at provincial and donor levels.
This effort will be led by the partner I-SAPS which has considerable experience in this regard.
The project also aims to partner with other projects and organisations such as the DFID-
supported initiatives of Alf Ailaan and Idara-e-Taleem-o-Aagahi (ITA) to feed into their policy
advocacy and awareness-raising mandates.