Fao Good Learning Practices For Effective Capacity Development
Fao Good Learning Practices For Effective Capacity Development
Fao Good Learning Practices For Effective Capacity Development
http://intranet.fao.org/fao_communications/idwgcd/
Thank you,
FAO Capacity Development Team
Knowledge and Capacity for Development (OEKC)
The designations employed and the presentation of material in this information product do not imply the
expression of any opinion whatsoever on the part of the Food and Agriculture Organization of the United
Nations (FAO) concerning the legal or development status of any country, territory, city or area or of its
authorities, or concerning the delimitation of its frontiers or boundaries. The mention of specific companies
or products of manufacturers, whether or not these have been patented, does not imply that these have been
endorsed or recommended by FAO in preference to others of a similar nature that are not mentioned.
ISBN 978-92-5-000000-0
All rights reserved. FAO encourages reproduction and dissemination of material in this information product.
Non-commercial uses will be authorized free of charge, upon request. Reproduction for resale or other
commercial purposes, including educational purposes, may incur fees. Applications for permission to
reproduce or disseminate FAO copyright materials, and all queries concerning rights and licences, should
be addressed by e-mail to [email protected] or to the Chief, Publishing Policy and Support Branch, Office
of Knowledge Exchange, Research and Extension, FAO, Viale delle Terme di Caracalla, 00153 Rome, Italy.
© FAO 2012
Table of contents
Overview and objectives 9
Introduction 12
Step 1 - Assess the context: Is it a learning issue? 19
1.1 The importance of context 20
1.2 Analyze the context: is it a learning issue? 21
1.3 Identify complementary support 23
Key learning points 24
Planning checklist 25
Step 2 - Strategically identify the target audience 26
2.1 Strategies for identifying target audiences 27
2.2 Work with counterparts in Member Countries 29
Key learning points 31
Planning checklist 32
Step 3 - Assess learning needs 33
3.1 Define the approach for a Learning Needs Assessment 34
3.2 Undertake a Learning Needs Assessment 34
3.3 Identify local resources to support sustainability 36
3.4 Developing a concept note 40
Key learning points 41
Planning checklist 41
Step 4 - Design content and select delivery modes 42
4.1 Define learning objectives 43
4.2 Define delivery mode and learning method 45
4.3 Organize learning contents 52
4.4 Define Follow-up support 58
4.5 Define logistics for delivering face-to-face learning 59
Key learning points 59
Planning checklist 60
Step 5 - Deliver the learning solution 61
5.1 Identify roles for delivering learning initiatives 62
5.2 Ensure quality and coherence in delivering learning initiatives 65
Key learning points 65
Planning checklist 65
Step 6 - Provide follow‑up support 66
6.1 Arrange Follow-up support 67
Key learning points 69
Planning checklist 70
Step 7 - Evaluate learning initiatives 71
7.1 Evaluation objectives 72
7.2 Customize approaches for results evaluation 73
7.3 Customize instruments for process evaluation 76
Key learning points 77
Planning checklist 77
Conclusions 78
Annex 1 - The international understanding 79
Annex 2 - Contents of Toolboxes 81
Glossary 92
3
Toolbox 1 93
Tool 1a: Assess the context: Is it a learning issue? 95
Toolbox 2 98
Tool 2a: Identify the target audience profile 100
Tool 2b: Participant Application Form Template 103
Toolbox 3 105
Select the learning needs assessment (LNA) approach
Tool 3a: Learning needs assessment: Process and tips 107
Tool 3b: Overview of data collection techniques 109
Tool 3c: Managing focus groups 111
Tool 3d: Key informant interviewing 113
Tool 3e: Developing surveys 117
Undertake the learning needs assessment (LNA)
Tool 3f: Learner analysis 120
Tool 3g: Learning needs assessment checklist 122
Tool 3h: Work setting analysis 124
Toolbox 4 126
Define learning objectives
Tool 4a: Essential principles for adult learning 128
Tool 4b: Tips and checklist for writing learning objectives 131
Tool 4c: Task analysis 134
Select delivery mode
Tool 4d: Blended learning 138
Tool 4e: What is on-the-job learning? 141
Tool 4f: Coaching and mentoring 143
Tool 4g: Study tours 147
Tool 4h: Is ad hoc technical coaching appropriate? 151
Tool 4i: Elements of Terms of Reference for technical coaches 153
Identify, organize and develop learning content
Tool 4j: Structuring the learning event 154
Tool 4k: Preparing written learning material 156
Tool 4l: Preparing a design outline 158
Toolbox 5 161
On-the-job
Tool 5a: Experiential learning 163
Tool 5b: The technical coaching process: approaches and tips 167
Tool 5c: Action learning 176
Tool 5d: Action plan for coaching 179
Tool 5e: Review of coaching progress 182
4
Face-to-face training workshops
Tool 5f: Learner’s log 183
Tool 5g: Key tasks for trainers/facilitators 184
Tool 5h: Key phases of a training workshop 187
Tool 5i: Post-workshop reflection and action plan 190
Tool 5j: Delivering effective PowerPoint presentations 194
Online collaborative
Tool 5k: Comparing face-to-face and online facilitation tasks 196
Toolbox 6 202
Tool 6a: Developing a plan for follow-up support 203
Tool 6b: Template for a follow-up plan 206
Tool 6c: Types of follow-up measures 207
Toolbox 7 209
Planning for evaluation
Tool 7a: Kirkpatrick’s results evaluation framework 211
Tool 7b: Designing an evaluation plan 215
Level 1: Evaluation of learning results: participants’ satisfaction
Tool 7c: Evaluation of participants’ satisfaction: methods and tips 221
Tool 7d: Open feedback form 223
Tool 7e: End-of-course evaluation of face-to-face training 224
Tool 7f: Daily evaluation of face-to-face training 227
Tool 7g: Mid-term evaluation of face-to-face training 228
Level 2: Evaluation of learning results: participants’ learning
Tool 7h: Evaluation of learning results: formal and informal methods 230
Tool 7i: Assessing changes in learners’ perception 232
Tool 7j: Example of pre/post-course knowledge test 235
Level 3: Evaluation of learning results: changes in participants’ behaviours
Tool 7k: Evaluation of changes in participants’ behaviours and practices: methods and tips 238
Tool 7l: Participant benchmark evaluation 240
Tool 7m: Example of participant benchmark evaluation 244
Process evaluation
Tool 7n: Process evaluation 251
Tool 7o: Evaluating the quality of facilitation 252
Tool 7p: Evaluating the quality of the coaching process 254
Toolbox 8 256
Tool 8a: Writing a concept note 258
Tool 8b: Planning checklist 261
Tool 8c: Checklist for quality assurance 265
Tool 8d: Checklist for event preparation 269
5
Acknowledgments
This Module was developed by Cecilia de Rosa, OEKC Capacity Development Officer with the
support of Ms Aliza Inbal Belman, OEKC Consultant. The following people contributed to the Module:
Astrid Agostini, Lorenzo G. Bellù, Sally Berman, Marco Boscolo, Rosana Frattini, Kevin Gallagher,
Lavinia Gasperini, Beatrice Ghirardini, Gabriela Gold, Claudia Hiepe, Joe Hoper (UN‑Habitat),
Helga Josupeit, Fred Kafeero, David Kahan, Jamie Morrison, Andrew Nadeau, Patrick Otto,
Jenny Pearson (Independent Consultant), Ana Peralta, John Preissing, Mike Robson, Mariagrazia
Rocchigiani, Stephen Rudgard, William Settle, Susana Siar, Ilaria Sisto, Katrin Taylor, Sophie Treinen,
Francoise Trine. Resources have been used and adapted, with permission, from the World Bank
Group and the United Nations High Commissioner for Refugees, Global Learning Centre.
Special appreciation goes to Skiprock Creative for the visual design of the module.
6
Acronyms
LM learning module
7
FOREWORD
This Module is the third title in a series that the IDWG, under the leadership of the FAO
Knowledge and Capacity for Development Branch (OEKC), has produced. It reflects the
multidisciplinary nature of Capacity Development, which cuts across the entirety of the
Organization, and the importance that FAO management and staff have placed upon it.
It is my view that this Module will be useful for FAO Officers and Collaborators
involved with developing and/or delivering learning activities in member
countries. This Module covers the following curriculum areas:
The Capacity Development Learning Modules aim to keep FAO staff and consultants abreast
of evolving Capacity Development practices in the international development community.
8
Overview and objectives
This Module is the third in a series of four that address capacity development
competencies in FAO. It is intended to enhance FAO’s practices in designing, developing,
delivering and evaluating its activities in support of learning in Member Countries,
while ensuring that learning leads to sustainable capacity development.
Learning objectives
By reading and practising the guidance of this Module, you will increase your ability to:
The following chart will help you tailor your PLP. Please identify the topics in which you are
most interested, and then refer to the ‘core’ and ‘recommended’ sections of the Module.
9
HOW TO USE THIS MODULE
This publication defines the seven Steps that make up the
Learning Management Cycle. Each of these steps is achieved
through Key Actions. Core Tools and Recommended Tools
are provided for accomplishing the Key Actions.
>> Key Actions are indicated by a and are clearly marked on the cover
page of each Step and further described in the text. In each chapter
you will find “FAO good practices” boxes that illustrate specific
applications of the concepts in FAO projects and programmes,
“Planning checklists” for coordinating and managing learning
initiatives, and links to specific tools in the Toolbox section.
10
Recommended
Core sections sections
Needs and interests
Module Toolbox Toolbox
4. Find out the most cost-effective Chapter 3 3a, 3b, 3f, 3g, 3c, 3d, 3e
way to undertake a learning needs 3h
assessment of the target audience
7. See which creative modalities, other than Chapter 4, 5 4d, 4e, 5a, 5b, 4f, 4g, 4h,
face-to-face workshops, have been used by 5c, 5d 4i, 5k
FAO to support learning in Member Countries
9. Find out practical ways to evaluate the Chapter 4, 7 7c, 7d, 7e, 7h, 7f, 7g, 7j, 7m
results of the learning initiative 7i, 7k,7l
10. Ensure effective coordination and quality in Chapter 1-7 7n, 8a, 8b, 8c 8d
the overall design of the learning initiatives
11
Introduction
FAO promotes learning for national stakeholders in Member Countries as part of its technical
work in support of national capacity development. It supports a broad range of learning
activities – from seminars for ministerial staff that last a few hours, to an experiential field school
course for farmers that lasts a season, or curriculum reform in agricultural higher education.
FAO supports formal and informal initiatives which use a variety of methods in different technical
sectors, as shown in the diagram below. The initiatives emphasize learning methodologies that stress
learners’ choice and ownership, learners’ participation and appropriate approaches for adult education.
-Diagram 1-
Over the past two decades, the international community has evolved considerably
in its understanding of the role of learning in developing sustainable capacity.1 In the
past, it was more common to view stand-alone training as sufficient to build capacity;
however, recent evaluations in this area2 have suggested that one-off interventions are
rarely successful in developing sustainable organizational or institutional capacity.
1 See, in particular, the OECD-DAC paper Training and Beyond: Seeking Better Practice for Capacity Development, January 2010,
by Jenny Pearson http://www.oecd.org/dataoecd/35/53/44696077.pdf and the Berlin Statement on International Development
Training http://siteresources.worldbank.org/WBI/Resources/BerlinStatementonInternationalDevelopmentTraining.pdf
2 Evaluation of FAO’s Activities on Capacity Development in Africa (2010); Using Training to Build Capacity for Development, An
evaluation of the WB’s Project-based Training http://siteresources.worldbank.org/EXTTRABUICAPDEV/Resources/full_doc.pdf
12
FAO, other UN agencies and development organizations have undertaken to embed all learning
activities in broader capacity development interventions which address gaps for individuals
and organizations, as well as factors in the enabling environment, in a well‑sequenced,
comprehensive manner. Annex 1, at the end of this Module, provides an overview of the
current international consensus on the role of learning in capacity development.
The Module uses the term “learning activity/initiative” to indicate any type of structured or
semi‑structured initiative or intervention with the primary aim of supporting improved work
performance and behavioural change of individuals in FAO’s Member Countries, in a way that enables
them to better contribute to the development goals of their own organizations and countries.
Implicit in this notion is the understanding that for FAO’s learning activities to support
the development of sustainable capacities, learning must be integrated into a portfolio
of interventions that address factors other than knowledge and skills (e.g. management,
motivation, incentives or governance) which can support a gradual uptake of changes
across the organizational dimension and the enabling environment.3 Further, a number of
different delivery methods should be used to have a better impact on participants.
Knowledge is usually acquired quite directly, for example through presentations, readings,
demonstrations, manuals, document reviews or exercises. However, participants will only retain that
knowledge if it is reinforced through application. Developing new skills requires time to master the
underlying knowledge and concepts and to extensively practice and experiment. Realizing a change in
attitude may take longer, and the new attitude needs to be consistent with local culture and practice.
3 Capacity development involves three dimensions, all of which can be addressed by a combination of modalities:
(1) an individual dimension: people’s knowledge, skills, attitudes and values, which can be addressed by learning;
(2) an organizational dimension: the overall functioning and performance of an organization, often noted in
changes in organizational mandates, systems, processes or priorities; and (3) the enabling environment: the
institutional set up of a country, its power and governance structure and the policy and legal frameworks. See
Learning Module 1 – Enhancing FAO’s practices for supporting capacity development of Member Countries.
13
In sum, triggering and managing a behavioural change requires not only a well-designed
learning initiative that address knowledge, skill and attitude development; it also requires:
-Diagram 2- FOLLOW UP
TO LEARNING
ORGANIZATIONAL
LEARNING SUPPORT
KNOWLEDGE
ACQUISITION
I know what ‘good fishing practices’ are and can name at least two
practices prohibited by customary and national laws. I’ve seen how
such good practices work and what it takes to put them in place
POLICIES AND
SKILL DEVELOPMENT
I’ve used and applied good fishing practices in my own context. I
LEGISLATION
can now find a solution to an existing problem using the new
knowledge and skills I’ve acquired
ATTITUDES
I am aware of the importance of good fishing practices for the
safeguard of fish species and I am helping to raise awareness of
other fishermen. I am motivated to continue using good fishing
practices as I strongly believe in preservation of species
PROCESSES INCENTIVES
AND EQUIPMENT
14
The learning management cycle
Good learning practice is based on the learning management cycle (LMC) shown in Diagram 3.
The diagram provides an overview of the process required to create effective learning situations
(also called the instructional design process). While all of the steps in the process are important
parts of learning management, their sequence may not be the same for every learning initiative.
For example, sometimes it may be more useful to identify the target audience after assessing
learning needs, or to undertake context analysis and learning needs assessment at the same time.
Your decisions throughout the cycle will be guided by the scope of the initiative being planned,
together with the time and budget available. The LMC is intended to help you define the “best
fit” for your learning initiative, while highlighting core aspects of the process that have to be
considered irrespective of the scope, time or budget. Because learning initiatives are very often
designed, developed and delivered by consultants or FAO external collaborators (i.e. NGOs or
other partners), FAO staff must be well-equipped to coordinate and manage the initiatives.
Toolboxes 1 through 7 correspond to each of the seven steps in the Learning Management Cycle.
15
Also, gender and culture should be considered throughout the LMC, including, for
example, when assigning activities, identifying participants or selecting facilitators. It is
critical to recognize the culture of the local context; failing to do so can negatively affect
the results of the initiative and, even worse, FAO’s relationships with national and local
counterparts. The chapters that follow focus on each of the steps in the LMC.
>> Whether the learning initiative is targeted to local or national organizations, ministerial
bodies, research institutes, individual farmers and producers or civil society, local and
national ownership are essential. One of the clearest indicators of ownership is the
quality of national partners’ engagement throughout the learning process.
>> The commitment of target groups is indispensable to being able to effectively
analyse the context; without such commitment, learning needs assessment may not
generate sufficiently in-depth information on needs and capacity constraints.
>> A strong sense of ownership among stakeholders is needed to ensure that they will help identify
appropriate participants for the initiatives and that participants are committed to learning.
>> To encourage learners to use the new learning in the work setting,
there must be support in the work environment.
>> If target groups have a clear sense of the benefits that the learning initiatives can have for
their organizations, they are likely to provide more useful input to improve the initiatives.
16
THE LEARNING MANAGEMENT CYCLE
Remember throughout the learning management cycle to:
Plan, Manage and Coordinate
See Toolbox 8
STEP
1 Assess context: Is
it a learning issue?
- page 19 -
STEP
2 Strategically identify
the target audience
- page 26 -
STEP
3 Assess
learning needs
- page 33 -
STEP
4 Design content and
select delivery modes
- page 42 -
STEP
7 Evaluate learning
initiatives
- page 71 -
STEP
6 Provide follow-up
support
- page 66 -
STEP
5 Deliver the
learning solutio
- page 61 -
17 18
STEP
1 Assess the context: Is
it a learning issue?
This chapter will look at ways in which FAO and national or local counterparts can analyse
the context before moving into the design of a training or other learning event.
Organization refers to both formal and non-formal organizations relevant to the Agriculture and Rural
Development sector, such as central and decentralized government agencies and ministries, social protection
services, inspectorates, laboratories, national agricultural research systems, global and regional economic
commissions, enterprises, cooperatives, chambers of agriculture, consumer groups, producer associations,
community-based organizations, NGOs, and formal and non-formal education and training institutes .
>> Sometimes the physical aspects of the work setting may not allow the use of the learning,
such as when the proper equipment or resources are lacking. An example of this, faced by the
IAEA-FAO Joint Division in Vienna, occurred when developing country scientists were trained
in a developed country setting in advanced techniques for pesticide residue analysis, and
then found it difficult to translate the learning to the reality of their laboratories at home.
>> The social aspects of the work setting may be inadequate, such as when there is a lack
of support from the community, managers, colleagues or organizational culture, or when
there are gender-related problems (e.g. a female worker takes a leadership skills course
but is unable to use her new skills once she is back in a male-dominated environment).
>> Learners might be left alone after the learning initiative without follow‑up,
or they might find that policies and organizational structures in their work
settings make it impossible to apply what they’ve learned.
>> Learners may not have sufficient motivation or incentive to use the learning
in their work, perhaps because it was not relevant to their work needs
or because there is no reward for improved performance.
20
To illustrate the complexity of context, it is interesting to cite a case in crop protection in Uganda.
Researchers and NGOs have been actively working against a particular banana disease for the past
five years; however, even if farmers and local officials have seen extension materials (e.g. posters,
leaflets), many do not know how to practically manage the disease. Many of those who do know how
to manage it do not apply the recommended disease control measures – which would protect the
whole community from infection – because the measures take extra time and require the farmers to
visit their fields every day instead of once a week. In other cases, they have discovered that by planting
alternatives to banana, they can make more money, although they may be reducing long-term soil
fertility and potentially damaging the environment. Also, when farmers are old or affected by disease,
they may not be able to do extra work unaided. Communities have responded by passing by-laws on
the management of banana plantations. This example shows that understanding the whole context is
essential to designing learning that can help the whole community protect their collective livelihood5.
Where organizations are the main stakeholders, assessing the organizational context is
important to gather a full picture of the situation. In some cases, such as when support to
learning is clearly going to be a major component of a project, the organizational context
assessment can be carried out in conjunction with a learning needs assessment.
Consider, for example, a situation in which there is a need to boost agricultural production
and address declining soil fertility in a given African region, and yet a certain country is found
to not be using sufficient quantities of mineral fertilizer. The question to ask is: Why is this the
case? Is it a matter of not being able to afford or obtain fertilizer? Is it that fertilizer application
is an investment with heightened risk in the case of drought? Is part of the problem that there
is no tradition of fertilizer use and it is a matter of individual knowledge and skills, or are other
contextual factors contributing to the situation? Some form of learning support may help
encourage increased use of fertilizer, but some other interventions can address macro issues
pertaining to the enabling environment (e.g. access to credit and financial incentives).
21
Table 2 presents the factors that may influence the context:
Guidance Does the person have sufficient and reliable information to carry out the job
(e.g. expectations from the job and feedback on performance)?
Resources Does the person have the financial resources and equipment to perform the
work?
Incentives Are the financial and non-financial incentives adequate to support work
performance (e.g. career prospects, salary increases)?
Knowledge/skills Does the person have appropriate knowledge and skills to carry out the tasks
or are some gaps evident?
Motivation Does the person have the appropriate motivation to carry out the job (e.g.
interest in the job area)?
Organizational processes Does the person have sufficient support from the community, from line
and support managers, peers or colleagues? Are relevant organizational processes in place
to facilitate individual work? (e.g. standard operating procedures)
Policy and Does the organization have the mandate and legitimacy to operate in the
macro‑economic area? Do policies and/or macro-economic context support individual and
environment organizational practice in the area?
There may be no clear cut answer to the question of whether something is a learning
issue; in most cases, however, support to learning can only partially address capacity
needs, as shown by the decision tree in Diagram 4. In addition, a simple tool is provided in
Toolbox 1 to help determine whether a capacity issue can be addressed by learning.
22
What is the capacity issue?
Assess organizational context and enabling environment
23
BOX 1: FAO good practices – Embedding learning
in broader support strategies
Changed focus of agricultural extension requires a shift in mindsets. Agricultural extension has an
important catalytic role in the development of the food crops sectors. So far, the focus has been to support
small-scale farmers to increase production and productivity primarily through the transfer of technology
from research to extension to farmers. Recently, however, greater focus has been put on improving access
to food for rural households by encouraging farm enterprise diversification. This requires a “market
approach” to extension and, hence, a change in mindsets.
Through the project “Strengthening the agricultural extension system through agro-enterprise
development” in Sri Lanka, FAO undertook to develop extension workers’ capacities in market-oriented
production by engaging them in a Farm Management Training of Trainers. At the same time, the project
sought to address the expected constraints arising from a context in which extension had traditionally
been perceived as a means to support production and productivity. To this end, an awareness campaign
was conducted to ensure that decision-makers at all levels understood and supported the changed
focus of extension services and that the required organizational changes were made in the Government
structures. (Source: Rural Infrastructure and Agro-industries Division (AGS), project TCP/SRL/3302 (D))
Improving agriculture statistics in Africa. The training/learning component of the global strategy to
improve agriculture statistics in Africa includes developing the capacities of national and local training
institute staff through training, twinning arrangements and mentoring support. At the same time, the
strategy seeks to support an organizational reform of universities to become more dynamic actors rather
than relying only on direct public funds as they had in the past. Obsolete university curricula and lack of
motivation/incentives of university professors are among the major contextual constraints that the strategy
ought to address. (Source: Statistics Division (ESS))
24
Planning checklist
Discussion
Decision still N/A
taken required
25
STEP
2 Strategically identify
the target audience
Identifying the audience at an early stage enables targeting the appropriate levels
and categories of participants to ensure that organizational goals or other desired
changes may be achieved. To maximize the impact of a learning initiative, participants
must be “the right people, at the right time, and in the right combination”7.
For example, in the case of the banana disease facing Uganda, a study tour was organized to a
neighboring country where the disease had largely been brought under control. The tour involved
visiting ministries, research stations, local authorities and farmer field school (FFS) groups working to
control the disease. The question to be addressed was who should attend the study tour. While farmers
would obviously benefit, it was decided that the local ministry of agriculture inspectors, local politicians,
community leaders and NGOs were just as important to create a coalition for change at a larger scale8.
>> Consider who should learn what to help bring about the improvements or the changes that the
learning initiative aims to trigger (e.g. people who influence policy-making in a given area).
>> Ask whether the target audience/target organizations are adequately linked to
broader constituencies and whether they are in a position to extend or transfer
their newly acquired learning and experience to members of those constituencies
(e.g. extensionists, training institutes, community motivators).
>> Consider whether the audience has the required institutional support to promote change
in this area and, if not, whether such support can be strengthened as part of the initiative.
For example, if there were individual champions who promoted change initiatives within
single departments of a Ministry, but who did not have full support from the Minister’s
cabinet, they could be supported through high-level joint advocacy measures.
27
Strategy
Type Appropriate use of strategy
description
Competitive Selection based Recruitment of new personnel to be trained for specific roles in the
on a competitive target organization. Selection may be phased, with performance in
application the first stage of the course determining continued participation.
process
Appropriate to ensure uniform high quality of participants,
particularly in initiatives to train the trainers. Example: train the
trainers on Collaborative Conflict Management for Enhanced
National Forest (Forest Economics, Policy and Products Division
(FOEP))
Targeted Invitation of Learning needed only for people fulfilling key functions within an
participants based organization, or people likely to act as agents of change within an
on highly specific organization. May be key decision-makers or people with specialized
job profiles technical skills. Example: Policy Learning Programme, Policy and
Programme Development Support Division (TCSP)
Widespread Supporting Supporting learning of a critical mass of people across the same
learning of many sector so that everyone in the organization or sector supports and
people in an understands how to implement change. Appropriate to build support
organization or for change. Example: livestock sector training (FAO Somalia) or FFSs
across a sector, in integrated pest management (various countries in West Africa)
often with different
job functions,
levels of expertise
Adapted from “Using Training to Build Capacity for Development: An Evaluation of the World Bank’s Project-
based and WBI training http://siteresources.worldbank.org/EXTTRABUICAPDEV/Resources/full_doc.pdf
Identifying participants for learning activities should be a collaborative effort between FAO
and local stakeholders. FAO may not make the final decisions, but can provide guidance on
profiles of people to include. Where participation is based on government nomination, external
factors may influence the selection and may not be linked to the relevance of the participant’s
work to the goals of the initiatives. For example, where the initiative involves free trips or per
diems, participant selection may be influenced by a desire to reward specific employees.
As a rule of thumb, always consider options for organizing learning activities that minimize
travel or other benefits and take place in or near the workplace. Where international events
are needed – for instance to share regional experiences – and participant selection is done
by government nomination, it is critical to maximize dialogue with partner organizations to
gain a good understanding and ownership of the initiative by the local population.
28
2.2 Work with counterparts in Member Countries
Some tips for working with government partners to ensure good
selection of participants include the following:
>> Attention should be given to clearly defining the required skills, education, experience,
job functions and gender of potential candidates in written course invitations or terms
of reference. It is important to set criteria that do not discriminate unfairly on the
grounds of sex, race or tribal origin. Detailed participant profiles make it easier for
partner organizations/governments to select the appropriate people. Characteristics
of suitable participants should be discussed with target organizations.
>> It is important to keep higher-level people (e.g. senior management, team leaders, key
community groups) well-informed and on board. As highlighted in the Evaluation of
Capacity Development Activities in Africa,10 there have been cases where an audience
has been selected without involving the structure to which they belonged, and this
seriously hampered the effectiveness of the learning and the ability/willingness of learners
to transfer the new expertise to members of the organization or the wider group.
>> Detailed admissions questionnaires will help identify potentially unsuitable candidates by
asking possible participants to describe their work and the challenges they believe the course
will help them address. This can also be done as part of a learning needs assessment.
>> Especially for long and resource-intensive initiatives, it is advisable to undertake follow-up
interviews by phone or Skype with identified candidates to help reconfirm the preliminary
choice made on the basis of applications. This also serves as an opportunity to explain
unclear aspects of the learning initiatives, such as the required time commitment.
>> When participants are selected well in advance of the course, it may be possible to ask
appointing officials for replacement options where advisable. In all cases, the selection process
should be as transparent as possible to avoid raising false expectations or claimed injustices.
>> With longer-term learning initiatives involving a series of courses, it may be possible
to use a preliminary online course to screen potentially unsuitable candidates. Only
those who complete and pass the first course would move on to the others.
29
BOX 2: FAO good practices – Identifying the target audience
1. The Policy Learning Programme has developed a questionnaire intended to gauge the background,
actual work responsibilities and motivation of course applicants in relation to the course objectives.
2. The Training of Trainers (TOT) for the Socio-economic and Gender Analysis (SEAGA) Programme
requires prospective participants to undertake some work in preparation for the workshop and provide
some evidence that they have previous experience in training and facilitation. A Technical Note about
the programme also specifies that participants are expected to prepare a follow-up training plan. Only
those who meet these selection criteria at a satisfactory level are admitted to the course. Criteria to select
participants are clearly indicated in the invitations.
3. For the TOT Programme on Collaborative Conflict Management for Enhanced National Forest
Programmes, the selection process is quite elaborate and consists of three steps. First, a general
announcement is shared with National Forest Programme (NFP) Focal Points to provide course
information and guidelines to identify suitable candidates. In order to propose their own candidates,
organizations commit to provide full institutional support to their nominees in using their learning, by
training others. As a second step, interested candidates complete a questionnaire with information
concerning their background and motivation. In the final step, potential candidates selected on the basis
of the questionnaire undergo a phone/skype interview with a team comprising Facility Coaches located in
the (sub) region, and FAO HQ. While this process is resource-intensive, it has ensured that the right people
are selected in order to achieve more effective and sustainable impact.
Supplementing official appointments: Despite rigorous selection procedures, countries may “appoint”
participants who do not have the profile desired by the course organizers. If the budget allows and if the
countries approve, additional people could be invited to attend the course as “resource people” rather
than participants.
An appropriate mix of participants: Mixing experienced participants with new, usually younger ones helps
to ensure that the knowledge and skills are transmitted to a younger generation of local/national experts.
This was the experience gained from a nine-week training in boat building in Bangladesh, sponsored
by FAO, the International Maritime Organization (IMO) and other partners (Fisheries and Aquaculture
Resources Use and Conservation Division (FIRO)).
30
a
31
Planning checklist
Discussion
Decision still N/A
taken required
32
STEP
3 Assess learning
needs
This chapter will look at ways to conduct learning needs assessments. It will also introduce
some discussion points on identifying and working with national partners.
>> What broad objective(s) is the learning initiative intended to address? How
do these objectives relate to participants’ work functions?
>> What skill and knowledge assets exist to help achieve organizational objectives? What skill
and knowledge gaps exist and what learning is needed to address those gaps? This should be
approached not only in terms of technical skill and knowledge gaps, but also soft/functional skills
that can support the use of technical capacities (e.g. managerial or project management skills).
>> What prior knowledge and experience do prospective participants have and
how is this likely to affect learning content and delivery methods?
>> What opportunities and obstacles exist regarding the use of learning in the work setting? How
may the learning activity be adapted to maximize opportunities and address these obstacles?
>> What sort of follow-up support is likely to be needed to support the implementation of learning?
In the day-to-day reality of FAO’s work, needs assessments are often a pro forma exercise: time
constraints, financial constraints or assumptions based on the assertion that “we already know what
needs to be addressed” are usually the main reasons for not conducting a formal needs assessment.
11 Using Training to Build Capacity for Development: An Evaluation of the World Bank’s Project-Based and WBI
Training; Evaluation of FAO’s Activities on Capacity Development in Africa http://www.fao.org/fileadmin/
user_upload/capacity_building/PC104-5EvaluationCapacityDevelopmentAfricaK8635E.pdf
34
Want to know more about how to select the
appropriate approach for a learning needs
assessment? Toolbox 3 contains tools for:
Where timeframes and budgets are very limited, learning needs assessments can be
done through written questionnaires to prospective participants and their managers
or community leaders as part of the application process; and/or targeted phone or
face-to-face consultations with key members of the participants’ workplaces.
Where timeframes and budget are more flexible, more in-depth methods – like focus
groups, pre-tests of participants or online forums – may be useful ways to ensure that
investments target actual needs and help build sustainable capacity to achieve goals.
Whatever desk reviews or expert consultations are conducted, a learning needs assessment
must involve direct input from prospective participants. This is for two reasons:
>> First, it is necessary to have an in-depth understanding of the objectives and existing
capacities of prospective participants, their organizations and the context in which
they operate, in order to ensure that the learning activity is adequately targeted to
their needs and objectives. External experts, even when they work on an ongoing
basis with target organizations, do not necessarily have an adequate understanding
of specific organizational conditions and the context in which individuals work.
>> Second, the process of conducting a learning needs assessment is important to build
internal support for the goals of the learning activity within target organizations and
to ensure greater client “ownership” of the capacity development process.
The “Pygmalion” effect is a concept in psychology that explains that people behave as they think others
expect them to behave. The experiments that led to the identification of this effect have some very
important lessons for conducting learning needs assessments. It suggests that people live up to their
perceptions of our expectations of how they will perform. In other words, they respond to the level of
confidence that interlocutors show in them.12
This means that the way FAO officers relate to, for example, indigenous peoples, local communities or
small-scale farmers, and the level of confidence they express in their potential will affect the findings of
the needs assessment. When the right approach and techniques are used (e.g. appreciative inquiry and
non‑directive communication style), people’s existing assets, skills, past achievements and unexpressed
potential are more likely to be uncovered. This represents the “capacity base” upon which the learning
initiative should be built.13
When sufficient time and resources are available, a needs assessment can be done through a learning
needs assessment workshop, which is particularly effective to support the development of learning
programmes and major curricula. In the workshop, the needs assessment is undertaken as a collaborative
process involving all major stakeholders, including some prospective target-audience members, subject-
matter experts from various sources (e.g. academia, national and international institutes), senior managers,
donors and supporters of the planned initiative. The advantages of undertaking a capacity development
needs assessment as a collaborative exercise is that it builds strong ownership, support and buy-in from
stakeholders and creates strong links with institutions, which are usually very proactive in using and
adapting the learning material, once produced, for their own needs.
Professional facilitators lead these workshops, with the support of instructional designers and learning
specialists. Workshop participants provide their perspectives and expertise to identify:
The outputs of the workshops include an outline for the learning programme and an initial identification of
potential participants and institutions that would most benefit from it. FAO has undertaken two capacity
needs assessment workshops with the Common Market for Eastern and Southern Africa (COMESA) and
the Permanent Inter-State Committee for Drought Control in the Sahel (CILSS) as a first step in setting up
a learning programme to support the New Partnership for Africa’s Development (NEPAD) Comprehensive
Africa Agriculture Development Programme (CAADP) Pillar III Framework for African Food Security. The
resulting learning programme is targeted at CAADP country actors and focuses on the knowledge and
skills in food security required to support the design and implementation of country agriculture and
food security investment plans. (Technical Cooperation Department and Office of Knowledge Exchange,
Research and Extension) (TCFS, OEK).
In addition, a workshop including national stakeholders in phytosanitary capacity worked through a formal
guide or checklist, the Phytosanitary Capacity Evaluation, which has been used to date in over 50 countries
(AGP).
36
>> the learning initiative addresses real and urgent problems which the local people strongly
care about, and that it mainly relies upon local expertise (i.e. local trainers, coaches, experts
or other resource people). Local people are more likely to understand the specific needs,
context and culture of the learning event participants and are more likely to understand
what material is relevant. Also, supporting the development of local experts’ individual
capacities contributes to strengthening national and local capacity as a whole;
>> appropriate institutional partners14 are identified and involved, which can, with
the right support, continue and possibly scale up the initiative; and
>> appropriate measures are taken to support longer-term sustainability of the initiative.
No matter how successful the learning initiative has been, it will have no long-term impact on national
capacities if it is not anchored in a national system that can sustain it after donor support has ended.
Identifying partners
There are no general guidelines to help with identifying national partners in learning
initiatives; what makes a partner “appropriate” depends on the local context, the type of
initiative and the subject-matter area. However, these are some key considerations:15
>> Knowledge and expertise in the subject-matter area and familiarity with group moderation
and facilitation: What type of expertise do they have in the technical area? Are they familiar
with participatory facilitation techniques and/or do they have strong potential to learn?
>> Motivation and incentive to operate: What is their motivation to take on additional roles
and responsibilities to sustain the initiative once FAO’s funding is over? What is driving their
engagement? Financial gains? Desire to promote change? Prospects for individual career shifts?
>> Ability to implement and deliver: Are the infrastructure and processes in place to implement
and deliver? If not, can these be supported with cost-effective efforts, given the goal?
>> Access to and credibility with a wide range of local and national stakeholders,
including civil society and small rural organizations: Is the partner a credible
interlocutor for a wide variety of stakeholders? Does the partner speak the local
language and does it have cultural understanding of the context?
>> Anchored in national structures: What type of linkages into national structures
and systems exist, and/or can such linkages be supported and developed?
>> Ability to relate, partner and mobilize resources: What is the partner’s potential
ability and readiness to actively seek new partnerships and attract new funding to
secure longer-term sustainability after FAO has exited from the programme?
14 The Evaluation of Capacity Development Activities in Africa found that “back up support from either an extension service or
implementing agency is often needed in order to ensure that capacities once developed are institutionalized. There is a need
to ground the projects in the agencies that will be in a position to provide post-project support” (paragraph 94, main report).
15 Paragraph 97 of the main evaluation report states, “In quite a few cases, stakeholders criticized FAO for having
followed a standardized approach to the selection of implementers and facilitators, thus not always making the best
choice or creating the best fit with local needs”. Some generic guidelines for engaging with national partners can
be found on the LenCD website ‘How to establish partnership for scale up’, which refers to the example of an FAO
Farmer Fields School. http://www.lencd.org/group/learning-package/document/how-establish-partnerships-scale
37
Building sustainability
FAO has supported the integration of promising approaches into national systems; a
number of examples are described in Box 4. The example of the farmer field schools
in Mali suggests that the following factors may provide some inspiration for how
a learning initiative can become self-sustaining after FAO’s support ends:
-xox-
Involve government structures and the private sector. The project “Strengthening the agricultural
extension system through agro-enterprise development” (see Box 1) adopted a two-fold approach to
ensure sustainability. On the one hand, the project works through government institutional structures at
central, district and local levels by establishing a core team of trainers drawn from different offices in
different divisions of the Department of Agriculture, Department of Export Agriculture and extension
training institutions. On the other hand, the project involves private-sector buyers in developing training
curricula, delivering training and providing back-stop support. In a number of cases, the private sector
implemented the training through their own extension service providers. This approach is expected to
create linkages between producer organizations and private-sector buyers, which will lead to increased
sustainability. (AGS)
Work through national education systems. The Global Plant Clinics initiative, sponsored by CAB
international (CABI), offers expert diagnostic service for any plant and type of problem in various parts of
Africa. It also trains “plant doctors” and scientists, supports plant health clinics – often held in local markets
or other public places where farmers can bring affected plant parts for diagnosis – and links extension,
research, regulation and input supply. In order to ensure that this approach makes a durable contribution
to plant health services, one strategy envisions crediting the time spent working in plant clinics as a
contribution towards degree qualification for general agronomy undergraduates. This strategy – which
would require that university authorities support a change to the curriculum – would benefit the local
community, which would continue to benefit from plant clinic services, while agronomy undergraduates
would have unprecedented exposure to real farmers’ needs. (AGP)
38
-xox-
Support a national trainer/facilitator pool as part of existing structures. In Cambodia, a National Trainer
Pool for Food Security and Nutrition was established in 2008 by the Council for Agricultural and Rural
Development of the Council of Ministers (CARD), with the support of the FAO-Netherlands Partnership
Programme (FNPP) on Food Security Poverty Reduction. Trainings can be requested by all government line
ministries, development partners, NGOs, civil society and private-sector organizations that wish to locally
organize a course on food security and nutrition either in the capital, provincial cities or at the village level.
The initiative is now fully owned by the national authorities who, through CARD, select and nominate
trainers, assist trainers in improving their capacities, manage trainings and advise client organizations,
monitor the quality of the curricula developed, manage funds, etc. Through the food security website
of the Government of Cambodia <http://www.foodsecurity.gov.kh>, CARD provides information to the
public about the Pool and trainings on a regular basis.
Work at national and regional levels simultaneously. While national training institutions are the main target
of the FAO’s TOT programmes on Enhancing Stakeholder Participation in National Forest Programmes
and Collaborative Conflict Management for Enhanced National Forest Programmes, FAO works
simultaneously with regional institutions (e.g. Tropical Agricultural Center for Research and Education
(CATIE) in Latin America; Regional Community Forestry Training Center for Asia and the Pacific (RECOFTC)
in Asia; African Network for Agriculture, Agroforestry and Natural Resources Education (ANAFE) and
African Centre for the Constructive Resolution of Disputes (ACCORD) in Africa to strengthen their ability
to support national training institutes. At the initial stage, FAO and the regional organizations train the
participants from national institutions, then provide joint ongoing mentoring assistance to build a common
approach. At later stages, FAO gradually withdraws and limits its role to backstopping when required,
leaving the regional organization to provide the support. (Forest Economics, Policy and Products Division
(FOEP).
Establish a national coordination unit. In Mali, the Integrated Production and Pest Management (IPPM)
Programme derives a growing proportion of its financing – currently 30 percent – from new projects that
arrived after the programme was in place. An effective FFS national administrative team attracts interest
and support from other projects that can benefit from access to experienced farmer facilitators and active
and receptive farmer groups. From this observation, a strategy emerged to develop an FFS “national
coordination unit” (NCU) in each country that will act as a “service provider” to coordinate new projects
with donors who wish to work at the community level, but who otherwise lack the access to networks
or the means to administer highly decentralized activities. The NCU provides staff highly experienced
in discovery-based training and FFS national programme execution. It offers to new partners a host of
services, from curriculum development to monitoring and evaluation. Most importantly, it facilitates access
to existing networks of trained facilitators and active farmers’ groups already familiar with a farmer-based
experimental approach to testing and adapting new ideas. In return, donors and potential partners would
finance the bulk of the training for the targeted cropping system on which the partner focuses. In this way,
everyone benefits: donors can dedicate more resources to the field and less to project administration;
governments have better oversight and alignment of capacity development because new donors will
not have to “reinvent the wheel”; district-level government and farmers’ organizations will have more
consistent access to resources for capacity development and communities will have a greater sense of
consistent support and follow through from technical support services. (Source: The West African Regional
Integrated Production and Pest Management Programme, a case study, AGP)
39
3.4 Developing a concept note
It may be useful at this point to develop a concept note for your learning initiative to share
with internal and external stakeholders in order to fundraise and build internal/external buy-
in. A concept note is particularly useful for identifying possible synergies with other internal
or external initiatives, so that necessary linkages can be built on time in the design phase.
>> the capacity issue and how the learning initiative will address it;
>> broad learning goals;
>> expected results and result indicators, and how the evaluation will be conducted;
>> design, delivery mode and follow-up plan;
>> relationship to other relevant learning initiatives, if any;
-xox-
40
-xox-
Planning checklist
Discussion
Decision still N/A
taken required
41
STEP
4 Design content and
select delivery modes
Whether you are designing the learning initiative or supporting and coordinating others to
do it, this chapter will help you understand how to select delivery modes, develop learning
content and activities and prepare for the delivery. Where possible, seek the support of a
professional Instructional Designer to ensure the quality and coherence of the initiative.
>> What do you want your target audience to be able to remember, understand or do differently?
>> What skills or competencies do you want them to learn, develop,
expand or improve, to support them in their jobs?
>> In what type of learning do you want them to engage (e.g. recalling definitions, applying
abstract concepts to reality, applying procedures, solving problems or something else)?
Addressing these questions will help you design good learning objectives and also:
To develop specific learning objectives that match real learners’ needs, it is useful to undertake a task
analysis. A task analysis is a detailed analysis of the actions and decisions that potential learners take to
perform their usual job tasks; it includes identifying the knowledge and skills needed to support those
actions and decisions. The results of the task analysis will help you identify, prioritize and organize the
contents on the basis of what learners really need, rather than on the basis of what subject- matter
experts believe is important. You can access a template for a task analysis from the Toolbox.
-xox-
Learning objectives should be specific, measurable, achievable and realistic. They should
consider the time available and be congruent with the goals of the learners’ organizations.
43
Learning objectives should not be used to describe the content to be covered, but to describe the
intended results of the initiative. Well-formulated learning objectives focus on intended outcomes
and reference the ways in which participants are expected to apply learning at work or in life.
Action verbs must be used when formulating objectives, because they imply an action that can
be observed and assessed later on. Generic verbs such as ‘learn’, ‘understand’, ‘be aware of’,
‘appreciate’ or ‘become familiar with’ should be avoided, as they are difficult to observe.
For example, the chart below shows bad and better ways to complete this objective
statement: “At the end of this programme/this session/this unit, you will be able to...”.
Bad formulation Better formulation How can you assess learning results?
..understand the concept ..define the concept of By asking learners to define the concept in
of Right to Adequate Food Right to Adequate Food their own words as applied in their context
..become familiar with ..incorporate impact By asking learners to create action plans
the steps of impact assessment into describing how they intend to go about
assessment programme design incorporating impact assessment in their
programmes
..learn how to design ..outline some basic By asking learners to apply principles of good
qualitative assessment principles in designing a qualitative assessment to their own project.
qualitative assessment
Objectives can be clustered logically around areas, themes or levels, so that you
define a sort of hierarchy among learning objectives. The areas, themes or levels
will correspond to the main topics or sessions of your learning initiative.
-xox-
Objectives should be set not only for courses, workshops or distant learning
material; you should also set objectives with the participants when you are
coaching or facilitating a job-shadowing activity, for example.
44
4.2 Define delivery mode and learning method
By the time you have completed the LNA and defined learning objectives, you will
have an idea of the delivery mode which best suits your learning initiative.
-xox-
In deciding which mode makes most sense for your learning activity, consider the following questions:
45
>> Is it important that participants learn new practices and skills while remaining in their local
context (because the context will greatly affect the implementation of such practices)?
>> Do target participants’ time schedules and geographic locations
enable online collaborative synchronous learning?
>> How important is it that participants get exposed to practices used outside
of their own country/region as part of their learning process?
Classroom: In-person, real-time courses, workshops and seminars delivered via an instructor or trainer
Strengths Challenges
>> Easier to keep participants >> More costly and logistically difficult to gather
motivated and engaged participants in one location; high costs may
>> Easier to facilitate discussion cause courses to be shorter in duration than
and peer-to-peer learning would be optimal to achieve training objectives
>> Relatively easy to integrate in a project cycle >> Participants are taken out of the work
>> Relatively easy to report outputs environments where they will be using new
>> Helps participants create networks and skills, which can, at times, be difficult to adopt
provides exposure that contributes to in their working environment, discouraging
building their confidence and leadership participants of less developed countries from
implementing the new skills (see example below)
>> Not appropriate for skills best
learned gradually over time
>> Wrong participants often attend
FAO example:
Pest risk analysis (PRA) training in the Caribbean: PRA is the process of evaluating biological or other
scientific and economic evidence to determine whether a pest should be regulated and to determine the
strength of any phytosanitary measures to be taken against it. Training was organized for the Caribbean
Commission through collaboration with a United States university; however, when trainees returned to their
organizations with requests for the equipment to do effective PRA, the requests were denied. This discouraged
participants from adopting the methods in which they had been trained. (Source: AGP)
46
On-the-job learning: Mentoring, coaching, job shadowing or study tours delivered via a
coach, mentor, facilitator or advisor in a work setting, either face-to-face or online
Strengths Challenges
>> Can be planned as formal ad hoc events or >> More difficult to report outputs
informally as part of ongoing technical work >> Difficult to separate from ongoing technical work
>> Enables learners to practise skills as >> Success depends on individual skills and
they are being learned in the setting techniques of facilitator, coach, advisor or mentor
where they will be used and to address >> It is essential that the mentor/coach is very
implementation problems as they arise familiar with the work context of the learner
>> Relatively lower cost than other modes
>> Learners can actively shape the situation
to their needs and the coach/mentor or
advisor can change methods, timing and
informational priorities on the spot
>> Learning can be paced over a long
period of time when necessary
FAO example:
Informal coaching of researchers/scientists in Uganda: As FAO was working with researchers and
laboratories on projects to control the spread of banana wilt disease, researchers began taking on the farmers’
perspective in research, understanding their valid observations about the disease and listening to their needs.
The FFS provided a meeting place for this. As a result, the banana research teams are now extremely proactive
in seeking out farmers who are fighting the disease and engaging with the community through radio phone,
public meetings and visits with extension workers. (Source: AGP)
Study tour from the Democratic Republic of the Congo (DRC) to Uganda: Study tours were used to take
representatives of local authorities, NGOs, community leaders and farmers from Goma in DRC to Uganda to
learn how the Ugandans had managed to succeed in controlling banana wilt (see above). One individual had
visited Uganda previously, but upon returning, he was unable to get his supervisors to adopt what he had seen.
A small amount of additional resources was used to finance a group of 16 and a facilitator-translator to visit
ministry, research and farm sites and discuss with Ugandan colleagues the challenges and issues they faced
in controlling banana wilt. One participant (a priest) financed his own trip because he felt it was so important
to learn about controlling the disease in the interests of his congregation. The visit has created a team spirit
among those who travelled, and there seems to be some signs for cautious optimism about local efforts to
reduce the impact of this disease. (Source: AGP)
47
Self-study (asynchronous): Self-paced e-learning,16 or self-study through television or radio
broadcasts, through the Internet, CD-ROMs, tapes, printed materials or other media
Strengths Challenges
>> Enables distribution of learning materials >> Requires learners to have a high
to learners who don’t have access degree of self-motivation
to computers or the Internet >> Only allows one-way flow of information;
>> Enables learners to work at their own pace and no opportunity for peer learning,
at convenient times, which is particularly useful feedback, questions or discussion
when learners are in different time zones >> In the case of e-learning, high start-up
>> Learning content can be adapted, costs for course development, but then
to some extent, to the needs and can be very cost-effective for training
circumstances of participants (e.g. large numbers of participants
through personal learning paths)
FAO example:
Self-paced e-learning: A series of self-paced e-learning courses have been developed as part of the EC-FAO
Programme on Linking Information and Decision-making to Improve Food Security. The courses are targeted
to food security professionals working in FAO Member States and cover a wide range of food security subjects.
At the end of 2011, over 67,000 users had registered for an online course or had ordered a CD. An evaluation
undertaken in 2009 noted that 88 percent of users surveyed noted that they had gained new knowledge and
skills from the courses which they were able to use in their work; 99 percent of respondents liked the learning
style offered by self-paced e-learning; 87 percent recommended the courses to others; 77 percent noted that
the courses improved the collection, management, analysis and/or dissemination of food security information
to key stakeholders, which is one of the major objectives of the EC-FAO Programme. The survey also noted
that more than 75 percent of users had ordered a CD version of the course, which demonstrated that
bandwidth issues and access to the Internet must always be considered when developing e-learning. Features
that users liked most were the use of practical examples, case studies, exercises and interactivity in the courses,
as well as the fact that the courses can be done at the users’ convenience and speed. (Source: Agricultural
Development Economics Division and Office of Knowledge Exchange, Research and Extension - ESA/OEK)
16 FAO has recently developed a methodological guide for e-learning to set the standards and provide
guidelines to technical staff in developing their own e-learning resources: Ghirardini, B. E-learning
methodologies: A guide for designing and developing e-learning courses, FAO, 2011.
48
Online collaborative (synchronous): Video or audio conference, or moderated, real-time Web-
based learning in which participants can interact with a moderator and with each other
Strengths Challenges
>> Suitable when there are budget, time >> Cost/logistical difficulty depends on
or geographical constraints availability of needed equipment
>> May be easier to attend for higher-ranking and cost of communications
officials or others who cannot take time off from >> Online facilitation requires specific set of skills
work duties to attend a face-to-face course to keep participants motivated and engaged
FAO example:
Virtual workshops: The Office on Knowledge Exchange, Research and Extension (OEK) offered a virtual
workshop entitled “Introduction to Knowledge Sharing Networks and Communities” to FAO staff and partners.
The workshop introduced participants to networks and communities and how they facilitate knowledge and
information exchange. The workshop took place online using an e-learning space to which all participants had
access. It was conducted with both asynchronous (not occurring at the same time) and synchronous (occurring
at the same time) tools. The asynchronous tool, called Moodle, provided a space which contained all the
learning materials and instructions. It was also a shared space where discussions took place and participants
uploaded their assignments. Once a week, all participants connected with each other and with the facilitator
using Skype and a telephone bridge line. (Source: Office of Knowledge Exchange, Research and Extension -
Knowledge Management & Library Services (OEKM), http://www.fao.org/knowledge)
Video conferences on gender: The Gender, Equity and Rural Employment Division (ESW) has often resorted
to video conferences for gender training in Member Countries. As with face-to-face meetings, ESW emailed
needs assessment questionnaires to participants prior to the event, so as to focus discussions on a set of
identified issues. Video conferences or virtual meetings, though, require skilled facilitation. A comparison
between face-to-face and virtual facilitation tasks is included in the Toolbox (Tool 5k). (Source: ESW)
Using a blend of delivery methods can often mitigate the challenges of any single delivery
mode and address learning objectives effectively. Blended learning can take many forms.
-xox-
Blending online and face-to-face events: An online pre-class event can be used to bring
learners with different levels of knowledge and skills to the same level before the face-to-face
class begins. Another approach can be to start with a core classroom event, followed by online
independent experiences which can include, for example, interaction with online resources
49
or e-mentoring services for continuous reinforcement. This could include the development
of communities of learners or further discussions on advanced topics of individual interest.
Online events can also be used to introduce and conclude a blended learning programme. For
example, they could be used to assess participants’ knowledge prior to and after a course.17
Blending face-to-face with on-the-job events: On-the-job learning support, such as coaching,
mentoring, job shadowing or study tours, can be used after face-to-face events to support learners
in implementing what they learned in the classroom. Mentoring and coaching can be provided
through individual one-on-one or small group face-to-face components. On-the-job support
might be seen here as providing the “missing link” between skills that can be developed through
training and implementation because it helps address “some of the primary blocks to change
such as self-belief, action orientation and risk taking”.18 Coaching and mentoring is particularly
useful when part of training-the-trainers programmes (see Box 5 below). In addition, study tours
can be arranged to follow a more formal training course and enrich the learning process.
-xox-
E-learning + classroom training: A blended approach was used successfully in planning community-
based adaptation (CBA) to climate change, targeted to rural agricultural extension workers. Bringing
e-learning to rural communities might be challenging if it is not carefully integrated with other learning
processes. In this case, e-learning came in after social mobilization and good practice identification had
been started, and it was followed by field demonstrations in a clearly communicated and understood
process. In this way, everyone was able to situate e-learning clearly with respect to their own situations and
could clearly define next steps for putting learning into practice.
Typically, the format consisted of two to three days of training which included both e-learning and oral
presentations of country-specific examples. The e-learning was undertaken through guided individual
self-study sessions in “e-labs”; with large groups, e-learning would be projected on the main screen of
the classroom and some devices were used to take advantage of the interactive elements of the tool (e.g.
multi-coloured cards for exercises or an audio/video system to screen short videos in the e-learning tool).
The number of participants varied from 15–30 (when using the e-lab) and from 10-–200 in the classroom).
The advantages of combining e-learning with face-to-face training (as compared with using e-learning or
face-to-face alone) are:
>> Face-to-face allows better tailoring and ad-hoc adjustment of training content to the specific
context (e.g. pre-knowledge of participants, country context, direct response to participants’
individual questions, embedding the content in the current status of related project activities);
>> E-Learning allows participants to refer back to sessions of particular interest after the course;
>> There is the possibility of copying the CD-ROM and disseminating it to others;
>> The younger generation of extension workers are fascinated by e-learning as a new
technology and are very eager to use it and improve their capabilities to use computers.
The introduction of this new technology through face-to-face also includes older, often
computer-illiterate extension workers on board who would have been excluded otherwise.
17 Ghirardini, B. E-learning methodologies. A guide for designing and developing e-learning courses, FAO, 2011, p.19.
18 Coaching federation research portal, http://coachfederation.org/articles/
index.cfm?action=view&articleID=851&filename=133-dyadic-learning.pdf
50
-xox-
Classroom training + mentoring: The training programme on Collaborative Conflict Management for
enhanced national forest programmes has a well-developed component of mentoring which is carried out
at specific stages of the learning process, i.e. when the learners are back in their respective countries and
start using the skills they have learned by engaging in the organization, facilitation and management of a
national level training. The aim of mentoring is to support them in the design and delivery of successful
conflict management trainings. Mentoring activities include:
>> Distance mentoring through e-mail and phone: This occurs while learners are
involved with planning and preparing for their in-country training. This includes
progress review according to an agreed action plan, revision and clarification
of tasks, exploration of problems and review of key learning points.
>> Direct mentoring as part of preparing the training: This takes place in person throughout
the three days preceding the training. The objective is to support learners to relate
their own session to the other sessions and the overall objectives of the training.
>> Direct mentoring during the in-country training: The mentor is physically present during
the training, providing psychological support, addressing challenging questions, filling
gaps and contributing additional comments. At the end of each training day, this includes
an hour of supported self-reflection on the content and the process of the day.
>> Direct mentoring following the training: This consists of a private, one-on-one session
with each mentee to look into the individual learning resulting from preparing and
conducting the in-country training. This session is also used to pick up issues for
future follow-up support and motivation for conducting future trainings.
>> Final assessment. At the very end, the mentor reflects his/her observations
and feedback about the learners’ performance in a trainer assessment tool
which goes to the programme coordinators. (Source: FOEP)
Study tours + coaching + in class training: In Togo, an incremental approach to learning for investment
planning was adopted in the context of support for the implementation of the Comprehensive Africa
Agriculture Development Programme (CAADP), whereby learning needs were initially addressed in a
crash face-to-face training course addressing basic competencies and knowledge on core themes such as
Sector Wide Approaches, Medium Term Plan development and monitoring and evaluation. As participants
proceeded in the development of the first agriculture investment plan, complementary on-the-job learning
support was provided. Subsequently, study tours opportunities were arranged to complete and enrich the
learning process, allowing Togolese representatives from government and producer organizations to learn
from their peers in Mali about the process of introducing a sector wide approach. (Source: Investment
Centre Division (TCI))
51
4.3 Organize learning contents
How do you move from designing learning objectives to designing your initiative? There
are several aspects that must be taken into account and it may be useful to approach this
step by creating a design outline. In the design outline, you succinctly describe:
-xox-
1. the broad goals of the initiative and the delivery mode that has been selected;
3. the content you plan to include, expressed in terms of key topic headings;
4. the resources you already have, those that must be adapted and those that you must identify or create;
Identifying learning content implies taking decisions on what type and depth of content
should be presented as well as on how it should be presented and sequenced.
Obviously, content design and development considerations will change depending on the delivery
mode. You may have to prepare a lot of written material for a course or a workshop, but only
draft a ‘lesson plan’ for yourself if you are preparing a job shadowing or a coaching session.
52
Table 5 presents various types of content.19
Unique, specific information that answers the questions: who? where? when?.
They are shown, exhibited and indicated.
Fact
Examples: data, lists, historical events
A group of objects, entities or ideas that: are defined by a single word or term;
share common characteristics; differ in unimportant characteristics; require a
Concept definition; and answer the question: “What is a …?”.
A flow of events that describes how something functions. They are events that
involve more than one actor. They can be divided into phases.
Process
Example: “the assessment process”
Predispositions to behaviour
Attitudes Example: content related to appreciating the “importance and urgency of
adopting measures to limit the negative impacts of climate change”
[Table 5] Types of content
It is important to find the right balance between quantity and depth of content. Learning
initiatives are not opportunities to showcase all the knowledge that is available in a given
discipline! To have an impact, content must be essential to the learning objectives that
have been identified and must be directly relevant to the organizational goals.
It may be useful to draw a distinction between content that is “nice to have” and
content that is essential because it directly supports the work/tasks that participants
must perform in their own settings to help achieve organizational goals.
Relates directly to the work or the tasks that Provides additional details that might be interesting,
individuals must perform to achieve their own yet are not supportive in attaining the learning
objectives towards organizational goals objectives
[Table 6] Table 6
19 Ghirardini, B. E-learning methodologies Guide, freely adapted by the author from Morrison G.R., Ross
S.M., Kemp J.E., Designing Effective Instruction, Wiley & Sons, Inc., Third Edition 2001.
53
Finding the right mix of technical and soft content
Most of FAO’s learning initiatives aim to enhance knowledge, skills and attitudes of key
actors in the areas of its specialization (e.g. food security, agriculture). To be sustainable,
however, learning in technical areas has to be complemented by development of
functional skills that allow a sustainable uptake of technical learning in Member Countries.
For example, individual capacities in project management, partnering and networking,
knowledge management, negotiation and advocacy and policy processes are often
essential and should be strengthened alongside other more specific capacities.20
-xox-
The e-learning course, entitled “Communicating for Food Security” (part of the e-learning curriculum
entitled “Food Security Information for Decision-making”) targets mid-level managers, technical staff
and field personnel who are involved in collecting, managing, analysing and reporting food security
information. It provides concrete guidance on how to inform and influence policy-makers’ decisions. It
also covers ‘soft skill areas’ such as lobbying and advocacy techniques for changing a food security policy.
Available on the FAO CD portal at: http://www.fao.org/capacitydevelopment/e-learning-services/en/
Once you have defined the content, the learning initiative needs to be appropriately structured.
The educational psychologist Robert M. Gagné suggests nine distinct “events” that make learning
successful and can easily be adapted to all types of learning initiatives. These are illustrated in the arrow
steps of the diagram below. Examples of how to address each “event” are provided in the Toolbox.
-xiagram-
Simulate
Inform Provide Enhance
Gain recall Present Elicit Provide Assess
learner of guidance retention
attention of prior the material performance feedback performance
Objective for learning and transfer
knowledge
1. 2. 3. 4. 5. 6. 7. 8. 9.
54
Of the “events” shown in the diagram above, eliciting performance and reinforcing practise is of
paramount importance. Research shows that learning composed entirely of frontal lectures and/
or PowerPoint presentations (PPT) is unlikely to be effective in facilitating learning and use of
learned skills, knowledge and attitudes after the course. Instead, participants should be given ample
opportunity to practise learned skills in the course, discuss their own perspectives, set goals for using
the learning and consider how learning relates to their specific workplace or country environments.
>> Case studies to practise technical skills enable learners to see how
theoretical learning can be applied in real-life situations.
>> Drawing up action plans enables participants to work through
how they might apply learning in their own contexts.
>> Guided discussions enable participants to learn through actively engaging
with materials, and generally aid in absorption of knowledge.
>> Class discussions based on participant presentations of case studies from their own
contexts enable participants to learn from each other’s experience and trainers
to gain a better understanding of participants’ needs and constraints.
>> In-class project assignments enable participants to gain hands-on experience with learned
materials and to receive feedback from trainers on their understanding of content.
55
-xox-
Case studies: Case studies are student-centred activities based on topics that demonstrate theoretical
concepts in an applied setting. Case studies are useful pedagogical tools that shift the emphasis
from teacher-centred to more participatory and student-centred activities. As such, they are highly
recommended to increase interactivity in workshops or other face-to-face settings. How to go about
developing case studies?
Typically, development of case studies is outsourced to consultants prior to training workshops. In the EU-
funded “All ACP Agricultural Commodities Project”, an activity designed to better understand successful
value chain development in the Pacific, the Trade and Markets Division (EST) piloted an alternative
approach in which consultants were paired with ministry staff in Member Countries to work together on
developing case studies. This approach combined methodological expertise and local experience and
resulted in increased development of national capacities as part of the workshop preparation process.
(Source: EST)
Simulations: Since the emergence of highly pathogenic avian influenza (HPAI/H5N1) in Asia, FAO and its
partners have developed national table-top simulation exercises (SE) to strengthen in-country capacities
in emergency preparedness and response for the prevention and control of zoonotic diseases at the
human-animal-environment interface. SE consists of two days of room-based events bringing together
professionals from different sectors involved in addressing emerging outbreaks in human and animal
populations (e.g. central, provincial and district levels from the Ministry of Agriculture and Ministry of
Health, veterinary and public health epidemiologists, laboratory diagnosticians, private service providers,
communication experts, environment and wildlife services, civil defense, farmers’ associations). The
simulation is based on a scenario describing a fictive outbreak, from its suspicion to its control. Participants
address questions covering the different activities which would be carried out in response to the different
phases of the evolving disease situation.
In Mauritania, where the exercise was conducted in June 2010, a large map was placed in the middle of the
room to enable better visualization of the outbreak areas and to illustrate the extent of the implemented
measures. The elicited discussions around the map were generally perceived as very useful, because
they triggered more lively exchanges and involvement among participants. On two occasions, mock
press conferences were held to force participants, who played the role of the authorities, to respond to
challenging questions posed by the organizers and fellow participants taking the roles of critical journalists.
Through the simulation exercise, the country’s preparedness and capabilities to control the emergence
of a zoonotic disease are assessed. The identified gaps are used to enhance the country’s contingency
plan and to develop a national action plan to improve the country’s preparedness and response capacity
to prevent and control zoonotic threats. A by-product is also that different participants establish channels
among themselves to discover the strengths and weaknesses of the plan and the procedures in place to
address a specific emergency situation. The approach also enhances communication, cooperation and
collaboration among the different sectors that would be involved in case of an emergency situation. The
project was developed jointly with the World Health Organization and funded by USAID. (Source: Animal
Production and Health Division (AGAH))
56
Supporting ‘learning by doing’
Experiential learning, illustrated here through the well-known FFS approach, is a good
example of a learning process that focuses on reinforcing learners’ practise via a ‘learning
by doing’ approach. This approach also proves to contribute to changes in attitude.
The experiential learning cycle is especially useful for practical skill training because most of its
techniques are designed to involve the learners in practising the skill. The experiential model
helps people assume responsibility for their own learning because it asks them to reflect on their
experience, draw conclusions and identify applications. Most importantly, learning occurs in the
farmer’s own local environment where farmers jointly diagnose problems, arrive at possible solutions
and commit themselves to take action – with extension staff playing the role of facilitator.
Started in the mid-1980s in integrated pest management on rice in Indonesia, the FFS
concept is increasingly being used to guide FAO’s approach in support of learning
across technical sectors. Examples of adaptations of the FFS concept are the Junior
Farmer Field Schools (ESW), FFS for Sustainable Land Management (Land and Water
Division (NRL)), Conservation Agriculture and Farmer Business Schools (AGS).
-xox-
A Farmer Field School (FFS) consists of 25–30 farmers who meet once a week for an entire crop growing
season. It is facilitated by extension workers or skilled farmers. Employing non-formal education methods,
the field is used as the primary resource for discovery-based learning. The process is facilitative and
respects the experience that farmers bring with them. Farmers work in small groups to ensure that
everyone’s ideas are shared. The activities are designed to respond to the immediate needs of farmers
and are geared towards encouraging creativity and independence. The FFS trainers play a crucial role in
ensuring that the environment and all resources contribute to the farmers’ learning experiences. (Source:
http://www.vegetableipmasia.org/Concepts/FarmerFieldSchools1.html
FAO has recently taken the FFS concept a stage further by developing the Farm Business School (FBS)
concept for interested graduates of the FFS School or members of established farmers’ associations or
groups. The approach uses some of the principles of the farmer field school – learning by doing and
gradual skills development over time. The FBS training programme covers an entire farm business cycle
from diagnosis and planning to implementation, evaluation and re-planning. (Source: AGS - The Farm
Business School)
57
4.4 Define Follow-up support
The design phase does not stop with the actual development of content and choice
of learning methods. It is important that you also consider which sort of follow-up is
necessary and feasible to reinforce learning and support learners. What budget, time
and resources can and should be made available to provide follow-up support?
-xox-
It is also important to start defining at this stage how you will evaluate your
programme or event. Whether it is e-learning or a workshop, a one-off event or
a multi-year programme, an evaluation plan should be set to verify that:
>> the process was effective, i.e. that the steps in the learning cycle management were up to
standard. This can be assessed as the initiative is progressing, which would allow you to
refine or change it to make it more effective and/or improve future learning activities; and
>> the initiative is producing the expected results. This allows you to refine
the design and delivery strategies or the content approach of the initiative
or to plan more effectively for future follow-up support.
More in-depth discussion on follow-up and evaluation are provided in chapters 6 and 7.
58
4.5 Define logistics for delivering
face-to-face learning
A checklist to guide you in preparing for face-to-face learning events is found in Tool 8d
in Toolbox 8. This checklist, useful for trainers, facilitators, coordinators and managers
of learning initiatives, should be used while planning and delivering an event.
-xox-
At this stage, you should have a clear picture in your mind of how your learning initiative will look. There
are also important logistical decisions that need to be taken, such as the following:
>> Who will deliver the initiative? Have trainers, coaches or facilitators been identified?
>> Have you ensured that trainers, coaches and/or facilitators have the appropriate competencies?
>> Have trainers, facilitators or coaches been familiarized with the goals
of the initiative and the learning content material?
>> Do contracts or Letters of Agreement need to be stipulated? Who will do it?
>> When will the initiative be piloted? Where?
>> Are logistics for the event confirmed (e.g. venue, air tickets, accommodations)?
>> Have all partners and relevant counterparts been informed?
-xox-
59
Planning checklist
Discussion
Decision still N/A
taken required
2. Has the optimal delivery mode been identified and have the
learners’ context and sustainability considerations been taken
into account in the decision?
8. Who will deliver the initiative, when and where? Have Letters
of Agreement or contracts been stipulated and have all efforts
been made to involve national/local resources?
60
STEP
5 Deliver the learning
solution
Effective communication, including inquiry and Presenting technical content clearly, adapting to
advocacy varying audience needs and learning styles
Facilitating group processes through participatory Facilitating learning needs analysis in technical areas
approaches
-Diagram 6-
[Diagram 6] Facilitative roles
62
The facilitator is a neutral third party, acceptable The advisor is used for his/her expertise in a
to all participants in an initiative, who has little or particular area. The advisor is a third-party expert
no knowledge of the subject matter and who has whose purpose is to help the client make informed
no substantive decision-making authority. His/her decisions. He/she does so by applying the area
role is to help a group increase its effectiveness by of expertise to the client’s particular situation,
diagnosing and intervening in group processes and recommending a course of action and, in some
structure. cases, implementing it for the client.
FAO employs internal and external facilitators when An advisor has some process-related skills and, in
delivering learning initiatives in its own areas of the FAO context, is often the same person as the
expertise. Because of their content-neutral stance, Technical Officer. He/she provides technical advice
facilitators have to work always in collaboration with to partners and is often called in as a resource for
subject-matter experts. on-the-job training as part of his/her regular work in
Member Countries.
The coach usually works one-on-one with people, The trainer is an expert in a particular domain and
helping them improve their effectiveness, although has responsibility for teaching a particular topic. He/
coaching may also be done in small groups. The she usually has good facilitative skills and knowledge
technical coach brings subject-area expertise in the of participatory techniques, which are used to
coaching relationship, but the heart of the coaching enhance participants’ learning experience.
role is helping individuals reflect on their behaviour,
explore new possibilities and find solutions for In FAO, trainers are the most common role. They
themselves. A coach jointly designs the learning combine an adequate knowledge of process with
process with the client instead of assuming that he/ content expertise. The role of trainer is associated
she knows how the client can best learn. almost exclusively with face-to-face workshops.
Facilitators, technical coaches, trainers and advisors are very important roles in FAO’s
activities in support of learning in Member Countries. The distinctions among them, however,
are only theoretical because these roles are often combined in the same individuals. The
advisor and the coach, for example, may be the same person, where the advisor adopts
appropriate coaching approaches and techniques. The trainer may play the role of the
facilitator where he/she integrates facilitative skills and participatory approaches.
This manual advocates that process-related skills are critical for any FAO staff/collaborator
who is involved in supporting learning of FAO’s counterparts and stakeholders in Member
Countries. In particular, strengthening process-related skills is important to allow a shift:
>> from trainers to facilitators of learning: process and content experts who are able to facilitate any
type of learning situation, whether in face-to-face, experiential or collaborative online situations;
>> from technical advisors to technical coaches: content experts who play an increased coaching
role so that regular advising tasks become genuine opportunities for learning and self-growth.
21 Freely adapted from ‘The facilitator and other facilitative roles’, R. Schwarz, in Organization Development, p. 409.
63
-Diagram 7-
ENHACED
PROCESS - RELATED
SKILLS
-xox-
In the project, «Amélioration des capacités de formulation et d’analyse d’impact des politiques de
développement agricole et rural sur la sécurité alimentaire et la pauvreté» (Improvement of capacities to
formulate and assess impact of agricultural development policies on food security and poverty) in Burkina
Faso, FAO used multiple learning processes as part of a three-year broad capacity development effort
that included technical support, on-the-job assistance and support in distance coaching and mentoring.
The key success factor was involving participants in defining the learning content and creating their own
learning plans based on their actual work needs; this increased their motivation and sense of ownership
in the initiative. FAO experts and the few international consultants deployed to the project did not play
the role of “experts” in the traditional sense of the term, i.e. those who have solutions on hand. They
acted more as coaches and reference points for expertise, i.e. as people willing to share their technical
skills and knowledge, merge them with other competencies from different sources and adapt them to the
context to ensure their relevance and usefulness. This occurred in the framework of a partnership among
different ministries (Environment, Agriculture, Economy and Finance), academic and civil society bodies,
the National Statistical Office and selected development agencies. In this collaborative process, the actors
sought solutions to rural development issues through appropriate policies, and analytical capacities were
created within the national institutions. As a result of this process, the government decided to restructure
the Ministry of Agriculture and created a unit for analysing, the impacts of socio-economic policy. In this
way the institutional sustainability of the capacity development achievements was strengthened. (Source:
Policy Assistance Support Service TCSP, ref. FNOP/INT/120/NOR)
Process-related skills can be learned and require that you practise a lot. While this
Module is not intended to make you a facilitator, trainer or technical coach, it provides
suggestions and tips to help you adopt appropriate approaches for delivering
face‑to‑face workshops, coaching people on-the-job or facilitating online learning
events. It also contains reference to the wealth of material that exists in the area.
-xox-
64
5.2 Ensure quality and coherence in
delivering learning initiatives
Toolbox 5 provides tools and guidance for delivering on-the-job and face-to-face learning initiatives;
specifically, including tools on experiential learning, coaching, facilitating and delivering training.
-xox-
Planning checklist
Discussion
Decision
Planning checklist still N/A
taken required
65
STEP
6 Provide follow‑up
support
-xox-
How much follow-up support is needed depends on a variety of factors, including the needs of
learners and the objectives of the initiative. For instance, learners may have difficulty applying
what they learned to practical situations, or they may need further information, resources or
expertise in order to be able to use what they learned. Technical assistance may be particularly
useful where learning needs to be adapted to be put into practice. For example, policy-makers
who participated in a workshop on policy modalities may need some form of practical support to
translate learned principles into policy mechanisms appropriate for their specific environments.
67
The extent of follow-up support also depends on resources available. Where resources do not exist
for refresher courses or on-the-job technical assistance, other, less expensive follow-up support
modalities may be used. Some less costly methods for providing follow-up support include:
>> scheduled online support sessions where learners are given the chance to
share experiences or ask questions to trainers or other experts online;
>> appointing technical coaches or mentors from former course alumni or more
senior members of participants’ workplace organizations; and
>> facilitating peer-to-peer dialogue and consultations through
list serves or websites for initiative participants.
-xox-
Alumni networks – In Mali and other project countries, alumni of an FFS meet as a group in one of their
own fields on a weekly or bi-weekly basis, rotating each time to the field of a different group member.
Here they discuss specific problems and advise each other on possible solutions. If solutions are not
forthcoming, the facilitator has access to district and national level expertise; this encourages two-way
flows of information between communities and technical support networks. The cost of these follow-up
activities (only the facilitator’s transportation cost) is only a fraction of that of an FFS. (Source: The West
African Regional Integrated Production and Pest Management Programme, a case study, AGP.)
Q&A service and online resources – After the completion of a course, a question and answer service will
be offered, together with access to additional online resources, to facilitate the transfer to the job. (Source:
FAO-GiZ Blended Learning Programme for Country Teams and Food Security Thematic Working Group
Members, part of a project on Developing the Capacities of Food Security Professionals to support the
Comprehensive Africa Agriculture Development Programme (CAADP) Pillar III).
GiZ: GiZ invests heavily in thematic alumni networks, according approximately 10 percent of course
budgets to alumni work. These networks provide ongoing peer-to-peer follow-up support and allow the
exchange of information on specific topics or issues among and with participants, alumni and partner
organizations. They are a way for GiZ to encourage the transfer of acquired skills from individual advanced
training programmes (such as a study trip to Germany) into the work environment in a participant’s home
country, making knowledge obtained in courses sustainable and transferable. Networks initially have
GiZ facilitators, but the expectation is that over time, network members will take over management of
the networks, enlisting needed inputs and support from other alumni. GiZ ’s experience has been that
programme managers are best able to set up alumni networks, because they are motivated by personal
contact. (Source: GiZ)
The follow-up plan is an outline document which helps you to lay out the most appropriate
follow-up measures for your learning initiative. Using this plan, you can figure out what kinds of
measures you wish to use, the kinds of resources you will require and the best roll-out strategy
for your plan. In short, the follow-up plan helps you establish a coherent strategy for a systematic
and realistic approach to promote sustained engagement in the topics of your initiative.
68
How to build a plan
To maximize opportunities for follow-up, begin preparing the ground before the implementation
of the learning initiative. Ideally, follow-up should be planned during the design phase of the
initiative and included in your design outline. The plan can be further refined as the initiative is
delivered and additional information is disclosed by participants. By the design stage, however, you
should already have a clear picture of the type of support that might be required for participants
to implement and use their learning. Make budget provisions for the follow-up in advance.
To build a plan, it is useful to first determine the opportunities in the work environment
that can be used to apply or internalize the skills, knowledge or attitudes that participants
will have acquired upon completion of the learning activity, as well as the challenges
that may exist. To this end, you can use the information you’ve gathered throughout
the previous steps of the learning management cycle, particularly when assessing the
context and the learning needs, when relating to counterparts to identify participants and
during the delivery phase. You can access a template follow-up plan in Toolbox 6.
-xox-
69
Planning checklist
Discussion
Decision
Planning checklist still N/A
taken required
70
STEP
7 Evaluate learning
initiatives
LEVEL 1
Tools 7c 7d 7e 7f 7g
LEVEL 2
Tools 7h 7i 7j
LEVEL 3
Tools 7k 7l 7m
Different types of information may be gathered for the evaluation, depending on its purpose. For
example, coordinators of learning initiatives often find that qualitative information (such as participants’
descriptions of their experiences in trying to apply learning in the workplace) tell them far more about
the use/transfer of learning than quantitative scores. However, quantitative ratings may be necessary
for accountability purposes. Similarly, for learning purposes it may be more important to gather
information about the quality of learning processes than about the exact nature of learning results.
This is important to bear in mind because, where evaluation resources are scarce, evaluation managers
may have to prioritize what sort of information to gather to best achieve the evaluation objectives.
72
7.2 Customize approaches for results evaluation
Process evaluations
Process evaluations look at the quality of the learning management cycle and inputs. They may assess:
-xox-
Process evaluations are a valuable source of information to improve the quality and results
of learning initiatives. Process evaluations can occur after the learning initiative or while
the initiative is being designed, developed or delivered. Process evaluation can be done
by instructional design experts or managers as part of quality assurance processes.
73
Results evaluations
Results evaluations look at the effects of learning on participants and their organizations.
The most commonly known framework for evaluating results of a learning initiative
is the Kirkpatrick model. Kirkpatrick divides results into four levels:
All levels except the first one are linearly related. In other words, learning is a necessary but not
sufficient precondition for behaviour change. Behaviour change is necessary but not sufficient for
organizational impact. Participant satisfaction, however, is not strongly correlated with the other
three levels of results. A learning initiative can result in learning even when participant satisfaction is
low and can be ineffective in achieving learning goals even when participant satisfaction is high.
-xox-
Level 2 - Learning can be measured either formally or informally. Formal measures of learning can
be done through pre- and post-tests to examine what was the starting knowledge of participants
and how the initiative added to that knowledge. However, for many courses, such tests may not
be feasible or particularly useful. Informal gauges of learning can include participant in-class
presentations, projects or on-the-job achievements. In addition, facilitators, coaches or trainers
can use questions on material to check participant understanding and reinforce learning.
74
Level 3 - Workplace behaviour outcomes is the most important level of results evaluation
of learning initiatives, and one of the most challenging. For learning initiatives to contribute
to organizational capacity, participants have to be willing and able to apply knowledge, skills
and attitudes acquired through learning. To understand how participants have used learning in
their workplaces, it is necessary to do a post-initiative evaluation of the behaviour outcomes.
Such evaluations should generally be done at least six months but no more than two years
after the initiative – enough time to ensure that participants have had sufficient time to
implement learning, but not so long that they are likely to have forgotten learning content.
-xox-
The project “Building in-country capacity for food security information products that support decision-
making” was assessed with a view to analysing the extent to which FAO’s training support was effective
in enhancing countries’ institutional capacities to produce more relevant, credible and accessible food
security information products (FSIPs) that support decision-making in food security.
Workplace behavioural changes (Level 3) were assessed several months after the trainings took place
through questionnaires, interviews and observations against a set of criteria for measuring FSIP
improvements.
Concrete examples in all countries showed that several months after the training, participants had retained
the learning, applied it successfully at their workplace and adopted new attitudes in carrying out their job.
Evidence of this workplace behavioural change is summarized below:
Source: Assessment Report, Building In-country Capacity for Food Security Information Products that
Support Decision-making, EC/FAO Food Security Programme Information for Action, December 2008
75
Level 4 – Organizational impact rarely results from stand-alone learning initiatives.22 For this
reason, while impact evaluations may be done of broader organizational capacity development
programmes which address the various aspects of capacity problems, they are rarely done for learning
interventions alone. Organizational impact of learning is often difficult to measure for three reasons:
>> There is generally an attribution problem. Even where organizations have changed, it
is often difficult to know the extent to which this was as a result of learning activities
rather than other complementary interventions or changing conditions.
>> There may be a time lag problem. Sometimes the impact of
learning on organizations is only evident years later.
>> It may be hard to find reliable quantitative measures for some forms of organizational change.
Organizational development and change, and the role of learning in supporting organizational
change processes, is the object of specialized research. Learning Module 4 of the Capacity
Development series will explicitly deal with organizational development and analysis.
Additional resources and examples of how to go about measuring the impact of capacity
development are referenced in the LenCD Learning Package23 and in Toolbox 7.
76
-xox-
Planning checklist
Discussion
Decision still N/A
taken required
2. Does the evaluation plan include both process and Level 1-3
results evaluation?
77
Conclusions
Over the course of the past two decades, the international community’s
understanding of the role of learning in developing sustainable capacity has
evolved considerably. In its 2006 “Guide to good capacity development practice”,24
the Development Assistance Committee (DAC) states (OECD, 2006):
“The traditional capacity building tools of technical cooperation and training have often
proved ineffective in helping to improve performance because they have not been linked
to the necessary organizational and institutional developments…This implies approaching
capacity development in an integrated way, so that individual skills and the organizational
settings in which they can be put effectively to work are created simultaneously.”
A subsequent 2010 report “Training and Beyond: Seeking Better Practices for Capacity
Development”25 highlights the following key points of the emerging consensus:
24 OECD-DAC The Challenge of Capacity Development: Working towards good practice, (p. 30)
http://www.fao.org/fileadmin/templates/capacitybuilding/pdf/DAC_paper_final.pdf
25 Training and Beyond : Seeking Better Practice for Capacity Development:, OECD-DAC, January
2010, Jenny Pearson, p. 2 http://www.oecd.org/dataoecd/35/53/44696077.pdf
>> Achieving sustainable capacity development impact calls for long-term perspectives. There
is a need to make strategic links between short-term activities, such as training courses,
and long-term learning and change goals for sustainable capacity development impact.
Additionally, there is a need to facilitate the continuity of long-term relationships that can
make valuable contributions to success and enable persistence through difficulties;
>> The quality of training design and training cycle management is fundamental to success;
>> Training has often been both inappropriately used and poorly implemented as the response to
capacity development needs. A results orientation can help to ensure that proposed training
activities are appropriately implemented to meet identified needs, and that progress and
the contribution to overall capacity development needs can be monitored and evaluated;
>> Greater attention needs to be paid to translating resources and materials, to
adapt concepts to the local context as well as into local languages, and this
can be achieved through more effective use of local resource providers;
>> Some donor agencies and Development Training Institutes (DTI) recognize
the need to change their approach, practice or role, and understand that they
need staff with soft skills in addition to their existing technical expertise.
Annex 2 - Contents of Toolboxes
>> Guidelines for identifying participants for the FAO Training of Trainers on
Collaborative Conflict Management for Enhanced Forest Management
http://www.fao.org/forestry/29010-0891ed3c1db6578171c4c0e7deb9833ba.pdf
Toolbox 3 – Assess learning needs
Tools
Tools
>> Witkin, B.R., & Altschuld, J.W. (1995). Planning and Conducting Needs
Assessments: A Practical Guide. Sage Publications, Inc.
>> Designing and using questionnaires/surveys – Extracts, by D.S. Walonick
http://www.promoteprevent.org/resources/designing-surveyand-questionnaires
>> Multi-stakeholder Processes Resource Portal http://portals.wi.wur.nl/msp/?page=1180
About using appreciative inquiry: http://www.kstoolkit.org/Appreciative+Inquiry
>> Asking open-ended and probing questions tip sheet:
http://ppa.aces.uiuc.edu/pdf_files/Asking1.PDF
>> Sustainability checklist, available on FAO Intranet under the IDWG on Capacity Development
http://intranet.fao.org/fao_communications/idwgcd/capacity_development_tools/
It is also available in the CD Learning Module 2.
>> Working with partners, LenCD website
http://www.lencd.org/group/learning-package/document/how-establish-partnershipscale
>> Practical resource about setting up partnerships
http://assets.wwf.org.uk/downloads/wwf_parthershiptoolboxartweb.pdf
Toolbox 4 – Design content and select delivery modes
Tools
Tools
Tools
>> Smith, M. K. (2002) ‘Malcolm Knowles, informal adult education, self-direction and andragogy’,
the encyclopedia of informal education, http://www.infed.org/thinkers/et-knowl.htm
>> About writing learning objectives http://www.park.edu/cetl/quicktips/
writinglearningobj.html http://www.nwlink.com/~donclark/hrd/templates/objectivetool.html
>> About task analysis http://classweb.gmu.edu/ndabbagh/
Resources/Resources2/Task%20Analysis.htm
>> A comprehensive ITC-ILO Guide for the design of face-to-face events called ‘Enhancing
Learning for Effectiveness’ can be found at:
http://www.train4dev.net/fileadmin/Resources/Publications/T4D_Guide_final.pdf
>> A good summary of Bloom taxonomy of learning domains
http://www.businessballs.com/bloomstaxonomyoflearningdomains.htm
>> WFP Toolkit - The toolkit aims to assist facilitators and trainers with the design and delivery
of Food Security Assessments (FSA) learning events http://learning.vam.wfp.org/pages/dpg/
pBookContent.asp?c=chap9&p=94. http://learning.vam.wfp.org/documents/Docs/FTK.pdf
>> Ghirardini, B. E-learning methodologies. A guide for designing
and developing e-learning courses, FAO, 2011.
>> Bersin J. (2004). The Blended Learning Book. San Francisco: Pfeiffer Coaching successfully,
J. Eaton and R. Johnson, Essential Managers, Dorling Kindersley Book, 2000.
>> Coaching across cultures, P. Rosinski, Nicolas Brealey Publishing, 2003.
>> CGIAR website provides access to a wealth of resources in scientific mentoring
http://www.genderdiversity.cgiar.org/resource/links_mentoring.asp
>> Coaching and mentoring network http://www.coachingnetwork.org.uk/Default.htm
>> International coaching federation, hosts articles and research about coaching
http://www.coachfederation.org/icf-research/icf-research-portal/
>> FAO study tour report - The Role of Women Producer Organizations
in Agricultural Value Chains, Practical Lessons from Africa and India,
Gender, Equity and Rural Employment Division (ESW)
>> Lessons learned from the Togo-Mali study tour on Sector-wide
Approaches in Agriculture, Investment Centre Division (TCI)
>> Learning Routes, IFAD - http://www.slideshare.net/ifad/
learning-routes-background-documentation
Toolbox 5 – Deliver the learning solution
On-the-job
Tools
Tools
Online collaborative
On-the-job learning
>> A good book on questioning techniques Making questions
work, Dorothy Strachan, 2007, Jossey-Bass
>> Anytime coaching – unleashing employee performance, Teresa Wedding
Kloster, Wendly Shervin Swire, Management Concepts, 2009
>> Useful website on coaching http://coach4growth.com/
>> Kolb, D.A. (1984), Experiential learning: experience as the source of learning
and development http://academic.regis.edu/ed205/Kolb.pdf
>> Educational Foundations of the Field Schools. http://www.vegetableipmasia.org/Concepts/
FarmerFieldSchools1.html. http://www.vegetableipmasia.org/docs/Nonformal%20
Educational%20Foundations%20of%20FFS.pdf. ISNAR, The experiential learning approach,
by Zenete Franca. http://www.farmradio.org/english/partners/archives/bdg/learning.pdf
>> Action learning resource sites. http://www.kstoolkit.org/Action+Learning+Sets.
http://www.fao.org/Participation/ft_more.jsp?ID=10423. http://www.adb.org/
Documents/Information/Knowledge-Solutions/Action-Learning.pdf
Tools
Tools
Tools
Tools
Tools
Process evaluation
Overall
>> Kirkpatrick, D. L. (1968). Evaluating Training Programmes.
San Francisco, Berrett-Koehler Publishers, Inc.
>> Using Training to Build Capacity for Development, An evaluation of
the World Bank’s project-based and WBI training http://siteresources.
worldbank.org/EXTTRABUICAPDEV/Resources/full_doc.pdf
>> If I were the facilitator – Assessing the quality of facilitation, taken from training
materials of Market-oriented farm management for trainers of extension workers,
2007 (AGPS) http://www.fao.org/docrep/011/a1298e/a1298e00.htm
>> FAO e-learning course, Impact Assessment of Large-scale Food Security Programme,
available at: http://www.fao.org/spfs/learning-from-results/e-learning/en/
Level 1
>> A good example for evaluating various aspects of a training course is
included in the FAO training materials for Market-oriented farm management
for trainers of extension workers, Module 7, Training Evaluation, pp. 23-
30 (AGPS) http://www.fao.org/docrep/011/a1298e/a1298e00.htm
Level 2
>> About writing test items
http://www.park.edu/cetl/quicktips/writingtest.html
http://www.go2itech.org/HTML/TT06/toolkit/evaluation/dev_eval.html
>> Changes in perception test form. FAO training materials for Market-oriented farm management
for trainers of extension workers, Module 7, 2007 (AGPS). The module also contains useful
examples of final training examination. http://www.fao.org/docrep/011/a1298e/a1298e00.htm
Level 3
>> Reflection, Consultation, Action template. Taken from Module 7 of the Market-
oriented farm management for trainers of extension workers materials,
2007 (AGPS) http://www.fao.org/docrep/011/a1298e/a1298e00.htm
>> Knowledge, Attitudes and Practices (KAP) Survey of the project “Rehabilitation and sustainable
development of fisheries and aquaculture affected by the tsunami in Aceh Province,
Indonesia”(Project OSRO/INS/601/ARC), 2009
http://typo3.fao.org/fileadmin/user_upload/oed/docs/OSROINS601ARC_2010_ER.pdf
>> About using Knowledge, Attitude and Practice (KAP)
evaluation and Most Significant Change (MSC):
http://www.anthropologymatters.com/index.php?journal=anth_matters&p
age=article&op=viewArticle&path%5B%5D=31&path%5B%5D=53
http://www.stoptb.org/assets/documents/resources/
publications/acsm/ACSM_KAP%20GUIDE.pdf
http://www.kstoolkit.org/Most+Significant+Change
Toolbox 8 - Plan, Manage and Coordinate.
Training: Training activity aims to provide the skills, knowledge and attitudes required for
employment in a particular occupation, group or related occupations, or for exercising
a function in any field of economic activity. For the purpose of this manual, the term
“training” will be used to indicate a learning initiative that takes place through in-class
or face-to-face interaction with an instructor, typically in the form of a workshop.
26 This and the following definitions are based on The International Standard Classification
of Education Glossary, ISCED, 1997 and 2011 (not yet formally adopted).
Toolbox 1 2 3 4 5 6 7 8
Tools
1A Assess the context: Is it a learning issue? CORE 95
References and resources
94
Assess the context: Is it
Tool 1A a learning issue?27
-xox-
WHAT: This tool helps you determine whether learning is likely to be an effective method to address an
identified capacity issue, such as when national partners are not delivering the project up to the expected
standard or are facing challenges as a result of changed circumstances.
WHEN: At the beginning of the Learning Cycle Management, before starting to plan the learning initiative
Instructions27
The approach should be customized for the specific use. It can be used to either
facilitate a self-assessment by the people involved, or to run an exercise with the
people who are affected, to achieve their involvement and ownership.
How to use it
Example 2: Country x has adopted a new Convention on pesticide management, requiring restructuring of the
Ministry of Agriculture’s organizational mandate.
27
Based on Instructional Design, P. Smith and T. Ragan, 1999
95
2. Identify who is affected by the capacity issue and how they are affected.
Who is affected by the Example 1: Project team, national partner Example 2: Ministry of
capacity issue? Agriculture
How does the capacity Example 1: The credibility of the national partner and future funding are
issue potentially affect at risk.
the achievement of the
mission and/or the goals Example 2: The Ministry of Agriculture will have to shift its mandate to
of the organization? help various constituencies implement the Convention.
Where there has been a change in circumstances and/or in the context, does this √
change significantly affect what individuals must understand, know or do?
Are relevant policies, strategies and regulatory frameworks supportive of the goals √
of the organization?
96
4. Determine if the capacity issue can be completely or partially addressed by learning.
Yes,
Possible considerations Yes No* partially
4.3 What learning objectives should be defined to address the capacity issue?
Example 2 - Ministry of Agriculture’s staff is able to provide advisory services to farmers and other groups on
new standards and procedures resulting from implementation of Convention
4.4 If learning can only partially address the problem, which are the factors/constraints that cannot be
addressed by learning activities? Please list the factors here:
Example 1 - The partner does not have external legitimacy to act and operate as required by the project, with
no ability to link up with other national and regional institutions.
The partner’s mandate should be reviewed and strengthened.
Example 1 - Assistance with mandate review and advocacy at a higher level to help strengthen the credibility of
the partner in the country and within the region
6. Next steps
Once the assessment is completed, there are two options:
>> If learning is needed to address capacity issues, the next steps are:
>> to identify the target audience and assess learning needs, as appropriate;
>> to define and plan for complementary modalities to address the issues
identified in the organizational and enabling environment.
>> If the capacity issue is not a learning issue, the next steps are:
>> to undertake a broader capacity assessment to identify causes of the
issue at the organizational level or in the enabling environment (the
Capacity Assessment Tool is available from the FAO CD Portal);
>> to define non-learning modalities (e.g. technical support, financial support, policy
review, access to knowledge and good practices) to address the issues identified.
97
Toolbox 1 2 3 4 5 6 7 8
Tools
2A Identify the target audience profile CORE 100
2B Participant Application Form Template CORE 103
References and resources
>> Guidelines for identifying participants for the FAO Training of Trainers on
Collaborative Conflict Management for Enhanced Forest Management
http://www.fao.org/forestry/29010-0891ed3c1db6578171c4c0e7deb9833ba.pdf
99
Identify the target
Tool 2a audience’s profile
WHAT? This tool is meant to guide FAO staff in defining the profile of the target audience for the learning
initiative jointly with the national counterparts who are being targeted and who are likely to propose or
appoint participants. It suggests the development of an “Invitation Note” to be prepared, discussed and
agreed upon together with relevant partners as early as possible in the planning process.
Instructions
Involve your key partners (particularly those who are likely to appoint participants to
the learning initiative) in preparing and sharing the invitation from the early stages
of planning. This is important to support their ownership of the process, secure
their support throughout the steps of the learning management cycle and ensure
their commitment to the results that will be brought about by the initiative.
Use the following sample to develop an invitation for your learning initiative. You
may use this sample in conjunction with the Application Form (Tool 2b).
Sample invitation
1. Short description of the learning initiative, the learning objectives and delivery mode:
Learning objectives:
Delivery mode:
100
2. Profile of candidates
Background: Explain the background that candidates should have (e.g. technical, managerial)
Gender and age: Explain which age range is targeted by the initiative, and whether there will be a quota/
percentage set for female candidates
Job function/role: Explain which functions and roles are targeted by the initiative
Prerequisite qualifications and skills: Explain the required education levels of candidates and prerequisite
skills (e.g. communication, project management) that must be mastered at the basic/good level
Minimum working experience: Indicate the minimum amount of work experience required in the subject
matter area
Language: Indicate the language and proficiency levels that candidates must possess
Expectations at the individual level: Explain what is expected from the people participating in the
initiative. Example: Selected individuals will commit to organizing at least two events to disseminate the
new knowledge or skills within the local community. Such events could be training sessions, individual
coaching sessions, information sessions, etc.
Expectations of the sending organization: Explain what is expected from the sending organization in
terms of support and follow up. At a minimum, organization management should support the objectives of
the learning initiative and any follow-up activities. This commitment should be demonstrated by alignment
between the initiative and the organization’s strategic plan and budget. Example: The organization will
commit to providing full institutional support to the individual for organizing the events described above;
it will also commit to provide the budget for scaling up the initiative the following year.
101
4. Description of selection process
This section describes the selection process, including the criteria by which candidates will be selected.
Description of the process: Describe the selection process. Example: The selection process will include
pre-screening of candidates by using an application form and a telephone interview with a team composed
of:
Description of the selection criteria: Describe the selection criteria to be used. Example: The following
criteria will be used to select the candidates:
Roles and responsibilities: Describe the roles and responsibilities of FAO and sending organizations.
Example: FAO will appoint the selection team in collaboration with national partners. National partners
will be kept informed at each step of the selection process through email or telephone. When required, ad
hoc telephone sessions can be held.
102
Participant Application
Tool 2B Form Template
Learning initiative:
A. General information
Applicant’s Name:
Contact address:
Age:
Gender:
Your highest education level:
Job function:
Organization:
Country:
B. Work-related information
1. To which technical area are your current work responsibilities related
________________________________________________________________________________________
________________________________________________________________________________________
________________________________________________________________________________________
____________________________________________________________________________
2. How would you rate your knowledge and skills in relation to (subject matter
area) (1=no knowledge/skills; 5= excellent knowledge/skills)
________________________________________________________________________________________
________________________________________________________________________________________
________________________________________________________________________________________
____________________________________________________________________________
103
3. In your view, what are your strengths in doing your work?
________________________________________________________________________________________
________________________________________________________________________________________
________________________________________________________________________________________
____________________________________________________________________________
4. What major constraints do you encounter in your job or in your work setting?
________________________________________________________________________________________
________________________________________________________________________________________
________________________________________________________________________________________
____________________________________________________________________________
5. What knowledge, skills or other benefits would you most like to gain
from the learning initiative? Please be as specific as possible.
________________________________________________________________________________________
________________________________________________________________________________________
________________________________________________________________________________________
____________________________________________________________________________
6. How do you expect to use what you have gained here once you have returned to your workplace?
________________________________________________________________________________________
________________________________________________________________________________________
________________________________________________________________________________________
____________________________________________________________________________
Thank you!
104
Toolbox 1 2 3 4 5 6 7 8
Guidance sheets
3A Learning needs assessment: Process and tips CORE 107
3B Overview of data collection techniques CORE 109
Tools
3C Managing focus groups RECOMMENDED 111
3D Key informant interviewing RECOMMENDED 113
3E Developing surveys RECOMMENDED 117
Guidance sheets
3F Learner analysis CORE 120
Tools
3G Learning needs assessment checklist CORE 122
3H Work setting analysis CORE 124
References and resources
>> Witkin, B.R., & Altschuld, J.W. (1995). Planning and Conducting Needs
Assessments: A Practical Guide. Sage Publications, Inc.
>> Designing and using questionnaires/surveys – Extracts, by D.S. Walonick
http://www.promoteprevent.org/resources/designing-surveyand-questionnaires
>> Multi-stakeholder Processes Resource Portal http://portals.wi.wur.nl/msp/?page=1180
>> About using appreciative inquiry: http://www.kstoolkit.org/Appreciative+Inquiry
>> Asking open-ended and probing questions tip sheet: http://
ppa.aces.uiuc.edu/pdf_files/Asking1.PDF
>> Sustainability checklist, available on FAO Intranet under the IDWG on Capacity Development
http://intranet.fao.org/fao_communications/idwgcd/capacity_development_tools/
It is also available in the CD Learning Module 2.
>> Working with partners, LenCD website
http://www.lencd.org/group/learning-package/document/how-establish-partnershipscale
>> Practical resource about setting up partnerships
http://assets.wwf.org.uk/downloads/wwf_parthershiptoolboxartweb.pdf
106
Guidance sheet - Learning needs
Tool 3a assessment: Process and tips
WHAT? This guidance sheet provides an overview of the process for carrying out a learning needs
assessment (LNA) and offers some tips.
WHEN? Usually after a context analysis has been conducted and learning activities have been found to
be a solution to the identified need, an LNA can be carried out. However, it also can be done concurrently
with the context analysis.
>> helps ensure that the learning content is relevant and useful to participants;
>> helps developers ensure that learning activities target the correct level for participants;
>> ensures that learning activities will build on what participants already know;
>> helps build participant motivation and organizational support to implement the learning;
>> builds beneficiary ownership and involvement;
>> provides a basis to evaluate results of learning activities; and
>> maximizes cost-effectiveness of learning activities by focusing content on actual needs.
107
2. Decide on data collection techniques.
>> What data collection methods should be used, given the number of people to be
interviewed, the resources and time available and the context in which individuals
operate? See Guidance Sheet 3b for an overview of the main methods.
>> Speak directly with participants. Do not assume that technical assistants, organizational
managers or external counterparts have a full picture of participants’ needs and constraints.
>> Where possible, get a range of perspectives from various stakeholders.
>> Focus on getting a clear picture of participants’ competencies, experience
and present work functions. A common mistake in designing learning content
is including basic information that participants already know.
>> Find out what challenges and constraints are anticipated for participants to implement
the learning. Learning activities may be able to help address these constraints.
>> An LNA is often performed inadequately because time and funding are insufficient.
It may be necessary to build internal support within your own department to
ensure sufficient allocation of resources to conduct an effective LNA.
108
Guidance sheet - Overview of
Tool 3b data collection techniques for
a learning needs assessment
WHAT? This guidance sheet provides a comparative overview of the data collection techniques that can
be used for a learning needs assessment (LNA).
WHEN? After analysing the context, decisions can be made about which LNA data collection technique
is most appropriate.
Time required
Resources
Method When to use Conduct Cost
Analyse required
or data
implement
109
Time required
Resources
Method When to use Conduct Cost
Analyse required
or data
implement
110
Tool 3C Managing focus groups28
WHAT? This tool provides a suggested process for managing focus groups for a learning needs assessment
(LNA).
Focus groups are a means of collecting information from a small group (5 to 12 participants)
in a systematic and structured format. Two very distinct characteristics of the focus group
interview process are: (1) the purpose of focus groups is to collect data; and (2) focus
groups use group interaction to elicit information from the group members.28
A focus group interview is an interview with a small group of people on a specific topic. Groups are typically
six to eight people who participate in the interview for one-half to two hours. The focus group interview
is indeed an interview. It is not a discussion. It is not a problem-solving session. It is not a decision making
group. It is an interview. The participants are typically a relatively homogeneous group of people who are
asked to reflect on questions asked by the interviewer. Participants get to hear each other’s responses and
to make additional responses beyond their own initial responses as they hear what other people have
to say. It is not necessary for the group to reach any kind of consensus. Nor is it necessary for people to
disagree. The object is to get high quality data in a social context where people can consider their own
views in the context of the views of others.
Source: Patton, Michael Quinn. How to Use Qualitative Methods in Evaluation, Newbury Park, CA: Sage, 1987.
Advantages Disadvantages
You can interview multiple people at one time. More reserved members may not feel comfortable inserting
their contributions in the discussion. Other participants may
try to dominate discussions.
Members of the focus group can build on It can be difficult to coordinate busy schedules.
each other’s comments and reactions. This
can yield synergistic discussion around topics
or themes.
28
Adapted from WBI Needs Assessment Knowledge Base, WBI Evaluation Group, with permission.
111
Advantages Disadvantages
You can observe verbal and non-verbal Participants can easily get “off task” if the facilitator does
interactions among participants. not maintain structure and control throughout the process.
You can get buy-in from key players. It may be costly if performed in person in different
countries.
Suggested process
1. From the list of information required for the LNA, identify those
elements that may best be obtained through focus groups.
2. Create a list of small groups of relatively homogeneous people
with similar background and experience.
3. Prioritize the information requirements for each focus group and use this
to create a facilitator’s guide or protocol for each focus group.
4. Use an experienced focus group facilitator where possible, and ensure you have a notetaker in the
room. The interviewer usually acts as the facilitator, introducing the subject, guiding the discussion,
cross-checking participants’ comments and encouraging all members to express their opinions.
5. If you facilitate the focus group, ask participants to reflect on your questions, provide their
own comments, listen to what other group members say and offer their observations.
Remember, the main purpose is to elicit ideas, insights and experiences in a context where
people stimulate each other and consider their own views along with the views of others.
6. Immediately after the focus group has ended, the facilitator and notetaker should verify that
all of the essential information from the group has been captured in a written document.
7. If appropriate, run several focus groups. This ensures that
you gather enough information for the LNA.
>> Have a clear and specific goal for the focus group.
>> Create a survey to be given out to participants in order to capture information
that may not be discussed in the focus group because of time limitations.
>> Carefully present each of your questions to the group, and allow the group members
a couple of minutes to think about the question and record their answers.
>> Complete a test-run of the focus group in order to identify potential problems, changes
to questions or additional materials that should be available to participants.
>> After a question has been answered, and before moving on to the next question, verbally
report back a summary of what you heard. This confirms for the group members that
they communicated what they intended to and allows them to make any suggestions
for adjustments in the event that their thoughts were not accurately represented.
>> Plan for the focus group to take between 40 minutes and three hours.
112
Tool 3d Key informant interviews29
WHAT? This tool provides elements of process for conducting one-on-one interviews for a learning needs
assessment (LNA).
WHEN? As part of an LNA when it is important that the interviewees are allowed to elaborate through
follow-up questions. It is particularly recommended when interviewing senior individuals, who may
appreciate the additional personal attention that the interview can offer over a survey or focus group.
>> Structured: A controlled format, where the interviewee is asked predefined questions
in a specific order. Following an interview protocol, interviewers read the questions to
the interviewees. The format may also include a choice of predetermined answers.
>> Semi-structured: A more flexible format, allowing new questions to be brought up
during the interview as a result of the interviewee’s responses to previous questions.
>> Unstructured: A fully flexible format, which does not ask the respondent
to select among predetermined answers, but instead advocates listening
to how each person uniquely responds to the question.
Individual key informant interviews can often provide in-depth context, stories and
discussion related to one or more topics pertinent to the LNA. They can help describe
an environment or a situation with qualitative data, by documenting the knowledge
and experiences of people using their own words. Interviews also offer an opportunity
for the interviewee to become familiar with the LNA and its objectives.
29
Adapted from: Conducting a key informant interview, University of Illinois Extension Service – Office
of Program Planning and Assessment http://ppa.aces.uiuc.edu/pdf_files/Conducting1.PDF;
Getting the Lay of the Land on Health: A guide for using interviews to gather information, written for the Access project
by S. T. Sherry and A. Marlow, 1999 http://www.accessproject.org/adobe/getting_the_lay_of_the_land_on_health.pdf;
WBI needs assessment knowledge base, WBI Evaluation Group, 2007, with permission
113
Advantages Disadvantages
Interviews typically allow for more focused Time requirements for interviewers and interviewees
discussions and follow-up questions. can be significant.
Individuals may offer information in interviews that Interviews have the potential to reduce the scope
they wouldn’t offer in a group context. and sample for data collection.
Interviews can be an excellent source of stories and Results of multiple interviews may contradict each
context. other or be difficult to analyse.
The interviewer can observe the non-verbal Interviewees may be biased or represent only a
behaviours of an interviewee. limited perspective on performance issues/themes.
2. Who to interview? Selecting a key informant should be done with care. The informant needs to be
someone who has personal knowledge or experience with a particular problem, or has professional
training in that area. Be sure to have a mix of people – people of different ages, ethnicity, religious
affiliation, educational level, etc. The informants must also be able to express themselves clearly.
3. How many to interview? This decision is largely a matter of time and resources (you will
have to write up and analyse all of the interview information!) It should, however, be a
sufficient number to begin to get a sense of the environment and to identify themes.
114
Conducting the interview
5. Introduce yourself and your objectives. When you call to set up the interview, introduce yourself
and your position. Thank the interviewee for her/his willingness to meet with you and be sure s/he
understands the purpose of the LNA and what you intend to do with the information you receive.
6. Address confidentiality issues. Be very clear about how much confidentiality you can
offer. Confidentiality is an important consideration in conducting key informant interviews.
Information you obtain in the interviews should not be directly attributed to any particular
individual. Assuring confidentiality will make your informants feel more comfortable
sharing information that may be controversial or of a personal nature. Also, reports on
the interview results will then focus on the content rather than on who said what.
7. Ask questions. Start the interview with some basic “ice-breaking” questions. Questions
like this are not a waste of time; they give both of you a chance to get acquainted and feel
comfortable with each other. Many interviewers find it helpful to start with general questions
and then become more specific as the interview progresses. It’s a good idea to have questions
ready ahead of time, but sometimes what the informant tells you will change things, so you
may have to ask different questions. It is also likely that someone may introduce a subject not
included in your questions; let them talk (within reason!). The point of this whole experience
is to allow the interviewees to tell you their stories, including their particular knowledge,
opinions and experiences. Give them the space to say what they need to say. If the person
deviates completely from the topic, then pull them back by referring to your questions.
8. Use active listening techniques, such as nodding your head, saying “uh-huh” or “Can you
tell me more about that?”. If the informant knows you are really listening, s/he will talk more.
Never let something you don’t understand pass without asking for clarification. You can do this
by saying something similar to, “’I’m sorry, I don’t understand how that would work,” or “If I
understand you correctly, you’re saying...”. Your role is to be a facilitator so you can get all the
information you can from this person; it is not to impose your own interpretations or perspectives.
9. Use prompts. You may also find people who are more reticent or not certain about what
specific information you are seeking. In these situations, you will want to use the “prompts”
that accompany each general question. The prompts are designed to assist interviewees in
thinking about the questions and what they know or have experienced. (see the link under
References and resources: ‘Asking open-ended and probing questions tip sheet’)
10. End with a summary. If you’ve been really good at establishing rapport and listening, you
may find it difficult to break off the interview. Beginning your summary of what they have
said will help the informant know things are winding down. This summary is important
because it gives you a chance to verify that you have understood the informant. Put what
s/he has said into your own words and ask if you have said it correctly: “Now let me see
if I have understood you correctly. You’re saying that.. . .”. If you have misunderstood,
this gives the informant a chance to correct you and clarify the response.
115
Recording and analysing the information
11. Keep track of what was said. There are several ways to keep track of the information
you have received. Sometimes interviewers make brief notes on 3 x 5 cards, which can be
grouped later in a variety of ways. Sometimes the interviews are taped so the interviewer can
listen again and make notes. It can be helpful to have someone else listen to the interview
to confirm that you have correctly understood what was said. Whatever method has been
decided upon, you need to be sure that you have some record of the main points. Usually it’s
a good idea to sit down right after the interview and put your thoughts on paper – a summary,
your impressions of the key informant’s feelings and anything else that seems relevant.
12. Analyse the information. After conducting several interviews, you will be able to begin
analysing your results. This process involves comparing and contrasting what your informants
told you. Overall, you will be looking for common themes among the informants. Are there
similar stories about the experiences of a particular population? If there are, which parts are
similar? All of these questions will help locate themes. It is, of course, also important to look
for differences. You will want to look for places where different types of people disagree.
Although there are no shortcuts to thoroughly reviewing and discussing interview information,
there are some approaches that may make analysis easier. It is helpful to analyse your initial
interviews to identify early themes that can be added to interviews conducted later. If you
will have a lot of interviews, you might actually list issues/themes that emerge and tally how
frequently these come up. You should also list themes by category or type of interviewee.
13. Reporting on the results. Once you have analysed your data, looking for the most important
common themes and disagreements, you will write up your results. The report can follow a
variety of forms, depending on your original purpose. Taking the time to produce a written
document that can be shared with others serves multiple purposes and is well worth the
effort. A list of your interviewees can be contained in a final report, but none of them should
be specifically attributed. Writing a report on your interview results largely follows the
process of planning and conducting the interviews. Elements of a report would include:
>> an explanation of why you conducted the assessment. What motivated you to do this research?
What were some of the assumptions you had about the issue before beginning the research?
>> who you interviewed by category and how many interviews you conducted over
what period of time. (The actual list of interviewees can be in an appendix.)
>> the general focus of your questions. (The interview protocol
with the actual questions can be in an appendix.)
>> the themes that emerged, including use of unattributed quotes
when these would help enliven the description.
>> a description of what your group concludes or will do as a result of what you learned.
116
Tool 3e Developing surveys
WHAT? Elements of process to develop surveys for a learning needs assessment (LNA)
WHEN? As part of an LNA when the budget is small, the target audience is geographically dispersed and
there is enough time to allow sufficient responses
A survey is a quick way to get a general sense of your learners’ key needs and challenges.
Both paper and electronic surveys may be distributed to audiences of varying sizes to gather
information relevant to the LNA. For example, surveys can be administered to members of
the target audience to gather data on participants’ expectations, demand for the training,
and reactions to the proposed delivery modes. Alternatively, surveys can be sent to all major
stakeholders to identify potential obstacles, gauge the level of buy-in and determine the
extent of resources available to support the development of the learning initiative.
Surveys are generally quicker than one-on-one interviewing and relatively low-cost. They should be
planned well ahead to allow sufficient time for participants to respond (between two and three weeks).
Advantages Disadvantages
Surveys give you the ability to contact a large Surveys require very clear questions and instructions.
number of people in many countries.
They yield a lot of data at a relatively low cost. Return rates are typically low.
An easy way to survey learners is to develop a set of questions and make them
available on the Web so that participants can answer them online. For those who do
not have easy access to the Web, you can send a Word copy of the survey.
117
The following checklist provides some guidelines on how to build survey questions and how to deliver
the questionnaire. You can also check the resources indicated under References and resources.
Checklist
Define objectives
1 Write down the purpose of your survey and what you hope to achieve as a result. This will help you
identify the specific objectives for the survey and the questions you want to ask for each objective.
2 Create questions for each objective. This helps you better define what you want to find out from the
survey. It will also help you avoid redundant questions and lengthy surveys.
3 Start with generic and easier questions (e.g. title, office, grade).
6 Ask about one piece of information per question. For example, the question “What training activities
did you attend in the last year and what did you find useful about them?” should be broken down into
“What training activities did you attend in the last year?” and “Please indicate what you found useful
about them”.
7 Group similar questions together. For example, you may build a group of questions on participants’
demographics, one on the job they perform and another on the training they have received.
9 Decide whether there will be mandatory questions and determine which ones they are.
10 Provide space at the end for any additional comments participants may want to provide.
11 Keep the survey short and indicate the time it will take to complete it. Try not to exceed 25–30
questions. Also, keep in mind that the length of the survey will depend on how many closed vs. open-
ended questions you have included: the more open-ended questions available, the longer it may take
to comment on them and to review the information gathered.
12 If you intend to follow up with some of the participants after the survey, the last question should ask
participants to indicate whether they are willing to be contacted via phone or in person. State that the
information gathered during a follow-up interview will remain confidential. Ask participants to provide
their contact information voluntarily (unless you made the contact information a mandatory part of the
survey).
118
No. Action item
13 Open-ended:
Allow respondents to comment freely on a subject. They are more time-consuming to analyse but
provide qualitative data that may shed more light on the question you are asking. When you build
an online survey, you can decide whether or not you want to limit the space provided for the answer.
Avoid negative or leading questions that may bias participants’ responses.
Closed:
They provide a pull of predetermined answers. You need to decide whether people can provide more
than one answer (multiple choice) or whether they are limited to one (mutually exclusive questions).
Generally, try to limit the responses options to seven (exceptions include, for example, lists of titles and
grades which may exceed seven. In such cases, use a drop-down menu rather than a checklist).
Scales:
Likert scale: Ask participants to rank or rate options. Examples: strongly agree, agree, somewhat
agree, disagree, strongly disagree; very high, high, average, low, very low; excellent, good, fair, poor.
Interval scale: Find out about a range of possibilities. Example: How much time do you spend on xyz-
related activities in a week? Up to 1 day, between 2-3 days, more than 3 days.
Also, explain how you intend to share the results of the survey (this is very important as respondents
usually like to know the results).
119
Tool 3f Guidance sheet - Learner analysis30
WHAT? This guidance sheet is to assist developers in identifying learner characteristics that might impact
the design of the learning initiative
WHEN? As part of the learning needs assessment (LNA); it addresses the “participants’ characteristics”
Some of the most important characteristics that affect learning include the following:
Baseline skills – Baseline skills are the initial levels of skills that participants bring to the
learning initiative. They allow learners to develop new skills that build on their existing
ones. Also, part of designing any learning initiative includes identifying what prerequisite
skills participants need. It is important to determine that participants indeed do possess
these prerequisites before they begin a learning initiative. If many learners do not
possess the prerequisite skills, then steps may be taken to adjust the learning initiative
to address that gap before moving on to the intended content of the initiative.
Prior knowledge of topic – If participants have good working knowledge about the background
of a topic, then the learning initiative can immediately focus on achieving the specific learning
objectives that are built on that base of knowledge. If, however, participants do not have a strong
foundational knowledge of the topic, additional time likely will be needed to introduce the topic and
establish the required working knowledge before moving into more advanced or complex skills.
Content for this tool is adapted from Dick, W., & Carey, L. (2008). The systematic design of instruction. Boston:
30
Allyn & Bacon; and WBI Needs Assessment Knowledge Base, WBI Evaluation group, with permission
120
is important to differentiate between attitude towards content, which relates to participants’
feelings towards a certain topic or subject, and their motivation towards learning, which may
include factors surrounding a particular learning initiative or process. Although not every aspect
of participants’ attitudes towards content and motivation to learn can be adjusted, knowing that
such attitudes exist can be helpful in making adjustments that might improve those attitudes.
Learning preferences and styles – Participants will have their own learning preferences, and
it is helpful to know, in broad terms, what some of these preferences are so that the learning
initiative can potentially be adjusted to suit them. Some learners, for example, may prefer to learn
by watching interactive multimedia presentations; others may prefer to read written text. Some
may prefer group activities, while others may prefer a more individual learning style. Rarely will all
members of a group agree about learning preferences, but understanding the preferred learning
modes can be helpful in designing a learning initiative so as to improve its overall impact.
A few of the many different theories on learning styles include the following:
>> Kolb, an organizational behaviour expert, looked at four basic styles: Activist, Reflector,
Theorist and Pragmatist. http://www.businessballs.com/kolblearningstyles.htm
>> Honey and Mumford developed a questionnaire based on Kolb’s styles.
http://nwlink.com/~donclark/hrd/styles/learn_style_survey.html
>> The MyerBriggs Type Indicator is an inventory used to determine personality disposition and
preferences based on Carl Jung’s theory of psychological types. http://www.myersbriggs.org/
my-mbti-personality-type/mbti-basics/; http://nwlink.com/~donclark/hrd/styles/jung.html
>> Felder and Solomon developed four categories with opposing perspectives: Sensing/Intuitive,
Global/Sequential, Visual/Verbal, Active/Reflective
http://www4.ncsu.edu/unity/lockers/users/f/felder/public/ILSdir/styles.htm
31
From: Understanding how people learn, David G. Reay, Nichols Publishing Company, New Jersey, p.83
121
Learning needs assessment
Tool 3g checklist
Instructions
Use this checklist to verify whether all important issues have been addressed in your LNA, regardless
of its scope or the methods to be used. Questions can be customized based on the target group,
the subject-matter area or the selected LNA approach. Remember that engaging the target
audience in genuine and active participation in this exercise is indispensable for its success.
2. What background do they bring to the learning initiative in terms of prior knowledge, skills,
experience, exposure, etc.?
6. How and where will learners want to learn (e.g. on-the-job, in class)?
7. What expectations will learners have of the initiative in terms of the work involved?
122
Areas to be addressed in an LNA
12. What are the current job functions of the target audience? (Depending on the depth of
needs assessment being done, answering this question can range from a brief description to
a very detailed analysis of job tasks).
13. What are the gaps in knowledge, skills or attitudes that hinder the achievement of individual
and organizational goals?
15. What are the possible consequences of not addressing one or more identified gaps?
19. What obstacles do participants face in fulfilling their work functions that may pose difficulties
for implementing the learning?
20. What support from peers and/or superiors will be available to participants in their work
environment to perform what they have learned?
21. How are participants currently interacting in the work setting to exchange information,
network or share knowledge?
E. Organizational context
23. How do the course goals relate to the specific needs, challenges and culture of participants’
organizations?
123
Tool 3h Work setting analysis32
WHAT? This tool provides suggestions for conducting an analysis of the work setting in which learners will
apply the new knowledge and skills. It helps to: identify the potential obstacles to implement and transfer
the learning to the workplace; and to plan follow-up measures and complementary capacity development
support.
Instructions32
Skills are not applied in a context-free setting; in fact, they are almost always intimately
connected with the setting in which they are most frequently used. Therefore, an important
part of designing a learning initiative is to consider the characteristics of the context – or the
setting – in which the skills will be performed. Understanding the performance context can
yield important information about how best to prepare participants for performing the skill in
that setting, and can influence both the design and the ultimate delivery of the initiative.
Look at the four areas below, and consider the questions in the checklists. Discuss them with the
target groups and other relevant stakeholders and ensure that conclusions from the discussion
are reflected and addressed in the learning initiative design, delivery and follow up.
1.1. Is the skill to be developed relevant within the actual workplace setting? (It would
be illogical, for example, to develop skills for using a particular piece of equipment
if that equipment is not, nor ever will be, part of the workplace context).
Adapted from WBI Needs Assessment Knowledge Base, WBI Evaluation group, with permission
32
124
2. Managerial or supervisory support
Managerial or supervisory support refers to issues regarding support, motivation and guidance
from the supervisor to the participant who is performing the skill. The presence of absence of
managerial support can have implications for the design of a learning initiative. If managerial
support is present, it is possible to integrate that into the design of the learning initiative by
indicating possible roles for the manager in the initiative itself. If such support is not present,
it may be helpful to develop a more robust set of self-practice and evaluative techniques.
2.1. Will supervisors either directly or indirectly support the acquisition and proper
implementation of the skill, or will participants be largely responsible for this on their own?
2.2. Will supervisory personnel directly communicate with the participant concerning this skill,
or is the participant alone largely responsible for maintaining ongoing focus on the skill?
3.1. Does the work setting’s current physical layout, design and/or facilities support the individual
practising the skill, or will the skill be difficult to implement in the current physical context?
3.2. What, if any, are the hindrances to the performance of the skill in terms of the physical
characteristics of the work setting (e.g. work layout, on-site equipment or tools)?
3.4. Can the performance of the skill be modified in some way to accommodate any perceived
shortcomings in the physical characteristics of the work setting? In what way?
4.1. Can fellow workers who are responsible for performing the same skill, or perhaps
those in the immediate work setting who previously carried out the same or similar
responsibilities, potentially influence the learners’ ability to implement the new skills?
4.2. Are there any gender or cultural considerations linked to the performance of certain roles
in the work setting (e.g. biases about women in the roles of managers or leaders)?
Communities of practice can be considered as part of the social aspect of the work setting, if they are
integrated into the work setting and if they support the skills. Whether these are computer-mediated
communities or simple social networking, communities of practice can play a decisive role in skills
performance because of their potentially powerful effect on community members. If such support
networks are in place, it is very valuable to integrate them into the learning initiative as much as
possible. Not all participants will be aware of the existence of these communities in their work settings,
and so providing information about them is an important part of delivering the learning initiative.
125
Toolbox 1 2 3 4 5 6 7 8
Guidance sheets
4A Essential principles for adult learning CORE 128
Tools
4B Tips and checklist for writing learning objectives CORE 131
4C Task analysis RECOMMENDED 134
Guidance sheets
4D Blended learning CORE 138
4E What is on-the-job learning? CORE 141
4F Coaching and mentoring RECOMMENDED 143
4G Study tours RECOMMENDED 147
Tools
4H Is ad hoc technical coaching appropriate? RECOMMENDED 151
4I Elements of Terms of Reference for technical coaches RECOMMENDED 153
Guidance sheets
4J Structuring the learning event RECOMMENDED 154
4K Preparing written learning material CORE 156
Tools
4L Preparing a design outline CORE 158
References and resources
127
Guidance sheet - Essential
Tool 4a principles for adult learning33
WHAT? Adult learners differ from younger students in many ways. This guidance sheet presents some
essential principles for adult learning and provides suggestions for adult-focused design and delivery
strategies.
Guiding principles 33
The principles presented below are built around the primary concept,
illustrated in the Learning Pyramid diagram, that adult learners have specific
requirements that must be addressed in any learning effort.
33
Adapted from Education & Counseling for Risk Reduction (ECRR) Curriculum, Center for Health Training (based on
materials by Helmich J., in Making Connections: Teaching and the Human Brain. Renate M. and Caine G., Addison
Wesley Publishing Company, 1994); https://courses.worldcampus.psu.edu/public/faculty/adults.html Dr. Gary Kuhne
for “ADTED 460 - Introduction to Adult Education,” a course offered through Penn State’s World Campus )
The Adult Learner: The Definitive Classic in Adult Education and Human Resource Development by
Malcolm S. Knowles, Elwood E. Holton III, & Richard A. Swanson, 2005, Burlington, MA: Elsevier.
128
1. Theory should be used to serve practice.
>> Adult professionals tend to be pragmatic in their learning. Use instructional strategies that focus
on practical, applied skills, using theoretical content only as necessary to inform discussion.
129
5. Pay attention to surroundings.
>> Learning involves both focused attention and peripheral perception. Trainers should
pay attention to room temperature, ambient noise, seating arrangements and
the use of visual aids (e.g. charts, illustrations, displays) to support learning.
>> Using music is a good way to cue memory and influence more natural acquisition of information.
6. Learning involves opening up to challenges and taking risks. Adults may feel insecure
in learning environments or be worried about appearing weak or incapable.
>> Establish an environment where participants feel supported and safe enough to ask
questions and try new things, where individual needs and uniqueness are honoured
and where abilities and life achievements are acknowledged and respected.
>> Treat participants as peers – intelligent experienced adults whose
opinions are listened to, honoured and appreciated.
>> Don’t ask people to take risks too early in a workshop or course (e.g. by engaging
in a role play exercise) unless they already know each other well.
>> Provide opportunities and allow time for people to establish themselves in the group.
>> Promote intellectual freedom and encourage experimentation and creativity.
>> Consider how to give constructive criticism in ways which
support learning rather than discourage it.
>> Allow for “consequence-free” practice activities where learners do not feel judged.
>> Use introductory exercises or “ice-breakers” to allow learners to
get to know each other at the beginning of training.
130
Tips and checklist for writing
Tool 4b learning objectives34
WHAT? A learning objective is a statement of what a learner should be able to do at the conclusion of
the learning initiative. This tool provides some tips and a checklist to help formulate effective learning
objectives.
WHEN? After the Learning Needs Assessment (LNA) and the Task Analysis (if one is done)
Tips 34
>> Learning objectives should be specific, measurable, achievable, relevant and timed. They should
consider the time available and be coherent with the goals of the learners’ own organizations.
Achievable, for all participants who complete the exercises and readings
Timed, so they do not involve a task or action that continues forever and would be difficult to measure
e-learning courses, FAO, 2011, and Instructional Design, P. Smith and T. Ragan, 1999
131
Wrong: it describes the learning activities Good: it describes the learning outcome
Learners will play a simulation in which they will Learners will be able to identify potential sources of
experience... water pollution…
>> Learning objectives should consider the appropriate range of cognitive domains. A useful question
could be: What type of learning outcome does the objective represent? In other words, what will
learners learn: will they memorize the step of a procedure or will they be able to actually perform it?
4. Analyse The learner is able to decompose and define relationships among components.
35
Source: Anderson and Krathwohl, 2001
132
Here are some possible action verbs, ranging from the lowest to the highest level:
>> To achieve overall coherence, the learning contents, learning activities and assessment
tests must be aligned with learning objectives. For example, if the learning objective
is about improving the learner’s understanding of a concept, then activities and
the final assessment test will have to be aligned with the same concept.
CHECKLIST
>> Is the objective expressed in terms of knowledge, skill or attitude?
>> Does the objective contain a clear, unambiguous description of performance?
>> Is the objective expressed in terms of outcomes?
>> Has an action verb been used to describe the observable actions that the
learner can show that will demonstrate that he/she has learned?
>> Is the objective based on accurate task analysis?
>> Does the objective contain details of the conditions under
which the performance is to be carried out?
>> Are the conditions based on a real working environment?
>> Does the objective include criteria for the performance standard to be achieved?
>> Are the standards measurable?
>> How will the achievement of the objective be assessed?
>> Does the learner understand and agree with the objective?
133
Tool 4c Task analysis36
WHAT? This tool provides step-by-step guidance for analysing and articulating the kind of knowledge,
skills and attitudes which learners should have to carry out their job. It helps answer what learners are
supposed to know and be able to do to perform their job.
WHO? Developers of the learning initiative and, where possible, instructional designers
WHEN? As part of designing the learning initiative, before defining specific learning objectives
Instructions 36
Task analysis is particularly indicated when the learning initiative targets highly technical or specialized
job functions. It ensures that learning content is relevant to learners’ needs. Task analysis will help you:
36
Adapted from Ghirardini B., FAO E-learning methodological guide, Section 3.6, p. 40
134
Step 3: Break up the tasks
Break up the tasks into:
>> steps (for procedural tasks); or
>> guidelines that should be applied to perform the tasks (for principle-based tasks).
The curriculum audience is composed of mid-level managers, technical staff and field personnel who are
involved in collecting, managing, analysing and reporting food security information.
After discussion with several food security experts, it emerges that the most critical aspects that should be
improved relate to: the selection of assessment methods and indicators; the analysis of the collected data;
and, very importantly, the preparation of effective food security reports to communicate research findings
to decision-makers.
Undoubtedly, all these tasks are quite complex and the way they are carried out depends on a given context.
This means that they are principle-based rather than procedural tasks.
Therefore, rather than providing learners with step-by-step instructions on what to do, we can provide them
with guidelines they can apply to specific situations.
Each task is then described and a set of guidelines are identified for learners to follow in order to correctly
accomplish the task. Let’s focus, for example, on Task 4:
135
Example of a task analysis
Step 4: Identify required knowledge
What does the food security professional need to know to apply these guidelines?
Timeliness of reporting
136
Task analysis template
Step 1: Inventory tasks
>> What is the overall goal of the learning initiative? (This should be
aligned with the target audience’s organizational goals).
>> What are target audience’s major responsibilities in relation to the overall goal of the learning initiative?
>> What are the critical aspects/tasks that the target audience
needs to improve to accomplish the overall goal?
1.
2.
3.
4.
5.
Describe how these tasks are going to be accomplished by answering these questions:
Describe each task and list the type of judgment, decisions or steps that are required for each of the tasks.
These explain “how” each task is accomplished.
Task Guidelines/steps
Task description:
What do target audience members need to know to make those judgments, take decisions or undertake steps?
137
Guidance sheet - Blended
Tool 4d learning37
WHAT? This guidance sheet provides suggestions on using “blended learning” – a combination of
different “media” (e.g. technologies, activities and types of events) – to create an optimum learning
programme for a specific audience. The term “blended” means that traditional instructor-led training is
being supplemented with other electronic formats.38
E-learning methods and activities can be integrated with more traditional classroom
and live, face-to-face (F2F) events. Traditional methods (e.g. face-to-face classrooms,
workshops, coaching and mentoring) can be combined with e-learning content
and methods to make the most of the two delivery channels. 37 38
Blending F2F training with e-learning can take different forms. Bersin (2004) identifies two main models:
>> Programme Flow: Learning activities are organized in a linear, sequential order
and learners have deadlines to accomplish the various assignments; this is similar
to traditional training, but some of the activities are conducted online.
>> Core-and-Spoke: A major course (e-learning or F2F) is provided and a set of supplemental
materials are available to reinforce the main course; these materials are optional and not scheduled.
The Programme Flow model is more suited for observable outcomes and
assessment purposes (including certification), since it enables instructors and
facilitators to easily monitor each step and formally track learners’ progress.
37
Adapted from Ghirardini, B., FAO’s e-learning methodologies guide, Appendix
38
Bersin J. (2004). The Blended Learning Book. San Francisco: Pfeiffer.
138
This approach has several advantages compared with a traditional F2F approach: (1) it forces
learners to come to the classroom prepared; (2) it allows the design of more efficient classroom
activities which are tailored to the specific needs or interests of the participants; and (3) it
reduces the total time in the classroom, which reduces the costs of the programme.
To optimize the design efforts spent in producing e-learning courses, trainers can adapt
and reuse e-learning materials in classroom sessions and training workshops.
139
They can also reuse media elements, such as illustrations and diagrams, and textual
content to create presentations for trainers and materials for learners. 39
7
Example: FAO blended learning programme for country teams and food security thematic working
group members
A blended learning programme has been designed to provide members of national food security teams with
the knowledge and skills required to design and implement country agriculture and food security investment
plans.39 The programme includes the following components:
>> Pre- workshop preparation: A questionnaire will be submitted to participants a few days before the
online phase. Participants will be asked to describe their role in the national food security system and
their areas of expertise. The questionnaire will aim both to help facilitators tailor the activities based
on participants’ profiles and allow participants to understand each other’s roles and responsibilities.
>> Online workshop (core component): The workshop is a mix of individual study of interactive e-lessons
on food security topics and online activities supported by facilitators and subject-matter experts.
Both synchronous and asynchronous communication are used for online discussions and group work.
The main outcome of the online component is an individual workplan that will help participants
reflect on their country situation and serve as a supporting resource for F2F workshop activities.
>> Bridge period: This is a period between the two core components of the course. Online
support is provided to particpants to complete their preparation prior to the F2F workshop.
>> Face-to-face workshop (core component): This is comprised of classroom events where
participants can present and discuss their previous work, practise communication principles and
techniques and proceed in their workplan with the assistance of a subjectmatter expert.
>> Q&A service and online resources: A question and answer service will be offered after the completion
of the course, together with access to additional online resources, to facilitate the transfer to the job.
39
The programme has been developed by FAO and GIZ (Deutsche Gesellschaft für Internationale Zusammenarbeit)
with the University of Pretoria as part of a project entitled ‘Developing the Capacities of Food Security Professionals’.
The programme aims to support the Comprehensive Africa Agriculture Development Programme (CAADP) Pillar III -
Framework for African Food Security (FAFS). It integrates e-learning materials on a wide range of food security topics
developed by the EC-FAO Programme on ‘Linking Information and Decision Making to Improve Food Security’.
140
Guidance sheet - What is
Tool 4e on-the-job learning?
WHAT? This guidance sheet provides an overview of different forms of on-the-job learning, including
coaching, mentoring, job shadowing, study tours and action learning.
WHEN? When selecting delivery modes for the learning initiative. On-the-job learning is particularly
suited when learning:
On-the-job learning includes among the most flexible practices to meet individual learning
needs. The individual learns while working on the job, i.e. by undertaking real tasks, using real
tools, facing real challenges and confronting a real environment. As such, on-the-job practices
are best suited to support individual ownership in learning and capacity development.
141
The following table presents some of the most widely used methods of
on‑the-job learning and describes how they are used in FAO.
Coaching Learner is paired with an expert from either within or TCI ad hoc technical
outside his/her work setting. Coaching may follow coaching; FAO Executive
a structured learning plan or can be unstructured, Coaching Programme
responding to needs as they arise. Learning can be
scheduled at regular intervals, ad-hoc (i.e. unscheduled) or
online.
Job Learner closely observes one or more colleagues Junior staff sit in a meeting
shadowing performing regular duties in the actual work environment. and shadow more senior
Can involve following one role or a variety of roles to get colleagues who facilitate a
a more general overview of the tasks an office performs. group discussion.
Allows time for learners to ask questions of colleagues. It is
generally a one-on-one activity.
Study tours/ Learner acquires new perspective on existing skills, Examples of study tours
exposure applied in a different context, and may also job shadow by FI, TCI and ESW (See
visits while on mission. The learner should generally be asked Tool 4g)
to complete a formal report outlining conclusions/
observations made during the period. It can be a one-on-
one or team-based activity.
Action Team-based learning through experience. Typically used Farmer Business Schools
learning to solve a real problem in the work environment. It is (See box 8 in the main
facilitated by an experienced colleague/advisor. document)
142
Guidance sheet - Coaching
Tool 4f and mentoring40
WHAT? This guidance sheet provides definitions for coaching and mentoring and discusses the
commonalities and differences between them. It also reviews when to use mentoring and coaching and
describes the key skills and attributes of coaches and mentors.
Definitions 40
>> Coaching is a process that enables learning for effective job performance or
development of long-term capabilities. Technical coaching is used in this Module
to indicate coaching in FAO-specific technical subject-matter areas.
>> Mentoring is off-line help by one person to another to make a significant transition in knowledge or
work. The mentor provides support and advice to facilitate both career and personal development.
Main differences
>> A professional relationship, with purpose >> Mentor acts as an advisor, guide and role model
and goals, that focuses on the future >> Mentor provides wisdom, experience,
>> Coach supportively holds the coachee practical help and support to facilitate
accountable for getting what he/she wants both career and personal development
and challenges his/her thinking process >> Can be a two-way exchange process
>> The relationship has a defined start and end between mentor and mentee
>> Coaches are generally experts in the >> Is an ongoing relationship with no
subject matter and also have knowledge clearly defined start and end point
and understanding of process
Successfully; The Pocket Guide to Coaching (Internal resource, WBI); FAO Executive Coaching Programme, CSH.
143
Commonalities
When is it used?
144
Ad hoc coaching can occur anytime, such as when:
>> national counterparts have taken on new tasks or projects which
require developing new or enhanced skills/work behaviours;
>> national partners’ projects, tasks, activities or products are not up to
standard and require focused support and improvement;
>> new knowledge has been learned which requires practise, supervision
and feedback to be internalized and institutionalized;
>> there are individual performance problems which prevent the organization
or project from progressing and/or achieving its objectives;
>> people need to learn how to operate with newly introduced
equipment, techniques, methods or procedures;
>> new people have joined a project/an organization and need to be brought up to speed.
E-coaching (i.e. coaching done online) can be done by any combination of:
>> skype/telephone coaching;
>> online platforms;
>> email;
>> video conferencing; and
>> where possible, face-to-face meetings.
The advantages of e-coaching are that it is less costly; supports virtual teams; and
allows coachees to express themselves over the phone, which they may find more
comfortable. The challenges of e-coaching are that it may require at least one face-to-
face meeting to build trust and establish an informal, safe environment; it may be difficult
to find the time to follow-up on a regular basis; and feedback is often not timely.
Technical coaches
Highly qualified technical coaches must possess knowledge and experience
in the subject-matter and also be good at process-related tasks.
145
Mentors
The Consultative Group on International Agricultural Research (CGIAR), which uses scientific mentoring
in its programmes41, offers a good description of a mentor. According to CGIAR, a good mentor:
appreciates individual differences knows how to develop skills is available, has open doors
is good at teaching and can balance direction and provides advice on career
communication self‑direction decisions
41
Nature: http://www.nature.com/nature/journal/v447/n7146/full/447791a.html cited in the
CGIAR website http://www.genderdiversity.cgiar.org/resource/links_mentoring.asp
146
Tool 4g Guidance sheet - Study tours
WHAT? This guidance sheet provides practical suggestions and tips on how to design study tours based
on good practices.
Study tours are on-the-job or informal learning situations in which participants can acquire know-
how or new perspectives on existing skills or see skills applied in a different context. Through study
tours, learners are invited to visit a specific location, see with their own eyes how work is carried out
and meet with key stakeholders to gain firshand information and insights about a specific topic.
Study tours have tangible and less tangible benefits. In general, study tours allow participants to:
Study tours can be used anytime as alternatives or complements to other types of delivery
modes. They work best when combined strategically with other modalities as part of broader
projects and programmes, although they have proven to be effective even by themselves. Job
shadowing can be associated with study tours, although this is not a necessary component.
147
Study tours can be used strategically:
>> selecting participants, based on clearly determined criteria, from the visiting country;
>> identifying the needs and priorities of participants and the needs
and capacity gaps of the sending organizations;
>> setting objectives for the study tour and choosing the topics, which
should reflect the composition of the participant group;
>> setting the agenda in collaboration with the visiting country/institutions and
the hosting country/institutions to enhance ownership by all involved;
>> identifying an appropriate itinerary and related activities, such as meetings to be held,
discussions to be facilitated, site visits to organize and demonstrations to plan;
>> hiring a local subject-matter expert, where appropriate, who can accompany the participants
and provide explanations, clarifications and additional insights in the subject matter;
>> arranging for debriefings at the end of each day or half way through the end of the study tour; and
>> arranging for a final debriefing, in which participants should be encouraged to develop
recommendations to bring back home to improve their own practice in the subject matter.
148
Tips and suggestions from FAO study tours
>> Study tours are most effective when they are proposed at the right time
and combined strategically with other project components.
>> A study tour should ideally last from three to four days and not exceed
seven days, as long tours can create information overloads.
>> Well-planned processes, clear division of roles and responsibilities and
commitment/ownership are critical for the success of the tour.
>> Selection of the participants and composition of the group should be done strategically.
For example, depending on the nature of the project, it may be advisable to mix
public-sector and private-sector participants, or individuals with operations and policy-
oriented functions, or decision-makers. Such tours offer valuable opportunities for
participants to create linkages among themselves, which are important to enhance
the overall capacities of the individuals and their sending organizations.
>> Study tour evaluations are best done using a three-step benchmarking tool (See Tools in Toolbox 7).
149
Study tour from India to General Santos Fish Port Complex (Philippines)
(as part of the project “Capacity Building in Support of Cleaner Fishing Harbours TCP/IND/3102 A, March
2007–December 2009)
This study tour was conducted over a period of five days and covered all aspects of fishing harbour
operations. The General Santos Fish Port Complex is located in General Santos City in South Cotabato on
the island of Mindanao in the southern Philippines. It is the main centre for landing marine fish in the area,
and local fishing boats as well as foreign vessels are allowed to unload their catches.
Two participants, one from each harbour, participated in the study tour to learn first-hand how a fishing
harbour in another Asian country was being operated and to pick up good practices to implement in their
own harbour.
The following criteria were used to select the participant from each harbour:
>> The person must have been directly involved in managing the fishing harbour for at least
two years, and intend to be involved in direct management for the next five years.
>> The person must be in a position to recommend and implement
changes in the management of the fishing harbour.
>> The person must be willing to prepare a write-up and presentation of the study tour
and present it upon return to relevant state officials and stakeholders. The write-up and
presentation must include recommendations on how the practices and lessons from the
visited fishing harbour would be applied to their own situation, or modified as necessary.
>> to observe the operations of a fishing harbour, learn lessons from the
experiences of the management authority and explore how the best practices
in the visited fishing harbour could be applied in their own situation;
>> to analyse the structure and composition of the harbour management group and the
operations pertaining to cleanliness and maintenance of sanitation and hygiene; and
>> to learn how stakeholders are involved in managing a fishing harbour, determine the costs
of running and maintaining the fishing harbour and see who is paying for what services.
Following the study tour, the participant from the Dhamara fishing harbour proposed
managerial, financial and technical recommendations to manage fishing harbours in India.
150
Checklist: Is ad hoc technical
Tool 4h coaching appropriate?
WHAT? This tool presents a checklist to support FAO technical staff to identify opportunities for technical
coaching as part of their regular ongoing work in Member Countries. It is intended as an orientation tool
to verify whether the conditions exist for ad hoc technical coaching with national or local counterparts, or
whether other solutions should apply.
151
Issue Yes No Comments
8. There are between one and six people who need technical coaching.
152
Elements of Terms of Reference42
Tool 4i for Technical Coaches
WHAT? This tool offers suggested elements of Terms of Reference (TORs) for technical officers and/or
consultants undertaking regular technical work in Member Countries.
WHEN? As part of delivery mode selection, after coaching has been considered a viable option
These TORs may apply, with some adaptation, to situations where the Technical Officer acts either as
the technical coach or facilitates a relationship between an external technical coach and the learners. 42
Inspired from: WB executive coaching programme, Coaches Terms of Reference, WB; and a
42
concept note on distance mentoring shared by the Investment Centre Division (TCI)
153
Guidance sheet - Structuring
Tool 4j the learning event
WHAT? This guidance sheet provides a suggested pattern for structuring and sequencing an effective
learning initiative, whatever delivery mode is chosen. It is based on educational psychologist R. Gagne’s
“Nine Events of Instruction”.43
Simulate
Inform Provide Enhance
Gain recall Present Elicit Provide Assess
learner of guidance retention
attention of prior the material performance feedback performance
Objective for learning and transfer
knowledge
1. 2. 3. 4. 5. 6. 7. 8. 9.
1. Gain attention. Present a problem or a new situation. Use an “interest device” that grabs
the learner’s attention. This can be thought of as a teaser – the idea is to grab the learners’
attention so that they will watch and listen while you present the learning point. 43
How? You can use devices such as:
>> storytelling
>> demonstrating
>> presenting a problem to be solved or a hypothetical situation (e.g. just imagine that...)
>> doing something the wrong way (the instruction would then show how to do it the right way)
>> appealing to a common memory (e.g. you all remember when...)
Based on Instructional Design, P. Smith and T. Ragan, 1999 p. 115. The diagram is
43
154
2. Inform learner of objective. This allows the learners to ‘sit in the driver’s seat’ in the lesson, such
as by organizing their thoughts around what they are about to see, hear and/or do. There is a
saying in the training field: Tell them what you’re going to tell them; tell them; and then tell them
what you told them. This cues them and then provides a review which has proven to be effective.
>> How? Describe the goal of a lesson and then state what the learners will be
able to accomplish and how they will be able to use the knowledge.
5. Provide guidance for learning. Although the learner’s attention was invoked at
the beginning, it must be refocused continuously throughout the lesson.
>> How? You can ask leading questions to help learners attend to the most critical features
of the lessons/contents, or use graphics and cues to direct the learners’ attention.
7. Provide feedback. Tell the learner that his or her response is correct, or analyse the learner’s
behaviour. The feedback needs to be specific; telling them why they are doing a good job
or providing specific guidance is better than just saying “you are doing a good job”.
>> How? Provide the results of a test or quiz or give verbal comments.
9. Enhance retention and transfer. The process of transfer, i.e. the application of new
knowledge and skills to a variety of real-life situations and future learning tasks, can be
enhanced by giving learners opportunities to apply their learning to a variety of circumstances.
Research suggests that spontaneous transfer rarely occurs. In many cases, learners require
prompting to see the connections between prior learning and a new situation.
>> How? Inform the learner about similar problem situations, ask learners to find examples
or apply principles in real-life conditions that they would anticipate encountering
or provide additional practise through action plans and follow-up support.
155
Guidance sheet - Preparing
Tool 4K written learning material44
WHAT? This guidance sheet contains suggestions for developing a manual, a booklet or any publication
as part of a learning initiative.
Organization of information 44
The first thing to consider is organizing the information into more manageable
groups to improve the reader’s comprehension and ability to access and retrieve the
information. This technique is called “chunking”. There are no hard rules on how big
a chunk is supposed to be, but you can think of it as a main piece of information or a
main learning objective, augmented by a few pieces of ancillary information.
From a layout perspective, you may want to discuss the main concept and place
it in a prominent position on the page, and put related information on a different
place on the page or format it in a different way. For example, you can:
>> add a case study that illustrates your main point on one side of the page, surrounded by a box.
>> add a tip or a note after you have discussed your main topic and separate it visually.
NOTE
Research suggests that human beings can
understand and remember no more than
seven, plus or minus two, items of information
at a time. This phenomenon is called the
“chunking limit”. Further, as the complexity of
the information increases, the chunking limit
decreases.
156
Secondly, consider how to sequence complex or large amounts
of information. Here are a few suggestions:
Reflection devices
To help learners reflect on what they are reading as they go through the materials and to help them
through their thought process, consider adding reflection questions at key points in your document.
Reflection questions are questions that activate users’ minds and prepare them for what they are going
to read next. This facilitates the reflection process and helps users retain information they are reading.
Questions also can be used to stimulate recall of prior knowledge. Before you introduce a new
concept, it is useful to start the paragraph or chapter with a question or a sentence that may
help bridge what users already know (based on their experience and background) to what
they are reading. Of course, this may be a bit challenging because you don’t know all of your
learners’ profiles, but if you are able to identify some common characteristics, you may be
able to provide this link between their prior knowledge and the new information. This strategy
helps learners retain information and facilitates comprehension of the new knowledge.
Memory devices
To help users retain information and new knowledge, you can use memory devices such as:
157
Tool 4l Preparing a design outline
WHAT? This tool suggests a template for a design outline to provide a clear picture of what the learning
initiative will look like. It can be used to help monitor the design/development work as it proceeds.
WHEN? After the Learning Needs Assessment (LNA) and the definition of learning objectives
Instructions
1. Overview – In the first part of the template, summarize key decisions about:
>> the title of the initiative;
>> its overall goal;
>> the delivery mode that has been selected or which is envisaged;
>> the learning objectives; and
>> the envisaged follow up.
2. Design plan – Reflect on the following and complete the design plan accordingly:
>> List each learning objective or each cluster of learning objectives on a
separate row of the design plan. Consider what type of learning each
objective or cluster of objectives involves: does it primarily involve acquiring
new knowledge, strengthening existing skills or acquiring new skills?
>> What content, in terms of knowledge and skills, will support the achievement
of each objective or cluster of objectives? List your content in terms of key
topics, using the results of the task analysis where one has been conducted.
Remember to include both technical and functional skills as appropriate.
>> What resources (e.g. readings, websites, previous learning material) will be
used to create content? List those that have already been identified.
>> What practical activities or exercises will be created to provide participants
with the opportunity to practise and apply the knowledge or the skills
(e.g. case studies, simulations)? List them in the table.
>> What type of follow-up measures do you envisage?
>> What means will be employed to assess learning and behaviour changes?
158
Template: Design outline
1. Overview
Overall goal
Delivery mode
Learning objectives
Envisaged follow-up
2. Design plan
1.
2.
3.
Toolbox 1 2 3 4 5 6 7 8
On-the-job
Guidance sheets
5A Experiential learning CORE 163
5B The technical coaching process: approaches and tips CORE 167
Tools
5C Action learning CORE 176
5D Action plan for coaching CORE 179
5E Review of coaching progress RECOMMENDED 182
5F Learner’s log RECOMMENDED 183
Guidance sheets
5G Key tasks for trainers/facilitators CORE 184
5H Key phases of a training workshop CORE 187
Tools
5I Post-workshop reflection and action plan CORE 190
5J Delivering effective PowerPoint presentations RECOMMENDED 194
Online collaborative
Tools
5K Comparing face-to-face and online facilitation tasks RECOMMENDED 196
References and resources
On-the-job learning
>> A good book on questioning techniques
Making questions work, Dorothy Strachan, 2007, Jossey-Bass
Anytime coaching – unleashing employee performance, Teresa Wedding
Kloster, Wendly Shervin Swire, Management Concepts, 2009
Useful website on coaching http://coach4growth.com/
Kolb, D.A. (1984), Experiential learning: experience as the source of learning and development
http://academic.regis.edu/ed205/Kolb.pdf
>> Educational Foundations of the Field Schools
http://www.vegetableipmasia.org/Concepts/FarmerFieldSchools1.html
http://www.vegetableipmasia.org/docs/Nonformal%20
Educational%20Foundations%20of%20FFS.pdf
ISNAR, The experiential learning approach, by Zenete Franca
http://www.farmradio.org/english/partners/archives/bdg/learning.pdf
>> Action learning resource sites
http://www.kstoolkit.org/Action+Learning+Sets
http://www.fao.org/Participation/ft_more.jsp?ID=10423
http://www.adb.org/Documents/Information/Knowledge-Solutions/Action-Learning.pdf
162
Guidance sheet - Experiential
Tool 5a learning
WHAT? This guidance sheet provides an overview of the process of experiential learning, with tips for
facilitating experiential learning sessions.
1. Concrete experience
2. Observation and reflection
3. Generalization and abstract conceptualization
4. Application
The focus of experiential learning, which is clearly illustrated in the farmer field school (FFS) approach,
is learning through discovery, experimentation, informed decision-making and group/community
action. As shown in several FFS examples, the approach is important because it may produce an
impact that goes beyond mere learning of technical skills, such as when participants master a process.
This enables learners to help others learn, apply technical principles, acquire increased self-confidence
and organize collaborative activities in their communities to institutionalize technical principles.
Because of its broad impact, experiential learning is among the most effective ways to develop overall
capacities of a community. You can find a summary description of the FFS approach as applied in
integrated pest management in Asia at:
http://www.vegetableipmasia.org/Concepts/FarmerFieldSchools1.html
163
The experiential learning cycle
1. Concrete experience:
The experience stage is the initial activity and data-producing part of the cycle. This
phase is structured to enable learners to “do” something. “Doing” includes a range
of activities, such as participating in a case study, role play, simulation or game, or
listening to a lecture, watching a film or slide show, practising a skill or completing
an exercise. In this phase in a typical FFS example involving rice integrated pest
management (IPM), farmers would go into a rice field and collect data.
2. Reflection:
In this stage, learners reflect on the activity undertaken during the experience stage. They
share their reactions in a structured way with other members of the group. They may speak
individually, in small groups or in the larger group. They discuss both their intellectual
and attitudinal (cognitive and affective) reactions to the activities in which they engaged.
The facilitator helps the learners think critically about the experience and verbalize their
feelings and perceptions, and he or she draws attention to any recurrent themes or
patterns which appear in the learners’ reactions. The facilitator also must help the learners
conceptualize their reflections so they can move towards drawing conclusions. In the FFS
IPM example, farmers at this stage would return to the meeting place to analyse the data.
164
3. Generalization:
In the generalization stage, the learners form conclusions and generalizations that might be
derived from, or stimulated by, the first two phases of the cycle. The facilitator must help the
learners think critically to draw conclusions that might apply generally or theoretically to “real
life.” This stage is best symbolized by the following questions: “What did you learn from all
this?” and “What more general meaning does this have for you?”. In the FFS IPM example,
farmers at this stage would use their data to prepare a presentation about field conditions
and take decisions for the rice field, such as whether to apply fertilizer or insecticides.
4. Application:
After learners have formed some generalizations, the facilitator must guide the learners into
the application stage. Drawing upon the insights and conclusions reached during the first
three stages, learners can begin to incorporate what they have learned into their lives by
developing plans for more effective behaviour in the future. Techniques used to facilitate the
application stage can include action plans, reviewing each other’s action plans, formulating
ideas for action, sharing action plans with the whole group and identifying additional learning
needs. The facilitator assists during this process by helping learners be as specific as possible.
At this stage, farmers in the FFS example would implement their decision in the rice field.
One example of how an experiential learning cycle can be applied to a different situation is
the case of the Maldive Islands after the Nargis cyclone. As part of its emergency intervention,
FAO organized awareness-raising campaigns to change fishermen’s behaviours and
attitudes towards risk in order to improve their compliance with standard safety rules.
Safety rules at sea are not theoretical; they are part of daily reality. While awareness campaigns are
effective to draw attention to an issue, learning might be required to ensure that the knowledge
is actually integrated into the fishermen’s practices. Experiential learning might be facilitated
as part of, or to complement, the awareness-raising campaign, in the following manner:
165
The role of facilitator
In experiential learning, the facilitator plays a radically different role than that of an
expert. For the approach to be successful, the facilitator takes a more ‘withdrawn’
facilitation role that promotes genuine ownership by the participants. He or she:
166
Guidance sheet - The
Tool 5b technical coaching process:
approaches and tips
WHAT? This guidance sheet describes the process and approaches that FAO Technical Officers/
collaborators might use to coach national counterparts or partners in specific technical areas, either in
planned or ad hoc coaching sessions. The process, with some adaptation, is also applicable to e-coaching.
Do Don’t
welcome the learner on a personal basis. jump into the contents too quickly.
initiate a discussion on the need and opportunity for diagnose the need for coaching.
coaching.
establish ground rules. get distracted by other activities (e.g. reading notes).
interrupt.
167
2. Identify outcomes.
In this phase, the technical coach works with the learner to identify specific outcomes that will
help develop or improve the learner’s performance or address the situation that originated the
need for coaching. At the end of this step, you will have gained an agreement on a set of specific,
measurable, achievable, realistic and time-bound goals. In particular, the technical coach will:
>> agree on learning objectives and identify the sequence
of tasks/stages required to achieve them;
>> draw up a learning plan together with the learner, where appropriate,
showing the main content/topics and objectives for each session
>> develop a task list, where appropriate. This is a detailed list of all the tasks
the learner will be able to perform as a result of achieving the objectives. This
can also include conditions and standards for successful completion.
Do Don’t
determine minimum performance standards for establish goals that are too vague.
objectives.
highlight choices and help learners prioritize assume the goals are understood without checking.
realistically (e.g. urgency, interest, resources
required).
listen and observe learner’s behaviours. tell the learner what do to or not to do.
168
3. Assess the current situation.
In this phase, the coach works to establish a clear understanding of the learner’s current
situation including the level of skills, knowledge and behaviour. The coach will:
>> help the person review what he or she has done or not done, and
how his or her actions have led to specific results;
>> examine the functions that the learner usually performs and what needs to be improved;
>> highlight the gaps in performance between the learner’s current
behaviour and the desired performance goals.
Do Don’t
mirror or paraphrase (e.g. this is what I hear you state what the problem is.
saying..).
help generate options (e.g. let’s brainstorm some tell the learner what to do or not to do.
possible options..).
ask for feedback around consequences of behaving jump to quick solutions before there has been a
in certain ways (e.g. what impact did your action thorough assessment of the situation.
have on..?).
encourage reflection on experience (e.g. what does ask closed-ended questions that shut down
it mean to you?) exploration by the learner.
track the insights or learning the learner has about plan insufficient time to generate information
current behaviour so that these can be used during required for planning.
the action planning.
help the learner discover his/her strengths and withhold your own ideas and feedback.
personal resources that will be needed to succeed
with the project or the changes ahead.
169
4. Create an action plan (see Tool 5d)
In this step, the technical coach and the learner will come to a mutual agreement about
specific action steps required to accomplish goals, and the coach will seek a commitment
from the learner to implement the action plan. In particular, the coach will:
>> help the learner maximize choices and options of action;
>> encourage independent decision-making and real commitment to action;
>> build a sense of confidence for the changes ahead.
Do Don’t
ask questions about possible solutions and direct or tell people what to do.
consequences (e.g. what would be the benefits for
you if you did that?).
facilitate alternative options (e.g. what additional be too general about the help you will provide.
things will you do?).
communicate your expectations for action (e.g. my withhold information or useful suggestions.
expectations is that the report will be completed
by..).
encourage independent decision-making (e.g. of the withhold your expectations of actions you want to
options we have identified, to which ones will you see taken.
commit?)
170
5. Provide ongoing feedback.
In this step, the technical coach will provide ongoing support to
the learner on the job. In particular, the coach will:
>> make him/her self available for a longer period of time, either in face-to-face or online situations;
>> provide specific and constructive feedback;
>> recognize and acknowledge achievements and changes in performance.
Do Don’t
monitor the milestones outlined in the action plan. give too much or too little direction.
ask for feedback on your coaching. provide feedback in an untimely and unfocused
manner.
There are advantages and disadvantages to coaching from a distance . Once e-coaching or
e-mentoring has been agreed upon, a number of tips should be considered for increased impact:
>> Make sure you have a private and quiet area to talk; this will help you focus
more on your coachee’s tone of voice, expression and manner.
>> Have at least one face-to-face conversation at the beginning of the
coaching relationship and at agreed intervals after that.
>> Plan ahead to ensure that your schedules incorporate the coaching session.
>> Prepare and use agendas to keep the conversation on track.
>> Prepare the next meeting’s agenda at the end of the current meeting.
>> Insert regular process checks into your conversations to make sure you are both on the same track.
>> Take lots of notes so you can review actions and commitments.
>> Ask the coachee to keep a learning log (see Tool 5f) to record behaviours and
actions that were helpful on the job and when encountering obstacles.
>> Remember that context setting and relationship building are still important, even at distance.
45
The Pocket Guide to coaching, a practical guide for leaders, WB internal material
171
Approaches to coaching46
>> Explain the purpose and process and check for understanding.
Example: “You know, it would save you a lot of time if you used a macro instead of
having to enter that whole paragraph every time. Do you know how to do that?”
>> Demonstrate.
Example: “Here, let me show you how. First, just …”
>> Have the learner practise while you encourage.
Example: “Now, you do the next section while I watch ... ”
In non-directive coaching, the learner’s active doing is central to the process instead
of being the third step as it is in directive coaching. The art of questioning is a skill that
needs considerable practise to develop. The general idea is to prompt the learner to
explore issues in depth, either by direct or implied questions. It also keeps the learner
fully engaged and responsible for his or her ownership of the learning process.
From the outset, the learner must explore alternative solutions to problems posed by your questions.
The learner will feel in control as well as satisfied when the solution is identified. What’s more, since
the learner will have done most of the work, he or she is more likely to remember what was learned.
Selecting an approach
An effective Technical Coach should be able to move between the two coaching approaches,
depending on the situation and on the learners’ starting skill level and will/motivation to learn.
With learners who begin with low skill/low motivation, for example, a directive
style might be preferred. This can then be changed into a less directive style as
the learners’ motivation and skills increase. The non-directive approach is usually
preferred because it reinforces learners’ ownership in their own learning.
Based on: UNHCR Management and Leadership Programme, Global Learning Centre with
46
172
This diagram illustrates the two approaches and the various techniques associated with them:47
How good are you in listening, questioning and providing feedback? Non-directive coaching
is sometimes described as “coaching by listening and asking questions”. You may need to
improve your skills in the following three areas to master the non-directive coaching approach.
Powerful listening
Listening is an underrated managerial skill that we all think we do well, although few actually do.
Since we are able to process what we hear at three times or more the speed that anyone can
speak, we all have a tendency to be preparing our response instead of truly listening. You can’t
be a good technical coach without really understanding the needs and concerns of the learner.
Listening is much more than the physical process of hearing. It’s a process in which you integrate
physical, emotional and intellectual input in search of meaning. Although complete objectivity is
rarely possible, good listening requires a conscious attempt to understand the speaker without
letting personal opinions influence interpretation of the speaker’s words. This means trying to
understand the message the speaker wants to communicate—not what you want to understand.
A coach who listens effectively uses three basic skills: sensing, attending and responding.
47
Freely adapted from: High Low Matrix Coaching Model – Coaching techniques for will and skill issues – Coach4Growth.com
173
Effective listeners also:
Effective questioning
As a technical coach, one of your primary responsibilities is to help learners define where they
“want to get” and “which way to go” to get there. One of the most valuable tools in your arsenal of
communication skills is the art of questioning: the ability to ask the right question at the right time.
Why ask questions? The obvious answer is to get information. But there is an even more important
reason: questions promote communication. By asking a question, you open the channel to begin
verbal interaction. You show learners that you’re truly interested in them. Once you’ve established
that rapport and opened the communication channel, you can begin to use questions towards other
ends, such as: gaining information, uncovering motives and gaining insight, gaining participation,
starting the learner thinking, guiding the discussion and checking your understanding.
Essentially, all questions are either closed-ended (i.e. can be answered with yes, no, maybe,
either-or) or open-ended (i.e. require elaboration or explanation). In order to get your learners
talking so that they can discover the answer or solution to their own problem (rather than your
solution to their problem), you need to ask open-ended questions most of the time.
What might have happened if you’d done it the Have you ever thought about the possibility of
other way? changing the system that you’re using slightly?
So, what’s the next step that you’re going to take? Wouldn’t you agree that four is too many in this
case?
Where are you thinking of holding the meeting? Have you checked it yet?” (Can be answered yes/no)
How do you think your colleagues will react if you Balancing the high cost of air shipment against the
take that approach? fact that they’re going to need the shelter goods by
October latest, are you thinking of using air or sea
freight?
174
Giving feedback
Providing feedback skilfully is extremely important in coaching. As a technical coach, you will need to
provide constructive feedback as the learner masters new skills. The following tips will be helpful:
>> Show consideration. Give feedback with attention to the needs and feelings of the
learner. Everything you say, even a flippant remark, is likely to be noted, analysed
and taken personally. Choose your words carefully and remember that the tone of
your voice and your own body language also send important messages.
>> Think of the coachee. Be sure that your feedback is truly for the benefit of the learner.
>> Withhold judgement on the learner. Do not evaluate the personality
of the coachees or judge their intentions. Focus your feedback only on
their actions and the objective consequences of their actions.
>> Be specific. Don’t make sweeping or general statements: be specific
about particular actions and always give examples.
>> Select priority areas. Effective coaches may observe several areas that could be
improved, but they feed back to the learner only a fraction of what they observe.
They recognize that too much feedback can paralyse the learner, whereas too little
leaves him or her unsure of what to do next. Judging the right quantity of feedback
and concentrating on the most urgent areas for improvement is a critical skill.
>> Choose the timing. You should provide the feedback as soon as possible, while the learner’s actions
are fresh. Be sure that there is time for discussion, and remember to choose your words carefully.
>> Check for clarity. Make sure your feedback is clear and unambiguous. Ask the learner whether
your feedback was understood. Observe signs of misunderstanding on the part of the learner
as you give the feedback. If in doubt, ask the learner to paraphrase what you have said.
>> Reinforce the positive. Look for things that the learner is doing right and mention
them specifically. Point out the positive consequences of these actions. Positive
feedback is easier to give than negative feedback, and praise is extremely
important. Be sure to balance corrective feedback with positive comments.
>> Make it private. The psychological environment is also influenced by the physical environment:
choosing the right place to give feedback is as important as choosing the right time.
175
Tool 5c Action learning48
WHO? Developers of the learning initiative and, where possible, Instructional Designers
Instructions 48
Use the following tool to help plan the steps you need to take to prepare an action learning event.
Background information
Number of people:
176
Logistics
Facilities available:
(e.g. photocopier, computer,
overhead projector, flip charts)
Constraints:
Problem:
(What specific problem will the event address?)
Possible solutions:
(Write down ahead of time your opinion about
potential solutions, but make sure you don’t
direct the group towards your conclusions!
Use them only if the group is moving in an
inappropriate direction and needs some help.)
Reflection activities:
(Think of ways that you can get the group
to reflect on their conclusions/actions and
when you would use those approaches.)
Next steps:
177
Action learning questions
Part of facilitating an action learning event is being able to ask the right questions to help move
a group through the process. Here is a sampling of questions that can help for various stages:
178
Tool 5d Action plan for coaching49
WHAT? This tool provides the learner with a suggested action plan to be used as part of the coaching
process. The action plan can be formal or informal, but it is always recommended to be in a documented
format. The plan might be used to review progress on achievements.
Instructions 49
1. Review your learning objectives as agreed with the technical coach. How are
you going to address these objectives? Specifically, what are you going to do
to achieve your learning objectives? These are your actions. Ensure that any
action is consistent with the goal of your group, unit or organization.
2. List the specific action steps you plan to take, and the timeline
for doing so. Examples of action steps include:
>> discussing the topic with someone who knows a lot about it;
>> observing someone who is good in that function and analysing what they do;
>> trying out a new behaviour and seeing if it gives the results you want;
>> asking for feedback from a colleague or supervisor on your behaviour in that area;
>> reading something, like a book or an article on the subject;
>> studying or taking a course on the topic.
3. Next, ask yourself what type of support/resources you will need in order to accomplish
each step. Support might include: support by peers or managers, time to take an
e-learning course, opportunities to practise in a forthcoming meeting, etc.
4. Determine by what date you could reasonably expect to complete that step.
5. Finally, you may want to review the questions on the next page
to see whether your action items are appropriate.
179
ACTION PLAN
Learner’s name:
Learning objectives:
Will talk with my colleague, who is excellent in Availability of colleague Before July 15
statistics, in order to understand how data are
compiled and how I can better incorporate them in
my planning activities.
Resources
>> Who will carry out the proposed action or help with it (formally or informally)?
>> Are the skills for carrying it out available?
>> How much time do you expect this to take?
>> Are special materials or equipment required?
>> What is involved in obtaining them?
>> If so, how much adaptation is required?
>> Is continual monitoring or follow-through required?
>> If so, who will do it?
Adapted from F. Zenete and Associates, Managing Agricultural Research for Development
50
180
Implementation
>> Do you have the authority to implement the action?
>> If not, who does?
>> How do you expect to go about getting approval?
>> How much support do you expect for your idea?
Effects
>> Who will be affected by this action?
>> How will it affect them?
Environment
>> What factors in the organizational environment might interfere with your doing this?
>> What factors in the organization will support your effort?
181
Review of
Tool 5e coaching progress51
WHAT? This tool provides a template for a periodic review of progress between the technical coach and
the learner.
Instructions 51
Describe any changes in the context which occurred during or after the
coaching. Then list each objective from your learning plan and your assessment
of your progress in meeting each one, using the following scale:
5= objective attained; 4/3= satisfactory progress towards objective; 2/1= little or no progress.
Name of learner:
Name of coach:
Area of technical Describe the technical area in which coaching takes place.
coaching:
Context Describe any change in context since the start of the coaching.
Date:
182
Tool 5f Learner’s log
WHAT? A tool to record the learner’s main learning points during or at the end of the coaching period.
While the log may be personal to the learner, it can also be useful for the coach and learner to monitor
and finally review achievements, and for the action plan.
Instructions
Learner’s log
183
Guidance sheet - Essential tasks
Tool 5g for trainers/facilitators53
WHAT? This guidance sheet provides an overview of the essential tasks that a trainer/facilitator must be
able to perform to deliver effective face-to-face training workshops.
>> achieve the training objectives while maintaining good personal relations among the group;
>> play several different roles as required: subject-matter expert,
listener, learner, adviser, catalyst, devil’s advocate, etc.;
>> provide a structure and framework for learning;
>> persuade participants to get actively involved;
>> deal with challenges;
>> visually reinforce participant and trainer communication and
discussions by using flip charts, overheads, wall cards, etc.;
>> be sensitive to cultural and gender issues;
>> have a good sense of timing and knowledge of the subject matter; and
>> be sensitive to the needs of participants.
Adapted from UNHCR Facilitated Group Learning, 2001, Facilitation of Learning, 2002, and WFP
52
Toolkit http://learning.vam.wfp.org/pages/dpg/pBookContent.asp?c=chap9&p=94
184
Trainer’s task: content and process
The distinction between content and process is essential to the trainer. In a training activity, the
content is the subject matter – the knowledge, skills and attitudes that the participants will be
learning. The process is how the learning is structured and how the group and individuals work
together. It is vitally important to strike a balance between the two elements. Too much emphasis on
the content may damage the process; too much emphasis on the process may lead to poor results.
The trainer’s main content-related and process-related tasks are described below.
Content-related tasks:
>> Keep an overall view and think ahead.
>> Encourage balanced participation.
>> Ask for information and opinions.
>> Summarize and reformulate key points.
>> Clarify and elaborate.
>> Concentrate on key issues and stop digressions.
>> Test for consensus.
Process-related tasks:
>> Try to establish a relaxed, amicable and cooperative atmosphere.
>> Be patient and understanding.
>> Keep communication channels open.
>> Encourage individuals.
>> Be alert, maintain eye contact and encourage by friendly nods, supportive smiles, etc.
>> Maintain harmony by reconciling differences in opinions.
>> Seek to perceive and verbalize feelings.
185
Training and facilitation styles/strategies
The following diagram53 illustrates the tension between content-related and process–related tasks that
a trainer/facilitator must perform, together with the various techniques or styles that he/she can use
depending on the situation and context. A good trainer/facilitator normally is able to remain within
the central areas of the diagram, by continuously involving participants through asking open‑ended
questions, showing interest in their ideas and helping them reach understanding, while using his or
her own knowledge of the subject matter to guide the discussion. The diagram also illustrates the
types of considerations that often influence which strategy or style to use in a particular event.
Brainstorming Demonstrating
Cultural expectations
Needed ownership
Of outcomes
Your skills as
A facilitator
Freely adapted from the Facilitation Rainbow, John Townsend and Paul Donovan, Master Trainer Institute, 1997
53
186
Guidance sheet - Key phases
Tool 5h of a training workshop
WHAT? This guidance sheet describes the key phases of a training workshop.
>> Preferably during working days >> Where possible, use local resources. If the
>> Three to five days is an ideal length specialists have little training experience,
>> Four sessions a day, normally no more bring in an external facilitator or help the
than 90 minutes each with three breaks specialists to improve their training skills
>> Avoid formal evening sessions, if possible >> A small team of trainers/resource
>> Avoid heavy sessions immediately after lunch persons is almost always best
>> Include a half-day break on the afternoon >> Coordinate carefully during the workshop
on the third day if the workshop >> Don’t have too many trainers/resource persons
lasts for more than four days on the floor at any given time (maximum of three)
>> If you bring people from outside
the country, make sure they co-train
with local resource people
>> If one trainer, maximum 15 participants >> Avoid interpretation where possible
>> If two or more trainers, maximum 25 participants >> Try to use local trainers and the
>> Don’t have a workshop for fewer local language, if possible
than six participants
187
2. The start of the workshop
How you start the workshop is important because it will set the tone for the
day and will communicate your personal style to the audience. In general
terms, you should cover the following topics during the first hour:
>> Welcome participants.
>> Introduce yourself and the co-trainers.
>> Get participants to introduce themselves or each other.
>> Review the agenda and workshop objectives.
>> Discuss the workshop methodology and norms/ground rules.
>> Describe the programme.
>> Outline administrative and logistical arrangements.
>> Use ice-breakers to get participants into their first active exercise.
In general, for virtually all training situations with more than eight to ten participants,
it is essential to work in small groups. That is the key underlying method to ensure
maximum participation for effective learning, whether you are engaged in a
simulation, case study, problem-solving session, discussion or role play.
There are several techniques for working in small groups, such as the “World Café”
style and many others. See the “References and resources” section at the beginning
of this toolbox for guidance to information on using these techniques.
188
4. The end of the workshop
Ensure you finish on time and are able to provide:
>> a summary of what was covered;
>> time for questions, discussions and comments;
>> time for evaluation of the workshop (see Toolbox 7); and
>> time for a short activity that encourages use of the new learning in the workplace.
5. Anticipate follow-up
Although we recommend that you indicate throughout the workshop how participants can use their
learning on the job, you should emphasize this at the end of the workshop. You will also have to
announce what type of follow-up is envisaged so that they expect to be contacted or approached back
home. Also, inform them of any post-course evaluations in which they will be asked to participate.
How will you encourage participants to use their learning in the workplace? Depending on the
subject or programme, you could include assignments that must be done on the job before the
course is considered ‘completed’. Your goal throughout the workshop is to combat the typical
workshop scenario where participants just leave, forget and go back to business as usual.
Possible types of short activities you can ask participants to complete during or immediately after
the conclusion of a workshop but before they leave are presented in Tool 5i. Activities may include:
>> action plans;
>> informal reflection on existing challenges for implementing the new
learning in the workplace and ideas on how to overcome them;
>> reflection/paired discussion on how and to whom participants
intend to disseminate the new knowledge.
6. Wrap up
Another important aspect of finishing the workshop is to summarize and close the
subject matter for participants. Link the various activities and sections you discussed
to the overall objectives and include any outcomes the group identified.
189
Post-workshop reflection
Tool 5i and action plan
WHAT? This tool is intended to assist workshop organizers in supporting participants’ reflection on how
they want to go about using their newly acquired learning. It consists of a form and a template action
plan.
Instructions
1. What is the most important thing you have learned in this course?
Based on the “Evaluation of market-oriented farm management for trainers of extension workers”, Module 7, AGPS
54
190
2. How has this influenced your understanding of your work
as a/an (your job title) _______________?
3. Which part of this course do you want to implement when you return to work?
What are the opportunities and the challenges for implementing it?
191
5. What steps are viable/realistic for you and your organization to
use and institutionalize the new knowledge/know-how?
After reflecting on the issues above, participants should be encouraged to fill out the
Action Plan using the template on the next page (it can be customized as necessary).
The following questions may guide you in completing and validating the plan:
>> What will you achieve by implementing your plan? What is the result you are after?
>> What steps do you have to take to implement your plan? What resources do you need?
Who might be your partners in this plan? When can you complete each step?
>> Are the skills for carrying it out available?
>> How much time do you expect this to take?
>> Are special materials or equipment required?
>> If so, what is involved in obtaining them?
>> If so, how much adaptation is required?
>> Do you have the authority to implement the action? If not, who does?
>> How do you expect to go about getting approval?
>> How much support do you expect for your idea?
>> What factors in the organizational environment might interfere with your doing this?
>> What factors in the organization will support you doing this?
192
ACTION PLAN TEMPLATE
Resources/ Support I
Steps/Activities Progress check Target date
Partners will need
e.g. This is what I e.g. The following e.g. I may need e.g. I will review e.g. I will
will do as a result partners/ support from the progress on accomplish this
of my training organizations technical focal each step on the step by...
workshop.... might be valuable point in FAO to following dates:
resources to help me...
explore..I may
also need
equipment to
carry out my
activity.
193
Delivering effective PowerPoint
Tool 5j presentations (PPT)56
WHAT? The tool offers suggestions for delivering effective PowerPoint presentations.
194
7. Use an easy to read colour format.
>> Pick a colour combination that offers a strong contrast between the background and the
text. Dark backgrounds with light coloured text or light backgrounds with dark coloured text
work best. If you know you will be presenting in a poorly-lit room, use a light background.
Conversely, if you know you will be presenting in a well-lit room, use a dark background.
>> Limit the number of colours you use to no more than three. Colour can help add interest
and can be used to emphasize key points. However, too much colour can be distracting.
>> Text drop shadows should be black or a darker shade of the background colour.
8. Keep images simple.
>> Images (e.g. graphs, tables and pictures) can help strengthen your presentation by
conveying your message in an interesting and often easy-to-understand format.
Before using art, ask yourself whether the art enhances or clarifies your message.
>> Unnecessary art and animation can create visual clutter and distract from your message.
>> Some art, especially photos and other high-resolution images, can also
greatly increase the amount of memory needed to run the slide show.
>> Sometimes scanned images and photos are too large for a film recorder
to handle easily, leading to delays in the slide presentation.
195
Comparing face-to-face and
Tool 5k online facilitation tasks57
WHAT? This tool guides you through the key differences between face-to-face and online facilitation.
Developing an Develop agenda for face-to-face Develop agenda for online meeting.
agenda meeting.
Designing Design activities for face‑to‑face Design activities for online interaction.
group activities/ interaction.
processes Many activities may run parallel.
Most activities are sequential; topics are
discussed one-by-one, with occasional Set up different “threads” for parallel
use of parallel sessions. discussion of various topics.
Checking Check equipment, facilities and venue. Check online workspaces and tools.
facilities
196
2. Phase 1: Starting the meeting
The way a meeting begins sets the tone for all the processes which follow. If the group
starts in a welcoming environment with clearly stated purposes and processes that are
accepted by all participants, the group stands a good chance of achieving its goals. If
these foundations are neglected, it will be hard for the group to function effectively.
Creating a Introduce yourself and any presenters. Introduce yourself and any presenters.
welcoming
environment Ask participants to introduce themselves. Ask participants to introduce themselves.
If participants don’t know each other, If participants don’t know each other,
consider using “getting to know you” consider posting a short list of questions
exercises, such as getting people to talk for participants to respond to relating to
to their neighbours. social and/or professional information.
These introductions usually take place These introductions may take place
within the first hour of a meeting. over a week or so. In an ongoing online
community, they may need to be
repeated as new members join.
Statement of Present the goals of the meeting. Present the goals of the meeting.
purpose, getting
group to align on Check whether participants all agree with Check whether participants all agree with
purpose the goals, or whether they would like to the goals, or whether they would like to
revise them. revise them.
Recap any revisions and display on flip Give a deadline (usually within a day or
chart or whiteboard. two) for changes and additions.
Introduce Present the agenda and explain the Present the agenda and explain the
process and process. process.
get group
agreement. Check whether participants are happy Check whether participants are happy
with the overall process and whether they with the overall process and whether they
want any changes/additions. want any changes/additions.
Takes place at the beginning of the Takes place at the beginning of the
meeting, and as the need arises. meeting, and as the need arises.
197
3. Phase 2: Providing initial information and encouraging discussion
Presentations Presenters make oral presentations, often Presenters post presentations to online
using tools such as slide shows. workspaces. Presentations are usually
written, but may include multimedia
Presentations are at a fixed time. elements.
Report backs Small groups report back orally to the Small groups report in writing.
plenary on their discussions.
Groups have more time to prepare
Groups usually have little time to prepare reports.
their reports.
Report backs usually take place over a
Report backs usually take place within few days or more.
the space of a few hours or less.
Decisions and Participants are asked to indicate Participants are asked to indicate
agreement on agreement orally or by a show of hands. agreement in writing or by responding to
action an online poll.
Decisions are finalized in “real time”.
Except in synchronous online spaces,
decisions can take days or more to be
finalized.
198
5. Phase 4: Concluding
Summarizing Summaries are done quickly with little Except in synchronous online spaces,
decisions and time for reflection. the facilitator has time to process written
agreement information about decisions.
They are presented orally in real time.
Written summaries are posted to the
workspace.
Thanking Thank participants and outline group Thank participants and outline group
participants achievements orally. achievements in writing.
and recognizing
achievement Physical gifts may be presented in real Physical gifts are less common and
time to reward exceptional contributions. cannot be presented in real time.
Contributions tend to be rewarded
Closing session takes place over a short through positive written feedback.
period.
Participants may have days or longer to
post their own thanks and recognition of
achievements.
Keep group Return group to topic if they stray (e.g. Discussion needs to be kept on track, but
focussed “Let’s note this point and get back to it there is greater scope for having parallel
later.”). “off-topic” discussions in a clearly
marked separate thread.
If a new issue needs to be discussed
before progress can be made on the If a new issue needs to be discussed
original one, explain to the group why before progress can be made on the
this is necessary and when you will return original one, explain to the group why
to the old topic. this is necessary and when you will return
to the old topic.
Summarize discussions thus far, or ask
participants to summarize. Summarize discussions thus far, or ask
participants to summarize.
Focus needs to be maintained over a
short period. Focus needs to be maintained over a
longer period.
199
ACTIVITY FACE-TO-FACE ONLINE
Manage conflict Conflict may be overt or hidden. Conflict may be overt or hidden.
Not always possible to deal with conflict Easy and sometimes more appropriate to
away from the main group. communicate privately with individuals to
help resolve conflicts.
Draw people Get real-time input (e.g. “Let’s hear from Get input over a period of time (e.g.
into discussion the people who haven’t spoken.” or “Let’s hear from the people who haven’t
“Let’s go around and hear what everyone posted.” or “Let’s go around and hear
is thinking about this proposal.”). what everyone is thinking about this
proposal.”).
Bring together Think “on your feet” and identify Reflect on and synthesize written input
diverse threads and synthesize the main discussion and post to the online community (e.g.
of a discussion threads (e.g. “We’ve got two proposals “We’ve got two proposals – to target
– to target farmers directly, or target farmers directly, or target extension
extension officers.”). officers.”).
Interpret silence Visual cues help to understand silence No visual cues to interpret silence.
and take action – people may look hostile, tired or
where necessary confused. Many external factors (e.g. personal
issues, existing power dynamics,
External factors (e.g. personal issues, other demands on time and attention,
existing power dynamics) may contribute technology problems) may contribute to
to silence. silence.
Ensure that Write key points on flip charts, Communications are automatically
ideas, decisions, whiteboard or computer. documented, and participants can refer
etc. are recorded to them at any time.
Make sure that key discussion points are
physically visible to participants. Discussions may need to be summarized,
“cleaned up” and prepared in report
Identify note takers among the format.
participants.
Manage time Time is managed very closely with exact Except in synchronous online spaces,
times (e.g. “five more minutes”; “the time is managed in terms of days or
next session starts at 10 o’clock.”). weeks (e.g. “Please respond to these
questions by Tuesday”; “We need to
Limited flexibility. Facilitator may ask finish this thread by the end of the
group if they are willing to extend week.”).
sessions or meeting by minutes or at
most a few hours. There are often greater possibilities of
extending meetings by days or even
Participants may need to leave the weeks.
meeting at a set time, or the venue may
be unavailable. Time constraints relate to deadlines
rather than physical presence.
200
ACTIVITY FACE-TO-FACE ONLINE
Monitor energy Physical and psychological energy needs Psychological rather than physical energy
levels to be sustained at a high level over a has to be sustained over a longer period.
short period.
No physical cues to identify low energy
There are often obvious physical levels.
indicators of low energy, such as
yawning, slumping in seats and reduced Participants are in diverse physical
participation and focus. contexts.
Facilitator can introduce a physical break Facilitator can introduce a new activity to
for all participants at the same time. re-energize the group.
Manage Participants occupy the same physical Participants are located in different
environment environment. physical environments.
Ensure follow-up By e-mail, phone and other media By e-mail, phone and other media
on decisions and
action
201
Toolbox 1 2 3 4 5 6 7 8
Guidance sheet
6A Developing a plan for follow-up support CORE 203
Tools
6B Template for a follow-up plan CORE 206
6C Types of follow-up measures CORE 207
Guidance sheet - Developing a
Tool 6a plan for follow-up support
WHAT? This guidance sheet offers some suggestions for developing a follow-up plan. It should be used
in conjunction with the template follow-up plan (Tool 6b).
WHEN? Plan should be developed during the design phase of the learning initiative, and should be
validated and acted upon after the initiative has been completed.
Background
The learning process that is enabled through a learning initiative should be viewed as part of a
larger, ongoing process rather than as a single learning event. Once a learning initiative has been
completed, participants return to a work environment in which they are asked to put in place
what they’ve learned. Systematic follow-up ensures that the learning acquired is transferred to
the work environment and has an impact on the organization in which the learners work. Hence,
the follow-up events and strategies that follow the actual learning initiative should be considered
as important and integral a part of the learning initiative process as the initiative itself.
203
What is the purpose of follow-up activities?
>> promote the ‘transfer’ of learning from the individual to the organizational level;
>> motivate participants to help diffuse new learning and innovations
within the work context, including building support to integrate learning
and innovation into existing systems or processes; and
>> support the development of learning networks and peer support groups which can provide
sustained on-the-job assistance in implementing the content of a given learning initiative.
The follow-up plan is an outline document which helps you lay out the most appropriate
follow-up measures for your learning initiative. Using this plan, you can figure out what kinds of
measures you wish to use, the kinds of resources you will require and the best roll-out strategy
for your plan. In short, the follow-up plan helps you establish a coherent strategy for a systematic
and realistic approach to promote sustained engagement in the topics of your initiative.
To maximize opportunities for follow-up, begin preparing the ground before the implementation of
the learning initiative. Ideally, follow-up should be planned during the design phase of the initiative
and included in your design outline. The plan can be further refined as the initiative is delivered and
additional information is disclosed by participants. By the design stage, however, you should already
have a clear picture of the type of support that might be required for participants to implement and
use their learning. Budget provisions for the follow-up should obviously be made in advance.
To build a plan, it is useful to first determine the specific kinds of activities and
opportunities in the work environment that can be used to apply or internalize the skills,
knowledge or attitudes that participants will have acquired upon completion of the
learning activity. To this end, you can use the information you’ve gathered throughout
the previous steps of the learning management cycle, as shown in the table below:
204
What information you may have gathered already... When? Link to tool
>> Whether there are contextual or >> During the initial >> Context analysis
organizational conditions that are likely assessment (Tool 1a)
to block implementation of learning of context
(e.g. incentive issues, resources)
>> Whether there is a need to work at a higher level >> When working >> Invitation note
(e.g. through advocacy) to ensure support by with (government) (Tool 2a)
managers for the results of the learning initiative partners to >> Application
>> The expectations of prospective identify and select form (Tool 2b)
learners, including in terms of post- participants
initiative support expected
>> Whether there is a need to provide refresher >> When assessing >> Learning needs
courses to ensure a standard level of soft and learning needs assessment
functional skills (e.g. project management, checklist (Tool 3g)
partnering, information and knowledge) >> Participants’
required for the uptake of technical skills. benchmark
>> What aspects from the physical and evaluation
social work setting may represent an questionnaire
obstacle to implementation of learning (Tools 7l and 7m)
(e.g. gender, team location)
>> Ways in which participants plan to
apply things they learn upon returning
to their work environment
>> Support participants will require in order to
apply the things they hope to learn once
they return to the work environment
>> Difficulties that participants are experiencing >> During delivery of >> Action plans
with grasping a specific topic or skill learning initiative (Tool 5d)
>> Theoretical issues that need supervised practise, >> Open feedback and
guidance and support to be applied correctly evaluation forms
>> Opportunities and challenges (Tools 7l and 7m)
for implementing learning >> Post-workshop
reflection and
action plan
(Tool 5i)
205
Tool 6b Template for a follow-up plan
Learning initiative: Example: Workshop on pesticide management for staff of national chambers of agriculture
Delivery date:
1. Example: Outdated work processes in learners’ work environment, which makes it difficult for learners to
implement new tasks
2.
3.
4.
Opportunities in the work environment that can be used to apply the new learning
2.
3.
4.
Follow-up support that can be provided to address constraints and leverage opportunities in work setting
2. On-the-job technical assistance to update work processes and test them in the context of the new project
3.
4.
In six months
206
Tool 6c Types of follow‑up measures
WHAT? An overview of common types of follow-up measures, together with examples of FAO’s use of
those measures, where applicable
Follow-up visits with There are ongoing missions in the area which can
on-the-job coaching and provide a cost-effective opportunity for observation,
mentoring additional guidance and problem-solving where
needed.
Study tours Tours can add value to a learning experience by TCI study tour in
allowing participants to see the application of Mali‑Togo
skills in a different context and build alliances and
networks. Tours require resources, although they
can be partially self-financed by participants.
Online coaching/ Learners are likely to encounter problems in TCI online mentoring
mentoring support applying learning that can be easily solved through support to national
online consultations. counterparts after
face‑to-face meetings
207
Follow-up measure Best used when.... FAO example
Face-to-face alumni Learners are in geographic or physical proximity Alumni network in FFS
groups/networks and can benefit from peer-to-peer follow-up in Mali
support.
Alumni online forums or Learners are geographically dispersed and can GiZ thematic alumni
list serves best learn from each other. Budgets for follow-up networks
support are limited.
Online knowledge
networks and
communities of practice
Help desks or focal There is an ongoing need for technical advice or Q&A service and
points knowledge resources that can be provided online. online resources –
FAO-GiZ Blended
Learning Programme
for Country Teams and
Food Security Thematic
Working Group
Members
Web-based resources Resources are readily applicable and can reinforce EASYPol resources
and toolkits theoretical concepts learned through other modes.
Works best with a help desk/coaching component.
208
Toolbox 1 2 3 4 5 6 7 8
Guidance sheets
7A Kirkpatrick’s results evaluation framework CORE 211
Tools
7B Designing an evaluation plan CORE 215
Guidance sheets
7C Evaluation of participants’ satisfaction: methods and tips CORE 221
Tools
7D Open feedback form CORE 223
7E End-of-course evaluation of face-to-face training CORE 224
7F Daily evaluation of face-to-face training RECOMMENDED 227
7G Mid-term evaluation of face-to-face training RECOMMENDED 228
Guidance sheets
7H Evaluation of learning results: formal and informal methods CORE 230
Tools
7I Assessing changes in learners’ perception CORE 232
7J Example of pre/post-course knowledge test RECOMMENDED 235
Guidance sheets
7K Evaluation of changes in participants’ behaviours CORE 238
and practices: methods and tips
Tools
7L Participant benchmark evaluation CORE 240
7M Example of participant benchmark evaluation RECOMMENDED 244
Process evaluation
Overall
>> Kirkpatrick, D. L. (1968). Evaluating Training Programmes. San
Francisco, Berrett-Koehler Publishers, Inc.
>> Using Training to Build Capacity for Development, An evaluation of the World Bank’s project-based
and WBI training http://siteresources.worldbank.org/EXTTRABUICAPDEV/Resources/full_doc.pdf
>> If I were the facilitator – Assessing the quality of facilitation, taken from training
materials of Market-oriented farm management for trainers of extension workers,
2007 (AGPS) http://www.fao.org/docrep/011/a1298e/a1298e00.htm
>> FAO e-learning course, Impact Assessment of Large-scale Food Security Programme,
available at: http://www.fao.org/spfs/learning-from-results/e-learning/en/
Level 1
>> A good example for evaluating various aspects of a training course is included in the FAO training
materials for Market-oriented farm management for trainers of extension workers, Module 7,
Training Evaluation, pp. 23-30 (AGPS) http://www.fao.org/docrep/011/a1298e/a1298e00.htm
Level 2
>> About writing test items
http://www.park.edu/cetl/quicktips/writingtest.html
http://www.go2itech.org/HTML/TT06/toolkit/evaluation/dev_eval.html
>> Changes in perception test form. FAO training materials for Market-oriented farm management
for trainers of extension workers, Module 7, 2007 (AGPS). The module also contains useful
examples of final training examination. http://www.fao.org/docrep/011/a1298e/a1298e00.htm
Level 3
>> Reflection, Consultation, Action template. Taken from Module 7 of the Market-
oriented farm management for trainers of extension workers materials, 2007
(AGPS) http://www.fao.org/docrep/011/a1298e/a1298e00.htm
>> Knowledge, Attitudes and Practices (KAP) Survey of the project “Rehabilitation and sustainable
development of fisheries and aquaculture affected by the tsunami in Aceh Province,
Indonesia”(Project OSRO/INS/601/ARC), 2009
http://typo3.fao.org/fileadmin/user_upload/oed/docs/OSROINS601ARC_2010_ER.pdf
>> About using Knowledge, Attitude and Practice (KAP)
evaluation and Most Significant Change (MSC):
http://www.anthropologymatters.com/index.php?journal=anth_matters&pa
ge=article&op=viewArticle&path%5B%5D=31&path%5B%5D=53
http://www.stoptb.org/assets/documents/resources/publications/acsm/ACSM_KAP%20GUIDE.pdf
http://www.kstoolkit.org/Most+Significant+Change
210
Guidance sheet - Kirkpatrick’s
Tool 7a results evaluation framework58
WHAT? This guidance sheet provides an overview of Kirkpatrick’s 4-level evaluation framework.
WHEN? Selection of the appropriate evaluation level should be done at the design stage of the learning
initiative.
58
Adapted from World Bank, with permission. The overview is based on Kirkpatrick, D. L. (1968).
Evaluating Training Programs. San Francisco, Berrett-Koehler Publishers, Inc.
211
When to measure:
At the end of the initiative
How to measure:
Group or individual participant interviews or participant questionnaires. In order to capture
possible areas for improvement of the training course, questionnaires should pose specific
questions about the quality of each of the lecturers and each of the course topics.
Level 2 - Learning
When to measure:
At selected intervals during and/or at the end of the learning initiative, when
learning is still fresh in participants’ minds. Tests also may be administered at the
beginning of the initiative in order to compare against end-of-initiative tests.
How to measure:
Tests, in-class projects, exercises and assignments or participants’ observations.
212
Level 3 – Behavioural changes and
workplace performance outcomes
When to measure:
From six months to one year after completion of a learning initiative. Evaluation should occur
a sufficient amount of time after the end of the initiative to ensure that participants have
had the opportunity to use their learned skills and practise them in the work setting.
Evaluations generally should not be done more than two years after the initiative, because
participants may have difficulty remembering how the learning experience affected their work.
How to measure:
>> Interviews or surveys of participants, their supervisors, policy-makers and/
or others who can report on participants’ performance on the job.
>> Observation of participants in their workplaces. For example, if participants were
trained to perform certain tasks using a different procedure, their use of the new
procedure can be monitored and compared with their practice before the training.
213
Level 4 - Organizational impact
When to measure:
Minimally one year after completion, or realistically at the end of the project/programme;
however, organizational impact generally will only be evident much later.
How to measure:
Interviews of organization management, personnel, clients or other stakeholders
before and after the learning initiative in order to compare organizational
performance/success at achieving organizational objectives.
>> There is generally an attribution problem. Even where organizations have changed,
it is often difficult to know the extent to which this was the result of learning activities
rather than other complementary interventions or changing conditions.
>> There may be a time lag problem. Sometimes the impact of
training on organizations is only evident years later.
>> It may be hard to find reliable quantitative measures for some forms of organizational change.
Because of the difficulties listed above, this Toolbox will not elaborate further
on Level 4 evaluation. Capacity Development Learning Module 4 will cover
some aspects related to organizational development and change.
214
Tool 7b Designing an evaluation plan59
WHAT? This tool describes a template to define the goals of the evaluation and the elements that should
be incorporated into the evaluation plan to achieve those goals. It consists of: (1) a worked example of an
evaluation plan; (2) an evaluation framework; and (3) a template of an evaluation plan
WHEN? An evaluation plan should be formulated before delivery, as part of the design process for a
learning initiative
Instructions 59
You may use this format to aid you in designing your evaluation plan, or you may fill in
the data on the template that follows. The level of detail in your evaluation plan should
reflect the scope of the evaluation and the cost/importance of the learning activity. The
example below is tailored to evaluating a beekeeping workshop for smallholders.
Do you envisage evaluation to measure the initiative’s results or the quality of learning processes? What is the
purpose: accountability to stakeholders or learning for improvement of future training activities?
Example:
The evaluation aims to assess both the quality of learning management processes and the results of the learning
initiative in terms of participants’ behaviour change in their work settings.
59
Adapted from World Bank, with permission
215
2. Evaluation objectives
What is the evaluation trying to achieve in concrete terms? The evaluation objectives should be based on
carefully defined desired outcomes and aligned with the programme’s learning objectives.
Example:
The evaluation will assess the extent to which:
1. the learning content was relevant to the needs of smallholders.
2. participants have acquired the necessary knowledge and skills to:
>> evaluate commercial and smallholder beekeeping;
>> provide needed policy and extension support to beekeepers; and
>> stimulate and train traditional beekeepers in their home countries by organizing
local workshops in modern methods of beekeeping management.
3. participants have been able to successfully implement learning in a way
that supports growth of beekeeping sector in their regions.
4. learning management processes have been in line with good practice standards.
3. Evaluation questions
Questions should be aligned with evaluation objectives and can be divided into evaluation levels.
Example: (This is a partial list of questions relating to the above objectives 1 and 2).
1. To what extent did participants learn required knowledge in:
>> hive management and production;
>> crop pollination;
>> queen rearing;
>> economics of production;
>> mechanization and technology;
>> sanitation and health?
2. To what extent have participants used learning upon their return to the workplace to:
>> evaluate commercial and smallholder beekeeping;
>> provide needed policy and extension support to beekeepers; and
>> stimulate and train traditional beekeepers in their home countries by organizing
local workshops in modern methods of beekeeping management.
216
4. Indicators and results measures
For results evaluations, each indicator should measure impacts, outputs and outcomes that are relevant in the
context of the results chain for the training programme.
What methods are you envisaging to use (e.g. pre-course and post-course questionnaires, interviews, group
feedback)? Who will collect the data and when?
Example:
Post-course questionnaires will be administered by the workshop facilitator at the end of the workshop.
Example:
Evaluation results will be communicated to internal and external stakeholders on the occasion of a mid-term
report on the status of the project.
The evaluation framework on the following page can help define evaluation objectives,
indicators and the appropriate data source for the type of learning initiative.
217
Evaluation framework60
Adapted from WFP Food Security Assessment Facilitator’s Toolkit, page 66 http://learning.vam.wfp.org/documents/Docs/FTK.pdf
60
218
Results‑based Kirkpatrick Type of
management Description Data source Possible indicators
level learning mode
term
Level 3 Measures Self-study Interviews, >> % of individuals able
how learning surveys with to carry out tasks
has affected individual independently up to
the way participants and agreed standards
participants their superiors
perform or F2F workshops Participant >> % of participants
behave in the benchmarking confident to
work setting evaluation disseminate their
forms, knowledge to others
observation,
>> Number of new
interviews,
activities or
surveys
products put in
On-the-job Observation place or adapted to
on the work fit new situations
setting, on-the-
job learning >> Number of projects
achievement or programmes
Outcomes form led by national
Online partners with
collaborative increased quality
61
61 This level of evaluation is rarely carried out as there are very important attribution problems with indicators.
For example, although a review of mandates and priority could be an indication of renewal and transformative
processes, it does not necessarily mean that this has resulted in better organizational performance or results.
Indicators at this level are therefore only suggested to encourage further reflection and discussion.
219
Template of an evaluation plan
220
Guidance sheet - Level 1
Tool 7c evaluation of participants’
satisfaction: methods and tips
WHAT? This guidance sheet describes methods and tips for carrying out a Level 1 evaluation.
WHEN? Design of appropriate evaluation instruments should start at the design stage of the learning
initiative, when developing the evaluation plan.
Among other things, participant satisfaction questionnaires can help you determine:
Evaluation methods
There are different techniques for obtaining feedback from learners in the context
of workshops, self-study activities and on-the-job learning events.
Questionnaires (or happy sheets) are most commonly used to solicit participant reaction.
Participants may not feel comfortable criticizing a colleague’s work, so it is better to keep
questionnaires anonymous where feasible. They are typically used for workshops, but can also
be used for distance learning programmes, online collaborative events or on-the-job situations
like study visits and coaching. Examples of questionnaires are provided in Tools 7e, 7f, 7g.
221
Open feedback forms are another way to gather written feedback from participants.
For informal learning situations in which you have only one or a few participants, you
may provide a ‘blank sheet’ format – a page with an opening sentence that encourages
them to provide as many comments as they wish on any aspect of the programme
or the initiative. An example of open feedback form is provided in Tool 7d.
Interviews can be conducted with participants after an event. This is more useful for one-on-one
situations (e.g. coaching or job shadowing) because it would be difficult and time-consuming to
interview enough participants from a group event. You can arrange separate telephone or face-to-face
meetings with the learner and the coach/experienced colleague. An independent party who was not
involved in the event will need to conduct the interviews. To conduct effective feedback interviews:
Group feedback can be useful for a group or team event. Arrange small groups either during or at
the end of event to provide feedback. You may need to give the group specific areas to address to
get the group started. Although there is the danger that one member of the group will sway others’
opinions, this is a good way for participants to develop their feedback into more concrete suggestions.
Where feasible, use daily or mid-term evaluation forms alongside the end-of-course form (Tool 7f or
7g) to enable participants to assess individual course components while they are still fresh in their memory.
Supplement closed-ended quantifiable questions with open-ended questions that enable participants
to freely list concerns and identify the most positive aspects of the course. Asking for a limited number of
“most ________” and “least ________” aspects of the course often results in more focused, richer answers
than simple open-ended questions. For example, ask “What were the three elements of the course that
you found most useful?” rather than “What course elements of the course were most useful?”.
Consider using rankings alongside or in place of ratings. Ratings scales (e.g. asking participants to rate
the usefulness of a lecture on a scale of 1–5), are generally used to ascertain the merits of various course
elements. Frequently, however, participants fill in rating scales in a uniform manner that makes it difficult to
differentiate between better and worse aspects of the course. Another possibility is using questions that
ask for rankings. For example, rather than asking participants to rate each course session, ask participants
to list the three most useful and three least useful course sessions. This may give a better indication of
which course modules are relatively weaker and could be improved.
222
Tool 7d Open feedback form62
WHEN: At the end of a learning initiative, when you prefer to leave the feedback areas open
Instructions 62
>> KEEP IT! – For the special things that you appreciated in the initiative
>> CHANGE IT! – For the things that you did not appreciate
>> ADD IT! – For the suggestions that you would like to offer
>> WHAT I WILL REMEMBER! – For the points that you will retain after the event
KEEP IT! I appreciated this: CHANGE IT! I did not appreciate this:
ADD IT! These are my suggestions: WHAT I WILL REMEMBER: I expect to retain these
points:
62
Based on the FAO facilitation skills I course
223
End-of-course evaluation
Tool 7e of face-to-face training
WHEN: At the end of the workshop, when you want to have specific feedback on certain aspects of the
training.
Instructions
Please complete this questionnaire to help us improve our activities in the future. Please
be honest and open. Your responses — no matter how positive or negative — are valuable
to us. To keep them anonymous, please do not write your name on the form.
1. The following statements relate to your satisfaction with the course. Please rate
them on a scale of 1–5, where 1 = “I do not agree” and 5 = “I totally agree”.
If the statement is not relevant for your course, please check “n/a”.
1 2 3 4 5 n/a
224
1 2 3 4 5 n/a
2. What did you find most useful in the activity and why?
________________________________________________________________________________________
________________________________________________________________________________________
________________________________________________________________________________________
3. What did you find least useful in the activity and why?
________________________________________________________________________________________
________________________________________________________________________________________
________________________________________________________________________________________
4. On what topics, if any, would you rather have spent more time—whether
or not they were addressed in the activity?
________________________________________________________________________________________
________________________________________________________________________________________
________________________________________________________________________________________
5. On what topics, if any, would you rather have spent less time?
________________________________________________________________________________________
________________________________________________________________________________________
________________________________________________________________________________________
225
6. Please name up to three things that we could do to improve similar activities in the future.
1. ____________________________________________________________________________________
2. ____________________________________________________________________________________
3. ____________________________________________________________________________________
7. Please list up to three things that you intend to do as a result of your participation in this activity:
1. ____________________________________________________________________________________
2. ____________________________________________________________________________________
3. ____________________________________________________________________________________
8. What type of follow-up could help you make this activity more useful once you return to work?
________________________________________________________________________________________
________________________________________________________________________________________
________________________________________________________________________________________
________________________________________________________________________________________
________________________________________________________________________________________
9. Please list any further comments you may have on any aspect
of the course personnel or classroom facilities.
226
Daily evaluation of face-
Tool 7f to-face training63
WHO: Participants
WHEN: At the end of each day of a face-to-face training, when the initiative lasts more than two days
2. What did you learn during today’s sessions that you anticipate using in your work?
4. What is the most valuable thing you learned today (knowledge or skills)?
Thank you.
63
Adapted from: Levels of evaluation based on Kirkpatrick, D., 1994, Evaluating
Training Programs: The Four Levels, San Francisco: Berrett-Koehler.
227
Mid-term evaluation of
Tool 7g face-to-face training
WHO: Participants
WHEN: In the middle of a face-to-face training, when the initiative lasts more than three days
1 2 3 4 5
2. In this training, how do you rate your participation in the small groups?
1 2 3 4 5
3. In this training, how do you rate your participation in the large groups?
1 2 3 4 5
64
Developed by Rosalie Husinga Norem and Patricia Colbert for SEAGA workshop held in Egerton University, Kenya 1998
228
4. What training methods have been used in this training which
really motivated you to get involved? (Please list)
5. What training methods have been used in this training which did
not encourage you to get involved? (Please list)
6. What tasks/activities have we done that were very interesting? (Please list)
7. What tasks /activities have we done that were not very interesting to you? (Please list).
8. Do you feel that there are some participants who do most of the talking? (Yes or no)
9. What content would you like to have further clarified before the end of this training?
10. What additional content would you like to have covered before the end of the training?
11. Do you have any comments or suggestions you would like to make?
229
Guidance sheet - Level 2
Tool 7h evaluation of learning results:
formal and informal methods
WHAT? This guidance sheet describes formal and informal methods for carrying out a Level 2 evaluation.
WHEN? Design of appropriate evaluation instruments should start at the design stage of the learning
initiative, when developing the evaluation plan.
Evaluation at this level can be done on an informal or formal basis. It is also known
as ‘learner assessment’ and allows you to determine the extent to which:
Informal assessment
Trainers and facilitators should frequently check informally during learning initiatives to
ensure that learners comprehend the knowledge and skills being taught. At the simplest
level, this is done by asking questions during the initiative/event to ensure participant
comprehension. In-class projects, exercises, activities or presentations also can be
used to assess participants’ absorption of course materials and ability to use learned
concepts. Informal assessment of learning is an integral part of any learning activity.
230
Formal assessment
Formal assessment takes the form of learning tests or quizzes at the conclusion of a learning event,
and may be useful where it is important to ensure that learners have grasped certain fundamental
concepts, ideas or skills in learning programmes. In certain circumstances, such as in self-paced
study, formal assessment is important in supporting learners’ motivation because it will allow
them to monitor the results of their own efforts as they proceed in the learning. In other types of
learning initiatives there may be challenges in assessing learning. These challenges may include:
>> Not all types of learning are easily measured. The easiest types of learning to measure
are knowledge or hard skills. Much harder to measure are soft skills such as those imparted
in management or leadership training or learning activities aiming to shift attitudes.
>> Pre-tests may be necessary to attribute participant knowledge and skills. Without pre-
tests, it is difficult to know whether correct test answers reflect learning or prior knowledge.
>> Higher-level participants may object to being “tested”. Testing may be viewed
by some participants as being disrespectful to their rank or position.
>> Learning scores are likely to reflect, in part, the skill of participants in taking tests and
not just participant learning. This may be a problem particularly with participants who are
not fully used to or comfortable with learning environments. When learning is tested, care
should be taken to ensure that tests are administered in ways that don’t discourage learners.
231
Assessing changes in
Tool 7i learners’ perception65
WHAT: This tool presents a method for assessing learners’ acquisition of new knowledge and changes
in their perceptions and attitudes (Level 2 learning). It also provides an indication about which follow-up
measures could reinforce changes.
WHO: Participants
WHEN: At the beginning and at the end of any learning initiative, after appropriate customization of the
questions
Instructions 65
Taken from Market-oriented farm management for trainers of extension workers materials, Module 7, AGPS 2007
65
232
The following example relates to the topic of farming, farm
management and market-oriented farm management.
WORKSHEET A
Don’t
Statement Agree Disagree know
Other comments:
233
WORKSHEET B - Changes in perception
At the beginning of the session/course/event, the following statements related to farming, farm
management and market-oriented farm management were asked. Now that this programme
has come to a close, read each statement again; think briefly about it and decide whether you
“agree” or “disagree” with the statement or if you “don’t know”. When finished, compare these
with your answers from the Worksheet A. Has there been a change? If so, what changed and
why? If not, why not? Record your answers in the space provided under “change/no change”.
Don’t
Statement Agree Disagree know
Other comments:
234
Example of pre/post‑course
Tool 7j knowledge test66
WHAT? This tool provides an example of a pre/post test to assess acquisition of new knowledge/skills
(Level 2).
WHEN? Design of appropriate evaluation instruments should start at the design stage of the learning
initiative, when developing the evaluation plan.
The course team is always looking for ways to serve you better. Please take a moment to complete this
short survey. It will help us know how we’re doing and how we can better serve your needs in the future.
66
Taken from FAO’s online collaborative course on climate change, agriculture and food security aimed at strengthening the
knowledge capacity of decentralized offices to address climate change. For post-course evaluation, the same form is used.
235
2. How confident are you Not A little Somewhat Very
in your ability to do Confident
confident confident confident confident
each of the following?
236
How confident are you in Not A little Somewhat Very
your ability to do each Confident
confident confident confident confident
of the following?
Thank you for completing this evaluation. We appreciate your input as we make every effort to improve our course.
237
Guidance sheet - Level 3 evaluation of
Tool 7k changes in participants’ behaviours
and practices: methods and tips
WHAT? This guidance sheet describes methods and tips for carrying out a Level 3 evaluation.
WHEN? Design of appropriate evaluation instruments should start at the design stage of the learning
initiative, when developing the evaluation plan.
Evaluation of performance outcomes, i.e. how participants have used their new learning on the job,
is the most important level of results evaluation. It is also, however, one of the most difficult levels
of evaluation. It’s normally carried out from six months to one year after the initiative has ended.
There are specific challenges to Level 3 evaluation. The major difficulty stems from the
fact that post-course questionnaires or surveys sent to former participants often have very
low response rates and do not produce very in-depth information about how participants
have used learning in the workplace. Other forms of evaluation may be prohibitively costly
or logistically difficult, particularly when learners are distributed geographically.
However, it is recommended that some form of Level 3 evaluation be done at the end of each
learning initiative. Below are some tips for improving the design of questionnaires and managing
the evaluation process to raise response rates and enhance the quality of information gathered.
238
Tips for designing post-initiative questionnaires and surveys
>> Inform participants at the end of the initiative that you will be sending
them questionnaires. Stress the importance of filling out these questionnaires
and inform participants when they should expect to receive them.
>> Administer post-initiative questionnaires from six months to one year after the
course is completed. Earlier than six months may be too early to report back on
how learning was used, and after one year, participants’ memories about learning
content and their willingness to respond to questionnaires may be reduced.
>> Use open-ended questions to ask for specific information about how learning was
used. Supplement closed-ended questions asking participants to rate their use of learning
with open-ended questions asking for specific details about how they have used learning
in their work situation. It will generally be possible to get a much better understanding
of how learning was used from responses to open-ended questions than closed-ended
ones. This information will help coordinators develop a much better understanding of
how learning is being used by learners, how it may be possible to provide supplementary
support and how future initiatives can be better adapted to participants’ needs.
>> Ask participants to rate the extent to which their ability to implement learning was
affected by various constraints/support factors. Among possible constraints that can
be rated are: support of managers, support of peers, availability of financing, availability
of equipment, understanding of how to apply learning in the work setting, presence of
supportive policy structure and presence of supportive organizational processes.
>> Remember that the purpose of post-course evaluation is not only to rate the results
of learning activities but also to develop a better understanding of how learners use
learning and how they may be better supported in implementing learning. In many
cases, participants may come from very different work situations and may use learning
in very different ways. Gaining an understanding of how learning is used will make it
possible to better target future interventions to the range of participants’ needs and to
ensure that supplementary capacity support is provided to learners’ organizations.
Tools 7l and 7m describe one possible method for carrying out Level 3 evaluation through
questionnaires. Another method, known as Knowledge, Attitudes and Practice (KAP) surveys,
can be useful to identify changes in people’s attitudes and practice in a certain area after
they have participated in a learning initiative. The “Reference and resources” section at the
beginning of this Toolbox includes some references on how this method has been used. The
Capacity Development Learning Module 2 also refers to the KAP survey in its toolbox.
239
Level 3 participant
Tool 7l benchmark evaluation67
WHAT: This tool presents a comprehensive, participatory system for needs assessment and learning
evaluation. It aims to provide a simple, cost-effective way to gather information on participant learning
needs and on the results of learning on workplace performance and organizational capacity. It does so
through three inter-locking questionnaires.
WHO: Participants
WHEN: The three questionnaires are filled out at three different stages: one at pre-initiative, one at the
end of the initiative and one six months to one year following the initiative.
Instructions 67
Stage 1. Pre-initiative needs assessment and setting learning goals. Before the learning initiative,
participants are asked to write three individual learning goals for the course. For each of their individual
goals, they should be asked to explain how they would like to use this learning in their work settings.
This can be done as part of pre-course application or registration forms submitted in writing or online.
Asking participants to design their own goals will increase their sense of ownership of and engagement
in the learning process. It also helps participants reflect on how learning should support their work and
may thus make them more proactive in the classroom to ensure that learning meets their needs. It is
also possible to ask learners’ supervisors to approve the learners’ goals in writing or to add comments
to the learners’ goals, in order to help prepare support for implementation of learning in the workplace.
The method was developed by Ms.Aliza Belman Inbal, who agreed to contribute it to the Module.
67
240
Stage 2a: End-of-initiative learning assessment. At the end of the initiative, participants
should be given their individual learning goals from the pre-course needs assessment and
asked to rate the extent to which they have achieved their goals and to comment on why.
It should be noted that the learning goal table is meant to supplement, not replace, a
standard end-of-course participant satisfaction questionnaire. Relating participant ratings
of learning to their individual learning goals, rather than simply asking participants to rate
course learning overall, gives a much clearer picture of the extent to which participants
believe they have acquired the knowledge and skills which are important to them.
Stage 2b: End-of-initiative setting goals to use learning. Participants are asked at the end of course
to choose the three most useful things they gained from the course, and to explain how they believe
they will use this learning in their work settings. Setting individual goals on how participants intend to
use their learning enhances participants’ sense of ownership over the evaluation process and provides
much richer and more nuanced understanding of the ways in which participants intend to use learning.
In addition, having participants set use goals before returning to the work environment will make it
possible to better gauge to what extent factors in the work environment hinder or help learning use.
It should be noted in this context that intention to use learning is not the same using learning, and
thus a third, post-course assessment of learning use is an essential part of participant benchmarking.
Stage 3: Assessment of how learning has been used. Six months to one year after the course,
participants are sent their responses from the end-of-initiative questionnaires (Stage 2b) and
asked to rate the extent to which they have been able to use learning as anticipated. For each of their
goals on how they intended to use learning, they are then asked to provide input on what factors
in their work settings supported or constrained implementation of learning. The learning use table
filled out by participants at the end of the course should be sent to them either electronically or by
mail, with a postage-paid, addressed, return envelope. Relating the questions about the enabling
environment to support implementation of learning to the information about the participant’s
goals on using learning will give a much clearer picture of where there are obstacles to learning
implementation. This information can then be used to better support these or future learners.
241
Personal benchmark evaluation template
These tables should be filled out in three stages. They can be administered electronically or on paper.
Questions have been shaded to differentiate between those to be administered at different stages.
What knowledge, For each of the goals Now that you have Comments
skills, or other benefits listed above, how do completed the
would you most like you expect to use initiative, please
to gain from the what you have gained review the goals you
initiative? Please be here once you return set for yourself. For
as specific as possible. to your workplace? each goal, please tell
You may list up to us the extent to which
three goals.68 the initiative met your
needs. Please rate on
a scale of 1–4 where
1 = “did not meet my
needs” 4 = “fully met
my needs”. If the goal
you set was not relevant
for the course, please
select N/A.
Goal 1: 1 2 3 4 N/A
Goal 2: 1 2 3 4 N/A
Goal 3: 1 2 3 4 N/A
68
A simpler way is to ask participants to choose individual learning goals from a list of topics rather than asking them
68
to write their own goals. The advantage to this method is that it can be used to compile quantitative data with
regard to which topics are of greatest interest to participants. The disadvantage is that, by limiting responses to
goals formulated by the training manager, it will not be possible to capture unanticipated learning goals.
242
Stage 3 – End of iTnitiative (how learning has been used)
At the end of the initiative, you were asked to share with us the three things you acquired in the course that you
thought would be most useful to you back on the job, and how you expected to use these things in your work.
For each of the knowledge/skill areas you listed above, please tell us the extent to which you agree or disagree
with the following statements, where 1 = do not agree at all and 4 = fully agree. Choose N/A if the statement is
not applicable to the knowledge/skill area you selected.
I have been able to effectively use the knowledge and skills I 1234 1234 1234
acquired. N/A N/A N/A
I received in the initiative the practical skills needed to use this 1234 1234 1234
knowledge/skill area in my work. N/A N/A N/A
I have enough support from my managers to use this 1234 1234 1234
knowledge/skill area in my work. N/A N/A N/A
I have enough support from my peers to use this knowledge/ 1234 1234 1234
skill area in my work. N/A N/A N/A
1. Are there other ways in which you have been able to use the knowledge, skills
or contacts you gained in the initiative since returning to your workplace?
Please give specific examples of any additional ways in which the initiative
has contributed to your ability to achieve your work-related goals.
2. Are there areas of learning in which it would have been useful for you to get
expert follow-up support in order to help you use this knowledge in your
workplace? If so, what support would have been helpful to you and why?
3. Please list up to three further suggestions on how to improve the
relevance and usefulness of initiatives like yours.
1. _____________________________________________________________________________________
2. _____________________________________________________________________________________
3. _____________________________________________________________________________________
243
Example of Level 3 participant
Tool 7m benchmark evaluation69
WHAT? This tool provides an example of how participant benchmark evaluation has been used in FAO
for Level 3 evaluation.
WHEN? Design of appropriate evaluation instruments should start at the design stage of the learning
initiative, when developing the evaluation plan.
Dear Participant, 69
We hope that this training course will be interesting and useful for all participants. In order to allow
us to improve our training programme and to assess the effectiveness of this particular course,
we kindly ask you to complete three questionnaires as part of the course evaluation process.
All the information you submit will be used in confidence by the FAO
Investment Centre to improve its capacity development work.
Stage 2 – End-of-training assessment: You will be asked to complete the questionnaire for Stage 2 at
the end of the training course. This information addresses the extent to which your expectations were
fulfilled and how you expect to be able to apply the knowledge and experience in your job situation.
The training course organizer will distribute these forms separately to each participant.
Please complete the forms by hand and then return them to the training course organizer.
He or she will then make two copies, keep one and forward one to FAO TCI.
Thank you!
Togo to Mali Study tour on sector-wide approaches in agriculture, sectoral M&E systems and the “Chambres d’agriculture”, TCI
69
244
Stage 1: What do you hope to get out of the training course?
Name:__________________________________________
Question 1:
From a job perspective, what are your main expectations from participating in the training course?
________________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
Question 2:
245
Question 3:
Please score the following training course topics in terms of their expected relevance for you:
(1= Not at all relevant, 2= To a little extent, 3= To some extent, 4= To a large extent, 5=Fully relevant)
Question 4:
For each of the three topics that you rated as being most relevant to you in Question 3 above,
please explain briefly why you consider them to be particularly relevant to your job:
Topic 1: _________________________________________________________________________________________
Why relevant?___________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
Topic 2: _________________________________________________________________________________________
Why relevant?___________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
Topic 3: _________________________________________________________________________________________
Why relevant?___________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
246
Stage 2: What did you actually get from the training?
Question 1:
Now that you have finished the training course, how would you rate the
overall usefulness of the training from a job perspective?
Please use a scale from 1 to 5 (1= Not useful at all, 2= Useful to little extent, 3=
Useful to some extent, 4= Useful to a large extent, 5=Extremely useful).
Rating:__________________
Question 2:
Were there any particular elements/topics that you were missing in this training
course which you would recommend including in future training courses?
Topic 1: _________________________________________________________________________________________
________________________________________________________________________________________________
Topic 2: _________________________________________________________________________________________
________________________________________________________________________________________________
247
Question 3:
Based on the experience/learning you have acquired from this course, please select
the top three topics (in prioritized order) that you consider to be the most useful to
you from a job perspective, and explain briefly why you find them useful (use the
suggested list of topics from Question 3 as guidance and add topics as needed).
NOTE TO TRAINERS/FACILITATORS: Participants should be given a copy
of Question 3 (Stage 1 Questionnaire) to answer this question.
Topic 1: _________________________________________________________________________________________
Why useful?_____________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
Topic 2: _________________________________________________________________________________________
Why useful?_____________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
Topic 3: _________________________________________________________________________________________
Why useful?_____________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
Question 4:
Are you considering sharing the experience/learning from this training with others? Y / N ______
If yes, please indicate how you may do that (e.g. post-visit briefing workshop/
seminar, dissemination of training report) and with whom.
How: ___________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
Why whom: _____________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
248
Stage 3: Have you made use of the training
course experience in your job?
Question 1:
Based on your work experience after the training course, please rate the extent
to which the experience/learning from the training on the three topics listed
under Question 3 (Stage 2 Questionnaire) has become useful to you.
NOTE TO TRAINERS/FACILITATORS: responses from Stage 2
Questionnaire will have to be sent to participants.
Please use a scale from 1 to 5 (1= Not at all useful, 2= Useful to little extent, 3= Useful
to some extent, 4= Useful to a large extent, 5= Extremely useful). If you have been
making use of the new experience/learning, please also briefly explain how.
Score (1, 2, 3, 4, 5)
Topic 1: _________________________________________________________________________________________
How? _________________________________________________________________________________________
Topic 2: _________________________________________________________________________________________
How? _________________________________________________________________________________________
Topic 3: _________________________________________________________________________________________
How? _________________________________________________________________________________________
Question 2:
Is there any other experience/learning from the training (not listed under Question Y / N ______
1 above) that has become particularly useful to you in your job?
If yes, please list these topics in the table below and explain briefly
how you are making use of this experience/learning.
Topic 4: _________________________________________________________________________________________
How? _________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
Topic 5: _________________________________________________________________________________________
How? _________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
249
Question 3:
For each of the topics listed under Question 1 and Question 2 above, please
indicate whether the following factors have been limiting or supportive of
your ability to apply the acquired experience/learning on the job?
Please use a scale from 1 to 5 (1= Very limiting, 2= Limiting to some extent, 3= Neither
supportive nor limiting 4= Supportive to some extent, 5= Very supportive).
Question 4:
Have you been sharing the experience/learning from the training with others? Y / N ______
If yes, please indicate how you may do that (e.g. post-visit briefing workshop/
seminar, dissemination of training report) and with whom.
How: ___________________________________________________________________________________________
________________________________________________________________________________________________
Why whom: _____________________________________________________________________________________
________________________________________________________________________________________________
250
Tool 7n Guidance sheet - Process evaluation
WHAT? This guidance sheet describes the importance of process evaluation. It should be read in
conjunction with the “Checklist for quality assurance” (Tool 8c in the Planning and Coordinating Toolbox).
WHO? Coordinators and managers of learning initiatives with the help of instructional design experts,
where feasible
Process evaluations look at the quality of learning management and inputs. These may include: quality
of needs assessment; quality of lectures; quality of learning materials; appropriateness of participant
selection strategies/participant mix; appropriateness of course length/content; appropriateness of
mix among lectures, discussions and activities; and quality and adequacy of follow-up support.
Process evaluation provides information on the quality of learning cycle management. It allows
facilitators/trainers to determine what needs to change in their learning plans and delivery so that
present or future sessions or programmes will be most effective for participants. Process evaluations
are a valuable source of information for improving the quality and results of learning initiatives.
The “Checklist for quality assurance”, contained in the Planning and Coordinating
Toolbox (Tool 8c), can help you evaluate the learning management cycle process.
Please note that the checklist is intended for your own learning because it can help
you identify whether your initiative was carried out in compliance with good standards
of learning cycle management; it should not be distributed to participants. Also, it is
indicative only. Some items may not be applicable to all types of learning initiatives, and
may depend on factors like the scope of the programme, available resources, etc.
The checklist has been designed in four main sections, but you may modify it as appropriate.
251
Tool 7o Evaluating the quality of facilitation70
WHAT? This tool presents a form that is suitable for assessing the quality of facilitation in a face-to-face
learning initiative. It asks participants to put themselves in the position of the facilitator. It is particularly
indicated for Training of Trainers programmes to encourage future trainers to begin to plan how they
would present the training programme to others.
WHO? Participants
Instructions
Allow 20 to 30 minutes to complete 70
1. Distribute the form below to each person. Then, divide the participants into
groups of three to four and ask them to discuss each point on the form.
2. During this exchange, participants can begin to record their answers on their form.
Participants do not have to agree. The purpose of the group discussion is to encourage
consultation and sharing of ideas to assist them in formulating their own answers.
3. Invite individuals to share some of their thoughts with the entire class. This should not
be a requirement. Some members may be reluctant to share their thoughts openly.
4. At the end of the session, the facilitator should collect all of the forms. These can be used to
improve the next presentation of the materials. Copy them and return them to the participants.
Note: It is important for the facilitators to refrain from defending themselves if criticized.
Be very open to learning from the participants about trainer performance.
A number of days have been spent in this training course. You have a unique
perspective to give on the training. What would you have done differently? Take
sufficient time to provide complete answers to the questions below.
Adapted from Market-oriented farm management for trainers of extension workers materials, Module 7, 2007 (AGPS)
70
252
What I would do differently if I were the facilitator:
Style of presentation
Use of visuals
Providing feedback
Other
253
Evaluating the quality of
Tool 7p the coaching process71
WHAT? This tool presents a form to identify the strengths and areas in need of improvement of your
technical coach.
Name of learner: 71
Name of coach:
Date: _________________________________
254
Circle the appropriate
Coaching process number
My coach displayed great sensitivity in the way s/he provided feedback to me. 1 2 3 4 N/A
I am generally satisfied with the way in which the coaching session was handled. 1 2 3 4 N/A
Additional comments you would like to make about this coaching experience:
255
Toolbox 1 2 3 4 5 6 7 8
Tools
8A Writing a concept note CORE 258
8B Planning checklist CORE 261
8C Checklist for quality assurance CORE 265
8D Checklist for event preparation recommended 269
References and resources
257
Writing a concept note72 for
Tool 8a the learning initiative
WHAT: This tool provides a template for describing key components of the intended learning initiative in
order to gain buy-in and support from major internal/external stakeholders.
WHEN: After the learning needs assessment is done and broad learning goals have been established for
the initiative
Instructions 72
The concept note should be as short as possible (generally not more than three
to four pages, although larger programmes may warrant slightly longer notes).
Below are the eight recommended ingredients for a concept note.
72
Adapted from World Bank Institute, Staff Development Group, with permission
258
2. Learning objectives
>> Describe key findings from the learning needs assessment.
>> Indicate broad learning goals for the initiative.
Example: “The goal of the programme is to enhance the ability of
smallholders to understand the benefits of microcredit.”
259
6. Quality assurance
>> Describe the mechanisms you will use to ensure the quality of the learning initiative.
Example: “Before implementation, the learning design and workshop materials
will be piloted and its outcomes reviewed by a manager, content expert from a
country office, at least two sample learners and/or one learning expert.”
8. Sustainability
>> Describe whether the initiative will be repeated in the current format and,
if so, whether plans have been made to make it sustainable.
Example: “The face-to-face workshop will be transformed into self-
paced e-learning after the initial pilot testing.”
>> Describe how the initiative will be institutionalized and integrated in countries’ local contexts.
Example: “TOT courses will be developed and offered so that the learning
solution will continue to be provided locally by national trainers.”
260
Tool 8b Planning checklist
WHAT: This tool provides a checklist for planning the learning initiative.
Discussion
Decision
Planning checklist still N/A
taken required
261
Discussion
Decision
Planning checklist still N/A
taken required
262
Discussion
Decision
Planning checklist still N/A
taken required
264
Tool 8c Checklist for quality assurance
WHAT: This tool provides a checklist to assure the quality of your learning initiative.
1. Context assessment, selection of target audience and learning needs assessment (Steps 1 – 3)
Not
Needs Notes and
Questions and considerations Yes feasible/
more work follow-up
relevant
265
2. Design and develop content, select delivery mode (Step 4)
Not
Needs more feasible/ Notes and
Questions and considerations Yes work not follow-up
relevant
266
3. Deliver the learning initiative (Step 5)
Not
Needs more feasible/ Notes and
Questions and considerations Yes work not follow-up
relevant
267
4. Follow-up and evaluation (Steps 6 – 7)
Not
Needs more feasible/ Notes and
Questions and considerations Yes work not follow-up
relevant
268
Tool 8d Checklist for event preparation
WHAT: This tool provides a checklist for the preparation of face-to-face events. It may need to be adapted
for other types of initiatives.
COMPLETED
WHEN ACTIVITY (√)
3 months Develop training concept note with related budget (See a suggested format
for a budget at the end of the checklist.)
2-3 months Identify workshop venue and location, and begin negotiations for
accommodation and transport package (See considerations for selecting the
venue at the end of the checklist.)
1-2 months Communicate with target groups: finalize participant list and start discussing
the agenda.
Assist participants with travel clearance process and with visa application, if
required.
269
COMPLETED
WHEN ACTIVITY (√)
Review the workshop preparation checklist and course agenda with all the
people involved in running the course.
>> liaising with hotel staff: providing hotel managers with the
workshop agenda, including break and lunch times; arranging the
distribution of participants’ welcome packs; and clarifying where
breaks and lunches will be held – if possible, arrange for flexible
buffet lunches, to allow learners to work during lunch breaks;
>> participants’ financial and travel issues: issuing daily subsistence
allowances (DSAs); confirming air tickets, etc.; and ensuring
that hotel staff and participants understand what is – and
what is not – included in the workshop package;
>> equipment in the workshop venue: ensuring it is secure after hours.
Ensure with hotel managers that participants will be picked up from the
airport, where possible.
1 day Decide who will welcome the participants at the start of the workshop.
Ensure that the venue’s Internet connection functions. If Internet access is not
available in the venue, they should identify inexpensive sources for Internet
access for participants during the course.
Test all the equipment to be used during the course. Ensure that computers
are compatible with the data projector, and load all the slides to be used on
to the computers.
Check that breakout rooms are available when needed, and that tables and
chairs are properly arranged in the rooms.
Find out where the venue’s and other’s keys are kept after hours.
270
COMPLETED
WHEN ACTIVITY (√)
Beginning Circulate a list of participants with their addresses, phone and e-mail details
and ask them to edit their entries. The list should be revised and distributed
at the close of the workshop.
Middle Provide participants with copies of all the learning event presentations or
prepare a workshop CD to distribute at the end of the workshop. The CD
could contain copies of presentations, handouts, photos from the workshop,
the participant list and other materials. Learners can then reproduce materials
for further in-country learning once they return to their offices.
End Provide contact points for the training team and indicate own availability for
further follow-up questions or advice.
Arrange for debriefings or after-action review with the training team and
organizers to review strong and weak points of the workshop.
Provide honest feedback to the venue staff coordinator for future events.
Budget73
The following format can be useful for setting the budget for the event.
DSA +
No. Participants Country Ticket terminals
Cap-Net, http://www.cap-net.org/sites/cap-net.org/files/Planning%20short%20courses.doc
271
Budgeted Actual
Description
Local Local
USD USD
currency currency
Facilitators
Venue
Lunch/coffee breaks
TOTAL
When facilities are not available in-house or at a ‘host partner’ institution, external facilities (e.g.
hotels, conference centres) are often used. Both options have advantages and disadvantages:
Advantages Disadvantages
74
Ibidem
272
WWW.FAO.ORG/CAPACITYDEVELOPMENT