Chapter8 Evaluation
Chapter8 Evaluation
Chapter8 Evaluation
Chhaapptteerr 88:: C
CU
UR
RR
RIIC
CU
ULLU
UM
ME
EV
VA
ALLU
UA
ATTIIO
ON
N
LEARNING OUTCOMES
When you complete this module will be able to:
OVERVIEW
8.0 Introduction
8.1 What is curriculum evaluation?
8.2 The CIPP evaluation model
8.3 Application of the CIPP
evaluation model
8.3 Stakes evaluation model
8.4 Eisners connoisseurship
evaluation model
8.0 Introduction
In Module 7, we discussed the implementation
of the curriculum plan. We looked at why people resist
change, the role of teachers, students, administrator and
parents in ensuring the successful implementation of
change. In this chapter, we will focus on determining
whether the curriculum plan implemented has achieved
its goals and objectives as planned. In other words, the
curriculum has to be evaluated to determine whether all
the effort in terms of finance and human resources has
been worthwhile. Various stakeholders want to know
the extent to which the curriculum has been successfully
implemented. The information collected from evaluating a curriculum forms the basis
for making judgements about how successfully has the programme achieved its intended
outcomes and the worth or value of the programme.
[Source: Letters to the Editor, New Straits Times, February 1, 2005]
ACTIVITY 8.1
Read the newspaper report at the beginning of the chapter and answer the
following questions
1. Do you think physical education be made an examination subject?
2. Do you agree with the writers opinions on the state of sports in schools?
McNeil (1977) states that curriculum evaluation is an attempt to throw light on two
questions: Do planned learning opportunities, programmes, courses and activities as
developed and organised actually produce desired results? How can the curriculum
offerings best be improved? (p.134).
Gay (1985) argues that the aim of curriculum evaluation is to identify its weaknesses
and strengths as well as problems encountered in implementation; to improve the
curriculum development process; to determine the effectiveness of the curriculum
and the returns on finance allocated.
to know if learners achieved the objectives or whether the programme produced the
desired outcomes. For example, the use of a specific simulation software in the teaching
of geography enhanced the decision making skills of learners. These outcomes can be
determined through formal assessment tasks such as marks obtained in tests and
examinations. Also of concern is whether the innovation was cost-effective. Was the
innovation efficient in terms of time to completion? Were there any unexpected
outcomes? Besides, quantitative data to determine how well students met specified
objectives, data could also include qualitative interviews, direct observations, and
document analyses
SELF-TEST 8.1
1. Identify the key words in the five definitions of curriculum
evaluation.
2. Why do you need to evaluate curriculum?
3. Whats the difference between formative and summative
evaluation?
Context
Input
Process
+
FORMATIVE
Product
SUMMATIVE
c) Process Evaluation (Is it being done?) is the provision of periodic feedback while
the curriculum is being implemented.
d) Product Evaluation (Did it succeed?) or outcomes of the initiative. Data is collected
to determine whether the curriculum managed to accomplish it set out achieve (eg. to
what extent students have developed a more positive attitudes towards science). Product
evaluation involves measuring the achievement of objectives, interpreting the data and
providing with information that will enable them to decide whether to continue,
terminate or modify the new curriculum. For example, product evaluation might reveal
that students have become more interested in science and are more positive towards the
subject after introduction of the new science curriculum. Based on this findings the
decision may be made to implement the programme throughout the country.
Context
Input
GOALS
PLANS
CORE
VALUES
OUTCOMES
Product
ACTIONS
Process
SELF-TEST 8.2
1. What is the difference between context evaluation and input
evaluation according to the CIPP model? Give specific
examples
2. What is the difference between process evaluation and product
evaluation according to the CIPP mode? Give specific
examples.
and other computer software in teaching science, mathematics and social sciences is
more widespread today. To evaluate the effectiveness of such a programme using the
CIPP model would involve examining the following:
Context: Examine the environment in which technology is used in teaching and
learning
How did the real environment compare to the ideal? (eg. The programme
required five computers in each classroom, but there were only two computer
labs of 40 units each for 1000 students)
What problems are hampering success of technology integration? (eg.
technology breakdowns, not all schools had internet access)
About 50% of teachers do not have basic computer skills
Input: Examine what resources are put into technology integration (Identify the
educational strategies most likely to achieve the desired result)
Is the content selected for using technology right?
Have we used the right combination of media? (internet, video-clips, etc)
Process: Assess how well the implementation works (Uncovers implementation
issues)
Did technology integration run smoothly?
Were there technology problems?
Were teachers able to integrate technology in their lessons as planned?
What are the areas of curriculum in which most students experienced
difficulty?
Product: Addresses outcomes of the learning (Gather information on the results of
the educational intervention to interpret its worth and merit)
Did the learners learn using technology? How do you know?
Does technology integration enhance higher order thinking?
ACTIVITY 8.2
With reference to Case Study 8.4.2
1. Suggest other questions you would ask regarding process
evaluation (i.e. implementation issues).
2. What data collection techniques would you recommend for
carrying out product evaluation to determine the teaching
and learning outcomes of technology integration?
INSTRUCTION
Antecedents
Student &
teacher
characteristics,
curriculum
content,
instructional
materials,
community
context.
Transactions
Communication
flow, time
allocation,
sequence of
events, social,
climate
Outcomes
Student
achievement,
attitudes,
motor skills,
effect on
teachers and
institution.
food critic, you must be a connoisseur of different kinds of foods. To be a critic, you
must be aware and appreciate the subtle differences in the phenomenon you are
examining. In other words, the curriculum evaluator must seek to be an educational
critic. When employing the procedure of educational criticism the following questions
may be asked:
What has happened in the classrooms as a result of implementation of the new
curriculum?
What are some of the events that took place? (eg. more students are participating
in field work, more students are asking questions in class, even academically
weak students are talking in group activities)
How did students and teachers organise themselves in these events?
What were the reactions of participants in these events? (eg. students enjoyed
working collaboratively in projects)
How can the experiences of learners be made more effective as suggested by
students, teachers and administrators? (eg. more resources are needed for
fieldwork, more computers are needed to integrate the internet in teaching and
learning).
You will notice that these questions places more emphasis on the process of learning
and the quality of experiences by those involved in the implementation of the
curriculum; namely, students, teachers and administrators. According to the
Connoisseurship Model, evaluators provide a description and interpretation of the
curriculum plan implemented:
1) Description: The evaluator records the actions, the features of the environment
and experiences of students, teachers and administrators. People who read the
evaluation report will be able to visualise what the place looks like and the
processes taking place. The aim here is to help the reader see the school or
classroom and get a feel of what the curriculum evaluator or critic is attempting
to understand and help others understand.
2) Interpretation: The evaluator explains the meaning of events reported by
putting it in its context. For example, why academically weak students were
motivated to ask questions; why reading comprehension skills improved; why
enthusiasm for doing science experiments increased and so forth.
To be able to describe and interpret the implementation of a curriculum the evaluator
has to collect data and the following are examples of activities an evaluator may engage
in:
o The evaluator observes what is going on the classroom and records
teachers and students in action using videotapes, audiotapes and
photographs.
o The evaluator keeps notes of what is done, what is said and more
importantly what is not said. The evaluator should strive to describe the
tone of the curriculum in action (Ornstein and Hunkins, 1998).
o The evaluator interview students, teachers and administrators about the
quality of the curriculum
o The evaluator would analysis students work
.
10
One of the great benefits of Elliot W. Eisner's activities has been the way in
which he has both made the case for a concern with connoisseurship and criticism, and
mediated these concerns for educators and researchers. The importance of his advocacy
of these ideas cannot be underestimated - especially at a time when rather narrow
concerns with instrumental outcomes and an orientation to the technical dominate.
Together they offer educators a more helpful and appropriate means to approach
evaluation, for example.
Advocating moving beyond technocratic and behaviouristic modes of thinking and for having a concern for 'expressive outcomes'.
Arguing that schools should help children create meaning from experience,
and that this requires an education devoted to the senses, to meaning-making
and the imagination. Eisner argues for a curriculum that fosters multiple
'literacies' in students (especially by looking to non-verbal modes of learning
and expression) and a deepening of the 'artistry' of teachers.
Over the time that Eisner has been writing there have been significant shifts in
the context in which schools have to operate. While there have been other voices
calling for changes in the culture of schooling (notably Howard Gardner in this
arena), the impact of globalization, growing centralization in many schooling systems,
reaction against more process-oriented forms of pedagogy, and a growing
instrumentalism education have served to make Eisner's message both more pertinent
to schools, and more difficult to respond to.
11
1. Aspects of the
curriculum to be
evaluated
2. Data Collection
3. Analysis of
Information
4. Reporting of
Information
12
to the questions. There is also the fear that the wordings used can bias client's responses.
Questionnaires are impersonal and since only a sample of subjects are given the
instrument, we not get the full story.
8.6.2 Interviews
Interviews are usually one-on-one situations in which an individual asks
questions to which a second individual (which may be a teacher, principal, student,
parent) responds. The person asking the questions is called the interviewer while the
person giving answers to the questions is called the interviewee. Interviews are used
when you want to fully understand someone's impressions or experiences, or learn more
about their answers to questionnaires. There are two general types of interviews
depending on the extent to which the responses required are unstructured or structured.
In an unstructured interview, the interviewer does not follow a rigid script and
there is a great deal of flexibility in the responses. For example; Why do you think the
recommended textbook for the course is difficult for low ability learners? The teacher
responding to such a question will give a variety of reasons. Some of the reasons given
may be of a general nature while others may be specific to certain sections of the
textbook. This makes the task of keeping track of responses more difficult. The openendedness of the question will require that the interviewer record all responses and
make sense of it later. The advantage of the unstructured interview is that it allows the
evaluator to gather a variety of information, especially in relation to the interviewees
knowledge, beliefs or feelings toward a particular situation.
In a structured interview, the questions asked usually require very specific
responses. For example, Is the recommended textbook difficult for low ability learners
because: a) there is too much content; b) the language used is beyond the
comprehension of low ability learners, c) or there are too few examples and illustrations.
Regardless of which type of interview is used, evaluators should ensure that each
question is relevant for its intended purpose. In the end, the data must be translated into
a form that can be analysed and this has to be dome carefully to preserve accuracy and
to maintain the sense of the data. The advantage of interviews is that it can get a full
range and depth of information and it develops a relationship with teachers and students
and it is more flexible. However, interview can take much time, can be hard to analyze
and compare, can be costly and interviewer can bias client's responses.
8.6.3 Observations
To gather accurate information about how a program actually operates,
particularly about processes. In other words to view operations of a program as they
are actually occurring. For example, can the people involved adapt to events as they
occur.
8.6.4 Documents
When we want impressions of how a programme operates without interrupting
the programme; we can review the memos, minutes, etc to get a comprehensive and
historical information about the implementation of the programme. However, we
should be quite clear about what looking for as there may be a load of documents.
13
Method
Overall Purpose
Advantages
interviews
Challenges
-might not get careful
feedback
-wording can bias client's
responses
-are impersonal
-in surveys, may need
sampling expert
- doesn't get full story
observation
to gather accurate
information about how a
program actually operates,
particularly about
processes
-view operations of a
program as they are actually
occurring
-can adapt to events as they
occur
focus groups
case studies
to fully understand or
depict client's experiences
in a program, and conduct
comprehensive
examination through cross
comparison of cases
common impressions
-can be efficient way to get
much range and depth of
information in short time
- can convey key information
about programs
14
ACTIVITY 8.2
1) What are some of the problems identified with the implementation of the
MALATI programme?
2) Based on the findings list the recommendations made.
15
DISCUSSION QUESTIONS:
1. Identify some problems in the implementation of the Primary School
Integrated Curriculum (KBSR) and the Secondary School Integrated
Curriculum (KBSM)?
2. Describe how the teaching of science and mathematics in English was
implemented in your school?
3. New curriculum often fail to become established in schools because the
importance and complexity of the implementation phase is not understood
Discuss.
READINGS
16