Longitudinal and Evaluational

Research
By: John Amada, Jyulier Malco,
Jason Mangilit
Longitudinal research outline
I. What is Longitudinal research
II. Method of data gathering
III. Pros/Advantages
IV. Cons/Disadvantages
V. Types
VI. Journal

Longitudinal Research
A type of research method used to
discover relationships between variables
that are not related to various background
variables.
Researches study the same group over a
long period of time hence the term,
longitudinal.
How is data gathered?
asking people questions, observing
people's behavior, and observing the
physical traces or results of people's
behavior for a long period of time.
Pros/Advantages
Useful on researches about development
life span issues.

Cons/Disadvantages
Requires a long and enormous amount of
time.
Types of Longitudinal Research
Panel Study/Cohort Study:
Involves sampling a cross section of
individuals.
Example: people within a common age-band,
and groups with some other date-specific
common experience, such as people
graduating from a university, having a first
child, or migrating to another country in a
given year or band of years

Retrospective study
Involves looking to the past by looking at
historical information such as medical
records.
Studies groups of individuals who are alike in
many ways but differ by a certain
characteristic (for example, bodybuilders who
do cardiovascular exercise and those who
don't) are compared for a particular outcome
such as heart attacks.
Journal Article
Evaluation research
Evaluation research outline
I. What is evaluation
a. Definition
b. Nature of Evaluation

II. Types of Evaluation
a. Formative
b. Summative

III. Quantity and Quality: Paradigmatic Choices in Evaluation Methodology
a. The Paradigm Problem
b. Quantitative Approach
c. Qualitative Approach
d. Paradigm of Choices

IV. Methods of Data Collection
a. Types of Data
b. Methods of Collecting Data
c. Mixing Methods
d. Method and Theory

V. Journal Article

What is evaluation?
Evaluation is a systematic review of
someone or somethings worth,
significance, merit using a criteria based
on a set of standards
Methods of evaluation
Evaluation is influenced by the situation,
how the evaluation is conducted.
Evaluation helps find the right methods to
use.
Types of data
Qualitative
used to obtain details of the subjective
experiences of programme planners and
participants in process-oriented
evaluations.
Quantitative
objective factual data that are not only
replicable, but also amenable to statistical
analysis.

How they are acquired
Primary data
Involves the use of research instruments.
Secondary data
Data collected other researchers,
organizations or government departments for
their own purposes.
Types of data
Design

I. Structured
- it imposes a structure on the data at the point of collection
- it has predetermined nature of the response categories
affords respondents little opportunity to express their individual
feelings.
Ex. Standardized assessment schedules or attitudinal
scales

II. Unstructured
- provide scope for individuals to respond in their own terms
Ex. Informal interviews and open-ended
questionnaires

Questionnaires
- Can be used to provide the main source of
data in a study
- Concerned with description and
measurement
- Can include assessment schedules,
sociometric tests and attitude rating scales

1. use an existing scale
2. construct a new scale

PRINCIPLES in using
questionnaires to collect data:
1. The questionnaire should be delivered with a
covering letter explaining the purpose of the
research.
2. It is important that the document itself looks
attractive and that all instructions are easy to
follow.
3. Questionnaires should not be too long
4. The order in which questions are asked is
important
5. Question wording should be clear and
unambiguous.

Closed questions
- Ones which require the respondent to
select one or more responses from a
predetermined list of possible answers.
- Yes or No
- Strong Agree or Disagree
- Choose a number
- Symbols and Pictures

Open-Ended Questions
- Allow respondents to answer in their own words,
rather than being restricted to choosing from a
list of pre-coded categories.
- They give respondents leeway to elaborate their
answers.
- They call for greater deliberation and a written
response.
- They can generate wide range of response
- they are best placed towards the end of the
questionnaire.

Formulating Open-Ended
Questions
1. Use language appropriate to the target
audience
2. Avoid leading questions
X Do you prefer..? Dont you agree that?
3. Always make sure that the frame of
reference for a question is clearly stated

General advice in developing an
interview schedule
1. It begins with the evaluator drawing up a
preliminary list of categories deemed to be
relevant given the evaluation situation.
- program literature
- related documentary materials
- data collected from unstructured,
exploratory interviews with principal
stakeholders
2. The evaluator proceeds to generate lists of
questions relevant to each category
Internal Interviewers
-using programme staff to conduct
interviews
Advantage:
Can help to keep costs down as well as
having other potential advantages
Disadvantage:
Programme staff can find it difficult to keep
the roles of service provider and research
interviewer separate

Group Interview or Focus Group
- provides relatively inexpensive and
efficient way of collecting data particularly
when the investigator is primarily
concerned with obtaining an insight into
the attitudes and opinions of groups,
rather than acquiring specific information
about individuals.


Group Interview or Focus Group
Group Size
There is no fixed size for a focus group interview, but the group
must nt be too large as this may inhibit some members from joining
in the discussion.
A typical focus group involves around six to twelve people who
are brought together by the researcher for the purpose of answering
a number of questions.
Group Facilitator
Encourages the participants to respond in their own terms
while simultaneously ensuring that the focus of the group is
maintained.
Group Setting
Allows the participants to qualify their original responses in the
light of comments made by other group members.

Group Interview or Focus Group
Advantage:
1. It can generate a lot of qualitative data fairly quickly; normally a
focus group session lasts one to two hours
2. It is relatively inexpensive to run and therefore a popular choice
when financial resources are limited.

Disadvantages:
1. Individuals may suppress or modify their true feelings when in the
presence of others.
2. Individuals may feel particularly inhibited when in a group where
participants not only know each other, but also have to work
alongside each other.
3. There is the question of the status of individual participants.

Group Interview or Focus Group
Preliminary stage
- It can help identify what participants see as the key
issues.
Critical stage
- It can be used to obtain feedback from participants
as to what problems are encountered and they are
resolved.
End of the programme
- It can be used to obtain participants perceptions of
the impact and overall effectiveness of the programme.

Observation
Systematic Observation
- when observation and recording are done
according to explicit procedures which permit
replication and rules are followed which permit
the use of logic of scientific inference
- involves much more than the looking and
listening as part of everyday life.
- the evaluator becomes the main instrument
of data collection

Roles of the evaluator
I. Participant observation
- involves the evaluator entering the social world of those engaged in
programme activities in order to provide a full and detailed account of the
programme

a. Complete participant
- the role adopted by the researcher is a covert or secret one; those
people under observation are unaware of the identity or intentions of the
observer
- there is a possibility that the covert participants may feel
constrained from asking questions in field setting for fear of appearing
unnaturally inquisitive, thereby running the risk of being exposed.
b. Participant-as-observer
- the reasons for the researchers presence are made explicit,
consequently, there is no threat of exposure.
- Overt observers might have the advantage of being able to freely
question informants, but the fact that they are seen as researchers may
influence how informants behave and respond to their questioning.


Roles of the evaluator
I. Non-participatory roles

a. Observer-as-participant
- contact with informants is brief, and although the role
is an overt one there is no attempt on the part of the researcher
to develop relationships with informants
b. Complete observer
- there is no social interaction whatsoever between the
observer and the informant during the course of data collection
- as is the case with the systematic eavesdropping or
when the behavior of subjects in laboratory-based experiments
is observed through a one-way mirror

Advantages:
1. Observational fieldwork is essential if the evaluator is to provide a suitably
descriptive account of the core features of any programme
2. When there is strong emphasis on participation, personal experience becomes an
important resource
3. Direct experience of programme activities enables the evaluator to draw on tacit,
as well as propositional, knowledge, in order to describe a situation or series of
events
4. In some situations, observational methods can succeed where other methods fail
or prove infeasible.

Disadvantages:
1. Gathering, recording, and analysing field data are costly activities, both in terms of
time and money.
2. There is concern about the reliability and validity of observational data.
3. There is always the possibility that individuals will consciously modify their
behavior if aware that they are under observation.
4. Fieldworker gets so caught up in the activities under observation that it affects
what is actually observed.

Documentary Sources
Kinds of documents
1. Primary sources
- documents compiled by individuals who have firsthand experience of the events
described
2. Secondary sources
- materials consists of documents produced by individuals who do not possess personal
knowledge of the situation
Ex. Written synopsis of a lengthy official report
3. Public documents
- administrative records held by national and local governments, official statistics and
reports of government select committees. Many of these documents are freely available
4. Private documents
Ex. File notes, records, case files
5. Solicited documents
- includes documents the individual is asked to produce by the researcher; hence the
researcher can exercise some control with respect to what information is included and the
manner in which it is recorded.
6. Unsolicited documents
- the researcher is unable to influence the kind of information that is collected and must
make do with what is available. The information will have been originally collected for a
purpose other than an evaluation study and is therefore referred to as secondary data.

Mixing Methods
Mixed-methods research designs, in some
cases integrating quantitative and
qualitative methods, are now an
established feature of programme
evaluation research and policy evaluation
studies.

Mixing Methods
Triangulation
- borrowed from surveying or navigation,
where it refers to the practice of
establishing the exact position of a given
object by taking readings or
measurements from multiple view-points.
Using more than one reference point
enables greater accuracy of
measurement.

Mixing Methods
Types of Triangulation
1. Data
- involves the creation of multiple data sets by collecting data in a variety of contexts
and settings at different points in time.
2. Investigator
- occurs when more than one researcher or evaluator investigates the same situation.
Using multiple investigators ensures that a number of different viewpoints are
represented.
3. Theory
- entails making use of a number of alternative or competing theories in examining the
data
4. Methodological
a. within-method entails applying the same method on different occasions or using
multiple techniques within a given method
Ex. Administering the same questionnaire to
respondents on separate occasions
b. between-methods refers to the actual mixing of methods in a single research
design
Ex. A questionnaire survey of programme activities can be combined with
non-participant observation of programme activities and qualitative interviews with
samples of recipients and key workers.
Mixing Methods
Advantages:
1. Allows the researcher to have greater
confidence in the research findings than is
the case when a single method is used
2. Way of guarding against threats to both
reliability and validity.
3. Involves cross-checking for internal
consistency (within-method) and tests the
degree of external validity (between-
method)


Method and Theory
Types of Evaluation
1. Method-driven
- rigid adherence to a particular method, or
collection of methods, determines the focus
and scope of an evaluation
2. Theory-driven
- the evaluator chooses those research
methods deemed best suited for testing the
particular theory in question

Journal
Thank you for your time. :)
Credits to Mam Rieta for the Evaluation
research portion of our power point