Research Design

Research design is defined as a framework of methods and techniques chosen by a researcher to

combine various components of research in a reasonably logical manner so that the research
problem is efficiently handled. The research design refers to the overall strategy that you choose
to integrate the different components of the study in a coherent and logical way, thereby,
ensuring you will effectively address the research problem; it constitutes the blueprint for the
collection, measurement, and analysis of data. It provides insights about “how” to conduct
research using a particular methodology. Every researcher has a list of research questions which
need to be assessed – this can be done with research design. The design of a research topic is
used to explain the type of research (experimental, survey, correlational, semi-experimental,
review) and also its sub-type (experimental design, research problem, descriptive case-study).
There are three main sections of research design: Data collection, measurement, and analysis.

The type of research problem an organization is facing will determine the research design and
not vice-versa. Variables, designated tools to gather information, how will the tools be used to
collect and analyze data and other factors are decided in research design on the basis of a
research technique is decided. An impactful research design usually creates minimum bias in
data and increases trust on the collected and analyzed research information. A good research
design is expected to produces the least margin of error.

Research Design Characteristics

There are four key characteristics of research design:

Objectivity: It refers to the findings related to the method of data collection and scoring of the
responses. The research design should permit the measuring instrument which are fairly
objective in which every observer or judge scoring the performance must precisely give the same
report. In other words, the objectivity of the procedure may be judged by the degree of
agreement between the final scores assigned to different individuals by more than one
independent observer. This ensures the objectivity of the collected data which shall be capable
of analysis and drawing generalizations.

1
Reliability: If a research is conducted on a regular basis, the researcher involved expects similar
results to be calculated every time. Research design should indicate how the research questions
can be formed to ensure the standard of obtained results and this can happen only when the
research design is reliable.

Validity: There are multiple measuring tools available for research design but valid measuring
tools are those which help a researcher in gauging results according to the objective of research
and nothing else. The questionnaire developed from this research design will be then valid.

Generalization: The outcome of research design should be applicable to a population and not
just a restricted sample. Generalization is one of the key characteristics of research design.

Types of Research Design

A researcher must have a clear understanding of the various types of research design to select
which type of research design to implement for a study. Research design can be broadly
classified into quantitative and qualitative research design or mixed research design.

Qualitative Research Design: A qualitative research design is concerned with establishing

answers to the whys and hows of the phenomenon in question (unlike quantitative). Due to
this, qualitative research is often defined as being subjective (not objective), and findings are
gathered in a narrative format as opposed to numerical.

Quantitative Research Design: Quantitative research design relates to the design of a research
project which uses quantitative research methods. The design varies depending on the method
used. Below are some quantitative research designs

1. Descriptive Research Design: In a descriptive research design, a researcher is solely

interested in describing the situation or case under his/her research study. Instruments we use to
obtain data (research method) in descriptive studies include questionnaires, interviews (closed
questions), observation. It does not involve manipulation of the variables. It is a theory-based
research design which is created to gather, analyze and presents collected data. By implementing

2
an in-depth research design such as this, a researcher can provide insights into the why and how
of research.

2. Correlational Research Design: Correlational research is a non-experimental research design

technique which helps researchers to establish a relationship between two closely connected
variables. Two different groups are required to conduct this research design method. There is no
assumption while evaluating a relationship between two different variables and statistical
analysis techniques are used to calculate the relationship between them.

Correlation between two variables is concluded using a correlation coefficient, whose value
ranges between -1 and +1. If the correlation coefficient is towards +1, it indicates a positive
relationship between the variables and -1 indicates a negative relationship between the two
variables.

3. Quasi experimental research design: The prefix quasi means “resembling.” Thus quasi-
experimental research is research that resembles experimental research but is not true
experimental research. Although the independent variable is manipulated, participants are not
randomly assigned to conditions or orders of conditions. Because the independent variable is
manipulated before the dependent variable is measured, quasi-experimental research eliminates
the directionality problem. But because participants are not randomly assigned—making it likely
that there are other differences between conditions—quasi-experimental research does not
eliminate the problem of confounding variables. In terms of internal validity, therefore, quasi-
experiments are generally somewhere between correlational studies and true experiments.
Quasi-experiments are most likely to be conducted in field settings in which random assignment
is difficult or impossible. They are often conducted to evaluate the effectiveness of a treatment—
perhaps a type of psychotherapy or an educational intervention. There are many different kinds
of quasi-experiments, but we will discuss just a few of the most common ones here.

4. Experimental Research Design: Experimental research design is used to establish a

relationship between the cause and effect of a situation. It is a causal research design where the
effect caused by the independent variable on the dependent variable is observed. For example,

3
the effect of an independent variable such as price on a dependent variable such as customer
satisfaction or brand loyalty is monitored. It is a highly practical research design method as it
contributes towards solving a problem at hand. The independent variables are manipulated to
monitor the change it has on the dependent variable. It is often used in social sciences to observe
human behavior by analyzing two groups – affect of one group on the other.

Characteristics of experimental design

1. The Manipulation of Predictor Variables

In an experiment, the researcher manipulates the factor that is hypothesized to affect the outcome
of interest. The factor that is being manipulated is typically referred to as the treatment or
intervention. Suppose, for example, a group of researchers was interested in the causes of
maternal employment. They might hypothesize that the provision of government-subsidized
child care would promote such employment. They could then design an experiment in which
some subjects would be provided the option of government-funded child care subsidies and
others would not. The researchers might also manipulate the value of the child care subsidies in
order to determine if higher subsidy values might result in different levels of maternal
employment.

2. Random Assignment

 Study participants are randomly assigned to different treatment groups

 All participants have the same chance of being in a given condition
 Participants are assigned to either the group that receives the treatment, known as the
"experimental group" or "treatment group," or to the group which does not receive the
treatment, referred to as the "control group"
 Random assignment neutralizes factors other than the independent and dependent
variables, making it possible to directly infer cause and effect

3. Random Sampling

Traditionally, experimental researchers have used convenience sampling to select study

participants. However, as research methods have become more rigorous, and the problems with
generalizing from a convenience sample to the larger population have become more apparent,

4
experimental researchers are increasingly turning to random sampling. In experimental policy
research studies, participants are often randomly selected from program administrative databases
and randomly assigned to the control or treatment groups.

4. Validity of Results

The two types of validity of experiments are internal and external. It is often difficult to achieve
both in social science research experiments.

(a) Internal Validity is the extent to which a piece of evidence

supports a claim about cause and effect, within the context of
a particular study. It is one of the most important properties of
scientific studies

 When an experiment is internally valid, we are certain that the independent variable ( X)
caused the outcome of the study (Y)
 When subjects are randomly assigned to treatment or control groups, we can assume that
the independent variable caused the observed outcomes because the two groups should
not have differed from one another at the start of the experiment
 For example, take the child care subsidy example above. Since research subjects were
randomly assigned to the treatment (child care subsidies available) and control (no child
care subsidies available) groups, the two groups should not have differed at the outset of
the study. If, after the intervention, mothers in the treatment group were more likely to be
working, we can assume that the availability of child care subsidies promoted maternal
employment

One potential threat to internal validity in experiments occurs when participants either drop out
of the study or refuse to participate in the study. If particular types of individuals drop out or
refuse to participate more often than individuals with other characteristics, this is called
differential attrition. For example, suppose an experiment was conducted to assess the effects of
a new reading curriculum. If the new curriculum was so tough that many of the slowest readers
dropped out of school, the school with the new curriculum would experience an increase in the
average reading scores. The reason they experienced an increase in reading scores, however, is

5
because the worst readers left the school, not because the new curriculum improved students'
reading skills.

THREATES TO INTERNAL VALIDITY

 Subject Characteristics (Selection Bias/Differential Selection) — The groups may have

been different from the start. If you were testing instructional strategies to improve
reading and one group enjoyed reading more than the other group, they may improve
more in their reading because they enjoy it, rather than the instructional strategy you
used.
 Loss of Subjects (Mortality) — All of the high or low scoring subject may have dropped
out or were missing from one of the groups. If we collected posttest data on a day when
the honor society was on field trip at the treatment school, the mean for the treatment
group would probably be much lower than it really should have been.
 Location — Perhaps one group was at a disadvantage because of their location. The city
may have been demolishing a building next to one of the schools in our study and there
are constant distractions which interferes with our treatment.
 Instrumentation Instrument Decay — The testing instruments may not be scores
similarly. Perhaps the person grading the posttest is fatigued and pays less attention to the
last set of papers reviewed. It may be that those papers are from one of our groups and
will received different scores than the earlier group’s papers
 Data Collector Characteristics — The subjects of one group may react differently to
the data collector than the other group. A male interviewing males and females about
their attitudes toward a type of math instruction may not receive the same responses from
females as a female interviewing females would.
 Data Collector Bias — The person collecting data my favors one group, or some
characteristic some subject possess, over another. A principal who favors strict classroom
management may rate students’ attention under different teaching conditions with a bias
toward one of the teaching conditions.
 Testing — The act of taking a pretest or posttest may influence the results of the
experiment. Suppose we were conducting a unit to increase student sensitivity to
prejudice. As a pretest we have the control and treatment groups watch Shindler’s

6
 List and write a reaction essay. The pretest may have actually increased both groups’
sensitivity and we find that our treatment groups didn’t score any higher on a posttest
given later than the control group did. If we hadn’t given the pretest, we might have seen
differences in the groups at the end of the study.
 History — Something may happen at one site during our study that influences the
results. Perhaps a classmate dies in a car accident at the control site for a study teaching
children bike safety. The control group may actually demonstrate more concern about
bike safety than the treatment group.
 Maturation –There may be natural changes in the subjects that can account for the
changes found in a study. A critical thinking unit may appear more effective if it is taught
during a time when children are developing abstract reasoning.
 Hawthorne Effect — The subjects may respond differently just because they are being
studied. The name comes from a classic study in which researchers were studying the
effect of lighting on worker productivity. As the intensity of the factor lights increased, so
did the work productivity. One researcher suggested that they reverse the treatment and
lower the lights. The productivity of the workers continued to increase. It appears that
being observed by the researchers was increasing productivity, not the intensity of the
lights.
 John Henry Effect — One group may view that it is competition with the other group
and may work harder than they would under normal circumstances. This generally is
applied to the control group “taking on” the treatment group. The terms refers to the
classic story of John Henry laying railroad track.
 Resentful Demoralization of the Control Group — The control group may become
discouraged because it is not receiving the special attention that is given to the treatment
group. They may perform lower than usual because of this.
 Regression (Statistical Regression) — A class that scores particularly low can be
expected to score slightly higher just by chance. Likewise, a class that scores particularly
high, will have a tendency to score slightly lower by chance. The change in these scores
may have nothing to do with the treatment.
 Implementation –The treatment may not be implemented as intended. A study where
teachers are asked to use student modeling techniques may not show positive results, not

7
 because modeling techniques don’t work, but because the teacher didn’t implement them
or didn’t implement them as they were designed.
 Compensatory Equalization of Treatment — Someone may feel sorry for the control
group because they are not receiving much attention and give them special treatment. For
example, a researcher could be studying the effect of laptop computers on students’
attitudes toward math. The teacher feels sorry for the class that doesn’t have computers
and sponsors a popcorn party during math class. The control group begins to develop a
more positive attitude about mathematics.
 Experimental Treatment Diffusion — Sometimes the control group actually
implements the treatment. If two different techniques are being tested in two different
third grades in the same building, the teachers may share what they are doing.
Unconsciously, the control may use of the techniques she or he learned from the
treatment teacher.

(b) External Validity is the validity of applying the conclusion of a scientific

study outside the context of the study. External validity can be classified as
Population Validity the extent to which the results of a study can be generalized from the
specific sample that was studied to a larger group of subjects
1. The extent to which one can generalize from the study sample to a defined population–
If the sample is drawn from an accessible population, rather than the target population,
generalizing the research results from the accessible population to the target population is
risky.
2. The extent to which personological variables interact with treatment effects–
If the study is an experiment, it may be possible that different results might be found with
students at different grades (a personological variable).

Ecological Validity the extent to which the results of an experiment can be generalized from the
set of environmental conditions created by the researcher to other environmental conditions
(settings and conditions).
 External validity is also of particular concern in social science experiments
 It can be very difficult to generalize experimental results to groups that were not included
in the study
 Studies that randomly select participants from the most diverse and representative
populations are more likely to have external validity

8
  The use of random sampling techniques makes it easier to generalize the results of studies
to other groups

For example, a research study shows that a new curriculum improved reading comprehension of
third-grade children in Iowa. To assess the study's external validity, you would ask whether this
new curriculum would also be effective with third graders in New York or with children in other
elementary grades.

Threats to External Validity

1. Explicit description of the experimental treatment (not sufficiently described for

others to replicate)
If the researcher fails to adequately describe how he or she conducted a study, it is
difficult to determine whether the results are applicable to other settings.
2. Multiple-treatment interference (catalyst effect)
If a researcher were to apply several treatments, it is difficult to determine how well each
of the treatments would work individually. It might be that only the combination of the
treatments is effective.
3. Hawthorne effect (attention causes differences)
Subjects perform differently because they know they are being studied. “…External
validity of the experiment is jeopardized because the findings might not generalize to a
situation in which researchers or others who were involved in the research are not
present” (Gall, Borg, & Gall, 1996, p. 475)
4. Novelty and disruption effect (anything different makes a difference)
A treatment may work because it is novel and the subjects respond to the uniqueness,
rather than the actual treatment. The opposite may also occur, the treatment may not work
because it is unique, but given time for the subjects to adjust to it, it might have worked.
5. Experimenter effect (it only works with this experimenter) The treatment might
have worked because of the person implementing it. Given a different person, the
treatment might not work at all.
6. Pretest sensitization (pretest sets the stage)
A treatment might only work if a pretest is given. Because they have taken a pretest, the

9
 subjects may be more sensitive to the treatment. Had they not taken a pretest, the
treatment would not have worked.
7. Posttest sensitization (posttest helps treatment “fall into place”)
The posttest can become a learning experience. “For example, the posttest might cause
certain ideas presented during the treatment to ‘fall into place’ ” (p. 477). If the subjects
had not taken a posttest, the treatment would not have worked.
8. Interaction of history and treatment effect (…to everything there is a time…)
Not only should researchers be cautious about generalizing to other population, caution
should be taken to generalize to a different time period. As time passes, the conditions
under which treatments work change.
9. Measurement of the dependent variable (maybe only works with M/C tests)
A treatment may only be evident with certain types of measurements. A teaching method
may produce superior results when its effectiveness is tested with an essay test, but show
no differences when the effectiveness is measured with a multiple choice test.
10. Interaction of time of measurement and treatment effect (it takes a while for the
treatment to kick in)
It may be that the treatment effect does not occur until several weeks after the end of the
treatment. In this situation, a posttest at the end of the treatment would show no impact,
but a posttest a month later might show an impact.

Similarities between true and quasi-experiments:

 Study participants are subjected to some type of treatment or condition

 Some outcome of interest is measured
 The researchers test whether differences in this outcome are related to the treatment

Differences between true experiments and quasi-experiments:

 In a true experiment, participants are randomly assigned to either the treatment or the
control group, whereas they are not assigned randomly in a quasi-experiment
 In a quasi-experiment, the control and treatment groups differ not only in terms of the
experimental treatment they receive, but also in other, often unknown or unknowable,

10
 ways. Thus, the researcher must try to statistically control for as many of these
differences as possible
 Because control is lacking in quasi-experiments, there may be several "rival hypotheses"
competing with the experimental manipulation as explanations for observed results

Principles of research ethics

There are a number of ethical principles that should be taken into account when performing
undergraduate and master's level dissertation research.

At the core, these ethical principles stress the need to do good (known as beneficence) and do no
harm (known as non-malfeasance). In practice, these ethical principles mean that as a researcher,
you need to:

(a) Obtain informed consent from potential research participants;

(b) Minimize the risk of harm to participants;

(c) Protect their anonymity and confidentiality;

(d) Avoid using deceptive practices;

(e) Give participants the right to withdraw from your research.

PRINCIPLE ONE
Minimising the risk of harm

Dissertation research should not harm participants. Where there is the possibility that participants
could be harmed or put in a position of discomfort, there must be strong justifications for this.

11
Such scenarios will also require (a) additional planning to illustrate how participant harm (or
discomfort) will be reduced, (b) informed consent, and (c) detailed debriefing.

There are a number of types of harm that participants can be subjected to. These include:

 Physical harm to participants.

 Psychological distress and discomfort.

 Social disadvantage.

 Harm to participants? financial status.

 An invasion of participants? privacy and anonymity.

PRINCIPLE TWO
Obtaining informed consent

One of the foundations of research ethics is the idea of informed consent. Simply put, informed
consent means that participants should understand that (a) they are taking part in research
and (b) what the research requires of them. Such information may include the purpose of the
research, the methods being used, the possible outcomes of the research, as well as associated
demands, discomforts, inconveniences and risks that the participants may face. Whilst is it not
possible to know exactly what information a potential participant would (or would not) want to
know, you should aim not to leave out any material information; that is, information that you
feel would influence whether consent would (or would not) be granted.

PRINCIPLE THREE
Protecting anonymity and confidentiality

Protecting the anonymity and confidentiality of research participants is another practical

component of research ethics.

After all, participants will typically only be willing to volunteer information, especially
information of a private or sensitive nature, if the researcher agrees to hold such information in

12
confidence. Data is not treated confidentially, whether in terms of the storage of data, its
analysis, or during the publication process (i.e., when submitting your dissertation to be marked).
However, this does not mean that all data collected from research participants needs to be kept
confidential or anonymous.

There are also a wide range of potential legal protections that may affect what research you can
and cannot perform, how you must treated the data of research participants, and so forth. In other
words, you don?t simply have a duty to protect the data you collect from participants; you may
also have (in some cases) a legal responsibility to do so. Since this varies from country-to-
country, you should ask your dissertation supervisor or Ethics Committee for advice (or a legal
professional).

PRINCIPLE FOUR
Avoiding deceptive practices

At first sight, deceptive practices fly in the face of informed consent. After all, how can
participants know (a) that they are taking part in research and (b) what the research requires of
them if they are being deceived? This is part of what makes the use of deceptive practices
controversial. For this reason, in most circumstances, dissertation research should avoid any
kinds of deceptive practices. However, this is not always the case.

Deception is sometimes a necessary component of covert research, which can be justified in

some cases. Covert research reflects research where (a) the identity of the
observer and/or (b) the purpose of the research is not known to participants. Cases where you
may choose to engage in covert research may include instances where:

 It is not feasible to let everyone in a particular research setting know what you are doing.

 Overt observation or knowledge of the purpose of the research may alter the particular
phenomenon that is being studied.

PRINCIPLE FIVE
Providing the right to withdraw

With the exception of those instances of covert observation where is not feasible to let everyone
that is being observed know what you are doing, research participants should always have

13
the right to withdraw from the research process. Furthermore, participants should have the right
to withdraw at any stage in the research process. When a participant chooses to withdraw from
the research process, they should not be pressured or coerced in any way to try and stop them
from withdrawing.

If your supervisor and/or Ethics Committee expect you to complete an Ethics Consent Form,
it is likely that you will have to let participants know that they have the right to withdraw at any
time.

14