Unit 2
Unit 2
Unit 2
(UNIT 2)
BMS 4th Sem
Sugandha Jain
RESEARCH DESIGN AND ITS TYPES
What is Research Design?
• The research design is a conceptual structure within which the research
is conducted. It constitutes the blueprint for the collection,
measurement and analysis of data.
• The research design is the backbone of the study. It supports the study
and holds it together.
• It is the researcher’s overall plan for answering the research questions
or testing the research hypotheses.
Process of Designing and Conducting a
Research Project
• What – What is to be studied?
• What about – What aspects of the subject are to be studied?
• What for – What is the significance of the study?
• What does prior literature/research say?
• What was done and how was the study conducted?
• What was found?
• So what now?
Hence, Research Design is?
• A framework for the research plan of action.
• A master plan that specifies the methods and procedures for
collecting and analyzing the needed information.
• A strategy for how the data will be collected.
• The planned sequence of the entire process involved in conducting a
research study.
Purpose and Advantages of Research Design
Designing of the
Research Project
Exploratory RD Conclusive RD
Descriptive RD Causal/Experimental RD
Example
Exploratory: An HR manager sees a sudden rise in employees’ absenteeism. The manager
may not be aware of the reason behind it at all. Under such level of extreme uncertainty, an
initial investigation of the problem may be required and the approach of research can be
exploratory. The manager may get to know that probable causes of a sudden absenteeism
can be some seasonal illness affecting employees, children of many employees appearing for
board exams, some competitive firm trying to poach employees at a large scale, change of
leadership causing content among employees, etc. Now the manager gets an idea that the
reason for sudden absenteeism could be any of these.
Descriptive: The manager may want to carry a descriptive research to better understand the
employees, their satisfaction levels and their personal problems in terms of health and
children’s education, etc. In case the manager finds out that recent change in leadership is a
matter of concern amongst many employees.
Causal: Further, an experimental research can be carried out to study if the cause behind
absenteeism is the discontent among employees due to new leadership. Experimental
research is done to establish cause and effect.
I. EXPLORATORY DESIGN
Exploratory Design
• Exploratory researches are conducted to resolve ambiguity, or gain a deeper
understanding of something.
• Its role is to provide direction to subsequent and more structured and rigorous
research.
• Eg. A review of market opportunities available to a prospective entrepreneur; an
informal survey conducted to identify the problem in the supply chain of a product;
different ways that women professionals adapt to manage work-family conflict
• As can be seen, studies of this nature are less structured, more flexible in approach
and are not conducted to test or validate any preconceived propositions; in fact
exploratory research could lead to some testable hypotheses.
• Also called pilot or feasibility studies. It is the first step the researcher takes into the
unknown, to explore new frontiers which determine whether a full-scale investigation
is worthwhile.
Exploratory Design
• Exploratory studies are also conducted to develop, refine or test the designed
measuring instruments.
• For example, in designing a questionnaire to measure the parameters an individual
looks at while taking an investment decision, one needs to first explore the benefits of
a financial instrument, which could be the advantages sought by a consumer while
saving.
• Another case could be that we identify the selection parameters a person considers
while enrolling for a pilot training institute. After an assessment is made about the
importance of the parameters considered, one can then work out the financial
feasibility of setting up a private pilot training institute.
• The nature of the study being loosely structured means the researcher’s skill in
observing and recording all possible information and impressions determines the
accuracy of the findings.
Purpose of Exploratory Designs
Exploratory designs are the simplest and most loosely structured designs. As the name
suggests, the basic objective of the study is to explore and obtain clarity about the
problem situation. It is flexible in its approach and it mostly involves a qualitative
investigation. The sample size is not strictly representative and at times it might only
involve unstructured interviews with a couple of subject experts. The essential
purpose of the study is to:
• Define and conceptualize the research problem to be investigated
• Explore and evaluate the diverse and multiple research opportunities
• Assist in the development and formulation of the research hypotheses
• Operationalize and define the variables and constructs under study
• Identify the possible nature of relationships that might exist between the variables
under study
• Explore the external factors and variables that might impact the research
Methods of Conducting Exploratory Study
A. Secondary Resource Analysis
• Carrying out research to explore the research problem using already available data is called
analysis of secondary data for exploratory research.
• The literature as well as secondary data can be a very economical and quick sources of
information in the initial research phases.
• Extensive LRs are generally done before starting any research to understand the existing theories
and findings about the research problem.
• But secondary data is data that was collected for some other research problem which may be
wider or narrower in scope than the current problem. Thus it has to be used wisely to identify
relevant information.
• While choosing secondary data, make sure it is suitable. The content and time for which the data
was collected should also be taken into account.
• Reliability and accuracy of data should also be examined. So although secondary data may be less
costly, researcher must use it carefully.
• There can be multiple sources of secondary data: internal records, externally published records
and syndicated data.
a. Internal Records
• Data sources generated within the business for various purposes.
• Eg. Accounting records, trade association data, sales performance
record, customer database, etc.
• This pre-existing data can be analyzed from the perspective of the
research problem.
• For eg. Retailer notices decline in revenue and wants to investigate
causes. Exploratory research can begin by looking at the daily
transaction data. From this, he can easily identify if the fall in revenue
is due to fall in footfall, or fall in average billing amount. Also the
product categories which have seen max fall can be studied.
b. Externally Published Data
• These include data published by various trade associations,
agencies, research firms, brokerage firms, government, etc.
• Secondary data can be found in directories, guides and indexes. Many
consulting companies also publish market trends data reports on a
regular basis.
• There are various literatures in the form of journals, magazines,
books, etc. that form the source of externally published data.
• Government also collects large amount of data on regular intervals.
Eg. Census data provides better understanding of demographic
characteristics. Rainfall data, pollution data, farming production,
credit dispersion, industrial development, etc.
c. Syndicated Data
• There are many business firms who collect large amounts of data
and distribute it to the companies for a fee. This type of data is called
syndicated data.
• Syndicated data is widely used across different business sectors.
• For eg. The overall television viewing data is collected by a research
firm and distributed to all the different broadcasters for a fee.
• Similarly, consumer attitude, purchase behaviour data is also
collected by a marketing research firm for multiple product categories
and retail environments and then provided to various product
companies.
B. Comprehensive Case Method
• This in-depth study is focused on a single unit of analysis. This unit could be an individual
employee or a customer; an organization or a complete country analysis might also be the case of
interest.
• They are by their nature, generally, post-hoc studies and report those incidences which might have
occurred earlier.
• The scenario is reproduced based upon the secondary information and a primary recounting by
those involved in the occurrence.
• Thus, there might be an element of bias as the data, in most cases, become a judgmental analysis
rather than a simple recounting of events.
• For example, Company X wants to implement a performance appraisal system in the organization
and is debating between a traditional vs a 360˚ appraisal system. For a historical understanding of
the two techniques, the HR director makes use of the theoretical works done on the constructs.
However, the roll-out plans and repercussions and the management issue were not very clear. This
could be better understood when they studied in-depth case Company Y which had implemented
traditional appraisal formats, and Company Z- 360˚ systems. Thus, the two exploratory researches
carried out were sufficient to arrive at a decision in terms of what would work best for the
organization.
C. Expert Opinion Survey
• There might be a situation when the topic of a research is such that there is no previous information
available on it. In these cases, it is advisable to seek help from experts who might be able to provide some
valuable insights based upon their experience in the field or with the concept. This approach of collecting
particulars from significant and erudite people is referred to as the expert opinion survey.
• This methodology might be formal and structured and might be useful when being authenticated or
supported by a secondary/primary research or it might be fluid and unstructured and might require an in-
depth interviewing of the expert.
• For example, the evaluation of the merit of marketing organic food products in the domestic Indian market
cannot be done with the help of secondary data as no such structured data sources exist. In this case the
following can be contacted:
1. Doctors and dieticians as experts would be able to provide information about the products and the level
to which they would advocate organic food products as a healthier alternative.
2. Chefs who are experimental and innovative and might look at providing a better value to the clients.
However, this would require evaluating their level of awareness and perspective on the viability of
providing organically prepared dishes.
3. Pragmatic retailers who are looking at new ways of generating footfalls and conversions by offering
contemporary and futuristic products. Again, awareness about the product, past experience with selling
healthier lifestyle products would need to be probed to gauge their positive or negative reactions to the
new marketing initiatives.
C. Expert Opinion Survey
• These could be useful in measuring the viability of the proposed plan. Discussions with
knowledgeable people may reveal some information regarding who might be considered
as potential consumers. Secondly, the question whether a healthy proposition or a
lifestyle proposition would work better to capture the targeted consumers needs to be
examined.
• Thus, this method can play a directional role in shaping the research study.
• However, a note of caution is also necessary as by its very nature, it is a loosely
structured and skewed method, thus supporting it with some secondary data or
subsequently validating the presumptions through a primary research is recommended.
• Another aspect to be kept in mind is that no expert, no matter how vast and significant
his experience is, can be solely relied upon to arrive at any conclusions, as in the example
stated above. It is also advisable to quiz different expert sources.
D. Focus Group Discussions (FGDs)
• Another alternative approach to interviewing is to carry out discussions with significant
individuals associated with the problem under study.
• This technique is most staunchly advocated and used for consumer and motivational
research studies.
• In a typical focus group, there is a carefully selected small set of individuals
representative of the larger respondent population under study.
• It is called a focus group as the selected members discuss the concerned topic for the
duration of 90 minutes to, sometimes, two hours.
• Usually the group comprises six to ten individuals. The number thus stated is because
less than six would not be able to throw enough perspectives for the discussion and
there might emerge a one-sided or a skewed discussion on the topic. On the other hand,
more than ten might lead to more confusion rather than any fruitful discussion and that
would be unwieldy to manage.
D. Focus Group Discussions (FGDs)
• Generally, these discussions are carried out in neutral settings by a trained observer,
also referred to as the moderator. The moderator, in most cases, does not participate in
the discussion. His prime objective is to manage a relatively non-structured and
informal discussion. He initiates the process and then maneuvers it to steer it only to
the desired information needs.
• Sometimes, there is more than one observer to record the verbal and non-verbal
content of the discussion.
• The conduction and recording of the dialogue requires considerable skill and
behavioural understanding and the management of group dynamics.
• In the organic food product study, the focus group discussions were carried out with the
typical consumers/buyers of grocery products. The objective was to establish the level of
awareness about health hazards, environmental concerns and awareness of organic food
products. A series of such focus group discussions carried out across four metros—Delhi,
Mumbai, Bengaluru and Hyderabad—revealed that even though the new age consumer
was concerned about health, the awareness about organic products was extremely low
to non-existent.
Process of Planning and Conducting an FGD
Step 1: Objective Setting
(outlining the questions, should be precise and comprehensive)
Step 2: Moderator Briefing
(understands the objective, aware of research topic, have background of research, good
command over language of FGD, aware of sensitivities of group)
Step 3: Participant Selection
(gender, age, group, socio-economic class, profession, geography, etc., representative)
Step 4: Discussion
(started by moderator, brief about topic, ensure participants are comfortable with topic,
discussion is not influenced by one single participant, observe group dynamics carefully,
manage dominants participants, encourage quiet ones, summarise responses and ensure
group agrees on moderator’s interpretation of their responses)
Step 5: Recordings and Note review
(helps to capture verbal and non-verbal cues, eliminate inconsistent responses, note
remarks that were missed during discussion)
II. CONCLUSIVE DESIGNS
Conclusive Design
• The findings and propositions developed as a consequence of exploratory research might be
tested and authenticated by conclusive research.
• This kind of research study is especially carried out to test and validate formulated hypotheses
and specified relationships.
• In contrast to exploratory research, these studies are more structured and definite. The
variables and constructs in the research are clearly defined with explicit quantifiable
indications or simply, the variables can be denoted in the form of numbers that can be
quantified and summarized.
• The timeframe of the study and respondent selection is more formal and representative.
• The emphasis on reliability and validity of the research findings assume critical significance as
the concluded results might need to be implemented, in case it is an applied research study.
• For example, if a research study has to be conducted to test the impact of a new data
monitoring programme on the inventory management system of a hearing aids’ manufacturer,
then the impact needs to be clearly discernible for the management to install the monitoring
system.
Conclusive Design
• It is to be noted, however, that it is not always the exploratory that leads to the conclusive.
Sometimes the hypothesized relationship to be tested might be spelled out by the manager as
the problem to be investigated.
• An example is testing the level of consumer satisfaction with different insurance policies that an
organization has offered to consumers at large.
• Conclusive research can further be divided into descriptive and causal research. This
categorization is basically made based on the nature of investigation required.
EXPLORATORY RESEARCH CONCLUSIVE RESEARCH
Is flexible and investigative in methodology Has a formal and definitive methodology that
needs to be followed and tested
Does not involve testing of hypotheses Most conclusive researches are carried out to
test the formulated hypotheses
Findings might be topic-specific and might not Findings are significant as they have a
have much relevance outside the researcher’s theoretical or applied implication
domain
II. CONCLUSIVE DESIGNS
1. DESCRIPTIVE DESIGN
Descriptive Design
• As the name suggests, descriptive research is undertaken to describe the situation,
community, phenomenon, outcome or programme.
• The main goal of this type of research is to describe the data and characteristics
about what is being studied.
• For eg. The annual census carried out by the Government of India is an example of
descriptive research.
• It is contemporary, topical and time-bound.
• It addresses the establishment or exploration of a formulated proposition.
(Propositions are statements that define the relationships among concepts. These
can be judged as true or false if they are related to observable phenomena. These are
used to state the properties of a concept or its relationship with other concepts.
When a proposition can be tested for its validity through empirical estimation, then it
is called a hypothesis.)
Descriptive Design
• For eg. a study might want to distinguish between the characteristics of the customers
who buy normal petrol and those who buy premium petrol. Is the consumption of
organic food more in affluent South Delhi as compared to the other areas in Delhi? What
is the level of involvement of middle-level versus senior-level managers in a company’s
stock-related decisions? Organizational climate studies are conducted in different
organizations. A study of inventory management practices in the best-managed
companies is another example.
• The commonality between all these research studies is the fact that unlike the
exploratory, these are being conducted to test specific hypotheses and trends. They are
relatively more structured and require a formal, specific and systematic approach to
sampling, collecting information, collating and testing the data to verify the research
assumptions.
• The findings of descriptive studies are largely of a diagnostic nature, i.e., the studies
indicate the existing symptoms of a particular situation without establishing the
causality of the relationship.
• Lacks the precision and accuracy of experimental designs.
Descriptive Research
A. Cross-sectional B. Longitudinal
Design Design
Example 2: R O1 X O2
R X O3
• The above scheme indicates that the two groups of individuals were assigned at random
(R) to two treatment groups at the same times.
• Both groups received the same treatment X at the same time.
• The first group received both a pretest (O1) and post-test measurement (O2).
• The second group received the post-test measurement (O3) at the same time as the first
group received the post-test measurement (O2).
Validity in Experimentation
Researcher’s goals while conducting an experiment:
Internal Validity: To draw valid conclusions about the effect of IVs on DVs.
• It tries to examine whether the observed effect on a DV is actually caused by the IVs in
question. For an experiment to have internal validity, all other causal factors except the one
whose influence is being examined should be absent. Internal validity is the basic minimum
that must be present. It is impossible to draw inferences about the causal relationship between
IVs and DVs if the observed effects on test units are influenced by extraneous variables. Control
of extraneous variables is a necessary condition for inferring causality.
External Validity: To make generalizations about the results to a wider population.
• The concern is whether the result of an experiment can be generalized beyond the
experimental situations. If it is possible, then to what population, settings, times, IVs and DVs
can the results be projected.
It is desired that an experiment is valid both internally and externally. However, in reality, a researcher might
have to make a trade-off between one type of validity for another. To remove the influence of an extraneous
variable, a researcher may set up an experiment with artificial setting, thereby increasing its internal validity.
However, in the process the external validity will be reduced.
Factors affecting Internal Validity
There is a need to control the influence of extraneous variables so as to ensure that the experiment has
not been confounded. The following may threaten the internal validity of an experiment.
1. History: If there is long time duration between pre-test and post-test, there can be other variables
that may come into play to affect DV and make conclusions spurious. The difference (O2 – O1) may
indicate the treatment effect. Even if this difference is positive, this may not be attributed to the training
programme as this may be due to an improvement in the general economic condition between O1 and
O2. This is because the training programme is not the only variable causing a positive difference between
O2 and O1. As a matter of fact, the higher the time difference between the two observations, higher are
the chances of history confounding an experiment.
2. Maturation: The units themselves can change over a period of time and the change can reflect on
their responses. Eg. bored, tired, older, informed, influenced, etc.
3. Testing effects: When a pre-test affects the post-test observations
4. Instrumentation or Observer: If different set of questions are used in pre-test post-test, then
comparability will be an issue. Using different observers and interviewers may alter the responses of
respondents based on how respondents perceive them.
Factors affecting Internal Validity
5. Statistical regression: When test units with extreme scores (either extremely favourable or
unfavourable) are chosen for exposure to the treatment. The effect is that test units with extreme scores
tend to move towards an average score with the passage of time. Eg. If the sales people with extremely
poor performance are sent for the training programme, an increase in sales after the training
programme may be attributed to the regression effect. This is because test units with extreme score
have more room for a change, so a variation is more likely to be there. Random occurrences (weather,
luck, festive seasons), might have helped poor performance of sales people in the pre-test
measurement, and turned them into better performers.
6. Selection bias: Improper assignments of test units to treatments. Selection bias can occur if test units
self-select their groups or are assigned to the groups on the basis of the researcher’s judgment. The
selection of test units to the treatment group should be random.
7. Mortality: Some of the test units might drop out from the experiment while it is in progress or some
may refuse to continue with the experiment. Eg. some sales people may quit the organization before
completing the training successfully. There is no way of finding out whether those who were not
improving quit the organization. It is also not possible to measure whether those who left would have
produced the same results as those who completed the training programme.
Factors affecting External Validity
• Difference in environment: The environment at the time of test may be different
from the environment of the real world where these results are to be generalized.
For example, a commercial advertisement may be shown to a set of prospective
customers and their reaction to the advertisement may be very favourable.
However, if the same advertisement appears while the respondents are watching TV
at home with their family members, they may not like to see it and switch to another
channel. In this example, the environment in the two situations is completely
different and has come in the way to generalize the results.
• Difference in population: Population used for experimentation of the test may not
be similar to the population where the results of the experiments are to be applied.
Suppose the students of a college are asked to perform a task that could be
manipulated to study the effects on their performance. However, the findings of this
study cannot be generalized to the real world when the same task is assigned to the
employees of an organization. This is because the employees and the nature of job
in this particular organization may be quite different.
Factors affecting External Validity
• Difference in time period: Results obtained in a 5–6 week test may not hold in an
application of 12 months. Suppose a company wants to launch ice cream in Delhi NCR.
The results of the survey conducted during the months of May and June may be
extremely favourable. These results would certainly not be applicable during the
winter months in December and January, thereby raising questions on the
generalizability of the results.
• Difference in treatment: Treatment at the time of the test may be different from the
treatment of the real world. This can happen when while testing the effect of a
treatment, it is administered in the form of a pill and in reality it is given as a part of a
meal.
Methods to control extraneous variables
• Randomization: Random assignment of test units and treatments to experimental groups. Because of
random assignment, extraneous factors will be operating in all experimental groups. An extraneous
variable is eliminated, for eg., if background noise that might reduce the audibility of speech is
removed. Unknown extraneous variables can be controlled by randomization which ensures that
expected values of the extraneous variables are identical under different conditions.
• Matching: Match the various groups according to confounding variables. Suppose there are 120 people
to be distributed in three groups, out of which there are 45 females. Then each of the three groups is
assigned 15 females. This way, the effect of gender can be distributed among all three groups. Likewise,
other confounding variables like age, income, years of work experience could be distributed among the
three groups. The other examples of matching variables can be price, sales, size or location of store.
However, it may be not possible to match all the confounding variables to various groups.
• Use of experimental designs: Some of the experimental designs may be very useful in eliminating the
influence of extraneous variables.
• Statistical control: If all the above discussed methods fail to eliminate the effect of extraneous
variables, there is still one way of handling the confounding variable. It may be possible to statistically
control the effects of this variable on the dependent variable by the use of a technique called analysis of
covariance (ANCOVA).
Environments of conducting experiments
• Laboratory environment: Experiment conducted in an artificial environment constructed
exclusively for the experiment. Eg. suppose the interest is in studying the effectiveness of a TV
commercial. If the test units are made to see a test commercial in a theatre or in a room, the
environment would of a laboratory experiment.
• Field environment: Experiment is conducted in actual market conditions. There is no attempt to
change the real-life nature of the environment. Eg. showing of test commercial in an actual TV
telecast.
• Advantages of lab over field. Lab experiments have higher internal validity as they provide the
researcher with maximum control over the maximum number of confounding variables. Since the
experiment is conducted in a carefully monitored environment, the effect of history can be
minimized. The results of a laboratory experiment could be repeated with almost similar subjects
and environments. Laboratory experiments are generally shorter in duration, make use of smaller
number of test units, easier to conduct and relatively less expensive than field experiments.
• Disadvantages of lab over field. Lack in external validity i.e., it is not possible to generalize the
results of the lab experiment. Experiments conducted in the field have lower internal validity. The
ability to generalize the results of the experiment is possible in case of a field experiment, thereby
leading to higher external validity.
Experimental and Control Groups
An independent variable can be manipulated over two treatment
levels:
• The group in which an experimental treatment is administered is
called experimental group.
• The group in which no experimental treatment is administered is
called control group.
CLASSIFICATION 1. Pre-Experimental a. One-Shot Case Study
OF (no randomization)
EXPERIMENTAL b. One-Group Pre-Test-Post-Test
DESIGNS c. Static Group
2. Quasi-Experimental a. Time-Series
(no control of environment, field
experiments)
b. Multiple Time-Series
Presented as: O1 X O 2
• Test units are not selected at random. Subjected to treatment X and both pre (O1) and post-
treatment measurement (O2) are taken.
• One may compute treatment effect as O2 – O1, which may not be true, as this difference could be
the result of uncontrollable extraneous factors like history, maturation, testing, instrumentation,
regression, selection and mortality and would make the design invalid for making causal
inferences due to the following reasons:
a. The economic condition might have changed during the two periods (history).
b. The test units may mature over time (maturation).
c. The pre-test measurement on the test units may influence the performance (testing).
d. The prices of goods might have changed over time (instrumentation).
e. Test units might not have been selected at random (selection bias).
f. Some test units might have left before the experiment was complete (mortality).
g. Test units might be self-selected on the basis of the current poor performance and may have a
better period ahead because of sheer luck (regression).
c. Static group comparison
Presented as: Group 1 – X O1
Group 2 – O2
• This design uses two treatment groups. Test units in both groups are not selected at
random. The first group (experimental) is subjected to the treatment X, whereas the
second group (control) is not.
• Both groups are measured only after the treatment has been presented. Thus, it is
critical to understand that in this design the exposure as well as the experimental
treatment is not under the control of the researcher.
• The treatment effect could be measured by O1 – O2. However, this difference could be
attributed to at least selection bias and mortality. Moreover, since the test units are not
selected at random, the two groups could differ prior to the application of treatment
and can make the design invalid for drawing causal inferences.
2. Quasi-Experimental Designs
• Called field-experiments or semi-experimental designs. These are required if
stimulus can occur only in natural environment.
• Researcher may not have control of extraneous variables and experiments.
• The researcher can control when measurements are taken and on whom they are
taken. However, lacks complete control of scheduling of treatment and the ability
to randomize test units’ exposure to treatments.
• As the experimental control is lacking, the possibility of getting confounded
results is very high. Therefore, the researchers should be aware of what variables
are not controlled and the effects of such variables should be incorporated into
the findings.
a. Time-series design
Presented as: O1 O2 O3 O 4 X O 5 O6 O7 O 8
• Involves a series of periodic measurements on DV. The treatment X is then administered
and a series of periodic measurements are again taken to measure the effect of
treatment.
• No randomization of treatment to test units. Further, the timing of treatment presentation
as well as which of the test units are exposed to the treatment may not be within the
researcher’s control.
• Because of the multiple observations in time series design, the effect of maturation, main
testing effect, instrumentation and statistical regression can be ruled out. If test units are
selected at random, selection bias can be reduced. Further, if a strong measure like
giving certain incentives to the respondents is introduced, mortality effect can more or
less be controlled.
• The major drawback of this experiment is the inability of a researcher to control the effect
of history. The results of the experiment may be affected by an interactive testing effect
because multiple measurements are made on these test units. If a researcher could keep
a record of key changes in various unusual economic activities and if no changes are
found, one can reasonably conclude that the treatment has exerted an effect on test unit.
b. Multiple Time-Series design
Presented as: Experimental Group: O1 O2 O3 O4 X O5 O6 O7 O8
Control Group: O’1 O’2 O’3 O’4 O’5 O’6 O’7 O’8
• In this design, the control group is added to the time series design.
• The experimental group is subjected to the treatment X, whereas the control group is
without any treatment.
• The treatment effect (sales training) is found by comparing the average sales of the two
groups before and after the training programme.
• The major drawback of this design is the possibility of the interactive effect in the
experimental group.
3. True-Experimental Designs
• The units are selected randomly for administering the experiments
and also the experiments are administered randomly.
• Here, the researcher is able to eliminate the effect of extraneous
variables from both the experimental and control group.
a. Pre-Test Post-Test control group/ Before-after with control group design
Presented as: Experimental Group: R O 1 X O2
Control Group: R O3 O4
• Test units in both experimental and control group are selected at random at the same time. Pre-
test measurements O1 and O3 and post-test measurements O2 and O4 are taken in the
experimental and control group at the same time.
• All the extraneous variables operate equally on both the experimental and control group because
of randomization. Therefore, the only difference is the effect of treatment.
• A = O2 – O1 = Treatment + extraneous variables
• B = O4 – O3 = Extraneous variables
• The extraneous variables would include history, maturation, testing, instrumentation, statistical
regression, selection bias and test unit mortality. However, it may be worth noting that the
interactive testing effect would be present only in the experimental group and would be missing
in the control group. This is because only the experimental group is subjected to the treatment.
• Therefore A – B = (O2 – O1) – (O4 – O3) = treatment effect which would include interactive testing
effect. Therefore, it is doubtful to generalize the results of the experiment.
b. Post-test only control group/ After-only one control group design
Presented as: Experimental Group: R X O1
Control Group: R O2
• Test units in both the experimental and the control group are selected at random.
• Post-test measurements are taken on both experimental (O1) and control group (O2) at the same
time.
• The post-test measurement (O1) on experimental group comprises treatment effect and all other
extraneous variables, whereas O2 comprises only extraneous variables. Therefore, the difference
in the post-test measurement of experimental and control group is taken as a measure of
treatment effect. Hence,
O1 – O2 = (Treatment effect + extraneous factors) – (extraneous factors) = Treatment effect
• As pre-test measurement is absent, the effect of instrumentation and interactive testing effect is
ruled out.
• As there is a random assignment of test units to both the groups, it can be approximately
assumed that both the groups were equal prior to the application of treatment to the
experimental group.
• Further, one can always assume that the test units’ mortality affects each group equally. One can
always justify these assumptions by taking a large randomized sample. This design is widely used
in marketing research.
c. Solomon four-group design/ Six-study design/ Ideal controlled experiment
Presented as: Experiment Group 1 R O1 X O2
Control Group 1 R O3 O4
Experiment Group 2 R X O5
Control Group 2 R O6
• Helps the researcher to remove the influence of EVs and interactive testing effect.
• Test units are selected randomly in all the 4 groups. Experimental and control group 2 are not given
any pre-test measurement, whereas experimental and control group 1 are. Both experimental
groups 1 and 2 are subjected to the same treatment X at the same time.
• As experimental and control group 2 are not subjected to pre-test measurement, we would need
their estimates to remove the influence of EVs and interactive testing effect. As test units from all
the 4 groups are chosen randomly, it can be assumed that all the 4 groups are equal before
experiment. Therefore, the pre-test measurements O1 and O3 on experimental and control group 1
can be used as an estimate of the pre-test measurement of experimental and control group 2.
• To conduct this design, the time and cost required are enormous and therefore, this design is not
commonly used in research. However, it guarantees the maximum internal validity. In businesses
where establishing cause-and-effect relationship is very crucial for survival, this design is useful.
4. Statistical Designs
• Basic experiments that allow for statistical control of external
variables.
• Using these designs, more than one IV can be studied and their effect
on DV can be measured.
• Each test unit can be measured more than once to make the design
economical.
a. Completely Randomized Design
• To investigate the effect of one IV on the DV.
• The IV is required to be measured in nominal scale i.e. it should have a number of categories. Each
of the categories of the IV is considered as the treatment.
• CRD assumes that there are no differences in the test units. All test units are treated alike and
randomly assigned to the test groups. This means that no EVs that could influence the outcome.
• For eg. the sales of a product is influenced by its price level. In this case, sales are a DV and price is
the IV. There are 3 price levels– low, medium and high. We wish to determine the most effective
price level, i.e., at which price level the sale is highest. Here the test units are the stores which are
randomly assigned to the 3 treatment levels. The average sales for each price level is computed and
examined to see whether there is any significant difference in the sale at various price levels. The
statistical technique to test for such a difference is called ANOVA.
• The main limitation of CRD is that it does not take into account the effect of extraneous variables
on DV. The possible extraneous variables in the present example could be the size of the store, the
competitor’s price and price of the substitute product in question. This design assumes that all the
extraneous factors have the same influence on all the test units which may not be true in reality.
• This design is very simple and inexpensive to conduct.
b. Randomized Block Design
• As seen, the main limitation of CRD is that all EVs were assumed to be constant over all the
treatment groups. This may not be true. There may be extraneous variables influencing the DV.
• In RBD it is possible to separate the influence of one extraneous variable on a particular DV,
thereby providing a clear picture of the impact of treatment on test units.
• In the previous eg., price level (low, medium and high) was considered as an IV and all the test units
(stores) were assumed to be more or less equal. However, all stores may not be of the same size
and, could be classified as small, medium and large size stores. In RBD, the EV, like store size could
be treated as different blocks. Now the treatments are randomly assigned to the blocks in such a
way that each treatment appears in each block at least once. The purpose of forming these blocks
is that it is hoped that the scores of the test units within each block would be more or less
homogeneous when the treatment is absent. What is assumed here is that block (store size) is
correlated with the DV (sales). Blocking is always done prior to the application of the treatment in
such a way that variability within blocks is less than the variability between them.
• In this experiment one may randomly assign 12 small stores in such a way that there are 4 stores
for each of the 3 price levels. Similarly, 12 medium stores and 12 large stores may be randomly
assigned to 3 price levels. ANOVA could be employed to analyse the effect of treatment on the DV
and to isolate the influence of EV (store size) from the experiment.
c. Latin Square Design
• This design helps in separating out the influence of two extraneous variables.
• Suppose we want to study the influence of price (treatment) on sales. Let there be three levels of price
categories, namely, low (X1), medium (X2) and high (X3). The sales could be influenced by two extraneous
variables, namely, store size and type of packaging. For the application of the Latin square design, the
number of categories of two EVs should be equal to the number of levels of treatments. This is a necessary
condition for the use of Latin square design. The store could be of size – small (1), medium (2) and large (3)
and type of packaging could be I, II and III.
• The rows and columns represent those EVs whose effect is to be controlled and measured. There are 3
categories of row variable (store size) and 3 categories of column variable (packaging type). This would result
in 3 × 3 Latin square.
• The treatment should be assigned randomly to cells in such a way that each treatment occurs only once in
each row and in each column.
• Use of this design helps to measure statistically the effect of a treatment on the DV and also the
measurement of an error resulting from two EVs.
• This design has a very complex setup and is quite expensive to execute.
d. Factorial Design
• Employed to measure the effect of two or more IVs at various levels. It allows interaction
between the variables. Interaction takes place when the simultaneous effect of 2 or more
variables is different from the sum of their individual effects. A person may like mangoes and
also ice-cream, which does not mean that he would like mango ice-cream, leading to an
interaction effect separate from the individual effects.
• So if researchers are manipulating two or more independent variables, how exactly do they
know which effects are linked to which variables? It is true that when two manipulations are
operating simultaneously, it is impossible to disentangle their effects completely. Nevertheless,
researchers can explore the effects of each IV separately by averaging across all levels of the
other IV. This procedure is called looking at the main effect.
• The main advantages of factorial design are: It is possible to measure the main effects and
interaction effect of 2 or more IVs at various levels; It saves time and effort because all
observations are employed to study the effects of each factor; The conclusion has broader
applications as each factor is studied with different combinations of other factors.
• The limitation of this design is that the number of combinations (number of cells) increases
with increased number of factors and levels.
Example 1: Researchers want to test the effects of a memory-enhancing drug. Participants
are given 1 of 3 different drug doses, and then asked to either complete a simple or complex
memory task. The researchers note that the effects of the memory drug are more
pronounced with the simple memory tasks, but not as apparent when it comes to the
complex tasks. In this 3×2 factorial design, there is an interaction effect between the drug
dosage and the complexity of the memory task.
Example 2: A university wants to assess the starting salaries of their MBA graduates. The
study looks at graduates working in 4 employment areas: accounting, management, finance,
and marketing. In addition to that, the researchers also look at gender. Employment sector
and gender of the graduates are the IVs, and the starting salaries are the DVs. This would be
considered a 4×2 factorial design.
Example 3: Researchers want to determine how the amount of sleep a person gets the night
before an exam impacts performance on a math test the next day. But the experimenters
also know that many people like to have a cup of coffee (or two) in the morning to help them
get going. So, the researchers decide to look at how the amount of sleep and the amount of
caffeine influence test performance. The researchers then decide to look at three levels of
sleep (4 hours, 6 hours, and 8 hours) and only two levels of caffeine consumption (2 cups
versus no coffee). In this case, the study is a 3×2 factorial design.