See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/349502367

Survey Design~Quantitative research

Presentation · February 2021

DOI: 10.13140/RG.2.2.10886.50242

CITATION READS

1 10,969

1 author:

Megh Raj Dangal

Kathmandu University
66 PUBLICATIONS 157 CITATIONS

SEE PROFILE

All content following this page was uploaded by Megh Raj Dangal on 22 February 2021.

The user has requested enhancement of the downloaded file.

Research Methodology: Survey Design

Megh Raj Dangal, PhD

Associate Professor, Kathmandu University
2021
[email protected]
98510-98003
 State question and hypotheses, identify variables

Determine design structure

Identify population and sample

Planning Stage

Design instruments and classify operational definitions

Select statistical test for resolving hypotheses/RQs

Stages of designing and carrying out a study

Execution Stage

Carry out plan, collect data

Analyze data, draw conclusions and evaluate

process
 Survey Design: What?
 “means for gathering information about the characteristics,
actions, or opinions of a large group of people”
 three distinguishing characteristics of survey research
 First, survey research is used to quantitatively describe specific
aspects of a given population (target pop and survey pop)
 Second, the data required for survey research are collected from
people and are, therefore, subjective
 Finally, survey research uses a selected portion of the population from
which the findings can later be generalized back to the population

“ The success of survey research depends on how closely

the answers that people give to survey questions match how
people think and act in reality” (Avedian, 2014, p. 4).

Survey Strengths
 Capable of obtaining information from large samples of
the population

 Inclusive in the types and number of variables that can be

studied, require minimal investment to develop and
administer, and are relatively easy for making
generalizations

 Provide estimates for the true population, not exact

measurements
Survey Weaknesses
 Unsuitable where an understanding of the historical
context of phenomena is required

 Difficulty assessing their own behavior or have poor

recall of the circumstances surrounding their behavior
 Surveys by type of study design (Avedian,
2014, p. 4)


 Descriptive – Graphical and numerical methods for summarizing
(describing) the data – Describe phenomena and summarize them.
 Graphs, tables and numerical summaries are all examples of descriptive
statistics

 Inferential – Making predictions based on the data– Inferential
Statistics uses methods for making predictions about a
population (total set of subjects of interest), based on data from a
sample (subset of the population on which study collects data).
 Measure associations, e.g. income and quality of life.
Survey Designs (Avedian, 2014, p. 4)

 Cross-sectional surveys: – Data collected at one point in

time selected to represent a larger population.

 Longitudinal surveys:
 – Trend: Surveys of sample population at different time
points.
 – Cohort: Study of sample population each time data are
collected but samples studied maybe different.
 – Panel: Data collection at various time points with the same
sample of respondents.
 According Bryman (2008), five prominent research designs are
outlined:
 Experimental and related designs (such as the quasi-experiment)

 Cross-sectional design, the most common form which is survey

research

 Longitudinal design and its various forms, such as the panel study
and the cohort study

 Case study design

 Comparative design
Six steps to better survey design (OECD,
2012)
 Step 1. Define survey objectives, use of results and
target population

 Define the objectives Research Questions Hypothesis


 Define the final use of the results

 Ensure a perception survey is the adequate tool

 Define target group(s)

Step 2. Draft survey questions/ Questionnaires
 Set up discussions with members of a target group to
identify key issues
 Translate those into questions and answer categories
 Draft simple and clear questions
 Keep the questionnaire short to maximise response
rate and concentration
 Ensure respondents have the opportunity to report
problems
 Questions for questionnaires
 1. Do the answers to the questions help meet the objectives of the survey?
 2. Do the questions address the most bothering issues of the target population?
 3. Is the language simple and devoid of technical jargon?
 4. Are key terms such as “regulation” clearly defined?
 5. Do you avoid asking two questions in one, i.e. do all questions only ask one question at a
time?
 6. Are questions clear and precise enough that they will be consistently understood in the
same way by all respondents?
 7. Are the formulation of questions and answer choices and their order as neutral as
possible, i.e. do they avoid suggesting answers?
 8. Are the answer choices and scales clearly defined and consistently understood across
respondents? Have both been chosen carefully?
 9. Does the target population have the capacity and knowledge to answer all questions?
 10. Have screening questions been included, that is, has the same question been asked in
different ways to identify consistent respondents and meaningful responses?
 11. Have tricky questions been included towards the end of the survey when respondents
feel more comfortable answering them?
 12. Is the questionnaire short enough to ensure that respondents will concentrate until the
end?
Step 3. Pilot and re-adjusting the questionnaire

 Test the survey on a smaller-scale target group to identify

weaknesses in the survey design

 Possibly ask volunteers to think aloud while answering

questions and analyse what motivated their answers

 Adjust questionnaire if needed

Step 4. Select respondents and the data
collection method
 Select a sample either by random sampling or other
methods
 Ensure that the sample size allows to draw valid
conclusions from the results
 Choose the data collection procedures: personal
interviews, telephone interviews, Internet surveys,
email surveys, etc.
 Maximise response rate through appropriate data
collection method
Step 5. Run the survey
 Ensure high response-rate through follow-up emails
otherwise conclusions to the survey could be biased

 Use trained interviewers to avoid unintentional influence

on responses
Step 6. Analyse the results
 Interpret results as perceptions rather than facts

 Take into account the response rate. A low rate means

that no general conclusions can be drawn

 Take into consideration the number and the way

respondents have been selected in the result analysis

 Understand how results were reached is essential to

draw policy conclusions
Errors: During surveys (Engel and Schutt (2010, p. 179) .
 Question Wording: Does the question have a consistent
meaning to respondents? Problems can occur with
 Lengthy wording Words are unnecessarily long and complicated.
 Length of question Question is unnecessarily long.
 Lack of specificity Question does not specify the desired information.
 Lack of frame of reference Question does not specify what reference
comparisons should be made to.
 Vague language Words and phrases can have different meanings to
respondents.
 Double negatives Question uses two or more negative phrases.
 Double barreled Question actually asks two or more questions.
 Using jargon and initials Phrasing uses professional or academic discipline-
specific terms.
 Leading questions Question uses phrasing meant to bias the response.
 Cultural differences in meaning Phrases or words have different meanings to
different population subgroups.
 Respondent Characteristics: Characteristics of
respondents may produce inaccurate answers. These
include
 Memory recall Problems remembering events or details about
events.
 Telescoping Remembering events as happening more recently than
when they really occurred.
 Agreement or acquiescence bias Tendency for respondents to “agree.”
 Social desirability Tendency to want to appear in a positive light and
therefore providing the desirable response.
 Floaters Respondents who choose a substantive answer when they
really do not know.
 Fence-sitters People who see themselves as being neutral so as not to
give the wrong answer.
 Sensitive questions Questions deemed too personal.
 Presentation of Questions: The structure of questions
and the survey instrument may produce errors
including
 Open-ended questions Response categories are not provided, left
to respondent to provide.
 Closed-ended questions Possible response categories are provided.
 Agree-disagree Tendency to agree when only two choices are
offered.
 Question order The context or order of questions can affect
subsequent responses as respondents try to remain consistent.
 Response set Giving the same response to a series of questions.
 Filter questions Questions used to determine if other questions
are relevant.
 Interviewer: The use of an interviewer may
produce error.
 Mismatch of interviewer-interviewee demographic
characteristics.

 Unconscious judgmental actions to responses.

View publication stats