Dcwguide A
Dcwguide A
Dcwguide A
Appendix A includes two examples of how providers may use the survey instruments
and/or their subscales included in this Guide. One example shows how a nursing home
decided to measure one topic of interest (Job Design) among CNAs. The other
illustrates how a continuing care retirement community (CCRC) constructed a multi-
topic survey instrument from among the scales/subscales in the Guide. Both scenarios
follow the steps laid out in Appendix C on data collection planning and implementation
issues.
A nursing home is experiencing high turnover among its CNAs. The Administrator
wants to identify the parts of their jobs that CNAs are most concerned about and those
that are least rewarding. Using this information, she would like to decide what actions
management can take to try to address some of these problems.
The Administrator has started to call the nursing departments at local universities in an
attempt to identify potential researchers with whom she can collaborate. She has also
asked her Director of Human Resources to obtain price quotes from data collection
vendors for conducting an employee survey of all 40 CNAs. Lastly, she has asked her
Director of Finance to assess what kind of budget the organization has for staff
development as she realizes she will have to act on the survey findings in order to
maintain credibility among her CNAs. The survey fielding/data collection period will last
for three weeks.
Step #4: Decide whether to include all members of the population or a sample
A-1
Step #5: Decide the topics, subscales, and/or formulas on which to collect data
The nursing home Administrator has heard rumors about tension between CNAs and
certain charge nurses. While she knows for certain that this topic area is one that her
CNAs will be asked about, she really wants to keep the employee survey broad so she
can really get at what may be causing the high CNA turnover she is experiencing.
Step #6: Decide how the questionnaire will be administered and set the response
rate goal
The Administrator was able to form a relationship with a local researcher who will
oversee the data collection process. Each CNA will receive an advance letter informing
them of the survey and its goals. After the letters are distributed, a CNA staff meeting
will be held to allow a question-and-answer period focused on the survey. The project
team has determined that the survey will be administered by the researcher at pre-
appointed times for each employee on each shift in a common area. Free food will be
available in this common area during these times. A lock box will be placed in this room
so that employees will feel comfortable responding. The goal is to have a 100 percent
response rate among CNAs.
Together, the Administrator and researcher examined the instruments in the Guide
around the topic areas of job satisfaction, job design and worker-supervisor
relationships. Using the advice of her research collaborator, the Administrator has
decided to keep the survey broad and use subscales of the Job Role Quality
Questionnaire. She is particularly interested in assessing the degree to which her
CNAs are concerned about the workplace environment and the degree to which they
find certain aspects of their jobs rewarding in order to inform an appropriate
organizational response. The JRQ includes five items to measure “job concern factors”
and six items to measure “job reward factors.”
The Director of Nursing has recruited CNAs on each shift with whom the research
collaborator, an objective outside source, will hold focus groups to get feedback on the
length and content of the questionnaire.
The researcher has agreed to provide frequent reports on response rates among the
CNAs via email and phone calls with the Administrator, Director of Nursing, and Director
of Human Resources. The survey will be conducted over a three-week period.
A-2
Step #9: Analyze data and present findings
Scores for each item in the eleven subscales were added across the 40 CNAs and then
averaged. The results for each item are as follows:
*For job concern factors, a lower score reflects better job design.
* For job reward factors, a lower score represents poorer job design.
The results of the survey show that the Administrator’s suspicion that the employee-
supervisor relationships may need to be strengthened has been reinforced. Based on
survey results, it appears that CNAs seem most concerned about the poor supervision
they receive. They report the least rewarding parts of their jobs to be supervisor
support and recognition they receive.
The Administrator and Director of Nursing are putting together a presentation for the
next CNA staff meeting to report the survey results. At that time, they will solicit CNAs
who would like to work on a team to develop a strategic plan for improving employee
supervisor relationships and the overall work environment of the nursing home.
A CCRC wants to see how committed its employees are, how empowered they feel,
and whether those who feel more empowered are more likely to be committed to their
employer. The Administrator would like to see how employees’ perceptions in these
areas differ across department so that an informed organizational response can be
developed.
A-3
Step #2: Specify the target population for data collection
This CCRC has a research unit on campus, so the Administrator will work with the
Director of Research on campus to develop a reasonable schedule. The Administrator
will also coordinate with the Director of Finance so the appropriate distribution of
resources across the CCRC and its research unit is clearly spelled out among all
parties.
The team determined ahead of time that the survey will be administered in-person,
since many of the employees are Spanish-speaking. Added expenses the team has
already considered include the hiring of outside interpreters, time staff spends on
completing surveys (and the overhead costs associated with a lengthy survey process
as a result), and efforts to increase response rate. The survey fielding/data collection
period will last six weeks, including survey administration and follow-up to improve
response rate.
Step #4: Decide whether to include all members of the population or a sample
Given the cost of doing in-person interviews and the number of staff members at the
CCRC, the Administrator and Director of Research decided to draw a random sample of
the 1,100 employees at the CCRC. Employees from all departments, on all shifts will
be included in the random sample. The research unit will ensure that enough
employees are drawn from each department to make appropriate comparisons and to
ensure confidentiality.
Step #5: Decide the topics, subscales, and/or formulas on which to collect data
A-4
The specific items selected, which include entire subscales,1 are:
Here are some statements about you and your job. How much do you agree or
disagree with each? (Likert Scale ranging from 1-7, where 1 = strongly disagree and 7
= strongly agree.)
Item #1. I will probably look for a new job in the next year.
Item #3. How likely is it that you could find a job with another employer with about the
same pay and benefits you now have? (Likert Scale ranging from 1-7, where 1
= not likely at all and 7 = extremely likely)
Items from the Conditions for Work Effectiveness Questionnaire (CWEQ I) and (CWEQ
II Short Form) -- Opportunity subscale (3 items):
Items #1-#3. How much of each kind of opportunity do you have in your present job?
(Likert Scale ranging from 1-5, where 1 = None and 5 = A Lot)
• Challenging work.
• The chance to gain new skills and knowledge on the job.
• Tasks that use all of your own skills and knowledge.
Step #6: Decide how the questionnaire will be administered and set the response
rate goal
The team determined that it would like a 70-percent response rate across the CCRC.
The questionnaire will be administered in-person. The Director of Human Resources
will work with each department head to schedule times for employee surveys. At least 2
interviewers will be available per shift. Survey interviews will be conducted on campus,
in areas away from the departments in which interviewed employees work. Advance
letters and reminder postcards will be mailed to selected staff at their residence, flyers
will be posted throughout the campus and a “Share your Voice!” kick-off meeting will be
held the week survey administration begins where refreshments will be served and door
prizes given out.
1
It is important to include all items in a subscale because our review and the findings on the properties of the
instruments reported in this Guide are based on the entire subscales (not individual items within each subscale). If
you choose to take only some items from a subscale, the properties we reported (e.g., reading level, reliability,
validity) do not apply to the individual items.
A-5
Step #7: Design and pretest the questionnaire
Because this CCRC has many Spanish-speaking employees, cognitive testing will be
done on the survey item translation. Pretesting will also be conducted with 10 English-
speaking employees.
Survey fielding will be conducted over a six-week period. The research unit on the
CCRC campus and the department heads will meet weekly to discuss the progress of
data collection. That way, the team is updated on the survey response rate and can,
subsequently, strategize on what types of efforts are needed to increase the response
rate (if any).
Below is an illustrative example of the scores that correspond to answers three workers
gave to the six questionnaire items. The CCRC will tabulate the results of all
responding employees (of the random sample) using this same scoring process.
To calculate the score for each employee for the behavioral intent to leave job subscale,
sum the scores given for all three items. In this example, below are the scores for each
worker on this organizational commitment measure.
Worker #1: 8 = (2 + 3 + 3)
Worker #2: 12 = (6 + 4 + 2)
Worker #3: 10 = (3 + 3 + 4)
A-6
To calculate the score for each employee for the opportunity subscale of the “Conditions
for Work Effectiveness Questionnaire II,” average the scores given for all three items.
In this example, below are the scores for each worker on this empowerment measure.
Higher scores on this measure indicate greater empowerment in the form of more
perceived opportunity, with possible scores on this 3-item measure ranging from 1 to 5.
At the individual worker level, worker #3 shows the greatest level of empowerment
(score of 4.0) followed by worker #1 (score of 3.3), with worker #2 (score of 2.7)
showing the least empowerment.
The average is usually the statistic used to indicate the summary score on a measure
across all respondents when using Likert-type response scales. Using the
empowerment measure above as an example, here is how to calculate the average
empowerment score for all respondents.
Working through this formula we get these figures below, for an average of 3.3 among
all three workers:
So, on average, this sample of workers at this CCRC tend to feel that they have “some”
opportunities at work. However, based on the score of 3.3, there is room for
improvement toward a score of 4 or 5.
Management believes the needs of each department may differ and has decided to put
together employee focus groups for each department. The goal of these focus groups
is to get a sense of the types of things needed to make employees feel more
empowered in their jobs. All employees will be given the opportunity to be part of the
focus groups. After examining each department’s focus group findings and comparing
across departments, management will work with teams of staff members (across all
departments and titles) to determine how to allocate resources across all staff in the
best manner. The ultimate goal is to increase satisfaction with the working environment
and to improve retention of staff. Results of each stage in the process will be shared at
all-staff meetings.
A-7
MEASURING LONG-TERM CARE WORK:
A Guide to Selected Instruments to Examine Direct
Care Worker Experiences and Outcome
CHAPTER 2: How This Guide Can Help Organizations Use Information to Address the
Challenges of Job Retention and Performance Among DCWs
http://aspe.hhs.gov/daltcp/reports/dcwguide2.pdf
References http://aspe.hhs.gov/daltcp/reports/dcwguide4.pdf
APPENDIX A: From Start to Finish -- Sample Scenarios of Using and/or Constructing Survey
Instruments http://aspe.hhs.gov/daltcp/reports/dcwguideA.pdf
APPENDIX E: Individual Measures from Chapter 3 that Use Survey Instruments to Collect
Data, By Topic http://aspe.hhs.gov/daltcp/reports/dcwguideE.pdf