PARTICIPANT GUIDE/CASE STUDY

Evaluating Public Health

Programs

Created: 2013
Evaluating Public Health Programs. Atlanta, GA: Centers for Disease Control and
Prevention (CDC), 2013.
EVALUATING PUBLIC HEALTH PROGRAMS

Table of Contents

EVALUATING PUBLIC HEALTH PROGRAMS CASE STUDY:

HYPERTENSION............................................................................................3
OVERVIEW ........................................................................................................................ 3
BACKGROUND: PART I ...................................................................................................... 3
BACKGROUND: PART II ..................................................................................................... 7
BACKGROUND: PART III .................................................................................................. 10
BACKGROUND: PART IV .................................................................................................. 13

PARTICIPANT GUIDE/CASE STUDY |2

 EVALUATING PUBLIC HEALTH PROGRAMS

Evaluating Public Health Programs Case

Study: Hypertension
OVERVIEW
In this case study, participants will engage in the six steps of the CDC Framework for
Program Evaluation: (1) engage stakeholders; (2) describe the program or project; (3)
focus the evaluation design; (4) gather credible evidence; (5) justify and state
conclusions; and (6) ensure use of evaluation and share lessons learned. The total
amount of time for this case study is estimated at approximately 5 hours (3 hours to
complete the activities and an additional 2 hours to review answers)1. The time allotment
per section varies, and is listed at the beginning of each step. Please allow additional time
for discussion of each step.

Please note: While this case study was inspired by real data, the people and events
described in this case study are fictitious. This exercise is intended for educational
purposes only.

BACKGROUND: PART I
A number of studies have reported the increasing prevalence rates of hypertension
among adults in Country X over the past 30 years. This increase in hypertension has also
been marked by a significant increase in reported prevalence rates of cardiovascular
diseases. In response, the Country X Centers for Disease Control and Prevention
(XCDC) of the Country X Ministry of Health funded the health bureau of City Y to deliver a
hypertension self-management program for adults aged 35 and older with uncontrolled
high blood pressure (≥140/90 mmHg). This evidence-based community intervention helps
patients take responsibility for managing their hypertension with assistance from health
care professionals, and support from families and the community.

The XCDC was the lead agency for the development, implementation and evaluation of
the hypertension self-management program. The City Y Health Bureau implemented the
program and was responsible for coordinating and communicating the program’s goals
and objectives with interested parties such as neighborhood groups and community
health center physicians. To ensure a successful and collaborative process, the City Y
Health Bureau convened multiple stakeholders from different levels – health care,
community, and patient – to plan the implementation and evaluation of the self-
management program.

1
Assumes four small working groups. Review time may take longer if there are more than four groups.
PARTICIPANT GUIDE/CASE STUDY |3
EVALUATING PUBLIC HEALTH PROGRAMS

In the initial phases of the program planning, the City Y Health Bureau identified and
recruited stakeholders through health center flyers, cardiovascular health care
newsletters, by word of mouth, and announcements at neighborhood group meetings.

The City Y Health Bureau invested significant time meeting with stakeholders to
appreciate their concerns about implementing the program. They relied on formal and
informal communication to ensure all perspectives were understood.

After the program began, new stakeholders became involved, including a pharmaceutical
company and representatives of the government health care insurance system. These
stakeholders were concerned with health care costs associated with medications and
hypertension screening. These interests were important to the project; therefore, XCDC
and City Y Health Bureau updated the program plan and focused the evaluation to
include these views.

As these stakeholders met and shared their ideas, the following concerns emerged:

Table 1: Stakeholder Interests

Early Stakeholders What Stakeholders Cared About Most

Family members of patients with Providing needed support to family

hypertension members with hypertension

Patients with hypertension Increasing self-efficacy and strengthening

support systems to self-manage
hypertension on a daily basis

City Y Health Bureau Preventing and controlling hypertension

among residents

Country X Ministry of Health Promoting national level policies that

prevent cardiovascular disease and risk
factors

Country X Center for Disease Control and Promoting national level policies that
Prevention prevent cardiovascular disease and risk
factors

Preventing and controlling hypertension

among the population

Community health center administrators Improving use of resources spent to

prevent and control hypertension; training
and building capacity of physicians and
PARTICIPANT GUIDE/CASE STUDY |4
 EVALUATING PUBLIC HEALTH PROGRAMS

Early Stakeholders What Stakeholders Cared About Most

other health professionals regarding the

hypertension self-management model

Community physicians (at community Preventing and controlling hypertension

health centers) among patients

Country X Hypertension League Supporting practical work and scientific

activities regarding hypertension and
related diseases

Country X Pharmaceutical Association Providing medication adherence

counseling training to pharmacists
through continuing education programs

Government health care insurance Controlling associated health care costs

companies

Neighborhood groups Developing environmental and social

supports in the community for people with
hypertension

Answer the following questions for Step 1 only. (40 minutes, including reading of
previous pages)
Step 1: Engage Stakeholders
1. Identify the stakeholders for whom you will involve in a program evaluation. Consider
the Utility standard when filling out the three columns in the table below. In particular,
think about:
• Who will use the results of the evaluation?
• Who can influence the use of the findings?

PARTICIPANT GUIDE/CASE STUDY |5

EVALUATING PUBLIC HEALTH PROGRAMS

Persons involved in Persons served or Primary users of the

program operations affected by the evaluation results
program

2. Refer to your completed table (question 1) and answer the following questions. Who
among the stakeholders will
a. Enhance credibility of the program:
_________________________________________________________

b. Implement the program changes:

_________________________________________________________

c. Advocate for changes:

_________________________________________________________

d. Fund, authorize, or expand the program:

_________________________________________________________

3. Complete the table below as follows:

a. List one stakeholder from question 1 from each of the categories (person involved
in program operations, person served or affected by the program, primary user of
evaluation results).
b. Describe how to engage each stakeholder in the evaluation process.
c. Describe a reasonable time commitment for each stakeholder’s involvement (e.g.,
quarterly meetings, weekly phone calls).

PARTICIPANT GUIDE/CASE STUDY |6

 EVALUATING PUBLIC HEALTH PROGRAMS

Types of Stakeholder How to Engage Reasonable Time

Stakeholders Stakeholder Commitment

Answer the following questions for Step 2 only. (1 hour)

Step 2: Describe the Program
Read the information below about the program goals and objectives. Then answer the
questions that follow.

BACKGROUND: PART II
Statement of Need (Statement of the problem): Hypertension is a major risk for
cardiovascular disease. Prevalence of hypertension (≥140/90 mmHg) was 39% among
residents aged 35 and older. A high percentage of patients with hypertension were
unaware of their condition, lacked access to proper and consistent treatment, and had
uncontrolled hypertension.

Goal: Reduce the proportion of adults with high blood pressure.

Objective: After 5 years of implementation, demonstrate a 25% increase in the proportion

of patients with diagnosed hypertension at participating community health centers with
blood pressure under control (<140/90 mmHg).

Program Description: Through a participatory planning process, the group of

stakeholders developed a long-term plan to successfully implement and evaluate the self-
management intervention in City Y to achieve this broad goal and objective. Components
of the intervention can be described at three levels: health care, individual/patient, and
community. A comprehensive approach would address each of these three levels.

• Health Care: Within the community health care system and among physicians,
the self-management model promoted comprehensive systems to support
patient self-management, which included physician telephone follow-up,
linkages to home blood pressure monitoring, and pharmacists trained to
provide self-management support and counseling. Additionally, community

PARTICIPANT GUIDE/CASE STUDY |7

EVALUATING PUBLIC HEALTH PROGRAMS

health centers may establish policies to increase patient adherence with

treatment, including medication and lifestyle changes.
• Individual/Patient: Another component of the intervention is self-management
training of patients with hypertension. Patients were referred to the training by a
staff member of a participating community health center in City Y. Over a six
week period, there is a total of 12 hours of training (2-hours weekly). The
primary objective of the training was to increase patients’ confidence to control
their blood pressure. This was achieved by participants’ mastery of the
essential skills for hypertension management including improved diet,
increased exercise, smoking cessation, self-control and effective
communication skills, awareness of community resources and treatment
options, and improved medication adherence and blood pressure self-
monitoring.
• Community: Some of the intervention activities at the community level
included establishing more collaborative partnerships and linkages between
community health centers and other community resources; availability of
community-based hypertension monitoring stations; conducting informational
media campaigns; and making changes to the environment that encourage
living a healthy lifestyle.

1. Based on the background information, identify and list the following elements of the
project on the table below OR as a logic model on the following page (or a flip chart):
• A minimum of three inputs
• A minimum of three activities
• A minimum of three outputs
• A minimum of one short-term outcome, one intermediate outcome, and one
long-term outcome

Inputs Activities Outputs Outcomes

PARTICIPANT GUIDE/CASE STUDY |8

 EVALUATING PUBLIC HEALTH PROGRAMS

Space for Logic Model

Inputs Activities Outputs Short-term Intermediate Long-term
Outcomes Outcomes Outcome

PARTICIPANT GUIDE/CASE STUDY |9

EVALUATING PUBLIC HEALTH PROGRAMS

Answer the following questions for Step 3 only.

Step 3: Focus the Evaluation
Read the background information below about the request for an evaluation. Then answer
the questions that follow.

BACKGROUND: PART III

The City Y Health Bureau implemented this hypertension self-management model training
program in one community health center. During the first year of implementation, the
bureau wanted to find out whether the program was implemented as planned (this is a
process measure) and whether the program had increased participants’ confidence to
control their blood pressure (refer to the short-term outcome from the logic model).

Before offering this program at other community health centers, the participating
community health center director and surrounding community health centers wanted to
learn from the evaluation whether the training program was effective, and make any
necessary improvements.

1. What is the purpose of this evaluation, as described in the text above? Please
circle all that apply.
a. Show accountability
b. Examine program implementation
c. Determine program improvement
d. Facilitate judgment about a program’s fate

2. Consider the purposes of the evaluation. What type of evaluation is this? Please circle
your answer.
a. Process evaluation
b. Outcome evaluation
c. Both process and outcome evaluation

3. Who are the users for this evaluation?

PARTICIPANT GUIDE/CASE STUDY |10

 EVALUATING PUBLIC HEALTH PROGRAMS

4. What is the use for this evaluation? Write a brief description.

5. Based on the utility and feasibility standards, is what you are being asked to evaluate
a reasonable request? Yes or no? Please explain. If you answered no, what can you
evaluate that is related to the request?

6. Based on your responses to the above questions, please give a minimum of two
examples of process and outcome evaluation questions you would need to ask.

Answer the following questions for Step 4 only. (20 minutes)

Step 4: Gather Credible Evidence
1. Record two evaluation questions you identified in Step 3 in the first column of the table
below.
2. Identify and list indicators for each question in the second column.
3. Identify and list the data sources or methods you will use to collect data about the
indicators in the third column. Select ones that can enhance the credibility of the data
with stakeholders.

Evaluation Question Indicators Data Sources /

Methods

PARTICIPANT GUIDE/CASE STUDY |11

EVALUATING PUBLIC HEALTH PROGRAMS

Evaluation Question Indicators Data Sources /

Methods

PARTICIPANT GUIDE/CASE STUDY |12

 EVALUATING PUBLIC HEALTH PROGRAMS

Answer the following questions for Step 5 only. (20 minutes)

Step 5: Justify conclusions
Read the background information below about the findings from the evaluation. Then
answer the questions that follow.

BACKGROUND: PART IV
The main objective of the self-management program is to demonstrate a 25% increase in
the proportion of patients with diagnosed hypertension at participating community health
centers with blood pressure under control. Twelve months after the start of the program,
the interim evaluation findings indicated an 11% increase among participating patients. It
was discovered that all of the participating community health centers adopted systems-
level changes within their healthcare delivery system to promote patient self-management.
However not all of the sites adopted a comprehensive self-management model.*

Figure 3 illustrates data collected over the two year period summarizing the number of
community health centers in City Y with policy or systems-level changes to encourage
patient self-management of hypertension. It was noted that Site A experienced the
highest level of participation in the self-management training sessions among their
patients.

Findings also indicated a significant increase in patients’ awareness and knowledge of

chronic disease related risk factors. An increase in patient self-management ability was
determined by a 15% increase in the number of patients who do self-monitoring of blood
pressure and a 20% increase in the number of patients who regularly receive counseling
and support from a pharmacist.

Participation in regular exercise also increased by 12%. Additionally, the evaluation

documented an improvement in adherence to medication regimens and community
environmental changes such as increasing access to safe and free physical activity
facilities. These changes were viewed by stakeholders as positive outcomes resulting
from the program.

*Sites with comprehensive self-management models addressed all 3 levels of the

intervention—health care, individual/patient, and community (see “Program Description”
in Step 2).

PARTICIPANT GUIDE/CASE STUDY |13

EVALUATING PUBLIC HEALTH PROGRAMS

.Figure 2: Graph of Changes in Health Centers

Number of Systems-level Changes at Participating

Community Health Centers, 2012-2013

3
Number of systems-level

2
Site A
changes

Site B
1 Site C

0
Baseline 2012 2013

1. Identify at least two evaluation results and write a corresponding recommendation for
each evaluation result in the space below.

2. Refer to your recommendations from the previous question, and consider the four
standards for evaluation. Then complete the following table and justify your answers.

Question Answer Answer Justification

Yes No
Utility: Have different Yes No
interpretations of the findings
been considered?
Feasibility: Are the Yes No
recommendations realistic for
the program to implement?
Propriety: Are the Yes No
conclusions and
recommendations reflective
and respectful of key
stakeholders, including those
served by the program?
Accuracy: Can the Yes No
conclusions be explicitly
justified?

PARTICIPANT GUIDE/CASE STUDY |14

 EVALUATING PUBLIC HEALTH PROGRAMS

Answer the following questions for Step 6 only. (20 minutes)

Step 6: Ensure Use of Evaluation
1. Describe various methods for communicating the evaluation findings:

2. In the table below, list the two stakeholders you identified in question 1 and then
complete the remaining columns with regards to communicating the evaluation results
to each stakeholder.

Stakeholder What to Method of Frequency

Communicate Communication

3. What will you do to make sure that the evaluation is reported in a manner that
encourages follow-through by the stakeholders?

PARTICIPANT GUIDE/CASE STUDY |15