A Virtual World Versus Face-to-Face Intervention Format to Promote Diabetes Self-Management Among African American Women: A Pilot Randomized Clinical Trial

Introduction

Type 2 diabetes is a complex chronic illness requiring continuing medical care and, ideally, patient adherence to numerous behavioral recommendations for self-management (ie, prescriptions for dietary change, physical activity, weight reduction, blood glucose self-monitoring, smoking cessation, and medication intake) [1] with the goal of achieving glucose control and preventing diabetes complications. Suboptimal control of diabetes places individuals at higher risk for diabetes complications [2].

There are considerable disparities in diabetes risk and outcomes in the population, with African Americans demonstrating among the highest diabetes prevalence and related morbidity and mortality [3,4]. Projected increases in incidence of diabetes may fuel even greater disparities in the future [3]. The traditional medical model involving repeated face-to-face visits over time may represent barriers to diabetes management, especially among underserved populations such as African Americans. Competing family responsibilities, distance to services, transportation difficulties and cost, cost of time away from work and other responsibilities, and difficulties accessing care [5] are among the reasons for limited participation in treatment among patients and may contribute to poor outcomes among African Americans.

With increased penetration rates of Internet use, at 81% in 2013 [6] (up from 71.7% in only 2011) [7], researchers have investigated the impact of delivering behavioral interventions via the Web. The Internet offers alternatives to the challenges typically associated with face-to-face lifestyle interventions through its potential for increased access to specialized behavior change experts, convenience to patients, and potentially lowered costs. However, while online alternatives show promising improvement in health behaviors and glycemic control, effect sizes have been small [8-11]. Limited human interactivity and engagement have been hypothesized as contributors to the small effect sizes [12,13]. In contrast, Virtual world technologies are potentially more suitable environments for supporting diabetes self-management programming. Through the use of three-dimensional (3D) environments that depict real places and avatars that represent people, virtual world environments offer opportunities for interaction, intense engagement, and opportunities for scripted immersive experiences, simulations, role-playing, and constructivist experiences, all important facilitators of active learning [14,15]. The use of virtual world environments continues to increase. There were 1772 million registered virtual world accounts in 2011, with 27 million users registered in Second Life alone [16,17]. The potential of virtual world environments for implementing or supplementing diabetes care interventions has been noted [18], but there is little evidence for the feasibility and potential effectiveness of such an approach [19].

This pilot study examined the feasibility of delivering a group-based diabetes self-management intervention via a virtual world environment (Second Life) and explored the potential effectiveness of the virtual world-based intervention, compared to a traditional face-to-face intervention, on self-management behaviors and glucose levels.

Methods

Design

A randomized clinical trial design was used. A detailed description of the study methods has already been published [20]. The Institutional Review Boards at Boston Medical Center and the University of Massachusetts Medical School approved the trial, and all participants provided written consent prior to participating in the trial.

Participants

Study participants were African American women identified from the medical record data warehouse at Boston Medical Center and affiliated community health centers as having a diagnosis of type 2 diabetes, age ≥18 years, English-speaking, HbA1c>8 at their last outpatient visit (within the previous 12 months), and excluded for medical conditions for which the intervention diet and physical activity would be contraindicated (ie, ulcerative colitis, renal failure, complications following abortion and ectopic and molar pregnancies, angina pectoris, and other forms of unstable ischemic heart disease and other conditions precluding brisk walking). Identified patients were mailed a letter to inform them about the study, to announce a phone call from study staff, and to provide the option to call in or opt out. The staff made up to five calls per patient (on different days and times). Those patients reached were informed about the study (ie, comparison of two formats for delivering a diabetes self-management intervention) and screened for interest and final eligibility (ie, self-reported ability to view a computer screen without difficulty, ability to read, no use of glucocorticoid therapy, no current participation in a weight loss program, and availability for weekly meetings). Fully eligible and interested women were invited to participate and scheduled for an in-person enrollment visit at the Boston Medical Center General Clinical Research Unit. At this visit, participants provided written informed consent and completed baseline assessments.

Randomization

Upon completion of baseline assessment measures, participants were randomized to either the virtual world-based intervention or the face-to-face intervention. Randomization was stratified by age and hemoglobin A1C measured at baseline using a block randomization scheme with a block size of 4, developed by StudyTRAX software (v3.0.0103).

Intervention Conditions

The intervention protocol was similar in both conditions, adapted from the Centers for Disease Control/National Institutes of Health program “Power to Prevent” [21], a widely available social cognitive theory-guided [14], evidence-based, and culturally appropriate behavior-change curriculum designed for delivery to African American groups with diabetes or pre-diabetes via face-to-face group sessions. The intervention sought to enhance diabetes knowledge, optimize attitudes toward diabetes self-management (ie, self-efficacy, outcome expectations), and develop behavioral self-management skills (eg, goal setting, tracking self-management behaviors and glucose levels, problem solving) to facilitate changes in diet, physical activity, blood glucose self-monitoring, and medication adherence. The first session used an individual format followed by eight weekly 90-minute group sessions (group size was 8-9 participants). A single intervention team (a registered dietitian who is a certified diabetes educator, and a nurse practitioner), trained in behavioral counseling and motivational interviewing principles, delivered all sessions in the virtual world environment or face-to-face using the same protocol consisting of a detailed intervention manual and materials (intervention delivery methods are described in greater detail elsewhere) [20]. Intervention fidelity was monitored, and providers were given feedback on behavioral counseling process and content. Participants in both conditions received a two-session computer training and were provided with an Internet-enabled laptop computer upon training completion (Internet access was standardized by providing high-speed 4th generation wireless modems to all participants).

The virtual world-based intervention was delivered in a mock open-air virtual world forum designed and programmed especially for the intervention with appropriate structures and visuals/displays (eg, food exhibits, confidence ruler, a ring of screens, exercise facilities). Participants were asked to log in 30 minutes prior to each session in order to troubleshoot connection or sound problems. A triage system to provide technical support as needed through the session was used. The face-to-face intervention took place in a large conference room at Boston Medical Center. All face-to-face participants received transportation vouchers to facilitate attendance.

Measures and Data Collection

Trained staff, blinded to study condition, conducted assessments at baseline and at 4-month follow-up. Clinical assessments included a non-fasting blood sample for HbA1c assays (specimens were analyzed at the Boston Medical Center laboratories) and measures of blood pressure, height, weight, and waist circumference using standard protocols [20]. At each baseline and follow-up assessment, two telephone-administered unannounced 24-hour recalls assessed diet and physical activity [22,23], blood glucose self-monitoring, and medication adherence. Survey measurements were verbally administered and included measures of depressive symptoms [24], self-efficacy for diabetes management [25], health literacy [26], social support [27], perceived stress [28], quality of life [29], demographics, and other characteristics, including baseline experience with computers and the Internet and post-intervention participant satisfaction. Intervention implementation costs (costs that would be incurred if the intervention were to be implemented outside the context of the research project) were tracked, including staff time, facilities, materials, and set-up (Second Life) for all sessions.

Data Analysis

An intent-to-treat approach was used to compare the virtual world versus face-to-face groups. Feasibility was assessed by comparing the attendance rate by session and the mean number of sessions attended within each arm. Implementation costs were also considered in determining feasibility. Cost estimates were based on expenditures from the trial and excluded the cost of equipment for participants in the virtual world groups. All primary and secondary outcomes were assessed for pre-post differences within each respective arm. The pre-post differences were then compared between the two arms. Differences between continuous variables were assessed using t tests. Binomial tests were used for categorical outcomes. Non-parametric tests were applied as appropriate. Analysis of potential mediators was conducted using the taxonomy and recommendations of Zhao [30]. Analyses were conducted in SAS 9.1 and R (version i386 2.153), all comparisons were two-sided and P<.05 was considered statistically significant.

We performed bivariate analysis of baseline characteristics to determine whether randomization achieved balance in both treatment groups across all characteristics. The results revealed a statistically significant difference in the proportion of participants with systolic blood pressure greater than 130 mmHg. While multivariate adjustment eliminated the statistical significance of systolic blood pressure at baseline, adjustment did reveal an imbalance in insulin use between the two treatment groups. Using a general linear regression to evaluate the association of insulin use on the pre-post change in HbA1c, we found that it did not have a statistically significant impact and did not affect our assessment of no difference between the virtual world group versus the face-to-face group.

Results

Of the 494 patients who were deemed pre-eligible based on medical records data (age 18 or greater, English-speaking, type 2 diabetes diagnosis, last HbA1c>8 within previous 12 months), it was not possible to determine the eligibility of 321 patients for reasons listed in Figure 1. Of the 174 (35%) who were reached for telephone screening, 62 (36%) were ineligible and 112 (64%) were eligible. From these 112 patients with known eligibility, 89 (79%) were enrolled and randomized, 46 of them to the virtual world intervention and 43 to the face-to-face intervention.

Table 1 summarizes demographic and baseline characteristics of participants in the trial: average age was 52 years (SD 10) and 90% of participants were over the age of 40; 60% had a high school education or lower; 82% reported a household income of US $30,000 or less; and experience with computers was variable. The single statistically significant difference between the virtual world and face-to-face groups at baseline was the proportion of participants with systolic blood pressure greater than 130 mmHg. A greater proportion of participants in the face-to-face group had elevated systolic pressure compared to those in the virtual world group (18% vs 10%, P=.04).

‎View this figure

Feasibility of the Virtual World-Based Intervention

Overall session attendance was similar across the two interventions, with an average 6.8 sessions (SD 1.8) among WV participants and 6.8 (SD 1.7) among face-to-face participants (P=.9). However, a significant difference was observed in the rate of completion of Session 1. Compared to face-to-face participants, fewer virtual world participants completed this session (78%, 36/46 vs 95%, 41/43, P=.02, respectively). There was also a slight although non-significant difference in the proportion of participants completing sessions 1-3 in the virtual world group (63%, 29/46) versus the face-to-face group (77%, 33/43) (P=.16).

Overall participant retention rate was 94% for clinical and psychosocial assessments, and 93% for telephone-based assessments, and attrition was lower in the virtual world group (1 participant, or 2%) compared to the face-to-face group (4 participants, or 9%) (P=.19). The one drop-out in the virtual world group was due to a new cancer diagnosis and unexpected hospitalization and occurred prior to session 1. Reasons for drop-out in the face-to-face group included dialysis onset and loss to follow-up (2 of these 4 participants attended Session 1).

Health Outcomes

Results from intention-to-treat analyses are shown in Table 2. There were improvements associated with both the virtual world and the face-to-face intervention conditions. Analysis of change from baseline to 4-month follow-up within the groups revealed a non-statistically significant 3.2% reduction in HbA1c in the virtual world group (P=.186) and a significant 4.9% reduction in HbA1c in the face-to-face group (P=.019). However, no significant differences between the groups were detected (P=.52). There was also a statistically significant within group decrease in the percentage of participants with HbA1c ≥9% in both groups, with 14% fewer participants in each group having a HbA1c value above 9.0% (P=.014 and P=.002, for virtual world and face-to-face groups, respectively), with no significant differences between the groups (P=.493). No significant within or between-group changes were observed in measures of blood pressure, total cholesterol, waist circumference, and Body Mass Index (BMI).

Behavioral Outcomes

Participants in the virtual world group experienced an 18.4% within-group increase in total physical activity and a significant 8% decrease in inactivity (P=.10 and P=.04, respectively), whereas participants in the face-to-face group experienced a 22.5% reduction in their total physical activity. There was a marginally significant between difference in total physical activity, with marginally superior effects for the virtual world compared to face-to-face group on total activity, light activity, and inactivity (P=.05, P=.07, and P=.025, respectively). The proportion of participants not adhering to blood glucose self-monitoring dropped by half in both groups (P=.001 and P=.002 for virtual world and face-to-face, respectively), with no significant between-group differences for this outcome. No significant within or between-group differences were observed for dietary outcomes of interest (ie, total calories, percent calories of saturated fat, fiber or dietary quality as measured by the Alternate Healthy Eating Index). Medication adherence decreased in the face-to-face group with 8.6% fewer participants reporting that they adhered to all medications as prescribed (P=.035), whereas there was a 1.2% increase in the virtual world group, although no differences between the groups with regards to change in self-reported medication adherence (P=.298).

Psychosocial Outcomes

Depression symptom scores and mental health functioning (as measured by the Short-Form survey [SF-12]) were marginally improved in the face-to-face condition only (P=.053 and P=.062, respectively), and there were between-group differences for depression symptom score pre-post change (P=.051). Improvements in self-efficacy for diabetes self-management were observed in both groups (P<.001), and there were no differences in self-efficacy improvements between the groups (P=.268). No within or between-group differences were observed for perceived stress or social support.

Mediation Analysis

We evaluated the potential mediation effects of select behavioral and psychosocial outcomes and found insufficient evidence of a mediation effect on HbA1c levels for changes in total calories consumed, total calories from saturated fat, total dietary fiber, alternate healthy eating index, diabetes self efficacy scores, inactivity levels, activity levels (household, light, and moderate activity), medication adherence post intervention, and blood glucose self-monitoring (data not shown).

Intervention Costs

The per-participant cost of implementing the virtual world intervention was US $186.39 greater compared to cost of implementing the face-to-face intervention (US $1117 vs $931 for virtual world vs face-to-face, respectively). Expenses associated with health care personnel (eg, diabetes nurse educator, dietitian, administrative staff) and educational materials were the same between the two groups. However, the virtual world group required additional technical personnel who trained and provided technical assistance to participants and the intervention team during each session, contributing to 13% of the total cost per participant in that group.

Participant Satisfaction

At the follow-up assessment, 97% of face-to-face participants agreed/strongly agreed with the statement “If I had a choice, I would attend diabetes sessions face to face rather than on a computer”, while 80% of virtual world participants agreed/strongly agreed with the statement “If I had a choice, I would attend diabetes sessions on Second Life rather than face-to-face at BMC or my health center” (P=.490). However, there were no differences between the groups with regard to whether they would recommend their program to other people; 100% of virtual world participants agreed/ strongly agreed with the statement “I would recommend other people to attend diabetes education sessions given in Second Life,” and 97% of face-to-face participants agreed/ strongly agreed with the statement “I would recommend other people to attend diabetes education sessions given face to face at BMC or a health center” (P=1.0).

Figure 2 contains pictures of virtual world intervention sessions and edited clips of the sessions (see also Multimedia Appendix 1).

‎View this figure

Discussion

Principal Findings

To our knowledge, this pilot randomized controlled trial (RCT) is the first study to compare the feasibility and potential comparative effectiveness of a virtual world-based versus an face-to-face-based diabetes self-management intervention. A previous publication reported on the feasibility of conducting individual virtual world-based visits with participants with diabetes, but the intervention did not include lifestyle modification, the study did not include a comparison condition, and metabolic outcomes were not reported [19]. Our study showed that it is feasible to deliver a virtual world group-based behavioral intervention originally designed for face-to-face delivery, to improve diabetes self-management among inner-city African American women. All participants who began the virtual world group completed the study, whereas 4 participants in the face-to-face condition did not.

Study findings show that the virtual world technology has tremendous potential for intervening and improving glucose control and diabetes self-management behaviors. Reductions in glucose levels were similar across both groups and were comparable to those of other group interventions [31]. Furthermore, HbA1c reductions were particularly significant among participants with the highest baseline glucose control (HbA1c≥9%).

Our study showed a marginal superiority of the virtual world intervention, compared to the face-to-face intervention, on physical activity (increase in total and light physical activity, and reduction in inactivity). A greater effect of the virtual world over the face-to-face intervention format on physical activity was also reported by Johnston et al [13]. In that study, the authors attributed this effect to the virtual world environment facilitating opportunities for the individual to initiate and practice healthy behaviors through an avatar with whom they identify. A study by Napolitano et al [32] that explored the usability of avatars for modeling weight loss behaviors provided additional support for the potential of virtual worlds for influencing diet and exercise behaviors. Two additional studies reported promising results from virtual world-based interventions for smoking cessation among rural teens [33,34]. Known as the Proteus effect, the practice of a new behavior by one’s avatar may influence the individual’s behavior in the real world [35].

The virtual world and face-to-face interventions were both comparable in terms of fostering blood glucose self-monitoring and enhancing diabetes management self-efficacy, and the face-to-face intervention was marginally superior compared to virtual world with regards to reducing depression symptoms. Social support interventions have improved depressive symptoms among people with diabetes, and it is possible that the face-to-face interaction influences participants in a different manner compared to virtual world interactions, potentially facilitating greater or a different type of social support [36].

Strengths and Limitations

There were technical challenges in the virtual world intervention that affected completion of the first session. Additional technical difficulties occurred with decreasing frequency over the course of the intervention. The technical support during the virtual world sessions was necessary to assure that each participant was able to navigate, hear, and interact in the virtual world. The two most commonly encountered technical challenges were strength of Internet connection and sound problems (could not hear or could not speak). virtual worlds demand significant bandwidth as they process enormous amounts of data to render the 3D spaces, physical interactions, and sound that characterize these environments, and our chosen Internet service, provided via wireless modem, was unreliable in the participants’ neighborhoods (inadequate cell tower coverage). We devised a triage system troubleshooting these problems (re-positioning in the home, and checks for headphone, laptop, and Second Life preferences options). Undoubtedly, virtual world platforms are rapidly improving and becoming more accessible and technically efficient.

Despite these challenges, participants seemed to be equally satisfied and engaged in both intervention groups. Consistent with the high level of satisfaction reported by participants in both groups, attrition was remarkably low in this hard-to-reach group of African American women. A previous study of a virtual world-based versus face-to-face-based intervention that targeted weight loss in a non-minority, educated and more affluent group reported a 13% drop-out rate in both groups, with 5 virtual world participants reportedly dropping out within the first 2 weeks of the program for reasons associated with technical difficulties [13].

The per-participant cost of the virtual world intervention was 13% higher than that of the face-to-face intervention, with the excess cost related primarily to the need for technical support staff to train and support our participants, as the study participants had variable levels of computer experience. This study is unable to answer the question of whether the increased cost of the virtual world intervention outweighs its potential benefit; however, it does provide evidence of feasibility and preliminary evidence of effectiveness for future larger RCTs that can answer such questions. It is important to note that the cost of virtual world interventions should be expected to decrease over time with improved technologies and Internet access for the wider population.

A particular strength of the study was the use of the virtual world format with a socioeconomically disadvantaged sample of African American women. African Americans constitute a high risk group with a high prevalence of diabetes, diabetes complications, and mortality [3,4]. The sample was middle-aged with most women having a high school education or lower, low household income, and variable computer experience. Furthermore, the sample consisted of 79% of participants who were reached and for whom eligibility was known, supporting the representativeness of the sample and potential generalizability of the study findings. Prior studies of Internet-based interventions have included primarily young individuals. While it has been hypothesized that demographic differences (including age, ethnicity, income, and culture) could impact the effect of interventions [37], our study found that the virtual world intervention was feasible and had benefits for our African American sample. The generalizability of Internet-based interventions also has been questioned based on potential selection bias by which individuals with low computer or Internet literacy may refrain from participation, may be excluded by the study’s eligibility criteria, or may more easily drop out from these interventions. For example, the only prior pilot study comparing a virtual world-based versus face-to-face-based interventions for weight loss [13] recruited virtual world participants via print and online media and excluded participants who had no access to an Internet-connected computer (73% of participants held college or advanced degrees and had incomes above $75,000). In contrast, our study systematically recruited patients from community health centers and a large safety net hospital using electronic databases, and minimized exclusion criteria in an effort to provide accurate data on the feasibility and outcomes of the virtual world intervention for inner-city African American women. The fact that the sample had variable computer experience and most participants had no prior exposure to virtual world environments provides further support for the potential generalizability of virtual world-based behavioral interventions.

Additional study strengths include the parallel content and structure of the virtual world and the face-to-face interventions, and their delivery by a single provider team, with only the intervention format being different across groups, and the tracking of cost data for both groups. While the possibility of reduced care cost has been an argument for Web-based interventions, very few studies have compared the costs associated with the implementation of virtual world versus face-to-face.

Conclusions

Future research is needed to test the comparative effectiveness of the virtual world and face-to-face interventions in larger, appropriately powered trials and with a longer follow-up. Future studies should also investigate characteristics of individuals who do best with each one of the two approaches. For example, it has been suggested that men may have a stronger experience of “presence” (ie, perceived realism, sense of being present) when being immersed in virtual world environments [38,39]. Furthermore, there is currently little understanding of potential mechanisms that facilitate health behavior change and adherence in virtual world environments. Future studies need to examine virtual world environment and avatar factors that facilitate health behavior change, including the degree to which one’s experience in the virtual world influences one’s behavior in the real world (Proteus effect) [35]. In 2010, the National Heart Lung and Blood Institute (NHLBI) convened a workshop entitled “Virtual Reality Technologies for Research and Education in Obesity and Diabetes”, which included behavioral and health researchers, technology experts, and representatives of the other National Institutes of Health institutes (National Cancer Institute, National Institute of Child Health and Human Development, National Institute of Diabetes and Digestive and Kidney Diseases, the NIH Office of Behavioral and Social Sciences Research, and the NIH Office of Research on Women’s Health). A report [40] from this workshop identified a number of research priorities, including the impact of using virtual reality technologies for fostering health-related behaviors and for extending the availability and capacity of health care providers (ie, “extended classrooms for diabetes education”). This study addressed both priorities.