Home electronic health information for the consumer: user evaluation of a DiTV video‐on‐demand service

Recruitment and sample

A number of different groups were recruited to view the videos and undertake the evaluation. These were:

• •

  Service users. An online questionnaire, activated when subscribers accessed the service, asked for interviewee volunteers. A total of 12 people offered their contact details, but eventually only five were willing to participate in the evaluation. The real user group consisted of three males and two females, aged between 25 and 60.

• •

  Elderly people. A group of eight elderly people (four men, four women) were recruited from the voluntary organisation Third Age (Third Age Project, 2001). No attempt was made to obtain people with particular demographic or other characteristics. Their ages ranged from 70 to 95, and they viewed a video (on chronic heart disease (CHD)) in a group at the offices of Third Age.

• •

  Diabetics. Three diabetics were recruited, the first being a friend of one of the students at City University and the others by “snowballing” (i.e. a friend of the first interviewee and then a contact of the second). All evaluators were females, between the ages of 20 and 30, and had been diagnosed at least two years before they saw the programme. It was not possible for these people to see the videos together and so individual sessions were held with two at City University. The third took the video home and was interviewed about her evaluation later, at her place of work.

• •

  Health professionals. These were recruited from City University School of Nursing and Midwifery, and consisted of two lecturers (one senior) in health and health policy who had been nurses, one nurse working at Whittington Hospital, and four student nurses. The lecturers and nurse saw the videos individually; two of the students saw them in a wider group (consisting of those with a general interest, described below) and the other had an individual viewing session with the researcher.

• •

  Health information professionals. Three health information professionals and one Master’s degree student working in a health environment were recruited, again by “snowballing”.

• •

  General interest (i.e. “lay”) volunteers. These were people who had no professional interest in health, nor a particular condition covered by any of the videos. They were, rather, simply interested individuals, 12 in total, recruited via the other interviewees listed above. Of these, six were City students, one an ex‐student, the others being their friends or colleagues. Included in the total are three friends of the researcher. In all, five watched the videos individually at home, reporting back later, the others with members of the other groups at City University.

Methodology

A semi‐structured interview approach was adopted. For the service users this was looser than for other groups, as there might have been issues in their usage which had not been considered by the researchers when drawing up the interview schedule. As they were interviewed about videos they had viewed in the normal course of their use of the Video Networks service, the evaluation was retrospective, and included details of the accessibility of the system, in terms of menu hierarchies etc. With regard to those participants who viewed the videos specifically for the evaluation, the programmes were copied onto standard videotape and shown on analogue televisions. The service users were interviewed individually. Where possible representatives from each group watched the programmes together, at City University, and discussed them in an open forum. Time constraints and availability problems meant that some evaluators viewed the videos individually, with the researcher. In two cases (one of the diabetics, and a health professional) videos were taken for home viewing, with the researcher interviewing the evaluators later, also individually.

Before the showings (and lending), participants were told a little about the research project and about the DiTV service from which the videos had been taken. (The actual service users had, of course, already seen the videos beforehand, for their own benefit.) An outline was given of the issues to be discussed after the showing. An evaluation form had been drawn up to reflect the aims of the exercise, and this was referred to but not looked at in detail until the viewers had watch the first section of the video chosen. This was so that they would have an idea of content and presentation etc. before needing to consider these and other aspects in detail, as outlined on the form. In every case, even where the researcher was present at the showing and had the opportunity to discuss the programme “live”, volunteers completed the evaluation form (one for every video viewed). This both provided a structure for the discussion and a written record of views.

The service user group was in a position to provide information on all the following topics, which were not appropriate to those viewing the programmes especially for the evaluation:

• •

  Reasons: for turning to the service.

• •

  Context of use: how use of the service related to wider health information seeking behaviour.

• •

  Relevance: the relevance and usefulness of the information regarding the particular circumstances for which it was sought.

• •

  Uses: how the information was used, and any benefits that accrued (such as being better equipped to cope with a medical problem or deal with a medical professional.

• •

  Usability of the service: the ease with which the videos were accessed from the menu hierarchy.

The areas discussed by all participants (i.e. the service users and the other groups) were:

• •

  Information content: the comprehensiveness and appeal of the information and, in its broader sense, its potential relevance.

• •

  Presentation: order of scenes, production, performance of doctors/lay participants.

• •

  Language and terminology: medical language, descriptions of procedures etc.

• •

  Format: appropriateness of the topic for the medium, and whether any of the content could have been better presented in alternative forms.

Evaluators could raise any points not covered above, either in person or on the evaluation form.

The videos

The service users, of course, had seen any one or more of the videos, and were asked for their views and reactions to the ones they had seen. For the other groups, a selection was chosen for viewing that reflected the range available and type available. After viewing the majority of the videos, it was possible to informally classify them to the extent necessary to choose a representative sample for evaluation. The categories, with the videos subsequently chosen, were:

• •

  Specific conditions (CHD, diabetes).

• •

  General condition management (pain management).

• •

  Procedural information (giving blood).

• •

  Practical information/demonstrations for specific demographic groups (exercise for older people).

The videos with predominantly medical content (CHD, Diabetes and Pain Management) had a similar production style. A GP or similar expert spoke to camera or, at times, voiced over a specific scene (such as a consultation). The videos were broken down into standard sections (accessible individually), such as “Symptoms” “Medication”, and “Patient experiences”. The latter consists of individual patients describing their experiences to camera. The Giving Blood programme is very different. It is not split into sections and also, unlike the medical conditions programmes, it follows a person through a procedure – it is chronological, and it places the patient as the central focus. Finally, the Exercise for the Elderly video (and others, such as First Aid) consists of demonstration activities.

Usage issues

Only the service user group were asked about issues related to usage, as those of other groups viewed the videos especially for the evaluation.

Information content

The service users were generally positive in their appraisal of the video content. The respondent who browsed through several declared that “generally they were very informative, not patronising and would be useful to go back to”, and another said of the diabetics video, that “the information was good – what I would want to know”. The osteoporosis video viewer said that it “made me aware of what I had, the reason that I had it, and what the prognosis is like and how to slow down the progression of the disease”. Although one interviewee complained that the programme, at 15 minutes, required quite a bit of concentration (in fact, none of the individual clips last more than five minutes), other interviewees felt that, in fact, there was not enough information included. One respondent even suggested an option to access more clips treating the same topic but in greater depth:

For example, it could have a couple of more clips on the latest research on the subject which it is talking about and new scientific discoveries or something similar, or maybe even some controversies involved in researching the subject concerned.

The groups viewing the videos for evaluative purposes only were also generally positive about programme content. The general interest evaluators said that the information was generally “comprehensive”, for both the Diabetes and CHD videos. However, the medical and information professionals felt that these videos “tried to cover too much ground too quickly”. There was “too much to take in”, and many topics were “poorly explained”. One of the medical professionals felt that the subject matter was “difficult”, and the conditions, particularly CHD, “complex”. The differences in opinions about the videos between medical professionals and other evaluators is interesting, and is explored in detail in the discussion section later in this report.

The elderly group saw only the video on CHD. Their views are valuable, partly because they saw the programme without members of any other group being present, partly because there were more of them to provide input (eight), but mainly because they are a group targeted by the Department of Health as requiring (digital) health information. The Wanless Report (Wanless, 2002) makes much of the fact that the population is ageing, and that the high number of elderly people in the future will place a strain on the NHS that could be reduced by higher engagement with and responsibility for their own health. The group was very impressed with the content of the CHD video, describing it as “marvellous”, “really informative” and “a good thing to help people with the condition”. They accepted that they were a group that needed information on coronary problems.

It is worth discussing the video on diabetes in some detail, as it was possible to recruit for the evaluation those who actually had the condition, providing the perspective of those at the front line and for whom the information was written and health professionals with an intimate knowledge of the condition. There were, in fact, interesting contrasts between the views of these two groups. The former were far more positive. The diabetics felt that the content, as one put it, “was all very relevant and covered the most important points”. The other diabetics also indicated they were happy with the content. The general viewers and information professionals also considered the content “relevant”, “comprehensive” and “interesting”. One of the medical professionals (a nurse), however, suggested that much of the material was not very clear and that some of it was too detailed. Another said there was, at 22 minutes, too much content. There were, from other quarters – the diabetics and an information professional – suggestions for the inclusion of additional material. One pointed out that the “important topic” of genetic inheritance was not covered. Another considered that it would have been helpful to have had a section on how a newly diagnosed patient should expect to be treated by the health service, in terms of engagement with surgeries, clinics etc. and the roles that might be expected of GPs, nurses and others. Finally, one of the diabetics argued that there should be something included about the psychology and mental/psychiatric implications of being diagnosed with the condition, or “at least” the social implications and how to deal with them. There were also questions that the content raised and appeared not to answer. These included why diabetes causes eye/heart/kidney damage and why fat needs to be controlled as well as sugar.

The Giving Blood video, made in a different style to the others of a purely medical content, was very positively regarded by all the evaluators. The content was considered “interesting” and “appropriate”, with the procedure described in an easy to understand and logical order. There were, however, (almost inevitably) suggestions for additional information to be provided. This was principally with regard to the benefits a blood bank gave to a hospital in treating patients and the value, therefore, of giving blood. As these benefits were not emphasised, in the opinion of the evaluators, the video was described as appearing to aim at the “already committed”. It was felt that the programme should have aimed more at recruiting donors. Highlighting the benefits one gave to others would, it was felt, encourage the activity. As a footnote to this point it may be said that as the videos are “on demand” those electing to view this particular one would already have considered the possibility of becoming a donor – just as those watching a programme on healthy eating may be already thinking about some action on their part. This is unlike the case where a broadcast programme may reach people who are watching by chance and get the idea of giving blood specifically from the programme.

Presentation

There were mixed views with regard to the presentational aspects. The presenters (i.e. doctors) were criticised as being “boring” and “uninspiring”, with one viewer, a nurse, complaining “they reinforced the stereotype”. When asked to expand she said they were “aloof, distant and unfeeling”. Members of the elderly group felt that the doctor featured was “typical” in this respect, although others said that their own doctors were friendlier. The video was redeemed by the participation of patients (who appear in a section entitled “Patient experiences”). These were described as “genuine”, “warm” and “interesting”. Given this, it is perhaps surprising that no one considered that there should have been greater participation by the patients. As with the case of Diabetes and Pain Management, however, there was a general feeling that instead of “lumping together” the patients in one section, they should feature at various points. For example, when the doctor describes the symptoms, it was considered better to have had a sufferer reflecting the professionals’ comments by talking of his/her own situation. In the video shown, the doctor is a voice‐over in a scene showing a man holding his chest and staggering. This was regarded as “distracting” at worse, and “unhelpful” and “unconvincing” at best.

Several viewers from the service users, the general interest and health information professionals groups thought that too much attention was made of the speakers and not enough on using the video medium to look at procedures, situations etc. As one of the nurses said, “the possibilities are there, but most of the programme was a guy talking”.

One point of interest regarding the production and presentation of the videos was opinions about the involvement of the general public/patients in the programmes.

The Pain Management, CHD and Diabetes videos all lead on health professionals, generally as “talking heads” explaining medication, processes etc. Public/patient participation is somewhat minimal, except in separate sections entitled “Patient experiences”, where patients also appear as “talking heads”, describing their own symptoms and treatment etc.

Evaluators considered that although there was sufficient involvement of the public in these videos the type of participation was not ideal. Nearly all evaluators, including the service users, felt that there should have been a far more dynamic involvement: “The patient is just there like a tailor’s dummy”. The service user who watched the programme on diabetes remarked, “that bit about diet – why didn’t they show a family shopping or planning a meal, with the heart‐attack guy saying something like “you’ve got to buy extra fruit – the doctor said it keeps the fat down”.

Illustrating the appeal of having the public more engaged in the programme content was the positive view of the programme on giving blood. This video followed a potential blood donor through the blood giving process. The focus was thus on the lay member of public, with the medical professionals reacting to him, rather than the “talking head” of the doctors, with the patient a secondary figure, as in the CHD and other medically oriented videos. The medical professionals appeared as they would in the real situation (i.e. to register him, take a sample etc.). Evaluators from all groups viewing this video felt that this was an excellent approach.

Interestingly, the diabetic group did not regard the degree or type of patient involvement as an issue. They felt that the split of medics in some sections and patients in others was “well done”, and were comfortable with the practice of having patients explain their condition in a separate section. One said that this did not interest her, as she knew her own experiences, and was more interested in latest developments in treatment, rather than the views of other sufferers. It might have been interesting to have shown the video to a group of newly diagnosed diabetics, who might have had a different outlook.

One other point was made with regard to the video participants. One of the health professionals described them as “all white and middle class”. Apart from this lack of inclusivity, she pointed out that much diabetes occurs in the Asian and Afro‐Caribbean communities. In addition to the lack of contributors from these groups, the information given in the programme did not allow for differences in diet or lifestyle, which affect the condition and its treatment. The diabetic evaluators did not mention this aspect. This is probably due to the fact that they themselves were all white. Also, it may be that the health professionals may know more about the condition than the sufferers, certainly in the field of epidemiology. The general interest viewers (one of whom was of African origin) similarly did not comment on this, possibly not knowing the ethnic bias the condition manifests.

Language and terminology

Evaluators’ views on the language used in the programme were mixed, although within‐group opinions tended to be similar. The health professionals, in general, considered the language to be too difficult and were somewhat critical of the language. The elderly group also felt there were many medical terms and expressions that they did not understand, but were quite accepting of this. Other viewers, including one of the health information professionals, thought the language was generally understandable, one going so far as to say that terms used were “well explained”. The elderly group had an interesting view of this aspect of the programme. They claimed that despite difficult terms, the language used was not a problem. One reasoned, to general agreement, that “there is so much around today that we don’t understand – we just let it go by!” Pressed on this point, others said that there were so many new developments they could not keep up with that they tried not to think about it. One said his video‐recorder was the oldest and simplest available, because the instruction book for new ones was “ridiculously complicated”. The video viewed (CHD) was beneficial, it was claimed, as it was not difficult to grasp the essential messages (in this case regarding diet, lifestyle and treatment etc.) even without a command of some of the vocabulary. Clearly, visual material is appropriate for those who have doubts about technical or medical vocabulary. Much material, however, consists of a “talking head”, where it could be argued that the medium is not being exploited to its full extent. Despite assurances that the video was “interesting” and “helpful”, it remains for further research to determine the amount and value of information that is actually imparted.

The diabetics felt the language was “pitched just right” with regard to the video on their condition. A nurse who had originally criticised the language later said she felt that long‐term diabetics would probably be familiar with the terminology, but newly diagnosed cases and carers would not, and opined that the video would be for these people too. The terminology used in the Giving Blood programme was considered very understandable by all groups. Unlike with others dealing with medical (as opposed to general health) issues, there were no complaints by the medical professional viewers or others about difficult technical or medical terms. This may have been because the information was presented within a specific and demonstrated (i.e. “acted‐out”) context, although evaluators did not consider this possibility. Any deeper analysis would have to work through the programme transcripts, run them through a reading level programme and then compare results with those found by interviewing viewers.

Format

There was much agreement amongst evaluators of all groups with regard to whether some (or all) of the information would be better presented in another form. Particular emphasis was placed on text, but viewers were also asked about diagrams or animation. People from every group felt that the sections on medication, in which a doctor talks about various medicines and their effects, could have been better presented as a table, listing medications, functions, effects etc. It was variously suggested that this could be on screen with a voice‐over giving additional details, accessible from a remote control button, whereby viewers could toggle between the video and the table, and, by contrast, as a replacement for the video section. Many people felt that, without such textual reinforcement, this section was ineffectual: “No‐one could remember all those details”, “You would have to be a quick writer to get all that down for when you needed it!” It was suggested by two of the health professionals that the classification of the medication (i.e. curatives, pain relievers etc.) was not explicit enough, and that a table would have been of much help here. Others (two health information professionals and a general interest viewer) felt that a written summary of every section would have been helpful. Sections on diet (in CHD, Diabetes, as well as in the Healthy Eating video), could, it was suggested, also benefit from recipes displayed on screen and, ideally, printable.

Evaluators’ preferences for material in written form is backed by research showing this to be beneficial. Studies have shown (e.g. Kitching, 1990) that generally patients forget half the oral information provided to them within five minutes of leaving a consultation. There is no reason to suppose their ability to retain such information from the television may be any better (although in this case there is, of course, the option to replay the tape). Studies in the USA (e.g. Wehby and Brenner, 1997) have shown that patients, in this case with chronic heart failure, find it difficult to retain information and that they may not appreciate the relevance of information provided by clinicians. Therefore, written advice, particularly if printable, would seem to be a good option.

A possible preference for text services is reflected in the logs. In total, 85 per cent of users did not view a topic video, though these users made use of accompanying textual information, and 89 per cent of user sessions did not include a view of a topic video. In the paper by Huntington et al. (2003) in this issue, striking differences are evidenced in the extent of take‐up of this service – which provides video information accompanied by text‐based materials, and that of a similar service (indeed, the videos are the same) in which only the videos are offered. The video‐only service had a greatly reduced reach figure (i.e. the number of people who used the service compared to the number to whom it was available).