Abstract
Provides a summary of a Department of Health funded research study investigating performance and impact of four pilot digital interactive television services in the consumer health field. These were launched in various locations in the UK in 2001. Text and video, interactive and transactional services were featured. Pilots were investigated using a combination of research methods. The four pilots’ performance varied, but overall there was sufficient evidence to suggest that consumer health digital interactive television has a healthy future.
Keywords
Citation
Nicholas, D., Huntington, P., Williams, P. and Gunter, B. (2003), "First steps towards providing the UK with health care information and advice via their television sets: an evaluation of four Department of Health sponsored pilot services", Aslib Proceedings, Vol. 55 No. 3, pp. 138-154. https://doi.org/10.1108/00012530310472633
Publisher
:MCB UP Ltd
Copyright © 2003, MCB UP Limited
Introduction
In 2001, the Department of Health (DoH) launched four pilot projects to explore the feasibility and effectiveness of health information and advice services supplied to the public via digital interactive television (DiTV). At the same time, the DoH commissioned a research team comprising academics from City University and the University of Sheffield to undertake a substantial evaluation of the performance of the pilot DiTV operators. This research incorporated a suite of methodologies through which a variety of types of data were collected.
The four DiTV pilots offered distinctive services. Although there were some overlapping features among these services, they had many more qualities that are distinctive. These qualities included the type of platform on which a service was transmitted, the amount and nature of content, the presentation formats used, and the degree of interactivity involved in each case. The four consortia were: Communicopia, Flextech Telewest – Living Health, Channel Health and DKTV (A Different Kind of Television).
The service developed by Communicopia aimed to extend the reach of the NHS Direct telephone and online service onto digital television. It was launched (in November 2001) to an audience in Kingston upon Hull and the East Riding of Yorkshire via Kingston Interactive’s (KIT) local ADSL telephone network, and later, on Video Network’s HomeChoice platform, to viewers in London. The service enabled TV viewers to access key sets of data from the NHS Direct Online Web site, including details of over 400 illnesses and medical conditions, support organisations, and advice about living a more healthy life. Utilising a mixture of existing text‐based material and specially produced video clips, the service provided users with over 20,000 pages of information, and video‐on‐demand offering both the perspective of medical professionals and that of patients. The service also provided users with interactive options such as health quizzes and an SMS text messaging reminder service for children’s vaccination dates. This service was promoted as NHS Direct Digital.
Flextech Living Health Ltd and partners launched a pilot service in June 2001 featuring a range of digital TV health applications to a potential average audience of 45,000[1] in the Birmingham area via the Telewest cable network. The applications included “NHS Direct in Vision” (talking to and actually seeing a nurse), a system for booking an appointment with a GP through the TV and a wide range (over 22,000 pages) of information pages covering local NHS services, healthy living, health conditions and treatments and a database of medicines. The pilot ran for six months and was then extended by Flextech Living Health for another six months.
Channel Health piloted a series of broadcast TV programmes (Bush Babies) to a national audience of over 5 million with linked interactive services via Sky. The programmes dealt with health (and financial) issues relevant to pregnant women and the interactivity enhanced and supplemented these by providing a range of text‐based pregnancy and maternity related information. Channel Health presented a text‐based information service linked to special broadcasts in its regular schedule on the Sky Digital platform. It experimented, on a local basis, with a package of other interactive services for pregnant women comprising mainly e‐mail support links between users and health professionals.
DKTV Ltd developed a service for the HomeChoice platform, launched initially to Newham residents. This service aimed to work together with public sector organisations to provide national and local interactive television content. The service relied on the HomeChoice video‐on‐demand, enabling users to select and find out more about the local services on offer. The video clips were presented by local service employees and provided users with the option to press a button to forward their details to local service providers. Originally intended to be rolled out on cable TV and broadband platforms in London, in the end only the broadband version of the service was launched and evaluated. This service was transmitted to approximately 500 potential users on Video Networks Limited’s HomeChoice service in Newham.
Aims
These were:
- •
To determine whether digital interactive television “worked” in a consumer health context. Did it, for instance, deliver the numbers, the particular audience profile, the ease of access and the hoped for health outcomes?
- •
To evaluate and compare the four DoH contracted consortia’s approaches to the delivery of health information on DiTV. To establish which approaches worked, with whom, and which of the approaches worked best.
The organisation of this paper follows the main performance indicators originally specified by the Department of Health in its briefing document. Most of these can be applied to each of the four consortia in turn. Indicators such as delivery of NHS services in a domestic setting by a commercial organisation, the costs of different DiTV services, and the effects of approaches used by broadcasters to combine NHS and non‐NHS health programmes and material, are examined in a separate section of this report, taking all four DiTV consortia together.
The key DoH evaluative criteria were:
- •
volume and pattern of use;
- •
users’ experience and perceptions of the service;
- •
the range, quality and appropriateness of the services offered;
- •
co‐ordination with other health services available;
- •
impact on NHS of supporting the service;
- •
impact on users’ use of health services;
- •
impact on users’ health status;
- •
impact on their perceptions of the NHS.
In addition, some comparisons were made between use of digital interactive television (DiTV), as a platform for health information, and other technologies such as touch‐screen kiosks and the Internet.
A number of factors made the meeting of the research objectives a difficult and challenging task because:
- •
The four consortia did not start at the same time, which made it difficult to compare the impact on each consortium of a health scare, seasonal illness or item of news.
- •
The periods over which the evaluations of the different pilots were conducted varied, giving some consortia longer exposure.
- •
Clearly, DiTV is a new platform that represents a development in the nature of television as a technology and as a viewing medium for the audience at home. The use of DiTV as a communications channel through which to transmit health information and advice – as evidenced in the current pilot exercise – is a pioneering innovation. Even on an established platform, it may take some time for a new service to become established in terms of its market share. In the case of the DiTV experiment, both the services and platforms were relatively new, not just in terms of content, but also in terms of the way users were invited to engage with the services being offered. While people may be accustomed to entering “interactive” mode when using the Internet, this is not true of television viewing. The long‐conditioned psychological orientation towards television is one of passive reception – with interactivity limited to channel surfing via the remote control, video recording of off‐air broadcasts for later viewing, and latterly, choice of camera angles at sports broadcasts. Full‐on interactivity with the television that involves content mining or two‐way communications, with the viewer being a sender as well as receiver of messages, is a new phenomenon for most viewers that will entail some re‐conditioning of their orientation towards the medium.
- •
The four DiTV consortia offered different products to different audiences in different places – Birmingham, Hull, London, or, in one case, the whole of the UK. Some were mostly text‐based, while others offered significant video material. Levels of interactivity varied from searching of text content – a kind of online encyclopaedia of health information, through video‐on‐demand – to personal medical records keeping, online appointments booking and two‐way, live and interactive, personal health condition diagnosis. All these distinctive features meant that we were not always comparing “like with like”.
- •
Data quantity and quality varied between consortia. While it was originally intended to centre much of this evaluation exercise on computer log data that gave measures of real‐time behavioural use of each service, this proved not always to be possible. No log data at all were produced by one consortium (Channel Health), while only summary data were made available for another consortium (DKTV). This meant that the comprehensive data comparisons we wished to make could not always be conducted.
- •
Access to users or potential users also varied between consortia. One significant factor in this context (and also related to log data availability) is that the DoH‐contracted consortia did not own or control subscriber databases for the various platforms on which their services were transmitted. The commercial organisations that owned the transmission platforms and controlled access to subscribers were not always willing to give access to commercially sensitive data from these potential digital TV health audiences. This meant that the data could not always be triangulated to best effect.
In view of the above, the research team had to modify the original research design as certain methodological options and envisaged data sources were closed down. Where real‐time log data were not available to measure user behaviour for these services, self‐report data were obtained instead via ad hoc surveys with users or potential users. Where direct telephone access to viewers and users was not possible for lack of subscriber telephone number information or permission to speak directly to subscribers from platform owners and operators, postal or online self‐completion questionnaires were used instead.
Methodology
A multi‐method approach was used by design and by necessity. In the advance planning stages, it was recognised that the research should attempt to study real‐time user behaviour in relation to these DiTV services, users reactions, and links between patterns of use and reactions and perceived impacts. Circumstances that arose during the course of the pilot exercise – and already outlined earlier – meant that standardised data could not be collected across all consortia. Nonetheless, the evaluative research maintained its commitment to assess use and user reactions as fully as possible throughout.
Log analysis
Analysis of digital logs was fundamental to this study. Logs were used to provide a bottom‐line assessment of how much these DiTV health information services were used at all by people who subscribed to the television distribution systems that carried them. The great advantages of the digital logs are not simply their size and reach, although the dividend here is indeed a rich and unparalleled one. Just as important is the fact that they are a direct and immediately available record of what people have done: not what they say they might, or would, do; not what they were prompted to say; not what they thought they did. The data are unfiltered and speak for themselves. Log data enabled the measurement of overall numbers of users for the interactive services, trends in usage over the pilot period, the amount of time users spent visiting sites, and the types of content they viewed. Detailed log data were obtained in electronic form for two consortia – Living Health and Communicopia. In the latter case, such data were available for the use of text‐based and video‐based content. Summary log data only were provided for DKTV and Channel Health generated no log data.
Industry audience data
Viewing statistics produced by BARB (Broadcasters’ Audience Research Board) were utilised in the case of one consortium – Channel Health. These data indicated how many viewers watched the consortium’s broadcast programme output linked to the project. A series of programmes targeted at women on maternity issues was broadcast as part of the regular Channel Health schedule on Sky Digital. Viewing of these programmes was measured by BARB’s audience measurement service. Viewing data are produced daily from a national panel of over 5,000 households. All TV sets and other related equipment, such as video‐recorders and set‐top box decoders, are linked to a data collection device installed in the home that automatically records which channel is tuned into when a TV set is switched on. Video recorded viewing is also registered. The presence of individual household members when a TV is switched on is recorded via a handset.
User reactions
Ad hoc user surveys (online, telephone and postal questionnaires) were carried out in respect of all consortia that were designed to supplement data obtained from real‐time measures of viewing. These surveys questioned identified users of each DiTV health service about their experiences, usage and opinions. Questions probed issues such as ease of use and navigation, usefulness of information, the way information was used, perceived impact on own health or dealings with health professionals, and trust and authority of the information.
With Living Health, data were obtained from two postal surveys conducted shortly after the service was launched and mid‐way through the pilot period. Questionnaires were distributed along with promotional literature to Telewest cable TV subscribers in Birmingham and yielded return samples of 450 and 723 respondents respectively.
With Channel Health, data were obtained from two user surveys conducted respectively during the early part and the later part of the pilot period. The first survey was conducted by Channel Health in part by post (n = 114 respondents) and in part via an online questionnaire (n = 322 respondents). The second survey was conducted in partnership with Channel Health by telephone interview (with fieldwork by Continental Research) and yielded 251 respondents.
With Communicopia, a postal questionnaire survey was sent to all potential subscribers to the Kingston Interactive Television platform in Hull, and 1,184 usable responses were returned.
With DKTV, a single telephone interview survey was carried out shortly after the launch of its Newham service carried on the HomeChoice broadband network in London. At the time of the research, there were 513 broadband subscribers in Newham and all were approached for interview. Interviews were successfully conducted with 281, of whom 35 had reportedly used or watched the DKTV service.
In‐depth interviews
The ad hoc surveys were supplemented in the case of most consortia with in‐depth qualitative interviews with either users of DiTV services or health professionals involved with these services.
With Living Health, depth interviews were conducted individually with six of the staff fronting the InVision service. In‐depth interview data were also obtained from health professionals concerning the GP appointments booking service. Twenty in‐depth interviews were conducted with Living Health users identified through one of the ad hoc surveys, four of whom had also used InVision. A further 31 InVision users were identified through caller logs, 27 of whom responded to telephone interviews and four participated in face‐to‐face interviews. Further qualitative data were obtained from 14 Birmingham City Council employees about the Web version of the service carried on their employer’s Intranet.
With Channel Health, qualitative data were obtained via the consortium among a small sample of pregnant women or young mothers and eight healthcare workers. These individuals were shown a demonstration of the Channel Health service and allowed to experiment with it themselves, before giving their impressions. Further in‐depth telephone interviews were carried out with two pregnant women about the interactive services supplied by Channel Health.
Communicopia held consumer panels once a month from June 2001 to explore issues of usability, information content and presentation. Each panel meeting involved between 10 and 15 participants, some of whom repeat attended. The sessions lasted between 90 minutes and 2 hours. Participants were presented with demonstrations of the service and opportunities to use it themselves.
Usability research
Serco Usability Services Ltd was commissioned directly by the Department of Health to conduct qualitative research with small, convenience samples who were given the opportunity to use service demonstrators in Serco’s observation laboratory. In all, 62 respondents were run across the four consortia. These data were examined alongside other data collected independently by the authors or in partnership with the pilot consortia.
Results
Use
Over the 12‐month pilot, the four DiTV pilots were made available to a potential combined audience of 5 to 6 million households, equating to 11.5 to 12 million individual viewers. The only nationwide service, Channel Health, attracted aggregated audiences of around 2.8 million to episodes of its Bush Babies series over the duration of its pilot. Based on self‐report data, it is estimated that 2‐300,000 people availed themselves of one or other of the digital health services, a reach figure of 2 per cent. At first glance, this would appear to be a disappointingly low figure. However, why the reach figure is so small overall is because most of the base of approaching 6m was contributed by Sky, and the reach of Channel Health in percentage terms was small (though it reached the largest potential audience overall). On Channel Health, 9 per cent of respondents who had identified themselves as viewers of Bush Babies reported using the maternity guide – the enhanced text service that supported the Bush Babies programmes.
Hence the much healthier reach figures for Living Health and Communicopia of 30 per cent and 20 per cent respectively (see below) are significantly diluted by the performance of the service on Sky. Even DKTV, which operated on a smaller scale than any of the other consortia, achieved a reach figure of 35 per cent. The specialised nature of some of the services (i.e. pregnancy information) and the short time‐span of their availability also needs to be taken into account in any calculation.
Individual service figures
Living Health was made available to an average of 45,000 subscribers. Over the six‐month period monitored, the logs tell us that 13,718 different visitors (households) used the system. Based upon this figure it is estimated that around 30 per cent of potential users (i.e. “households”, as wherever mentioned in this report) accessed the service during the period – a healthy figure by any standard, which shows the level of penetration these innovative health services are capable of achieving. This is especially true when one considers that, unlike daily news, for example, the kind of information/transaction offered was for ad hoc information needs. In consequence, subscribers may not have required any health‐related services in the relatively short time of the pilot. The potential for growth appears there too, with well over eight in ten (84 per cent) of questionnaire respondents saying that they would be either likely to or very likely to access Living Health if the service continued to be broadcast. Interviewee respondents were equally enthusiastic: “It will definitely become a prime source of information for me if it carries on – and if it doesn’t I will be disappointed”. Although this finding did not quite square with the falling trend of use, the initial surge, and then decline, in usage may well have been due to its novelty value and early marketing efforts, and is probably unlikely to signify long‐term stagnation.
Communicopia (NHS Direct Digital) was available to approximately 10,000 potential homes. Over the five‐month period monitored 1,965 different households used the system. Based upon this figure, it is estimated that about 20 per cent of potential users accessed the service during the pilot.
In regard to Channel Health, BARB data indicated that Bush Babies episodes aggregated (across their repeat showings) audiences of over 300,000 in the first (six‐week) phase, over 200,000 in the second (five‐week) phase, and 2.3 million in the final (three‐month) phase. According to self‐report questionnaire data, 27 per cent of Channel Health viewers watched Bush Babies. This is quite impressive, given that the target audience was pregnant women, not only a low percentage of the population as a whole, but, presumably, a low number even of Channel Health viewers. Channel Health calculated that 160,000 out of 700,000 annual births in the UK could occur in Channel Health receiving homes.
Channel Health is estimated to have a monthly reach figure among Sky viewers of approximately 15 per cent. Given a Sky Digital audience base of around 5.7 million households then the audience of Bush Babies was estimated to be about 200,000 homes, or about 3.5‐4 per cent of all Sky subscribers (assuming all Sky users receive Channel Health).
Finally, DKTV: during the two‐month survey period 142 users availed themselves of the service, out of the 403 to 513 households receiving it – a reach figure of up to 35 per cent.
In comparing the performance of the pilots, Living Health performed particularly well. In terms of audience reach, Living Health appeared to perform much better than the Communicopia/NHS Direct Digital, the service with which it had most in common – reaching an average of 30 per cent of the potential audience as compared to its rival’s 20 per cent. Adjusting the reach figure for the different operational period lengths, Living Health is estimated to have a higher figure by about one‐and‐a‐half times compared to the NHS Direct service on Communicopia. The reach figure for both services declined after about the 35th day of operation.
Return visits
The data show that once someone has used a DiTV health service there is a good chance they will use it again. About 60 per cent of Living Health users said that they would use the service as needed. In fact, the logs showed that 41 per cent of Living Health users visited the service again during the pilot. Living Health users who first visited the channel in July/August, shortly after launch, on average visited the channel three times or about once a month. Considering, again, the ad hoc nature of the need and the fact that everyone knew this was a pilot, these are quite impressive statistics.
A slightly smaller proportion (37 per cent) revisited NHS Direct Digital over the period January to May 2002. It is also worth noting here that rate of return visiting may have been partly affected by (negative) changes to the positioning of Communicopia/NHS Direct Digital in the broadband environment in which it was transmitted. No logs were available for Channel Health, so the data are probably less reliable. However, subscribers were asked how often they watched the service and 56 per cent said they watched at least once a week, 20 per cent watched at least monthly and 24 per cent watched it infrequently. Return visits could not be calculated for DKTV.
Time online
Short online consultations proved to be the norm. Living Health viewers spent on average six minutes on a visit (session), whereas the figure for Communicopia/NHS Direct Digital was over seven minutes. Users viewing more pages in a session recorded longer session times. NHS Direct Digital users spent approximately 50‐100 per cent more time on a visit as compared to Living Health users. This difference is thought to be due to the viewing of videos by NHS Direct Digital/Communicopia users – video viewing is plainly more time‐intensive. No data were available here for either Channel Health or DKTV.
Screens viewed
For the services for which logs were available, there was evidence that once people accessed a service they showed a significant degree of interest in it. In the case of Living Health, 39 per cent of users viewed more than 20 pages during a visit and these figures proved to be stable over time. In the case of NHS Direct Digital, 44 per cent of users viewed more than 20 pages, demonstrating, perhaps, a significant interest in what they saw. However, this service is a menu‐heavy service and has a number of menu screens associated with both the video and text service.
There were high “failure” rates too. In the case of Living Health, 19 per cent of users viewed one to three pages and these users were unlikely to have penetrated past the menu screens and would not have viewed an information page, except for the daily news headlines. For Communicopia/NHS Direct Digital the equivalent figure was also 19 per cent. Note that this service was very menu rich and to reach a number of information pages users had to go beyond three menu pages.
Finally, logs also revealed that only 12 per cent of those accessing the Living Health InVision (Broadband nurse) service page actually completed a session (i.e. engaged in an online consultation with a nurse). These figures may provide evidence that early users were “checking out” the system for future reference, not having an immediate need.
Personal characteristics of the DiTV health consumer
Age/Gender
In the case of Communicopia/NHS Direct Digital, younger respondents and, interestingly, especially male respondents were more likely to use the text service as opposed to the video service compared to older and female respondents. Men aged 55 and under were just over one‐and‐a‐half times more likely to use the text service than women 55 years and under. The relationship was also true of the video service. Furthermore, men over the age of 55 were about twice as likely to use the service compared to women under the age of 55. As for service preference, men reported a preference for videos over text.
For Living Health, older users tended to be men. A higher proportion of younger Living Health users (compared to NHS Direct Digital) were women. In the case of the InVision home nursing service men and women used the service equally, although for different reasons. Women tended to do so on behalf of others (particularly children) whilst men wanted answers to their own health problems (maybe the privacy of the home was an incentive).
Not surprisingly, given the nature of the Channel Health service (i.e. pregnancy information) those aged over 45 were four times less likely to use the service than younger viewers were. Women were twice as likely to use the service as were men. This still represents an encouraging interest in the service on the part of males. Interviewees all confirmed the interest of their partners in the programme and, indeed, how the content and information helped them understand and manage their situation: “we watch the programme together whenever we can – where (partner’s name) is at work, he watches on his own later – or I watch it a second time, with him”.
Social class
It would appear that DiTV attracts low‐income users. This is encouraging in that it supports the argument that DiTV throws an ICT health lifeline to those who have been excluded from the digital revolution – the less well off and potentially socially excluded.
People living in an area with a low incidence of £20,000+ income earners were more likely to use the Living Health service. Respondents from wealthier areas were half as likely to use a DiTV text information service, as were those people who came from less well off areas. In addition, users from lower income areas were more likely to say Living Health was useful compared to users from higher income areas. It was the same for the other services too.
Communicopia/NHS Direct Digital users from postcode areas with a low incidence of £20,000+ earners were about twice as likely to use the service. Channel Health users from middle and lower social classes were two to three times more likely to have viewed the service as were respondents identified as being from a higher social class.
Children
There was a high level of use of Living Health in areas where there was a higher incidence of 0‐14 year olds. This finding could not be confirmed among users of NHS Direct Digital on KIT or for those using Channel Health’s Bush Babies service. However, the researchers have found elsewhere (Nicholas et al., 2001), high levels of usage of other health information services (especially touch‐screen kiosks) among the young (under 15s).
Health concerns
What health topics consumers decided to view or what medical conditions they wanted to look up is plainly of major interest to health information providers everywhere. However, we need to be careful how we interpret these data as choice is determined by a number of factors – what content is provided, how visible or accessible that content is, what health topics interest users and what medical conditions they – or their family – may suffer from. Comparisons between channels are difficult because of differences in the content, media, audiences and interactive services provided. The general indications were that users turned to DiTV health services to address real information/health needs rather than to browse for recreation.
In the case of Living Health the sections that users appeared most interested in were the “Illness and treatment” section followed by “Women’s health” and “Men’s health”. The illness and treatment section accounted for 36 per cent of all pages viewed. The most popular topics in the illness and treatment section were back pain, depression, impotence, Aids, and irritable bowel syndrome. Popular topics under Women’s health were orgasm problems, dyspareunia, thrush and cystitis. For Men’s health interesting topics proved to be impotence, premature ejaculation, sexual infections, gay sex and sexual health help. Clearly, the privacy of one’s home appears to be the ideal environment in which to explore issues that may be too intimate or embarrassing to research via a publicly placed information terminal.
For Communicopia/NHS Direct Digital the “A‐Z of conditions” was the most popular section by some margin and accounted for 57 per cent of text pages viewed. The second most popular section was “Not feeling well”, and accounted for 13 per cent of pages viewed. Interviewees and questionnaire respondents for this project tended to stress their desire to avoid visiting a doctor: “If I find out that the problem is not serious I can avoid going to my GP. You have to make an appointment, it’s a long way, and the doctors are over‐stretched anyway.” An established pattern of popular topics viewed emerged in the “A‐Z of conditions” section from about February with diabetes, lower back pain, asthma, mellitus appearing in the top ten of subjects viewed in each of these months.
For Communicopia, the most popular on‐demand videos were the “Foray for health”, “Diabetes” and “Coronary heart disease”. These three videos accounted for 42 per cent of topic videos viewed. Each topic was represented by a series of videos that the user could view independently. For topics where only one video was available, the hypertension (downloaded 54 times) and MMR videos (downloaded 44 times) were the most watched. There were no views of videos on testicular cancer or ulcerative colitis. The latter might be considered disappointing to the DoH, as there is some concern that men are not undertaking regular self checks to spot this condition early. This raises the issue of whether topic positioning in the digital service should be used strategically to help ensure important topics are viewed.
Role and impact of digital health information
Well informed consumers
It would seem that DiTV creates well‐informed health consumers. Two‐thirds of Living Health users (67 per cent) said that the information they obtained had either helped or helped them a lot in becoming better informed about their condition. Data from in‐depth interviews confirmed this, with respondents enthusing about having information so easily to hand: “It has given me a whole new source of information, and right there at home.” Similar findings came from Communicopia/NHS Direct Digital users; 90 per cent said they felt better informed about a condition after having used the service. Pre‐launch focus group interviewees felt that people would be, as one put it: “much better informed about health matters”.
Helps with the doctor
DiTV helps in dealing with the doctor. Well over half (55 per cent) of Living Health users queried the service for information about their consultation with the doctor either before, after or before and after their consultation; 40 per cent of Living Health users felt that the information they found had helped or helped a lot in their dealings with the doctor. This was also a theme among depth interviewees: “I now go to the doctors with a much better idea of what’s wrong with me, and what I can ask about.”
These are views echoed by NHS Direct Digital/Communicopia subscribers; 73 per cent of respondents felt that the information they found had helped a little or helped a lot in their dealings with the doctor. Just over one in four NHS Direct Digital users, 27 per cent, said that they would use the service to look for information that they would not want to discuss with their doctor. Plainly, this is an important role for DiTV health services.
And the same was true about Channel Health users. Interviewees felt that they were far better armed when they met their GPs, midwives and obstetricians, and praised the Bush Babies series for giving them an excellent knowledge base regarding their condition: “It gave me so much information that I know I wouldn’t have got elsewhere – I don’t go fishing for information in books – and it’s easy to forget what the doctor tells you.”
Used to put off a visit to the doctor
Over half (53 per cent) of Living Health users confirmed that they had used information they found to replace a visit to the doctor. Those reportedly using the Living Health service were just under twice as likely to use information found as an alternative to seeing the doctor. Older users and men, however, were less likely to use information in this way. NHS direct online, health books and magazines and the Internet were also significant as sources used as an alternative to seeing the doctor. Importantly, the younger the respondent the more likely they were to substitute information for a visit to the doctor. Older users, those aged over 55, were half as likely to use information found on Living Health as an alternative to a doctor’s visit, as compared to younger people.
Where a person had an interest in general health, prescription drugs, healthy living and an interest in alternative health, then these people were more likely to use an information source as an alternative to seeing the doctor. This was strongly apparent in interview data as well as questionnaire returns (“I maybe know more about my health than the average person, and have a good idea about when I need to go to the doctors. I always do some research first now I have an extra resource.”). Indeed, Living Health was considered by subscribers as an important, if not a more important, source of information compared to the NHS Phone line. This has to be regarded as significant in the decision to roll‐out digital health to the nation.
Information on NHS Direct Digital also impacted on whether viewers used information found as a substitute for a visit to the doctor. Those using the service were just under one‐and‐a‐half‐times as likely to use information found in this way. This result was only significant at the 10 per cent level and further research is required to confirm the result. It was found, however, to be the least important information source compared to medical magazines and books, NHS Direct Telephone line and the Web. This suggested that the impact of this service was not as important as these other sources, probably because of its relatively limited content base.
Leads to an improvement in health
One‐third of Living Health users said that the information found either helped or helped a lot in improving their condition. Nearly 62 per cent of Communicopia/NHS Direct Digital said that the information found either helped a little or helped a lot in improving their condition and 15 per cent said that it had helped a lot. In the case of Channel Health, three out of four women who used the enhanced text service, supporting the broadcast programmes, said that they found its information reassuring. Interviewees tended to say that the way they approached their pregnancy was healthier because of the programme contents: “My diet was better, I exercised, I felt better about what was happening. The programme helped me in all of these things.”
Whether the service improved the user’s condition or helped in understanding the condition appeared to depend on the extent to which people understood the nature of the Living Health service. This points to the importance of design, navigation and documentation; possibly, also, digital literacy training.
There was one surprise result. Whether the user found the system easy to view impacted on whether the person expressed a positive health outcome as a result of receiving digital health information, but the relationship was the inverse of what we might have expected. Those finding the system easy to view were not those who felt that there were significant or positive outcomes resulting from use. There is a sense here that the harder you have to work at it, the more appreciative you are of the benefits.
Use of the different digital products on offer
The relative popularity of each type of service (text and video based services and interactive and transactional) was assessed. Health commentators have championed the video for consumer health information (especially for those with literacy and language difficulties), and their take‐up was of particular interest. Two pilots in particular presented a significant amount of content in video format. Channel Health produced a broadcast series of TV programmes on maternity issues and Communicopia provided a video‐on‐demand service comprising material on a variety of health topics.
Video Vs text
Of Channel Health’s Bush Babies respondents, 70 per cent just viewed the video programme, 23 per cent reportedly viewed text and a video, while 7 per cent reportedly just viewed the text.
Both forms proved popular to Communicopia/NHS Direct Digital users. A total of 41 per cent of those who had used the NHS Direct Digital service claimed that they only used the text service and that they had not requested a video; 6 per cent said that they had only used the video service and 54 per cent said that they had used both text and video.
However, text proved to be the most popular; 60 per cent of NHS Direct Digital users said that they preferred the text service while 38 per cent said that they preferred the video service. This was reversed for men aged under 36. Focus group participants felt that some information presented in video form would have been better as text: it was suggested that information on medication, for instance, could have been better presented in tabular form.
Young men preferred videos; 63 per cent of men aged under 36 said that they preferred to watch videos rather than read text compared to 26 per cent of females in this age group who said so.
Users of NHS Direct were also asked to rate the importance of each service, the health video and text service. The average scores were similar, though there is suggestive/weak evidence that videos are easier to understand (3.4) compared to text (3.3) and that videos are more interesting (3.3) than the text service (3.1). Focus group interviewees tended to be of this opinion: “Of course, it’s not just the information – if it is boring or dull, people won’t take it in. Watching a video is at least a bit more interesting.” Only 34 per cent of NHS Direct digital users, however, agreed with the statement that watching a health video was a big improvement on reading the text, one third disagreed and one third of users had no opinion. Men aged between 36 to 55, and men and women aged over 55 were more likely to say that watching a health video was an improvement. Of this group, 42 per cent agreed compared to 31 per cent of females in the 36 to 55 age group, only 20 per cent of those aged under 36 (male or female) agreed with this statement.
Transactional services
These services explored the potential of DiTV as a two‐way medium where the user becomes an information sender as well as receiver. Such applications represent more advanced forms of interactivity and require a different mindset on the part of users who engage in a customised activity geared to addressing their specific problems rather than ones of a more general nature. These are genuinely new applications of TV. The applications tested in the pilots included visual interpersonal communication with an NHS nurse (InVision), online appointments booking with one’s GP, and the maintenance of personal medical details online, in this instance personal immunisation records. In addition, one consortium (Channel Health) tested a small‐scale e‐mail support service for a specific group – pregnant women.
- 1.
(1) Living Health – InVision. Despite the obvious warmth with which both consumers and nurses received the service, relatively few people chose to use it. A total of 163 users from a potential audience of around 38,000 subscribing households in four months appears low. Four possible explanations suggest themselves: a small potential user population; the discouragement of casual users by the channel itself; the lack of publicity for these services; and, possibly, most importantly, the novelty and unfamiliarity of the services. Nonetheless, the number of people (1,380) who activated pages leading to the “point of no return” connection button indicates much potential interest. Clearly, the issue is how you convert these “lookers” into users.
- 2.
(2) Living Health – GP surgery bookings service. Use of this facility was plainly very low with just 30 people making an online appointment with their doctor over a period of six months. This was partly to do with the fact that there were only three surgeries in the pilot, and that the surgeries did not “sell” the service sufficiently to their patients. Given the amount of work that would have been involved to fully sign‐up to the project and that it would only be available for a period of six months, this was probably unrealistic. An open‐ended (time wise) roll‐out might have produced different results.
- 3.
(3) Communicopia/NHS Direct Digital. Views to the vaccination service accounted for 0.14 per cent of use; approximately 28 people used it. The service consisted of a reminder of when a jab was needed and users had to enter all relevant personal details. The take‐up can be regarded as disappointing, but one has to question the value of this to a consumer when they know that it will only be in place for a short period.
Usability
Data were obtained from user reaction surveys, follow‐up qualitative interviews and Serco usability lab research. DiTV is thought to be an easy‐to‐use health information medium. In practice, however, usability of the pilot services was found to vary with the individual user and the nature of the service being used.
Of Living Health respondents, 28 per cent said that it was “very easy” to find the information. Just under half found it either OK, hard or very hard to find information on the channel. Interviewees complained of the number of screens required to navigate (“there is hardly any information on a page – even the shortest of topics has six or seven pages to it”).
Most participants in Serco’s DKTV usability sessions were unclear about where to find the DKTV service within the HomeChoice menu on a broadband platform.
Some services proved almost trouble‐free. The interactive service linked to the Bush Babies television series on Channel Health was universally described as being easy to use (although even here there was some concern about using arrow keys and sub‐menus). Interviews with users of the e‐mail service indicated that they experienced difficulties: “It wasn’t Channel Health – it was the e‐mail service that didn’t work very well.”
Digital visibility the key
In an analysis reviewing the menu position of the NHS Direct Digital on KIT (Nicholas et al., 2002), it was found that the service became more difficult to access as its sign posting became ever more removed from the television service’s opening menu. This led to the proportion of new visitors (as a percentage) of all users declining alarmingly. New users did not come through because of the increasing difficulty of finding the service. Those people who battled through to find the service, however, showed their tenacity by making more extensive use of the channel when they arrived.
In laboratory‐based usability tests, Serco found instances of poor positioning leading to a lack of use – DKTV’s “further details” button, for example, was missed by several viewers engrossed in the video content of the channel.
In sum, the number of clicks to obtain content is a critical feature with DiTV interactive services, and determines whether people find what they need.
Gender and employment
Women were less likely to find the Living Health system useful “all of the time” compared to men. Women were also more likely to say that the information offered was full of medical jargon.
Users living in higher unemployment areas were more likely to say the Living Health system was easy to navigate. Clearly, this finding requires further investigation before a full explanation can be offered.
Age differences
The elderly are clearly a source of concern when it comes to DiTV health services. On the one hand they offer the prospect of convenience and doing everything in the comfort of one’s own home, but on the other they require a certain amount of dexterity and the elderly tend not to bring with them digital familiarity.
Not surprisingly then, the age of the respondent was found to impact on how easy the user found it to understand the Living Health service. Users over 66 were more likely to report that they ran into difficulties. Furthermore, older NHS Direct Digital users, particularly women aged over 55, found the service difficult to use.
Focus group sessions in which Communicopia/NHS Direct Digital videos were shown to elderly people revealed some anxiety about the march of information technology. Older participants felt that they would not have access to digital television in their lifetime and would be unsure of how to use it even if they did: “I can’t even use a video (recorder) these days – it’s got so complicated”.
It helps if you are technologically literate or if you use a variety of health information sources
Unsurprisingly, users with a greater experience of technology were more likely to find the navigation and menu structure of Living Health easier. Users reporting that the general KIT DiTV service was easy to use were more likely to have used the NHS Direct Digital service provided by Communicopia.
Many interviewees who had used Living Health, or used the Bush Babies interactive service, or viewed the videos on NHS Direct Digital tended to use these in concert with a variety of other sources – Internet, books, etc. These information hungry respondents were used to various menu systems, indexes and different information configurations, and found it easy to adapt their skills to the medium of DiTV. (“I have always had an interest in health – now I can use the Internet and my television to supplement the books I have always used.”)
Contrasts between views of medical professionals and lay consumers
The DiTV services were meant to be consumer friendly, obviously. Nevertheless, there is probably a tendency amongst health professionals to underestimate the consumer. Thus nurses who watched the NHS Direct Digital videos felt that the language and terminology used was often too difficult for the lay viewer. However, information professionals, general consumer interest groups and others all felt there was no problem in this area.
Authority, trust and branding
DiTV is a very new platform for health information and issues over whether the information could be trusted, and whether the NHS brand was visible and what it meant were of concern to the DoH. Clearly, matters are complicated by the fact that, with so many parties involved with the content, production and distribution of the digital services, ownership and responsibility are far from clear.
Of the users on Telewest’s Birmingham cable platform, 60 said that they would trust the health information found on DiTV for most things; 38 per cent said they trusted the information and only 3 per cent of users said that they did not trust the information found. A substantial proportion (43 per cent) said they would not use the service should the NHS not be involved, and a large majority (81 per cent) thought the NHS should be involved with digital television, a finding echoed in interviews: “I just assumed the NHS was involved – it’s a public service isn’t it?”
For certain types of people the NHS brand really meant something. DiTV subscribers who had either used the Living Health service, which carried NHS branded health information, or had heard of the service were more likely to say that the NHS was a symbol of trust than DiTV subscribers who had not used the service. There were, however, digital users who did not buy into the NHS brand. DiTV users visiting the doctor less frequently and those less interested in health information were less likely to accept the NHS as a symbol of trust, were less likely to recognise the NHS symbol, and were less likely to say that the NHS branded information could be trusted. Younger respondents were also less likely to recognise the NHS as a symbol of trust compared to older respondents.
Communicopia/NHS Direct Digital users were asked to rate the trustworthiness of health information sources out of 5. Sources where the NHS appeared in the name performed well – NHS Direct phone and the NHS Direct service available on KIT scored respectively 3.9 and 3.8. This score placed these services below doctors and nurses (4.5 and 4.2) but above medical magazines and books (3.0).
Around half of Channel Health viewers who were surveyed for their reactions to its service were aware of NHS involvement and their reactions to this were largely positive. NHS branding was seen as offering more authority and credibility to the content (“It was nice to know that this was real information from a proper source. It was obvious the programme was helped by the NHS”.) The more respondents had viewed the Bush Babies TV series, the greater their awareness of NHS involvement became.
Satisfaction
Satisfaction was a key DoH performance measure. They were not to be disappointed. A total of 45 per cent of Living Health viewers said that the service had been very useful, 48 per cent said that the service had been quite useful, while 7 per cent said that the service had not been useful. The main reason people gave for finding the service helpful was that they felt reassured by the information given – 50 per cent said so; 15 per cent of users said that the advice helped them deal with the condition themselves while 20 per cent said that the advice helped them to contact the right service. Interviewee respondents also indicated that the service was very useful, enabling them to, for example, avoid visiting the doctor, research a condition for a friend, and check information about medication. If DiTV was rolled out nationally there would appear to be significant gains.
The InVision service went down extremely well and comparisons between the telephone equivalent are instructive. Respondents rated the service either very satisfactory (76 per cent) or satisfactory (24 per cent) – 100 per cent satisfied in other words, versus 97.8 per cent (combined totals) for the NHS Direct telephone‐only service. Additionally, 100 per cent of InVision customers polled said they would use the service again and/or recommend it to friends/family – versus 97.8 per cent for telephony.
Finally, 88 per cent of those who had used both InVision and the telephony service preferred InVision (the rest had no preference). These results suggest that the nurses were able to offer an equivalent service while having to engage the camera and operate the InVision system in addition to the normal CAS (clinical assessment system) software. Interviewee respondents all rated InVision above the telephone only NHS Direct – enthusing about the facility to see the nurse (rather than about how images could be sent to them): “It was so good to see the nurse – it was obvious she was interested. With the telephone service you cannot tell how much she is paying attention.”
The 35 respondents who claimed to have viewed or used DKTV were asked to rate their overall satisfaction with DKTV along a four‐point scale from very satisfied to not satisfied at all. Two respondents were very satisfied, 21 were quite satisfied, seven were not very satisfied and one was not at all satisfied. The remainder did not know (or care). The average satisfaction score indicated that these respondents were only “quite satisfied” with the service. When asked how often they thought that they or their family would use DKTV in the future, most of these respondents (20) thought they would use it about the same as now, with nine saying they would use DKTV more often and six saying less often.
Most people who used the Channel Health/Bush Babies service thought the service quite useful – 60 per cent said so. Those in social class D/E were less likely to say that the information would be useful for them: 17 per cent of this group said the information would be useful for them, compared to 40 per cent from other groups. Furthermore, those living with a partner said that the videos were useful to them: 56 per cent said this compared to 37 per cent of married and 8 per cent of single users. For those users who were pregnant, 83 per cent said the information was useful to them compared to 29 per cent who were not pregnant.
People who used Communicopia/NHS Direct Digital generally found what they were looking for. A total of 15 per cent of NHS Direct viewers agreed and 69 per cent disagreed with the statement that in general users could not find what they are looking for on the service. Of the users, 81 per cent said that if they needed health advice they would consult health information on KIT. There is suggestive evidence that carers (informal and formal) were less likely to say that they would consult the service compared to non‐carers. This needs further research, particularly as it is contrary to what one would expect.
Comparisons with other digital platforms
In parallel with the DiTV study, City researchers were also investigating other digital platforms providing heath information and advice[2]. On the basis of this work (Nicholas et al., 2001; Williams et al., 2001, Huntington et al., 2002), the following comparisons have been made:
- •
Compared to the Web and kiosks, DiTV attracted a greater percentage of its users at weekends. This is no surprise, as the DiTV service is the only “home only” service. Many people may only have access to the Web at work or in a public place, and the kiosks are, of course, sited away from the home.
- •
Compared to the Web, DiTV attracted more repeat visits. This could be ascribed to easier access, but it could also be a function of more limited choice – it is possible that Web users revisited for health purposes, but to another competing health site. During the pilot period, there was no competing health information service on any of the specific DiTV platforms being investigated. Hence, Telewest subscribers, for example, had no option but to return to the Living Health channel.
- •
DiTV users took a third longer to view a screen than kiosk users, possibly reflecting the fact that they were using the service in the comfort of their own homes.
- •
DiTV users viewed three times the number of pages that kiosk users did. This is because DiTV users have the time to penetrate the system. Furthermore, users cannot scroll on DiTV and will inevitably view more pages. It is not uncommon to find health information, but not menus, spread over up to seven to eight pages.
- •
DiTV users spent the most time viewing – approximately five to six minutes during a session compared to three to four minutes for Web users and about one minute for kiosk users. This is thought to be because DiTV users had more time to view pages and, in the main, were forced to read the content, as they did not have a printer (some KIT users had printers attached to their TV sets).
- •
Compared to kiosk users, DiTV users were more willing to interrogate the system beyond the menu screens. Kiosk users were thought to be rushed for time and may have been unwilling to invest the time necessary to interrogate what may be a new system for them. As previously mentioned, DiTV users view the service in their home, at a time convenient to them, and on a system that they have used before and hence may be far more willing to interrogate the system.
This all suggests that, because users are viewing a system in their own home and one with which they are familiar, they will interrogate more topics, view more pages, will have longer search sessions etc. The log metrics point to a greater use of DiTV. To date, the data are lacking to give a comprehensive analysis of the reasons why such differences in use of online health on different platforms occur. However, it seems highly likely that differential ease of use of the technologies and information systems is one factor. Clearly, with kiosks, users may not have seen the system before and they might be at a loss to use the service effectively. This is not so true of the DiTV service, where users have some previous relevant experience in the use of the medium. In fact, our research generally points to greater use in households where the TV plays a central role – i.e. where households have a previous experience of TV viewing. If you like, the technology learning curve is lower. In relation to the use of television interactively, however, industry research has indicated that significant numbers of subscribers to mainstream television services offering interactive capabilities do not use the interactive elements of these services. In any case, interactivity on mainstream broadcast television tends to occur at an elementary level (e.g. choosing between different camera angles at a sports contest or selecting enhanced news footage behind a regular news broadcast). The more complex level of interactivity represented by the digital health pilots requires a different degree of involvement on the part of viewers.
Overall conclusions
DiTV appears to have the potential to attract a large audience. This audience comprises an extensive number of occasional users rather than a large one‐off audience, but that probably reflects the nature of the health information need. Reach figures indicated that significant numbers of people who have access to these online services use them from time to time. Encouraging take‐up figures suggest a viable future, and evidence shows that when people use the services once, they are likely to do so again.
Health is a popular topic likely to attract viewers to watch regular television programmes, but it can also encourage viewers to use television more interactively. They are prepared to search for information that is of use to them or their families. It should be noted, however, that they might not be prepared, as yet, to search as much for health information as Internet users. Hence, interactive TV sites must be easy to use and enable users quickly and efficiently to reach the information they seek, and the content has to be worth seeking.
What is also encouraging is that the digital health services appear to reach groups that the DoH have targeted because they have proved that can be difficult to access via other methods. Furthermore, DiTV has achieved a different demographic profile of users from that of other platforms, which shows that a suite of online technologies is needed rather than all the investment going to a single, supposedly convergent technology.
As well as low‐income groups, DiTV reaches older users, especially older male users. There is also evidence that it can encourage younger male users who are noted for not checking up on their health. It can also be effective at reaching specific groups with specific conditions, e.g. pregnant women.
Low take‐up of Living Health’s InVision and online GP bookings service suggests unfamiliarity and an uncertainty with transactional services. The level of use recorded for Communicopia/NHS Digital Direct’s immunisation records service reinforces this point. This may indicate a need both for user friendly interface designs and possibly also training designed to promote interactive TV literacy. Alternatively, it may also indicate that some applications, such as an appointments booking service, where a degree of negotiation and flexibility are required among patients and health service providers, are more difficult to transfer into the online environment.
People appear to be selective in their use of different media. Pages and topics accessed via DiTV are different from those accessed on the Internet or a publicly located touch‐screen kiosk, due to considerations of privacy, time availability and the amount of choice offered. With regard to format, video appears to be less consulted than text services. However, it might be the case that, while the absolute number of videos consulted is less than the absolute number of text pages examined, users spend more time with videos than text. So overall, distribution of time to these formats might work out about the same in the end.
Again, there is the issue of “application effectiveness”. This means finding the most appropriate, effective and acceptable format for a particular type of application. Focus group interviewees suggested that text was more appropriate for certain kinds of information (e.g. basic facts), and more easily consulted than having to run through a video to find a particular item. Video has a different psychological functionality for users. Text is factual, while videos can succour emotional needs too. This point is important both in respect of the construction of online libraries of health information, and also in relation to “live” transactional links. There is mounting research on the use of computer‐mediated communications – outside the health sphere – that has shown that the rules of interpersonal communication online may vary from those that prevail in face to face communication. This research may have important implications for the design of user interfaces and decisions about the types of formats that could most effectively be applied to facilitate online health‐related communications.
Even where there was no facility (i.e. printer) to copy information, time spent perusing pages has to be regarded as minimal. Further work is needed to examine how information is assimilated and used by DiTV subscribers. For instance, users may feel better informed by video, but absorb less factual information than they would from text.
The health topics viewed, whilst differing across platforms and formats, nevertheless appear to suggest the services were used very much for consultation/reference with regard to specific conditions, rather than for general browsing or recreational use, which one might have expected. Users of Living Health acknowledged that this was a primary reason for using the service. Digital health services provide support for a number of specific functional needs on the part of NHS patients and members of the general public. Future research should attempt systematically to map out these needs and determine the delivery platforms, formats and content that will effectively satisfy them.
The information retrieved appears, for all services, to have made some impact on dealings with doctors, and there is some evidence to suggest that people are using it as an alternative to making GP appointments. Our data provide support for the belief that digital health services could help the NHS in terms of reduced demand or economies in relation to certain offline services. Furthermore, and in a broader sense, if online health information can cultivate greater self‐care and adoption of preventative “medicine”, further economies may be felt in an over‐stretched NHS.
Self‐report data indicate that having information does help one manage medical problems, in a conditional way. The degree to which this occurs also depends on the type of user. For example, Living Health users stated that the type of service it provided could help them in relation to consultations with a doctor. There was further evidence that it could serve as an alternative to information from a doctor – though non‐users indicated that they preferred getting medical information straight from the doctor, whether spoken or in writing, than any other information source.
For the pregnant women of Channel Health, the TV/interactive service was seen mostly as a supplement to seeing health professionals. With their particular condition, it was still essential to have physical diagnoses and checks with doctors and nurses, etc. There was some indication that a service such as Channel Health’s interactive elements could be used in relation to reminders of appointments – as endorsed by health professionals. However, as the Living Health experience demonstrated, there is a relatively slow take‐up for an actual online appointments booking service.
Summing up, the DiTV online health service pilot delivered mixed results across a variety of services. Early audience uptake of these services indicated that a market for digital health does exist. The public are likely to welcome an online health information service sourced by a trusted brand. Health‐related programmes, especially on themes of special interest to niche audience groups, will attract viewers, as will health‐related videos on demand, provided they are of good production quality. Enhanced information services that lie behind familiar broadcasting will also be accessed, provided the user interface is not overly complex. More sophisticated interactive services that invite two‐way flows of information between the user at home and health service provider are likely to take longer to become established. In popular parlance, television is a “lean back” medium that has not traditionally invited its users to actively engage with it. In contrast, the personal computer is a “lean forward” medium that has always encouraged a high degree of interactivity on the part of its users.
Different technologies are associated with distinct psychological dispositions on the part of their users. This distinction must not be forgotten when considering television’s potential as an interactive technology. Over time, these two technologies may become largely indistinguishable, as will the psychology of users in each case. Until then, any application of television as an interactive health information medium must be mindful of these distinctions and take them into full consideration when determining the nature and form of digital health services on the box.
Notes
- 1.
1 The Telewest digital household figures for Birmingham were: 1st June 2001: 38,149; 1 December 2001: 51,963. This provides an average household figure across the pilot of 45,056 homes.
- 2.
2 The Web, the kiosk, digital TV and the changing and evolving face of consumer health information prevision: a national impact study, 2000‐2003. DoH funded.
References
City University (2002), First Steps Towards Providing the Nation with Health Care Advice and Information via their Television Sets. An Evaluation of Pilot Projects Exploring the Health Applicants of Digital Interactive Television, City University, London.
Huntington, P., Williams, P. and Nicholas, D. (2002), “Age and gender user differences of a touch‐screen kiosk: a case study of kiosk transaction log files”, Journal of Informatics in Primary Care, Vol. 10 No. 1, pp. 3‐9.
Nicholas, D., Huntington, P., Williams, P. and Gunter, B. (2001), “Delivering consumer health information digitally: platform comparisons”, International Online Conference, Olympia December 2001, Learned Information Limited, Oxford, pp. 145‐53.
Nicholas, D., Huntington, P., Williams, P. and Gunter, B. (2002), “Digital visibility: menu prominence and its impact on use of the NHS Direct information channel on Kingston Interactive Television”, Aslib Proceedings, Vol. 54 No. 4, pp. 213‐21.
Williams, P., Nicholas, D. and Huntington, P. (2001), “Walk in to (digital) health information: the introduction of a digital health information system at an NHS Walk‐in Centre”, CD & Online Notes, Vol. 14 No. 2, pp. 4‐7.