Open access in context: a user study

6.1.1 Article and abstract page views

For the first six months of 2003 176,000 views were made per month and this rose very sharply from July 2003, reaching a peak of about 287,000 in November 2003 – a 60 per cent increase all told. Apart from a seasonal dip in the December, use remained at this level until March 2004. Use then dipped significantly, falling to a relatively low level of 220,000 in August 2004. Use subsequently rose in the September to January period, reaching a new high in January 2005 of 424,000 views. The peak in January 2005 was over twice as much as recorded two years previously in January 2003 (Figure 2).

The main finding is that use of NAR has increased – and dramatically so. It increased by 143 per cent from January 2003 to January 2005[1]. Given that NAR is a well established, research journal with a defined, specialist readership, the fact that so much growth could occur in such a relatively small time has to be regarded as surprising. NAR went open access on 1 January 2005 and, significantly, usage increased between November 2004 and February 2005 by 19 per cent. On the face of it then, there appears to be an open access impact, especially as most of the increase (16 per cent) occurred between December 2004 and January 2005 when the journal changed to full open access.

Consider the differences users searching in December and January encountered. Firstly, those people searching in December who were not registered users would have found that articles published July to December were embargoed and they could not view the full‐text of these articles. Secondly, those searching in January might have been influenced by the announcements of the journal moving to OA – after all, this was the first journal to adopt this form of open access publishing, whereby the existing subscriber model simply changed to an OA one and the publicity would have inevitably generated interest. Hence an increase in usage would have been expected as a result of the journal changing publishing mode.

However, care needs to be taken in ascribing the 19 per cent increase wholly to open access as growth has been occurring for a long time and that pre‐2005 major increases have already taken place. Thus usage had increased in the two year period prior to the move to open access by 143 per cent, an increase of about 4 to 5 per cent a month. The danger is that the factors driving this usage may also be driving the increase in January and February 2005 that appear to be the result of open access.

An additional complicating factor was that January and February were traditionally recovery periods from the traditional December low. Hence part of the 19 per cent growth posted maybe accounted for by this recovery. To get closer to the truth it was decided to compare the month on month percentage increases immediately before and after the open access launch – November 2004 to March 2005, to the same period in the previous year (Table I).

Both periods saw a percentage fall in December although the fall in 2003 was greater; −12.8 compared to −9.9. The main recovery in 2003/2004 occurred in January when usage increased by 31.2 per cent although much of this was a recovery from the falls in December 2004. This increase was significantly higher than the increase in February 2005 of just 5.4 per cent. Looking at the difference between November and February the increase over this period was 19.2 per cent in 2004/2005 and 10.4 per cent in the 2003/2004 period. It seems that the increase in 2004/2005 was higher, by nearly as much again, as the increase recorded in 2003/2004. It seems that usage really did increase as a result of OA.

It was decided to focus in on the change in article usage between the three week period before the winter semester break and the three weeks after. Figures 3 and 4, respectively, give the weekly article downloads for 2004/2005 and 2003/2004. Comparing these two periods[2] average daily use grew 6 per cent in 2004/2005 (Figure 3) but fell by −9.5 per cent (Figure 4) in the same periods[3] of the previous year. Taking the previous year as an example it would have expected that usage would have fallen but usage in the open access period actually increased.

It is apparent from Figure 3 that usage from January was more assured. The highest weekly use for 2004, of 72,935 article views, was recorded in week 50 – the week beginning 9 December. This figure was then exceeded in the second week of January 2005 by 4 per cent and again in week 5 by 6.3 per cent. In fact in the 2005 period considered, the high in December 2004 was exceeded in all weeks except the first. This only happened once during the 2003/2004 period (see Figure 4).

6.2.1 Type of page viewed (article, abstract)

Views to table of contents and lists of issues have been excluded for this analysis because open access is all about opening up full‐text to users who could not access it because of the subscription model and therefore the concern is only with access to content. Figure 5 gives the daily use of items (articles or abstracts) viewed within an online session. Article only sessions accounted for about two‐thirds of use, abstract only sessions for a quarter and sessions where both were viewed accounted for 10 per cent of sessions. Given that users had a choice to view an article or abstract the fact that one in four sessions just viewed abstracts is evidence of the enduring popularity of the abstract.

Table II gives the percentage month on month changes in usage for the periods before and after the introduction of open access. Although there was a lift in abstract only use in December 2005 of 27 per cent this in part was compensation for the fall in abstract usage in November, there is a downward trend in abstract only session use in the first three months of 2005. The real growth in usage came from article only sessions (a 19 per cent increase in January) and in mixed sessions, where articles and abstracts were viewed (a 25 per cent increase in January). Thus, as result of moving to an open access model the signs were of a shift from abstract only sessions to a greater use of articles. This would have been expected.

Focusing on the three week period before the winter semester break (weeks 48 to 50, 2004) and the nine weeks following the break (weeks 2 to 10, 2005) abstract only sessions decreased by 13 per cent but article only usage increased by 7 per cent; sessions viewing both increased by 10 per cent. This provides more evidence of a shift from abstracts to articles, as result of the move to open access.

The high proportion of article only sessions is partly a function of the fact that for PubMed users, who made up about a third of users (Figure 6), abstracts were viewed on the NAR site as part of a sifting articles for relevance process and partly by the fact that it was just as easy to view an article as an abstract. Thus, looking at users who did not access the site via PubMed, just under two‐thirds (58 per cent) viewed articles, 27 per cent abstracts and 16 per cent both. This compares to just 1 per cent of PubMed users who viewed abstracts on the NAR site, 85 per cent of these users viewed articles and 14 per cent viewed articles and abstracts on the site.

6.2.2 Type of article viewed (free/non‐free status)

Open access is a mechanism for making articles free to view and the impact on usage of articles once not free to view is a key part of the investigation. Of course, free material was already being offered free prior to the introduction of full open access and its impact must be seen in that context. For the purpose of this analysis free status articles were:

• •

  articles available free after release from a six month embargo period;

• •

  articles from the two issues a year which were released within the (six month) embargo period – these were the first issue in each year and issue 13 – these became open access in 2004; and

• •

  all articles published once the journal went open access from the January 2005.

Sessions were grouped by what was viewed: those that just viewed free items, those that just viewed non‐free items and sessions where both were viewed. Free article sessions increased notably in June 2003, the date search engines were allowed in, by about 30 per cent and there after increased until January 2005 when all articles became free. In terms of sessions where just non‐free items were viewed the use of articles remained constant throughout the period.

Free article only sessions accounted for about two thirds (63 per cent) of article views in the first two quarters of 2003. This rose to three quarters from the first quarter 2004; but did not rise above 81 per cent. Sessions in which non‐free items only were viewed accounted for about a quarter of all session in the first two quarters of 2003 and this fell to 14 per cent by the last quarter of 2004.

Figure 6 gives the distribution of referrer information across quarters in 2003 and 2004 for free item sessions only and helps answer the question who used free items. At the beginning of the period free item use was predominately accounted for by users coming in from PubMed (47 per cent) and those coming from elsewhere in the Oxford Journals site (33 per cent) or from the NAR homepage. This changed from the third quarter 2003. From that time free articles were increasingly used by people coming in via search engines and it was this group that were behind the increased take‐up of free articles. Those coming in via search engines made up a quarter of free item use in the third quarter of 2003; however, they had increased their use share of free articles to over a half (51 per cent) by the fourth quarter of 2004.

In conclusion, it can be stated that the long term growth in article usage was driven by the demand for free articles.

6.2.3 Age of article

It was decided to focus on articles six months old or more recent. Before January 2005 these would have been embargoed and only available to subscribers and users linking in from an external site such as PubMed or PMC. After January 2005 articles six months and more recent were free to access. This analysis examines the question of whether the views to these newer articles had increased directly as a result of the journal going open access. Figure 7 gives the number of views to older and younger material. And there does appear to be quite a major open access impact. There was a rise in usage of six month or recent material after January 2005. The rise in usage in January 2005 is true in regard to both sessions where just recent material was viewed and in sessions where both young and old (Mixed) material was viewed (solid black line). This is looked at in more detail below.

It was decided to investigate the daily average[4] for the months November 2004 to March 2005 (Table III). A 5 per cent trimmed mean was used to handle some of the fluctuations of data; however, this did not fully accomplish this. Thus the percentage increase in usage of older material in between December and January was about 21 per cent. However, some of this increase represented a recovery from the low December semester break figure which showed a fall of 11 per cent. A comparable trend was recorded for the period 2003 to 2004, although the pickup from the December fall (−7 per cent) did not really happen until February (25 per cent). Generally, the percentage increase of older material between December 2004 and January 2005 was much in line with what was expected.

In terms of average daily article views for sessions just viewing the most recent 6 month material this increased by 42 per cent on NAR becoming open access. Again, part of this increase reflects the poor figures recorded in December, which recorded a fall of 11 per cent on the previous month. Comparisons with the previous year (2003‐2004) were made difficult as a result of a 64 per cent month‐on‐month increase recorded in February. This increase was exceptional and should be considered in the light of the month on month fall in December (2003) of 27 per cent, which was followed by a fall in January (2004) of −3 per cent and in March of 17 per cent. It is believed that the real increase in views for sessions just viewing recent material in January 2005 was about 40 per cent.

With regard to those people viewing both old and recent material in a session their use of articles doubled (103 per cent) between December 2004 and January 2005. Clearly these users have taken up the open access opportunity and increased their usage; this in part reflects the number of sessions including a view to recent material in their sessions. Interestingly this group's usage declined by 11 per cent in February.

A further investigation was undertaken to examine a narrower time window: the three week period (weeks 48 to 50) prior to the winter semester break was compared to the nine week following the break (week 2 to 10). The use of material six months and older increased just 3.5 per cent, while recent material, which was previously embargoed, increased by 27 per cent and use by those sessions viewing both increased by 79 per cent.

In conclusion, there has been an increase in the use of more recent material as a result of it becoming freely available under open access. This increase is currently quite small in absolute numbers; however, this may increase as users become more aware of the full implication of what has occurred.

It might be expected that open access would benefit the type of user who likes to keep up to date – clearly with OA “current awareness” now comes free. Clearly current awareness users will return more frequently. However, there was an insufficient length of time to test this assertion. The above analysis of recent (below six month) material suggests a positive result – a 40 per cent increase in those just looking at current material.

Clearly there is a digital visibility (Nicholas et al., 2002) issue here – how do “currency” interested users know the journal has gone OA? The gain in mixed aged material suggests that users are finding recent material as a result of exploring or linking in the site. It would be expected that, in the long term, users would become aware that recent material is now free. So the full impact of OA may well take a period (18 months) to be fully realised.

6.4.1 Types of user

Registered users (subscribers) v. non‐registered users. Clearly the success of open access will be determined by whether it draws in non‐subscribers. Figure 11 examines article usage broken down by whether the user was a registered user or not by day. Use by registered scholars for the first six months of 2003 was running at more than double the usage of non registered users: approximately 2,900 views a day compared to about 1,300. From July 2003 paths narrowed and use from non registered users was increasing at a faster rate. In fact, by October 2004 there were nearly as many non‐registered use as registered article use (3,900 daily views). Usage by both groups reached a peak of about 5,600 daily views in January 2005. The extent of non‐registered use shows that there was a strong demand for NAR outside of the consortia/big deals.

Use by both registered and non registered users appeared to follow the academic pattern with less use recorded in the summer, when many researchers were away at conferences, and greater use recorded in the winter and spring period. In addition both groups recorded a strong increase in usage, especially in the period October 2004 to March 2005, when another driver was impacting (see following search engine section). Usage by non registered users increased at a faster rate compared to registered users particularly in 2003 – 140 per cent, compared to 35 per cent for the period May 2003 to November 2003.

A more focused analysis comparing the three week period (weeks 48 to 50) prior to the semester break to the nine week period after (weeks 2 to 10) argues that, counter intuitively, use by registered users actually increased in the period following open access compared to use by non‐registered users.

An examination of the use distribution of referrer link by registration status of user showed that most registered users (predominately academic organisations), entered the site via PubMed (40 per cent) but increasingly did so from search engines (25 per cent), and also via OUP menus (22 per cent). For non‐registered use it was mainly via search engines (40 per cent) and PubMed (25 per cent).

In conclusion, use by non‐registered users has always been an important feature of NAR. The door was always open to such users (via PubMed Central) and as the door opened even wider (because of open access and search engine access) then this has driven usage levels even higher.

Organisational affiliation. The change in the type of people using the journal was investigated. A reverse DNS lookup on the IP numbers was completed and organisation type was extracted. DNS information was not available for 51 per cent of IP addresses. Figure 12 gives the distribution of organisation type for registered users for each quarter from 2003 to the second quarter 2005. The main organisation type, as expected, was academic and accounted for about 87 to 90 per cent of registered use.

Examining the month on month figures (Table IV) for academic and net provider use (organisations registered as internet access providers), January saw academic use record its highest increase of 30 per cent, however, this was making up for the low December figure (−19 per cent). The overall pattern is muddied by the usage increase in March by net providers and this may reflect users accessing the NAR service from home during the spring break. The spring break had a negative impact on academic usage (−4.4 per cent). Taking the November 2004 to February 2005 period there was not a lot of difference in growth of article usage between Academic and Net Providers: 16 per cent and 17 per cent.

Geographical location. Clearly OA is potentially hugely beneficial to people in poorer countries where users cannot afford to access, for what is to them, are expensive databases.

Figure 13 gives the item (articles and abstracts) use by month by DNS country type. The main growth in use has come from USA registered users who viewed approximately 41,000 items in January 2003 and about 150,000 in January 2005, an increase of 200+ per cent. However, many users with a DNS address registered in the USA were not located in the USA. In fact many of these users would be located abroad. More certainty can be had of the other geographical designations.

African usage continued to be rooted to the floor of the Figure, but there were signs that during 2005 use grew. South American use, similarly towards the floor of the Figure, also appeared to be on a slow rise, possibly boosted by open access. However, the greatest impact appears to be on East European countries (Figure 14).

Table V gives the month on month percentage changes over the period covering the introduction of open access. The biggest increase was the growth in usage in January 2005 of 88 per cent. Part of this increase was a clawing back of the traditional December loss (−15.4), a consequence of the winter semester break. The period November 2004 to February 2005 saw article usage increase by 59 per cent. However, this period was a seasonal growth period for Central and Eastern Europe countries and last year (2003/2004) this region notched up an increase of 30 per cent.

A comparison of the three week period prior to the winter semester break with the nine week period following the end of the break for 2004/2005 and the previous year 2003/2004 showed that article usage in the open access year increased by 43 per cent compared to 26 per cent in the previous year. So there is clearly an open access effect although this effect has occurred at a time when this region seasonally increases its usage. It is estimated that the increase in article usage by East Europeans as a result of going open access is about 20 per cent.

Referrer link. The interest here lies with the last site from which NAR was accessed. It is known from previous CIBER studies (Nicholas and Huntington, 2004) that net provider users will tend to use search engines to access and search the net. This use of search engines shows up in the referrer logs. It was decided to examine for changes in the last site visited for both registered and non registered users. Referrer information was not available for a third (32.9 per cent)[5] of IP addresses.

Figure 15 gives the monthly article use by referrer group for 2003 to the second quarter 2005. Referrer information was classified into five groups: OUP journals, PubMed, Search engine, external link and other. OUP menus referrer users were those who accessed via the NAR home page or from other OUP web site pages. With regard to external links these users had arrived at the main OUP site via an external organisation or website such as a link on a university site, a search engine.

PubMed is an external (gateway) website from which users could access NAR material and is a service of the US National Library of Medicine and administer by NCBI. PubMed includes links to full text articles and other related resources, for instance to NAR. Search engine users were classified as just those using one of the following search engines Google, Yahoo, MSN, Altavista, Ask and Earthlink. “Other” relates to all other referrer groups (for example www.highwire.stanford.edu, www.genome.org, www.sciencedirect.com).

There has been a substantial shift in the make‐up of users by referrer link ‐ most notably there has been an increase in the share of accesses via a search engine. The increase has come from a position where they accounted for less than 1 per cent in the second quarter of 2003 to account for about a half (49 per cent) of sessions in the second quarter 2005. This represents a dramatic and major shift in scholarly information seeking behaviour. However, other groups have remained relatively stable.

Search engine robots were allowed to index the site for the first time in June 2003. Thus from June 2003 users who favoured using a search engine to access web based scholarly information could find the NAR site. This represented a shift in the way that users could navigate to NAR and to navigate to information within NAR. With a search engine users could both navigate the net to find the site and navigate the site to find content.

Thus in June 2003 the digital “flood gates” were open and in the global environment that is the web this is clearly the prospect of large demands being made, although one might of thought that the highly specific and difficult nature of the content would afford some protection.

Daily movements. Figures 16‐18 give the daily movement of each referrer link category and these show real differences. Figure 16 plots article use by those finding the NAR via a search engine, Figure 17 concerns OUP menus and external links and Figure 18 looks at daily movements of the Other and PubMed categories.

The daily movement of those entering via a search engine highlights the falling of use in the summer months. Usage more than halved between May and July. This is thought to reflect the fact that search engines are most likely to be used by students, a group of users who are unlikely to use the service in the summer vacation months.

A comparison between article usage of those entering via a search engine during the three week period (48 to 50) before the semester break and to the nine week (weeks 2 to 10) period after the break shows little change.

Figure 17 details users coming in via OUP menus and external links. Use by either of these two groups did not fall dramatically during the summer vacation months.

A comparison of article usage before and after the Xmas semester break showed there was a 30 per cent increase in usage from those accessing the site via OUP menus. This is thought to be explained by the fact that those users coming into the site via the home page were more likely to be alerted to the fact that all of NAR was now free.

Figure 18 gives the daily use movement for those accessing the site via PubMed and Other referrer links.

Usage by those entering the site via PubMed increased by 12 per cent in the post nine week (2 to 10) open access period and this might reflect the updating of links on the PubMed site.