Portfolio of Learning - Investigating Index New Zealand title usage over 12 months

Investigate the Index New Zealand (INNZ) title usage over 12 months to get an idea of what journals are being accessed, when and how often. Present this information in a graphic form to the INNZ team/Content Services for discussion as a way to a) get more out of the listed statistics we regularly receive and b) challenge our current thinking and perspectives as to who our customers are and their access trends.  Evaluate how this knowledge has made a difference in how we decide on what journals to look at taking up.

BOK 6 - Research Analysis.
BOK 10 - Quantitative & Qualitative Evaluation.

Index New Zealand (INNZ) title usage over 12 months:

September 2011 - September 2012: Top 100

Aim

Investigate the Index New Zealand (INNZ) title usage over 12 months to get an idea of what journals are being accessed, when and how often.
Investigate how useful the monthly statistics we receive actually are.

Present this information in a graphic form to the INNZ team/Content Services for discussion as a way to a) get more out of the listed statistics we regularly receive and b) challenge our current thinking and perspectives as to who our customers are and their access trends. 

Evaluate how this knowledge has made a difference in how we decide on what journals to look at taking up.

Method

Utilize the monthly statistics provided to INNZ by the National Library of New Zealand for the past 12 months from September 2011 to September 2012.
I chose to use these statistics as they were the easier of the two ways we receive relevant statistics to access and assess. As the data was already provided to INNZ in monthly installments it removed the extra step of gathering it myself which was what the second method of gathering such statistics required. Because of the size of the data set the actual excel datasheets from which these results are taken are unable to be included on this blog.

Results

These are the results of the top 100 hits on INNZ records for specific journal titles for each month over the past 12 months (from September 2011 to September 2012). A hit is counted every time a user clicks through from the search results to a single record for a particular journal title.

Table 1. The combined totals for each month of the top 100 journal hits.

	Total
Sep-12	29829
Aug-12	35347
Jul-12	18436
Jun-12	28310
May-12	38742
Apr-12	37514
Mar-12	31691
Feb-12	17554
Jan-12	0
Dec-11	16338
Nov-11	19012
Oct-11	23326
Sep-11	29481
Total	325580
	27132

NB: there was no data available for January (whether this was because the data was not available or that there was no data created by users in the first place is uncertain).

Table 2. The average totals for each month of the top 100 journal hits.

	Average
Sep-12	301
Aug-12	357
Jul-12	186
Jun-12	286
May-12	391
Apr-12	379
Mar-12	320
Feb-12	177
Jan-12	0
Dec-11	165
Nov-11	192
Oct-11	236
Sep-11	298

Figure 1. The combined and average titles for each month

Because of the amount of information that was available in this data set consideration had to be given to what areas to focus on and investigate. What follows are the areas that have been selected for this study (see the 'Next steps' section below for future areas of study using this and subsequent data sets). 

General trends

• School holidays
• There is a general trend for the hits to follow the timeline of the New Zealand school terms and holidays
• Totals for each month versus average totals for each month
• Comparing the two bar graphs, while the trends in usage follow the same general pattern, what is noticeable is how much lower the average totals are. This suggests that there is a rather large tail of titles that are accessed only perhaps one or two times a month, thus dragging down the averages. If we were to average over all of the titles (not just the top 100 and including those that were not 'hit') we would expect a much lower average pulled down by the majority of titles that are only accessed once or twice a month, or not at all.

As well as the numbers of hits the dataset provided the names of the top 100 titles for each month (unable to be linked through to this blog). The following trends were noted in the following areas. 

Top 10 titles (receiving hits ranging from 340 – 3455)

• Dominated by general interest and current affairs-based titles.
• Predominantly newspaper titles, both ceased and current.
• Regularly filled in with the Listener, North and South, and the New Zealand Woman’s Weekly.
• Medical-based journals appear in four of the months, education-based and agriculture-based in two of the months and one month each of chartered accountants, and management

Newspapers

• Dominate the top 10 for every month (ranging from six to eight of the titles in each month), including ceased titles (Evening Post, Dominion) as well as current.
• Titles are spread throughout the top 100 titles.
• The main city papers are the most used (NZ Herald, Otago Daily Times, Dominion Post, and the Press).

Bios/Biographical information

There is a significant prevalence of newspaper hits not only in the top 10 but throughout the top 100 titles.

Education

• A noticeable presence within the top 100 for each month, ranging from nine titles in July 2012 to 25 in March 2012, with a mean of 11.
• Focuses of titles ranged from early childhood education to tertiary research.

Maori/Pacific Island

A presence within the top 100 for each month, ranging from four titles in May 2012 to seven in July 2012, with a mean of five.

Medicine/Health

A noticeable presence within the top 100 for each month, ranging from six titles in December 2011 to 17 in April 2012, with a mean of ten. One in every ten titles is therefore is medically-related.

Online

According to this data only 27/1200 hits were for online records.

Ceased

A presence within the top 100 for each month, ranging from three titles in Jul 2012 to five in September 2012, with a mean of four.

Discussion

Because of the amount of information that was available in this data set consideration had to be given to what areas to focus on and try to interpret. The following are the areas that have been selected for this study (see the 'Next steps' section below for future areas of study using this and subsequent data sets). 

School holidays

There is a general trend for the hits to follow the timeline of the New Zealand school terms and holidays.

School holidays:

Term 1	30-Jan	5-Apr
Term 2	23-Apr	29-Jun
Term 3	16-Jul	28-Sep
Term 4	15-Oct	20-Dec

Usage starts to drop off from Oct, through to Feb, due to the end of the school year and school holidays.

The no data result for January could be because the schools are closed and no one is accessing INNZ during that time. A comparison against previous years and the following years may shed further light in this phenomenon if similar results were discovered.

Top 10 titles

These titles serve to give us an idea of the top areas of interest for our users (general interest and current affairs). It would be worthwhile to regularly compare these titles over time to see if they generally remain the same or if topics of interest fluctuate.

Newspapers and Bios/Biographical information

The significant prevalence of newspaper hits (on both ceased and current titles) suggests that the significant part of our time spent indexing newspaper articles containing both current events and biographical information is worthwhile.

Education

Perhaps the 25 titles in March correspond to the start of the New Year when teachers are fresh and looking to develop their professional development? As the majority of our users are librarians and teachers directing their students towards us (this is borne out in the data we collect elsewhere on ISP addresses) it makes sense that education as a topic would be so prevalent.

Maori/Pacific

• While it is pleasing to see the amount of Maori/Pacific titles there are this result needs to be taken with caution as it might be suggesting that due to our low number of Maori/Pacific titles in the first place, the titles that we do have are only being accessed because they are the only ones available. If this is the case (and further investigation is required to validate this hypthesis) it should serve to remind us here at INNZ of the work we need to do to continue to develop the range of titles we offer.  
• Another possibility is that it is not our Maori/Pasifika users who are accessing these titles but other users. If this is the case then we may need to take into consideration the various Maori-based methodologies that our Maori users are utilising and see how well our product matches their specific needs.

Medicine/Health
The prevalence of medical-related titles suggests that INNZ is being used as a reliable source of information about all things medical as it provides one of the few opportunities for free access to professional medical journals.

Online

27/1200 hits for online records is a lot less hits than expected and suggests the following:

• That the kinds of records users are interested in are not online (see for example the hits on ceased newspaper titles, none of which are available online).
• That the data collection software does not pick up whether a record is online or not. This requires further investigation.
• That the online results are so low because once hit they are accessed whereas with the print titles users bounce around them until they find an online title with the information they want. This would explain why there is a lot more print results.

Ceased

These results suggest that there is still value in titles that are no longer being published or have been superseded by new titles. The fact that almost half of them were newspaper titles suggests that they are being used for biographical information.

Conclusions

What journals are being accessed, when and how often.
The very low rate of 'online' hits raises the question of why this is the case. Is it because when users are hitting these records they are finding what they want and accessing it. Does this in turn suggest that users may be bouncing off the print only titles they find until they find a relevant online hit. Or do these results suggest that online hits are not being recorded successfully? Further investigation is therefore warranted.

 

How useful the monthly statistics we receive actually are.

While the data tells us what INNZ records are being 'hit', i.e., clicked on by INNZ users, it does not tell us whether the user is then requesting that record, either by clicking through to the online full text version or requesting a PDF copy through Collection Delivery. Therefore we are only getting a portion of the information we need to use these statistics in a way that can direct what titles we focus on indexing and what titles we can afford to move away from. We were expecting a lot more online hits as then we would be able to deduce that

The most use these statistics seem to be is in highlighting the names of the titles that are being looked at (even if we don't know whether they are being accessed and actually used). We can get an idea of the topics that are being accessed by our users which can be used to direct the topic areas we are weak in and pick up appropriate journals accordingly.

At the end of this investigation I am left with the feeling that I am still not understanding exactly what all of this data is telling me.

Present this information in a graphic form to the INNZ team/Content Services for discussion as a way to
a) get more out of the listed statistics we regularly receive and
b) challenge our current thinking and perspectives as to who our customers are and their access trends. 

While this information was not presented to the INNZ team or Content Services (the branch wherein INNZ sits) it was presented and discussed within the senior members of the INNZ team. It has spurred us on to establishing a dialog with the appropriate report managers to see what other datasets we can receive in order to investigate this area more fully. For example finding a way to include all of the titles (and not just the top 100) to see how the results really are spread. We need to dig deeper into the topic areas of titles and investigate the actual areas they cover in order to see where our users are spending most of their time and whether this can then shed further light on our topic strengths and weaknesses.

How this knowledge has made a difference in how we decide on what journals to look at taking up.

The new knowledge gained from this investigation has not yet made a difference on the journals we decide to index. It has however focussed our awareness on the kind of data-based information we are actually getting and how difficult it is to work out exactly what it is telling us. Subsequent discussions around what datasets are available to us at INNZ has helped to refine our focus and has led to us requesting more specific datasets (such as OPAC search counts) that come closer to giving the information we require. We are now in the process of investigating these new datasets to see what information we can most effectively use.

Next steps

Compare these results against the next 12 months and previous 12 months to see what, if any, other trends can be deduced.

Include results for all hits, rather than just the top 100 to see if doing so distorts the current results, or gives a more accurate view of the range of titles being accessed.

Investigate how to use the data to work out when the amount of hits a title (or range of topic-related titles) gets becomes too low to warrant indexing and then work out what topic areas are attracting enough hits to warrant the indexing of similar wish-listed titles that we have on our waiting list.

Compare the general subject areas trending in these results to those areas required by the different sections of our users to see how well they match up e.g. the topics of education required by our school users.

Investigate the very low rate of 'online' hits. It is because users find an online resource and use it straight away while they bounce from print resource to print resource searching for the online version? Or is it because the data gathering programme used was not specific enough to gather such online information?

Further to this, continue to investigate what data is available that could show us what resources our users are actually utilizing from their searches and how they access them – whether print or online. Do they click through to the fulltext online resource? Do they order the PDF version through our Document Supply area? Do they just use INNZ as an index and locate the actual resource somewhere else? How to they get the information they want? Once we discover these access trends we can use the information to direct how much focus we give to the format of the different journals we index, pick up for indexing, or cease to continue to index.