The ARK Project: Analysing Raptor at Kent

It is indisputable that the use of e-resources in university libraries has increased exponentially over the last decade and there would be little disagreement with a prediction that usage is set to continue to increase for the foreseeable future. The majority of students both at undergraduate and post-graduate level now come from a background where online access is the de facto standard. Add to this the ubiquity of mobile devices in the form of netbooks, tablets and smart phones and it is apparent that a considerable percentage of the service provision from libraries does and will continue to involve on-line resources.

By online resources we are not referring solely to journals and databases but also to local resources such as library catalogues and research repositories and even local ‘live’ services such as the ‘Find a free PC’ service, provided via a mobile app at the University of Kent, and similar services in many other Higher and Further Education institutions.

The number and range of on-line resources a university may be supplying to its staff and students is now enormous and continues to grow. Similarly the offer from many online subscription services continues to increase in breadth as well as size as more and more archived resources are digitised and made available electronically.

Managing access to these on-line resources is key to the provision of library services which satisfy the needs of the library’s users. The ability to manage provision and access to these services successfully is greatly enhanced by the availability of accurate and timely statistics on current and past usage of e-resources and the identification of trends which can inform future subscriptions and the provision of infrastructure to facilitate access.

Answers to  Librarians’ Questions

Library managers need answers to questions such as:

• Are the e-resources subscriptions we provide being used at the level we expect?
• Is there a usage trend upwards or downwards?
• Which schools or departments are the main users of a resource?

There may be other questions relating to security and compliance – can we be confident that only those authorised by the terms of the licence are accessing these resources?

Until recently data relating to the usage of e-resources has not been readily available to library staff in a granular form that gives the degree of detail needed. When a user accesses an e-resource at Kent they do so through EZProxy middleware or through Shibboleth-federated access management software. These methods are common to many university libraries though alternatives do exist. Using either of these methods to authenticate to e-resources creates an entry in a log file. Log files usually take the form of small text files which contain a log of events relating to the use and release of an application. Primarily log files are there to help with diagnostics when things go wrong and to provide an audit trail of usage. They can also be used to analyse security issues and to identify weaknesses in a system.

The main problems with log files for the average non-technical user is that they are written in an almost impenetrable form and that each log file refers to only one event – ie access by a single user on a single occasion. For a member of the library staff, compilation of these log files to provide useful information for analysing access to e?resources would be a tedious and complex task.

However in 2011, a team at Cardiff University, with funding from JISC, developed the Raptor Tool Kit which takes a lot of the work out of this process and makes the data stored in the log files accessible in the form of graphs and downloadable data tables [1].

How Raptor Can Help

So what does Raptor actually do? In a nutshell, Raptor sits on a system monitoring, in the case of library e-resources, authentication events. It extracts the relevant information from the log files of these events and stores that information in a database. The front end of Raptor accesses that database allowing users to view the event information in summary or queryable form.

The University of Kent was an early adopter of Raptor and experimentation in 2010 convinced the IT and Library teams that the tool kit had strong potential to become a useful tool. Early tests on temporary hardware did not provide an opportunity for detailed exploration of Raptor’s installation, performance and use as a production service. For these reasons, and because we were convinced that Raptor outputs could make a real difference to our understanding of the way e-resources were being accessed, we were very pleased that our bid for JISC funding to continue working with Raptor was successful. The ARK Project (Analysing Raptor at Kent) [2] began in June 2012 and final reports will be published in January 2013.

The main aims of the Analysing Raptor at Kent Project were:

• to install Raptor as a production service;
• to design and publish reports and graphs in collaboration with stakeholders in the Templeman Library;
• to evaluate and analyse the usefulness of these reports to inform the strategic planning of  library and other staff.

We also wanted to explore the possibilities for hooking up the Raptor database (or an exported copy of it) to other analysis and presentation tools such as Microsoft Reporting Services. Raptor is still in the early stages of development and I think it is fair to say that the user interface and some of the processes necessary to customise and set parameters for output have proved to be somewhat obscure to some library staff.

Installation of Raptor proved to be straight-forward and documentation in the Raptor wiki was adequate to get the job done. Matthew Slowe in our Server Infrastructure Team documented the process and will make this available to the wider community.

Early meetings with all stakeholders produced a list of reports that librarians felt would be most useful to them. Raptor comes with some standard reports out of the box – the number of authentications over set periods (day, week, month, year), authentications per school, authentications per affiliation type etc. Additional reports specific to the e-resources to which the hosting organisation subscribes can be added – for instance Authorisations to Science Direct resources grouped by School. The process for the creation of bespoke reports involves externally editing XML files on the server and would not likely be seen as a task appropriate to library staff (though of course some organisations may employ librarians with sufficient technical skills). This is one of the limitations of the current version of Raptor since even though some post-processing, formatting and customising is possible within the application interface - setting time periods and sorting results for instance - new reports and major changes would not be available without reliance on IT staff.

Once configured with a relevant set of standard reports, Raptor provides the non-technical user with a quick route to producing and customising a number of charts and tables which can be output as a combined PDF file. For instance a library user may wish to create a report of authentications to a particular online journal provider, say Science Direct and can select this from the Raptor reports menu. Raptor will display, under separate tabs, a graph, a table of results and export and print options using the report’s default values. Standard UI tools are provided to customise the report. The start and end dates can be set using date pickers, drop-downs control the ranking of the results and the number of results displayed can be set to make the graph more readable. Within Raptor the user can also choose from a number of other options to set the graph type and position of labels.

Raptor also gives the user the option to use filters and to change the grouping of results; but in the current version some knowledge of the underlying data from the log files is required to make meaningful choices. The drop-downs and auto-complete features, whilst convenient and easy to use, draw directly from the log files or from LDAP, so terms presented may be somewhat obscure to non-technical staff.

Despite these limitations, these reports have been well received and staff likely to be responsible for producing them have provided mainly positive feedback on the user interface and ease of use. Raptor also features an export facility to allow users to work externally from the application on Excel formatted and CSV data. Many library staff are experienced Excel users and saw great value in being able to further analyse data and to present it in a wider variety of ways than those available in the current version of Raptor.

Exporting the Raptor data to Excel does give users, depending on their Excel skills, a much broader range of options. The data can be filtered, sorted and normalised and Excel features a huge range of chart and graph types.  The facility to add, format and position labels and annotations is much improved over the native application. Excel users can also aggregate data from more than one Raptor report, or even from other databases into a single Excel spreadsheet. Advanced users might use look-up tables and calculated fields to enhance results and presentation. However, ad hoc Excel graphs and spreadsheets are still limited to the data output from pre-configured Raptor reports.

Further Development

Currently the Analysing Raptor at Kent team is exploring how we might use Microsoft Reporting Services (MRS), a tool which already has a user community at Kent, to publish standard time-based reports to Web pages or to Sharepoint – authentications for the last month by school or resource for instance. MRS, through the much improved Report Builder 3.0 tool,  may also be a route to equipping library ‘super-users’ with robust analysis and presentation tools without adversely affecting either the resources of the IT teams or the performance of Raptor. Data sets, SQL queries and data views can be defined and published to all users. Well-designed wizards within Report Builder make the creation of visually excellent charts and parameterised reports with drill-down features a possibility for a wide range of staff without the need for specialist skills and training.

We are also looking at how we might interface with other databases to enrich the Raptor data further. This might be basic look-up tables to resolve LDAP abbreviations for schools to more meaningful terms, but the possibilities go much wider. There may be value in cross–referencing results against e-resource usage or in identifying unusual patterns of use by individuals or groups which could identify areas for concern or help to target additional support. Should we be looking at offering more support to students who do not appear to be using e?resources in the expected way or whose patterns of resource access suggest they might be struggling to meet deadlines? Building on the work of the LIDP Project at the University of Huddersfield [3], we would be interested in looking for correlations between e-resource usage and student grades. Could we drill down further and examine whether there are particular disciplines where online journals can be demonstrated to have a positive influence on outcomes?

The RAPID Project at Newcastle University [4] also looked at further uses for Raptor and concluded that, with a little work, the application could be used to record and analyse a diverse range of access events such as entry and exit to and from a building or room – perhaps a PC cluster room, via an electronic door. Many libraries employ entry gates software so the opportunities for interfacing Raptor with this task and the resultant data seem an obvious area for investigation.

We have also been able to use Raptor to provide evidence of e-resource usage by the students of schools who were not previously thought to be accessing these resources. Raptor provides very valuable business intelligence which, combined with other indicators, can support the development of internal resource-charging models.

One area that needs a little thought is that of compliance with the Data Protection Act. There is great value for the Library and the University in analysing archived logs, but we may need to anonymise these data beyond a certain period. We need to decide on processes and schedules to make sure data are anonymised on a rolling schedule whilst preserving the richness of the information held in the log files. The username is a link to a great deal of other data – programme of study, modules taken, personal details such as age, gender and status. There is a risk that the opportunity for useful analysis would be lost if we do not take this in to consideration when designing the anonymisation procedure.

Conclusion

Raptor is in the early stages of development but has already proved itself as a useful tool for librarians and other staff. The standalone version of the software will continue to develop and provide further features and options. The ARK Project has shown that Raptor can be used by non-technical staff to produce useful information quickly which will assist library staff and senior managers to design better targeted  e?resource provision and to be confident that charging models are appropriate and fair. The data produced by Raptor can be used with other database and presentation tools to give further options for analysis and the production of rich reports and charts. Cross-referencing Raptor data with other databases presents exciting opportunities for learning more about the impact and potential for the use of e-resources in Higher and Further Education institutions.

Although brief, Analysing Raptor at Kent has proved to be an interesting and useful project, allowing us to build on our early experience with Raptor. I am confident that Raptor will continue to develop, and skilled analysts will glean interesting and previously undiscovered insights into how library resources are used and how they might be used in the future.

Further information and discussion on the ARK Project can be found on the Project blog [5].

Acknowledgements

We are grateful to JISC and our programme manager Chris Brown for supporting us in our efforts throughout the course of the ARK Project.

References

1. The Cardiff Raptor Project Web site http://iam.cf.ac.uk/trac/RAPTOR
2. Analysing Raptor at Kent Web site: http://www.kent.ac.uk/is/projects/ark/
3. LIPD Project at the University of Huddersfield http://library.hud.ac.uk/blogs/projects/lidp/
4. The RAPID Project at the Newcastle University http://research.ncl.ac.uk/rapid/
5. The ARK Project blog http://blogs.kent.ac.uk/ark/

Author Details

Leo Lyons
Project Manager
Learning & Research Development
University of Kent

Email: l.lyons@kent.ac.uk

Leo Lyons is a project manager at the University of Kent working within the Learning and Research Development department. Over a 20-year career in IT he has been a trainer, has run help desks, and designed and coded database management systems for the public sector. His work in recent years has included projects on engaging with alumni, e-folios and improving the presentation of university course data.