AGORA: The Hybrid Library from a User's Perspective

Agora is one of the five Hybrid Library Projects that began in January 1998, forming part of phase 3 of the elib programme investigating issues surrounding the integration of digital and traditional library resources. It is a consortium-based project, led by the University of East Anglia; partners are UKOLN [1], Fretwell Downing Informatics and CERLIM (the Centre for Research in Library and Information Management). The project also works with several associate groups: libraries, service providers and systems developers.

The initial six months of the project concentrated on the development of a prototype HLMS (Hybrid Library Management System) based on Fretwell Downings VDX software. The prototype was evaluated and the results fed into the system definition for the first "real" HLMS which emerged in the spring of 1999. The document lays out the Agora vision of the HLMS, most explicitly in the requirements catalogue, which sets out over 145 detailed requirements. [2]

Prioritisation of these requirements was undertaken by the library associates and project staff to determine the functions to be included in Release 1 of Agora and those to be included in the later Release 2 version [3]. Once Release 1 had undergone compliance testing in the autumn of 1999, it was installed at the library associate sites ready for the next phase of the project - the undertaking of case studies. This article will look at the main focus of the case studies, summarise the results and evaluate how this work can be used to inform the future direction of the project and the ongoing development of the Hybrid Library concept.

Case Studies

The case study proposals were submitted to the Agora Board in May 2000 and were accepted in principle at that stage. Further progress and planning was aligned with both the training requirements of the library associates and the installation of a fully configured server at each of the case study sites.

Case Studies were to be undertaken at the four library associate sites, with a fifth study to be carried out by UKOLN, at the University of Bath. The case studies were seen as a change of focus from the previous work of the project, moving away from a technology focus and towards a process and policy focus - a human approach designed to inform the wider community of the reality of the hybrid library.

Four case studies took place in August-September 2000 and used the first release of the Agora HLMS. University of East Anglia had some technical difficulties so delayed the start of their study until late September. This resulted in UEA being able to migrate their case study to Agora Release 2.

Methodology

Similar methodology was used across the case studies. All the studies gathered data by way of a questionnaire. The same basic questionnaire type was used by most with the addition of specific questions relevant to each associate site. All the sites offered some form of introductory/training session, although the number, length and complexity of these sessions was determined locally. Additional qualitative data was gathered using follow-up interviews and in one case the use of journals for interlending staff. All users participating in the study were supplied with system documentation. This documentation was prepared locally to meet the needs of the specific case study.

The case studies were varied in nature concentrating on users from a variety of different groups:

• Professional and paraprofessional library staff
• Academic and faculty staff
• Postgraduates/research students
• Distance Learners

No undergraduates were able to participate as the studies took place during the summer vacation.

Case study results

The case studies covered four major issues; the creation and use of landscapes, the utility of collection level descriptions as resource metadata, user reaction to, and use of, cross-domain searching, and the use of interlending as a document delivery mechanism within the system. The case study process and experience also threw up some findings with regard to the possible implementation of such systems and the conduct of any further case studies.

Landscapes

The concept of information landscapes is integral to the Agora HLMS. The term is used to describe a way of presenting different views of information resources to users according to their interests and needs.

Evaluation of Release 1 by the library associates and the compliance testing process had revealed significant shortcomings in the interface to the system, particularly in relation to the creation and use of landscapes. Whilst these

shortcomings hindered comprehension and use, the associates were confident that users would be able to provide some evaluation of both the functionality and the appearance of the system as a whole. The ultimate goal of the studies was to provide feedback on how the hybrid library concept worked in a library environment.

The case studies examined a number of different landscape options;

• A single pre-configured landscape put together to reflect the perceived user group requirements e.g. subject specialisation.
• A range of pre-configured landscapes
• Access to all resources with the ability for individual landscape creation

All the case studies received feedback about the landscape function, and several common themes emerged.

Firstly, there was confusion about the meaning of the term landscape. Users did not feel the functionality matched their conception of the term and that the term could be misleading.

Secondly, all users, be they professional librarians or students, found the landscape function over-complex and difficult to understand. But, despite misgivings about the actual functionality, there was a general acceptance that with training, documentation, and extensive help within the system, landscapes could be more relevant and useful, particularly in the area of making users aware of new or undiscovered resources. Landscapes were also seen as an important means of introducing users to a subject by exposing them to a wide array of resources, albeit at a shallow depth.

Thirdly, all agreed that more research needs to be undertaken to ascertain how best to describe the concept of grouping information in this way and how landscapes are used by different user groups.

Collection Level Descriptions

In order to provide information landscaping it is necessary to match information about users, against information about resources. This is done by surrounding the landscapes with descriptive metadata; Collection Level Descriptions which facilitate the discovery and organization of resources.

There were 58 CLDs in Agora Release 1 - one CLD for each of the available resources which included catalogues (internet and library), subject gateways, commercial databases and other mixed media. Reaction to CLDs was mixed.

It was acknowledged that CLDs were useful as a way of learning more about unfamiliar resources.and in assessing their relevance for inclusion in different landscapes. However, there was a consensus that more work will need to be done to enrich the CLD content to provide more information about the different collections. Further enhancement and refinement of the controlled language to aid searching is also felt necessary with special mention being made of the lack of a developed subject schema.

This conclusion tallies with a recent Dlib paper [4] authored by eLib project staff in which it was acknowledged that "use of controlled language," was one of the areas that would need to be addressed. It is hoped that the RSLP and JISC-funded HILT (High-Level Thesaurus) project [5], in which Agora is a stakeholder, will provide a set of recommendations to facilitate cross searching by subject.

Cross-Domain searching

There can be no doubt that the principle of cross-domain searching was proved in the use of Release 1 of Agora. Users were pleased with the ability of the system to provide images, bibliographic records, URLs and full text. However, the success or failure of searching was dependent on the construction of suitable landscapes and on the quality and reliability of the resources within the landscapes. Large collections of unfocussed, poor quality resources inevitably led to numerous marginally relevant results. There was some evidence that research students were happy with the wide range of resources accessed, but library staff members were more demanding in their requirements, wanting to create smaller landscapes of similar resources.

Cross-domain searching across a number of resources had the effect of highlighting the resources that were returning errors or were unavailable. Continual error messages were disappointing for users and had a detrimental effect on their confidence in a particular resource. This demonstrated how important it is for the Service Providers to keep all users of their systems up to date with planned downtime, changes in IP address or data structure, so that any interruption in supply can be minimised.

Ensuring that the ‘right’ resources are available proved to be as important as ensuring the reliability of the resources within the system. Both in the context of discussions regarding landscapes and searching effectiveness, participants noted resources they wish to have added in order to make the concept and operation of the system as a whole more acceptable. Regardless of the utility of landscaping, searching and delivery of results, lack of appropriate resources was seen as a significant disincentive to use of any HLMS.

Interlibrary Loans

The integration of discovery and searching functionality with request and delivery functionality is a major step forward in the seamless integration of interlending as a delivery mechanism within the Agora HMLS. Interlending requests can be generated by the completion of a blank request form within the interface or can be generated as a result of selection from a search hit list. When this functionality was tested in a case study scenario it was reported as being easy to use and an improvement on the institutions own manual ILL system. In fact it was interesting to note that users felt that the integration of search and locate with request and deliver was a natural progression. They did not see interlending as a separate function from other forms of document delivery although most library organisational structures make this distinction. In this sense the users readily accepted the hybridity that Agora offered.

Both users and administrators of ILL approved this integrated approach but felt that more work needed to be done to make the system ready for operational use. Proper and extensive training on, and use of, the system were seen as crucial.

Process

In carrying out the case studies, the Library Associates and Project staff also gleaned information about possible issues in the implementation of such systems and for the conduct of future case studies.

Perhaps the most important lesson in both cases is to have a stable system. Whilst it is in the nature of new systems to have ‘teething’ problems, the unreliability of resources and problems with the system itself (both in local configuration and functionality) served to inhibit the gathering of results for the intended purposes of the case studies, even where a ‘negative’ result was deemed acceptable. More importantly for the future of the HLMS, these difficulties proved to be a disincentive to use of the system, both within the case study itself and in regards potential future use.

Another, related lesson, was that proper configuration of the system is essential. Whilst in some instances, such as the UEA case study, changes in the underlying software could not be anticipated, in most, proper time for configuration did exist. The difficulty was the lack of training and knowledge on site to deal with local configurations and the need to rely on ‘bug-fixing’ at a distance. Either more knowledge is required at each site, or future implementations might need to consider remote database installation and administration at the source of the system expertise.

Another issue was the extent of configuration required. Agora/VDX is a complex, highly integrated system and requires a great deal of configuration. Within a project , adequate resources to achieve such configuration are difficult to find therefore more time, if not resource, needs to be set aside for configuration within that context. In an implementation scenario, one would assume that the commercial provider would be able to provide more assistance than was available within the current project.

Conclusions

The case studies, though somewhat limited in time and scope, have provided valuable information about the way in which users have interacted with the Agora HLMS. It has given the project team a chance to step back and reflect on the progress made so far and has offered a sense of direction for the work still to be done to define what the community requires from a hybrid library system.

A number of conclusions can be drawn. Landscapes are a crucial part of the building blocks for the HLMS, however have proven difficult to explain to users. There is an underlying assumption that users will know and understand the language that is being used, the reality is quite different. This is a training and documentation issue, as well as a language issue - the terms that are to be used in any hybrid library system must have a broad acceptance across library and user communities.

Users approach their information requirements from a range of different perspectives, expectations and experiences and, the success of any future HLMS system will depend on it being made understandable across the range of user types and user experiences.

One way to manage this variety of users and expectations is to by setting up a range of pre-defined interfaces offering different iterations of the HLMS specific to the needs of different groups. This was successfully done in the case studies with the setting up of pre-defined landscapes. It should also be noted that Agora Release 2 offers more sophisticated landscape options, allowing for the creation of group landscapes for differing user groups (eg. in HE, faculties, different student groups, academic staff, tutorial groups). This will help to simplify the perceived complexity of landscapes by offering simple default landscapes for the more naïve user or the full power of the functionality to the more expert user.

There are no firm conclusions to be drawn from the use of CLD's in the Agora HLMS. However, there is evidence to indicate that CLDs need to be fuller and richer, providing more detailed information about each separate resource, and enhancing searching across CLDs for the purpose of resource discovery. It may also be possible to place service availability information in the CLDs.

Users were enthusiastic about the possibilities offered by cross-domain searching and were generally pleased with the resources available. However, users clearly identified additional resources that they would like to see incorporated in any future iteration of the hybrid library. Prominent amongst these resources were internet search engines and commercial booksellers. Development work by Fretwell Downing has now moved this forward so that non-z39.50 targets can be added including services such as Amazon and Alta Vista.

Ensuring that resources are high quality, reliable and the resources most needed, is crucial to the success of any HLMS, particularly as number of resources available increases. In regards the maintenance of resources, there needs to be a structure in place to ensure that the HLMS is aware of all pertinent changes to access details. This can be accomplished in a number of ways; all service providers could provide regular updates to any changes in their access details; the system could ‘poll’ supplier data; or there could be an agreed central repository of access data for consultation. UKOLN's Z-directory is an example of a centralised directory of Z39.50 targets in the UK and has links to similar initiatives in other countries. UKOLN plan to expose the configuration information contained in the directory using XML and the Explain-Lite DTD, which will allow Z39.50 brokers (and other clients) to configure themselves automatically. In the future, it is anticipated that there will be a DNER service description service, either bundled with or complementary to a DNER collection description service, that will provide a similar function across the DNER.[6]

The examination of interlending offered evidence that users are very amenable to the further integration of document delivery with the discovery and searching of resources. Indeed, most see interlending as merely another mode of document delivery to be chosen from a suite of options. Certainly, even the basic interlending functionality offered to users within Agora was felt to be superior to current systems. Interlending staff, however, added a note of caution, noting that the system as configured for the case study was incapable of supporting operational interlending despite its obvious potential. In short, the users approved but some ‘plumbing’ needs to be done to render the system operational.

In all, the case studies showed that the users of library systems have embraced the concept of the hybrid library and are ready for an operational HLMS. The problems seem to be, not so much rejection of the idea of the HLMS, but rather impatience that present systems, including Agora, cannot deliver the vision that the users themselves already possess. In this regard, the evidence of these case studies offers the library community both hope for the future of the HLMS and a challenge to meet.

References

[1] UKOLN - UK Office for Library & Information Networking: <http://www.ukoln.ac.uk/>

[2] The Agora system specification is available at http://hosted.ukoln.ac.uk/agora/documents/documents.html

[3] Library Associates were: University of East Anglia, University of Hull, Heriot-Watt University and Bath Spa University College.

[4] Dr E.V. Brack, David Palmer and Bridget Robinson "Collection Level Description - the RIDING and Agora Experience" D-Lib Magazine September 2000 Volume 6 Number 9 http://mirrored.ukoln.ac.uk/lis-journals/dlib/dlib/dlib/september00/09contents.html

[5] HILT (High Level Thesaurus Project ): http://hilt.cdlr.strath.ac.uk/

[6] Z-Directory http://www.ukoln.ac.uk/dlis/zdir/

Author Details