Web Magazine for Information Professionals

CLUMPS in the Real World

Juliet Dye and Jane Harrington look into evaluation procedures for the CLUMPS projects.

In attempting to establish the requirements of users in the context of the four e-Lib Clump Projects we decided to take a two-pronged approach. Firstly we tried to look at some typical user profiles and see what their search requirements were likely to be. Then we related this to what we found when we did some actual searches. This article will provide the reader with some pointers to the relative merits or otherwise of the projects. Some similar points emerge from both stages.

What do people want?

The first question that should be asked is what is it that users might actually want from clump services? Probably any search undertaken is likely to be for one of two things: -

What, then are the questions that the user is going to be asking if the search is successful and throws up some possible hits?

Who is a typical user?

The next thing we tried to identify was whether there was any such thing as a "typical user". Trying not to be influenced unduly by our academic bias we identified three levels of user who might want to access clumps.

  1. The student who may or may not live or work close to their place of study, so is looking for somewhere more convenient to use. Alternatively their own institution may not have sufficient subject coverage in their chosen area so they need to find somewhere else which will provide them with a suitable range of material
  2. The researcher/academic who may have more specific demands and may need to undertake a quite extensive literature search, or they could be looking for hard-to-find material.
  3. The wider public who may have a work or study related need or could be searching for material purely for personal use.

Skills Profile

It goes almost without saying that these different categories of users will have very different skills when interrogating catalogues or databases. They might be very experienced searcher who can define their terms precisely and need little or no help, or they may have very limited experience beyond using their own OPAC. In the latter case they will either require substantial assistance or screens will need to be designed for this level of naivety.

What do Users Want?

Whatever their skills profile however, it is likely that they will actually want very similar things from any single search of a shared library resource. Readers should forgive us for using some poetic license when coining a new phrase in defining what any clump project is about when we use the term "One Stop Search".

Search Results

Once the search has been made it should be possible to refine it down if there are too many hits, probably by language, date or medium at the very least. The search results should be able to be sorted by a range of criteria with location probably being the default as this is likely to be the most popular way of listing material. However other appropriate ways should be offered such as chronologically or alphabetically by author or title.

The user should be offered a brief list of hits in the first instance, which they can then expand to a full record where desired. At this point the location becomes particularly critical, especially for split-site institutions or organizations.

Lastly the user will then need to be able to access current holdings information. If they do want to look at material or borrow it they must know whether that item is available.

After the Search

Once the search has been completed where then does the user go from there? As access is likely to be the key question it is essential that the user is easily directed to the relevant information they are likely to require. Ideally there should be a link directly from the search screen into the relevant library home page or summary screen. That next screen should provide answers to the following questions for the user:-

These last two points highlight areas that service providers will need to address in terms of what exactly the role of the librarian is in this process. To what extent is it the librarian or the interface which is the intermediary? Also what will be the charging mechanism if there is a document supply scheme?

The Future

The last aspect to consider is where the projects go from here. The term "inter-clumping" is not an attractive one but the opportunities it could provide certainly would be. The future presumably will bring the possibility of searching across a range of clumps or the possibility of moving from one to the next until search terms are satisfied.

The Actual Experience of Searching

Our evaluation of the Clumps did not attempt to be comprehensive or rigorous. Time was fairly tight and in any case our brief was simply to sample these resources and present our initial reactions as non-technical librarians. We could not pretend to be naïve users. But our daily contact with such people, combined with our own lack of familiarity with the Clumps, meant that we could make a good stab at predicting possible problems.

Some observations could be made without actually doing any searches at all. However we obviously needed a common thread and we chose to look for material on ‘Japanese management’. We chose this because it was fairly specific without being overly narrow. It is also a topic which could be the subject of all kinds of student project, or could be an enquiry from a member of the public about to be involved with a company run in this way. We didn’t aim to judge the absolute quality of the information we found – the Clumps are (or were) in an early stage of development, after all. Instead we tried to note what happened and what we thought of this.

We were very conscious of the need to note the good points as well as the bad (and therefore very relieved to find both). Although it became clear that we did raise matters which the Clump project managers had not thought about or noticed, it was also inevitable that some criticisms would pinpoint problems that they were only too well aware of. In order to keep the mood of the session upbeat we started with the ‘discouraging’ things we had found and ended with the ‘encouraging’ features that we had liked. These have been amalgamated in the section below however. We did not name particular Clumps in the evaluation and although it was sometimes possible for the informed members of the audience to identify them, there were several points which applied to more than one anyway. The checklist of headings which emerged was as described below.

Screen Puzzlement

Did the opening screen make us feel anxious or irritated? Was there too much background information, making it look confusing? Was it immediately obvious where to enter your query?

Sometimes it wasn’t clear exactly what choices we were being asked to make. One might pick something that looked promising, such as ‘main web site’ and then find out that it was not an appropriate choice and have to try again. Do I want ‘gateway’ or ‘research collections online’? How are they different? Sometimes a choice would quickly return you to the opening screen in a kind of loop. This irritates users with some knowledge. Those with very little knowledge would probably be either worried or just terminally dismissive.

Looking at a lot of different screens makes it very obvious that busy screens are distracting and moving screens induce nausea. Lots of white space plus blues, whites and yellows seem very easy on the eye.

A feature we personally liked was the combination of a simple map and hot links to give an immediate physical orientation. Perfect geographical fixing may be unnecessary however?

Terminology

Not everyone immediately recognises the professional terminology relating to client server architecture. We thought that some users might not instantly know what a ‘target’ means for instance. To the lay person this might be a library, a subject, a collection or a server. It seems perfectly proper to be led into using structured language where this is necessary and effective, but it needs to be explained. Another example was the term ‘collection description’, which might have been a description of all the libraries being searched but turned out to be a list of special collections. Similarly ‘gateway’ can be an ambiguous sort of word.

Conversely there were some good examples of natural language where the user is asked to fill in the blanks in a sensible sentence e.g. ‘ Display each record in [] notation and sort by []’

Managing the Customer

There is a central tension between getting the user to refine the search so that it is efficient and allowing him or her to search widely. It’s important to make it very clear what the choices are. If the default search is going to be all locations, this information should be up front and obvious so that the customer can actively decide whether to choose a quick foray or risk a longer wait for an exhaustive investigation.

To the user ‘search’ is usually a more important concept than ‘target’. The provider may want to reverse that weighting in the interests of orderly searching. We suggest that it may be psychologically important to have both on the same page.

Truncation and the use of a wild card character was sometimes specifically encouraged and sometimes not. It’s not a technique that occurs naturally to many users, but it’s a useful one and we felt that on balance it was worth highlighting the possibility.

A prominent ‘reset’ button, which allowed you to wipe out everything and start again was very welcome. Repetition of important instructions at top and bottom of the screen was also helpful.

Online help was at different stages of development. We saw some very clear pages such as ‘How to use this service’ but also some vacuums, for example in help with collecting together your results from various searches. Often there was just a message saying that this was still to be written, which is understandable. The problems of online help are not peculiar to Clumps. You want enough on the screen to give you the confidence that you know how to proceed without endlessly clicking back and forth. But you don’t want to be held up by having to read whole paragraphs of explanation, or having to do a lot of scrolling down. Some compromise has to be reached using buttons or hot links for more detail. The less cluttered the screen the less off-putting it was to use.

The Search Experience

Inevitably we had some confusing experiences. These could have been due to our misunderstandings; to known snags which would be addressed at some point; to shortcomings or variance in the basic indexing of the source data; to problems with the underpinning systems; or to limitations in the records which were available at the time. Only the experts would know.

An example of such confusion involved a subject search (on Japanese management) which turned up records at one member library only. After examining those records we chose a particular book called ‘Japanese multinationals’. An author-title search on that book revealed that in fact it was also held by a second clump library.

We assumed that the user might then conclude that a subject search on ‘Japanese multinationals’ could be worthwhile and we did one, plus another on ‘Japan’ and ‘multinationals’. The result was zero records, even though we knew there was something in there The actual explanation could have been that some sites’ Z-servers are not yet working when it comes to subject searches. It would not inspire confidence if it turned out to be a routine occurrence however.

Error messages are disconcerting at any time but especially when they don’t really explain the problem e.g. ‘JavaScript error’ or ‘combination of specified databases not supported’.

It was very reassuring when it was clear that something was happening - the reporting of how a search was going, how many targets had been processed, which servers were down and so on. Simple terms like ‘searching; presenting; connecting; awaiting response; no hits; complete,’ all make users feel that they have an overview and are in control.

Dealing with Results

A major drawback, we felt, was the lack of information about availability of a particular item. We understood why this is difficult, but anyone doing a search for anything other than bibliographic reasons would be likely to be interested in exactly where something is and exactly how available it is.

Interloan request as an immediate hot link is excellent from a reader’s point of view. Obviously great care will be needed in clearly explaining any limitations of access to such a service and any associated charges that are being incurred.

Collecting together one’s results and doing something with them involves several possibilities and we had a variety of experience. Ways of rearranging and sorting your display in various ways were always provided, but guidance on how to do this wasn’t always forthcoming. At best you were prompted, at worst you were left to figure it out.

Clear signposting on how to view your search history is helpful. We didn’t have time to investigate what could be done with these, but double clicking to re-run it or to examine it again would be a predictable preference. Clear invitations to delete or re-use might be helpful. Another development might be the facility to transport a search to another clump altogether.

Exporting the results caused a few problems. Copy and paste into Word worked by and large, albeit with some strange characters and formats sometimes emerging. Some on-screen help about this might be welcome. We tried downloading some results to disk and got all the html along with them.

Our overriding impression was that the Clumps could be an exciting and accessible tool, which mirrors the way our users are thinking. In our experience many of them want to search out resources independently for themselves, even though they also want guidance on how to do it. They can be also be surprisingly willing to travel about in search of what they feel they need. At our own university, the guide to libraries in other institutions is one of the more popular ones with students and access to other online catalogues is a common query. The Clumps project could have a major impact on this sort of activity.

Author Details:

Juliet Dye
Deputy Manager Harrow IRS Centre
University of Westminster
E-mail j.n.dye@westminster.ac.uk


Jane Harrington
Library Manager Marylebone Campus
University of Westminster
Email: j.harrington@westminster.ac.uk