MIDAS: Manchester Information, Datasets and Associated Services

anne mccombe

MIDAS: Manchester Information, Datasets and Associated Services

Anne McCombe describes a service that provides a wide range of datasets to the wider communities.

MIDAS [1], based at Manchester Computing, University of Manchester, is a National Dataset Service funded by JISC [2], ESRC [3] and the University of Manchester [4], to provide UK academics with online access to large strategic and research datasets, software packages, training and large-scale computing resources. The datasets are supplied by arrangement with CHEST [5] and The Data Archive [6].

Anne McCombe was appointed on October 1996 to promote awareness and use of MIDAS in the academic community.

After a brief history of MIDAS, I want to talk about the WWW made us think again about how we publicize, document and deliver the service and what we have done to exploit the growing use of the Web, and what the Web has done to help us attain the unattainable.

History

In May 1993, Manchester Computing Centre was designated one of the first National Datasets services, and capital funding to replace the national scalar service on Amdahl (running VM/CMS - the common user interface of the day) with a UNIX-based machine was provided. At that time, MCC already had a substantial holding of on-line datasets including the 1981 and 1991 Censuses of Population and several large government datasets. In 1991 Manchester had inherited some of the national scalar computing service users that had been using the national service on the Amdahl at the University of London Computer Centre, and the London Office of Manchester Computing was opened with two member of staff to provide local support to users in the London Institutions.

It was named MIDAS (Manchester Information Datasets and Associated Services) a couple of months later to reflect the new focus of the service.

The Cray CS6400 Superserver was installed in February 1994. Manchester Computing had its first Web Server in December 1993, and the first documents to be put up were the National Service Newsletters [7]. Comprehensive on-line documentation was provided via the Gopher.

Web Service gets going

By March 1995 we had a 'rapidly developing' WWW server for the MIDAS service. It contained general information about the service, pointers to the appropriate sections of the MIDAS gopher for detailed information about each service, registration information and links to other related WWW servers. Work started on mounting the CSS Notes (documentation on MIDAS services) in PostScript and WordPerfect 5.1 and registration forms in PostScript. Information on Courses, the midas@mailbase.ac.uk list [8] and how to login via telnet, support arrangements - all these were reformatted for browsing with a graphical interface browser such as Mosaic or Netscape. Both forms of on-line information were maintained.

For documentation - a gradual transition

The primary on-line source for detailed documentation on how to use most MIDAS services has been textual and in some cases remains so (but not for long, see below), and the information is accessible directly from a graphical Web browser which displays the information in the same way as would the Gopher or Lynx.

The most popular MIDAS documentation (SAS, SPSS, NQS and the Current Documentation list) was translated into HTML format. We are looking at how we can best serve our users' documentation requests to have extensive Web-based documentation and still be able to provide paper copies to users on request.

The more appropriate medium for informing users of new developments was fast becoming graphical, and if information was "on the Web", it was expected to be in HTML format, have pictures, provide hyperlinks to related information. A WAIS searchable catalogue of datasets and software [9] was set up at NISS and a guide to statistical software on the Web [10] were set up.

MIDAS already had an extensive amount of Census help and information on its gopher server. Census-related material appeared elsewhere in the World Wide Web, and graphical images such as maps and graphs illuminated the information; Web interfaces for some of the software packages for processing Census data were being developed. To draw this information together, the MIDAS Census Gateway [11] was set up. The Web provided us with the mechanism for giving access to Census users in t he UK to related research in the UK and all over the world.

Soon we shall see this gateway develop a Visualization Gateway to the 1991 Census: a new project which takes deliverables from two NTI projects, KINDS [12] (set up in 1994 to provide an intuitive interface to the spatial data) and ARGUS [13] (developing techniques to allow the visual exploration of spatial data), to create an intelligent, highly interactive, on-line mapping and visualization interface to the 1991 Census data and associated boundary data held on MIDAS.

In recent years the spatial data resource [14] on MIDAS has grown considerably, with the acquisition of global and national digital map datasets from Bartholomew and the digital census boundaries for the UK. The satellite data archive was acquired in 1995. To offer potential users an introduction to the SPOT data and to provide a quick reference for registered users, we built an interactive index of SPOT images [15] containing 'quick- looks' of each scene along with various metadata. A similar site for Landsat is under development. We hope that this resource will attract users from many branches of research, not just from mainstream geographical research and remote sensing. We must be able to show how it can be used, and to some extent the KINDS project can do this.

More information than we provide is sometimes offered by the suppliers of the data, and as the newer Web interfaces are written we include links to these other resources.

Many users of MIDAS users are from the social sciences. It is the data that's important and as far as possible we want to make the data easy to identify and simple to extract and/or process. A Web interface was built for the CSO (now Office for National Statistics [16]) macro- economic time series databank and the Ingres database, but we had to ensure that the data was only available to authorized users, and we had to wait a while for Web password security to be established before releasing the interface. The interface has been greeted enthusiastically by users and it has the potential to significantly widen access to this important teaching and research dataset. The ONS Web interface [17] was demonstrated at the MIDAS User Forum in July 1996.

What did our users think?

In the summer of 1996 we conducted a electronic survey of 400 of our MIDAS users. Response was low but nevertheless informative. Telnet was the most popular form of access (75% of responses), but we expect the use of X (currently 16.2%) to increase in the future. 86% considered themselves to be only novice or intermediate users of UNIX . Virtually all respondents had Web access which they preferred to the gopher. We presented the results to the MIDAS User Forum and many people indicated they would be happy to see information migrating to the Web, but it was pointed out that some students did not have Web access and that the gopher was quicker to use.

This situation is likely to change. Increasingly access to the Web will be provided and there will be the expectation that if the information is worth having it will be on the Web. We asked the MIDAS User Group how they would feel about us transferring all our on-line documentation to HTML (yes, yes, yes) and to consider the implications of transferring the support from text to the HTML.

The User Group agreed that we should transfer our support to the Web interface, and we hope that by July of this year we shall have all our on-line documentation in the new format. The old format will remain but decline in its 'up-to-dateness'.

Where to next

MIDAS is developing, so is the Web, so too are our users. The culture of sharing resources is growing, and, as was brought to my attention last week at the Arts and Humanities Data Service [18] launch, it is not just the resources that we should share, but the method of extracting and processing the data.

We know the sort of questions we want answers to. How many people eat fish one or more times a week in this region. Show me the streets where more than half the occupants have central heating. Let me ask my next question. Gradually we are getting there.