WebWatch: Surfing Historical UK University Web Sites

It has been said that those who ignore history, are condemned to repeat it. In the Web world we can be so excited by new developments that we may forget approaches we have taken in the past and fail to learn from our mistakes. This article describes how the WayBack Machine [1] was used to look at the history of UK University Web sites.

The Survey

The survey was carried out by entering the URL of the entry point for UK University Web sites, recording details of the availability of the Web site in the Internet Archive (including earliest and most recent dates and numbers of entries) and providing a link to enable readers of this article to obtain the most recent results.

The full survey findings are given in the Appendix.

Viewing The Past

A "Web Tour" has been developed which provides an automated view of the oldest pages. The display changes every 10 seconds. The Web tour is available at [2].

Discussion

Brief comments are provided on interesting findings in the Appendix. It was noted, for example, that in a number of cases images were not displayed. This will have been due to the site containing a robots.txt file which banned robots from accessing directories containing images for the site, such as the institution's logo, standard navigation images, etc. When the Standard For Robot Exclusion [3] was first released it was applicable for indexing robots such as the robot used by the AltaVista search engine. The consensus view was that it was advisable to ban robots from accessing directories containing images, as these contained no text to be indexed. We can now see that robots provide a wide range of functions which include archiving. There are advantages to be gained in allowing robots to access directories of images, not only so that the Internet Archive's robot can archive a Web site's images, but also to allow indexing robots which can provide a search facility for images (such as the Google image search [4]).

A number of Web sites contained "Best viewed in Netscape 2" style messages. The community probably now accepts that designing best sites for a specific browser is not advisable and it is probably sensible to avoid promoting a specific browser on an institution's entry point.

There appeared to be a number of instances of use of non-native Web technologies such as Java which required browser plugins on institutional home pages.

There were also a number of instances of use of frames or content provided by third parties, (such as usage counters), which were not fully functional in the archived Web site.

Conclusions

The Internet Archive can be a fun resource to access, especially for Web managers who have been developing Web sites for a number of years. However in addition to the embarrassment factor, ("did I really use the <BLINK> tag!"), there are also valuable lessons which can be learnt from our approaches to Web site development in the late 1990s. It can help us to reflect on the processes which lead to the chosen design; the technical decisions which where used and the way in which we sought to support the users of our Web sites.

This will, of course, be a continuous process. The decisions we are making today are the potential embarrassment of tomorrow. Perhaps the most importance message of this survey is the need to preserve access to to our digital past. The Internet Archive may provide one mechanism for preserving the past (provided, of course, that we allow robots to access our resources). Some organisations, however, may not feel comfortable in relying on a third party. In this case there will be a need to develop a digital preservation or records management approach which embraces institutional Web sites.

References

1. Wayback Machine, Internet Archive
   http://www.archive.org/
2. Display of UK University Entry Points From Internet Archive, UKOLN
   http://www.ukoln.ac.uk/web-focus/site-rolling-demos/universities-archive/
3. A Standard for Robot Exclusion, The Web Robot Pages,
   http://www.robotstxt.org/wc/norobots.html
4. Google Image Search, Google,
   http://images.google.com/

Appendix 1 - Survey

A summary of the findings is given in the following table.

The information in the table was initially collected between 20-23 December 2002

Author Details

Brian Kelly
UK Web Focus
UKOLN
University of Bath
Bath
BA2 7AY

Email: b.kelly@ukoln.ac.uk

Brian Kelly is UK Web Focus. He works for UKOLN, which is based at the University of Bath