Web Watch: A Survey of Links to UK University Web Sites

Background

On 10 February Phillip Simons sent the following query to the web-support Mailbase list: “Can anyone tell me if there is any way of detecting who is linking to a particular URL? We want to see who still has our old URL on their links pages.” [1]. The replies suggested a couple of approaches: looking at referer (sic) fields in server log files and using the link feature provided by a number of search engines to report on pages containing a link to a resource or web site.

A number of services are available (including LinkPopularity.com [2] and LinkCount [3]) which provide an interface to the “back-link feature of a number of search engines. Why have such services been set up? Why are web managers and information providers interested in details of links to their sites? There are a number of reasons. The LinkPopularity.com site makes the following points:

Need to find additional sites that might link to you?

Run a Link Popularity report for competitors’ sites and contact sites that link to them.

Thinking about buying a product on the Internet?

Run a Link Popularity report to see what other sites say about it.

Links to your web site not only help to generate traffic. Due to recent developments in search engine technologies they can generate additional search engine traffic to your site - search engines such as Google make use of citation analysis to rank hits, so that a resource which has many links to it will be given a higher ranking that one with few links.

Since the number of links to web site resources improves access growth in the numbers of links may be a useful performance indicator for your web service. A comparison with the numbers of links to one’s competitors may also prove interesting. And as Roddy McLeod, manager of the EEVL subbject gateway said on the lis-elib list last November [4]:

On 4 Nov, Brian Kelly helpfully suggested using info on the number of sites linking to a service as the basis for impressing potential advertisers.

I tried this, pointing out to a potential advertiser that EEVL had, according to HotBot, 1099 sites linking to it, whilst there were only 18 sites linking to their site, and suggested that what they needed was more exposure.

It seems to have worked, as they have agreed to buy an ad on the soon to be released new design EEVL site. For how much?…well, its not a lot of money, not even enough to buy a plane ticket for the BARD to return from his Himalayan retreat where he is currently “resting”, but its a start. It also helps that the site in question is the sort of information service which might just fit in to any future perfect portal (or whatever they are being called today).

The Tools Availalable

Two popular web services which can be used to report on link popularity are LinkPopularity.com and LinkCount.

The output from LinkPopularity.com is shown in Figure 1. As can be seen it gets its results from the AltaVista, Infoseek and Hotbot search engines.

Figure 1: Results from LinkPopularity.com

The output from LinkCount is shown in Figure 2. It gets its results from the AltaVista search engine.

Figure 2: Results from LinkCount

As can be seen LinkCount enables the results from two sites to be compared.

Caveats

Although a number of popular search engines can be used to report on links to a web site a number of caveats should be noted:

• Only links from conventional static web pages will normally be recorded. Links held in databases will not be recorded.
• Link information held on resources which are not harvested by search engines will not be recorded. This could include password-protected resources, Intranets, bookmark files and pages which are protected by a robots.txt file.

In addition to the limitations on the information stored by the search engine vendors, it should also be noted that there are several issues associated with the reporting process:

• Should internal links be included in a survey (e.g. links from www.foo.ac.uk to other resources on www.foo.ac.uk)?
• Should links to subdomains be included in a survey (e.g. links from www.bar.foo.ac.uk to resources on www.foo.ac.uk, links from www.foo.ac.uk to resources on www.bar.foo.ac.uk, etc.)?
• Should links to alternate versions of a domain name be included (e.g. www.bham.ac.uk and www.birmingham.ac.uk)

Conclusions

The results published in the Appendix provide a useful snapshot of the number of links to institutional web services. The survey has helped to identify a number of deficiencies in the use of the AltaVista and InfoSeek search engines for monitoring back links, namely:

• The incomplete coverage of the search engines.
• The volatility of the results, especially in the case of AltaVista.
• There is a requirement for link information to be provided in reusable format for post-processing.

Perhaps the best advice is to treat link popularity information with the same healthy scepticism that you would treat web statistics.

References

1. web-support archive, Posting to Mailbase list, 10 Feb 2000
   http://www.mailbase.ac.uk/lists/web-support/2000-02/0113.html/
2. LinkPopularity
   http://www.linkpopularity.com/
3. LinkCount
   http://linkcount.mycomputer.com/
4. lis-elib archive, Posting to Mailbase list, 10 Nov 1999
   http://www.mailbase.ac.uk/lists/lis-elib/1999-11/0015.html

Appendix 1: Survey Results

The AltaVista and InfoSeek search engines were used to count the number of links to UK University web sites. The survey was carried out between 14-18th February 2000.

The following table contains details of the number of links to UK University web sites. Column 1 gives the name of the institution and a link to the institutional web server. Column 2 gives the number of links to the main web server (e.g. www.foo.ac.uk), excluding links from the institution’s own domain (e.g. excluding links from foo.ac.uk). Column 2 gives the number of links to the all web servers in the institutions domain (e.g. *.foo.ac.uk), excluding links from the institution’s own domain (e.g. excluding links from foo.ac.uk).

Note that in the survey the syntax of the search string used was: link:www.foo.ac.uk -host:foo.ac.uk (numbers of links to central web server) and link:foo.ac.uk -host:foo.ac.uk (numbers of links to all web servers).

Note the following limitations in the survey:

Domain Name Variants

The survey will not detect links from unusual domain names used within an institution: e.g. a link from www.maths-lse.ac.uk will be included in the numbers of links to www.lse.ac.uk.

Short and Link Domain Names

Institutions which make use of a long and short form of their domain name (e.g. www.bham.ac.uk and www.birmingham.ac.uk) will have misleading results.

IP Numbers

Links from sites within an institution which have an IP number rather than a domain name will be included in the lists of links.

InfoSeek Limitations

The InfoSeek search engine appears to report on pages from other servers in a domain when this has been excluded.

Table 2 gives a summary of the top 5 web sites with the most links to the central web server as reported by AltaVista.

Table 3 gives a summary of the top 5 web sites with the most links to all servers within the domain as reported by AltaVista.

Author Details

Brian Kelly
UK Web Focus
UKOLN
University of Bath
Bath
BA2 7AY

Email: b.kelly@ukoln.ac.uk

Brian Kelly is UK Web Focus. He works for UKOLN, which is based at the University of Bath