Web Magazine for Information Professionals

Search Engines Corner: Moving Up the Ranks

Tracey Stanley looks at how search engines rank their results.

Previous articles in this column have concentrated on how you can get the best out of search engines when trying to track down information on the web. However, it is also possible to use search engines as tools to promote your own web sites and pages to a wider audience; ensuring that others can effectively find their way to your information. A recent survey from the Georgia Institute of Technology claimed that some 86% of web users use find web sites using search engines as their main tool [1]. Furthermore, a large search engine such as Alta Vista can boast that it can attract some 19 million hits [2] to its web site every day. Obviously this means that search engines are a major resource which you can use to draw users to your web site on a global scale.

Most search engines use relevancy ranking to distinguish between the many thousands of sites which they retrieve in response to a set of keywords provided by the user. Relevancy ranking is important because it helps to provide some semblance of order over the millions of possible sites which can be retrieved. Thus, those sites which contain a high incidence of the specified keywords, or have those keywords in certain places within the document will receive a higher relevancy ranking than those which don't. Search engines list the sites they retrieve in terms of their relevancy, so it is important to get your site as high up the relevancy ratings as possible if you want it to get seen. As search engines typically display the results page by page, perhaps 10 hits at a time its important to get your site into that all-important first ten. Many users will tend perhaps to look at only the first page of results they get from a search engine, rather than scrolling through pages and pages of hits.

How Do Search Engines Rank Search Results?

Search engines use a wide range of methods in order to rank your search results. These are some of the most common devices:

Improving Your Relevancy Ranking

Unfortunately there is no standard set of criteria for relevancy ranking, so there is no way of ensuring you can get a high relevancy ranking in all of the major search engines. There are, however, some tricks which you can try in order to maximise your success in as many search engines as possible.

The Title Tag

The title tag is used by many search engines as an indexing tool. Alta Vista, for example, enables you to restrict a search to documents which contain your specified words in their titles. The reasoning behind this is that a document which contains your words in its title is more likely to be relevant than one which simply mentions your keyword somewhere towards the end of the last paragraph.

The title tag appears in the head of a html document and defines the information which appear in a web browser title bar at the top of a screen. It is structured as follows:

<head< <title>Put the title of your document in here</title> </head>

The tag is often ignored or overlooked by web authors. Quite often it is left blank, or a fairly generic title for a page is used; such as 'Index Page' or 'Home Page'. However, it can be a useful tool in helping to get your site adequately indexed by the search engines. Thus, if you have a description of your site in your title tag this can be utilised by the search engine, and aid people trying to find your site.

Think carefully about the title of your document: a title such as 'Peter's home page', for example, isn't going to be very meaningful when it gets indexed by a search engine. If your page is actually all about vegetarian cookery, nobody is ever going to find it unless you provide them with some sensible information to make this a bit easier. You should prepare a brief, descriptive title for your site; keep this short, and make sure that it conveys your topic effectively.

Meta Tags

Meta tags are used in the head of a document in order to convey some descriptive information about that document. This can include the author of the document, keywords associated with the document content, date of last update and description of the site in general.

Certain search engines will use meta tags, if they encounter them in a document, in order to index that document. Search engines which support meta tags include HotBot, Alta Vista, InfoSeek and Webcrawler. The most important meta tags for search engine indexing are the description and keyword tags. If a search engine can't find any other information it will simply index all of the words which appear in your document, and will use the first few words of your document as a short abstract. However, if you provide meta tags in your document search engines can then use these instead to index your document. If you make your meta tags as meaningful and comprehensive as possible this will aid people in finding your document.

The meta description tag can be used to provide your own description of your document - this is likely to be more useful and meaningful than the first few words of your document which search engines will otherwise use as an abstract. For example, if you have a document which covers hints and tips for web searching you could use the following type of description in the meta description tag:

<meta name="description" content="Hints and tips for using search engines.">

Your description should clearly describe your site in a single sentence if possible. You can also include the meta keyword tag to provide some keywords to be associated with your document, e.g.:

<meta name="keywords" content="search engines, hints, tips, help, examples, advice, web searching">

Alta Vista allows you to provide a description and keywords in the meta tag up to the length of 1,024 characters [3], and it will then use the description tag as an abstract for your document. It's worth sitting down and brainstorming a number of keywords which will accurately reflect the content of your site. You should include the basic concepts, but also expand on these to cover related information as well.

Some search engines - Hot Bot, for example - will actually weight documents higher for relevancy if they use meta description tags and the relevant keywords are found in those tags; so the meta tag can be extremely useful in improving your relevancy rating.

Meta tags have often received a bad press because of the practice of some web authors to use a technique known as keyword spamming (see my earlier article on this for more information [4]). However, because of this practice many search engines will now actually give you a lower relevancy ranking if they detect that keyword spamming is occuring on your pages! In some cases the offending site might even be removed from the database altogether.

As a result of keyword spamming some search engines are refusing to use meta tags at all for indexing purposes. Excite, for example, doesn't have any plans to introduce the meta tag for indexing of the documents in its database. Excite claim that there is too much evidence of spamming occuring to make it worth their, or their user's whiles to use meta tags for indexing and searching [5].

Organisation of Information on Your Web Page

The way that information is organised on you home page can also make a great different to the success that others have in finding your site. Some search engines will give a document a higher weight for relevance if specified keywords appear in the first paragraph of that document: the reasoning behind this is that a document is more likely to be relevant if the required words appear in the first paragraph than if they appear somewhere near the end of the document as a passing reference.

Hence, its often a good idea if you can organise your information to ensure that you cover, or at least mention, the key concepts contained in your document in your first paragraph. For example, a home page for a University Chemistry department might cover something similar to the following in its first paragraph:

 

Welcome to the Chemistry Department at the University of Poppleton.  
This page contains information about our undergraduate and postgraduate courses, 
funding and research opportunities, lists of our staff, our links with industry 
and resources for chemistry on the Internet.

Thus, anyone searching for information on Chemistry at Poppleton, for courses, funding, research opportunities, staff, links to industry or other chemistry resources on the web should be able to track down this page.

If you are using an image on your home page you should ensure that you use alt in your image tag to provide some text information about the image. Search engines will index this text; so if a large image appears at the top of your home page you should think carefully about the text information you associate with this image, for example, a picture of a chemical structure should perhaps include the name or type of the structure in the alt information provided.

You should try not to put information which isn't directly relevant to your main topics on your home page; so, for example, it wouldn't be helpful to quote a line of poetry at the top of your home page as this would be indexed by search engines.

Submitting Your Site to a Search Engine

You can submit your site to a search engine database manually simply by following the links on the home page of the search engine. Alternatively, you can use an automated submission service such as Submit It [6]. Submit It enables you to automatically add your web site to around 20 search engines.

A few pointers are worth bearing in mind when submitting a site to search engines:

References

[1] Graphic, Visualisation and Usability Centre, Georgia Institute of Technology. GVU's Seventh User Survey
http://www.gvu.gatech.edu/user_surveys/survey-1997-04/bulleted/use_bullets.html, April 1997.

[2] Alta Vista, About Alta Vista Search
http://www.altavista.digital.com/av/content/about_our_story_3.htm, 2 November 1997.

[3]Alta Vista, Alta Vista Search: Adding an URL http://www.altavista.digital.com/av/content/addurl_meta.htm, 2 November 1997.

[4] Stanley, T. Keyword Spamming: Cheat Your Way to the Top, in Ariadne, issue 10
http://www.ariadne.ac.uk/issue10/search-engines/, July 1997.

[5] Excite, Excite Help: Getting Listed
http://www.excite.com/Info/listing.html, 2 November 1997.

[6] Submit It Web Site
http://www.submit-it.com/ 2 November 1997.

[7] Mathsearch Search Engine
http://www.maths.usyd.edu.au:8000/MathSearch.html 2 November 1997.

[8] InfoSeek Web Site, InfoSeek: Results
http://www.infoseek.com/AddUrl?pg=DCaddurl.html, 2 November 1997.

Author Details

Tracey Stanley
Networked Information Officer
University of Leeds Library, UK
Email: T.S.Stanley@leeds.ac.uk
Personal Web Page: http://www.leeds.ac.uk/ucs/people/TSStanley/TSStanley.htm