Web Magazine for Information Professionals

Search Engines Corner

Tracey Stanley reviews 'Northern Light', which offers features not available elsewhere.

<!DOCTYPE HTML PUBLIC “-//SoftQuad//DTD HoTMetaL PRO 4.0::19971010::extensions to HTML 4.0//EN” “hmpro4.dtd”>

New search engines appear all the time on the web. Usually there is little to choose between the tools available other than the largely cosmetic differences of interface design However, every so often a new search engine comes along which seems to promise to take us to the ‘next level’ of web searching in terms of capability and functionality.

Northern Light is the latest contender to emerge in the battle for your searching custom. Northern Light is a search engine with a couple of interesting extra features which set it apart from many of the other search engines currently available on the web. First of all Northern Light offers access to not only web pages but to a collection of on-line journal articles and other materials which would usually only be available by subscription. It is possible to search just the web, just the special collection or both when using the search engine.

Northern Light also offers an interesting feature which claims to improve the ease with which you can sort out useful and relevant information from a set of search results. Results are automatically sorted into custom search folders which are created dynamically according to the most frequently occurring subject areas in your search results.

In terms of ‘look and feel’ Northern Light is pretty similar to most other search engines currently available on the web. It has a simple search form on the home page [1] into which you type your search terms. There are no advanced searching options available at present, which is a bit disappointing as it makes it difficult to combine more than a couple of terms in a search, or to restrict a search by language or date. Apparently Northern Light are planning to introduce a ‘power search’ option in the near future, and this will include options for field searching, truncation and stemming and full Boolean searching. [2]

The searching language is very similar that currently used by other search engines such as Alta Vista or Hot Bot. Search terms can be combined by using the plus sign (i.e.: +holidays +Scotland to find documents containing both the words holidays AND Scotland). Boolean OR and NOT are actually supported (i.e.: holidays not Scotland to find information about holidays which are not in Scotland, holidays or Scotland to find information about either holidays or about Scotland). Phrase searching is also supported, using quotation marks (i.e.: “holidays in Scotland”).

Northern Light also uses relevancy ranking in order to sort your results and display them in relevancy order on screen. This again, is similar to other popular search engines available on the web.

The real difference between Northern Light and other search engines emerges when you see what it does with your search results, especially in its use of custom search folders.

Custom Search Folders

Northern Light builds your search results and categorises them into folders according to subject and type of resource which are created on the fly. This enables you to see at a glance the different sorts of results that your search has retrieved. These folders might include educational sites, commercial sites, personal pages and FAQ lists. This can be useful in assessing the likely quality, value and relevance of sites retrieved from a search at a glance.

Custom search folders appear on the left of the screen when your search results are displayed. In order to explore the value of custom search folders I performed an example search on the topic of Scotland. This returned over 288,000 documents, sorted into the following custom folders, according to category:

At a glance it is possible to see the sorts of sites which are being retrieved from this search. There are a number of documents from the Special Collection (more about which later on), plus other educational, personal and governmental sites. There is also a collection of pages from specific web hosts (e.g.: www.hmso.gov.uk) and some other miscellaneous topics (e.g.: Macbeth).

If I was interested in government information about Scotland I could take a look at the government folder. This folder opens up to display a number of subfolders covering related topics:

This is quite a useful way of ‘drilling down’ through a large number of search results in order to find relevant information. At each level I am informed how many documents have been retrieved in that category, and this can potentially save time in wading through hundreds of search results in order to find useful and relevant information.

However, I am a little unsure about the categorisation. How, for example, would I find information about tourism in Scotland using these folders? Sites about tourism might be scattered across a number of different folders - including government, personal pages, educational pages, Scottish history etc. Finding information on this topic would not be immediately obvious from using the custom folders, and I might find it necessary to do quite a bit of guessing in order to track this down.

If I instead perform a search on +Scotland +tourism I get more sensible results. This search retrieves 8,724 documents, sorted into the following folders:

This gives me a mixture of commercial, personal and tourist organisation web sites, as well as a separate category for Bed and Breakfast accommodation. As with all search engines, it does seem to help improve the value of your results if you can be fairly precise about your information needs.

I’m intrigued by the inclusion of business law in this list; looking in that folder reveals a number of sites offering business information searching which don’t seem to have any direct relevance to my search. I’d imagine that each of these sites contains an instance of the words Scotland and tourism, but in separate contexts.

In order to categorise information by subject, a hierarchy of over 200,000 subject terms has been put together by librarians working for Northern Light. This provides a standard vocabulary which is used in sorting the sites. Subject terms are applied automatically to web resources as they are indexed in the database, so human intervention is not involved at this point.

Documents have also been sorted by source - both by top level domains such as educational, government and country, and by individual hosts where a substantial number of pages from a single host have been found. The ‘personal page’ sites are identified by the presence of a tilde (~) in the URL, or by a name-like subdirectory.

It isn’t clear whether or not a record can appear in more that one folder if it is relevant to more than one category. For example, a document from the School of English at the University of Edinburgh might contain information about Macbeth. Would this document be categorised as an educational site, or a site about the subject Macbeth?

Proficient use of the custom folders, and competency in navigating around them effectively will probably take a bit of practice. However, it is worth spending some time becoming familiar with this potentially extremely useful feature as it helps enormously in tracking down useful resources from a mass of search results.

The Special Collection

This is a collection of over 3,400 magazines, journals, books, newspapers and newswires from a range of content providers. It is possible to search this collection at the same time as searching the web, or you can search it separately. The titles in this collection are mainly American trade publications and the collection is strong in business and finance publications such as banking . There are also some academic titles, including:

Abstracts from these articles are available for free; however, the full text articles themselves need to be purchased on-line before they can be viewed. Prices range from $1 to $4 per article and there is an alternative subscription option which enables you to purchase up to 50 articles a month for $4.95, from a selection of titles. This has to be paid in advance.

As a further enticement to buying articles on-line, customers have a guarantee that they can get their money refunded if an article which they have purchased doesn’t in the end meet their needs - even if it has been printed or downloaded. As long as a refund is requested within 30 days of purchase the customer account can be recredited. This is an interesting approach to document supply, and an additional incentive for first time users to try the service. The pricing for articles also seems to be pitched competitively, although I’m a little unsure as to how useful the sources will be for UK academic users as a large proportion of them seem to be targeted at American business users.

The service is, in fact being supported through sale of articles, rather than through advertising revenue. It remains to be seen how much revenue article sales will generate, and whether this will be enough to ensure that Northern Light still around in a year or so.

References

[1] Northern Light home page , http://www.northernlight.com
[2] Notess, G. Northern Light: New Search Engine for the Web and Full Text Articles, in Database 21 (1): 32-37, Feb-Mar. 1998.
 http://www.onlineinc.com/database/awards/award2.html

Author Details

Tracey Stanley
Networked Information Officer
University of Leeds Library, UK
Email: T.S.Stanley@leeds.ac.uk
Personal Web Page: http://www.leeds.ac.uk/ucs/people/TSStanley/TSStanley.htm