Search Engines: FAST, the Biggest and Best Yet?

FAST - new but better?

FAST - Fast Search and Transfer ASA was established on July 16th 1997 and the search engine (1) arose out of a project initiated at the Norwegian Institute of Technology, Trondheim. Their system is powered by Dell Systems and they have an impressive grouping of partners, such as 3Com, CompuServe, Corbis, Lycos, and almost inevitably, Microsoft and Sun. The aim of FAST is to index the entire web by the end of this year; they expect to be able to index one billion documents and beyond. Their byline is ‘All The Web, All The Time TM’.

The FAST database currently holds in the region of 200,000,000 webpages, which compares very favourably with AltaVista (2) (150,000,000) and Northern Light (3) (160,000,000), although if we accept figures of 800,000,000 webpages currently available, they still have a long way to go in their quest to index all of them. However, be that as it may if you’re looking for a single search engine and you require comprehensiveness in your searching, it is certainly worth taking a look at.

At first glance the interface is plain, simple and uncluttered - it looks like all other new search engine interfaces before they start to get involved with portals, free email accounts and links to commercial offerings such as Amazon. It certainly makes a refreshing change from the overly busy (some might say messy) opening pages that other search engines have. There is the usual ‘search for’ box, with a pull-down menu offering to search for any word, all words or a phrase search. It’s at this point that I began to have one or two misgivings about the ability of the search engine, so I immediately headed over to the ‘Help’ option. It is a shame to have to report that I might as well have not bothered - it consists of information that anyone who has ever used a search engine will be familiar with - double quotes for a phrase search, + sign to include and - sign to exclude, with a couple of examples. In this it performs badly in comparision with other search engines which do provide valuable and informative help screens. This is a theme that I’ll be returning to in this article - simply because a search engine has a large database doesn’t mean that it’s going to be a good and effective way of searching the Internet.

Returning to the search screen I was intrigued to note a link through to the FA Premier Division website (4). I have no idea why it’s there, and I couldn’t find out, but it’s there! Returning to the search box, I ran a quick search and have to say that I was impressed with the speed; the results were returned faster than any other search engine I’ve ever used. FAST claims that it can search through all 200,000,000 web pages in less than a second and on the evidence of the searches that I ran I’m not going to contradict them! They also say that they can rebuild their document index in 12 hours, leading to fresher results than search engines that take days or in some cases weeks to do the same thing.

Let’s now turn to look at the results that were returned, and compare them with those of the other two heavyweights, AltaVista and Northern Light. I did a number of quick searches and have included the results in the table below. All the searches were conducted at the same time and since FAST does not deferentiate between cases, they were all done as lower case searches.

The results generally do suggest that FAST is more comprehensive than its two main rivals, at least in the area of database size, though it did fare badly with the dyslexia search. However, it doesn’t appear to be as far ahead of its rivals as it perhaps might like to be seen, and I found the ‘ariadne’ example particularly telling! In case you’re wondering about the last search that I did, for my sins I have a namesake in the US, who was a gay porn star, so after running the first search I ran a second to try and eliminate some of those references. I then checked them to see how many of the first ten returned pages were mine or about me. This isn’t really a fair test, since I didn’t give the engines any more to work with than that, but once again FAST came out on top.

At this point I could of course stop and summarise by saying that FAST is comprehensive, and as its name suggests, that’s exactly what it is. Relevance seems to work well, with an emphasis on finding words in the title of the web pages, rather than (for example) on links, repetition of key words and so on. If you wanted to introduce FAST to your novice users who wanted to do quick and dirty searches you could do a lot worse. However, how does the search engine stack up against the others when we begin to delve a little bit further?

The answer has to be ‘not very well’, I’m afraid to say. The only search options that you get are the ones that I’ve already mentioned. I looked and tried in vain for truncation, searching using upper case, searching in titles, URL and host. Admittedly FAST do have options to search for FTP and MP3 files, (which actually links through to the Lycos search engine), but it does not come close to offering the same facilities as AltaVista for example, which provides much greater variety in terms of images and video clips for example. This is perhaps a slightly unfair criticism, but no-one ever said that the Internet was a fair place; in order to compete search engines have to offer more than the ability to search web pages. Both AltaVista and Northern Light have been in the business for much longer than FAST, and they have been capitalising on that by adding all the extra facilities. Both have their own unique selling points - Northern Light with its special collection and custom created folders provides access to data otherwise difficult to get hold of, and AltaVista has very sophisticated search options. FAST has a large database, and returns results with blistering speed, but is this enough in the short term to challenge for the top spot? The lack of extra features is a real drawback to its use, and although I may well use it in addition to my current favourites, it isn’t going to displace them. In the longer term however, if these issues are addressed, and it can continue to index a larger database of web pages then it will be difficult to ignore it.

References

1. FAST
2. AltaVista
3. Northern Light
4. FA Premier League

Author Details

Phil Bradley
Internet Consultant
Email: philb@philb.com
Web site: http://www.philb.com