Search Engines. Which is the Best?

So Much Information!
The internet has been so successful, that its growth has been almost exponential. The number of sites and pages has grown and grown like Topsy. According to Northernlight there are over 50 million pages on the internet. Imagine trying to index that lot!
Because of the explosion of data on the internet, it's would be surprising if all of the search engines could keep up with the supply of data and demand for information. The simple truth is that they can't! Some seem to be doing better than others.
The Search Engines and Directories are not charities, it costs money to provide the sort of computing power to run a search engine. They can only recoup this and make a profit from advertising. Some appear to be so income oriented that it's hard to find their "add-url" or "add-site" page. Neglecting to update their index might save costs in the short term, but must surely push their long term prospects into decline as users learn that they can find more current information elsewhere.

Index or Directory?
Before saying more about search engines, let's clear up one common misunderstanding. Search engines, or indexes, work by indexing web pages, allowing very flexible and powerful searches. Directories list whole sites by placing them into one or more categories.

The Best
Search Engines - Alta Vista and Northern Light
Directories - Yahoo

Our "best" recommendations come from using them to find information, and also noting how quickly or reliably they index our own sites. If you have your own website, and submit your sites to search engines, we believe that your experience will probably mirror ours.
Just over a year ago, we launched our first website, now we run six, with more planned. We recently made chart of the number of our sites listed in directories, and the number of pages listed from each site, on each of the major search engines. Our 6 sites are as follows, with their launch dates.

Site Name	Launched
24carat.co.uk	28.Nov.98
numis.co.uk	02.Feb.99
18carat.co.uk	03.Mar.99
southshore.co.uk	06.Aug.99
9carat.co.uk	09.Aug.99
goldsovereigns.co.uk	11.Oct.99

Results Table

Engine	Type	24 carat	numis	18 carat	south shore	9 carat	gold sovereigns	Rating	%
altavista	Index	387	21	70	75	-	32	585/1123	52%
excite	Index	65	8	20	1	-	-	94/1123	8%
google	Index	619	34	114	70	12	-	849/1123	76%
hotbot	Index	140	3	-	-	-	27	170/1123	15%
infoseek	Index	72	31	48	55	1	-	207/1123	18%
lycos uk	Index	581	33	-	-	-	-	614/1123	55%
lycos web	Index	-	-	-	-	-	-	0/1123	0%
magellan	Index	-	-	-	1	-	-	1/1123	0%
northern light	Index	691	39	128	75	158	8	1099/1123	98%
ukplus	Directory	1	1	-	-	-	-	2/6	33%
webcrawler	Index	21	-	-	-	-	-	21/1123	2%
yahoo	Directory	1	1	-	1	1	1	5/6	83%

Conclusions
You can see for yourself that Northernlight has got more of our pages than any other engine apart from the most recently launched. We are confident it will catch up, in fact we probably only submitted eight pages to it.
The only other index which approaches it for thoroughness is Altavista.
Five of our six sites are listed on Yahoo, and show up whether using yahoo.co.uk or yahoo.com
If an index does not contain all the data, then it cannot possibly provide the most information.
Why use any but the best? Search portals make their money by selling advertising to companies pitching for your custom, you have complete freedom over your choice of search facility. Poor quality complacent search engines don't deserve your custom, and don't deserve to survive.
Vote with your fingers!

We would be delighted to hear from Search Engines, Users, and Webmasters.

All comments welcome!