Page 1 of 3: Introduction
How can people find your website? It’s quite simple, they have it bookmarked, or, they find it via a search engine. Lets be honest and say “How many people have my site bookmarked?” – If it isn’t many, you need to get listed in search engines.

What is a Search Engine?
Everybody knows that the World Wide Web is an enormous place. Nobody however, knows every page that makes up the World Wide Web. Enter the Search Engine – “A means for finding the World Wide Web you want to find”

The most popular Search Engine is arguably Google. Using Google is an absolute doddle; you visit the page, where you are slapped in the face by a search box, the logo and a few links at the bottom. Enter your search keywords and press enter/click submit – usually within 1 second you are presented with between 1 thousand and 1 million results.

Results will appear in order of their individual “page rank” within that search engine, which is the ranking of that page or site, according to the rules that google applies for ranking. The simplest way to look at this is to think of the most popular sites on the web as having the best ranking.

Fig 1, 2: Google.com and Google results

Interpretation of the results can be found here: http://www.google.com/help/interpret.html

Crawlers
There are two types of Search Engine, namely “crawlers” and “directories”. Crawlers may in some cases be referred to as bots or spiders but they will all refer to the same category of search engine. A Crawler is a robot which will relentlessly crawl the web and cache web pages or cache hyperlinks to web pages.

The crawler can index hundreds, thousands, perhaps millions of pages over a short period of time, this makes them useful for picking up the enormous amount of documents that exist on the web. The downside to a crawling bot is that it will index everything and there is no scope for checking the quality of the pages, the validity of the links and the relevancy of the page to its metadata and categories.

Common crawlers include MSN, AOL, Alltheweb and of course, the best of the lot, Google. ;-)

Directories
Directories are predominantly "human-powered". Similar to how you set your home PC’s folders and directories up, websites are categorised and grouped according to their page content and subject area. This ensures that the quality and relevancy of links within each category of a directory should be an example of what you are searching for. Directory listings are less achievable than crawler based listings, as an editor will usually moderate and approve the submission before it is added. This method of submission is more exclusive for this reason.

Common directories include DMOZ (which powers the Google directory, also known as ODP (Open directory project)), Looksmart and Yahoo.

How do I submit to a search engine?
Free submission
Submitting websites is (at the time of writing) mostly available free of charge. (For major search engines)

Google, AltaVista, DMOZ, Yahoo, Alltheweb and many others are all currently still available with a free submission service, although paid listings as a quick alternative is creeping into the higher profile search engines such as Google and AltaVista.

A free submission generally comes with a disclaimer, which will state that additions have no guaranteed date or time of inclusion and no guarantee of any listing at all.

The time between submitting to a free submission service and inclusion is approximately 6-10 weeks at present, but it could be anything up to 2-3 months, depending on a few factors.

Submission to crawlers like Google may not always be necessary. If your site has a lot of reciprocal links on the Internet already there is a very good chance it’s already indexed. The more links you have, the higher the possibility that this has already happened.

To find out if Google already has you indexed, enter your url into the search box. E.g. www.something.com – If you get a returned match of your site then you are indexed! Try searching for: “site: www.something.com <keyword>” and see what pages you can discover.

If you are just starting out with a site and don’t have many reciprocal links elsewhere, you will need to submit. In both cases, its most likely that you will need to submit to directories such as DMOZ and Yahoo manually as these don’t tend to happen automatically.

Paid submission
Paid Submission is springing up all over the Internet at the time of writing (June 2002) and most Search Engines are pushing quite hard on it. Faster submission, better frequency of re-indexing and various other appealing factors have encouraged business and individuals seeking a far quicker method of getting noticed and found via Internet search queries.

Yahoo, Google, AltaVista, Overture, WebWurld, FAST Include and Inktomi are all popular methods of getting listed and for some purchases, submission and regular re-indexing every 48 hours occurs on many different search engines, reducing the need to manually submit regularly to a variety of websites.

Costs vary according to the services requested and the level of exposure that is desired. Yahoo for example offers a non-refundable $299 USD charge for "within 7 days" submission to the Yahoo directory, one of the most expensive options, but this would increase traffic and “findability” on a decent scale due to the enormous usage that Yahoo has attained. In the United Kingdom, WebWurld offer some paid submission services to Inktomi, FAST and Looksmart, the first of which includes some of the major web crawlers (MSN, Hotbot, AOL, BtOpenWorld and more) for approximately £25 a year. FAST includes a few directories into the mix as well as the aforementioned crawlers and thus, demands a higher cost. Looksmart, which is a directory based engine which fronts many sites, is similar to FAST but at a higher cost still. (http://www.webwurld.com)

Tracking
Tracking visitors is an important aspect of any website, not least to get a reasonable idea of who is visiting your site, when they visit and what they look at. Search word reports and referring URL reports are two useful ways to discover this information.

Search word reports will report which search terms, if any, were used when a visitor “visited” your page via a search engine result.

Referring URL is the term applied to the Website Address which has the link to your page within it. This is an example of a referring URL, from google:
http://www.google.com/search?q=%22iframe%22+and+%22tutorial%22&hl=en&lr=&start=10&sa=N

The URL above has been generated by the Google search engine, the query string (the URL text after the question mark) contains the parameters passed into Google, which in this case are: "iframe" and "tutorial".

Tracking software and web statistics analysers are available in abundance and best of all, most are free, although many offer a paid service which offers more specialist features and some analysis tools. The services provided by these however only indicate the "top level" of traffic, if you want to dig deeper into your stats and produce some very complex, detailed reports, you need a program like sawmill, which you can use to "mine" your logfile and pick out and cross reference everything you require.

Online free services generally require a piece of script and a small image to be placed somewhere on your page. (Invisible counters are usually a feature of paid services) Depending on the complexity of the script will depend on what is logged, but generally, a JavaScript can log the important information such as domain, IP address, browser used, operating system, referring URL, pages viewed and time spent browsing. (Amongst other information)

2 good examples of these, are site meter (http://www.sitemeter.com) and stats4all (http://www.stats4all.com) – Very little time is needed to subscribe and set one up, a piece of code is required by both on all pages that you want visitors to be logged on, plus a small image displayed on your page which will advertise the service and in most cases provide a link to the providers website.

---

Go to: Next page >>> | Home | Top