Search engines
making your site visible

Getting visitors

Visitors will only find a site by word of mouth (which includes email, blogs and forums these days), through a search engine or adverts. Generally speaking it’s better to get a good number of the right visitors than lots of random visitors. You’ve probably read about pumping up search rankings and other ways of blagging visitors—here’s what I set out to do:

• make a good site (you can’t con people into visiting again if the site’s no good);
• rely on page content and search engine indexes to help people find it;
• avoid paying for search engine submissions and advertising if possible.

Search engines are useful but probably won’t be enough on their own.

Web site design

This site is simple and fast, the message is clear and it’s easy to find things. There are no pop-up windows, animations, award logos, software badges, affiliate buttons, banner ads or contacts from a link exchange. The sound clips are MP3 and don’t need a special player. You don’t need to register anywhere to use the site.

Some “professional” sites are impossible to use because of all the junk. User interface experts say a third of visitors don’t wait more than 8 seconds for the first page.

Web site links

A link exchange won’t help unless it’s a high value site targeted at your audience. You don’t want to waste your visitors’ time sending them to a low value list.

I told my best contacts when I launched the site and word of mouth was far more powerful than I expected. These visitors are more likely to be tuned-in to what you’re doing. The word of mouth network is definitely worth spending some time on.

Web site statistics

I recommend using a web statistics tool to tell you who’s visiting, where they’re from and what they’re looking at. This helps you keep the site relevant. Most web hosts offer one and there are several freeware statistics sites. (I’ve used Matrix Stats which cost about £10 a year and works well, Urchin which is very good, AWStats which is OK and Webalizer which is poor. Urchin is used by Google Analytics.) Stats will show:

• number of visits, including search engine crawlers;
• where referred visitors are from and server names;
• which pages are being used and which are most popular;
• entry pages where visitors have used links or bookmarks;
• internal page hits by visitors exploring the site;
• search words and phrases used by search engines;
• page errors and download errors;
• visitors’ browsers and operating systems.

Search engine tips

Search engines will only find a site if you tell them it’s there or if other indexed sites link to it. It’s worth remembering the blindingly obvious, that searching always uses the words that are there, not what you meant to say, and not pictures, diagrams or logos. Some directories ask for keywords and you’ll probably use them in your page titles and in HTML headers anyway. It took me a while to get this right.

Search-friendly HTML

There’s a lot of folklore about search engine friendly HTML techniques and contradictory advice on just about every angle. I designed the site for users rather than search engines and I don’t think I’m missing any magic tricks that would get better rankings. Here are some common sense steps.

• Use good TITLE text for all the pages.
• Use ALT tags on images (this also helps visually handicapped and text-only readers).
• Use good heading text (be as explicit as possible).
• Use headers (H1, H2, H3, etc.) for their meaning, not to make the text different sizes.
• Use headers in order within each page (e.g. so the relevant H3s are under an H2).
• Use good anchor text (search engines get meaning from links).
• Use good file names and don’t worry about delimiting the words.
• Don’t worry about META NAME tags (see below).

If trial searches for your own site fail, look at the HTML source for the top result pages and see if it gives any clues.

There are other ways to write your pages; this is just my best guess from the research I’ve done, and it works. I have many top page ranks and many top 10 page ranks on Google.

Meta name tags

Meta name tags are used by some search engines and not by others, you can safely forget about them. I have tried with and without—information in meta name tags is effectively ignored by all the big search engines.

Submitting this web site

None of the major search engines will index a site quickly unless you pay, and their free services vary widely. Things were changing rapidly as I went along and it’s not easy to make sense of the search engine network. In some cases it’s hard to work out how to submit a site and what the terms are. Some different nationality sites have different options, so it’s worth trying variations (e.g. excite.com and excite.co.uk). The trend is to charge for submissions but many are still free. I didn’t use a mass submission service; the free ones tend to be a front for junk mailers.

Free search engines I used

Search engines I didn’t use

Although I didn’t submit to Yahoo, Excite and Lycos, their crawlers visited and I get searches referred by them and many others (e.g. MSN and AOL).

Web site submission lead time

This is the sequence of events with Google and ODP from July to November 2002, as an illustration of submission lead time. I also had a number of hits by FAST WebCrawler and other spiders but it’s difficult to track the source and timing because their search engine network is complex (Lycos above).

My Google index was originally updated about once a calendar month: the crawler visited for a day or two and about 4-8 weeks later the index was updated. This meant the index could lag the site content by up to 3 months. These days updates are generally on Google within a week or so. Here are a few tips for checking your Google visibility.

• Search Google for your site and check all the pages are indexed.
• Review the words in your best phrases highlighted in Google cache pages.
• Look at the cached page on Google to see what date it was crawled.
• Subscribe to Google’s free web and site search.
• Read Google’s Webmaster page.
• Use a word frequency counter like SEO keyword analysis.

The most useful online guide I found about search engines, submission, etc. is www.selfpromotion.com Robert Woodhead’s site. It has good advice and coverage (although I’m sceptical about search ranking tricks) and provides some useful tools.

Summary of web site publicity

Here’s a summary of what worked best.

• Tell your best contacts and email people who might be interested.
• Choose half a dozen search engines including Google and submit your URL.
• Post links on relevant link pages and forums without spamming.
• Keep the site up to date, check it and add to it regularly.
• Check the search words people use and fine tune page titles and headings.
• Ask relevant sites for one-off links but don’t spam.

By the end of 2002 my hit rate was good enough and although I keep tweaking I don’t plan any major work on HTML or search engines. It’s now simply a matter of keeping the site up-to-date.

Here’s the kind of growth rate you can expect on a popular site without advertising. Slow but steady.

Getting a foothold on the web can be a slow and frustrating process but you only have to do it once. The web crawling strategy of Google has changed considerably since 2002 so your experience may be completely different but the basic principles are still the same.