Indexing



Search engines are essentially massive databases of domains and sub-domains for sites. When a person searches for something, the search engine instantaneously searches the domain names to come up with relevant listings. The method for doing this differs by engine and involves factors beyond the scope of this article. Instead, we are going to focus on figuring out if your site is in these important databases.

If you want to be included in the rankings on a search engine, your domain name has to be in the search engine database. Put another way, you have to buy a ticket if you want to win the lotto. Your chances of getting free traffic from rankings are far better if you have as many urls from your site included in the database as possible. Having just your home page indexed isn't going to cut it.

If your site has 500 total pages, you ideally want 500 urls included in the search engine index. While this is admittedly a simplification of the situation, the more pages included, the better chance you have of getting high rankings under different keywords. The more high rankings you produce, the more free traffic you can get and the easier it will be to survive the loss of one ranking.

To check how many pages of your site are indexed in a particular search engine, simply do a search on the particular engine for “site:yourdomainname.com”. Pay particular attention to Yahoo, as it often will fail to pick up many of your pages. The result of this search will tell you every page the engine has found on your site and included in its database.

If you discover only a few pages have been indexed by the search engine, you need to take a look at your site. A common problem for sites using databases concerns dynamic urls. Many programmers will write session ids and so on into the urls. The search engines have massive problems reading these urls and will often fail to include them in their databases. You have this problem if the internal pages of your site have tails like:

“yourdomainname/id43/s-876783/product=46830

If you have this problem, the best solution is to create static pages without the dynamic urls issues. The session ids and so on should be moved into the code of the page and removed from the url. You will probably need a programmer to make the fix.

If you do not have this problem, you need to figure out how to get your site into the indexes. There are plenty of articles telling you how. The first step, however, is just figuring out if the search engines have found you.