There are literally thousands of tools that are available on the market to help you choose keywords to optimize your WebPages, but one of the most underutilized tools that you should be using is your web logs. Your web log is the key to telling you who and how people are finding your website. It will also tell you if there are new trends in which people are actually finding your website by keywords or phrases that you may not necessarily have optimized for.

All internet marketing and search engine experts will tell you that if you are not in the top 5 records of any search engine results then you are simply not going to get any traffic. This has a certain element of truth to it but is not totally correct. In fact we have had a couple of WebPages recently that were not optimized for any specific keywords or phrases, but these pages have been pulling 50 hits a day, leading to sales even though it was on the 95th page of the search engine results.

To pull 50 hits a day off a keyword on the 95th page obviously prompted my interest and this is the point that I want to bring to your attention. The only reason that we were able to find this out was that we have the ability to review our own web logs. Let me explain how this works ?

Every time you visit a website you leave a small piece of information that the website owner can in fact collect. The information you leave behind includes the page you visited, your IP number and a series of raw data that includes the website that referred you to the site, your browser, the language you are using, the operating system you are using, session details, character set etc. As the website owner you are able to collect this information and use it to help you focus your site or identify that your search engine optimization program is working.

For example, if a person goes to the Google search engine and searches on the term "car detailing", when the person clicks on the link in that search result to your website, the website that they goto will see in the Raw Data the following:

http://www.google.com.sg/search'hl=en&q=car+detailing&btnG=Search&meta=

The HTTP Referer in the example above is the Google Singapore website and the search was done in English. You can tell this by the en after the question mark. After the q it tells you that the search term was "car detailing".

What you should be doing on a daily basis is to look in your web logs and to look at who is referring people to your website. For example if you find a keyword or phrase that people are using at Google, MSN or Yahoo to get to your site that you have not optimized for, then you need to first do a search on those search engines using those keywords to find out where you are positioned in the search results. Obviously if you are positioned highly, then you may not need to do any work but if you were not ranked in the first page of results, then you could certainly look at optimizing your webpage for those keywords.

One of the challenges I know that many people have is that they use a third-party hosting company to actually host their websites. Often these companies do not give you access to the web logs. They will though give you access to a statistics program that usually only looks at the number of hits you get, where the hits are coming from a country perspective and the domains that are referring people to you but they often do not include the key terms used by individuals at the search engines to find you.

I should note that you can in fact build your own statistics program. It is possible for you to collect all of the information discussed earlier when a visitor visits your website. All you simply need to learn is a little bit of ASP. There are lots of great ASP programming sites that will be able to help you achieve this.

One of the other free tools available on the internet that will allow you to track what search terms are being used to get to your site is available on Google. Under the webmaster function in Google, they provide a Sitemap function that tracks what search terms people are using to visit your site and what search terms are being used where your website shows up in the search results. The downside to this tool is that it only shows the top 20 results and is only for searches done on the Google search engine but it does cover results from all Google Search Pages across the Globe.

The best tool to help you work out how people are finding you and what terms they are using to get to your website is your web logs. If you do not have access to them, you are limiting the potential of your website as you may have a clear niche market and not know it exists if you do not have quality web logs.