Everyone in the computer field has heard of a search engine and for those of you who haven't a search engine is a program on the internet that is designed to search all of the websites and their documents for specific keywords. The search engine then produces a list of documents wherein these keywords are found. The major online search engines are Google, Yahoo and MSN. Search engines have a large database that is used for the storage of downloaded and processed pages. It is usually referred to as the index.



There are some common principles that are used to activate the search engine. When you use a one, it first sends out a "spider" whose job it is to collect as many documents with the related keyword as possible. The spider can be compared to a web browser; however the difference lies in the fact that the web browser shows all of the information on a page while the spider has no visual components. It works with the HTML code of the page.

The next program you will encounter is the "crawler" that helps to find links on a page. It is with the help of the crawler that the spider knows where to go. Following these links, the crawler can find website documents that were previously undectable on the search engine. It is then the "indexer" that analyzes each page and each part of the page such as headers, text, special HTML tags, etc. These three programs collectively constitute the common search engine principle.

There are some crawlers that are better than other, more common crawlers. These crawlers do a deep crawl on a website to access as many pages as possible which contain the keywords that are mentioned. These deep crawlers can also gather pages that have not been submitted! However, it is always better to search on the larger search engines; because the larger the engine, the higher the number of pages that are listed on the site. There are some search engines that can follow frame links, and some that cannot. So it is better to use those that follow frame links as it provides for a complete search of your web page.

Crawlers are found in crawler-based search engines where the listings are created automatically. However, there are also human powered directories where it depends upon humans for its listings. There are also hybrid searches that work on a combination of both of these types of search engines. Some of their searches are created automatically while others depend upon humans.

Next in line is the results engine. This is the program that extracts the search results from the database to rank pages. It is through this program that the order of the pages that best match the user's query is arranged. There are ranking algorithms that have to be followed to arrange the web pages in an orderly fashion.

The next important part of the search engine is the web server. It is the web server that is responsible for all of the interactions between the user and other searching components. The web server comes with a HTML page which has an input field. It is through this input field that the user can actually specify the exact query or information he/she is searching for. The web server also fulfills another need, and that is to display all search results related to the needs of the user in the form of an HTML page.

By following the common search engine principles and organizing your website according to these principles, you can surely place your website on the top listings of Google, Yahoo and MSN.