The history of search engine optimization began in the early to mid 1990's when web site owners realized sharp spikes in web traffic following the inclusion of a web page to a search engine for indexing. Many web site owners began to pay for frequent admission to the search engines in order to be indexed since it resulted in more web traffic which translated into more conversions and more revenue.

At about the same time, web site optimizer firms began business with experts who specialized in finding a way to make web pages more visible to the search engine spiders. Although most such firms did so by studying the algorithms used by the search engines and rewriting the web pages in a way that would utilize the characteristics of the algorithms to the benefit of the customer, other, less scrupulous search engine optimizers used tactics such as place keywords with a high number of hits in white font in the white space of the web page. Thus, the words could not be seen by humans looking at the pages, but the search engine results returned these pages at a higher rate because the high impact keywords were seen by the search engine.

The downside to such 'black hat' tactics was increasing lack of relevance to the search query in the search engine results page (SERP). For example, if a searcher queried the keyword 'vitamin C" and saw a SERP with the first three links devoted to 'sex', the searcher is going to believe that the search engine is unreliable. The search engines developed techniques to filter out such optimization techniques, which simply made the black hats firms more creative in their goal to increase the amount of web traffic to their customers' web sites.

By the beginning of the 2000's, both sides realized the benefits of working together, rather than competing with one another. Especially following the marketing plan of 'paid inclusion' or pay per click, the search engines realized revenue from successful linking of web pages to the SERP, while web page owners wanted conversion on the increased web page traffic. The benefits of relevance could be seen by both the search engines and the web page owners.

Optimizers still use techniques to increase web traffic to the customer's web page in ways that strain to be relevant while search engines continue to develop ways to forestall tactics which muddy the results page with non relevant entries.

Both the insertion of keywords and phrases which make the web page look differently to spiders than it does to humans viewing the page and the use of so called link farms which purport to make a web page appear more popular to the search engine spider than it really is result in search engine results pages which are less relevant to the search terms.

The guiding principle which should lead search engine optimization techniques is to promote the benefit and usability to the human visitor. A popular concept in web pages today is to remind the viewer that their web pages are written for humans, not for the search engine spiders.