For months now, the 7-year-old search engine has been testing new crawling and indexing systems with the hopes of making web searches more intuitive and accurate. The project, called “Bigdaddy,” is much more than a software update. It's a total refit of the search engine's infrastructure and software.



“…It's not just a data push or algorithm update,” says Search Engineer Matt Cutts of Google. Three data centers have so far been updated with new ones coming online every 10 days, a pace that's expected to increase. Web users can access the updated Google search by typing in the IP address of the updated servers in their address bar: 66.249.93.104, 64.233.179.104 or 216.239.51.104. All of Google's reported 85+ data centers are expected to be updated around the end of February or early March, and the closer we get to that time, the odds increase that your Google search will hit a “Bigdaddy” data center.

What ‘Bigdaddy' addresses

The biggest issue “Bigdaddy” will fix is hijacking redirects of URL requests—what happens when someone redirects a request for one website to another though illicit techniques. These kinds of redirects are called “302 redirects.”

“Bigdaddy” will also add something Google calls “canonicalization,” which will instruct a search engine how to decide which of a series of URLs is the proper one to insert into the Google index. For instance, most people would consider these the same URLs:

www.example.com

example.com/

www.example.com/index.html

example.com/home.asp

In reality, these URLs are different and could conceivably return completely different content for each request. However, Google's “canonicalization” will pick the URL that best represents what the user is likely asking for.

What ‘Bigdaddy' does

“Bigdaddy” will also bring different crawling and indexing criteria, the groundwork for more advanced algorithms, larger databases and faster, more effective indexing of content.

Additionally, Google will be able to index new kinds of content. Testing is underway on a new search engine spider based on the Mozilla browser that can read links within images and even within Flash video.

What ‘Bigdaddy' means to you

Brand Identity Guru has spent the past week researching what the implications of “Bigdaddy” will be for our Boston Search Engine Optimization clients. By comparing the results, we found that “Bigdaddy”-powered keyword searches returned improved rankings for 77% of our clients while 17% stayed the same and 6% (mostly those without a good number of incoming links) experienced a decline.

“Bigdaddy” data centers also reported much higher numbers of search results: searching for “landscape software,” the non-“Bigdaddy” search engine returned about 30 million results while “Bigdaddy” returned over 60 million. We've concluded, then, that sites ranked well because of solid on-page SEO techniques and inbound linking will benefit from this update. But there are still many unanswered questions about the implications of “Bigdaddy.” However, Google (as usual) is remaining mum on specifics in the hopes of preventing people from taking unruly advantage of the system.