Google’s Head of Research on SERP Split Testing
In an interesting article in yesterday’s Sunday Times, Peter Norvig (Google’s director of research) mentioned that a small percentage of Google users are diverted to alternative versions of it’s search engine:
On any given day, Google may be testing a couple of new tweaks of its algorithms – the formulas that decide where web pages appear in the listings. Norvig said the company may divert 0.1% of its traffic through a trial version of its search engine to see if users like the results.
Now 0.1% of its traffic may sound inconsequential, but when you execute over 91m searches per day in the US alone (source: searchenginewatch.com) that amounts to 91,000 searches using modified technology.
This would also account for some of the strange appearances with Google’s SERPs (screenshot of request is there). While the Google Adwords quality feedback system has been known about for some time (click on an AdWords link from Google SERPs, go back to the SERPs and you may see a ‘Was this link useful?’ request), this new system seems to solicit similar user feedback for organic listings. (Here’s a short but interesting side discussion about tainted feedback responses WebMasterWorld.com.)
This is an obvious attempt by Google to weed out the countless spam sites that have plagued their index for quite some time now. Of course, the effectiveness of getting your users to do your work for you must be questioned, and I very much doubt we will see this functionality rolled out to a wider audience (getting prepared to eat hat now that I have said that ).
(You can find some other interesting discussions about Spam over on the Google Webmaster Group.)
As you know Richard, I am a big fan of split testing. Split testing is the first rule of online marketing. And Google seem to have their heads firmly screwed on when it comes to online marketing.
Comment by Dave Davis — November 20, 2006 @ 5:19 pm
And I am very much hoping I get to hear about some of the results from your current batch of tests
Comment by Richard Hearne — November 20, 2006 @ 6:10 pm
Well, right now, the test I am preforming has one fundamental flaw. Lack of traffic. The test at this rate will take weeks, if not months more to get accurate results. My goal was to measure results from organic SEO, but that takes time. I do not want to waste adwords money on this particular one.
Now that I have learned the basics, I am going to start testing it on another higher traffic with established traffic and credibility, it is just horrendously difficult to incorporate the tags into a CMS. Even the most simple.
I have already started the research into developing a CMS designed specifically to use this tool.
Comment by Dave Davis — November 21, 2006 @ 2:46 pm