Google Voting Experiment

I was presented today for the first time with the now infamous Google (Digg-style) Voting Experiment.

The experiment has been available to the general public since November, 2007. It allows a searcher to "vote-up" or hide a particular search result. Further, the user can suggest a web site deemed particularly relevant to the search at hand.

I checked some of the usual suspects and much of the coverage was fleeting. SearchEngineland did a basic summary post, and I couldn't find any mention of it on SEOMoz. ReadWriteWeb offered some reasons why this experiment was inconsequential, which seemed to be the general theme at a few other places.

Given my current opinion of relevancy and how it is manipulated by the SEO industry, this seems like a step in the right direction. Any true measure of relevancy must consider the user. Wouldn't it make sense to actually ask the users if search results are relevant to them based on their queries? Wouldn't that be a more accurate gauge than counting back-links?

Gathering user input on which results are considered most relevant will be tricky. Voting directly on the results page would seem like the most obvious place to start, as Google has done in their trial, but from a usability perspective it's a bit awkward. That is, in order for a user to properly determine the relevancy of a given result, they must first visit and review the site proposed. At that point, many people will have left the search result page and it becomes unlikely that they'll return simply to vote. Google hasn't yet started popping open a new window when the user navigates to a result listing, but this may be a way to mitigate this issue.

The bigger problem looming on the horizon is the SEO community in general. Just as they've flocked to Digg and Mixx and Reddit and every forum on the web, you can be certain that any mechanism  available to influence result listings will be set upon by every opportunistic optimizer out there.

Some possible ways to keep the hordes from the gates include:

• Utilizing only a sample of user input.
• Limiting the availability of the voting interface to any given user.
• Limiting the number of times a user can vote in a specified time frame.
• Identifying trends in user voting and taking appropriate actions.

The likely solution lies in a combination of the above. Nevertheless, this is not an easy problem to solve.

I'm encouraged to see Google dipping their big toe into the waters of direct user feedback. Until they determine how to effectively use this information to inform relevancy while at the same time avoiding the legions of search marketing professionals that will try to game the system, search results will continue to be inordinately influenced by non-relevant information.

Comments

[Peter]

Google has had a similar feature available on the Google toolbar for IE for several years now. The main differences are that the toolbar version only permitted up and down voting and the votes had no impact on personalized results. Of course, Google claims that the toolbar voting feature had no effect on search results. Then again, Google isn't exactly known for its trustworthiness.

[CMP]

Peter -
Thanks for the comment. I've seen the voting function on the Google Tool bar (the yellow and blue smiley / sad faces), though these seem to operate outside the context of any specific user initiated search. That is, I can for for a web page with these but can't specifically indicate that I feel a web page is relavent to a given search term.