Today, Google posted (finally) the first in what was promised (two months ago) to be a regular series of posts intended to open up more about aspects of internal operations. The original post was laden with disclaimers to the effect that certain truths could not be revealed due to the nature of their business. Since then, I've been waiting (not alone, I'm sure) to see just how much Google would be willing to share.
Today, Google announced Google Ad Planner, taking a direct shot at services offered by comScore and Nielsen. Differing from Google Trends for Websites, Ad Planner is aimed at media buyers looking to identify target sites for ad placements. In a market where Nielsen and comScore charge hefty sums to advertisers for access to their data while at the same time being loathed by publishers for the inaccuracies of their panel based measurement systems, Ad Planner is likely to gain acceptance fairly quickly.
There is also a level of integration with DoubleClick's MediaVisor product. The possibilities for integration between the DoubleClick and Ad Planner are endless and will give Ad Planner a distinct advantage over competing services. The service is currently in closed Beta right now but I'm hoping to get my hands on a login so I can test it out.
So, as I'm sure you're aware, late last week Google announced the release of Google Trends for Websites.If you haven't checked it out yet, or read one of the many reviews, it basically offers (approximate) site traffic volumes, top keywords, and related URLs for sites Google has a sufficient amount of data for.
Google joining the ranks of potentially (or known) inaccurate sources for traffic data (comScore, Alexa, I'm looking at you..) asides, a few folks got seriously up in arms over this release. SEOBook.com screamed "The Death of Privacy," and Lisa Barone (doing her best high school sophomore imitation) said "Not Cool, Google."
Why do people forget the contract we have with Google. We allow them to index and store our site content and use their free analytics tool without any pre-conditions on how they may use they data collected in the process. To be clear, I believe that Google wields way too much power over the Internet economy. But that doesn't matter.
If you don't want Google to have your data then stop using Google Analytics, uninstall the Google tool bar (which everybody loves to complain about), and block the GoogleBot using Robots.txt. If you want to continue to use their free services and benefit from the SEO-bubble we're all living in, then please shut up.
There are some things I don't like about Google, but then there are things like this that remind me of what an innovative company they are. No other company today embraces the internet community quite the way Google does and (most times, it seems) they truly strive to provide tools with real value.
Today, on the official google webmaster central blog, Google has announced the release of the Webmaster Tools API. While the initial function set is limited (outlined below), this is a nice way to provide access to webmaster data and promises to deliver greater functionality in the future.
Here's the list of currently supported features:
An immediate use for the Webmaster Tools API (as functionality expands) would be an automated notification system for crawl errors, Sitemap errors, etc. I'm sure I can think of a few more.
What creative uses do you think a fully functional Webmaster Tools API could be used for?
Yesterday Google announced updates to their query recommendation algorithm that will process newer information in a more timely manner.
I tried a few searches for items that have recently been in the news and the results were good - for example, a search for "Iowa" recommended "Iowa Flooding". The same search, however, also recommended "Iowa DOT." I suppose the state department of transportation would be in high demand given the recent flooding, but seeing that recommendation is occurs to me that what's missing from these suggestions is context.
It would be helpful to the searcher is Google could present some bits of information indicating why a given search term is being suggested. Here's another example - I searched for "Father's Day" and among the suggested queries was "Father's Day, 2007." As a searcher, it's unclear to me why I should be interested in Father's Day from last year. Providing some of the deterministic data along with these recommendations would help me understand the relevance of this recommendation to my intentions. After all, relevance is what Google is all about.
TechCrunch is reporting that Google and Yahoo are announcing (right about now) a partnership that will most likely involve outsourcing Yahoo's search business to Google. Given the amount of dominance Google already has in search this will certainly raise market concerns as such a deal could push Google's share of the search market to near 90%.
I've been fairly vocal in the past in my opinion that Google has way too much power over the internet economy at present and this certainly doesn't help matter. For me, I'd much rather have seen the Microsoft deal go through so somebody, somewhere, could introduce real competition into the market. Online search is in dire need of innovation at the moment and giving Google a bigger slice of the pie will do nothing to foster that innovation, regardless of how creative the company has been in the past.
Update: For those of you interested in the gritty details, TechCrunch also has the the Yahoo SEC Filing which outlines some specifics of the deal.
Any other thoughts on this out there?
A few weeks back Udi Manber, the VP of Engineering for Search Quality, made a much publicized post on the Official Google Blog. In the post, he promised an increased level of transparency from the search giant and told us to watch that space for future openings of the kimono.
Well, since that first bit of bait, here's what we've gotten -
Late yesterday, Udi Manber, VP of Engineering for Search Quality over at Google offered up a post on the Official Google Blog titled - Introduction to Google Search Quality. What drew me in was this quote -
This blog post is part of a renewed effort to open up a bit more than we have in the past.
Having high hopes I read on, hoping for some insights into the (largely) black box that is search relevancy. Unfortunately, I was disappointed. The post dedicated a fair amount of time to detailing just how hard the job of the Search Quality team is, and how every second they have to handle hundreds of queries, parsing billions of documents to pluck out and order the relative few. And how they have just milliseconds to do so. Some time was also spent on the structure of the group, internationalization efforts, and some past history.
That's all great, but it feels like a setup to me. Frankly, this post did nothing to "open up a bit more" and even reiterated the point of why they need to be so secretive -
We are, to be honest, quite secretive about what we do. There are two reasons for it: competition and abuse. Competition is pretty straightforward. No company wants to share its secret recipes with its competitors. As for abuse, if we make our ranking formulas too accessible, we make it easier for people to game the system. Security by obscurity is never the strongest measure, and we do not rely on it exclusively, but it does prevent a lot of abuse.
So in one blog-breath Google tells us their opening the door a crack while at the same time applying a few more dead bolts. It smells of PR to me and not a genuine desire for transparency.
I was also surprised to see this item receive fairly sparse coverage around the blogs and news sites. I don't go crazy with my RSS subscriptions like lots of folks do, but I've got 40 or so relevant ones in Google Reader and I've only seen this bit from SearchEngineLand - and that's nothing more than a recap without any commentary.
Perhaps I'm being too quick to judge. The post is, after all, labeled as the first in a series. So far, nothing has been revealed, but I'll be watching that spot in Google Reader closely over the next few weeks.
Related Posts: The Expanding Web
Yesterday Google announced a private beta of FriendConnect - a standards based (OpenID, oAuth, and OpenSocial) collection of widgets that can bring social networking type functionality to any site or blog. The announcement has garnered its fair share of coverage (Mashable, Yahoo!, SearchEngineLand, Outside the Lines) so instead of simply rehashing what's already been written about FriendConnect, I'd like to look a little deeper.
With the CPC contextual text link model, Google opened up online advertising to the masses. With AdWords, even the smallest web site could bid on keywords and arbitrage traffic - effectively eliminating all barriers to entry. With FriendConnect, we will see the same effect. Any web site, no matter how small, will be able to add social networking components. Initial availability will include user registration, invitations, member galleries, message posting, and reviews, as well as OpenSocial applications.
This is again Google's long-tail approach to the web. Commodotize a feature set once controlled by a small number of vendors, and in so doing reach the vast potential of 100+ million web sites out there. With FriendConnect, Facebook and MySpace lose their prominence as social media platforms and the web itself becomes the platform. Consider how the game changes when your OpenID follows you from site to site and you know which friends are active on which sites. Combine this with the data portability movement and suddenly the web begins to resemble, well, life. Walk into any room, see whos there, recognize your pals and, wherever you go, you're the same person.
On the heels of a highly isolated StumbleUpon disappearance, my string of internet anomalies continues. The PageRank has gone missing from the google toolbar. As of this morning, for every site I visit, my google toolbar in both IE and FIrefox has lost the little green progress bar. Upon rollover, I get the following message "No PageRank information available."
I checked a few other computers in the office and we're all seeing the same thing. Anybody else out there seeing this?
Michael Martinez over at SEO-Theory.com recently posted a thoughtful article which considers the rate of expansion of the web and questions whether search engine technology has maintained pace.
What’s refreshing about this post is where it differs from the typical SEO blog entry. So often, folks in the SEO industry start a blog in an attempt to create and nurture a personal brand. The unfortunate reality, though, is that there’s only so much SEO content to go around. Much of what gets written in the pursuit of SEO Identity is either rehashed SEO basics or a variation on The Ultimate List theme. None of this is very interesting and in the long run is counter productive to the desired goal.
I figure Mr. Martinez is after the same with his blog. However, his posts at least make me think and offer a critical perspective on the search industry. In his latest, there are a few points I agree with and a few I question.
Content Inclusion
In examining the completeness of a search engine’s index, Mr. Martinez states that in order to deal with limited capacity Google has implemented a “Web Apartheid.” That is, not all sites get indexed and Google determines which of the fortunate (relatively) few make it in.
In Google’s defense it should be noted that a huge number of spam sites have been dropped out of Google’s index, but when all is said and done Web spam is a legitimate part of the World Wide Web because neither Google nor any other search engine has the authority to determine what is and is not part of the Web.
This argument falls apart a bit when you consider that all search engines attempt to provide the searcher with “relevant” results. Whether you agree with how the engine determines relevancy or not, this implies a qualitative process. If we allow Google to determine which sites are relevant for a given query, we must also allow them to determine which sites are not relevant to any queries and thus be excluded.
Link Building
As a result of this Apartheid, Mr. Martinez states:
For the search optimization community, then, the primary challenge today is the same as it was ten years ago: we have to ensure that our protected content is included in as many search indexes as possible… The more inbound links a page has, the more likely the page will be crawled and indexed. Simply having a page crawled and indexed is the SEO’s first priority. Getting the page to rank is the SEO’s second priority.
The implication here is that significant link building is required in order to get your content indexed. With the exception of truly large sites (millions of pages), my experience has shown that getting content indexed is fairly straight forward. I’ve managed a number of content driven sites, all of which had new pages indexed immediately without any external inbound links. This includes new sites that were created and all it took was the submission of a sitemap to get pages indexed. Again, for significantly large sites I would expect the engines to take their time and index slowly as you prove your worth. I would be interested to hear an account of, say, your average blog owner who must go out and procure new inbound links before new posts get indexed.
Link Counting
I haven’t found too many instances of people who get paid to perform search engine optimization for their clients openly criticizing the link based page rank algorithm. I suppose it’s a case of not wanting to bite the hand that feeds, but Mr. Martinez bucks the trend and comes out against link counting – and on this point we agree completely. Mr. Martinez introduces the issue as
… gross error of judgment that has yet to be acknowledged by Google.
He further provides excellent background on the history of link based page rank.
Larry Page and Sergey Brin tested their PageRank hypothesis on the Stanford University Web site, where pages were not embedded with links designed to assist with crawling or commercial promotional links. Just because they were able to show that Stanford’s probable most important pages were better linked than other pages did not mean their model was relevant to the real World Wide Web. Google’s founders and investors failed to reconcile the wrong assumptions with reality.
This is evidence that the SEO industry is operating from a faulty premise. Mr. Martinez reiterates his point that aggressive link building was originally an attempt simply to get pages indexed and thus:
The valuative [sic] citation model that Page and Brin believed existed in fact never existed at all, except possibly on academic and government Web sites.
And finally:
The World Wide Web cannot be accurately valued through its linking structure because the linking structure was never designed to be a valuation process.
Excellent points, all well made. Not only is the link based model built upon an unsound foundation, the structure of our relevancy ecosystem is weakened each day as SEOs everywhere continue to try to use this flaw to influence natural search rankings for their clients.
There’s much more to Mr. Martinez’ post and I recommend you read it in its entirety. Unlike much of the line-towing that goes on in the SEO industry, this entry is notable for its thoughtfulness and consideration of how the status quo simply isn’t sufficient. The search optimization industry clearly needs to evolve and it won’t unless its practitioners ask the hard questions – both of themselves and the search engines.
Related Posts: Universal Search
Last last week, news broke of a potential Google algorithm update. As we move further into this week confirmations abound - including a call for feedback from Matt Cutts and millions of comments from SEOs reporting non-trivial changes in search rankings.
What's most notable about this, and past Google updates, is the rampant speculation on conjecture on what the algorithm changes consist of. Once again I need to restate that Google simply wields too much power these days. Unspecified changes to the existing (fully) unspecified algorithm simply create confusion, a bit of panic, and an inordinate amount of wasted time.
Here's an analogy. Imagine a local business owner running advertisements that contain driving directions to her store. She's spent a fair amount of time providing directions from various points, including back-roads and highways, to accommodate her geographically diverse customer base. Now imagine that the Federal Highway Administration in conjunction with State Departments of Transportation and various local agencies rearrange all the roads in a manner they think is a little better. And in so doing they tell no one of their plans, publish no maps, and only tacitly admit that changes were ever made. Now our poor local business owner has no recourse but to go out and drive all the roads herself, from all origination points she's interested in, and republish her directions.
Sounds a little koo-koo, no?
Related Posts: Google, SEO, and the Ruination of the Internet, The Expanding Web
Just when I'm wondering what to write about next, I see this -
Official - Mahalo is Spam, According to Google's Internal Spam Documents
"Wow," I think, "this is going to be good."
In a number of ways, this post on SEOBook.com is great blog fodder (or link bait, which ever you prefer). Let us count the ways:
#1 Final Notes on Spam
SEOBook.com quotes the leaked Google documents as follows:
When trying to decide if a page is Spam, it is helpful to ask yourself this question: if I remove the scraped (copied) content, the ads, and the links to other pages, is there anything of value left? if the answer is no, the page is probably Spam.
What is amazingly ironic about this statement is if you ask the same question about any Google search results page, you will undeniably determine that page to be Spam.
#2 Jason Calacanis Venom
I still don't get why so many people with the letters S, E, and O present and sequential in their job title (bestowed upon them or self-proclaimed) have so much built up anger toward Mr. Calacanis. Well, let me restate that. SEOs don't care for some of what Mr. Calacanis has had to say about them in the past, even though he tried to make ammends at SES NY this year. What I don't get is why everybody is so insecure. Surely these folks are true marketing professionals that have well rounded skill sets they can fall back on as the online link-baiting business slowly starts to fall apart. Right? Surely they're not worried that the true foundations of SEO are really quite simple and one day their clients will wise up and the demand for their sevices will decline dramatically as SEO practices become integrated into standard in-house site building / content creation processes.
#3 What should be indexed, anyway?
By their own definition, Mahalo is a search engine. A curious question, then, is why does Google index the Mahalo pages at all? Search results pages from Yahoo! or Ask.com or MSN aren't indexed by Google, and the favor is graciously returned by each of these sites. So why, then, are the Mahalo pages being indexed? As an alternative search engine, shouldn't they stand alone? As a user, there's marginal value to me if I exectute the Google search, only to be directed to another page full of externally pointing links. Unless, of course, the Google search results page is as relevant as it should be. The same question can be asked for other "aggregator" type sites which function, essentially, as search engines. Think of Shopping.com or PriceGrabber.com, for example - the vast majority of the content on those sites is unoriginal, copied from other sites (merchants or product review synidcation), and provide nothing more than a search function to their users.
Should results page from vertical search engines be indexed by the Big Boys? What about results pages from general "alternative" search engines? This is a tricky proposition that will become more complex as the major Search Engines move more toward becoming destination / content sites themselves.
At SES NY today, the 1:30pm session was focused on Universal Search. The speakers for the discussion were:
For those not familiar with the term, Universal Search refers to a blended search results page where Google combines listings from its video, image, local, news (etc.) engines with those it gathers from spidering web pages.
The onstage conversation was the livliest and most engaging I've seen at SES so far. Mr. Battelle was persistent in his attack insistance that the continuing trend toward Universal Search was slowly turning Google (and other search engines) into the destination, rather than the road map. Recently released comScore data supported this theory, showing a 3% lower click-thru rate when Universal Search results are displayed to the searcher.
On the surface, this may seem ideal from a user experience perspective. Getting the searcher to the data they're seeking as quickly as possible is what it's all about, right? And that was the position taken by Mr. Menzel. The situation gets muddy with the realization that Google is also, slowly, becomming a media company. In many Universal Search results, you will find Google Finance for stock quotes and Google Video (YouTube) for funny clips of Chuch Norris. Mr. Battelle pointedly asked Mr. Menzel if Google favored their media properties when presenting Univeral Search results. Naturally, the Google stance is to always show the most relevant results. Of course, "relevancy" is determined by Google's algorithm, which relies on many, many data points of which Google is likely to have the best data for their own properties, so you do the math.
For the thoughtful SEO crowd in the audience, the question is this - What happens when Google becomes the destination? Even if the issue of Google showing more of it's own content on the results pages is put aside, where does it leave the average search marketer if the searcher never leaves the results page? In this scenario, the monetization and ROI of SEO goes right out the window as the user never makes it to the content owner's site.
It's an uncomfortable situation to be in. The entirety of the web allows Google to spider their content with no real guidelines on exactly how much of it can be displayed within a Google property. Would it be going too far for Google to allow a video from Break.com to be played direclty from the search results page? The inclination is to say yes, but wouldn't that also be a better user experience?
On stage today, Mr. Menzel said several times that Google wanted nothing more than to deliver the searched for information to the user as quickly as possible. Following this line of thinking to its logical conclusion, Google starts to look more and more like a destination. And if that's the case, then we're witnessing the beginnings of a game changer for the SEO world.
Related Posts: Yahoo Semantic Web
Most years, I forget that daylight savings time has arrived until one of the (many) electronic devices in my home reminds me.
Given the number of devices that will adjust to the time change without prompting, I was surprised to find that many people are using Google to figure out what time it is. This morning, 11 of the top 13 searches in Google are related to the time change - my favorite being the direct-and-do-the-point "what time is it now."
In the early 1990's I would awake on a Sunday morning and not realize what was going on until I turned on NY1 News or ventured out to get a newspaper. Some years, it would be mid-day before my clocks were sprung forward and in a way I liked not knowing. With so many information sources surrounding us these days, it's impossible to enjoy a similar level of ignorance.
This morning I also awoke to find that President Bush has vetoed a bill that would ban waterboarding. It seems we've sprung both forward and backward at the same time.
Sometimes it's better not knowing.
I was presented today for the first time with the now infamous Google (Digg-style) Voting Experiment.
The experiment has been available to the general public since November, 2007. It allows a searcher to "vote-up" or hide a particular search result. Further, the user can suggest a web site deemed particularly relevant to the search at hand.
I checked some of the usual suspects and much of the coverage was fleeting. SearchEngineland did a basic summary post, and I couldn't find any mention of it on SEOMoz. ReadWriteWeb offered some reasons why this experiment was inconsequential, which seemed to be the general theme at a few other places.
Given my current opinion of relevancy and how it is manipulated by the SEO industry, this seems like a step in the right direction. Any true measure of relevancy must consider the user. Wouldn't it make sense to actually ask the users if search results are relevant to them based on their queries? Wouldn't that be a more accurate gauge than counting back-links?
Gathering user input on which results are considered most relevant will be tricky. Voting directly on the results page would seem like the most obvious place to start, as Google has done in their trial, but from a usability perspective it's a bit awkward. That is, in order for a user to properly determine the relevancy of a given result, they must first visit and review the site proposed. At that point, many people will have left the search result page and it becomes unlikely that they'll return simply to vote. Google hasn't yet started popping open a new window when the user navigates to a result listing, but this may be a way to mitigate this issue.
The bigger problem looming on the horizon is the SEO community in general. Just as they've flocked to Digg and Mixx and Reddit and every forum on the web, you can be certain that any mechanism available to influence result listings will be set upon by every opportunistic optimizer out there.
Some possible ways to keep the hordes from the gates include:
The likely solution lies in a combination of the above. Nevertheless, this is not an easy problem to solve.
I'm encouraged to see Google dipping their big toe into the waters of direct user feedback. Until they determine how to effectively use this information to inform relevancy while at the same time avoiding the legions of search marketing professionals that will try to game the system, search results will continue to be inordinately influenced by non-relevant information.
That's right. Ruination.
I happened across a recent blog post that condemns the increased frequency with which online marketing experts are recommending the use of social media sites as a means to influence search engine results. The thinking goes that hordes of SEO-types are infiltrating the social web seeking to leverage the platform to generate back-links to their maybe-sometimes-but-not-that-often relevant content.
Well, I hate to be the bearer of bad tidings, but it's not just the social media sites that are at risk. The situation described above is merely a symptom of a much larger issue. The real problem stems from Relevancy, how it's computed by the search engines, and the SEO industry it has spawned.
(as an aside, any term that can be used freely as a noun, verb, and adjective in the same sentance has to be suspect. I'm an SEO performing SEO so my client's site will be SEO).
What am I talking about, you say? Here's the crux of it - "Relevancy" is a subjective concept that the search engines are attempting to determine by objective means. You cannot determine a qualitative measure by using a purely quantitative analysis.
And so what's happened is this. With the exception of the farthest reaches of the long-tail, adding your target keywords (or phrases) to your Title and Meta Tags and on page copy and then throwing in some H1s and BOLD tags for good measure cannot enable the search engines to determine the most relevant page for any search term given the fact that there are likely hundreds of other sites out there that have done the same thing for the same search terms. So what we're left with, then, is Page Rank.
By now, the Google Page Rank Algorithm is widely known. Simply put, the more back-links you have, preferably from high-ranking pages, the higher your own page rank will be. In an environment that is as content-rich and comptetive as, well, the entire internet, simply publishing a piece of content and hoping it gets noticed and hoping further that the good-folks of the web will graciously bestow upon your content bundles of back-links is wishful thinking indeed.
In a more traditional marketing model, a product is produced and then marketed. Clever or Sexy or Serious advertisements are created and distributed through various media. Much of which is online these days - "banner" advertising is far from dead, especially as broadband penetration deepens and rich media ads become more pervasive. Historically, (pre-internet), TV and Radio and Print were the primary distribution vehicles, but the effectiveness of these was (is) hard to measure. With internet-based advertising eveything became trackable and the numbers were appalling (1% Click-Thru rates!).
Google revolutionized internet advertising through contextualized CPC ads. Suddenly advertisers were given control over when their ad was shown and could determine how much they were willing to pay for the acquisition of that online visitor. Highly measurable, ROI could easily be determined and in many cases even an arbitrage was created.
And then everybody looked to their left a little. And what they saw were the natural search results, just sitting there, promising free traffic. No CPC, if only my site was on the front page....
Thus was SEO born.
And so what happens? In an attempt to overcome the fact that 1000 other websites are using the same keywords in Title and Meta tags, etc., the diligent SEO goes on a link building campaign. The objective search engine bot spiders and picks up links mostly indiscriminently, and the hard-working SEO knows this, so just about any link is a good link (and, man, those social media sites sure are great places to build links).
If you follow this closely enough, SEO really becomes a game of trying to convince (fool) the search engines that your content is more "relevant" than somebody else's.
If I hire 1000 people in the Ukraine to spam the social media sites, does that make me more relevant? If I succumb to the daily phone calls from the company that wants me to pay bloggers to write about my content and provide back-links, does that make me more relevant? Or what if I simply buy Text Link ads? Maybe these are considered "black hat," but they're only taking a perfectly acceptable SEO practice to it's logical extreme.
And it happens every day.
I'll admit that "Ruination" is a strong word. But the current notion of relevancy is broken and an entire industry has sprung up to take advantage of the situation.
