On Google's Spam Woes and the Need for More Players in Search

Wednesday, January 26, 2011

Fresh on the heels of search newcomer Blekko’s launch in November, and the ensuing debate over whether it’s particularly wise for anyone to challenge Google, the last few weeks have seen a surprising, though gratifying, spate of criticisms of the declining quality of Google’s results for some searches. And, surprise surprise, by and large the critiques run parallel to issues raised by Blekko Founder, Rich Skrenta, in the months leading up to Blekko’s launch, as well as to arguments we’ve been making since Helioid’s first launch in 2008. Finally, it seems as though people are catching on to the fact that Google isn’t infallible, that it doesn’t handle every search perfectly, that the kinds of searches it struggles with are actually increasing in number, and therefore that there is indeed room in the field for fresh players.

On his personal blog, Stack Overflow Co-Founder, Jeff Atwood, laments the increase in content-copying search spam sites regularly beating out Stack Overflow pages in Google searches, simply by scraping Stack Overflow content and displaying it with more ads. On TechCrunch, Vivek Wadhwa regales us with tales of his struggles against the mass manufacturers of quasi-content, like Demand Media, which are increasingly dominating Google’s search results, without actually offering users any worthwhile information. The response from Google has been to brush these complaints off as being in response to a “recent uptick” in spammers infiltrating the top results, but Atwood points to complaints dating back to 2009, from Richard McManus and Paul Kedrosky touching on the very same issues. Kedrosky in particular shares his personal horror after struggling to find trustworthy comparisons of dishwasher brands online and being bombarded with page after page of search spam, and concludes that the “appliance search” genre is too spam-laden for even Google to handle. Clearly this problem isn’t due to an uptick in spammer activities confined to the last few months. And all of these complaints resonate strongly with an issue we discussed in our very first blog entry in 2008, which had first been raised shortly prior by Nova Spivack: that the exponential explosion of content on the web would sooner or later start to strain the ability of keyword search engines to consistently place the most relevant pages amongst the top results. It seems as though the success of content mass-manufacturers like Demand Media and Associated Content is helping Spivack’s prediction come about sooner rather than later.

[caption id="" align="aligncenter" width="368" caption="Web Spam Example"][/caption]

The good news is, the creeping dissatisfaction with at least some Google searches may help abate some of the knee-jerk skepticism of new search engines like Helioid or Blekko, especially when they propose innovative ways of filtering out the noise plaguing certain Google searches. Blekko CEO Rich Skrenta actually touches on a similar issue to the ones raised by Kedrosky and Mcmanus in the CrunchTV interview we discussed in our last blog post, concerning song lyrics. Skrenta rightly points out that if you search for lyrics online, because it’s so easy for search spammers to copy and paste lyrics supplied by trusted sites, you end up getting bombarded with gratuitous ads and may even pick up some malware. Blekko has dealt with this problem by only drawing results for lyrics searches from a limited list of trusted sites added by users under the “/lyrics” slashtag. Blekko’s ability to avoid Demand Media filler content by drawing upon user input has been one of Skrenta’s favorite reasons to give for switching to Blekko. Even a slight decline in the quality of Google’s results may push more users into experimenting a bit, and making that switch, and a little extra adventurousness on the part of the users will make for a more fertile environment for innovative search start-ups in general, including future releases of Helioid’s web exploration tools.

Blekko and Dexterous Search

Wednesday, January 12, 2011

This past November, search newcomer Blekko opened its figurative doors to the public, and the appearance of a search engine that aims to intuitively boost user dexterity in refining results has been much welcomed. Blekko lets users narrow or reorganize search results by adding slashtags to the query. For example, a search for "net neutrality /liberal" returns only results on net neutrality taken from top liberal blogs and political sites. A search for "black swan /date" returns a chronological list of results for black swan, including the most recent reviews. Users can create new slashtags, by compiling a list of trusted sites over which they want to search using said tags, and can subsequently make their slashtags public so friends and other users can use the benefit of their peers’ experience is certain domains.

The major complaints heard from Blekko’s detractors aren’t particularly surprising: Google does search too well for anyone to possibly get a foothold and creating and using slashtags is too complicated for the average user. In an interview with TechCrunch shortly after their launch, Blekko Founder and CEO, Rich Skrenta, briefly addressed these issues. As he more or less put it, with regards to the first complaint, Google doesn’t handle every search perfectly, and therefore there’s room for an additional player to pick up the ball wherever Google drops it. Blekko doesn’t need to be a "Google-Killer," it just needs to find a niche or collection of niches in the widely varied ecosystem of web search. And with regards to the second complaint above, concerning the complexity of use, Skrenta notes that they’ve deliberately gone after the "power users" to start with, and are using these advanced users’ efforts to improve the search experience for their less involved visitors. Sites listed as trusted under a particular slashtag can be used to filter and improve results for searches related to the tag, even if those searches are made without the benefit of slashtags. Good answers.

Blekko Screen Shot

We think Blekko might just have staying power, and possibly enough of a draw to nip at Ask’s heels one day in the hopefully not too distant future. But regardless of how Blekko itself fairs, this development in the progression of ambitious search startups is an exciting one. The folks at Blekko have masterfully put into action a couple of the general guiding principles that have been part of our core philosophy at Helioid since day one. Namely, that at least some searches can be greatly improved with added user dexterity in manipulating search results and that search in general can be improved with new, innovative ways of incorporating user feedback. It’s quite gratifying to see a startup exploring these ideas so effectively, and hopefully if Blekko does establish a loyal user base it will evaporate the myth that no search startup short of a genuine Google-Killer has a place in the field.