London - It looks as clean and simple as it always has: the little white letter box into which we type our search queries and begin our light-speed journeys across the world wide web. Behind the unchanging façade, however, Google’s engineers are engaged in a frantic, never-ending battle to make sure that the search results we get are the ones we expect.
The battle is not new. But over the past few months, for perhaps the first time in the 13 years since Google revolutionised the way we surf the web, it has felt like Google is not always winning.
Perhaps you have noticed. Useless web pages are appearing high up its search results, pages from never-before-seen websites that promise the information we are looking for, yet seem to contain only ads. At its worst, it is as if the spam filter has been flung open and all the monstrous emptiness of the web has been revealed. Google admitted: “We’re not perfect.”
What is going on? The bottom line is that Google is a victim of its own success. It dominates the web. To get traffic to your webpage, you have to appear high in Google’s search results. Meanwhile, spending on online advertising passed spending on newspaper ads last year and is set to grow by 10 percent this year. Since advertisers pay per page view, or per click on an ad, the spoils from clawing your way up the Google rankings are high indeed.
There are huge incentives to try to game the system – which is why there are ever more parties trying to game the system.
Google says its search quality is higher than it has ever been, in terms of relevance and comprehensiveness. At the same time, it admits it is going to have to tackle new phenomena, such as blogs that automatically plagiarise other people’s content just to steal their traffic, or so-called “content farms” which produce vast quantities of low-quality webpages.
“As pure web spam has decreased over time, attention has shifted instead to ‘content farms’, which are sites with shallow or low-quality content,” Matt Cutts, Google’s principal engineer, wrote.
“In 2010, we launched two major algorithmic changes focused on low-quality sites. Nonetheless, we hear the feedback from the web loud and clear: people are asking for even stronger action on content farms and sites that consist primarily of spammy or low-quality content... .”
Google already tweaks its algorithm two or three times a day. It’s as if the scientists at Coca-Cola had to come up with a new secret formula every few hours. The worry is that the internet is changing around it, and that Google itself is the problem.
The system that Google’s founders, Sergey Brin and Larry Page, invented blew away the less sophisticated search engines that existed before. By judging the importance of a page by counting the number of pages that linked to it (and judging the importance of those pages), they changed the way searching worked. Ever since, web publishers – from porn merchants to multinational media giants, and from individual bloggers to corporate brand managers – have spent their time working on “search engine optimisation”.
This has marketing departments creating “landing pages” buried inside corporate sites, to hit all the different possible permutations of a search query relating to their industry. Bloggers linking to each other. A whole army of advisers to tell publishers where to put what keywords on a page.
And now, content farms. Intriguingly, one is on a roadshow of investors right now, ahead of a New York stock market flotation this week – a flotation that got a last-minute spanner put in the works by Cutts’s blog post.
Demand Media, founded by Richard Rosenblatt, the boss of MySpace who sold the social networking pioneer to News Corporation, has created a mirror-image version of the Google algorithm, which suggests a page that would do well in search results – and then builds it. The company and thousands of others, has been criticised for producing low-quality content often barely better than spam.
Demand Media is trying to raise $58m (R409m). Analysts think Google’s hint of a clampdown could hurt demand for Demand Media shares. “If Google tweaks its algorithm in a way that causes Demand’s content to fall in the search rankings this could decrease traffic significantly and lead to material revenue losses,” Rory Maher, of Hudson Square Research said.
Such is the power of the Google algorithm. The company doesn’t just order the internet for us. It shapes it. - The Independent