You would be surprised (and shocked) by how many people use Google as the major or even sole information source for daily business. In the worst case for business-critical decisions. And many of those are actually convinced that Google serves their needs. The problem simply is that you never know what you do not know.

OK … no bashing! … Google is not bad. Assumed that you always keep in mind what kind of tool you are working with and use it properly. Google suggests to give an answer, but factually it does not. Google only provides sources (= webpages) of answers.

But there is a limitation with the ranked sorting of the Google results. The hits on top of the list neither necessarily give a reliable answer nor a comprehensive answer nor the right answer at all. The webpage with the best answer (or the right one at all) can be hidden somewhere on page #5 or #11 of the results. And – let’s be honest – with many searches people do not go beyond page 1-2 of the Google results.

This is no issue when a you are looking for a movie you would like to see with your girl friend, or the map of the zoo, or the true age of Lady Gaga. But it is bad for business and potentially threatening the existence of a company when decisions are based on superficial Google use. You give the responsibility for your business … to a biased ranking algorithm not under your control.

Well, this constraint and menace is not new. But sometimes people take the bait to disregard, owing to a general tendency to concede time pressure, to oversimplify and to prefer quick answers instead of sound ones. But now, the well-known “soft” evidence once again has been proven by hard facts. A recent study by Nadja Hariri on the patented Google PageRank mechanism found that …

• documents ranked by users as most relevant were on positions 5 (most), 1, 2, 18, 20, and 36 – so, do not trust the ranking
• the mean values for the precision for most relevant documents were nearby between 38.82% and 31.18% on pages 1-4 – so, do not ignore results down the list
• altogether, the correlation between user’s and Google’s rankings was rather low
• “it is better for users of the search engines, especially Google, to examine at least three or four pages of the retrieved results”

Hariri’s conclusion: “users evaluate retrieved information in such a subjective way that search engine ranking cannot be in complete accordance with their views of relevance”. Search tools are simply far away from being able to equivalently replace human brains. While a decade has gone by, Hariri still is hooking up with Hawking et al. (1999) who stated that “the standard of document ranking produced by public web search engines is by no means state-of-the-art”.

So, watch out! Do not trust search engine rankings. And take care that Google is not taking your business into his hands.

P.S.: Please allow me to anticipate expected criticism. Could those results be biased by surrounding cultural conditions? The study was done at one location only. And – this frank note has to be allowed –  at a location not expressly known for goodwill regarding the US including US-based services. Well, sure, some influence can never be completely excluded. But Hariris’s findings are support by a series of earlier studies. And, by the way, it is also in line with my own decent but extensive experiences.

• “Relevance ranking on Google – Are top ranked results really considered more relevant by the users?” by Nadjla Hariri in Online Information Review 2011 35 (4), pp. 598-610. (online access needs subscription)