Occam's Razor

Google pretty much owns the search engine market, but why? Perhaps it was because they were the first to do it well. In the old days of search engines when we were forced to use sites like Lycos and AltaVista, finding useful stuff on the web was excruciating, requiring you to go through many pages of results (and usually wait … these were back in the connect-by-modem days). Google figured out how to turn relevance into an algorithm. Basically, the more sites that link to a page, the more relevant it is.

There’s more to a search engine than that, of course, but that was the big innovation. And not surprisingly, once that was figured out other search engines figured out how to do this too. For most search engine queries, you will get a set of similar results.

But with Google search, you get more, but in this case more maybe less. Specifically, what you get is what Facebook figured out, probably after seeing what Google was doing. Google will watch your behavior closely and give you more of what it thinks you like. To do that of course, it had to learn a whole lot more about you. And most of us are happy to comply, since most of us are logged into Google accounts. Even if we are not, given that we leave cookies in our browsers, not to mention IP addresses, so Google can usually figure out it’s you. With every search engine query and use of its services it learns more about you. Google probably knows you better than your spouse does.

The downside is that, like Facebook, you end up in a filter bubble. Google knows if you swing left and so your queries are unlikely to show links that swing right, at least not on the first page. In short, Google and other search engines undercut its own quest to provide relevant search results by providing you links to stuff you are more likely to click on. They are relevant as long as you want results that reflect your biases.

It’s a profitable strategy for Google. It just watches you behind the scenes. Its powerful algorithms give you more and more reasons to invest time with Google and its many services. I plead guilty because like almost everyone I have a Google account. I use GMail extensively. I long ago stopped using an email client. I do all my email using GMail’s web interface. I do turn off the marketing (thanks, Google!) and, gosh, with all that space and it’s amazing search interface I can find pretty much anything in my email in a few seconds spanning more than a decade of use. Since I have lots of clients, it’s quite a value to do things like easily figure out what their issue was six month ago. I’m willing to pay to use GMail, if I have to.

But I don’t have to use Google search. Since discovering DuckDuckGo, I rarely use it for search anymore. That’s because DuckDuckGo is kind of retro in a way: it provides more relevant results by getting you out of search personalization. I’m getting more relevant results with it. Moreover, DuckDuckGo doesn’t track your usage. It doesn’t know who you are. It doesn’t follow you around with ads. It just gives you highly relevant search results with incredible speed. I can hardly press the enter key before I get a page of results.

My result is now a lot of highly relevant content I wasn’t seeing before, mainly because it wasn’t on the first page of results. Google was figuring I didn’t want to see this other stuff, but I do. Actually, it was feeding me links that matched my biases, hoping I would stick around. With DuckDuckGo though, searching is becoming more useful again. Moreover, I am learning stuff by reading sites I probably wouldn’t have otherwise. I’m becoming more informed, getting actual news and perspective. Real news after all tells you objective truth, not selective truth. I’m moving out of my own filter bubble and into a wider world. I am gleaning actual insight. It’s neat and kind of humbling. But it shouldn’t be. Rather, the question should be why did Google and other search engines put us in this bubble in the first place? Of course: they were chasing mammon, putting their best interests ahead of yours.

To use DuckDuckGo regularly, you have to change your default search engine. It’s not hard to do; it’s just something we don’t think about because for most of us we can’t imagine there is something out there better than Google. You’ve been sold a bill of goods that isn’t quite true.

Try it for a week and tell me I’m wrong. I’ve been using it for a few months now. I can’t see ever going back.

For more reasons of the virtues of this search engines and some of Google’s slightly evil side, watch this video: