Occam's Razor

It’s no secret that my blog’s hits are way down compared to five or ten years ago. Trying to figure out why this is has been hard, and hasn’t been aided by my general apathy. There are days when my hits are in the single digits.

I recently saw a one-day spike of 97 page views, which turned out to be one person skimming lots of my pages. With 2038 posts and content going back to December 2002, you would think that I’d be getting a lot of traffic because I have a ton of content. But I’m not. Most of my stuff is being ignored because search engines don’t see it as relevant anymore.

At least that’s what I’ve learned in my latest research into the issue. Crazily, this site would probably get a lot more hits if it were a lot smaller. What would be left would be mostly my most accessed pages, i.e. the “relevant” ones. Merely keeping the old blog posts around, or posts that are very dated or rarely read, is apparently keeping many people from finding my site in the first place.

It wasn’t always that way, but over the years search engines (and Google in particular) have been concentrating ruthlessly on relevancy. If they don’t see your content as relevant, then for all practical purposes it will sit forever at the bottom of their search index. So if you are on the web because you enjoy encountering serendipitous content, well, a search engine can’t help you. In fact, it will get in your way. Even worse, there’s no easy way to discover serendipitous or random content. In their holy and obsessive quest to highlight only relevant content, I think search engines actually become less useful.

The exact process search engines use to determine relevancy is not known, but the general outlines are understood. If your content is newsy, i.e. topical, i.e. recent, it will rank higher. This is because in most cases people are searching for answers to a problem, so what’s current is more likely to be relevant than older stuff. The more sites that link to your content, the more relevant they think it is. It’s the 21^st century equivalent of being popular by popular people saying they like you. Curiously, the algorithms search engines use are smart enough to figure out fake popularity. If you have a campaign to convince other sites to link to your site if you link to their sites, search engines will notice this stuff, and assume you are trying to game the system.

Yet Search Engine Optimization (SEO) is all about gaming the system, i.e. convincing search engines that your content is relevant. The SEO industry is huge, with lots of shysters out there offering to boost your site’s search engine ranking. While the broad outlines of getting better search ranking are pretty well known, no one can say for certain if a given strategy will work. Moreover, search engines are constantly tweaking their algorithms on the quest for even more relevant search results.

The result of all this is that site owners become like hamsters on a wheel, engaging in what is often a fruitless effort to boost their search ranking. Of course, organizations and companies that can afford to do so can achieve some success. It’s like lobbyists buying political influence in Washington: your site too can be more widely read, if you bring the right amount of bread or hire people with enough expertise in the SEO field. Or you can pay sites like Google directly to highlight your site using their Adwords program.

And there are plenty of people making careers out of this stuff. They pore over their daily Google Analytics reports. They attend SEO conferences. They watch YouTube videos on line on how to boost their search rankings. The tradeoff though is either time or money, and since how you spend your time effectively is money, it’s largely the same thing. You can probably boost your site’s traffic if you are willing to spend the time on it, or hire people to spend the time on it on your behalf.

So in trying to be helpful, search engines are creating an unequal playing field, providing increasingly tailored search results and giving a bias to those with time and money. It gets ridiculous sometimes. Google learns so much about you from your Google account and previous searches that it returns results that are perhaps too skewed toward your biases, making you miss important stuff like perhaps actual fact-based news. I wrote about this recently when I compared Google’s search to DuckDuckGo’s search. I found DuckDuckGo’s search results a whole lot better than Google’s because I was outside of Google’s filter bubble.

Even DuckDuckGo though is still on a quest for relevant search results. It has no serendipitous search capabilities either. Perhaps some search engine will find some niche market in returning serendipitous content. Then maybe my search rankings and site traffic will go back to the good old days.

One of the steps to returning to those good old days for me though might be to move the old and unsearched content to a new domain, say occams-razor-archive.info, or just purge it never to be seen again. I’d have to be careful not to actually link to a new archive site though, because that too could lower my site’s search rankings. And I’d have to disguise this blog into something it is not, and spend a lot of time tweaking it so that search engines will pay more attention to it. Probably the best way to increase my page hits would be to start a completely new blog, and fine tune it with punchy short paragraphs, post of no more than 500 words (so viewers don’t lose interest), always using an active voice and creating post titles that are carefully researched to get high interest.

I doubt I will do that. That takes a lot of energy I don’t have. Frankly, I sort of resent the search engine system we are forced to use. I disagree with the way it doesn’t rank the breadth of my content as important, when it should. There’s nothing I can really do about it though other than join this game, which doesn’t interest me. So this blog is likely to rank lower and seem less relevant in the years ahead.

But you can help me fight back. If you like what you read, then bookmark my site and come back to it regularly. Add it to your feed. Tell your friends, “Hey, I found this neat little site on the web!” Use the handy form on the sidebar (or off the menu if using a mobile device) to get emails when I make a new post. Show Google that you value longer and serendipitous content like my site. Maybe in time they will learn that in chasing the holy grail of relevancy, they are effectively hiding a lot of hidden gems across the web.