Reading The Wall Street Journal For Free Despite Its Google News Cloaking

In Read The Wall Street Journal For Free, I explained how to get Wall Street Journal content for free via Google News. But in the past few weeks, I've noticed a problem. You can't find some of the articles that the Wall Street Journal publishes using Google News, not if you look for them using their exact titles. And maybe I'm wrong, but I kind of get the feeling the Wall Street Journal has done this to have all the benefits of Google News traffic without people deliberately using Google News as a way past the paid barrier.

Here's an example. Via Techmeme, I see this is a popular Wall Street Journal story today:

When I click through from Techmeme, I can't see the entire article -- I get that "Free Preview" message instead. No problem. I go to Google News and search for it by its exact title. And when I do that, it's nowhere to be seen -- even if I do a phrase search, where I'm ensuring that I get only pages that have all those words in that exact order.

It should come right up. The WSJ is a Google News source, and this is the exact title of one of its articles. Wazzup?

To find the article, I have to do something different. I have to search for key terms that I know are in it, like this:

See that source: part? That restricts the search to the Wall Street Journal. And the first listing that comes up is this:

Now click on that link (from Google News, if you want to see the entire thing), and you get this as the headline:

See? Different headline from what Google News is listing. That means a different headline is being fed to Google's news search than a human sees. In the SEO world, we call that cloaking. Except for Google News, where we call it "First Click Free." This from Search Engine Land explains more about that:

If I wanted to be all snarky and controversial, I'd write (over at Search Engine Land) about how the Wall Street Journal isn't making use of the First Click Free program's allowance of cloaking in the way it is supposed to be done. That's because to my understanding, I should as a reader see exactly the same content that Google's spider saw. I don't. I see an article that has a different headline.

Cloaking! Real cloaking, allowed by Google. That's not what bothers me. What upsets me is that the cloaking is being done because, from what I can tell, the Wall Street Journal wants to play both sides of the fence. They want the free traffic from Google, but they don't want the hassle of figuring out how to properly "trap" second clicks -- and they also don't want people to "abuse" getting their free content on Google News. So they feed out different headlines, I'm guessing.

If so, lame. In fact, the entire thing is lame -- there are so many ways to get Wall Street Journal content now for free that the pay barrier should go. And I say this as someone who actually does pay for access. It's just easier and less hassle for me to get in through other methods. At the very least, let readers of Techmeme click through without a pay barrier.

Interesting observation, which I intend to look into in some more detail tomorrow.

There is a possible alternative explanation:

Once an article has been fed into Google News, its in! It doesn't get re-crawled, so any subsequent changes to the article wouldn't get picked up, unless the article was resubmitted on a new URL.

I might try a wider sample to see how things stack up.

Comment by Adam C | May 15, 2008 11:06 PM

Although Adam C. might be right it could also be possible that they have a title for search (a title for spiders) and a headline for the users (more journalistic styled). And perhaps they have chosen to provide the title instead of the headline in their Google News Sitemap (I assume they have one).

Greetings from Madrid, it is always interesting to see the sources in action.

Comment by Dictina | May 20, 2008 7:54 PM

Danny, you have got deep insight into things related to search industry. Your blog is enriching and empowering.

Comment by varinder | May 29, 2008 6:30 AM

Want to comment? If you are signed into TypeKey, you'll see a form below. No form? Click on the sign-in link below, and you can sign-in or sign-up for a free account. Sorry you have to use TypeKey, but I use it to avoid comment spam. All comments currently appear automatically after posting.

Leave a comment