Dear WSJ: To Avoid Google Disease, Please Put A Condom On Your Content

I’d thought I’d heard it all in the debate over Google and newspapers, but yesterday Wall Street Journal managing editor Robert Thomson took it up a notch. He accused Google of making people slutty. If we’re using sexual metaphors now, here’s another one. Why doesn’t the Wall Street Journal and News Corporation in general put a condom around all of its content, to protect itself from Google? There’s a good brand called robots.txt that will help.

The robots.txt condom

The accusation was lobbed at yesterday’s Web 2.0 Summit session called “Wither Journalism.” Be sure to watch the video, which I’ve listed at the end of this post. I’ll also link to key sections of the video, as well. It’s a stunning contrast between two cultures, Thomson and Google’s vice president of search products and user experience Marissa Mayer, who was also on the panel

Google’s goal, Thompson said, isn’t really to send people from its site to other places. In reality, Google seems to have a nefarious plan to keep everyone on its own site:

Google wants to be the home page or wants to be the front page, and Marissa unintentionally encourages promiscuity. It’s about digital, the whole Google model is based on digital disloyalty. It’s about disloyalty to creators.

There was a murmur through the audience as the promiscuity reference that I’ve bolded was put out there. Even moderator John Battelle exclaimed, “Wow.” I’m not sure if the reaction was due to a new crass low point being reached in the overall debate about Google and newspapers or whether so many people likely disagreed with Thompson.

Somehow, in Thomson’s mind, Google has trained people that they don’t need to read newspapers at their own sites. This is despite the fact that Google didn’t even have its own news product until 2001. That it wasn’t the first news aggregator out there. That it wasn’t even the first search engine out there.

Hearing stuff like this, it’s hard not to agree with Google CEO Eric Schmidt’s assertion that Google gets blamed for changes wrought by the internet itself (see my Google CEO Eric Schmidt On Newspapers & Journalism article for more on that).

Thomson had also said earlier:

Having a debate about it obviously is important, which is clearly what we’ve been doing, provoking and stimulating and maybe unintentionally insulting.

Congrats. You were insulting. Not only did you say that Google encourages people to be news sluts, but you also accused many good people of being “net neanderthals:”

There are three types of people in the world.

There are the net neanderthals who think everything should be free all the time.

There are people who pay for ISP, 19.99 a month of whatever, and actually a lot of them think they’re getting access to content buffet. [I think that’s the amount he said, hard to hear]

And there’s a third group of people who are already on top of that paying for content.

Let’s skip the fact that there’s some debate on whether neanderthals were actually idiot savages. Really, if I expect some content for free, I’m some type of brute?

Last night I watched an episode of The Simpsons, broadcast on Fox, which is owned by News Corp, which owns the Wall Street Journal. It didn’t cost me anything. Sure, I watched it through Direct TV, and I pay a subscription fee for that. But it’s also broadcast over the air for free. My TV with an ordinary antenna would have picked it up. Later, if I wanted, I could have watched news content from Fox for free on the same channel.

Why am I a highly valued viewer in the “real” world but on the internet, if I read something for free at the Wall Street Journal, I’m some type of savage taken down the wrong path by Google?

Why also is the Wall Street Journal itself feeding those savages? Over at Nieman Journalism Lab, earlier this year WSJ executive editor Alan Murray explicitly talked about how and why the WSJ deliberately puts stuff outside a paywall. Be sure to watch that video and read the interview summary. At one point he says:

If it’s a big news story, if we report a takeover and — we could hold that behind the pay wall, but if we do, BusinessWeek or someone else will simply write a story saying ‘The Wall Street Journal is reporting x,’ and they’ll get all the traffic. Why would we do that?”

Note that it’s BusinessWeek that gets mentioned there, not some blogger, not some aggregator but another mainstream media outlet that could potentially do what mainstream media outlets have long done to each other, report on what each other is saying. But no one calls them out on that these days. It’s the aggregators, the search engines and the bloggers that get attacked.

I don’t know what research Thomson has to back his “three types of people” model. I think things are more subtle than that. I think there are plenty of people who will pay for stuff even if they’re used to expecting lots to be free.

In the real world, we have free and paid. On the web, we can have it as well. Paywalls DO have a place, and I’m not opposed to them. But the puzzling thing is that the Wall Street Journal has used its own paywall very well to both make money on the web yet also still get people from Google. They have their cake and get to eat it too. It’s a head-scratcher. What exactly is Thomson so upset with Google for?

Let’s go back to Thomson’s suggestion that Google is trying to keep traffic within its own site. Just after talking about Google’s “disloyalty” to creators, he discussed a unique feature of Google News that shows quotes. He challenged the audience to search for Hamid Karzai to see how Google fails to promote clickthrough:

In tiny, tiny font, you’ll see where the origin of those stories are …. there’s is absolutely no intention on that page to drive traffic.

Bring it on. First, most people are probably searching for hamid karzai at Google itself, not Google News. They’ll see this:

hamid karzai - Google Search

At the top of the page, three articles are listed. Those are smaller than the main link to the overall news results “News results for hamid karzai,” but they’ll still pull traffic. In fact, I challenge Thomson to give us a case study. It’s easy enough for him to know when he’s featured in that section. Is the WSJ really not getting traffic?

Also prominently featured are links to non-news sources. Fair to say that not everyone searching on Google for “hamid karzai” is after news content?

Now let’s drill into those news results. See? Google’s trying to keep people within its own site by doing that, driving them into Google News!

Sure, you can argue that. But you can also argue — and I will as someone who’s watched search habits for 14 years now — that the bigger issue is that people who DO want news content fail to actually go to a dedicated news search engine. So that link is part of the overall Universal Search change designed to help surface the right content for them.

On the news results for hamid karzai, you get this:

hamid karzai - Google News

Now you can see what’s ticked Thomson off. To his eye, the top of the page is dominated by a quote, with the entire intent being to pull you from that into a further page of quotes (as I’ll show) rather than send you on to news stories. That’s to the eye, by the way. A search marketer would know that the actual result is much different.

The quote feature is just over a year old. My guess is that most people bypass it, moving down to click on one of the actual news stories shown. I say that because typically, things that do not look like “real” results usually get little click through (ask anyone who runs search ads about this). As I’ll point out, Thomson — if he wants — can tell us if he really gets no traffic from these or not, in relation to the amount of traffic he gets overall from Google.

How about that font? Look below the quote, and you’ll see Reuters is credited (and overall, Google’s found 120 occurrences of that quote on other news sources). The credit isn’t “tiny, tiny” to me. It’s maybe one size lower than the quote itself.

Still, that’s not the actual font he’s so concerned about. See how there are two arrows in the screenshot above? The one to the right points to a link that lets you get to a page of quotes from Karzai:

Google News quotes

This is the page that Thomson was so upset about, the one he says where there’s absolutely no intention to drive traffic, in part because of the “tiny, tiny” font used to source the publications. In reality, the source font is identical in size to that used for the quotes.

Now, I’ll readily admit, most people who make it into a quotes page like this probably aren’t clicking off to the news publications. But then again, I think most people at Google News aren’t heading to these pages at all. Most of them are probably clicking on actual news stories, since that was their main intention for doing a news search in the first place.

Also keep in mind that quotes like this tend to show high on a page mainly when you search for the name of a newsmaker. Let’s look for a story was on the front page of the Wall Street Journal as I wrote this, about Nokia suing Apple. When I do that search, I get this:

nokia sues apple - Google Search

See the arrow? There’s the Wall Street Journal listed at the top of the news unit. As I said, the WSJ can measure the traffic they’re getting off links like this. How much free traffic did they get?

Let’s drill in:

nokia sues apple - Google News

There’s no quote shown at the top of the page, because we’re not searching for about a newsmaker. Instead, what we get is the Wall Street Journal ranking at the top of the search results. And trust me, anyone who runs a news site will tell you being in that spot is sending you lots and lots of traffic.

If Google’s intention was really to keep traffic from papers, papers would see a trickle, not the flood they get. And Thomson either knows this is true for his own publication or deliberately prefers to put out misleading information.

After hearing Thomson make more accusations against Google, I’d finally had it and thought (again) why doesn’t the Wall Street Journal (and all of News Corporation, for that matter), just block Google? It’s easy. You put up a robots.txt file, and your content is out of Google.

At that exact moment, I cheered John Battelle from afar for asking that exact question:

If Murdoch’s so mad about this, why don’t you just put in robots.txt, don’t crawl?

And Thomson responded:

We can do that any time we like. But obviously we’re at the, the idea that the web has entirely evolved is ridiculous. You’re asking that question as if we’ve reached the epitome of our life, and we know that that’s not true. I think one of the questions for Google is, is the very definition of the verb to Google going to change over the next couple of years. Will it mean in the end you’re sort of rummaging around [something] content?

I couldn’t catch the exact word he used at the end of that quote. But the implication was that Google again had done something wrong. And the main point is that Thomson didn’t answer the question he was asked. At all.

Thomson took a simple question — why don’t you block — provided a non-answer and then spun things back on Google. Well, you can’t say he’s masterful.

I say, enough the hypocrisy. Let’s go back to the sex metaphor that Thomson kicked things off with.

A robots.txt block is like putting a condom around your entire web site. It keeps Google away. So if Thomson really thinks things are so promiscuous out there, have the WSJ practice a little safe news sex. Roll that robots.txt file on.

Otherwise, stop prattling on about how wronged you are by Google. The WSJ and other News Corporation sites are knowingly having unprotected indexing relations with Google. Don’t then complain that you’ve caught some type of neanderthal disease from it. Especially, when it turns out, all that’s really happened is that Google keeps knocking you up with millions of visitors.

Sadly, and I do mean this with all sincerity, underneath there are signs that Thomson is actually a reasonable person. That he doesn’t have completely crazy views. That when he’s not singing from what appears to be the hostile Murdoch songbook, he could actually be a partner for sensible change rather than as a poster child for the “newspapers are dinosaurs” crowd.

At one point in the session, he said:

We need to have a rational debate that’s almost platform neutral about professional journalism. Because in the end it’s not so much the means of delivery but the means of creation. And it’s that very issue that is the sort of thing that we should all be talking about. And it doesn’t mean that professional journalism is the antithesis of citizen journalism. There’s almost a false dichotomy there.

Agreed. So let’s have that, please. A rational debate, without accusations that aren’t backed up, which don’t hold water.

For more, see these reports of the session:

You can also watch the entire video here:

Condom picture modified under Creative Commons license from here.


  1. says

    Sexual metaphors are always fun…
    On the web, journalists can build a “reputation” and have “regulars” who follow us, but that is about as “slutty” as we get because we work for ourselves.

    Tell me this, is it more slutty to do the same work and then turn around and pass the buck on to a pimp (looking at you Murdoch and Thompson), or is the web an evolution in journalism?


  2. says

    Great post! I generally agree with you; however, I empathize with how stressful it can be for print (or even web) content creators to see the gradual creep towards more & more content being shown within Google’s interface.

    That said, it’s up to the news providers & book writers out there to evolve accordingly, just at the music industry had to get it together when it comes to music-sharing. Can WSJ provide something of value on their site that can’t be replaced by a snippet on Google? If not, then they need to suck it up and realize they’re in a classic symbiotic relationship with the big G. Sure, Google can be a bit parasitic, but ohhhh how you love to get that traffic.


  3. Nick says

    I use to agree with you 100% on the relationship between Google and these news outlet. But I am starting to see it more and more from the news outlet’s point of view. That’s not to say I agree with them or can stomach their hypocrisy; but I see their point of view.

    1) Google has a near monopoly on search.

    True, News Corp can fill out robots.txt and remove themselves from Google’s index. But than their traffic will plummet. Google = Search = Traffic. News Corp has no leverage in dealings with search engines, because Google is essentially the only player.

    2) I don’t think it’s about links as much about brands.

    Apple makes the record labels a lot of money. But the records label hate them. Why? Because Apple has interjected themselves between the labels and the consumers. Apple controls the labels interactions with their customers. I think that’s what News Corp is really afraid of happening. Their brands will be eroded to nothing more than an article at the end of a Google News link.

    I think Google is the unfortunate target since they are rich and monopolistic. But apply the logic to any aggregators – Google News, Huffington Post, Drudge Report, etc. These sites don’t produce news (I know Huffington Post has started to, but ignore that for a second) but they become the source of news in the consumers eyes. Huffington Report can be appear to be a full service news source w/o actually producing content. Huffington Post becomes the valuable brand that people go to for “news” and the papers are left to try to eek out an existence with ads on individual articles.

    Google knows better than any site that one of the best ways to make money on the internet is to keep eyeballs on your web properties as long as possible. And although they send alot of eyeballs to the WSJ and other newspaper, they are keeping the portal to themselves.

    Now, do I think Google and Huffington Post are doing anything wrong? No. The internet is a wide open place and you can build a property linking to news articles; more power to you. And do I think newspapers are whining? Yes. Google or not, their brands are going to erode.

    The only way New Corps and other newspapers will compete is by building their own destinations on the web. To convince consumers that is a better news site than Google News. That’s a much better than the other crappy ideas they’ve had that you’ve documented.

  4. says

    Nick, on the traffic front, agreed. They don’t want to lose the traffic. And since they don’t, apparently that traffic DOES have value to them and they are receiving it. But to hear the WSJ execs tell it, Google devalues everything it touches and is a vampire that is sucking the live from their publications. Plus, the reality is that they get the traffic AND the paywall. That’s how it works right now on the Wall Street Journal. They get traffic despite having a paywall.

    I agree, News Corp is worried that anyone can fill their shoes. That’s really part of the digital loyalty aspect that Thomson is worried about. Google (along with many other sites) lets people choose what they effectively want to watch, just like cable television does. If you want me to tune into your newspaper, then you need to give me a compelling reason to be there.

    The WSJ has lots of compelling content. But there are other publications out there, as well. And much as Thomson might like to believe, the WSJ does not cover everything I want to know about. Nor does it cover everything as well as some other publications out there. My reading habits, and those of others, aren’t going to change to suit his outdated idea that my day should begin with the WSJ.

    For me, personally, my day begins with Techmeme. If good WSJ stuff is there, then I’ll visit the WSJ. My day continues with Twitter, where if I see it passed along, I might also visit. My day usually ends with a hard copy of the WSJ that I pay actual hard cash for, where if I have time, one of my favorite things is to sit with the paper and read.

    For Thomson, Google seems to be an enemy that steals his rightful readers. In reality, Google is a distribution partner — one that unlike cable isn’t charging him money, either.

    As for the aggregation blogs that comment on what others are reporting, like the Huffington Post does often, agreed — I can see that being irritating. But I also know those sites can send a publication traffic, too (just as my own Search Engine Land site both sends and receives traffic, from articles we summarize and to our own original content).

    Unfortunately, I think there’s plenty of “mainstream” news that’s coming off of blog mentions that never get reported that way. Or when mainstream articles tap into the expertise of sources that run blogs, quite often they don’t link out (not because of the reporters but because that’s just now “how things work” in mainstream publications).

    In addition, I’ve never seen the print journalism space fight against the constant television ripoffs of their reports in the way they fight internet publications.

    A “Hulu” destination for news could be compelling, if the papers can get it together. Even more so if they include some of the non-mainstream publications that also produce content.

    But they also do have destinations right now. And Google sends those destinations plenty of traffic. They just haven’t figured out how to make more money off those destinations after banking so much on ads and getting hit with one of the regular downturns ads take. Will they complain about all the free traffic they get when the ads pick up again? Doubt it :)

    One thought is that I’d love to participate in a wire service that allowed me to quote from a variety of good publications to my readers at a reasonable cost. Don’t know of anything like that out there. Couldn’t get the AP to respond to my requests on tapping into their wire. I’d like that even if my quoting is already fair use just because I know there’s more that I can do if I license material and because I’d like to contribute to the quality journalism I know is out there.

  5. says

    I see that Danny copied it correctly… so, is the title supposed to be “Whither Journalism” — or is “Wither Journalism” the most horrific pun Google has ever let slip? :-)

  6. lwy says

    I really want to read the entire article, but the badly flashy ‘congratulations you are todays 1m user…’ ad is so distracting. My eyes hurt. =(

  7. says

    Yeah, I shifted to a new ad system this week and am diligently shutting down some of the more extreme ads that are in there. Sorry about that. I got caught off-guard myself when an audio ad started playing on my own site. I hate that! And I inadvertently caused it!

  8. says

    In many ways, when Robert Thomson talks about Google he is talking about the effect of the Internet as a whole but Google makes for a more precise metaphor.

    And he is right in that Google increasingly tries to be a bit more stickier every year, and with new products like sidewiki, etc.

    At the end of the day, GOOG is piggybacking on other people’s content. The headline and first para are the top two most valuable parts of news content, every para after declines in value.

    How much value is there in the remnant traffic that GOOG sends to WSJ or other news sites. Very little. Everyone likes to talk about “all that traffic” Google sends to news sites, but no one talks about the value of that traffic and the ability of newspapers and news sites to monetize that traffic.

    Do you get paid depending on how much traffic you get to your blog? If you did you’d starve. You get paid because of all the other things that you do. News journalism cannot survive on traffic it generates, that’s the horrible truth and that’s why people like Robert Thomson say things have to change. Neanderthal might be a strong word but believing all content will be free because it is currently free is backward thinking, and it is foolish thinking.

    BTW, thanks for pointing out this panel. And in the interests of full disclosure Robert Thomson was editor of the US Financial Times when I worked there and I continue to have a lot of respect for him.

  9. Thomas P says

    I agree with Nick. I am no mainstream media fan, but I think we are giving Google too much of a free pass here. Aggregators are sucking a lot of the value, and quality content creators are at a huge disadvantage here.

    Eventually, as the value accumulates to aggregators, some of these aggregators will end up owning dominant content outlets, and that ownership would have been wrested unfairly. At that point, we are going to whine about the monopoly of content and distribution being together, and these are going to be massive internet-scale monopolies.

    We can see this in internet video. My prediction is that YouTube will leverage its unfairly gained distribution advantage (copyright content aggregated) and pretty much come to dictate terms to content providers.

  10. Peter says


    I am a webmaster of a web site with valuable content. If things are so simple, would you care to explain to me how do I allow Google to index my site for its search engine (thus sending me traffic) but *refuse* it the opportunity to integrate it in the ever-growing list of Google “services”, such as Places, Maps, Local Search, etc where Google helps itself to various information culled from sites, pictures and other “structured” info? (And since recently without even linking back to the source, at least for pictures!) Perhaps I am missing something, but I kind of view *that* use of data as theft. I didn’t sign up for that. And I would like to rectify it via robots.txt by saying “no” to rehashing of my content. Again, I am fine with and actually encourage web indexing.

    When the web was invented, the “social contract” was simple: the search engines would have the right to index content, in return sites would get traffic. I fail to see how allowing Google to do whatever the heck the want with content fits that model.

    Anxiously awaiting your response.

  11. says

    I’ve met Robert Thomson and I’m having trouble imagining him using the word “slutty”. Is that a direct quote?

    Or did he actually just say “promiscuous” (which is in the direct quote)? This is quite a different word to “slutty” and doesn’t hold a wholly sexual meaning. It also just means “indiscriminate”. I’m guessing you know this already and I’m 100% certain that Thomson does.

    Not that I disagree with you on the substantive argument about whether publications should put their money where their mouth is or shut up.

  12. coldbrew says

    would you care to explain to me how do I allow Google to index my site for its search engine (thus sending me traffic) but *refuse* it the opportunity to integrate it

    Peter: Do you mean how does one have its cake and eat it too? Do you mean you want the advantages of distribution, but you want all revenue that can possibly be derived from *your content*. I’m sure Bestbuy would like to have Apple’s hardware margins, and that Apple would prefer Bestbuy only have Apple products to sell.

    Are you people able to think objectively? Even for a short amount of time? Do understand the idea of cherrypicking?

    Foremski: Google’s job is not to help you figure out the value of your content or how you monetize. You seem to have some notion of entitlement.

    I may be missing something here, but I certainly have no dog in this race. What is wrong with you people? One cannot simultaneously reap the advantages AND belittle the methods.

    Simply put, content creation and distribution is not nearly as costly as it once was. Large bureaucracies are no longer needed for production and/or distribution. The sooner you realize that, the better your business will be.

  13. coldbrew says

    I’ve met Robert Thomson and I’m having trouble imagining him using the word “slutty”. Is that a direct quote?

    Caitlin: Was it written as to convey a direct quotation? Would you like to provide an inroad into any other non-issue? Your question has an answer in the content Sullivan provided (video or text).

  14. says

    Tom, I think there’s a lot more value in a news story than the headline and a summary. I also think that Google sends publishers much more than “remnant” traffic. For some publishers, Google is their top traffic source, if not one of the top ones.

    News journalism perhaps does survive on the traffic it gets now. Bear in mind some news publications actually purchase ads against search terms to get traffic. If they’re spending money on these ads, they’re making money off the traffic (or they’re pretty dumb for buying the ads). But many of them get far more traffic for free from natural listings. That’s one reason why many news publications have full-time people dedicated to do nothing but generate traffic from search engines. That traffic is valuable, not that Thomson mentions this.

  15. coldbrew says

    FYI: Google’s revenue is primarily derived from search advertising where the end user buys something. They have only recently gotten into display advertising and my understand is that is not yet lucrative.

  16. Peter says

    @ Coldbrew,

    > Do you mean how does one have its cake and eat it too?

    No, it’s two different cakes and using one to get the other is called being a monopoly. What I am trying to call attention to and what I think you’re missing is that Google has changed the fundamental “social contract” that had existed and for which robots.txt-type defense was originally developed. Robots.txt works for a search engine and I have no problem with Google-the-search engine. I like it. I do, however, have a problem with Google Places, Maps, Earth, etc etc, and do NOT want my data there simply because these service compete (or will compete) with what I do. THESE SERVICES ARE NOT THE SAME AS A SEARCH ENGINE because they are not designed to drive traffic to other sites (in fact, they are designed to keep traffic within Google) and therefore the naïve or, I’d say, disingenuous defense “but you’re getting some clicks from our index” is not valid and can not be used.

    Look at it this way: if a company – like WolframAlpha or similar started taking your data and republishing it as “answers” without giving you a link, what would you do? Unless they offered money, of course, you’d block them via robots.txt and may be via an IP block for a good measure. Hell, even if WA or a similar service offered a handful of clicks in return, they would be blocked. However, Google is in a different position: you can’t block it due to MONOPOLY they’ve achieved as a search engine. But the way it stands now, Google is out to drive everybody who collects any valuable data on their site out of business by republishing it without permission, using clicks from its search index as an excuse to do so. If they are so honest and well-meaning, let’s ask them to offer separate robots.txt for each new service they launch! Will they do it to prove that they are not evil? I doubt it.

  17. says

    Thomas, I know I can seem like a Google fanboy at time, but I’m not. I have issues with lots of things that Google does.

    The problem is with the newspapers that complain about them, what do they want? What does Thomson want? He complains that Google’s somehow ripping the WSJ off. But he doesn’t want to block them. He could do it at any time. It is totally within his control.

    I’m really tired of the debate. I want to get beyond the accusations and talk about what the solutions are. Does he think Google should be licensing the right to summarize headlines and summaries? That seems to be the AP’s view, not that I can get anyone from the AP to actually talk to me about this.

    If that’s the case, where’s fair use fall into this? Many mainstream publications also summarize content they find elsewhere. Do they fall under the same rules? And many mainstream publications are quite happy to depend on Google as a primary research tool to compile the articles they write (see Do Newspapers Owe Google “Fair Share” Fees For Researching Stories?. Should Google be licensing the right to summarize all pages that it indexes across the web? And if so, will the WSJ be paying a subscription fee to Google for the commercial uses it makes when its reporters use it rather than a service like Lexis-Nexis?

    I’m fine with not giving Google a free pass. I just don’t know what Thomson wants from them. I think he wants Google to write the WSJ a big check for the privilege of listing WSJ stories. OK, I can roll with that. But then will the WSJ in turn be writing Google a check for each click it receives? For the millions of clicks each day that Google already send it free of charge? For which, despite the WSJ’s protests, it is actually making some money off of. Perhaps much money?

    I seriously would like the debate to move on from the accusation phase to what the solution is supposed to be phase. We just don’t seem to be there.

  18. coldbrew says

    Peter: What you seem to be describing is exactly having one’s cake and eating it too. Obviously, I can’t claim to know whether Google will let you appear in their organic SERPs, but not anywhere else in their database. I don’t even know if their infrastructure would allow for such things. All I do know is that people whose livelyhood directly depnds on such things seem wholly unable to remain objective about the idea altogether.

  19. says

    Peter, I agree. There’s been a general movement across the entire search space (Google + Bing, even Wolfram Alpha) to extracting information and direct answers with the assumption that a source link is enough. That’s disturbing. The less people need to click through, the more I’d like to see sources pulling this information find a way to license or reward sites that generate these so called free “answers.”

    Most of your question really seems to be about local content. Well, Google actually licenses to gather information from third parties like Acxiom. If you buy a yellow pages ad, or you have a white pages listing, or you have some type of public record about your business, then you need to focus your upset on the parties handing out that info. Why does your Yellow Pages provider think the should make money selling you an ad and then reselling your listing to Google (and Yahoo and Bing)? As for public records, well, they’re public records.

    Still, I agree. As Google pulls from more and more resources, it needs to explain the ways you can remove yourself if you want to. One thing they did last year was launch a center for publishers (see Google’s “Submit Your Content” Page Changes Into Content Central, One Stop Shopping For Publisher Advice). I’d like to see them expand that, to make it even easier for publishers who want to opt-out to opt-out. Most publishers, of course, don’t want to. The WSJ is a case in point.

    Where are you finding they use pictures without linking to the source?

  20. says

    Caitlin, I’m paraphrasing him in the lead of suggesting people are slutty in their behavior. I don’t think that’s an incorrect paraphrase. From the link you provided, the first definition listed is:

    “Characterized by or involving indiscriminate mingling or association, esp. having sexual relations with a number of partners on a casual basis.”

    I’ve got the exact quote further in the story. Watch the video, and you can decide for yourself.

    This is the managing editor of a major news publication who ought to know words. He used the word promiscuous, and I think he well understood the connotations associated with that. He could have simple said “disloyal,” which still is charged, but not quite so much.

  21. says

    Speaking as a news consumer, OF COURSE I’m more slutty when it comes to news on the Internet. This is because I can get to the NYT and the WSJ and the Guardian and the BBC and CBC whatever other interesting news sources there are out there. Why would I settle for a cut-n-pasted warmed-over AP wire story in my local rag when I have access to some of the best journalism in the country?

    Tom Foremski,
    Maybe the problem is that news people need to learn to write for the web. If the headline and first paragraph are the most valuable and there is not compelling reason for me to click on your story past that, then you are either a bad writer, or you are reporting a basic fact like “so and so died”.

    Blogs have long learned the trick of Teaser + “More after the jump.” Maybe paper sites could learn to write for Google, make the opening paragraphs into pitches for why I should read further into your story. Shouldn’t be that new of a trick. I mean, that’s the game of headlines and pull-quotes used to sell paper out of the box on the corner.

    And honestly, it’s put up or shut up time. If you truly feel that the traffic from Google is low-value remnant traffic, then go ahead and pull the plug on showing up in search results. robots.txt will end all that Google “stealing” from you and the unwanted remnant traffic in one fell swoop.

    Ultimately though, this is all misdirected. Google isn’t your culprit, Craigslist is. You guys funded the journalism off of the sales of advertising and classifieds. Now we can get those for free, up to date, searchable and nation-wide. That’s the reason you can’t monetize all those hundreds of thousands of new readers.

  22. Peter says


    > Where are you finding they use pictures without linking to the source?

    Hmm, I thought I saw it in one local search, but I can’t replicate it right now. I guess without a proof, I am taking it back. (Although the “credit” line for photos is not linked, only the thumbnail image is… and I could’ve sworn it wasn’t before).

    Still, I am glad you’ve partially agreed with me that data on “answer-type” services should not be taken for granted.
    Sorry… forgot to say great post – can’t wait to read your next one!

  23. Michael Hill says

    Your analogy with watching a broadcast of the The Simpsons with reading a WSJ article via Google deserves another look. For one thing, if that article were behind the paywall, then it would be more akin to getting to see an HBO show for free, which many would consider some sort of piracy. But let’s say it’s outside the paywall. Well, the local affiliate would have a business relationship with Fox that would allow the broadcast of that episode, as would, in fact, DirectTV. Money would change hands in that transaction. Why doesn’t Google have to have a similar relationship? Even if, you might argue, the WSJ should be paying Google, just as the networks used to pay their affiliates.

  24. says

    Michael, Thomson suggests that that there are savage like people who just think when they go on the web, they should get stuff for free.

    In the real world, plenty of media outlets provide TV shows, radio shows and even print publications for free.

    I don’t think Murdoch, Thomson’s boss, has ever suggested that people watching his Fox television channel are savages that have unreasonable expectations because when the tune in, they expect the programs for free.

    Whether money changes hands behind the scenes to let the consumer have that material for free is beside the point. The consumer doesn’t know. And actually, money does change hands. That’s done through advertising. But again, no one suggests the consumer has somehow done anything wrong. But on the web, Thomson does suggest this.

    By the way, Google does have a deal that allows the Wall Street Journal to put its material outside the paywall. It’s called First Click Free. Go back up, click on that link where I said the WSJ gets to have its cake and eat it too. It explains more. The WSJ makes plenty off that deal. It gets traffic. It helps convert some of those visitors into subscribers. Google gets to list the WSJ content and have a better news search service.

  25. Peter says

    And if I were in the news press business, I would also consider the following strategies & tactics:

    * Insert a story in robots.txt to prevent it from appearing on Google for 6-12 hours. This way, people will gradually learn that stuff on your own homepage is always fresher than stuff they’re getting through an aggregator. If you can get the other guys to do the same, so much the better.

    * Offer your own aggregator (but not algorithm-based, put a human on the job) linking to stories beyond your site that COMPLEMENT your coverage. Looking at Google’s “clusters” of 100+ articles for each news item, I always wonder – how many of them are exact copies of each other and anyway, what’s the BEST one? I don’t want to read 100 to find out. If my newspaper not only did an excellent job covering a topic but also suggested to me 2-3 more articles that cover the same topic from a DIFFERENT angle, I would love that!

    * Stop writing generic content. 95% of stuff in NYT is predictable and can be replaced by reading the wire. Or Google News. Even if it’s marginally better written, it’s not that compelling. Krugman is interesting, whether or not you agree with him. Nobody goes to Google News to find Krugman’s articles. You go to the NYT and find it. Publish a directory of your writers on the FRONT PAGE. This is who you’re promoting.

    So, more personality, more opinion, more quality, keep Google waiting for a few hours and offer your readers a window onto the world beyond your newspaper.

  26. interval says

    Its simply more of the same loathing of new technology. The web has indeed changed everything; but not the way purveyors of mass media were hoping. I remember when I first started in IT in the late 80’s, my boss at that time wanted to try to make the office paperless. It took over 30 years, but its starting to happen. Newspapers are a waste of trees, and news gathering and editing is not quite the chore it was when physical paper was in play. Like the buggy whip, new organizations need to realize their time of charging premium fees for the kind of work they do is over. Its time to get a new career, not try to shame people into giving them the money they were used to “in the good old days”. I don’t have anything personal against anyone in the news media profession, but I know when to shoot a lame horse.

  27. MyTriState says

    The WSJ would lose tons of ad revenue from their pay per impression banners on their site if they didn’t have a flood of free users coming in from search engines like google. Any newspaper site would be NUTS to block google!

  28. says

    Yes, I agree, I’m a bit tired about this argument over Google and the value of the traffic they send.

    I think there is a larger tragedy: even if the aggregators gave every penny they make to the content producers it still wouldn’t be enough to cover the costs of the creators. Advertising is not a good way for news creators to make money, it’s gong to have to be what I call a “Heinz 57″ model, newspapers will need a wide variety of revenues from ads, lead gen, affiliate marketing, etc. It’s no fun being a publisher if you have to manage so many revenue streams but I don’t see any other solution.

  29. says

    Funny, most news sites seem to be working to get more Google crawls and more Google clicks and the WSJ is asking for less.

    I can’t imagine anything more stupid than insulting a female Google VP publicly with wanton accusations of promiscuity if I were running a media outlet.

    Why does NewsCorp want to start a war they can only lose?

  30. says

    I have always been skeptical about the value of the PPC campaigns, I think that targeted traffic is far more important than PPC which opens the door for fraud. Unfortunately, clients are generally of the mainstream opinion that a site that has 10000 random visitors per day is better than one that has 500 distilled visitors from a specific sector. People are also often forced to write “original content” on nonsensical, everyday, commonsense topics such as “10 easy steps to lose weight” or “SEO for dummies”. Thus, I believe that search engines waste the time of the writer and the reader by creating a demand for irrelevant content and link-spamming robots. I am in the process of launching our international business directory, The Cornerstones of World Business, but I am not going to just roll over and go with the flow. Yes, it is so much slower to contact people and partners personally and write articles with genuine content, but in the end, I believe the yield will be much more solid and reliable. If more people would focus on improving their website with relevant content instead of doing whatever it takes to win the battle for PR and keywords we would have a better internet for all. It is not Google that controls us, it is we who influence their policies with our greed for traffic. There is always someone willing to take shortcuts and work a little less – write a string of keywords instead of an article, etc. Eventually, others catch on and think of doing the same, very soon they run into each other and begin to compete and eventually we have and industry and the Google-worshiping society we have now. People spend incredulous resources and time to appease search engine robots rather than create a good site for their human users. Google as a corporation is only doing what comes naturally to them: if we surrender control, they will gladly take over and if we let go of our values, they will make up some new ones for us.

  31. says

    I’m not lucky enough to run a big news site like WSJ, but I guess there’s a lot of pressure in it. Perhaps when in the old days with some money you can control everything, nowadays it’s much harder, and sometimes the “littler” guys get more control for less money –but they’re smarter and more internet-marketing-savvy.

  32. Rich says

    Those who are with Google here (and on other aspects) don’t realize Google can take over their own market they dominate today. Then they will sing a swan song too.

  33. says

    Hey guys. If I can pipe in? I am concerned. I write news stories about recent events and pull from multiple sources, making it an editorial explaining the legal ramifications, etc. (A bus accident for example) I include myself and my firm with a link to my offices at the bottom. Now I am told this is plagiarism unless it was written by someone at CNN or Bloomberg or installs a canonical link to another story that MUST REALLY be the “source”, that does not contain my news editorial? Is journalism school a pre requisite to writing a news worthy story? I always thought journalists were very biased to the left. So how are they any less or more reliable that a lawyer who editorializes about recent injuries in the news, for example?