Google’s Love For Newspapers & How Little They Appreciate It

It was a hostile audience. It was June 2007, at a conference center in London, where newspaper and magazine publishers were hearing how a new industry-backed search engine rights standard called ACAP was coming along. The day ended with an “issues” oriented panel. The audience didn’t seem that pleased with me telling them they were full of it on how important they thought they were and how awful they thought they had it from Google in particular.

I didn’t phrase it like that, but that was the essence of my attitude. I’d rarely encountered so many people in one place with such a sense of entitlement. Worse, these were supposedly my own people. Newspaper folks, where I got my start in journalism. What an embarrassment.

I’m not talking the rank-and-file of newspapers, however — the reporters and editors doing the grunt work. This crowd was full of publishers or editors of a different type, not wordsmithing and story assignment but looking out for the business issues.

ACAP — the Automated Content Access Protocol — was a convoluted system being developed at the time to “solve” the problems that newspapers and some other publishers felt they had with search engines. In particular, that they felt they should be able to selectively decide which pictures could be printed, how long stories could be listed and a number of other things all of which largely already could be controlled through existing systems (my past post, Search Engines, Permissions & Moving Forward In Copyright Battles, goes into this in more depth).

ACAP’s real goal, of course, was to establish a way that newspapers could demand Googlegeld, their own version of Danegeld, a tribute tax they felt entitled to get just for being listed in Google. The panel started with a progress report on how ACAP was going, with the audience then asking the panel questions or simply making statements.

Over and over, people kept using the phrase “quality publishers” and how they hoped ACAP would protect these publishers and how “many” publishers were behind it.

I’d had enough. I can’t recall my exact spiel, but it went something like this. I explained to that group that ACAP was far from backed by most publishers. That on the internet, there were millions of publishers, while the newspaper groups backing ACAP mounted to a few hundred, if that. That these millions of publishers have a diverse set of concerns about search engines that ACAP was far from addressing, since it was so newspaper-centric. That an online shopping site is also a publisher, as is a small blog, as is a social media site, as is a vertical news site — and none of these groups had been invited to participate in the hallowed discussion of a supposed new robots.txt 2.0 system.

I also explained that unlike virtually all other publishers on the internet, newspapers were given extraordinary special status with Google. They were among the very select few to be admitted into Google News and receive the huge amounts of traffic it could send their ways. That many small blogs with excellent content struggle for admittance that these other publishers just got handed to them on a silver platter.

I then got very personal. I explained that I was also a journalist, publishing what I considered to be quality content as well. Indeed, I’ve published content on my topic (search engines) that I know has been of far superior quality than that published by many supposedly “quality” publications. So for them to argue they were somehow “quality publications” deserving special treatment was arrogant not to mention simply incorrect.

And now I’m hearing the same old crap again, and I’m feeling the same way I did back then. Some samples in the past few days. First from Robert Thomson, editor-in-chief of the Wall Street Journal:

Meantime Thomson said it was “amusing” to read media blogs and comment sites, all of which traded on other people’s information.

“They are basically editorial echo chambers rather than centres of creation, and the cynicism they have about so-called traditional media is only matched by their opportunism in exploiting the quality of traditional media,” he said.

Robert, I’ve been creating original content on the internet for about 12 years longer than you’ve been editor of the WSJ. Shut up. Seriously, shut up. To say something like that simply indicates you really do not understand that all blogs are not echo chambers.

I mean echo chamber? Sorry, that’s the mainstream media, too. I cannot tell you how many times I’ve seen stories emerge on the internet only to later appear in a mainstream publication. The mainstream papers read what the web publishes, then write their own stories, then all the mainstream pubs do their own versions of echoing each other.

I like getting quoted in the Wall St. Journal and all. It’s nice for the profile, and on the odd occasion I might get surprised with a link back to my site. But is this story about getting traffic from search engines (I got quoted in it) from the Wall St. Journal echo or original content? What was the Wall St. Journal putting out in 2007 that dozens of independent blogs about SEO hadn’t said already, in more depth and in more quality?

And what the hell is AllThingsD? Why are you running that and the Wall St. Journal. Are you just echoing the WSJ there? No, of course not. But why don’t you have it within the Wall St. Journal, since that’s the hub of where your traditional quality is supposedly at.

But let’s not stop with Thomson. Let’s go on up to Rupert Murdoch, who says Google’s stealing his copyright in a recent Forbes article:

“Should we be allowing Google to steal all our copyrights?” asked the News Corp. chief at a cable industry confab in Washington, D.C., Thursday. The answer, said Murdoch, should be, ” ‘Thanks, but no thanks.’ “

Let me help you with that, Rupert. I’m going to save you all those potential legal fees plus needing to even speak further about the evil of the Big G with two simple lines. Get your tech person to change your robots.txt file to say this:

User-agent: *
Disallow: /

Done. Do that, you’re outta Google. All your pages will be removed, and you needn’t worry about Google listing the Wall St. Journal at all.

Oh, but you won’t do that. You want the traffic, but you also want to be like the AP and hope you can scare Google into paying you. Maybe that will work. Or maybe you’ll be like all those Belgian papers that tried the same thing and watched their traffic sadly dry up.

Perhaps all the papers should get together like Anthony Moor of the Dallas Morning News suggests in the same article:

“I wish newspapers could act together to negotiate better terms with companies like Google. Better yet, what would happen if we all turned our sites off to search engines for a week? By creating scarcity, we might finally get fair value for the work we do.”

Please do this, Anthony. Please get all your newspaper colleagues to agree to a national “Just say no to Google” week. I beg you, please do it. Then I can see if these things I think will happen do happen:

  • Papers go “oh crap,” we really get a lot of traffic from Google for free, and we actually do earn something off those page views
  • Papers go “oh crap,” turns out people can find news from other sources
  • Papers go “oh crap,” being out of Google didn’t magically solve all our other problems overnight, but now we have no one else to blame.

Look, I jumped out of newspapers back in the early 90s because it was clear they didn’t know what to do about online. I will never forget being in a conference room at the Orange County Register when it was being debated whether the paper should go to CompuServe, AOL, MSN or freaking Prodigy. Prodigy! And I had SEEN THE WEB, and I knew that’s where things were going — so I got out. And since then, I’ve watched the papers fumble along.

The papers can’t get coordinated on anything. Anyone remember Pathfinder, that was supposed to be the Time-backed portal for news. Yeah, that did well. What, a decade of the web, and none of the papers could put together their own version of Hulu? The only thing you can all agree on is that you hate Google News for “stealing” so much from you — despite Yahoo News still being the larger news site. But Google makes a better target, plus I suspect some papers might have favorable placements with Yahoo that makes them not want to yell about the Big Y.

Stop yapping. If the papers think they’re such hot stuff, make your own Hulu. Get on with it. But spare me this whining. The AP is on again to protect news content from “misappropriation,” whatever that is — and if it’s that I can’t link to an AP story with a short summary, bring it on.

To the AP, I ask again that someone read my open letter from last year, Hey AP! How About Running A Real News Web Site?. Fix your problems; don’t look for scapegoats.

As for being legal, let’s talk now about the dirty secret of how newspapers operate. They misappropriate content all the time.

Look, I was in a newsroom for years. A newspaper graphic needed doing? You found a book with a drawing, used that without asking the author for explicit permission because shoving in a mention in the “source” line was good enough. Following on a story that a rival paper wrote? You damn well read that other story, which got you up to speed, but heaven forbid you ever mentioned that the other publication came out with the news first. If you did, that was only if you could do a story that suggested you had the “real” scoop that the other publication had wrong.

I was particularly bemused by the Daily Telegraph’s editor going off on Google about two years ago, given as I’ve covered before how twice I had material from my web site outright stolen by the Daily Telegraph. In my years of reading the Daily Telegraph, I was also bemused at how they treated private photos on Facebook as if they were their own exclusive picture library. Some woman died? Well, she’s dead — let’s just use photos from her Facebook profile that we can get. No need to ask permission.

Geez, people are blocking Google Street View cars in England, but did any of those people ask the news photographers shooting the scene if they got releases for taking their pictures. Newspapers “ripoff” people all the time shooting their pictures without permission because “it’s news.” (They actually have protections allowing them to do this, but I think you get the bigger point).

Yeah, AP, when you’re questioning the legality of search engines, let’s open up that big can of worms of what your business model is all about. That’s productive. Rather than fix your problems, keep doing those dinosaur death throes.

[Postscript: See Larry Dignan’s AP eyes news aggregators; Risks exposing its lack of value add for some specific examples of AP stories that aren’t exactly original content

What about the Guardian? As it tells [PDF] the UK government (and see this Telegraph article and this from PC Pro):

  • Search engines and aggregators have things “skewed heavily” in their favor
  • Since search engines get the “lion’s share” of news-related revenues (though the Guardian doesn’t back this up), news publishers are in jeopardy
  • Search engines actually generate too much traffic, which means the Guardian has too much inventory and can’t make as much money
  • There’s no way for the Guardian to take money directly from consumers (apparently charging for subscriptions, like the Guardian does offline, hasn’t been thought of as a solution for online)
  • Blocking search engines isn’t a solution because there’s then “no alternative route to market.” (Amazing — Google sends too much traffic, but pulling out and reducing the traffic flow means they won’t make more money — instead, they apparently won’t get found at all. So much for their content being so compelling that people might just go to them directly)

Gosh, it was about a year ago I sat at a panel at the Guardian, designed for its reporters, and talked about ways they could (and they wanted) to generate traffic from search engines. Doing keyword research, looking for trends, all that. And Google was by far — by far — the biggest referral of traffic the Guardian got. If I recall, it sent something like 3 million visitors to the Guardian per day.

Poor babies. See my memo to Murdoch above on how to install a robots.txt file. And don’t whine people won’t be able to find you. If you’re that good, they’ll seek you out.

Seriously, the Tribune and the New York Times saddled themselves with debt, and that problem is somehow Google’s fault? The Guardian’s had a decade to figure out how to earn off the internet, and it complains to the UK government that it can’t succeed? And Murdoch complains about Google at the same time his own company works to draw more traffic from Google through SEO efforts — just like every other major newspaper out there? WTF?

My suggestion is simple. Stop looking to blame Google for your failings. Figure out a better business model rather than blowing hot air about the privileged positions you occupy.

One example of this is First Click Free. That’s a program from Google that allows you to put your registration-only or view-by-payment content directly into Google. Newspapers have been able to use that program for about three years, if not longer. In contrast, “ordinary” web sites only got the go ahead late last year.

First Click Free is also a huge solution to the supposed problem the Guardian and others put out there about charging. It’s an express license from Google to charge people for content and yet still have your content get traffic from Google. The Wall Street Journal especially knows this well — see my Reading The Wall Street Journal For Free Despite Its Google News Cloaking post for more about what it does. And despite that article explaining how to bypass the Journal’s pay wall, I highly doubt most people will. For most of the Journal’s readers, it’s probably more convenient just to buy a subscription in the same way I’ll just buy a DVD or MP3 file rather than hunt for a free version online.

Newspapers get special treatment, both with First Click Free and with the extraordinary amount of traffic they get from Google. And while their top managers go off on renewed Google rampages, they still continue to work to get even more traffic. It is stunning hypocrisy, and certainly not what you’d expect from smart business people. But given how badly their papers seem to be going, I suppose they aren’t so smart.

Finally, don’t diss the blogs. My past post Blogs & Mainstream Media: We Can & Do Get Along get more into this.


  1. says

    Had last night a discussion with a CEO of a big publisher with a 25 m. budget. He asked me: thanks for your advise, but tell me first, why don’t we just stop with internet and save $$$$$$$ a year? Why do our newspapers have to be on the web?
    I answered: you’re asking why newspapers have to exist.

  2. says

    TESTIFY Brother Testify
    Last the GMG was looking for a SEO role – which seams to have run into the sand must chase that up some day.

  3. says

    Nice rant. I don’t disagree with your key points. Blaming Google for a crumbling business model is silly. This is a transformative time that demands much more than pointing fingers at innovators.
    I’m Anthony Moor. I want to clarify that I don’t consider myself aligned with Murdoch or any others who might be demonizing Google.
    Quite the contrary, I emphasized to reporter Dirk Smillie that search engines are the default home page for people using the Internet, and as such, direct a lot of traffic to us. That traffic is important. I don’t believe Google is “stealing” our content. And I was being a bit tongue-in-cheek about “turning off” to Google. We don’t matter much to Google. I was musing about what might happen if all news sites turned off for a week. What would people think? Would they survive? (Maybe.) I wasn’t suggesting we block Google from spidering our content. That wouldn’t test the “what if digital news went dark” hypothesis. In any case, none of that will fix our own broken business model.
    Google organizes the Web. Something needs to do that. My concern is that they’re effectively a monopoly player in that space. Oh sure, there’s Yahoo, but who “Yahoos” information on the Web? I understand and recognize the revolutionary nature of the link economy, but I’m concerned that it’s Google which defines relevance via their algorithms. (Yes, I know that they’re leveraging what people have chosen to make relevant, but they’re still applying their own secret sauce, which is why we all game it with SEO efforts) and that puts the rest of us in a very subservient position.
    I wonder if there isn’t another way in which the Web can be organized and relevance gained that reduces the influence of Google and returns some of the value that Google is reaping for the rest of us? I predict that someday there will be and all this talk of Google’s dominance will be history.

  4. says

    I have to agree with you.
    Google has nothing to do with newspapers failing in fact if we see it from a marketing perspective Google is helping the newspapers. Newspapers have been failing for the past years because they actually don’t know how to handle the web. And it can really turn into a proven fact if we see how badly designed and how badly coded the newspapers are. I’ll take a risk and point out that 80% of the newspapers online are not SEO friendly.
    Another point would be marketing, Internet marketing is not the same as publishing marketing it don’t translate well. Can you imagine newspapers selling their ads by pixels in the web? of course not. They don’t even use the per click / per view scheme. Some of them are selling by week, by month depending on their positioning and zone. I’ve been on the web for 12 years now as a Software Engineer and have had my fair share of newspapers so what I’m saying is a fact.
    Newspapers this days are decaying people are just not advertising like they used to. some newspapers have announced 60% to 75% losses in advertisement only this pass 3 months. Now that’s really scary since 80% of the revenue comes from newspaper ads.
    Also the paper used is not getting cheaper, a roll can be up to $800 to $900 dollars. AbitibiBowater reported losses by the billions and they are the main source of paper for a whole lot of newspapers. If they go out of business now that would certainly put a huge hole in the newspaper industry. See
    Newspapers just don’t have a correct business model to handle the web. My suggestion to newspapers is get a good SEO company to come on board and also a good advertising agency specialized in internet marketing for publishers.

  5. says

    > I wonder if there isn’t another way in which the Web can be organized and relevance gained that reduces the influence of Google and returns some of the value that Google is reaping for the rest of us? I predict that someday there will be and all this talk of Google’s dominance will be history.
    Only when the rest of us catch up to its level of sophistication. But by then, we’ll have some other “emerging superpower” to complain about.

  6. Glenn Kelman says

    Hi Danny,
    You complain of newspapers’ smugness, but do you worry this post is smug? Yes, you became Internet-savvy ahead of many of your colleagues, and for that you deserve accolades but that does not mean that the rest deserve bankruptcy and unemployment. It makes all of us in technology feel better to think so, but I don’t agree.
    Many journalists, bloggers, musicians and artists are struggling to make a living from the Internet. So are many technologists, for that matter. Opting out of Google’s index isn’t a realistic option for anyone who depends on web traffic, but that doesn’t make Google infallible.

  7. says

    Glenn, I don’t mean for the post to be smug. I made a point of saying I don’t have fault with the rank-and-file folks in the newspaper space. I was lucky to be able to jump ship for a variety of reasons. Not everyone could. But those who couldn’t, I still have a lot of friends there. I just helped one of them who got laid off move into his own blog. They were all (and continue to be) largely let down by owners that could not find the way forward to the web.
    In terms of opting out of Google, I just cannot agree with anyone who complains about being in Google — getting traffic for free — and then complains that Google is ripping them off. If you can’t afford NOT to be in Google, then clearly you’re getting essential value from the service.
    That’s not to say that Google and search engines don’t need to consider ways to support and work with various publishers. And they do. But some prominent newspapers continue to act and sound-off as if they are deserving of special treatment and somehow having been specially abused. Everyone deserves decent treatment — and in fact, they’ve been treated better

  8. says

    The problem is that not only the Internet itself but certain elements of it (like Google) must be addressed as new evolutionary landscapes, not merely big businesses. This has a whole variety of implications requiring new structures and economic forms that better balance and protect both private & public interests.
    I advocate a new structure called an
    Individually-Controlled/Commons-Dedicated Account as part of that needed solution. Initially designed for Political MicroDonation (under $1) but with much broader implications. For some info on what the heck I’m talking about you can see my blog which will have other links.
    Chagora & Civilization Systems
    What’s required is a little attention to issues of scale in social/economic/political systems. Not an easy issue to comprehend but too important to ignore.

  9. says

    Danny, that is a delightful rant. But I think you’re stealing a few bases.
    For example, you say, “newspapers were given extraordinary special status with Google. They were among the very select few to be admitted into Google News and receive the huge amounts of traffic it could send their ways.” Doesn’t it occur to you that Google News gains a lot of credibility and thus traffic by linking to brand-name newspapers? And that Google monetizes this traffic because it drives more traffic to the ads on
    You rightly point out that newspapers brought this state of affairs on themselves by inviting Google in. Now they whine that it was a bad deal. But the fact that they made the mistake doesn’t mean it wasn’t a mistake–or that they shouldn’t be trying to repair the damage now.
    You’re right that the newspapers need to put up or shut up. But don’t get preachy about Google loving newspapers. Google is acting in its own best interest, not that there’s anything wrong with that.

  10. says

    Danny good article.
    I just tweeted the other day.
    Will government bale out the next hard hit industry, newspapers? Wasn’t because of the price at $1 or $2. Compare newspapers & auto industry Whats wrong?
    It’s not that newspapers are too expensive, people just want a different product. Just like the auto industry, people were still buying new cars and trucks, but it had to be one that suited their needs, be a price, fuel economy or whatever. Gone are the days where manufacturers dictate to the consumer what to buy.
    Maybe some of these larger newspapers should hire a super affiliate to help them cash in on all that traffic. Why can other business models make money online but newspapers can’t, is it because they understand the relationship between them and their traffic.

  11. says

    It’s kind of funny that all the Google Ads below are for newspapers — one for the WSJ home delivery…
    My real point is that I think the problem is simple: Online advertising is undervalued. If you asked newspapers twenty years ago if they would like to eliminate their printing and delivery costs and bring the news directly to millions more people someone would say: “Great! We can give away the content and support ourselves through advertising only.”
    But today, the print ad is still really expensive and the online ad is relatively cheap.
    To me, this isn’t as big of a paradigm shift as people are making it out to be. People still get their news and value news from established sources. So why is it when I search CNN or the NYTimes I get ads for t-shirts and mortgage refinancing?
    It doesn’t make sense that advertisers are paying less for content that meets the eyeballs of relatively affluent people sitting at computers. Charge more for advertising, drive traffic through search engines, enhance content for free using bloggers, use technology to reduce costs. Am I dumb or does that sound pretty simple?

  12. Dominic Young says

    I have been part of the ACAP project since the start, and also work for a newspaper publisher. So in some peoples eyes I guess I am both stupid AND evil. But I’ll try to make a sensible, personal, comment.

    I was also at the conference in London when Danny Sullivan participated in the discussion about ACAP. At least I think I was.

    I, like he, don’t recall his exact spiel but I don’t share his recollection of the event – neither the hostility of the audience nor the vehemence of Danny’s views. It is true to say that Danny has previously expressed, at private events as well as public ones, a certain scepticism about aspects of ACAP. But I don’t remember the vituperative “embarrassment” and anger that comes across so strongly in this article.

    In fact, six months after that conference he wrote the following:

    “I think it’s been very useful that some group has diligently and carefully tried to explore the issues, and having ACAP lurking at the very least gives the search engines themselves a kick in the butt to work on better standards. Plus, ACAP provides some groundwork they may want to use. Personally, I doubt ACAP will become Robots.txt 2.0 — but I suspect elements of ACAP will flow into that new version or a successor.”

    Nothing Danny said at that conference or at other ACAP meetings he attended – or, in fact, wrote – reflected the condemnation of ACAP and newspapers he now expresses. His view was always sceptical, balanced and helpful which is why ACAP sought his views and wise counsel and why he was able to participate in the “hallowed” discussion of ACAP which was also, and still is, open to any other person or company who wanted to participate.

    The anger he is now expressing is new, to me at least, and seems largely directed at the newspapers who, it would seem, Danny considers to be almost moronically stupid.

    Of course, Danny is entitled to his opinion, so vehemently expressed, and if he is right he will be able to look gleefully on as the media steer their ships relentlessly to their own doom.

    But one thing remains unaddressed by his rant – even if in his view any newspaper wanting to change the way they work is idiotic, why should they not be able to make that choice for themselves? What is it about ACAP which is objectionable? If its the case that anyone choosing to exercise the functionality of ACAP – something other than a choice between the status quo and his tritely offered “block everything” robots.txt file – will find themselves to have made a terrible mistake then whats so scary about it and why is Google so unwilling to implement it? Surely the only people they hurt will be themselves.

    In other words, if he thinks the only people who stand to lose are the newspapers, why not let them learn from their mistakes rather than rail against their inability to see the world through Danny Sullivan’s (or Google’s) eyes?

  13. says

    Dominic, glad you were able to comment in the end!
    It wasn’t that the audience was tossing eggs at me. I mean, people were nice and polite. I meant that they were hostile to the view that they somehow hadn’t been hard-done by Google. That very clearly was my recollection from what many people voiced. And I certainly didn’t go into attack mode at that attitude. I do remember giving that spiel, but I certainly didn’t tell them quite so bluntly that they were full of shit! :)
    My embarrassment isn’t over ACAP — it was (and remains) over this sense of entitlement that comes from some of those in the newspaper industry. That they still seem to believe they occupy some special industry that should get wonderful perks, instead of having to reshape itself and ensure its own survival as many other industries have to do. Rather than do fundamental fixes, this current wave of “blame Google” does continue to be an embarrassment.
    Nor do I have particular anger at ACAP. I think it has been useful at least as an exercise for the industry to explore the permissions issue. But I’ve also watched ACAP continually be trotted out by some as if it is the ultimate fix — and if the search engines don’t get on board, they’re somehow obstructionist.
    If you look at the current ACAP blog now, Mark writes:
    “The idea that “Acap only addresses the small minority of content owners” is perhaps a useful mantra but it means nothing. This is like saying that the Robots Exclusion Protocol only addresses the minority of content owners, or that Creative Commons only addresses the minority of content owners. Both are true, but this tells us nothing. More than 800 sites including a growing number of very substantial content owners are now expressing their support for ACAP, and Google cannot continue to ignore this in perpetuity (however much Cohen continues to suggest that this is an insignificant minority).”
    800 sites is nothing compared to the number of sites on the web. It is indeed an insignificant minority. Oh, but these are “substantial” sites, right? Again, that shows an attitude that other sites are somehow not important enough. They are — but ACAP does not represent them nor represents the entire spectrum of sites out there that have robots restriction needs.
    My main issue with ACAP really is that it feels like what the music industry wants to do with DRM. If only we can get all these technological restrictions in place, then we’ll be able to sell music the way we want to. But DRM is dying. Consumers don’t want it. But music is selling fine without it.
    My fear is that the newspaper industry will buy too much into the idea that they can somehow foist ACAP controls onto Google along with the threat of removing all their content from being indexed. I think the result would be a lot of newspapers realized they’d cut their own throats and also wasted time better spent on figuring out improved business models.
    It’s especially a waste of time because much of what ACAP was trying to do could already be done with smarter application of robots.txt. I’m not saying it’s all or nothing — block or don’t block. People can selectively choose what they want to have listed or not. But ACAP ultimately seeks to find a way to indicate a licensing payment mechanism, right? It’s a technological proposal for a new business model that might not even fly.
    But newspapers can try anything they want. Why don’t they simply start their own portal now, with ACAP as the controlling mechanism? Instead of poking at Google and complaining, start their own Google and show it how it should be done.
    I love newspapers. I want them to survive. And I didnt write this piece because I have a hatred of ACAP or some great love for Google. I wrote it because you have a lot of newspaper executives spewing bullshit about how the internet operates and how they receive traffic. These executives seem to think yapping about stuff is going to fix the very fundamental problems the newspaper industry faces. I don’t think it will, and I think they’re leading you and others in the industry toward a murkier future. I want them to stop looking for scapegoats and start fixing issues that existed before Google itself existed.

  14. says

    Daniel, thanks for your comment. Until literally a few weeks ago, Google made no money directly off Google News. But yet, it does and did help drive traffic to
    I’m not suggesting that Google’s somehow altruistically offering Google News for the betterment of the world. Google’s a business. It launched Google News following the success of other preexisting news aggregators like Yahoo News.
    I just don’t see how Google suddenly gets to be the brunt of all these publishers with complaints. And in terms of “love,” the point I’m making is that for all the complaints newspapers have about their supposed mistreatment by Google, they’ve had a lovefest compared to those who run ordinary web sites.
    Want to cover news as an independent journalist? Good luck getting included into Google News — but the Wall Street Journal waltzes right in. Want to put up a paywall around your content but still get Google traffic. As I said, newspapers got the ability to do this years before ordinary web sites did. That’s the love I’m talking about.

  15. says

    Anthony, thanks so much for commenting and clarifying the context of your quote.
    If there were no news sites at all, yes, I think they’d be missed. People entirely undervalue news. A blackout might make them understand this more.
    In terms of news search, Google’s not a monopoly. Yahoo still has the larger share, to my knowledge. Google’s just the bigger target that people seem to poke at.
    I think information is already being reorganized in other ways. Digg sends sites lots of traffic (though if those in the news industry think Google’s somehow biased, take a close look at how gets well place with the Digg algorithm and system). Twitter I think is especially changing how people hear about news.
    I suppose the bigger issues isn’t Google but just how do the news publications earn off the web. I think Murdock and gang think the model is charging Google for the right to list their headlines. That threat helped Google cave with the AFP and AP. I think the very big pubs might pull it off, as well. But the smaller pubs aren’t that necessary. And even with the bigger pubs, Google might just decide thanks but no thanks, we just won’t list you.

  16. says

    Danny, thanks for the thoughtful response. I still think Google’s preferential treatment of newspapers (love feels like a strong word to me, but I realize I’m quibbling) may simply be the price Google pays in order to maintain their cooperation and the concomitant benefits. Of course, now that the newspapers are threatening to be less cooperative, Google may also change its offer. It will be an interesting renegotiation to establish the balance of power.
    By the way, I also agree with you that independent journalists / bloggers often have more insightful content than brand-name newspapers. But I suspect that the major newspapers still hold the lions’ share of brand equity–which unfortunately matters more than the content itself in determining market value.

  17. DougE says

    Right on Danny. Like you, I saw newspapers flailing online in the late 90’s and left for a company that actually got it. That company happened to be Google. It’s sad that now newspapers blame Google for their demise, when they had plenty of opportunities to dominate online content themselves. The truth is that many newspapers tried charging for content at first and it was a miserable failure for all except the WSJ. Then they engaged in purely defensive battles instead of trying to understand what was different about this new medium and developing new ways of reaching readers. I’m glad you’re telling it like it is, but I wish newspapers, which I love, were better listeners.

  18. says

    Excellent article and agree regarding the arrogance of the newspaper editing and publishing class and the fact that they are often their own worse enemy.
    The problem though is that theee is no business model that will work for them where they don’t move their operations online.
    Two main reasons, much cheaper production costs and, more importantly, online, like TV has the potential to be current. 24 hours is now too old.

  19. says

    Very thought provoking post Danny, and despite its length wholly engaging.
    I too share much of your perspective as regards the current stance of many ‘traditional publishers’ to Google. Unfortunately times have changed, and as a result people both obtain and read their news in a completely different way to what they used to. As a result they have moved away from their daily ‘quota’ of news, to a more ‘real-time’ environment more akin to the lifestyles we lead.
    No longer are newspapers the ‘first’ to break the story, no longer do they solely influence perception. That said, we may hear our news elsewhere first but we often look to the newspapers to confirm news – and as such they still carry much trust and authority and provide us with the bigger picture.
    There is no getting away from it though, online has changed the way people communicate, from one of simple one way communication to one of two-way engagement. People like to have a voice and express it, just look at the success of Youtube, Twitter and MySpace.
    As such the role of the newspaper has changed, and that is not just down to Google – although Google obviously has some part to play in that. Newspapers are now not the source of the story, but more of a point of confirmation. For this reason – surely it is the newspapers that need to evolve – to find their new ‘place’ in society.
    Whether or not that place involves Google and/or the other search engines is up to them.

  20. Anders says

    You’re right, newspapers aren’t special. They don’t provide anything that people can’t get anywhere else.
    What’s that? Oh, the local alderman in your town was busted for taking kickbacks from a paving company in his district, because of a reporter from your local paper sifting through the state sales taxes and incorporation papers and happened to notice a link?
    Awesome. Now when that newspaper folds because of twits like you who receive that content for free off of some fucking google twit feed, and when that alderman’s son comes to power in five years and loots the shit out of your town’s tax coffers for him and his buddies, don’t come whining to me.

  21. Anders says

    Doug E.
    Um, typical of a self important google nerd like yourself, you didn’t do your homework before you opened your mouth about newspapers.
    Specifically, you wrote: “The truth is that many newspapers tried charging for content at first and it was a miserable failure for all except the WSJ.”
    I can think of exactly three papers, off the top of my head, but no more than that, who tried charging for content at first: NY Times, Columbus Dispatch and some paper in Arkansas.
    The NY Times charged $7.95 a month, or $49.95 for a year, for “TimesSelect”, which was mostly just a pay wall to access the paper’s columnists.
    At the time the Times stopped charging, in 2007, they had 221,000 people subscribing to TimesSelect. Yeah, what a miserable failure that was. The Times stopped because they felt like they might make more money from ads from a greater number of free surfers like yourself.
    Most people in the business expect to see the Times begin charging again for content very soon. Along with most other papers.
    And if you don’t want to pay for it? Good, go away. We at newspapers DON’T WANT YOU. Just go away, period. You’re nothing more than a freeloader who has benefited from a paper like the Times’ brilliance in exposing one piece of corruption after another in this country.
    Fortunately for us still in the newspaper business, lots of other people value what they receive from a newspaper, and will be willing to pay for it. 221,000 were willing before, and when they realize that, “Oh, gee, google isn’t exactly giving me diddly squat when it comes to producing actual, real news content on their own and really are to newspapers nothing more than what Bill Gates was to Apple’s initial operating system (pirates, thieves) in the late 1970s and, gee, what the Times gave me was a whole hell of a lot better than some pimple-faced blogger asshole’s opinion on things”, then a whole lot more of them will pay a few bucks a month to get that content back.
    And then you, Doug E., will be part of a company whose free lunch on the backs of real journalists will be over, and we’ll be the ones laughing in the end.
    Unless you want to start your own news-gathering operation, that is. I look forward to that.

  22. Anders says

    And one more thing, Doug E. and Danny and the rest of you geeks here: Please, PLEASE, PLEASE don’t read a newspaper.
    Please don’t read a newspaper, because when you don’t, my community of newspaper reading people will be better than yours. My community’s citizens will vote more (as a recent Princeton study of what happened to Ohio and Kentucky towns when one of the Cincinnati daily papers went under showed – people voted less, their kids had worse test scores than kids in newspaper-reading cities and their towns had more local government corruption).
    Please don’t read a newspaper, because I’ll have the edge on you in lots of ways. I’ll know better what to look for, say, in my town or city’s next municipal bond hearing, based on what the local town/city council said in the article I just read about it.
    You, on the other hand, will have a sales tax levy suddenly imposed on you and your family by a town council who didn’t have to worry much about any protestations, because there was no article in any paper in your town, because there is no paper in your town and you didn’t know about the meeting and what was on the agenda and how controversial the sales tax increase might be.
    Now, I have more money in wallet than you, because I went to the council meeting and argued along with my other paper-reading brethren about a different alternative based on what the local reporter found out from interviews with other business or government leaders.
    Please don’t read a newspaper, because I’ll love that moment when I bump into you at a cocktail party and any number of current events subjects come up, and I’ll get to watch you embarrass yourself in front of me and my friends over your thumbnail, google-headline-sketch-but-no-deeper-than-that knowledge of the subject. Then, I can wade in and offer more nuance and detail to the subject because I read a couple of really well-reported stories on it in the NY Times from a few days before, thanks to my paid subscription.
    Just seeing your wife’s reaction to my better informed takedown of your observations/opinions will be worth the price of the subscription.
    So, in short, please, PLEASE don’t read any newspapers, you one-track mind google dorks.
    You’ll be giving me one of the best advantages over yourself I can think of in the marketplace. And, as an informed person.

  23. says

    Anders, I love newspapers. I pay for newspapers. I even subscribed to the LA Times despite them having made it so hard:
    But newspapers are not special in the sense of deserving some type of special business treatment. That’s what I’m saying. The newspaper executives are beating up on Google as if they are failing banks looking for a government handout. They’ve had more than a decade of the internet — before Google even existed — to figure out a new way to make money. They haven’t done that. And if they can’t get their stuff together, a lot of them are unfortunately going to disappear.
    That would be a shame. I want papers to exist. The type of reporting you’re talking about. I know what that’s all about. Like I said, I used to work for a newspaper. And like I said here:
    I have a real concern that important stories won’t get reported.
    You’re upset with the wrong person, and you’ve misread what I’ve written. Newspapers have value. Newspapers should survive. I read them, and I love them. And if you do too, you’d better hope that the people running them get their business act together. Otherwise the only thing we’re going to be talking about at a cocktail party is how sad it is they all went away.

  24. says

    I’m late to this one but want to chime in because the post and the ensuing conversation are so important.
    I too prickle at all the Google Bashing and am upset that search engines have become a convenient straw man for what ails news companies.
    The web did dramatically alter the distribution (and hence the consumption) of news. This started happening back in the 90s and the writing has been on the wall for a decade and half.
    Web content distribution undercuts two big things that are dear to news organizations.
    One is the ad-based revenue model, whether that means page advertising or classifieds. Page ads would still be viable if print subscriptions were viable. As for classifieds, they’re simply better in every way online, whether you’re looking at Craigslist, eBay, AutoTrader, you name it.
    The second is that digital distribution undercuts commodity articles. Major news sources do excellent online when they either create unique content or garner reputations as being the trusted curator of other’s content. As Danny stated, newspapers in particular have been cutting local reporting and adding wire stories and syndicated pieces in cost-cutting efforts. In doing so, they have undercut what differentiates them as information providers. While this commodification has slowly eroded offline readership, it’s a death-sentence online. There have been some horrible management and editorial decisions at news companies, but part of the decay is the marketplace saying that commodity articles or rehashes of other articles are no longer valued or needed.
    Danny, you are correct when you say that Google and Google News give mainstream news outlets special treatment, and it is in their interest to do so. Their business model relies on returning the best, most trusted results; if Google can’t do this, they too will wither. This is symbiotic relationship between media orgs and search engines should be embraced, not avoided. Search is not just about traffic, it’s about visibility. If you write an article that can’t be found, it might as well not exist.
    I do not want to see the end of expert news gathering. Media organizations would do well to think about their strengths and unique value to their readers, whether that is expert investigative journalism, local news, or expert analysis. Not every outlet is suffering, and even a few print newspapers (the Economist, for example, which only recently invested heavily in the web) are thriving because they offer something valuable and unique.
    I’ve spent a lot of time in recent weeks defending search engines and Google News. Luckily, I don’t have to do this at my company, as management and editorial are well aware that the benefits search engines give us far outweigh what we give up.
    As a non-profit, we’ve absorbed the financial shocks better than most, but by no means is NPR immune. Even in this dark time, our mission to create a better informed public means we worry about our content and our audience first. While I’m new to news, when I worked in the private sector, a relentless focus on giving the customer the best was the only way you survive.
    My biggest concern with search engines is not whether they are “stealing” our content, but rather am I doing my best to optimize the huge gift of authority they have bestowed on our site. I am also particularly keen on working with our local member stations to increase the visibility of their excellent local content.
    Javaun Moradi,
    Product Manager, Search
    NPR Digital Media

  25. Anders says

    Oh, OK then.
    If I’ve insulted the wrong person, apologies. I just interpreted what I thought were a few too many digs at newspapers in general.
    My rant still applies to many others who I bet run in your circle, but I guess not to you.
    Newspapers were not “late” to the web, though. In fact, newspapers were some of the first to hit the web.
    They decided to give all their content away at first, to build critical mass. Some of that thinking has been a mistake. Then again, maybe not.
    Most big newspapers now have at least 1 million unique visitors a month. The NY Times had 16 million last month. They’ve reached critical mass in eyeballs, in other words.
    Before, when their websites were just starting out, they didn’t have that kind of critical mass. People are coming to newspaper websites, and they keep coming back, along with new people every day. They generally like what they’re getting.
    My guess is this: About 10-percent of those current unique visitors will pay to continue to keep receiving that paper’s website content.
    That’s the general rule of sales – 10 percent will buy what you have to sell, provided it’s a decent product and reasonably priced.
    Now, you’ll say “well, 90 percent won’t pay for it, that’s awful.” And they’re right, 90 percent won’t pay. But once you have a critical mass of people already receiving your product, a return rate of 10 percent means papers now have 100,000 people paying a few bucks a month for content. Now they have, let’s say if the price per month is $4.99 for the content, $500,000 a month in new revenue coming in, or $6 million a year. That’s still not huge money really.
    But it’s a start.
    The trick for papers will be to keep that 100,000 people paying for the content happy and satisfied with what they’re getting. The 90-percent who may flee the site won’t matter much to advertisers anyway. Most advertisers expect only a 10-percent “buy rate” with the ads they place in print papers today anyway.
    Advertisers will like pitching to those 100,000 paying paper customers, because they’re locked in to the site and come back every day. I think the term is “stickiness” to the site that is attractive to advertisers.
    And the paper will like it because not only can they now charge higher rates to advertisers to access that 100,000 people, but now they finally have revenue from the subscriptions themselves.
    Newspapers will defeat local blogs or aggregators because their content will be superior. Flight to quality is always what happens in the end.
    Yeah, newspapers have made lots of mistakes. But I see hope on the horizon.
    And if I’m wrong, you can chide me for it at that cocktail party.

  26. Steven L says

    I hear alot of Physical Newspaper vs Internet; and until recently haven’t considered a different medium.

    Now I think i’d rather get newspapers and magazines from some type of ereader device. Like the one Amazon has, or a larger Apple Ipod intended for reading, in addition to its other uses. Then I could go to the Apple Online Store or other similar service and select them all, get billed all in one place, and have them all synced to my device everytime I log in.

    Then I can read the newspaper the way it was intended, lounging in the chair :).

    I won’t hold my breath.

  27. says

    I’ve been thinking about your essay all week, and finally published a longer response in the Seattle area’s main startup blog, TechFlash.
    The point that you seem to be ignoring is that no business, not even Yahoo, can afford to be excluded from Google’s index; as everyone knows who complained about Google’s favoritism toward Knol, it would be inappropriate for Google to use its index as leverage for competing in areas like news aggregation.
    Perhaps you find traditional journalism to be, at this point, a contemptible enterprise (your friends in journalism notwithstanding) but there has to be room in the world for some writers and artists to charge directly for their goods. Google for its own excellent reasons makes such an endeavor more, not less, difficult. And that in the end may make the Internet a poorer rather than a richer place.
    I think Google can be part of the solution to this problem, but first we have to acknowledge it’s a problem rather than saying Google, love it or leave the index!

  28. says

    Glenn, by no means do I find traditional journalism to be contemptible. Far from it. In fact, I linked to an earlier post I did expressing real worries that it is important and hopefully will survive. We need it. What’s contemptible is the way some of the newspaper owners and execs have acted over the past nearly 20 years now, to not come to grips with digital and figure out a business model that will help their businesses survive. And after all this failure, several now seem to think Google somehow owes them a living.
    No, I don’t see Google as the problem. I don’t see how a site that sends newspapers millions of visitors per day is somehow hurting them. Moreover, First Click Free is indeed an ideal gift to them, if they know how to use it. The Wall St. Journal does; other papers just need the guts to charge for some of their content, as well. And yes, there are going to be some savvy people who will use Google News as a way to bypass pay walls on a story by story basis. These are also the same people would would just skip reading anyway. But others will pay for convenience and because of a feature package that they might find worthwhile.
    In terms of your essay, it’s some of the paper’s themselves that are complaining about Google and how much it supposedly gets off of them, so pulling out of Google makes perfect sense, if they really believe what they say. Stop propping it up; if your brand is so big, people will seek you out. But the reality is they simply hope to pressure Google into paying “carry fees” to list headlines and summaries. And the larger issue with that is many sites list articles — do we have to pay carry fees as well? And as pointed out recently, both the NYT and WSJ have “aggregator pages,” as well.
    Bottom line to me is that the managers should get on with seeking solutions, not looking at Google as some type of money bags that they think they can dip into as a quick fix. That’s the attitude they give off, and that doesn’t address their fundamental problems. I mean, the AP does have a paid deal with Google. Apparently, that’s not solving their woes.

  29. cooper says

    Danny — I’ve been doing Internet journalism since ’95 and your arrogance rivals that of some of the worst newspaper managers I’ve met along the way. Talk about an echo chamber. Your arguments about why mainstream media is failing is stale, boorish and unenlightened. It’s the same drivel one hears from every blogger and it’s tedious. Let’s try to at least have an honest debate. Yes, old-line media is responsible for the decline of their business model, but that was true long before the Internet came along. Their franchise has been shrinking for years; the Internet sped the demise faster than anyone could have imagined. On the other hand, aggregators, whether search engines or portals, are not, on average, capable of replacing the lost revenue dollar for dollar, which means the inherent cost structure built into a traditional media operation is unsustainable. That’s the reality. But it’s also true that aggregators don’t give a rat’s ass about the cost of producing content and could care less about referring traffic to the content creator. And why would they, since they only lose money when a user leaves their site for another. This is especially true among the portals. Don’t believe me? Then you’ve never sat in on an internal biz dev discussion with a major online distributor. The cards are stacked against the content providers, since distribution — and the only real distribution that an advertiser cares about generally is repeat traffic and big numbers — is key to long-term success. Until distribution is equalized, it will continue to be true. Microsoft’s 80~% of the browser market and Google’s control of search traffic makes it extremely difficult for media companies to create a biz model (loyal, repeat traffic) that can support their cost structure — and I don’t care how much SEO they do. And, unless someone steps in, it’s only going to get tougher. All Internet users should be concerned about msft’s Yahoo designs, it’s Facebook “investments”, and Google’s ownership of YouTube and it’s own portal aspirations. You’re naive, Danny, if you think Google is the answer. You’re nearly as naive as the old-media managers and having a shouting match with them everytime they complain is a waste of everyone’s time and confirms you really don’t understand what’s happening at a macro level. And that’s a pity, becuase this really is an important topic.

  30. says

    Cooper, Google doesn’t have distribution discussions for Google News — except in the two or three cases where companies have sued them or threatened to sue them for including them. Otherwise, they just let you in for free — and send you hundreds, thousands, hundreds of thousands of visits per day. And they do all that without preventing you from leaving (except with AP, because AP itself wanted news hosted on Google).
    I can appreciate that other sites may operate differently. But Google News is the main target here.
    I also did not say Google is the answer. I said the opposite — that these media managers are looking to Google as if that’s the answer, if they could only free some money out of Google’s grasp.
    Bottom line is that even without Google, even without controlling the Internet Explorer, plenty of sites get traffic. But the newspapers largely haven’t seemed to figure out a way to earn off that traffic. And that remains a pity, because it is an important topic and problem that needs to be solved.

  31. cooper says

    Danny — Fair enough. Google doesn’t errect barriers to google news, but GN traffic is immaterial vs. Google search, macro-ly speaking. Newspaper sites can’t monitze the traffic b/c they are local creations, with local advertisers, who don’t care about users/readers coming in from India if they own an auto parts store in Indiana. That’s not necessarily the case for for national media, which can make a compelling advertising argument for more global traffic. Nor is it true for bloggers, who don’t have the same cost structure and are estatic to generate 100K a year using AdSense. It’s also true that local advertisers are using AdWords to offer their products directly to the consumer, bypassing local media companies. So what’s the solution? A better mousetrap — a market-beater local search engine married to compelling local conent that isn’t available anywhere else. Easier said than done.

  32. ryannagy says

    Thanks for the post.
    As someone who is new here, I find it highly annoying that you do not clearly identify who you are (I assume this is Danny Sullivan’s blog). Nor do you have dates on your posts. It would be nice to know how current a post is, as this “industry” can change very quickly.
    That being said, thanks for you efforts on behalf of the SEO and SEM community.
    – Ryan Nagy

  33. says

    There’s a date at the bottom of every post. In the sidebar, there’s an “About” area that links to information about who I am. But I’m hoping to do a redesign in the near future that makes it easier to see these things!

  34. Secret says

    I really hate google. It is so annoying. Lately I can’t go onto any site without having that stupid “The page you’re looking for wasn’t found” toggel page?
    Who invented that? And for what purpose?
    I use to love google but how do they expect me to download past papers with that stupid page that keeps popping up.???????????