I guess I can add a story to the many out there about someone who tracks down a lost phone using a Find My Phone-type of service. In this case, it was my son’s. I tracked it down to the exact classroom where the kid who took it was sitting.

My son calls when he arrives at school each day, so we know he arrived safely. Today, he left his phone out on his bike after locking up and heading to class. By the time he realized his mistake, the phone was gone. The case was still there, right on his handlebars. The phone itself had been taken out. Clearly, it was stolen.

When he called to tell me from the school office. I fired up the Find My iPhone service from Apple. That’s enabled on his phone, but there was no recent location.

About 45 minutes later, a location appeared on the school campus, out on the PE area. I got hopeful. It seemed likely another student had taken it. I triggered the phone to ring, and I also sent a message to it asking that it get turned into the campus office for a reward. Then I headed over to the school, with my MacBook Air and a mifi in hand, so I could keep tracking.

In the school office, there was much amazement that you could even track a phone like this. I waited for someone to take me out to the PE area, on the off-chance that maybe the phone had been ditched there, while we watched the screen. I was explaining more about how the feature worked when suddenly, the location changed again. This time, it jumped to a block of buildings with only three classrooms.

One of the office administrators walked me down to the block. Only two had classes in them, one far away from where the location pin-point was showing. So, we went to the other class and asked the teacher if anyone had turned in a phone. No one had turned in a phone, but he had confiscated one after someone’s phone rang.

Yep, it was my son’s. The kid who took it must have switched it on when outside, turned it off, then turned it on again when he got back into his classroom. It got my message to ring, and since you can’t have phones switched on while on campus, she’d known something was wrong to took it from him.

If you don’t have a Find My Phone service on your smartphone yet, do turn it on. They do work. My recent CNET column explains more about this: These ‘find my phone’ services will ease your mind.

Boost Mobile is Sprint’s pay-as-you-go sister company. I kind of doubted they’d let me activate my Sprint Nexus S phone on the Boot network. After an hour of transfers with both Boost and Sprint, turns out that was sadly a correct assumption.

I got my Nexus S earlier this year, purchased from Sprint so that I could test out Google Wallet. Google took so long rolling out Google Wallet that I ended up canceling my Sprint account. Even though it was a business expense, I never used it. I found the network to be poor. I resented the “unlimited” pitch that wants to charge extra for certain phones and even further if you want to use a hotspot with them.

By canceling, I ended up paying the full termination fee, which should mean my phone was free and clear to use elsewhere. Of course, unlike GSM-based phones, you can’t just slot in a SIM card from another carrier. It has to all be done online or with a phone rep.

At first, it seemed hopeful. Boost has a great daily plan, where you pay only $2 for unlimited use. When you use up your money, the phone just stops working until you reactivate it. I though this would be perfect for my occasional use needs.

The Boost rep got me through the initial sign-up process, but when they tried to activate the phone, it turned out that Sprint was still reporting it as active and tied to its network.

That was annoying. I’d canceled ages ago, been charged an early termination fee, and Sprint was still effectively locking it to their network?

Calling Sprint was a nightmare. The rep there was incredibly polite, but it took ages to figure out that I just needed this phone unlocked or somehow disconnected from Sprint.

In the end, a Boost supervisor got merged into our call. She explained that unfortunately, Boost couldn’t support my device for the “technical troubleshooting” that would come up, especially since it was a Sprint phone.

That’s probably what annoyed me the most. It’s not a Sprint phone. It’s a Google phone. It’s a pure Google Android device that’s designed to use the frequency of the Sprint network. It should totally work through Boost. The only reason it can’t work is because, my guess is, Sprint won’t allow Boost to do so.

That makes sense from Sprint’s point-of-view. You don’t want your sister company undercutting your overpriced cell phone options by letting them support high-end phones. And for most people, it’s probably fine. If you’re going to use Boost, you’re probably not looking to pay that much for a handset, either.

But still, Sprint — you made plenty of money selling me this phone. Lighten up. Let me or anyone who wants to use an Android phone on Boost, if it supports Android phones — as it does.

This is no surprise to many, but it underscores just how much the US cell phone industry sucks. Last year, I walked off a plane in London, bought a pay-as-you-go SIM card from 3 and slotted it into my Nexus One. I had full web and phone access. No problem. I did the same thing this year in Australia, with Vodafone. No problem.

But in the US, we’re screwed, and it’s not just an Android issue. My several old iPhones — fully paid for — still remain locked to AT&T.

By they way, Virgin Mobile — which uses Sprint — won’t activate anything other than a Virgin phone either. But hey, “it’s the perfect excuse to get a new phone,” it says.

No, it’s not. Not if you have perfectly good phone.

Meanwhile, the whole point of this exercise was to try again with Google Wallet. You know, the mobile payment system that Google is advertising on buses all over New York, I saw earlier this month, but which still works with exactly one phone, the Sprint Nexus S?

I at least got Google Wallet running on the Nexus S, after dusting it off — only to discover that despite having a Google Wallet account already through the web — and with that account synced to the Google Wallet app on my phone — the phone still wanted me to enter the details for the same credit card I’d already registered.

Yeah, Google Wallet’s going to be a winner.

Related Articles

• How To Give Your Old iPhone To Your Kids, Cheaply
• How Google & Apple Sold Out The Cell Phone Revolution
• Review: Real Life With The Galaxy Nexus Android 4.0 Smartphone

I’m drowning in Cyber Monday emails — mostly from merchants I’ve already done business with and so far, simply suggesting that my previous business with them isn’t that worthwhile. I’ll pick on TomTom and 24 Hour Fitness, in this post.

TomTom emailed me this:

Hmm. That Go Live Top Gear edition? Yeah, I just bought that at the beginning of the month for the full $270 price. Now they’ve dropped the price $70. I’m supposed to appreciate that?

No, what I’d have appreciated was if I’d been given the credit, instead. I sure would have appreciated when I called to ask about getting some credit, if the TomTom rep hadn’t made me think I was insane to find this strange.

Meanwhile, 24 Hour Fitness emailed “Cyber Monday Specials” to me today. I thought — good timing. My existing membership is about to expire, and this promised some pretty good savings. I headed over to the site, and here’s what I got:

Click on that Join Online Today button, and you have to enter your ZIP code to find clubs near you. Then after you pick a club, you land on the club page, which pitches the Thanksgiving Holiday Specials page, which sends you on a circular loop:

Instead, you have to ignore the most prominent pitch and instead select the “See Prices/Join” option off to the right. Do that, and you can get the offer as shown — but only if you selected a “Sport” level club or lower.

I actually want this membership because it promises access to the higher level “Super Sport” gyms. My existing 24 Hour Fitness gym is a cesspool. And now it’s finally being renovated, which means closure for a month next month.

Directly across the street is a new Super Sport gym that the Sport members aren’t allowed to use, during the renovation. Thanks, 24 Hour Fitness, classy move. But if I can upgrade my membership to use the new, nice gym for the same price I pay now or less, that’s a bargain! Sign me up!

Unfortunately, if you select a Super Sport gym, this offer isn’t available for purchase. That’s odd, because Super Sport gyms aren’t excluded, only the Ultra Sports are:

“Excludes Ultra-Sport level clubs,” you can see — no mention of Super Sport being excluded.

Now, if you click on the “Click for details” link shown, it simply takes you over main “How To Join” page that offers no further specifics. Are Super Sport gyms included or not. I had to call to find out. The phone reps said no (naturally) even though the ad didn’t make this clear.

I could do without the specials, thanks. They aren’t making me feel very special.

Like many, I’ve been watching the terrible news in Japan closely. It especially strikes home because I live in an earthquake zone, a tsunami zone and have the San Onofre Nuclear Generating Station 30 miles from my home. Checking up on the plant’s emergency plans was a less than reassuring experience today.

San Onofre Can’t Run Its Web Site Properly

My first thought was to see what San Onofre itself had to say about a nuclear emergency. Unfortunately, I couldn’t find the official web site using Google. Oh sure, it looks like Google has it:

But if you try to click from that listing to the official site, you’ll just get this:

What’s happened is that Southern California Edison, the electrical utility that owns most of San Onofre, has moved the page. When, I don’t know. But it was moved.

Normally when you move a page, you do what’s called a “redirect,” which is like a forwarding address or telephone number. If anyone tries to find the page in its old location, you automatically forward them to the new location. This is a best practice for web sites.

SoCal Edison isn’t doing that. Worse, they’re also not reporting the old location with what’s called a “404″ error code, which tells your web browser (and search engines) that the page is no longer valid. Instead, it uses a “200″ code, which means everything is “OK.” This is another best practice failure.

As a result, Google is continuing to list the old page because it hasn’t been told that it’s moved and that everything is fine. I know that building a web site isn’t nuclear science. But it’s sure not reassuring that San Onofre is getting this simple thing wrong.

By the way, despite San Onofre’s error, Bing:

and Blekko:

– both of which are Google’s competitors — have managed to find the new location and list it in their search results, proof that Google isn’t perfect.

Back to the San Onofre plant site. When I finally found it here, I started clicking through the pages, using the navigation links listed on the left-side. One of these was about decommissioning:

This section of the page really stood out for me:

Decommissioning began in 1999 and the majority of the plant’s structures and facilities are expected to be decontaminated, dismantled and removed from the site by 2008.

This is 2011, three years after when decommissioning of SONGS 1 — one of the three reactors on the site — was expected to be completed. So this copy was written before 2008 and hasn’t been updated since?

Again, it’s another thing that I don’t find that reassuring to read. From the US Nuclear Regulatory Commission web site, it seems that much of SONGS 1 has been dismantled but that decommissioning isn’t fully complete — there’s still more stuff to be dismantled, and there appears to be no ETA for when everything will be done.

So hey, San Onofre — maybe an update is in order?

In Case Of Emergency, Find Your Phone Book

Another page on the site deals with what do to in case of an emergency. There are sirens that will go off, though where exactly where these stretch to isn’t said.

There’s also an “Emergency Planning Zone” or EPZ that’s a 10 mile radius from the plant. Beyond that, there’s a Public Education Zone (PEZ) from 20 miles out. Education is only needed here, not planning, because evacutation would be “highly unlikely.”

Phew. All very reassuring. But wait, what if we want more specific details? For the EPZ, we’re told:

All residents within the EPZ receive instructions about emergency plans including protective measures, evacuation routes and shelter locations. These instructions are located in the customer guide section of the Pacific Bell Telephone Book, Orange County South edition.

For the PEZ, it’s similar:

Further information is located in the customer guide section of the Pacific Bell Telephone Book, Orange County South and San Diego County North editions.

Kids, you might not know what a phone book is. It’s a big huge binder that contains phone numbers. Numbers of people were printed on white colored pages; businesses had ads printed on yellow colored pages. That’s why you hear about these things called “White Pages” and “Yellow Pages.”

You might occasionally see these “phone books” appearing on your doorstep. Hopefully, you kept them, because that vital information you may want won’t be on the web. It’ll be printed on dead trees.

Don’t get me wrong. Those dead tree books probably will be working better than the internet during a nuclear emergency. But seriously, you can’t also put the plans online?

The Phone Book From An Extinct Company

But wait, as they say, there’s more! Pacific Bell used to be the local telephone company for much of Southern California. But I remember being confused when I moved back to SoCal in 2008, after having been gone for 12 years. I needed phone service, and PacBell was gone. There was no single phone provider any longer – and PacBell, along the way, got sucked into AT&T.

So that “Pacific Bell Telephone Book” with your emergency instructions? Good luck finding it. I’m virtually certain it doesn’t exist. And that tells me San Onofre doesn’t regularly review the emergency instructions that it posts on its own web site, shades of those emergency oil spill response plans that listed a spill consultant who was long dead.

Hiding The Far More Informative Site

Now in writing all this, I went back to the web site and finally noticed on the home page this easily overlooked link leads to a completely separate “community” site:

That link really leaps out at you, doesn’t it? And it’s nowhere to be found on the aforementioned emergency planning page or apparently other pages on the site.

That’s too bad, because the community site is more helpful. You can even play a clip of what the warning siren sounds like:

But looking for evacuation routes or shelter locations? That’s fobbed on on “check with your city.” And way over there, where San Onofre doesn’t bother to link to, you can even find those phone book guides.

San Clemente is the closest city to the plant, and it maintains a nuclear emergency page here that does seem to be loaded with really useful information. It even tells city residents how they are eligible for free potassium iodide (KI) pills and why those are issued. PDFs of those phone book guides are also offered:

Some Suggestions

I should have found the San Onofre Nuclear Generating Station web site — the better site I’ve just described — in Google and Bing for a search on “san onofre.” I didn’t, and San Onofre can help with that.

I’d suggest that all the content from the Southern California Edison site about San Onofre be relocated into that community site. If there’s some legal reason this can’t be done, then certainly link more to the new site.

Also, a little SEO, please. Pages within the site lack descriptive HTML title tags, which would be a help, I’d say.

Get more links out to the individual cities, where they have more planing info. Post that phone book info right on the site. And update the information, so that you’re engendering trust.

More Information

I certainly don’t mean to fearmonger in this post. The safety of San Onofre has long been debated and will continue to do so. I’ve read any number of articles like this gem from Beverly Hills “Patch” that’s not written by talking to any experts and just assumes that because California is expected to have an 8.1 quake on the San Andreas Fault at some time, that’s more than the 7.0 quake that San Onofre is designed to withstand — so we’re perhaps screwed.

The reality is that exactly where a quake hits, how deep, the type of shaking and all sorts of factors can mean that a quake that’s lower on the Richter scale can do more damager than a “bigger” one — or a bigger one may not do as much damage as you think, depending on your location, to my understanding.

In another “Patch” article, this one from San Clemente, San Onofre’s chief Pete Dietrick talks more about that, and what the plant is built to survive. The Press-Enterprise, the San Diego Tribune, the Orange County Register and the Los Angeles Times all have interesting articles that look at the plant’s protections — as well as experts who still wonder if we’ll be unpleasantly surprised. You can also get updates on San Onofre news from the plant itself via its Twitter account.

Postscript (March 17, 2011): The Orange County Register followed up on this piece and got SCE basically fobbing things off as a Google problem:

“We don’t ‘send’ them to either. Your questions are why Google sends them to sce.com and why our page is not current. We can only help with the latter,” said Alexander in an e-mail.

That’s not true — and in fact SCE did start redirecting, which is good, though maybe someone should tell their spokesperson. Here’s more from the comment I left at the OC Register article:

Sure, Alexander is correct — they’re not “sending” anyone to either of the web sites. But they are publishing them, and they’re doing that in order to spread information, and they’re being inept in doing it.

When he says there’s nothing they can do to direct people from the broken link on their own site to the correct information, he’s dead wrong. They could redirect to the new location.

And, in fact, when I looked just now — they did. So much for the “that’s Google’s problem” attitude.

Of course, anyone reaching the info at the SCE site will still probably not easily find the single link to the far more informative community site, nor will they find it if they read the emergency planning page. They will, however, still see today that they should open up their “Pacific Bell” phone book for information.

Last October, I wanted to deal with duplicates within my iTunes library. I tried several options. Among them was the mistake of buying TidySongs. The software failed to work as advertised. Moreover, both TidySongs & Clickbank failed to refund the $39 I paid as promised.

“Free” Download

I really should have known better. The web site was the first warning. Right at the top, it promises a free download:

Sure, the software will download for free. But what you get is a trial that does nothing. “Free demo download” would be more accurate.

Duplicates Not Found

Still, I decided to throw the dice and spend the money. I was tired of all those duplicates cluttering up my library. I ran the full software against my library. Disappointing. With around 1,600 duplicate songs, it found only 120 of those using the Name, Artist and Album options. Shifting to just Name and Artist only took the figure up to 150.

Refund Offered!

Suffice to say, a duplicate detection tool that only detects about 10% of the duplicates in my library isn’t that great. It kind of sucks. I emailed my results to TidySongs, and the customer experience manager Kelly McPhee seemed pretty nice and reasonable about the entire thing. She wrote back:

You can request a refund through  Clickbank since you used them to make the purchase.  They are a TidySongs affiliate.  You’ll request a refund through their website. TidySongs will only be able to identify duplicates that are exact duplicates.  I’m sorry that TidySongs was not the right program for you.

Then again, this was kind of odd. I didn’t buy from a “TidySongs affiliate.” I bought from TidySongs itself, using a link off the TidySongs web site. ClickBank appeared to be the way TidySongs sold direct.

I used the Customer Support form she pointed me at and never heard back. This was in mid-October. In mid-December, still wondering about that refund, I went back to the form again.

Second Time Not A Charm

This time, I found no option on the form to request a refund. Everything simply pointed over as if there was a technical support issue. Nonetheless, I diligently entered my information. This time, unlike the first time, I got an automated support ticket.

Every two days after that, I got a prompt asking me if I wanted to close that ticket, because no one had acted upon it. I ended up renewing it about five times in a row.

I also went back to Kelly, saying nothing was happening with the ticket. Her response?

The refund will need to be made through Clickbank since you used them to make the purchase

Passing The Refund Buck

Now kind of losing my mind, I went through the ClickBank site. As it turns out, no, I didn’t need to request a refund through them. At any point, TidySongs (or CloudBrain, the company that makes TidySongs), could have issued a refund. I pointed Kelly to the page advising ClickBank vendors about this:

If a customer contacts you directly to request a refund or cancelation, you can initiate a refund or cancelation ticket from within your ClickBank account. A link to open a ticket on any order is available when you search for transactions (orders) from the Transaction page under the Reporting tab. Simply click on the T in the left hand column.

Way back in October, when I first contacted TidySongs, it could have issued a refund right then, rather than send me on a merry dance through ClickBank. That it didn’t speaks volumes about the company and kind of feels like a scam.

Escalating The Refund Request

After the New Year, I closed the original support ticket, which was going nowhere. I think there was a prompt to escalate the issue. I did that. Now someone from ClickBank finally got involved — going back to TidySongs asking for permission to issue a refund, since it was now outside ClickBank’s 60 day window.

No response. Instead, one of those two day prompts to keep the ticket going started up. And again. And again. The last time it happened, I emailed both ClickBank and TidySongs to say I didn’t care how it happened, someone should issue me a refund. And if I didn’t get the refund as promised, I’d just channel my future energies into a blog post about my experience with both companies.

My ticket was closed today, due to inactivity. Well, at least I got $39 of ranting out of the experience, I suppose.

I vote by mail. This weekend, I started making my choices. I came to the decision over candidates for Orange County Treasurer, a race between Shari Freidenrich and Keith Rodenhuis. I thought I’d visit each candidate’s web sites, to learn more about that. I got a shock to discover that the Rodenhuis site was down on Saturday.

Today, it’s Sunday. That site has been down for about a day and perhaps much longer. Crazy. Election day is only two days away, and your web site goes down?

What’s wrong? Chances are, with a big dose of irony, Rodenhuis hasn’t paid enough money to keep it up. The message I get is “bandwidth exceeded:”

This type of message is common if you’ve picked a really cheap web hosting company that nickle-and-dimes you on the bandwidth front (FYI, I use Tiger Tech, which does NOT have this issue).

Paying the bill for more bandwidth will fix the problem. I have no idea of Rodenhuis has tried this yet or not. He has no Twitter account that I can reach him on. His Facebook page doesn’t appear to be monitored. I left this message yesterday on his page, saying that his site is down:

So far, no reply. Heck, I’d call him just to tell him, because he simply might not know. But the web site, which I can read through cached copies at Google and Bing, doesn’t seem to have a phone number listed.

To go really old school, I consulted the sample ballot the OC Registrar Of Voters sent out, to see if he left a phone number in his statement. Nope. Only a web address, for a web site that’s not functional. His rival did list a number — and has a functional web site, to boot.

We’ve been hearing rumors that the Albertsons supermarket would be closing soon here on the Balboa Peninsula, as well as all the stores in the complex near it. Apparently this is true, not that there’s been any local press about it.

I just heard back from my local city councilman, Michael Henn, who told me:

The City received the WARN Act notice from Albertsons about a month ago that the store would be closing on August 19th. The leases for all tenants expire on August 31st, as I understand it. So, I expect the center will be dark as of that date. The owner of the center is processing final permit approval with the Coastal Commission and construction plan approvals through the City for the remodeling, which I expect will start some time in the next few months.

The shopping center takes up nearly an entire city block. The supermarket is one of two that serve the area, so we’re not stuck — but it’s going to be a big crunch. Also departing will be these local businesses:

• Carl’s Jr
• KFC
• Fantastic Sam’s
• Let It Roll Bicycle Rental
• China Kitchen
• Newport Nails

And several others that I can’t recall off the top of my head, right now, including a dry cleaners.

Perhaps the biggest concern I have is that the entire center is going to be closed for months. I mean, the area doesn’t look that great to begin with, but what a mess. And how strange to do this right in the middle of the summer rush period.

The neighborhood rumors we’ve been hearing is that the Pavilions supermarket that’s nearby in Lido Marina Village will be relocated into the new shopping center. And further, that the property owners want to redevelop Lido Marina Village in some way — with new housing potentially going up on the site of the current city hall, after it relocates to a new direction.

Here’s a marked-up map on Google, that pinpoints the locations:

View Balboa Peninsula Redevelopment & Albertson’s Closure in a larger map

Earlier this month, the Daily Pilot did report about the Lido Marina Village redevelopment. But there was no mention of the Albertsons center redevelopment, and how that might play into things.

Personally, I hope the old city hall site gets turned into a park. The city just had a study session yesterday, so I’ll be checking back to see what’s coming out of that and paying more attention to things, myself.

And how sad — there go some of my Foursquare mayorships

I decided to upgrade my iPhone 3G to iOS 4 today, despite worries that ultimately, it would just slow my phone down more. We’ll see. After an hour of waiting just for the phone to backup, I got a tip off Twitter that let me get through the upgrade process painlessly in 45 minutes flat. Here’s what I did.

Big hat tip to Jake Feuerbacher, who told me to do a restore rather than an upgrade. It’s something he picked up off this Apple support forum thread, and it did the trick.

Backing Up … Forever!

Initially, I had loaded iTunes, plugged in my iPhone and got a message that there was an update available — the iOS4 software. I had selected Update, which caused iTunes to start backing up my iPhone before it would even apply the update.

Now, backing up is a really good thing to do. I like the concept of backing up. But after about an hour, I was wondering how long this would take. When I tweeted the seemingly endless process, I got a number of responses from other people in the same situation. At that aforementioned Apple thread, you can see similar complaints.

Restore, Rather Than Update

The quick and dirty method skips doing an update. Instead, you use the “Restore” option. Do this only after you’ve downloaded iOS 4. When you choose Restore, the iPhone will look for the latest version of the iPhone software on your computer (which will be iOS 4). Then it will “restore” this version of the software, effectively upgrading your phone.

The downside? Well, the restore means you’ll end up with a “clean” phone that knows nothing about your previous settings, the apps you had installed and so on. However, that’s where Sync comes in.

Syncing Back To Normal

Be sure that you’ve fully synced your phone. Make sure that iTunes has a record of all your apps, your playlists, your purchases and so on. If you’ve done this, after doing a restore, you’ll be able to resync your phone and bring back all your apps.

This worked fine for me. There is a risk, however, that something might go wrong. If you haven’t backed up your phone, and iTunes goes haywire, maybe you’ll have to start over.

I decided it was a risk worth taking, since the backup was taking ages. Plus, on my Windows machine, I kept getting “Mobile Backup” errors anyway. I threw the dice, and they came up good for me.

Sidenotes

A few further comments on the process.While my apps came back, none of my settings within them did. So expect to spend some time setting your email back up, Twitter and so on. That was a fair trade-off, for me.

I don’t sync my photos. That’s probably the most important data that I can’t replace on my iPhone. Instead, on my Windows machine, I simply copy the photos over manually from my iPhone to my Windows machine. Not syncing may have helped speed up the process.

Also, iTunes kept screaming that I hadn’t copied some purchased items from my iPhone to iTunes, warning me not to restore before doing this. Despite syncing all my apps, I kept getting this warning. Maybe there’s some music that didn’t make it. Or maybe one of my two digital movies that came with DVDs were causing the warning. Don’t know, don’t care. But sure would have been nice if iTunes had given me some better and more specific guidance.

In the end, I did see both movies I had purchased in iTunes, so I guess they didn’t die.

I also had one horrifying moment. In the initial sync, synchronizing my music wasn’t turned on. I toggled to do this, to the playlist that I keep synced with my phone. No luck. I kept getting little Windows errors that a some required file was missing.

I exited iTunes and restarted. It then saw my phone and asked if I wanted to start a fresh phone installation or restore. Gulp. Restore? But I’d just restored! And synced! And I didn’t have 45 minutes more to spend on it now.

Despite my better judgment, I chose to restore. In about 2 minutes, that was over, then the phone started syncing all my contacts again, my calendar and my email. Then I went back to my music area, tried to sync that again and success! I was even able to sync those purchased movies.

Ahead of next week’s FTC workshop on journalism and the internet, I gave an informal briefing to several people from the FTC about the differences between how search engines, aggregators and news blogs all gather content automatically and through human editing. They seemed to find it useful, so I thought I’d share it more widely and expand it a bit.

Search Engines

Most major search engines such as Google, Yahoo and Microsoft’s Bing are “crawler-based,” which means they use crawling software to automatically visit pages from across the web.

When a search engine’s crawler comes to a page, it makes a copy of that page, storing that in what’s called an “index.” You can think of an index as being like a giant book.

When someone searches at a search engine, software called an “algorithm” effectively flips through the book, finds pages that match the person’s query and list the pages that the software deems most relevant.

In the case of news articles, these are included in a major search engine by virtue of being web pages. Perform a search, and you might get general results that mix news articles, Wikipedia pages, shopping sites, blog posts and more all together. For example, look at the results for a search on breast cancer guidelines from Google:

At the top is what’s called a news “OneBox,” a special display of content from Google News, which I’ll explain more below. But below that are “regular” results, which have information from a variety of web sites. Included within these results are two news articles that I’ve indicated, from the Seattle Times and from USA Today.

It’s important to understand that people who tap into a regular search engine for news may be in a variety of search modes. These are my own definitions, and there may be more than these:

1) Breaking News Mode: They’ve heard about something breaking, such as a rumor that Jeff Goldblum has died earlier this year (yes, it was just a rumor), so they conduct a search looking for news and information about the topic.

2) Researching News Mode: They’re interested in a news topic generally, but it’s not a breaking event. For example, someone might be looking into issues they heard about with inauguration ticket problems from when President Barack Obama was sworn in earlier this year. A search for that on Google brings up these results currently:

This is no longer a breaking story. However, you get plenty of news reports about it showing up nearly a year later, matches that lead to the Wall Street Journal’s news blog, the Washington Post, The Huffington Post, ABC News and CNN, to name a few.

This is an example of where the “search tail” or the “long tail” kicks in. While the popularity of a search topic may dramatically drop off hours or days after breaking news first appears, there are still plenty of people who do a smaller number of searches on those topics over time. And because there are a lot of different news topics, a newspaper or other news publication can receive a substantial amount of visitors from search engines from “stale” or “old news” topics.

3) Just Searching Mode: News publishers may also receive traffic via regular search engines from people who are not deliberately seeking news content. For instance, consider this search for best summer books:

There’s no reason to assume a searcher is looking for content specifically from news publications in response to that search. There are easily thousands of non-news web sites that cover books. And yet, news publications have high visibility. NPR appears twice at the top, with USA Today and The Guardian showing further below on the page.

Finally, I said that the search engines make a copy of pages they visit. That can suggest to some, especially in the heated environment at the moment with some news publishers attacking Google as a supposed content thief, that search engines are publishing their news content without permission.

Yes, all the major search engines provide a way for you to read a news article (or any web page) right on their own search engine, without leaving it. These are called “cached copies,” and you access them using the “cached” links that Google, Yahoo and Bing all offer. Here’s how the feature appears at Bing, below the page’s description:

The feature appears similarly at Google and Yahoo.

Very few people, to my knowledge, access pages this way. Moreover, publishers are totally in control to turn off this feature. They don’t have to be cached. In addition, publishers can opt out of being listed within search results entirely — no cached copy, no listing, nothing.

It sure would be easier if none of the search engines cached pages, of course, at least from the perspective of having to explain how the services don’t really use content but rather point at it on other sites.

My post, Search Engines, Permissions & Moving Forward In Copyright Battles, goes into more depth about cached pages, how it’s been ruled legal so far in the US, the type of copying search engines do to make content searchable and the confusion with that being reprinting. I recommend reading it, for those who want to learn more about the matter.

News Search Engines

Many major search engines operate similarly to regular search engines. They have crawlers that find pages, which go into an index, which are made searchable. For example, here are matches for “emission talks” from Google News:

The key difference between news search engines and regular ones is that news search engines cover fewer sources. Rather than looking across the entire web for content, they’ll only visit a select list of news sites, a few thousand sources to maybe up to 30,000. They’ll also visit these sites constantly through the day, staying alert for when news is posted.

Unlike with regular search, none of the news search engines offered by Google (Google News), Yahoo (Yahoo News) or Bing (Bing News), present cached copies of news articles. However, Google and Yahoo allow you to read news right on their sites in another way, through licensing agreements. For example, the AP has deals with both companies to host AP stories within their respective news services.

As with regular search engines, publishers can opt-out of inclusion at any time.

News Aggregators

News aggregators, commonly shortened to aggregators, is a terrible name for incredibly useful services that bring together headlines from multiple news sources all into one place. The news stories are all “aggregated” together, hence the aggregator name.

The major news search engines I named above also have aggregator sides. Remember how you could search on Google News about emission talks? Well, you can also just browse news headlines and discover that topic:

That screenshot is the Google News home page. Google uses an algorithm to look at all the news stories out there and assembles them together to effectively make a custom newspaper for its visitors. Yahoo News does the same, as does Bing to some degree.

Ah, so this is how search engines are ripping off newspapers! They’re using content from all these newspapers to steal visitors away from the newspapers’ own web sites! Yes and no.

The search engines are only showing headlines and sometimes short summaries, along with thumbnail images, to create their blended news pages. You can’t read the actual stories, except in the few cases where there’s an explicit licensing agreement.

Instead, people click from the news aggregation site to the news sources themselves. The search engines typically content such linking is fair use. For the most part, news publications — with some notable exceptions — have been happy being listed, in exchange for the traffic they receive.

Beyond the major search engines, other aggregators exist. For example, I constantly use the Techmeme aggregator to keep up with news in the tech space:

Where as Google picks stories for its aggregator purely on an automated basis, Techmeme uses a combination of automation and human editors. So does Yahoo News, by the way.

The Drudge Report is another aggregator:

I’m not a regular reader, but to my understanding, Drudge is primarily human-powered, where editors are manually scanning news sites and deciding what to feature.

AllThingD, which is owned by News Corp that has spoken against aggregators, ironically offers another example of aggregation — the Voices section where AllThingsD editors pick interesting articles from across the web. Here’s an example of where my publication appeared in the Voices section recently:

Personally, I find it flattering to get mentioned in Voices, not to mention I’ll take the visitors who my discover my site when it is featured there. I also appreciate that AllThingsD understands the odd situation it’s in, when its corporate owner is speaking out against something its editors find useful. The site maintains a page addressing issues on this, saying in part:

We are fully aware of the controversies around how linking and aggregating is done on the Web and we, in no way, are attempting to “scrape” original content created by others. Instead, regarding third-party posts, we are trying to point readers of this site to other posts from around the Web that we admire and are trying to do so in the quickest manner possible.

The Internet is full of terrific content that is not ours and we want to help our readers find it by making editorial suggestions–Look, Mom, no algorithm!–of posts we think are worth their time.

Aggregators run by the major search engines respect the ability for people to automatically opt-out from being included. Those that use human-power may require that a publication request that it is not linked to. They might also not honor such requests, arguing that they can link to any public document on the web that they’d like.

Personal Aggregators / Newsreaders

If aggregators sounds cool as a reader (or potentially evil to some publishers), the world gets even more complex when we talk about personal aggregators or newsreaders. These are services that allow you to take in content feeds from the sources you select, in order to form your own super personalized newspaper.

For example, here are headlines from a variety of news sources that automatically flow into my personal aggregator, Google Reader:

All the publications you see shown above — the New York Times, the Wall Street Journal and the Los Angeles Times — explicitly put their headlines out through a feed (also called RSS) in order to have individuals subscribe to their latest news.

Even the Associated Press, which seems to view its headlines and story summaries as content that should be licensed by companies such as Google, freely invites those with personal aggregators to take its feeds:

As long as it’s for personal, non-commercial use, the AP is fine with its feeds being used. Commercial entities are supposed to see permission, as the AP says:

AP provides these RSS feeds to individuals for personal, noncommercial use under the following terms and conditions. All others, including AP members or Press Association subscribers must obtain express written permission prior to use of these RSS feeds. AP provides these RSS feeds at no charge to you for your personal, noncommercial use.

In my experience, however, the AP is relatively unique in being this exclusionary of feed content.

I think a key point to personal aggregators is that there are some publishers who wish aggregators never existed. That somehow, someway, they wish they could push that genie  back into the bottle. That if there were never aggregators, more people would somehow come directly to them each day.

I have a future “Ode To An Aggregator” post where I’ll give my own view on why I think aggregators likely give publications gains, not losses, in terms of visits. The short story is that we’re well past a world where people start their day with a single publication. Or if they do, aggregators serve an important opportunity for other publications to also be seen alongside someone’s primary news choice.

Even if public aggregation sites were suddenly outlawed, it’s hard to imagine that personal aggregation would disappear. The future seems to be for personalized mix-and-match news reading to continue.

Blogs, News Blogs & News Sites

Bloggers are probably the biggest challenge to news sites that believe they’re being “ripped off” somehow in an internet-related way. Bloggers is a big word, however. I’ll try to define it a bit more, along with some use situations.

At the most basic level, a “blog” is simply a web site that is characterized by using publishing software with certain characteristics: each new page or “post” typically pushes previous posts further down on the home page of the site. Archives are built, allowing you to find pages by date or category. Comments are often allowed. Typically, “trackbacks” may be shown, reflecting other blogs that link to a particular article.

Some bloggers cover personal topics and have no ads. Some bloggers cover personal topics and yet are commercial in nature. Some blogs cover news, but only link to what others have written. Some blogs cover news, writing both original content and linking to others. And this is only part of the spectrum.

When I hear publishers speak out against blogs, the view in my mind that they are angry about is maybe something like Gawker or Andrew Sullivan, highly visible sites that often highlight stories from other publications, sometimes quoting from them and adding their own commentary. Some publishers view these sites as “cherry-picking” all the good stuff and costing them visitors that won’t bother reading the original source.

For a good case study of this debate, see The Death of Journalism (Gawker Edition) from Ian Shapira, a Washington Post reporter who was initially pleased to have his article featured by Gawker but then reconsidered as an editor suggested that his story was “stolen.” Also see The Time Gawker Put the Washington Post Out of Business, which among other things, highlights how ironically, the Washington Post is pushing bloggers to, well, blog about its stories.

Of course, blogs like Gawker and Andrew Sullivan may also have original content. So might the Huffington Post, which can be notorious in some quarters for seemingly appropriating a story elsewhere and making it seem like a HuffPo article.

I deal with getting the balance right every day at my own publication, Search Engine Land. There, we review over 200 sources each day to spot interesting news. Each day, we feature some items that have been spotted by others. However, we also regularly have our own original content that in turn might be cited by others.

News publishers with a conservative view on fair use would seem to wish news blogs and non-mainstream news sites away. The reality is that they are unlikely to go. And proposals for new “hot news” laws are complicated when mainstream publications themselves borrow from each other or get tipped to stories from blogs.

Moving Forward

Right now, we seem to be moving further toward an impasse. Two major news publishers, the AP and News Corporation, continue to make noises that they feel even merely linking to their content is potentially a copyright violation. Both have suggested they’ll withdraw their content from Google or other search engines that don’t comply with their demands. There’s also the suggestion that News Corp may try to enlist other news publishers to pull out of Google in particular.

As my Thoughts On A “Killer” Bing-News Corp Deal & The Myth Of An “OPEC For News” and Josh Cohen Of Google News On Paywalls, Partnerships & Working With Publishers articles touch on more, news blogs in particular pose a challenge to any would-be news boycott. News is not solely discovered by mainstream news publications, nor legally is it clear they could completely shut down summaries of news out there. In addition, some mainstream news publications are perfectly happy to continue partnering with blogs or Google. Reuters, for example, is a believer in the “link economy.”

I’d like to see less antagonism and more discussion among interested parties to find a way that everyone can get along. Personally, I’ve felt both the AP and News Corporation have been overly hostile. Just last month, we had the managing editor of the Wall Street Journal accusing people of being net neanderthals. In turn, that can create hostility on the other “side” of things.

Part of the solution might be some type of common subscription that blogs could take out entitling them to make use of content from mainstream publications — the ability to quote, summarize and so on without fear of legal threats. It might be that people might simply be buying what’s already covered under fair use. But I think plenty of bloggers also appreciate the symbiotic relationship they have with mainstream media and would like to support them in some way, especially if they can more easily use their content. Of course, such fees would have to be reasonable. Blogs aren’t gold mines. But perhaps something could be done.

From the publishers, I think the aggregators, bloggers and others need something back. First and foremost, probably a little respect that they’re not all just rip-off artists that provide no value. In an economy where the mainstream media keeps laying off staff, many blogs are actually hiring journalists to do original journalism. They’re succeeding not by ripping off the mainstream media but for being fast, nimble and adapting to an internet world in the way mainstream publications should have.

Respect also means more credit, so that if a story tip comes off a blog (or forum or elsewhere), this is made clear. Crystal clear, too, such as in the form of an outbound link. Let’s see an end to the days of the mainstream media not linking out.

Anyway, those are some concluding remarks that I’ll continue to ponder.