The death of robots.txt?

Last night I linked to an interview with Rupert Murdoch in which he says that News Corp will probably de-index their sites from Google.

I figured it was all bluster. Search engine traffic is more valuable that Murdoch suggests, and there are probably plenty of people in high places at News Corp who know it.

But Cory Doctorow suggests:

So here’s what I think it going on. Murdoch has no intention of shutting down search-engine traffic to his sites, but he’s still having lurid fantasies inspired by the momentary insanity that caused Google to pay him for the exclusive right to index MySpace (thus momentarily rendering MySpace a visionary business-move instead of a ten-minutes-behind-the-curve cash-dump).

So what he’s hoping is that a second-tier search engine like Bing or Ask (or, better yet, some search tool you’ve never heard of that just got $50MM in venture capital) will give him half a year’s operating budget in exchange for a competitive advantage over Google.

Jason Calacanis has suggested this approach as a means to “kill Google.”

But it may actually be neither the death of Google, nor the death of News Corp if they are so foolish as to carry out this plan. It could be the death of the robots exclusion standard. I would guess News Corp would use robots.txt to de-index their sites. But it’s a “purely advisory” protocol that Google is under no obligation to honor. They could continue indexing News Corps if they so choose. So could every other search engine, big or small. And I’d guess they would if big content providers started going exclusive with search engines.

If News Corps puts all its contend behind a pay wall, this point is moot – Google and other search engines won’t be able to index it, and robots.txt will be fine. But it’s something to think about.

(Hat tips to Jay Rosen for the TimesSelect link and Chris Arkenberg for the Jason Calacanis video)

  • Share/Bookmark

Murdoch: We’ll probably remove our sites from Google’s index

Rupert Murdoch has suggested that News Corporation is likely to make its content unfindable to users on Google when it launches its paid content starategy .

When Murdoch and other senior News Corp lieutenants have criticised aggregators such as Google for taking a free ride on its content, commentators have questioned why the company doesn’t simply make its content invisible to search engines.

Using the robots.txt protocol on a site indicates to automated web spiders such as Google’s not to index that particular page or to serve up lionks to it in users’ search results.

Murodch claimed that readers who randomly reach a page via search have little value to advertisers. Asked by Sky News political editor David Speers why News hasn’t therefore made its sites invisible to Google, Murdoch replied: “I think we will.”

Mumbrella: Murdoch: We’ll probably remove our sites from Google’s index

(via Jay Rosen)

I’d be quite happy to see News Corps shoot themselves in the foot, but I have the feeling people who actually know what they are talking about will stop this from happening.

  • Share/Bookmark

Key 23 acquired, Technoccult shuts down

I’m proud to announce today that Key 23 has been aquired by News Corp (for, of course, an undisclosed sum). From the news release:

We’re proud to announce that Key23 has been acquired by News Corporation. Together we’ll continue to improve how people discover and share their memeplexes, with a big emphasis on the power of community. We’re excited to be working with the News Corp team – they definitely get what we’re all about and our potential.

“Key23 is an important acquisition for News Corp., instantly doubling our understanding of how to manipulate ‘countercultural’ audiences and providing an ideal foundation on which to meaningfully increase our presence in those niches,” News Corp.’s Chairman and CEO Rupert Murdoch said in a statement.

I’m taking my share of the l00t and moving to the Carribean. Therefore, this site and my other blogs will shut down. In the mean time, you can check the work of my namesake Clint Finlay at The All New, All Different Key 23.

  • Share/Bookmark

Outfoxed: Rupert Murdoch’s War On Journalism

Torrent: 703.05 MB AVI (RAR compressed).

You probably already know what this is: Robert Greenwald’s excellent documentary challenging Fox News’s “Fair and Balanced” doctrine.

Buy on DVD for $9.95.

See also:

The Corporation.

magical assault on corporations by Wu.

The Corporate Egregore by LVX23.

“My Love War with Fox News” by LVX23 in Generation Hex.

  • Share/Bookmark

Technoccult Presents

<a href="http://psychetect.bandcamp.com/album/return-to-the-wasteland">Awakening by Psychetect</a>

Archives