Delicious’s Data Policy is Like Setting a Museum on Fire

Yahoo! is going to shutter its social bookmarking service Delicious, the web learned today, and with it will sink an incredibly valuable source of collectively curated knowledge. You can easily export your own bookmarks (no verdict yet where we should all meet up to import them to) but what if you want to export other peoples’? That’s at least half the value of the service, socially curated discovery.

Tonight I thought I’d go loot a little from a burning building owned by a company not interested in putting out the fire. Specifically, I went to extract the top 50 links to pages that had been tagged by users with both the words “Twitter” and “International”. Where else are you going to find a reading list of the best collected written works and other multimedia about almost any given topic? Unfortunately, automated extraction is blocked by the site and the rickety, antiquated API appears focused on returning you little more than your own bookmarks. If there’s a clear way to accomplish export of not just my bookmarks, but all bookmarks with one or more tags, from all users – I haven’t been able to find it yet.

Sponsor

Doing it DIY Style

Using data extraction tools like Needlebase (read all about it) or Yahoo’s own Dapper (our coverage), I could have extracted every link ever tagged Twitter and International on Delicious in about 120 seconds. Then I could have pulled down all the links to articles that Delicious users have categorized as worth saving and pertaining to Portland and History. Then maybe Mapping and Gender. Then dogs and jokes or beans and flatulence.

Or whatever. But can I? Nope, Yahoo! blocks all automated extraction of data from Delicious. The company apparently is going to let this unique cross between a museum, a library and a crazy old collector’s attic burn to the ground. I’d like to take a few things with me before that happens, please.

In the screenshot above, for example, I’m trying to use Needlebase to go through every page of links bookmarked both Twitter and International, pull down the titles, links, tags and number of times each link has been bookmarked. Then I would export in CSV, import it into Google Docs, sort by the number column and archive like the top 50 articles about International use of Twitter. I could do that in about 2 minutes, if Delicious would allow it. And then I’d move on to another topic and do the same thing. But can I? No! Right now at least, the site prohibits automated extraction of its data. I sure hope Yahoo! changes its mind. Otherwise, there goes an incredible body of curated knowledge. What a sick tragedy.

Where else are you going to find a reading list of the best collected written works and other multimedia about almost any given topic?

One community of non-profit technologists has been bookmarking links with the tag “NPTech” for years – they have 24,028 links categorized as relevant for organizations seeking to change the world and peoples’ lives using technology. Wouldn’t it be good to have that body of data, metadata and curated resources available elsewhere once Delicious is gone?

What someone probably ought to do, as Karl Long said to me on Twitter today, is scrape all the public bookmarks and data and put it on Bittorrent. That would be against the rules, though.

Please, please Yahoo! let us save some of what you’ve got, before it goes to waste.

Discuss