Local government data: how to make it really open

Powered by Guardian.co.uk This article titled “Local government data: how to make it really open” was written by Simon Rogers, for theguardian.com on Wednesday 7th July 2010 07.00 UTC

We all thought Coins was going to be the government’s promised “tsunami of data”, but the real data storm is going to come when local government (under Downing Street duress) will release every spending item over £500.

This should be a moment to celebrate, for developers, journalists and everyone concerned with how councils spend our council tax. Instead, campaigners are united in anxiety that what we might get could just be more of the same.

And it all started so well. Local government secretary Eric Pickles told councils that:

“I don’t expect everyone to get it right first time, but I do expect everyone to do it”.

Well getting it wrong might be the default position for some local authorities. CountCulture’s Chris Taggart is concerned about data company Spikes Cavell’s SpotlightOnSpend muscling in on local government data (you can see his latest post on the issue here).

The upshot seems to be this, councils hand over all their valuable financial data to a company which aggregates for its own purposes, and, er, doesn’t open up the data, shooting down all those goals of mashing up the data, using the community to analyse and undermining much of the good work that’s been done.

Paul Bradshaw reports that a Help Me Investigate page has been set up over the issue, to see how widespread it really is.

Spikes Cavell has been stung by the furore – chief executive Luke Spikes has pledged to allow raw data downloads, according to Information Age.

As it is, there is a real fear that councils could get it clangingly wrong. Openlylocal’s data scoreboard shows that only 15 out of 434 local councils are publishing open data at the moment – only seven of them in a truly open format.

There seems to be a panic up and down the country among councils suddenly faced with releasing data they’ve previously kept to themselves – presumably combined with beffudlement over why they have to do it at all. If that panic translates into a default position of outsourcing the task, then we have real problems.

The thing is, there are no shortage of official guidelines showing exactly how to release the data. The Local Data Panel has a concise and clear set of principles for local data release – worth reading for their clarity alone. The Open Knowledge Foundation does too.

Essentially, they boil down to some pretty simple ideas:

1. Make it open

No T&Cs about not using the data for commercial use, no restrictions on access. Make the data available to anyone to do whatever they want to with it. That’s the only way that the data information revolution is going to work.

2. Make it readable for computers

The data needs to be in a format that any computer can use – no more PDFs, thank you very much. If developers can’t build applications and campaigners can’t analyse it, what use is it?

3. Make it granular

The days when we only wanted official statisticians to just put the numbers together in a way we could understand are gone. Now we also want the full, disaggregated data too. It’s the only way it will ever be useful for someone wanting to gather the true local picture of local spending. Let us worry about whether the dataset is too big or not. It’s not your problem anymore.

4. Make it quick

Just get the stuff out there. We’d rather have it as it is – and then get it revised later than have to wait months for it to be finalised. The government has provided express permission for local authorities to do this. So just do it.