I listened to Roy Orbison and I got Open Data wrong

I think Roy Orbison’s approach to Open Data is not good at all. I will get there in a minute. I know, you will think this is just a little stunt to get your attention (Ed.: well, it is…) — but I promise it will make sense soon.

Last week I was given the opportunity to talk about The Open Data Delusion to the GeoMob crowd. A couple of months have now passed since the original article was published. I have had an opportunity to refine my thoughts and elaborate a little on why I think Open Data is leaving me disappointed.

If you know me and have met me between 2010 and 2015, you probably thought I was a true believer in Open Data. Up to a point, I still am. I think Open Data has been a good thing for Governments and Citizens alike. I was at one point called “The Open Data Rottweiler” by my colleague Nick Halliday. But I also like to recognise hype, it has become apparent to me that Open Data has been venerated beyond its merits.

The original sin of Open Data is Transparency. Transparency is a good thing, you will tell me — and it is when it constitutes the inspiration and the method by which things happen. Unfortunately for Open Data, Transparency became very quickly the end goal, rather than an inspiration.

A key sinner in this context is a man that will soon be in the history books for a variety of reasons. The man who, at the height of the Parliamentary Expenses Scandal, managed to transform Open Data: from the outcome of an increasingly transparent Government, into the preferred method leading to the final goal of increased Public Sector Transparency.

Just imagine the effect that an army of armchair auditors is going to have on those expense claims. — David Cameron, 25 May 2009

Many got into this bandwagon, and I was not immune to it. The rhetoric of “Transparency as an end, achieved by way of Open Data” is a strong one, and it did have limited initial success that encouraged activists. What this rhetoric fails to understand is that Transparency lives through political cycles. It comes — and goes — in waves. The high points of this political wave is, more often than not, concurrent with a change of Government; it comes as a response to the electorate’s sense of disenfranchisement and detachment from its political leaders. This explains the cultural shifts that brought New Labour to espouse Freedom Of Information in 1997 and the Conservatives to adopt Open Data as their own platform in 2010.

As we all should know by now, as Governments stick to power they naturally drift away from the Transparency rhetoric. It is a natural (and, up to a point, acceptable) phenomenon. What puzzles me is that most Open Data activists have, so far, failed to accept it. If they did, they could be able to surf that wave. As a Freedom of Information Commission is set up to “review” (i.e. curtail) the FOI Act, the political context is no longer fertile for Openness to be a positive drive. Hence why the disappearance of the various transparency advisory groups should not be take us by surprise and the move to the Data Government Programme (where have they forgotten the “Open”?) is a more than natural response.

You might object to this by remarking that there are still parts of Government operating on the basis of Openness. The wealth of data released by DEFRA (and especially its Environment Agency) are certainly a good example. However, this is also where Transparency is showing its limits. The Department has shown so far (under Secretary of State Liz Truss) a big focus on figures. Although the drive to Openness has been welcome, it is arguably becoming increasingly meaningless as we move on. Knowing that DEFRA has released a grand total of 11,007 datasets, accurate to the unit, does not tell us anything about the quality of those datasets, the procedures that generated them, the guarantees on keeping them updated — if anything, it makes it sound very difficult to keep them promptly updated. It is difficult to believe that the department will go on in this direction under the new Secretary of State Andrea Leadsom. Most importantly, it does not tell us much about which problems these dataset could help address. I use the word “problem” willingly.

Let me do a little historical digression. “Open Data” was first heard of in 2005 as a response to Hurricane Katrina. A cataclysmic devastation, counterpointed by a lack of Government action, turned into a successful citizen activism story by showing that data gathered by volunteers from public sector website helped bring relief to the population affected by life-threatening floods. Open Data was something that could save lives. At the beginning, Open Data was something that citizens did.

Almost 10 years later, things have changed. As a response to the floods in South West England in 2013–2014, it was the Environment Agency that proactively released datasets, including a very high-quality API that publishes level of river gauges almost in real time.

These two flooding events have in common one feature: data was a means to an end. There was a problem seeking a solution, and that solution came by means of Open Data. In both cases, the release of data did not involve tens of thousands of datasets. In the US case, Open Data actively contributed to savings lives; in the UK, data enabled communities to start manage risks locally, which is amazingly empowering.

Both stories are powerful. They show how data can have an impact on everyone’s life. That the Governments prefer the PR coming from the release of thousands of datasets to the way more positive press they can get from Open Data positively impacting citizens’ lives, escapes my understanding and it stinks of short-termism.

I think the rhetoric has outstripped reality — Emer Coleman

The rhetoric around Open Data has always been centred on promises, rather than outcomes. Outcomes of something “Open” are difficult to evaluate, most of the times because data producers might not even be aware of who the users are. But on the other hand, credence has been given to rather dubious promises, including the infamous McKinsey report which boldly states

Open data — public information and shared data from private sources — can help create $3 trillion a year of value in seven areas of the global economy.

So, allow me to get back to our good old Roy Orbison. You might remember his song “You got it”.

Anything you want, you got it.
Anything you need, you got it.
Anything at all, you got it. — Roy Orbison

This has been the approach of Open Data activists in the past 10 years. The transparency-focussed asking for everything they wanted, the usage-oriented asking for anything they needed, both groups trying to grab as much as they could. You don’t need to be opposing Open Data — and certainly I do not — in order to recognise that this approach does not work. Part of it is determined by the obvious recognition that we live in a world of finite resources. Government resources are especially finite, and in the context of Brexit they will be even more so. We can keep pushing for good quality data, update guarantees, and so on, but in the current climate this is unlikely to lead to any good development.

The only possible way for Open Data to succeed is to start again and think about Katrina and the UK flooding; i.e. start again from a set of problems; start again from finding a solution to those problem; start again by understanding how using data can make a solution more effective. If we do encourage a problem-solving approach to Open Data, we will be able to navigate the Transparency wave: use it to push for more data when it’s high, consolidate on existing processes when it’s low. If we don’t, there is a huge risk, a risk that increasingly looks like a certainty, that no one will be talking Open Data for a long, long time.