I’ve written before about something that would really set a rocket under the opening up of data: the vigorous pursuit of the useful stuff.

When we’ve been given access to transport data, wonderful things have happened. When we get real-time feeds, useful services follow hot on their heels. Let’s make those infrastructural building blocks of services available for free, unfettered use: the maps, the postcodes, the electoral roll, your personal health records.

(Ok, I didn’t mean the latter two. Or did I? It gets complicated. Still writing that post…)

Here’s a vision:

Roll forward to a time when the first priority of any service owner within the public sector is not “how shall I display the accounting information about the costs of this service” (or indeed “how shall I obfuscate the accounting information..?”).

No. Instead, it is: WHERE is the service? WHEN is the service? WHAT is the service? HOW DO I USE the service? (And maybe even: WHAT DO PEOPLE THINK about the service?)

Those basic, factual jigsaw pieces that allow any service to be found, understood, described and interacted with. From a map of where things can be found, to always-up-to-date information about their condition, and a nice set of APIs with which others can build ways in.

The genius of this type of thinking being that many of the operational headaches of current service delivery simply fall away. They are no longer a concern for the service owner. “Our content management system can’t show the information quite like that.” “We haven’t got the staff to go building a mapping interface.” “We’re not quite sure how we’d slot all that into our website’s information architecture.”

Pouf. No more. Gone. The primary concern becomes: is the data that describes this service accurate (or accurate enough–with some canny thinking about how it might then be written to and corrected), and available (using a broad definition of availability which considers things like interoperability standards).

Well, Paul. Nice. But what a load of flowery language, you theoretical arm-waver. Can’t you give a more practical example?

Well, reader. Yes I can.


That’s right. Public conveniences. A universal need. A universal presence. But where are they? When are they open? And what about their special features? Disabled access? Disabled parking? Baby-changing?

There’s actually a bit more to think about (once you start to think hard) than just location and description. But not a whole lot more. The wonderful Gail Knight has been banging this drum for a while, and has made some good progress, especially on things like the specification for data you’d need to have to make a useful loo finder service.

Why’s this really interesting? Really, really interesting? Because having got a good idea of the usefulness of the data [tick] and a description of what good data looks like [tick] we then find all the other little gems that stand between A Great Idea, and a Service That Ordinary People Can Easily Use.

Who collects the data? Where does it get put? Who updates it? Who’s responsible if its wrong? How do people know they can trust it? Can people make money from it? (I could go on…)

Bear in mind that any additional burden of work on a local authority (who have some duties around the provision of public loos) probably isn’t going to fly too high in the current climate of cuts. Bear in mind also that anyone else who does a whole load of work like this is probably going to want something in return. Bear in mind also that “having a sensible standard” and “having a standard that everyone agrees is sensible” are two different things. Oh, and I need hardly add that much of this data will not currently be held in nice, accessible, extractable formats. If, indeed, it exists at all.

Two characters usually step forward at this point.

The first is the Big Stick Wielder (“well, they should just make councils publish this stuff. Send them a strong letter from the PM saying that this is now mandatory. That’s the standard. Get on with it. It’s only dumping a file from a database to somewhere on the Internet, innit?”) BSW may get a bit vague after this about precisely where on the Internet, and may, after a bit of mumbling start talking about a national database, or “a portal”, or how Atos could probably knock one up for under a million… (and it’s usually at this point that some clever flipchart jockey will say “Why just loos? Let’s make a generic, EVERYTHING-finder! Let’s stretch out that scope until we’ve got something really unwieldy massive on our hands”.) We know how this song goes, don’t we?

The second is the Cuddly Crowd-Sourcer (“forget all that heavy top-down stuff, man. We have the tools. We have some data to start from. Let’s crack on and start building! Use a wiki. Get people involved. Make it all open and free.”) CCS’s turn to go a bit vague happens when pushed on things like: will this project ever move beyond a proof-of-concept? how do we get critical mass? does it need any marketing? can people charge for apps that reuse the data and add value to it? how do we choose the right tools?

Both have some good points, of course. And some shakier ones. That’s why this is a debate. If it were clear-cut, we’d have sorted it by now, and all be looking at apps that find useful stuff for us. And isn’t just a matter of WDTJ (Why don’t they just..?).

My suggestion? CCS is nearer the mark. Create a data collection tool which can take in and build on what already exists. Use Open Street Map as the destination for gathered data. Do get on with it.

Matthew Somerville’s excellent work to get an accurate data set of postbox locations and the Blue Plaque finder are obvious examples to draw inspiration from. Once in OSM, data can be got out again should the need arise. There will be a few wrinkles around the edges as app developers seek to make a return on what they build using the data. There may well be a case for publicly-funded development on top of the open data. But get the data there first. Make it a priority.

Because if, after years of trying to make real-world, practical, open, useful services based on data we continue as we are, with a pitiful selection of half-baked novelties and demonstrators of “what useful might look like, at some point in the indeterminate future” we’re badly letting ourselves down.

Basically, what I’m saying is: if we can’t get this right for something as well-defined and basic as loos, a lot of what we dream of in our hack-days and on our blogs about the potential of data will just go down the pan.



OK, so it seems it already exists. Or at least a London version of it anyway. Don’t you love it when that happens? Would be good to see how it progresses, and what its business model looks like. I like the way that data descriptions have been used e.g. “Pseudo-public” for that class of loos which aren’t formally public conveniences, but can easily be accessed and used – e.g. those in libraries, and cooperative shops. The crowd-update function looks good too.

In a way, this also shows up another headache that arises when spontaneous services start to appear: there is only one set of loos in the real-world. But each representation of them in an app or online service must go through the same process of ensuring accuracy and extent of coverage. Distributed information is always tricky to manage. Should we hope that several competing services make it into production, with the market determining which succeeds? Will that be the one with the best data? Or is there scope for an underpinning data service that feeds them all? (But then we court the central, mega-project problems again…)

Answers on a postcard, please.

Preaching to the unconverted

I’ve been getting this blogging thing all wrong. Three years of grinding out thoughts about public services and technology, generally pointed towards an audience already versed in the issues, have all been for nothing.

I’ve been missing the real audience. The one that truly needs to understand more about this stuff.

A spirited discussion on Tuesday with a doughty advocate for public transparency convinced me that I need a change of approach.

Our debate arose from his astonishment that it wasn’t possible for “government” to say at any one time how many people it employed. Despite this being an “obvious” factual issue in his eyes, no amount of requests seemed to be able to produce a meaningful answer.

My response “well, it’s not really a meaningful question” – didn’t go down too well. Even having navigated the complexities of what “being employed” might mean, with all its colour and texture of vacant posts, secondments, part-funded posts, long-term absentees and part-timers, I felt there were still problems with the concept of such a broad question.

If asked by an economist with a specialism in operational research or organisational productivity, I could possibly, possibly see some sort of tangible purpose to a question, but more likely a version targeted at a more specific organisation or sector than just “all of government”. Possibly.

I know this is heresy: information should be free, yadayadayada, and the motivation of the questioner unimportant. But open your mind just for a moment to the possibility that context may have some value, in light of what came next in our debate.

The moment when I realised I’d got all my public service technology blogging pointing in completely the wrong direction was when my interlocutor said “you technical guys – you can sort all this out – surely the systems know how many people are on each payroll? Just add them up every night. You could if you wanted to.”

Here was an acclaimed expert in transparency of information, someone who’d spent much of his professional life pursuing the dark corners of government’s secrecy and intransigence. And he thought that a few lines of code and a dictat to “just f-ing report it daily” would meet this requirement.

(A spurious requirement, I’d say, as the journalist asking the question would be likely to write the same story whatever the actual number they got in response to their question. Any Big Number would do the job – and hey, if no meaningful answer came forth, that would be an even better story. “How stupid are they! They don’t even know…” Win, whichever way you look at it.)

I blame the Daily Mail, of course (shorthand for any form of lazy, populist, press). As with most difficult public policy issues, from asylum seekers to disability claimants to identity, there’s always an easy, quick answer that will get heads nodding in the pub and taxi.

But which is almost always utterly, hopelessly, WRONG. Who wouldn’t like an easy answer to a hard question? To avoid any deeper thinking about the subject. Or acknowledgement of history, personal responsibility or sense of others? To gloss past the difficulties that arise when something that looks (from a huge distance) a tiny bit like a simple, familiar, backyard activity is attempted on a scale of tens of millions of people and transactions.

So here’s the plan: a post, or small series of posts, called “The Daily Mail Reader’s Guide to Public Services Technology”.

Taking some of the favourite old chestnuts (Why can’t they count X? Surely if everyone just had one ID number? Why so many different systems essentially doing the same job?) and really, anything else that begins with: “I don’t see why they can’t just…”

And writing them up in language that DM folk may identify with. Analogies from golf clubs, caravan parks, tea shops. You get the drift.

I’ll make a start, but do please add your suggestions here for topics that you’d like to see given the treatment.

Agile, waterfall and muppets

There’s been some very good debate of late about how to do it all better, with a heavy emphasis on the role that Agile methods might play. “It” in this case being not just government technology, but extending to policy development, communications and more.

There seems to be a massed rebellion against the substandard, the lame, the failed and the apparent deathgrip that a few large suppliers have on taxpayer billions. This is a very good thing, of course.

I found a couple of recent pieces very insightful (in different ways). Alistair Maughan kicked off a lot of debate with a provocative piece arguing that Agile was doomed to fail in a public service setting. This from one of the architects of arguably some of the biggest, most expensive and (inarguably) rigid ICT contracts imaginable. Adam McGreggor from Rewired State came back with evidence of Agile success, and a rebuttal of the good lawyer’s key arguments.

These arguments really seem to me to be based around an overarching sentiment that Agile weakens the ability to hold contractors (and their clients) to account for the delivery of fit-for-purpose products. Whether as a result of mismatched expectation, fuzzy requirements, incompetence or downright fraud.

Our defence against these–particularly the latter two–has traditionally been the much-lambasted public procurement process. As Anthony Zacharzewski put it with characteristic tact and style, we should have a care about throwing out the defences that these processes are designed to provide, for all the tales of woe that can also be laid at their door.

Stepping back a little from all this, I am left wondering what points are really being argued here? Is this about a methodology, or is this about ensuring that the right people are getting through the door? Amazing things can happen when the usual barriers are thrown down and the real, trusted experts are brought in. (I hope to see real-life evidence of some of this next week.)

After all, with amazingly insightful and competent people involved, respecting each other, listening to reason, taking risks where necessary, being flexible–and above any suggestion of conflicted interest or perverse incentive–even the most traditional requirement/specification/build approach is going to perform pretty well. And just imagine the horror of a perversely-motivated behemoth muscling in (let’s call them Anders*nAgile, say) with their Chicago-schooled ScrumBizAnalysts(TM) charging a couple of grand a day to dance in a slightly different style to the same old tunes.

If this is about trust–and I think a lot of it might be–then we need to be careful not to confuse “method” arguments with those that are more about “gatekeeping”. A great sage in the public sector IT world once said that the only HR rule you ever needed was “No muppets”. He had a point.

Not quite public

That old question came up recently: What really good online service experiences has government ever given us?

As usual, the first (and, sadly, often the last) answer: the online tax disc service. There’s no doubt it’s a properly good use of the online channel to save people hours of queueing and paper fiddling. It achieves its magic not with any fancy visual design–its interface isn’t that great (those five questions up front–why, just why?). And it still stubbornly refuses to update its strapline from one that was phased out several years ago. (Did you notice? No, of course not. Straplines are irrelevant.)

What matters are two bits of genius: one, the removal of any burdensome personal identification at the front end. No Government Gateway, no personal identifiers. Just a reference number that you type straight off the paper form that’s sent to you when its due. That’s it. If you have that number, and you have the means to pay, a tax disc will soon be on its way. Whoever you are. It’s about the car, not you.

The second miracle is the joining up of databases at the back-end. The car’s registration is used to call on MOT and insurance databases (information from completely different sectors, let alone organisations) to save you digging out slips of paper and doing all that queueing only to find out that one of them is a little bit out of date. Don’t underestimate how valuable that service join-up is.

But this post is not about tax discs. It’s about another online service, also from DVLAVOSA [updated: the MOT scheme is run by another Dept Transport agency, VOSA], that far fewer people know about.

And it’s not to illustrate a service point, for a change, it’s to explore an information point.

The MOT.

The evil cousin of the tax disc. It doesn’t display its expiry date for the world to see on your windscreen. Well, it does if you choose to fill in the little sticker that you get with your MOT pass certificate. But that depends on your choice. And nobody is going to punish you if you don’t do it, or if the little sticky peels off and gets lost. So we know what that means.

When’s your car’s MOT due? Yes, you! Do you know? Without finding the last certificate, which, let me guess, isn’t about your person or your desk as you read this.

You could find out online, you know. There’s a nifty little utility here. And what do you need to get access to that magic expiry date? Well, you need the registration number of your vehicle, naturally. Which you probably know.

And you need a reference number from your last test certificate (or failure notice).

Really. No, I’m not joking.

Ok, I’m exaggerating a little: if you don’t have the test certificate to hand, there is a fallback. You can use the number on your blue V5 form. The one that we oldsters still call “the logbook”. Now, let me take a wild guess as to where your logbook is kept? Any possibility it might be in the same drawer as your… You’re there already aren’t you? Give yourself a quick kick on the inside of your shins, DVLAVOSA service designers.

So we have a potentially brilliant online service, that, if promoted, could stop tens of thousands (my guess) of people slipping past their MOT expiry dates without realising. The only time they think of these things is in idle hours at their desks at work, while the documents they need languish in a dusty study drawer at home.

And what would make the service brilliant? Just making it usable on the basis of the registration number alone. Which would mean that anyone could look up anyone else’s MOT expiry status. (The crowd suck through their teeth…is that, I mean is that, ok?)

And is it?

The point (which I have finally got to) is that MOT status information is a curious dataset. It’s not quite private (well, it’s barely “protected” to any appreciable extent), and it’s definitely not public. Instead we’ve built a little friction around accessing it (needing to drag out a hard-to-find bit of paper rather than an easy-to-find remembered–or seen in the street–fact).

Does it feel like personal data to you? Would it bother you if your nosy neighbour could look up your missed test date and start leaving little passive-aggressive notes on your windscreen? Or should it be a public data set? Nothing to hide, nothing to fear and all that. And the bloody tax disc expiry date is printed loud and clear for all in the street to see, isn’t it? What’s the difference?

The only risks I can think of that are headed off by this rigmarole are the nosy neighbour one, or possibly a local garage touting for business on the basis they’ve spotted your car is coming up for a test soon, or a miserable Lazy Wail underling sitting in a grey basement tapping in slebs’ car registrations in the hope of getting a pathetic non-story.

That’s not a lot, is it? Am I missing something? Is that the entirety of the reason why we are denied an incredibly easy-to-implement online tool which would save us real time and real money?

Over to you. And over to you, DVLAVOSA, if you’re reading. Which I hope you are.

I’ll revisit this concept of quasi-public data soon. Things that aren’t quite public, aren’t quite private, and may well be personal. Things like the electoral roll, for example :)

The speed camera and the Public Data Corporation

Think of a speed camera.

Think of the proposal for the Public Data Corporation.

One of them has attracted controversy. This seems to be based on instinct or ideology, without much groundwork being put in on the complex models and circumstances that surround it, and what it might mean as part of a bigger picture.

Its supporters see it as a way of bringing some order to a complex system; of ensuring that things actually do move more quickly by introducing an element of regulation. That it will actually bring some accountability and ensure things don’t run recklessly out of control.

Its detractors see it as a cynical front for raising cash for the government.

Oh, and the other one is a speed camera… :-)

What I’m saying, of course, is: we don’t really have much evidence as yet – perhaps it would be good to tease some out before taking a strong position either way?

Number numbness, part 2

Our story begins with a number being bandied around. 1,800,000 to be precise. That being the reported cost in pounds sterling, of London’s New Year’s Eve fireworks. [Update Jan 2012: this post was written after New Year’s Eve 2010; the cost a year later of these breathtaking fireworks was–perhaps remarkably–only £100k higher.]

It certainly is a big number.

And when I see big numbers, particularly those getting a lot of bandying, one word tends to come immediately to mind:


I know—boring old analysis again. We’ve been here before. That fortune the BBC were spending on taxis. Well, perhaps not, set in the context of what they actually, you know, do?

Much easier to say: cor that’s a shed load of cash; shouldn’t have spent it in a time of austerity; looks terrible; could have given it all to charity, people are going to let off their own, anyway, aren’t they? (Don’t assume that I don’t agree with these sentiments, mind you. But this is a post about what might lie behind the number…)

What ran through my head were the usual dull questions: so, that £1.8m: what’s that the cost for, exactly? Just the physical fireworks themselves? Probably not. What about the labour to set them off, and the safety checks? And what about planning, permissions, policing? And clean-up and fire cover and first-aid provision?

There’s a lot involved. And crucially, were the fireworks not to have been fired, what would be the actual cash saving to be realised as a result? Probably somewhere south of £1.8m, I’d say. Costs aren’t as neatly packaged up as all that in real life—there’ll be commitments and interdependencies that will bring that number down.

So having had a bit of a think about the composition of costs, I might then think about comparison. Yes, I know it’s a lot of willy-waving between ‘global’ cities trying to look all impressive on each others’ news channels, but if Sydney, NYC, Paris etc. had spent, say, £3m each, would our reaction be any different? Maybe, maybe not. But it gives a sense of scaling. (I loved this comment in response to my request for a rational basis by which one might judge how much should have been spent.)

And in terms of other context, what about other big lumps of money associated with New Year’s Eve in London? Policing generally? Transport costs (other than Wonga’s wedge)? Stewarding? First-aid? Clean-up? If the £1.8m were considered as a proportion of the whole picture of London’s NYE activity, would that make a difference?

What of the direct beneficiaries of that £1.8m?—that’s a fair chunk of business for someone. I know it’s the last refuge of the unreconstructed Keynesian scoundrel to bleat on apologetically about trickle-down multiplier [durrr, my bad] effects, but could that make a difference to your view? We naturally assume that nearly two million has been spanked away out of the country on Chinese gunpowder and cardboard, but what if all the pyrotechnics had been hand-assembled in a big social enterprise thingy by East End orphans? Would that sway us?

Would it buggery. Because none of the above, interesting though they might be in theory, have any real connection to the popular reaction. It’s all about the symbolism. Only the sight of Boris himself standing on a barge in the middle of the Thames setting fire to a pile of fifties with a blowtorch could be stronger. It’s fun, it’s frivolous, it’s WASTE.

I suggest that trying to pare the costs down wouldn’t have been much use either. Apart from the tedious fact that full-on Big City celebratory fireworks actually do cost a ton of money, there’s a whole load of data we don’t have about which costs were fixed and which variable. “Well, open all that data up too” I hear you shout—sure, and that might help with some of the analysis, but it won’t change the public reaction.

No, only a complete absence of fireworks would serve to sway that. And then we’d have the screaming headlines “They’ve BANNED New Year’s Eve” and “London falls dark—a miserable, depressed city shuffles towards 2012 (and don’t even think about investing or doing business here—we’re shut)”. “OLYMPICS??—they can’t even organise a fireworks display!”

Not easy, really, is it? And this is what happens when you get a Big Number being bandied. As more spending data is opened up, I hope, I really do, that we develop other skills—the ability to analyse in context, to compare, to understand downstream effects and dependencies—in balance. But I worry that the temptation to focus on cheese budgets and, of course, fireworks, may grab all the attention.

And then, as I commented to Francis Maude and the Transparency Board last month, we end up aware only of costs, and not of value. And Oscar Wilde knew what that would turn us into.

Central or decentral?

Yes, nice easy question. Should be a short post.

One of the debates that stuck in my mind at the UK GovCamp 10 came from a session hosted by Alastair Smith. Ostensibly about the ‘UK snow’* and what that had meant for the likes of local authorities in delivering services and information. At least that’s what I think it was about. One can never quite tell with unconferences.

The difficult issue of managing information in disrupted conditions. One of my favourite subjects, be it weather, strikes, train disruptions or pandemics.

“How to tell people about school closures” is an excellent example.

Why’s it so difficult? Here’s a little list:

It’s a highly localised decision. It’s taken by the headteacher of a school, often at short notice. What if they’re stuck in snow, or can’t communicate their decision to anyone? We’re talking about disruption here, remember?

It’s highly time critical: if the information is to be useful it has to be delivered in the very tight window between decision and parents’ departure for school (or rearrangement of childcare, or whatever) and almost by definition this will be outside normal working hours.

There are no obligations or penalties associated with how well it’s done. (There may be a motivating issue about OFSTED reporting of absence, but I consider that secondary to the actual information process, so am discounting it from this analysis.)

There is no consistent, expected place to find the information. In some areas schools brief local authorities, in others local authorities brief local radio, there are numerous instances of online information, but little in the way of standardised approach.

Kids are involved. Kids who may just have a conflict of interest were there to be any opportunity to game the information. Just possibly.

A variety of tools are used to try and get the message out: from notifications that are actively sent to parents (by SMS, email or phone) – so-called information ‘push’; to information made available for consumption (by web, radio or pinned to the school gates) – the ‘pull’ side. Some parents and schools have developed cascade networks, formal or informal, to pass on the message. Others haven’t.

Do we have any plus sides? Well, the only one of note is that snow closure is usually predicted, to a greater or lesser extent. Something I suspect that fuels even more ire when information management fails. Surely, we cry, they must have know this might happen? Why weren’t they prepared?

Accustomed behaviours are highly personal. Parents have become used to a particular information channel, be it the radio or the web, and any changes to that will cause even more confusion, at least at first.

All complex stuff – did someone say that public service information management was easy?

But where the GovCamp discussion got most interesting was when we tackled the nub of the problem – the overarching philosophy of whether it was worth trying to centralise information at all in such circumstances. Even at the highest level, opinion is divided between attempting to centralise so that information can all be consumed in one place, and ensuring that it is maintained as locally as possible to guarantee its speed and accuracy.

For there are classic trade-offs in this decision. There is no unequivocal ‘right’ answer.

Get it to a central point of consumption (or data feed that can be consumed elsewhere) by whatever communications protocols and brute force pressures you can: advantage – easy to find; disadvantage – very difficult to make foolproof, prone to error.

Or keep it distributed, and make it easier for people to get closer to the source of the decision to get the most accurate picture: advantage – saves money, fast-when-it-works, accurate; disadvantage – hit-and-miss, accessibility, findability.

The list of challenges above should make it clear why this is far from the trivial information management problem that some might assume. One chap in the GovCamp session maintained that all it would take would be a firm hand of authority to be laid on headteachers to comply (“or else their school would be assumed to be open”). I fear that view represents a hopelessly outdated approach to getting things done that actually work.

I’ll come off the fence. I think the answer to a problem like this doesn’t lie in ever more sophisticated linking and aggregation. Building big central solutions, even with a grass-roots crowdsourcing component, probably isn’t going to work.

Instead, my experience and my gut are combining to suggest that local is the place for this information. Ubiquitously local – on school sites, via SMS, on the radio, via local authorities. Keeping them in step is the challenge: but a challenge that’s more worthy of effort than building elaborate information pipelines and monumental repositories.

*if you’re wondering why this phrasing is used, there’s some background here – which might also show why I’m so interested in it.