The Geek’s Handbook of 1001 Privacy Formations

Douglas Adams got there first, of course. He usually did.

In a universe where past and future lost their rigid meanings, and time was as capable of twists and reverses as any spatial dimension, Adams gave us Dr Dan Streetmentioner.

The good doctor realised the limitations of conventional grammar in describing these temporal quirks, so produced the Time Traveller’s Handbook of 1001 Tense Formations – an exhaustive treatise on how to describe them properly.

It will tell you for instance how to describe something that was about to happen to you in the past before you avoided it by time-jumping forward two days in order to avoid it. The event will be described differently according to whether you are talking about it from the standpoint of your own natural time, from a time in the further future, or a time in the further past and is further complicated by the possibility of conducting conversations whilst you are actually travelling from one time to another with the intention of becoming your own father or mother.

Most readers get as far as the Future Semi-Conditionally Modified Subinverted Plagal Past Subjunctive Intentional before giving up: and in fact in later editions of the book all the pages beyond this point have been left blank to save on printing costs.

I’m reminded of all this by something I read tonight about Lord Winston’s response to a noisy passenger on his train. I’ve done a few notes previously in similar territory of what is and isn’t (or might and might not be) ok when private things happen in public.

The concepts of “public” and “private” spaces simply aren’t as clear-cut as they used to be: social platforms enable all sorts of breaches in traditional conventions about what constitutes fair game in terms of the public gaze. How public is public, anyway? Local to a train carriage, to the audience of a prominent media figure, or to those with special interest in a particular hashtag?

The vocabulary just doesn’t exist to describe the nuances of things that would previously never be seen, but now are.

(I toyed with the ugly term “broad-availabling” when I first saw what happened when you brought together a search function and a Twitter account with only a handful of followers. In no way could our cellar-dwelling ranter be said to be broadcasting, but given a certain level of energy and enthusiasm, his reckons could find themselves amplified as anything traditional media might emit.)

So we need a Dr Dan for the new realms of the publics and the privates, and all the lands that lie between. Might go a bit like this:

He’s been sharing details of our purribrate life with his mates again! – [Translation] My paramour told his friends what that gesture I do in my insta selfies really means.

Problem with the railways is everything goes to shit when they’re run by companies in the pubravaging sector. – [Translation] My train is late again because it’s operated through such a convoluted structure of joint ventures, subsidies and overseas quasi-governmental entities that nobody really knows who’s in charge.

If they dare to search my privennials before my job interview, I’ll report them to the ICO, so I will. – [Translation] I am concerned that my social media trail may have unforeseen repercussions.

Verification: I can’t even

I can’t even – and neither can they…

Yes folks, it’s back again! The Queen’s Speech today promises yet another Mumsnet/Mail pleasing crackdown on one-handed websurfing – age verification!

Ha, brilliant – so obvious – all we have to do to send the kids back to the era of damp grotmags in the bushes is do a bit of proving-who-you are when someone clicks their way to a nacky site. No proof, no nacky.

Couldn’t be easier!

So how are they going to make it work then?

Short answer: they can’t.

Longer answer: they’d have to solve the Big Problem, and also some Littler Problems.

The Big Problem is an ancient conundrum: how do you build a checking system that’s solid enough to be worth doing, but not so solid that it doesn’t immediately bugger up the life of someone who loses access to their digital self?

Solid example: imagine you have a password that will ‘prove’ who you are wherever you use it, to anyone (we gloss over here how that trust might actually be set up). Lovely! But anyone who nicks that from the Post-It on the side of your monitor can then start buggering up your life. So you add a special chip they have to hold at the same time, and a scan of their toeprints that has to match, and…and…you’ve got something that’s so clunky that no one will be able to use it reliably.

Less-solid example: you have to upload a paper document of some degree of ‘officialness’ – perhaps a driving licence or similar – or type in some reference number from it – and someone on the other end agrees to let you in. Cue instant exchange of document scans – anyone’s will do – and reference numbers between bulging-balled/clitted teens.

Or you could try and connect identity to payment; the “credit card as key” approach – cue even more bad things happening involving credit cards and real hard money.

So that’s the Big Problem: any system with very strong trust is a magnet for people who want to do bad things with it. And I’m not talking about watching-porn-bad-things. Because that’s not bad. But that’s a whole different (mass) debate.

But let’s assume we do want to have some system that’s worth doing: we have two options – build a central identity register (think of it as a single digital “you” that can be checked, tracked etc.) and have you prove your right to be identified as that person; or establish the trust in other ways.

Without rehashing all the central registry arguments – though you can check out Achilles & the Tortoise for a bit of light relief (tl;dr vulnerable to attack by undesirables, or misuse by a State gone Bad, all eggs in one basket) suffice to say that government thinking of late has steered away from such a thing. For now.

The alternative approach rests on a nice workaround: if you can prove who you are to organisations that already know about you – and they do their job to an agreed quality standard – then that trust can be taken, well, on trust by other services. Your bank went to huge amounts of trouble to find out who you were, so if they say you’re you, you probably are. And actually, for age verification, they don’t even need to say much about you to the porn-keepers – merely confirming that someone’s at the door with age >= 18 (or whatever) will do the job.

The great Dave Birch has done the most elegant job I’ve seen of describing how you’d do this.

All neat and compact and a whole lot less terrifying than having a great, groaning Database of Everyone sitting in a Cap Gemini data centre.

This is essentially what the government’s Verify programme of identity assurance is currently trying to do. It involves solving a number of Littler Problems.

– what sort of organisations know enough about enough of the population to be able to accurately and reliably work at the scale of millions of people?

– how good is their data, and might they have to ship in data from other sources to fill in any gaps?

– what’s in it for them? i.e. what’s the business model for them to do all these verifications?

– how’s everything going to be kept safe, and how can that be shown to everyone’s satisfaction?

– how much risk should we plan in? Identity is never ‘proven’ as such; merely claimed within an accepted range of risk. Otherwise systems would be unusable by normal humans, and break all the time.

– who picks up the bits when things go wrong? (which they will – no system is 100% safe) – this of course harks back to the Big Problem – if you really want a universal key to lots of services through a simple interface, have you also opened up a bottomless pit of liabilities when that trust is compromised?

and so on. Incidentally, all that while facing the spectre of individual government departments who have their own wide-ranging databases about us and who may continue to itch, as they’ve always itched, to use those databases to vet you against. Why rely on transferring trust from a third party when you can assure it in-house, they might say?

So that’s a crash through what’s involved as a result of today’s declarations. Not really that easy, huh?

Oh, and you do all of the above and you still have to do some incredible amounts of Whack-A-Mole to stop other porn sites springing up that you might not know about, and who might not give a stuff about these crazy UK requirements to prove age oh dear me hahahahaaaaa… That’s why it’s a “they can’t” overall – damn ‘inter’ bit in internet again. Gah!

Or maybe this isn’t about the porn sites at all – but about seizing control over everything that’s pumped out to us! HAH! You may choose your own favourite conspiracy at this point. (But yeah, quite possibly some elements aren’t mere conspiracy.)

You’ll hear people saying that other countries manage central registers, and why can’t we? You’ll hear people saying that we just need to trust the state a little more – and of course will someone think of the kids? You’ll hear armchair service designers telling you that it really isn’t all that difficult, and politicians saying “well of course we now hand this one to the clever technologists to implement; we know their grate branes will Find a Way…”

We’ll see, won’t we?

But as I say, don’t go thinking this is in any way real policy. It will keep a lid on tabloid outrage, hopefully, perhaps for a bit, just until something more distracting comes along.

Open platforms

Complicated journey

How do you make something freely available so that anyone can use it, but also build sustainable businesses on top of it?

It’s an aspiration that drives Wikipedia, innumerable open web projects and, in recent years, the thrust of releasing UK government – and government-funded/subsidised – data for reuse.

It’s also a complicated balancing act in terms of basic economic theory: prices find their natural level, and if something’s available for free, there’s always going to be a tension in competing with it. If there’s no added value, there’s no sustainable business.

Of all the sectors where data has been opened up, it’s in transport that I think the most visible and tangible advances have been made.

Transport data has many lovely qualities about it – it’s highly structured in time and location; it has extreme real-time relevance; and it affects people.

I don’t doubt that concentrations of heavy metals in the soil affect people, but with the best will in the world, not in the same way – and not in a way that’s likely to affect what time they get up or which route they choose to get to work.

The web, and then the apps, revolutionised the way we consume travel information, but none of this could be possible without the underlying data.

So is the freeing up of data entirely without complications?

First, a small diversion into history.

Long ago when I tinkered with these things for a living, I was much taken with the power of the bottom-up service “Fix My Street” to allow people simply and quickly to report defects in their locality. A quick phone pic, an upload via an app, and the matter was routed to the relevant authority – putting the burden on them to receive, process, and respond – all the while knowing their response (or lack of it) would lie in public view.

My exam question at the time: should such a service be given oxygen through association with government’s “official” channels for doing stuff?

There were some curious arguments thrown at me at this point: “but we spent lots of money on our own sites – people should use them” [NOPE, NOT BUYING THAT]; “it will erode understanding of who actually provides services, and therefore local accountability at the ballot box becomes less clear” [ER, MAYBE A BIT? BUT SERIOUSLY?]; and “if you put information like that into public view, people will use it to find potholes, drive into them, and claim against the council” [*WTYRF?]

And yet, and yet. We are merely fallible humans, and if there’s a buck to be made… Which brings me, in a very roundabout way, to the Delay Repay Sniper.

The train operators – possibly out of a sense of decency and fairness, but more realistically under the thumb of the regulators – now operate money-back schemes for many of their services. If the train runs significantly late, or is cancelled, you can claim from them. My operator, like some others, calls it “Delay Repay”.

The burden is on the unfortunate traveller to work out what went wrong, and to make the claim. Sometimes this is easy (I’ve known train operators bring claim forms through the carriage, but that was pre-web days) but often it’s not. You are too busy trying to rescue your day to log the details of just how long you were delayed. Or things have got so chaotic that all concept of which actual train got cancelled/delayed is lost in a mire of misinformation.

If you’re not a season ticket holder then your claim is further complicated by having to dig out the precise fare you paid, which means finding the ticket that you chewed up in disgust after sitting outside New Cross Gate for 90 minutes.

So it’s very likely that Delay Repay is massively underclaimed in practice.

And, as the theory so rightly predicts, if there’s data, and there’s untapped value to be squeezed from it, there’s a business opportunity.

Some clever folk have built this Delay Repay Sniper (DRS) service to do just this.

They get a feed of data from Network Rail every day. For a very modest monthly subscription they will then crunch it around to make sense of it (in its raw format it’s not easy to read or analyse) and email you every day to tell you which (if any) of your preferred routes had problems. They also offer more elaborate features such as the ability to make automatic claims for delays on a particular route.

This, and indeed much of the DRS service overall, has a particular appeal for the season ticket holder. Their routes, fare and train times are usually quite predictable.

But why wouldn’t the train operators just publish this performance information openly on their own sites?

Hmm. Let me think about that.

You see the problem? Although everyone involved is very clear that making a claim when you aren’t entitled to it is fraud, and this is very bad (which it is) – there are certain difficulties in practice.

You don’t even have to travel on a train to make the claim – because of course you can’t, by definition, if it’s been cancelled. You can’t even rely on a swipe at a ticket barrier to show intent to travel – who would leave the concourse (or even, in the case of my journey from home, my house) to do that if the signs (and apps) are all saying “cancelled”?

The Delay Repay claim form I use also asks me to say how long I was delayed. What does that mean? How much additional time it took me to reroute, bus, cycle and hike to my destination, end-to-end? Or how much the train I’d planned to get was delayed?

They don’t specify – because it’s not in their interests to do so – nor is it a clear concept. So they let the user choose how long they were delayed, in bands from 30 minutes to 120+. (My view on this is simple: if the train is cancelled, it’s always entered as a 120+, even if the next service comes along in 25 minutes. If they run it with a delay, then I use that time. My appointments, decisions and connections depend on trains running. They cancel; their problem. If I’m reading this wrongly, I welcome any official guidance…)

So DRS creates the potential for widespread fraud – enabled by the release of data. Perhaps “creates” is too strong – the potential already exists – but it certainly makes it a lot easier. To put it another way, DRS do show people where the potholes are so they can drive into them, exactly as my gloomy local government contact predicted all those years ago.

The train companies are fighting back, of course. Since DRS set up shop, the Delay Repay form has added a Captcha (to hamper automated applications) – an additional tick required to confirm the journeys were actually real (or really intended, I guess) – and stronger warnings against fraudulent claims. They’ve also changed the way that log-in works so that I have to manually fill out all the fields pretty much every time I use the form – passive aggression in interaction design if ever I saw it.

I’ve also had claims reduced or knocked back for being not as delayed as I’d thought – it’s not really worth fighting over each of these, because of some of the ambiguities of terminology mentioned above.

They hint that they’re using analytics to find the patterns of the “world’s unluckiest commuter” whose train is always the precise one that’s been cancelled. Or even, in extremis, would they scan social media to find those holiday snaps from Ibiza when the claim is for a dreary March morning in Ifield? Ok, maybe that’s going too far, for now…

Warnings based on statistics are one thing, mind you – prosecutions or withdrawal of tickets are an entirely different matter. I’m looking with interest for the first court case; because I am certain it will come. It’s massively in their interests to find someone to hit, and hit them hard. [See update below, 28 April 2017]

There is no doubt that an arms race is underway. DRS emailed me with the latest technical changes at their end to get automated claims working again, for example, in response to the introduction of the Captcha verification.

If one pays for a service, one wants to at least recover the cost of subscribing, so there will always be temptation. And in the mind of the commuter, perhaps the moral issues are more complex. All those missed claims because the information wasn’t at hand? Surely it’s fair to make up a few of them here and there? That time when they dumped me off the train at Purley at midnight, then fast-ran it through my bloody station…

You can see how the arguments stack up. I feel a certain level of sympathy for the operators, of course – they have to pay out for delays, and they will only ever be able to manage, not eliminate, fraud.

There’s also a strong whiff of inequality about all of this – the information-rich get a better deal than those who aren’t aware of what and how to claim. I can see ways to improve that, but they’d all require the operators to do – and spend – more. Probably unlikely to happen, in that case.

So – no great conclusion other than to marvel at what complex moral and societal issues surround even something as simple as historical train information.

I can certainly see that DRS add enough value with their unpacking of the stream of raw data, and their email alerts and other services, to give them a business model.

At least until a competitor arrives to undercut them. Market forces tend to keep running, even if the trains don’t.

You’ll be pleased to know that I wrote this over a succession of heavily delayed train journeys. And yes, I am a DRS subscriber.

*The insertions in this popular phrase are “Yellow” and “Rubbery” in this, my favourite variant of it.

Update: 31 March 2016

As ever, starting discussions in an area like this quickly leads to new and better information. What I learned, thanks to Chris Northwood and others, is that DRS don’t get a pre-packaged delivery of this data every day from the train operators. It wouldn’t make sense, really, if you think about it – why would a train operator do that?

What they’re doing (perhaps they’d like to add a comment?) is drawing on the Network Rail feeds, which are, more or less, made available as open data. I duly signed up just now and had a look. Gosh, it’s raw. Really raw. Hefty chunks of JSON, yours to do with as you wish.

It nicely demonstrates an open data business case. DRS are adding tremendous value by taking it in each day, crunching it into something usable, and sending people the precise parts that are most useful to them. Well done them for spotting the opportunity (whatever the motivations of its users may be) and creating a business on top of the data.

The argument remains open as to whether train operators should do that legwork for their customers – if they really wanted to help them – but it would simply add a cost that they’d have to cover somewhere else. Value is value – whoever adds it. There are no free rides here.

Update: 28 April 2017

And here’s that court case I predicted…

Twitter’s not ageing. You are.

Remember the first day at school? You made all those new friends? You didn’t really have to do anything after that, for 5 or 7 years or whatever. They were just there. Your relationships with your peers changed, you got older, then you all left, and that was largely that in terms of it being “a community”.

When I hear the cry “Twitter’s changed – it’s not what it was!” – this is very much what it reminds me of: a natural lifespan, an inevitable decay.

I used to think, optimistically, of Twitter as “the chatroom of the world”. We’ve all been in forums with boundaries set by topic, or demographic, or real-life membership of something else. Not this one.

For the first time, somebody had been bold enough to try and run a global, real-time, searchable messaging infrastructure (flaky, but it ran), a name-allocating system that mostly managed the disputes (though I’d still LOVE to know how some people really got their id…) and some protections against the worst of human behaviour (let’s call that work in progress, hey?). (Those three things are all you need to do to create one, really. Good luck!)

And what did you do, as a user? You had a big initial phase of making connections; following people. Then you pretty much tailed off – either sticking around a fixed number, or sporadically following back interesting souls who popped up, or occasionally reaching into a new network.

Largely, relationship formation in a social network seems to be characterised by lots of early activity, then not a lot. (If anyone knows of any stats available that map this pattern with actual data, shout. I’ve looked in vain for years now.)

I’ve often thought of the xkcd 10,000. Always be aware of the vast numbers of people who haven’t found the things you have. Their experience today may reflect precisely what yours was many years ago.

And yet because of its scale and uptake, Twitter was different. It was so pervasive in some communities (media, government, tech, comedy…) that there simply weren’t another 10,000 out there in many cases. Very few fresh ingredients to fall into the soup and keep it all tasting nice. Of course it was never going to grow like the investors demanded it did. Dur.

So if you wanted to keep Twitter fresh for you, you needed to work at it. And we didn’t.

Maybe we expected the same energy and adventure you find in week 2 of a relationship to be there in year 7. Uh-uh. Maybe we forgot that we’ve all got older? Maybe the new people who came later felt, inevitably, that they were outsiders – more keen on throwing rocks or picking fights than in anything particularly social. Lots of dynamics at play, and it’s impossible to account for everything in a few lines here.

But when I think of the concept, now, of organising a tweet-up? Or even a #ff? They aged, not because Twitter aged, but because our connections did.

And yet, connections become friendships (or habits); less easily discarded than acquired.

So “the changes” we perceive over 10 years become an inevitability: less of a consequence of “Twitter changing”, and more of us…not.

Happy 10th, you marvellous bastard. There’s no chance of you seeing a 20th.

Biting the bullet

Shall we just do it? Just build it and get this over with?

We have it anyway, don’t we? Just in a distributed and not-very-accountable way. So why not do it properly?

The stuff I wrote yesterday about registers is just a part of a vastly bigger story about information, people, and government.

[tl;dr of that piece: using ‘registers’ – lists of authoritative data – to make government services better has lots of benefits, and raises interesting questions]

It’s a story that’s so big it doesn’t really have a beginning, or an end. How we meet the needs of people, society, democracy, everything – with technology, data, organisations, everything.

So I’ll home straight in on one part. Probably the most sensitive registry of all would be a register of citizens. Of people. Of the entitled-to-vote. Of permanent residents. Yes, tricky, hey? Let’s just call it people.

The Promised Land of a canonical list of people sat (sits?) behind the for-the-moment-abandoned (I expect this to change/is changing!) concept of a national identity card.

It sits behind lots of other things too – either as the manifestation of the ultimate authoritarian state, or as the lubricant for a trillion safer, more secure, more efficient digital transactions. Depends on who you ask, what they’re trying to sell, and the weight they give to various arguments of logic, experience, ideology and emotion.

It’s hugely political, obviously. The argument that it is “poor civic hygiene” is usually high on the list of “why nots”. A future government may be in a position to do all sorts of terrible things to its people if it can track and target information very precisely at individual level, or even make people appear and disappear at will, through manipulating a central megadatabase.

(But Estonia!)

And that’s to say it’s even possible to procure, build and operate such a beast. The track record at this scale isn’t great.

(But Sweden!)

It’s so sensitive that registers of personal or sensitive data have been explicitly excluded from the current scope. Instead, Verify is doing sterling work to do digital identity checking through the use of third parties – essentially using what outside organisations know about people as a proxy for government’s knowledge, then accepting that trust as being good enough for subsequent interactions with government. A very neat, and widely welcomed, sidestep around the problems and concerns that bedevil a central people register. But it has limitations – you can use it to check facts about people, but you can’t write information back to it, or assemble a master list of people you could then sign up for electronic voting (or any other new thing you dreamed up).

(But Singapore!)

So none of this means that the clamour for a central people register has gone away. It never will. It’s what James Randi once described as an “unsinkable rubber duck.” An idea that no matter how many times you unpack it, debunk it, resolve it…will always bob back to the surface. It’s so tempting. The perfect answer for those who love hierarchy and are convinced that hard-edged systems can save the world. (But Estonia!)

Yes, yes, ok, Estonia etc. – there needs to be a better response available to the “But Estonians”. Your vulnerable minister and officials will be regularly swept over there to marvel at how all this digital identity and database stuff just…works. Nobody dies because of it, the tanks don’t roll in, there isn’t a monitoring screen in every house. I’ve asked a lot of people who should know about this stuff what the solid counter should be to the But Estonians. Curiously, I haven’t found one yet. Have you?

And then, I think – hang on, is any of this resistance actually meaningful?

We may not have a single people register, but we have lots of things that are a lot like it. You may be surprised by some of the questions you get asked when you use Verify. How did they know that? They know lots, really, those identity providers. That’s why they’re identity providers. They’ve spent years buying and integrating things about you. It helps commerce operate. But it’s private, opaque, unaccountable. Sure, it’s not government, but it’s still a thing.

Or what about the Police National Computer? Who knows how they refer to you? But they know things about you. Try getting stopped in the street by the cops and not showing any “ID” (don’t start me off on that term…but full disclosure: I have done this, just to see what happened.) You’ll find some of their questions to you, and their radio checking, pretty interesting too.

So whether it’s done through a single unique identifier (ooh – somebody said “just use the National Insurance Number!” DRINK!) or through the patchwork of private and occult registers, we live in a database state anyway. The infrastructure, and the surveillance powers, are already such that pretty much any bad consequence could already happen (is happening?). Data sharing work is developing apace. If one of the main concerns about a centralised people register is its vulnerability to attack, then those concerns apply to the private registers too, no? Ok, but the prize is bigger, but still… The police manage to do it. Experian manage to do it.

Is all the protestation just for show, really – we attack the thing we’ll be able to see because we can’t attack the things we can’t?

My personal view on this (as a non-practising civilian with a lifelong interest in civic data) is that the central register has some benefits. But enormous risks. And that the risks scale faster than the benefits. You aggregate that much in one place and the consequences of error, or breach, or yes, totalitarianism, are unthinkable. So it’s a bad thing.

My friends Achilles and Tortoise teased out some of these issues for me a while ago.

But I’m not convinced I’m right. That would require a level of evidence I don’t have, or a level of ideology I find distasteful.

Help me out here – what would it really take to sink, or float, that rubber duck?

At least for a bit?

Registering a concern

I’ve got out of the habit of blogging. The problem with that is that posts grow, anyway, inside my head. If they aren’t tended, they tangle furiously, and before you know it you’ve got Sleeping Beauty’s garden to hack through before you can get anywhere.

Then a wise man said: “start small. iterate.” So I will.

A relatively new part of the UK government’s strategy for making public services better is the concept of ‘registers’. They’re explained very well here – in an line, they are “authoritative lists of information you can trust”. Nothing new in that idea, really. Anyone who’s ever built any kind of database has worked with that concept.

Where things get fruity, as always, is when you introduce complicating factors like scale, stability, organisation and good old human nature. (This isn’t an exhaustive list.)

Scale? A register with 100 records behaves differently to one with 64.1 million. You won’t be able to maintain them both using the same tools and techniques.

Stability? The register of countries recognised by the UK government (the launch example shown at Sprint 16) changes quite infrequently. A register of every business created in the UK changes almost by the minute. Different dynamics, different issues. (Something like a register of holders of a particular licence that it’s quite hard to qualify for will lie somewhere between those two examples.)

Organisation? As soon as information supports the business of several organisations, other things start happening. Trust, for one thing. If you’re accountable for sending out information to a particular group of people, then at least if it’s your own database of recipients you’re using you have some sense of end-to-end control over what happens. Get that data from somewhere else, and accountability is separated, possibly diluted. Whose cock, as they said in my early days in Whitehall – when things were somewhat less enlightened and a lot less diverse than now – is on the block?

And that brings us to the human nature bit – if those other things get complicated, then the human instinct is to self-preserve. To silo (please, please forgive me). To build walled gardens. To duplicate, fudge, kludge and, of course, waste… From homespun spreadsheets here and there to mighty Oracle instances, the idea of canonical information is readily sacrificed on the altar of expediency, or just plain old survival. (Obligatory Upton Sinclair reference here.)

So – registers. Top idea. They’ll be definitive. They’ll be owned. They’ll feed and support other systems.

But will they, y’know, work?

I’m not sure. I used to be a young turk information systems type. Now I’m a greybeard, retired from the fray to the far less capricious world of pixels. I’ve seen centralisation, federation and linking of data. The rise, and the fall, of many programmes. The Citizen Account; the Single Business Register; the Government General Practitioner; health resources; gazetteers; land and property; military warehousing systems… you name it, I’ve feebly tinkered with bits of it.

I can’t not like the starting point set out in Paul’s post: begin with the simple principles – and with manageable scale, clarity of ownership, low volatility and all that. Establish how they’ll work, then do more. Start small. Iterate.

So what’s nagging me? I guess having seen so many “the one true…” projects founder, I’ve got a few scars and prejudices. Look hard at almost any data set, and it’s less canonical, less binary, than you might think. That list of countries? So should Palestine be on there then? Perhaps it should, for some purposes – but not for others? I don’t know – I’ll leave that to the foreign policy people to thrash out. But you get my point. Another of my inspirations in this area wrote that “Digital is political“. True indeed, and data is also very often political.

Then there’s this question of volatility. There are inevitable limits on how well you can keep up with fast-changing data. For some purposes it may be sufficient to know which companies held a particular licence as of the first of every month. Other requirements may need that status to be verifiable on a minute-by-minute basis. There’s some serious analysis to be done to make sure that an authoritative register can meet all those needs. Because if it can’t, the hydra’s heads will start to sprout…

And on that “feeding other systems” point – the risk of one hell of a dependency culture springing up. Yeah, er, we’re down today because one of the tables in our system is fed from a, yeah, and that’s down, and, er, well the API’s been flaky for a while and we’re not exactly sure whose problem it is to fix coz it’s on the boundary… And so it goes. To say nothing of the points of failure and vulnerability should someone want to pop a nasty bit of grit into the gears of the government machine… So there’s that.

The trust thing will take some real thrashing out too: you’d think that concepts such as company registration, or the existence or non-existence of a school or hospital would be nice, uncontroversial matters, with clear alignment to one or other government structure. You might think that; I couldn’t possibly fail to quote Francis Urquhart.

Lastly – the old curse of change programmes – it’s really easy to do new, good stuff. It’s very hard to stop doing old, bad stuff. Will teeth be required?

To conclude: I guess I’m cautiously supportive of the concept. I’d really like to see some more development of the vision for what future registers may be created.

I know – it’s a fine line between writing cheques you can’t cash, and setting out some aspirations. But would it be going too far to draw up a wish-list of just some of the more appealing candidates? An authoritative gazetteer, perhaps? The registry of land and property ownership? Companies? Charities? Patents? The Electoral Roll?

I don’t really mind which – juicy ones like those, offering enormous scope and value – or just more niche stuff: good parts to build better engines. But more, please. A little boldness goes a long way.

Not people though. Not citizens/subjects/taxpayers/voters. You and me. Uh-uh. That one’s definitely out of scope.

Or is it? (I’ll leave further exploration of that one for the next post…)

They did actually say that thing about cocks in my first meeting in my first central government job. I had to include it as a piece of social history. Sorry.

UPDATE: 18 March

It didn’t escape me yesterday that the Budget contained something very relevant to all this: £5m to build a new, open address register. It was one of the triggers for writing this post.

I won’t say much, as it isn’t yet clear who’s going to do this and what they’re going to do. Though I think I’m safe in saying that this time it won’t just get shoved through the door of 123 Buckingham Palace Rd in a brown envelope, to be blown on “stakeholder engagement meetings” before any actual work gets done.

Charlotte Jee has written a good summary here; much of the background is nicely encapsulated here.

I shall merely add:

– there is some devil in the detail of the differences between a gazetteer, a database of delivery points/postcodes, and underpinning geospatial information; their contributors and users are not homogenous, so it’s always good to be clear exactly what’s on the table here (which I’m not yet, because no detail has been published)

– IF this is to be the new, authoritative register, then what’s its relationship to the other one – the one that got flogged off? Both could be maintained in parallel, but that would clearly be absurd. One would have to be the master to the other. Which? How? Who makes that happen? Who pays? And so on.

That’s the big plus, and the big minus, of “authority”.

And even in 1746 Goldoni knew that this was a splendid way to create a farce.

What you probably won’t get to vote for

It took me a while to work out what was bothering me about the election campaign. Of course it’s also what’s been bothering me every day, in relation to politics. The issues that we’re told will decide the election? The policy building blocks from which any government must inevitably build its castle? The NHS, the deficit, the tax-dodging corporates?

They’re not the issues.

They’re shadows on the cave wall – implications and consequences of the bigger stuff, but they are not the issues.

So what is this bigger stuff?

Here’s a small, rough, incomplete (and not that neatly bounded) list of what I reckon some of it involves. Your task, dear reader, is to try and fathom why your choice tomorrow will have very little tangible relationship to any of it.

It might well be because they’re issues that are genuinely insoluble, but in that case why are we all indulging the dear politicians in the illusion that levers exist that can be pulled on from Whitehall, or anywhere else? Why indeed? What if we all understood the reality of the situation, and were able to be honest about the nature of the choices required?

Here goes:

1. An older, healthier population. Better healthcare = longer lives = changing morbidity profiles. There’s positive, obvious feedback in this system – the more able we are to prolong life, the more prolonged lives become candidates for intervention. Whether or not we use an NHS, or any other mechanisms, to address that are subservient to the deeper issue: we’re changing the nature of our population and our diseases. Do we understand where that’s going to lead, and are we prepared to build the structures that are required in response? If we’re not, then we have to make some very tough choices about families, care and ultimately people. Hard-edged ones that no squabbling about here-and-there £bns of NHS funding can mask.

2. The nature of international businesses. And perhaps beyond that, to the nature of business structures themselves. Getting all righteous about Amazon, Starbucks and all the other border-tricksters, from the non-doms to the Jersey-wraiths, is all well and good as an academic pursuit, or as material for blustery speeches. But it’s not just chance, or bad drafting, that mean taxation is so hard to pin on those who really don’t want to pay up. It’s down to those structural issues – how we regulate company formation, location, transfer pricing etc etc… And to what extent can those ever realistically be determined or limited?

3. Why we educate? If it’s just about matching skills to needs, why are we so haphazard at it, and why the huge apparent lags in adapting as those needs change? Is it just in the nature of changing a very large system? Or is it all just so much habit? Or… dur dur durrrr… do we stick kids in schools the way we do, and do things to them like we do, for other, darker, more manipulative social reasons? We ok with that?

4. The power of information. There’s less friction than we ever imagined possible in the way information flows. That changes markets, industries, liberty… Everything. And yet the nature of government – and particularly the attitudes of virtually all our politicians – are geared to denying, belittling and ignoring the reality of all that change…and all that potential. (Singapore’s Prime Minister Lee writes some pretty impressive code, y’know…)

5. The indexation of us. Ok, this is a bit of a favourite hobby horse of mine, as regular readers will know, and it’s very much bound up with no.4. But what’s really going to happen in the longer term to the way the state recognises us as individuals? And by recognise, I mean identify. And by identify I mean index – classifying, coding, linking, databasing us. That entropy flow that means the pressure to link us up and track us down will always exceed any force in the opposite direction. (Don’t know about you, but it keeps me awake at night.)

6. Selfishness & inequality. Bluntly: a lot of this country is innately, has been educated to be, or is perversely rewarded for being selfish, hateful, sexist – do I need to spell it out? Is that us? Is that who we really are? Are we ok with ‘freedoms’ that leave us free to hate, to distort, to manipulate? Maybe we are. But what if we were able to even imagine being better than that?

7. The tension between executive and judiciary. We think fondly of a British concept of fairness. It’s that fairness that means we build some of our rules (like tax thresholds) with softer edges. We don’t like hard edges; they hurt when you trip over them. So systems (like tax) get really complicated as we taper and titrate the rules. Whatever our government may act (or want) to do, has the counterbalance of judicial review. So that means there’s an inevitable (and perhaps highly desirable) tempering to political will. Do we really understand this dynamic, and are we honest enough to admit the caps and constraints it brings?

I could go on with a few more…the ground rules of government intervention in markets; a lead actor on the world stage or a tired extra; clutching past glory or refreshed for the future…but I hope the point’s coming over.

This is a half-hour sketch, not a manifesto. If nothing else, push a little harder when you see a “policy” being touted, to see what the thing is underneath that’s not really being acknowledged. There’ll usually be something there.

Vote wisely, people.

“No excuses”: in memory of Adam Bojelian

This week, I lost a force for good in my life. I want to say a few words about him, and why that was.

It’s possibly unlikely that the untimely death of a 15-year-old stranger would have great significance for a 47-year-old man who’d never met him.

But not if that young man was Adam Bojelian. He was one of the most exceptional people I’ve ever come across. He had a multitude of health issues throughout his life. Most notably, he had almost no control of his body, other than the ability to blink. But by blinking, he could communicate. And in communicating, he had a window to the world.

This level of incapacity would be the end of anything approaching a life worth living for most people, I suspect. Not for Adam, though.

He had a gift for poetry. For taking pleasure in the smallest of things – the feel of fresh air on his face in the park. For his dog, Charlie. For finding small ways to entertain and involve those around him – in hospital, and in his online world through Twitter. He organised quizzes and football sweepstakes, and he was indefatigable in his work for children facing health challenges and for those who support them. Here’s a link to the fundraising cause he asked to be his legacy.

Most of all, he brought attention to a really important issue: the need to listen directly to the voices of children in the healthcare system – to put their needs and feelings first. In this, he managed to get a great deal of attention, and involvement, from some seriously senior people in the NHS.

He had another gift, too. The thing he gave me.

Adam was – and always will be – my “no excuses” person. He never grumbled about how his life had turned out, or showed any sense of self-pity. He’d say when he was feeling low, or poorly, and get on with it. With some of the heaviest limitations it’s possible to imagine, he made no excuses, but found things that he could achieve, and achieved them.

If Adam could do all that…then what excuse are you going to use for not making something of significance with your life, my friend?

Online, nobody knows if you’re blinking.

Even if it’s because of a tear, as I am now, writing this.

The force has gone but the effect will remain. Thank you, Adam. You were, and are, brilliant.

RIP Brave Adam

MOT status check: a five minute business case

I have just written this business case in five minutes. Go on, knock it over! [Spoiler: the update lower down is where it gets knocked over, in some style!]


Millions of people have no idea when their MOT expires. If they’re lucky, their local garage sends them a reminder based on the last test date. But mostly they have to rely on a piece of paper in a filing drawer at home.

So they let them lapse, leading to compliance problems affecting insurance, enforcement, automated renewal of vehicle licensing etc. etc.


Fortunately there’s an online service to check MOT status. You’d think there would be – it’s exactly the sort of simple, non-personal, bounded data storage that computers are really good at looking up.


To use it you need TWO pieces of information: vehicle reg no; and either the old MOT certificate (or MOT failure note) reference, or a reference on the V5C (log book).

Thing is, pretty much everyone knows their reg no. but guess where people keep the V5C? Generally in the same drawer, right next to the MOT certificate. So what is the point of the online service if it forces you to go and run your fingers over the piece of paper that contains the answer you’re looking for? And of course when you think of these things, sod’s law says you are NEVER at home, with your filing drawer in reach. It’s a bloody nonsense.


Simple modification to the business logic: take away that requirement for a second piece of information.

At the front end, just ask the user for the reg no.

Hidden from view in the back end, tell the system that the correct V5C/cert reference has been entered. Dig through all that ancient legacy code(?) and set the “HAS_V5C_BEEN_PROVIDED” flag to “TRUE”.


Massive improvement in compliance. And a really huge new online transaction success story to shout about AT PRETTY MUCH ZERO COST.

Policy implications:

(And this must be where the problems lie in this complete no-brainer…) It will mean an acceptance that your MOT record is essentially public. Anyone can look it up if they know your reg no. Short of some really outlandish edge cases where the Sun might write some crappy story about Tulisa’s car having an out-of-date MOT, I really can’t see why this shouldn’t be a matter of public record. After all, the presence (at least until this year) of a visible tax disc has been a publicly viewable example of a car’s compliance with the law, and nobody’s ever got upset about that.

Even in the wildest imaginings of what risks might arise, they will never outweigh the benefit of making an MOT status check online service that is actually USABLE.

Rant over. Go on then – what did I miss, and WHY HASN’T THIS J F BEEN D?

(I wrote about this over three years ago, and have mentioned it several times since to senior officials in the world of government motoring administration. No answer ever came back. Should I have been more persistent, or just punchier?)


So, I tweet the head of DVLA, and I get properly ZINGED. Because it turns out that it did get JFD. (Just f—ing done)

Try this, the new Vehicle Enquiry Service (beta).

I just didn’t f—ing know about it. It’s not obvious in search – which is still dominated by that page linked to above, asking for that elusive second piece of info.

(It also asks for Make – which may possibly thwart a small percentage, and could be argued as being redundant, but I’m hopeful has been shown in testing to be a better way to handle mistyped registration marks than simply offering the (wrong) vehicle details back and hoping the user spots that their Ford is now a VW…)

Without doubt it is a GREAT service. An EXCELLENT service. It does exactly what I hoped. It does more, actually. It shows vehicle tax expiry too – which of course becomes essential as the tax disc is now being phased out.

Quite rightly, given the service is in beta, there’s still work to do on fixing that search journey, and I’m sure before very long that other page – https://www.gov.uk/check-mot-status – will be tweaked to reflect the new functionality.

I’ve got humble pie running down my face, and I’m ok with that. Shall we just reflect how far things have come that a well-intended (but clearly underinformed) blog post can pop-up – get a useful response directly from an agency CEO within a couple of hours, with not a hint of spin, snark or press officer flannel – and lead to a better informed me, and hopefully you, dear reader?

(Off to get a bigger spoon to eat that pie with)

No exit?

It’s the morning after the poll before.

I’ve just read an interesting James Ball piece on why the lack of exit polling was a Bad Thing. At first I thought it was going to be a journalist’s lament that journalists were being deprived of bones to pick over in that difficult bit between being allowed to talk about it and having any actual results to talk about. Oh dear. How awful.

But setting my wholly unfair cynicism aside, the real point of James’ piece was that this lack of information in some way diminishes our understanding of, and competence in, democracy.

[Personally, I don’t much like the practice, if I’m honest. As a voter, I’ve done my thing, and on the way out I really don’t see how it’s anyone’s business who I am or what I did. That’s just me.]

As James put it:

…we won’t know whether it’s because 16- to 17-year-olds, given the vote for the first time, turned out en masse, or whether it’s because of a last-minute change of heart among pensioners (who currently lean towards no).

This detail and data is what lets academics, journalists and politicians alike work out what really happened and how voters act.

And at this point, one of my favourite questions pops up. “Whose needs are being met here?” If the exit polling were restricted to basic questions on participation (e.g. age profiling) that might be useful to help find the missing turnout. But anything on actual preference will only have one effect – to allow the focusing of “messages” on defined demographic slices. To turn the overall outcome by taking close aim at particular levers to shift particular sub-groups. And is that ok? I’m doing a big hmmm in my head.

Surely the whole point is that messages – or better still, policies – have to read across an entire electorate, and be measured in the aggregate. Focus-group all you like in shaping those policies, but I just can’t see how exit polling serves wider democratic interest anywhere near as much as it serves party interests. If that’s the case, let the parties pay for them, and the media rake over them – but let’s be clear about who really benefits.

I suppose to some extent opinion-to-exit comparisons also provide a measure of public mendacity, but is that essential to make democracy itself more effective?

Vote for change