I had a useful argument with a mate recently about the controversy over Palantir signing a contract with the National Health Service to provide AI and data visibility across all of its very many incompatible systems. The substance of the debate wasn’t really about concerns over spying or anything; it was more like me saying “the NHS is so huge it should hardly be outsourcing anything at all” and him saying something like “that’s mad, come on, even NASA doesn’t roll its own SQL databases”. It has been worrying at my mind for a while, and so now I am going to bother you lot with it.
Of course, the specific company Palantir raises a bunch of issues, but I don’t think you need to go down that rabbit hole. Because in my view, the important thing is not the actual ethics and behaviour, but the risk that’s being taken. Even if Palantir were the most upstanding company in the world, with a CEO who really believed in “don’t be evil”, it would be problematic for the NHS to be getting into bed with them. And the reason for that is that Palantir’s business model is based on a very strong version of “vendor lock-in”, where they have many more “forward-deployed engineers” than the usual IT consultancy, and where much more of the understanding and process knowledge is kept on their side of the organisational boundary than on the client’s side.
Straight away, this is ringing alarm bells for me, because it’s breaking almost all of Mariana Mazzucato and Rosie Collington’s sensible principles for governance of public sector consultancy contracts – transparent pricing, a clear end date, knowledge transfer as part of the specification. But even more so, when you’ve got vendor lock-in on the core management information functions of a medical system serving millions of people, then it hardly matters how much you trust the provider today. The problem is that you’re stuck with them. If Palantir were to transform from my best case scenario described above, to the worst version of the worst conspiracy theory about them, then there’s nothing that the NHS could do about it.
This is actually something I’ve been thinking about for a lot of my life in the context of IT consulting in general. (And specifically, for most of that period, about SAP systems). Running a database isn’t a core competency for most companies. But managing the freaking company is a core competence, or at least it ought to be. And one of the big messages from management cybernetics is that the distinction between these two things is not as clear as you’d think. A company is an information processing system. The distinction between code and data is very context dependent, and so is that between the nuts and bolts of storing and retrieving bits and bytes, and the decisions made about what bits and bytes to store and what interpretation to put on them when they’re retrieved.
And so … there might be IT things which the NHS does which don’t have the health service equivalent of “Brompton-ness”, but fewer than you might think, and anything which Palantir might be interested in seems to me to be very much on the other side of the line. In general, although big organisations are aware of the risks of vendor lock-in, my guess would be that if the question ever becomes worthy of discussion, then it might be time to worry that you’ve already gone too far in putting a function that’s crucial to your company’s survival on the other side of a corporate boundary.
This is, of course, not specifically an IT problem. There are plenty of companies which literally outsource their strategic thinking to consultancies, as Sonny Boy Williamson said “don’t start me talking”. But it feels like it’s easier to do this without it being obvious when you’re bringing The Computer into it.
As a long-time technology consultant, I can tell you that the key to using consultants or a vendor (with its consultants to implement their solution for you) is to make use of them to build enough of an in-house understanding so you can maintain and evolve the capability yourself. Using an off-the-shelf database rather than building your own is viable if you have (and can easily replace) in-house staff who know how to manage the database.
Unfortunately, many buyers find tech scary, confusing, and risky, and would prefer to close their eyes and let someone do it all for them. They'll have a consultant come in to set up their off-the-shelf database for them, and the next time they need anything added or fixed, they're at a loss because the next consultant won't necessarily understand the way the first consultant went about it.
So we had the "outsource your IT" stage, where vendors convinced buyers ("tech is not your core business!") that they would magically make the yucky tech disappear by managing it all for them. Then they were stuck with the vendors, having to pay through the nose for every little change, assuming the changes they wanted were even possible.
Then clients saw new businesses that understood tech were using it to take their business. So vendors sold them on "digital transformation", which was installing an off-the-shelf solution - a core banking system to can catch up with the fintechs, an ecommerce platform so Amazon won't eat your lunch. But it turns out those solutions are high-maintenance, confusing configured, inflexible, and need expertise to use well.
Now AI has the potential to make it easier to build, assemble, and/or customize in-house tech solutions. But buyers who are excited because they, once again, see it as a way to avoid having to truly understand tech, will not end up any better off.
It's all made a bit worse because AI is being sold as something that replaces the people who understand tech ("at last!!"). But I can vouch from first-hand experience that you really, really need to put a lot of effort and expertise into using AI to build systems. Using AI for building software is like putting a chainsaw on wheels. It can do a lot of work very quickly. But if you don't have someone knowledgeable guiding it, you might not like the results.
I think you're right that what is essentially "managing patient records" should absolutely be a primary competence of the NHS and they should have the technical capability to build this in-house, but also that your mate's right that there's no way the NHS will be able to acquire/maintain the expertise needed to build this in-house, but *also* that this project is highly likely to fail whoever tries to implement it it so it's probably a low-impact concern in the greater scheme of things. The NHS is a very big, very *viable* system, and she's generally not for turning.