toward a sensible AI-skepticism

future proofing your takes

I had been thinking about this for a while, but obviously it became significantly more topical this week. I’m not sure whether I would describe myself as an AI sceptic[1]; I have a horrible feeling that my actual position is “person who reacts in opposition to the last thing he heard”. And so I find myself bouncing between takes, skidding away from “it’s just autocomplete on your phone” and running slap bang into “we’re maybe three months away from making God”.

One aim of my original manifesto was to be a bit more low-energy and create the equivalent of ambient music for take-merchants, so I’ve been wondering – what are the building blocks of a version of AI-skepticism which minimises the risks of looking really stupid in a few years’ time, without giving more ground than necessary? Here’s my bullet point first guesses and “notes to self”:

• Don’t make scepticism depend on assumptions about resource input. This is partly because a lot of the figures going round on the internet about gallons of water and so on are speculative to the point of bollocks, and it’s very difficult to check them. It’s also because, as DeepSeek has shown us this week, assumptions of this kind are intrinsically fragile to technological progress. More or less the whole history of AI research has been a tug of war between “just do the calculations more cleverly!” and “your so-called clever assumptions are in a blind alley, just get more processing power”, and given the point on that pendulum where we are now, it seems reckless to me not to anticipate the next swing. And it’s rarely a good idea to make very strong assumptions about how much resources ought to be used by one industry rather than another.

• When making assumptions about the limitations of AI in general, check that there isn’t a key premise that’s only really true of ChatGPT. Although I would still argue that the LLMs which have achieved breakthrough success so far are, at some level, “Unoriginality Machines”, this isn’t necessarily always true. Remember, for example, that AlphaGo not only managed to defeat Go masters, but did so by playing in a style that seemed utterly alien to them. What seemed to have happened there is that by repeatedly playing games against itself, it explored much more of the space of possible moves, and it was the human beings that had got into a locally optimal rut. I am going to qualify this argument quite a lot below, but it’s worth remembering that different architectures can have very different characteristics.

• Don’t go too hard on “hallucinations”. Looking at the way that neural networks work in very general mathematical terms[2], I have convinced myself that it is not possible to completely eliminate hallucinations, because it is a fitting problem which will always have residual error. However, for similar reason, I think it will always be possible to reduce the hallucinations by improving the fit and expanding the training data set. And consequently, I wouldn’t stake anything important on it being prohibitive in terms of cost to get an AI which makes fewer errors than a human being on most problems of interest.

And on the other hand:

• I think it’s definitely sensible to be skeptical of AGI. I really can’t see anything beyond sheer optimism behind industry claims that it’s imminent and doesn’t require any more technological or conceptual breakthroughs, so you can be as sceptical as you like of it, and then save face in the future by changing your mind if and when those breakthroughs happen. This also implies, to my mind, that a good use of sceptical effort is in identifying problems where solving it looks like it’s of the AGI order of difficulty. (I made this claim about analysis of planning consultations last week).

• There’s also a very important role for scepticism that AI is in some way or other outside the price mechanism or the normal priorities of political economy. This is particularly obvious when someone suggests we should forget about some obviously crucial issue because the AGI will solve it for us, but it’s also in my view perfectly sensible to be sceptical about future economic benefits, whether they will in fact justify current venture capital investments and whether projects which aren’t economically viable without subsidies and exemptions from environmental or social regulation should be made so because they’re AI.

My own scepticism, to the extent that it isn’t just rooted in being an awkward bastard, as noted about, is based in what I’d called “No Silver Bulletism”, after the relevant chapter of Fred Brooks’ “Mythical Man-Month” (published fifty years ago this year). I think that AI, like every other information technology, will end up creating complexity as well as processing it, that the robots will get in each other’s way just like we do, and that consequently we are going to systematically overestimate the benefits of the technology during the initial phase. It’s the Dinorwig Spoil Heap Problem – when the returns seem huge and the inconveniences are minor, you don’t notice that you are making structural and architectural decisions which will come back to bite you massively at a future date, when enough of the inconveniences have piled up that they’re not minor any more.

And that’s enough from me for a while on this subject – apart from a short joke post on Friday I’m going to try to find other things to write about apart from AI for a few months.

---

[1] I have absolutely no intention of ever spelling this consistently; don’t write in about “license”, “premis” or “organisation” either

[2] I’ve been, I think, guilty of sloppy use of the phrase “vector average”, to describe the process of selecting an answer from the space of training data. I am probably going to continue using the phrase out of laziness (and getting occasionally picked up for doing so), but it’s not right. Pedantically, I should have been saying “tensor” rather than “vector”, but this is a minor sin (particularly since a tensor is a generalised vector, and in a lot of applications the tensor is in fact just processed as a long string). More importantly, I should have been saying “regression” instead of “average”. The model is calculating a weighted sum to optimise its loss function, finding the right direction and orientation in the vector space of its dataset. It is a problem of fitting a high-dimensional hyperplane through a high dimensional vector space, but there are connotations to the word “average” which I don’t intend; it’s not very like an arithmetic mean. Of course, these things are fractal; this qualification of “vector average” has introduced several more errors and oversimplifications for people to have a go at, which is why I’ll probably keep using the original phrase.