the legend of john henry's cerebellum

yonder lies a vibe codin' man

Before heading off on holiday, I went to the annual conference of the Cybernetics Society, where I was lucky enough to catch up with Richard O’Rourke, one of the people who I think is doing cybernetics the right way these days. There might be a few posts in this series, because the things he’s been working on have very much clarified a lot of important things for me.

Particularly, Richard’s most recent papers have dragged out of the archives this wonderful piece by Ross Ashby, which seems to anticipate lots of the problems of LLMs and strategies for their solution, from the perspective of 1956, and from the perspective of someone who had built a “homeostat” which was miraculously able to stabilise a fluctuating voltage. The crucial insight that Ashby had – which the current AI industry is, to be fair, rediscovering extremely quickly – is that in his view, the key feature of intelligence is discrimination, being able to spot good from bad.

If you have a very efficient good-from-bad-spotting algorithm, then you can do a lot; particularly, you can make use of the “regulation by veto” method to use any external source of variety as a potential solution generating machine. And in this way, Ashby makes the case (in 1956!) that it’s obviously possible for human beings to make a machine that can solve problems that its designer couldn’t, as long as they’re able to recognise a solution when they see one. The problem is not really one of devising an algorithm to solve problems, but one that explores the space of solutions in an efficient way when it presents them to the evaluator.

Anyway, much more to come on this, but for the meantime, I think one of the consequences of Ashby’s argument, as set out by Richard, is that everything depends on how efficient your good-from-bad spotter is. And that this could be profoundly optimistic for the human race (or, I guess, pessimistic for that part of it which was hoping for a life of exorbitant riches gained from replacing human beings by robots).

The human brain is a really, really powerful wrongness detection engine, optimised by millennia of evolution. As far as we know, there’s a big general purpose neural network, with a bunch of specialised modules for detecting things like symmetry and cheating. And it’s contained in a surprisingly energy-efficient lump of biochemistry, mostly in your head but with help from a lot of nerve tissue elsewhere (“gut feel” is apparently a real thing).

We don’t know all that much about how you would write down what the human brain does as an algorithm, but it does look very much as if, considered as an algorithm, it’s significantly better than anything currently runnable on silicon. And when I say “significantly better”, I mean that in the sense used for comparing algorithms, in which the difference between a really good and really bad algo can be “half an hour versus the heat death of the universe”.

I think this matters, because as the number of variables and the size of the search space increases, the advantage of having a better algorithm tends to grow quickly. Since the starting point is one in which we are very close to maxed-out on our ability to fill that gap by pushing more electric power through expensive silicon, even for problems like writing computer code and drawing pictures of Taylor Swift in the style of Studio Ghibli, my guess is that we’re not far from the point at which people realise that it’s going to take another technological and mathematical leap at least as fundamental as the invention of neural networks themselves before they can start doing some of the things claimed for them.

So, I’m twisting the dial on “sensible skepticism” a little further in the direction of skeptical.There are going to be lots of really important problem domains where current AI technology can hugely improve its performance.But the ones I’m personally interested in – the solving of management problems in large organisations – are, I’m now prepared to bet, likely to maintain their intractability.