This is, in some ways, a restatement of a post from a couple of weeks ago, but in a slightly different (a bit more philosophical) way. Basically, the question is that of whether we should stop talking about “artificial intelligence” and return to Ross Ashby’s original terminology of “intelligence amplification”.
In some machine learning contexts, people talk about “human in the loop” as a strategy for doing things. It’s a phrase that’s been adopted by designers of agentic systems, as one potential way to design a workflow.
But it’s not just one way among others! In most applications, there is no other way to do things. There is always a human in the loop, because at some point, the agentic workflow has to come to an end and someone has to decide whether it has succeeded or failed. That someone is a human, and the process of either accepting the agentic output or making adjustments and starting again – that’s the loop.
POTENTIAL BIG CAVEAT: If the workflow was meant to be proving a mathematical theorem, this is more debatable – if I was quibbling, I would say that there’s still a human being potentially needed to verify that the output has been correctly parsed and the problem that’s solved is the one that was posed, but for something which is amenable to formal proof methods, I can see how you might design a workflow which terminated without human approval in any but a trivial sense. I think this is more of a demonstration of how unusual mathematical theorem proving is, though, and of the likelihood that progress in theorem proving, however impressive, doesn’t necessarily imply the possibility of similar results in other fields.
With that caveat out of the way, I think this is hard to argue against – there is always a human in the loop, it’s just a question of where the human being is placed, and how many times the human decision maker is called upon per round of feedback. Which is a design consideration; the system as a whole needs to be designed around this step, to make the most efficient use of the human’s time and energy.
In other words, Ashby’s principle of the intelligence amplifier would suggest that the purpose of the things that we refer to as AI is to do exactly this; to organise the available data in such a way as to make the most efficient use of the time and energy of the human decision maker, by presenting him or her with candidate solutions that have the highest achievable probability of being accepted. If you keep this objective in focus, it makes it a lot easier to think about AI problems, without getting yourself tied up in jargon.
It also, I think, illustrates that LLMs are a social technology; they’re a continuation of past techniques for economising on the scarce resource of “decision maker time and capacity”. They are a way of organising the data so as to present the most likely and helpful solutions. But that’s also what data is! “Data”, in modern context, means information about the world, collected and tabulated in such a way as to make it maximally likely to be useful for some purpose.
In Wittgenstein’s much-quoted phrase, “explanations come to an end somewhere”.In any workflow, no matter how it’s organised, someone has to, at some point, say “that’s it” and decide whether there’s a success or failure.There are only two ways of doing this; either an individual decision maker or some consensus of a group.(It’s not obvious to me that these are really two distinct ways).I have written a lot about the way in which accountability and individual decision making can get confused and concealed in organisations by scale and industrialisation, but I don’t think it can be eliminated entirely.So that’s the fixed point that the rest of the system needs to be designed around.
This is very clarifying, especially of AI as just an alternative data-processing system. Just like a bureaucracy.
Part of my work is prioritising infrastructure for planning and execution from a large portfolio of potential projects. The method is, essentially,
1) I have an MCDA model that takes in a bunch of quantitative data about our portfolio and some judgement calls.
2) I discuss the output with my boss and we produce a rank-ordered list which may or may not match the model output.
3) He and I discuss that list with the Executive Director who decides what the final list would be.
Now from the ED’s perspective, a sufficiently capable AI could be doing either or both of 1 and 2 for all it matters. He is the fixed point. In these terms, AI is quite prosaic: a Chinese Room puts a list in front of the decisionmaker, and it may have in it two guys or a computer or both. They’re just different ways to put a decision in front of a person.
In software development, when we talk about "human in the loop", we mean having a human as a gate *before* the system does something potentially harmful or irreversible.
If you use a "vibe coding" tool to build software and deploy it straight to production, you technically do have a human in the loop, but too late to stop something bad from happening like publishing all of your customers' credit card numbers.