I have Many Thorts about the regulation of agentic artificial intelligence (partly influenced by this excellent new book, mainly about financial regulation). But I think they might have to wait until Friday, because before I can get into them, I need to explain, to myself and to you lot, how these things work, and consequently where the “regulatable surface” is easiest to find.
This is not in any way original to me – I adapted it from Cory Doctorow, he credits Cal Newport, and I first got the idea (without understanding it at the time) from Ben Recht. All I’m really adding is a bit of a narrative, and using an analogy that most of us bored office workers understand –macros.
(For the non-office worker, a macro is a feature of Word and Excel which allows you to record a lot of operations and then repeat them with a single keystroke. So, you might have one to select some data, turn it into a bar chart, change the colours on the chart to match the corporate style, put currency signs on the axis, etc. But macros can get very sophisticated indeed, particular since you’re actually able to edit them and write commands as if they were a programming language. Some people have really really big macros that practically amount to a piece of enterprise software in their private directory. Which, of course, is an insane thing to do, but it means that a lot of the problems of AI agent regulation are very similar to problems I’ve already seen in the regulation of IT in banks).
As I say above, the main use of macros is to automate repetitive tasks. Let’s pretend I’ve got quite ambitious and the task I want to automate is “asking for help with writing macros”. So the simple macro I want to write is something like:
SELECT one of my mates from [a list] of people who I know are sometimes helpful
WRITE AN EMAIL with the content “Hiya mate, I’m trying to write a macro to do $X, what do you think I should do?”
WAIT until the email is answered
SHOW me the advice (if you’re feeling fancy, format it to cut off unnecessary headers, maybe cut and paste it into a word document so I can see advice from different people together)
In order to run this macro, I need to point it to the list of contacts, supply the text to replace $X and then off we go. It’s not saving me very many keystrokes though.
So, maybe I notice that sometimes my friends just take pity and write the macro for me. Let’s add some code to handle that:
IF the reply contains executable code <--- just assume this is easy to identify
THEN run the code;
IF the code works
THEN alert me to the success, write an email saying “thanks so much mate yer a lifesaver LOL” and terminate
ELSE paste the output into another email, add the text “doesn’t seem to work mate does this make any sense to you” and wait for another response
This is potentially more of a time saver, and you can see that this macro already has the important property that everyone is worried about – I am executing code I haven’t checked, and if my mates are incompetent or practical jokers, I am likely to end up in a pile of smoking wreckage saying “the computer has done something which I did not predict or intend!”. I can turn it into even more of a corporate IT guy’s nightmare by adding another subroutine:
IF the reply contains a URL including “stackexchange.com”
THEN download whatever code you find on that webpage, run it and respond to the email as if the code had been sent directly.
There are other ways to grow my macro. I could add:
IF the reply contains “sorry mate if that doesn’t work im out of ideas LOL”
THEN SELECT another $mate from [the list], send another email, appending output so far and adding “$mate told me to do this but look at this crap LOL please please if you could please mate owe you big time”
It is tricky to express this next one tersely, but it is also the case that if I have mates who are prone to say “you need to break this down into a sequence of steps”, then I can write subroutines to create more query emails for each step, then combine them together. That becomes important in the next step, where we replace “a randomly selected mate from my list of mates” by “a chatbot”, and point out that this hypothetical macro does neither more nor less what an “AI Agent” does. It automates and organises the process of repeatedly querying an LLM, trying out its responses and updating your query to take into account the results. It saves keystrokes, is what it does.
[I suspect some people will choke on the idea that a macro could successfully identify code, spot the meaning of “sorry can’t help” or know where to split tasks. To some extent, maybe just imagine all my friends communicate in an incredibly structured way, or that I have set the macro up to make web forms that constrain how they can respond. But it’s easier to just suspend disbelief I think].
Unfortunately, the hijacking of ordinary language words for technical purposes is as inevitable as it is destructive. (The economists’ hijacking of “rationality” is a particularly bad example, but the alternative is to go full Stafford Beer and expect people to get along with “the algedonistic metasystem of the anastomotic reticulum”). The words “attention” and “transformer” don’t give you much clue that what you’re looking for is “finding some way of keeping track of very small coefficients that tend to get lost off the end of the calculator when you’re dealing with very big matrices that are almost all zeroes.” And the word “agent” has so many connotations of, well, agency, that it distracts you from considerations which would be front and centre if it was described as a “macro for organising and keeping track of LLM prompts”.
I think this also has implications for how we think about the regulation of agentic AI, now that we are worried about “rogue agents”. The frontier labs seem to be strongly of the opinion that this phenomenon is a failure of alignment and needs to be addressed by making LLMs that don’t go rogue. In my analogy, this would be like me telling the corporate IT people “it’s fine, I just need to get some new mates who are not practical jokers, maybe we could ask stackexchange to do better quality control” rather than “for pity’s sake stop downloading random bits of code you don’t understand and executing them”.
(More to come on this bit on Friday – I have let that joke stand because I like it, but it is unfortunately not quite as simple as that. There’s no real equivalent of corporate IT for the whole world, and there’s a lot more potential upside to be lost by completely banning agents than by completely banning my macro.)
Furthermore, I think this is a vindication of Ross Ashby’s design principle (as explained to me by Richard O’Rourke) that AI systems need to be thought of as an “intelligence amplifier” for the human decision maker, who has responsibility for sitting in the middle of the feedback loop checking things. The problem with my macro – and, as far as I can see, the problem with all the rogue agent episodes so far – is that it’s iterating too many times without coming back to anyone who can supply the necessary context.
At the end of the day, if you repeatedly call an unpredictable function, feeding its own output back into the input, you rather lose my sympathy when you say that you couldn’t have predicted the outcome. Of course you couldn’t!
A related question that has been troubling me lately is how to ensure that the human decision maker in the loop actually does the job of checking things. At least my experiences with generative AI tools suggest that they are often very good at producing reliable-appearing output - indeed, better at the appearance of reliability than at the substance of it. At some point doesn’t that appearance itself become a design flaw, because it lulls the user into a false sense of security?
The macro analogy does something the “agentic AI” framing tends to obscure: it makes visible where the human was supposed to be.
In your example, that human is the person who wrote the macro — the person who is supposed to check whether the code pulled from Stack Exchange is actually doing what they intended. The moment the macro starts iterating without returning to that check, the system hasn’t necessarily “gone rogue.” The feedback loop the human was supposed to sit inside has simply been removed.
That’s the same failure mode I see in so many of the frauds I examine. The invoice approval system that routes everything through one person who also happens to be the vendor. The audit that verifies the process rather than the underlying fact. The credential checked once and then assumed to remain valid forever.
In each case, the control exists. What’s missing is the moment when someone with both context and accountability has to look at what the system produced and ask: does this actually match what we intended?
Your Ross Ashby point lands exactly right: intelligence amplifier, not intelligence replacement.
The regulator’s problem is that “the human checks the output before the next iteration” is a design principle everyone agrees with — and almost nobody actually builds into the system.