transparency is infrastructure
agentic spam and its cures
I participated in some unusually productive and thoughtful social media Discourse, yesterday, occasioned by this piece of techno-optimism which was doing the rounds:
My personal first reaction was that the author was guilty of some serious yaddayaddaing here – that tiny phrase “these 20 ppl” conceals the most difficult task in organising a child’s birthday party, which is constructing the guest list. But other than that, it’s not that weird or undesirable a fantasy. You could actually, if you knew what you were doing and were familiar with a lot of APIs, code yourself a bot to execute these tasks, so it’s quite a nice thought to imagine being able to “vibe code” such a bot into existence by just giving plain language instructions to your AI agent.
But then Daniel Knowles made what I think is the crucial point – you can’t just assume that the world stays exactly the same, and the only thing that changes is that now you have an AI agent. As people said in the debate, it’s “like the difference between driving a car in a car ad, and driving in urban traffic”. Or alternatively, the central problem of fantasy fiction – that it’s a bit ridiculous to just drop magic spells into a medieval setting, unless you have some explanation of why the existence of magic hasn’t completely changed the whole basis of society.
The author of the extract above actually kind of gives it away; it takes some kind of brass neck to base your techno-optimism on a proposition like “imagine if everything in life worked as well as digital advertising!”. The digital ad brokerage networks he’s talking about are notoriously bad; they cost a lot to operate, are riddled with all manner of clickthrough fraud and pretty conspicuously fail in the alleged goal of serving relevant advertisements to people on the web[1]. He might also have used restaurant reservation apps as an equally dystopian outcome, and I notice that even in his example, he’s assuming that his agent is going to be allowed to make reservations at two different bowling alleys, an absolutely pestilential practice which is causing more and more restaurants to refuse to deal with them at all.
So how might things actually work? It’s harder than you might think to predict. A lot, I think, depends on whether costs come down, or whether AI tokens remain comparatively expensive to generate. Most of my intuitions, including the analogies to clickthrough fraud, SEO slop and email spam, are based on the near-zero cost of proliferating externalities in the online world as it currently works. If that turns out not to be the case (either because the energy cost problem isn’t solved, there ends up being some sort of copyright levy on training data, or monopoly pricing really holds up), then it’s actually a much simpler case to analyse. All that happens then is that AI servants are a bit like human servants, in that only some people can afford them, those people have an advantage over everyone else, and they still seem to spend a surprising amount of their own time on things like party planning.
If agentic AI is available to the masses at low cost, though, then there will have to be lots of guard rails imposed. People keep telling me that various of the current problems of hallucination will soon be solved, and I’m not in a position to gainsay them – but right now, there is not very much actively malicious use. If you think you hate the pollution of the online space with useless AI slop, wait until you see the space being flooded with things that have been purposefully designed to deceive and misinform.
If there isn’t some system which works at least as well – and therefore costs at least as much, which is to say a hell of a lot – as the anti-spam protections of the currently existing platforms, then there’s just no future for agentic AI. Nobody is going to send the little computer program with their bank details out into a system that’s a massive arms race. (Rupak can tell you all about this, because this is the story of every algorithmic trading venue that’s ever existed, a very complicated and very political dance where you need the high-frequency traders to provide liquidity, but you also need to keep them under control so that the venue doesn’t become hostile to “real money” which allows orders to actually execute).
And that makes me, in turn, a bit more optimistic for the prospects of AI regulation. Basically, the only viable future is one in which AI agents have some means of identifying themselves, credibly establishing their provenance and certifying themselves as non-abusive; otherwise, no system can afford to take the risk of interacting with them. It might even be necessary to build in some kind of “requirement for explainability”, to certify that the agent is acting for transparent reasons rather than strategically to manipulate some other aspect of the system. (I would guess that there would need to be some protocol to allow an agent “negotiating” with a bowling alley to demonstrate that it wasn’t booking multiple slots with the intention of cancelling all but one). Some people in the industry might be fantasising about a new world without regulation, but there’s no such thing – these are creatures of pure information, they can’t exist or interact without making themselves legible.
[1] They also pretty conspicuously don’t make use of stablecoins for payment purposes, which makes me wonder whether they’re all that great an example for the alleged low costs, high volume and instantaneous settlement that he’s making either.
“Basically, the only viable future is one in which AI agents have some means of identifying themselves, credibly establishing their provenance and certifying themselves as non-abusive; otherwise, no system can afford to take the risk of interacting with them.”
For a similar reason, I’d argue that companies providing identity verification software are going to become increasingly important to online functioning. As the costs of producing passable online content decreases, the ability to identify that this information did indeed come from a verifiable (and potentially non-AI) source is going to be essential towards navigating the post-gAI environment.
Visions like that might be superficially tempting, but to me they always really illustrate the intellectual bankruptcy at the core of this whole wave of hype.
Like, let's just imagine that this agent could actually do this (which is huge- the last mile problems abound). How many of those tasks involve labor where the figuring out what you want is very different from the asking for it? The magical choice of twenty people to invite and sending them a note (as if this was a big ask when email address books exist), the choice of where to go on vacation (if you know what you want to see, then this is irrelevant, if you don't, this is indistinguishable from a tour company), editing the memo that, if it needed to be sent at all, presumably was mostly information already in your head and needed to be fed into the agent by writing it down, the notion that the 'negotiations with the bowling alley' weren't, again, a person needing to express their preferences between times and prices, etc., Like, these aren't hard things- they might also be *inevitable* things. The idea that tech utopia is that tickets are one notch easier to get- I dunno, it reeks of an impoverished imagination.
And the idea that this means that ads just go away or something feels hilarious to me. We have ten years of hilarious and horrifying examples of classifier being effectively 'hacked' by single pixel changes to images and the like, and someone thinks that letting robots (whose primary challenge remains *making the right decision in a noisy human world*) will make some kind of hyper-rational, Mr. Spock responsible fiduciary choice for you instead of *being even easier to make click on garbage* is comical- especially because the clicking agent is probably available for scammers and advertisers to test against!
And that's even before we get to the fact that the current iteration of these technologies- cloud hosted, as yet unprofitable, backed by enormous amounts of computation- is almost certain to be more-or-less ad-supported soon. Between Claude's Golden Gate obsession and Grok's white genocide system prompt, it's clear that these, like any piece of software, aren't oracles descended from the heavens- they're boxes that say what their owners want them to say, and it seems certain that will soon include messaging that is to the benefit of their owners and not their users.