While Charles Keating was dividing his time between his financial frauds and his public puritanism, two Israeli filmmakers were operating on the opposing side of the culture war, while progressively making a worse and worse name for themselves. Cannon Films attracted a bad name in artistic terms, due to the plethora of ludicrous, poorly acted films they put out, classics like “Enter The Ninja” and “Electric Boogaloo” which tended to feature either copious female nudity, Chuck Norris or both. And they attracted a bad name in financial terms too, for their sharp accounting practices, and their habit of pre-selling distribution rights at high-pressure sales meetings in Cannes on the basis of an outline script or a poster, then delivering a film significantly worse than promised because they ran out of money. If you were around in the 80s and ever wondered why Superman IV: The Quest For Peace was so bad, that’s why. But this is not the story of Yoram Globus and Menachem Golan. It’s the story of the man they eventually sold their failing film company to, Giancarlo Paretti.

Paretti was a minor player in the tangentopoli scandals of the period; if you are ever tempted to believe stories of how aggressive and cynical Hollywood people can be, recall that they sold one of their crown jewels to a fraudster who was not even a second tier player by Italian standards. In fact, Paretti was not just a bag-carrier, but a bag-carrier of the second order – he was the junior partner of Florio Fiorini who was himself, although a board director of ENI and a conduit of money to the PSI, basically a bag carrier to the real political and financial players. Before he got into the film business, Paretti’s main media industry experience was running “Diario”, a series of newspapers run as partnerships with the PSI which had tiny circulation, but significant revenues from selling advertisements to businessmen who wanted political favours.

But even a relatively minor player can do a lot of damage if he manages to get control of a corporate vehicle through which large amounts of finance can be raised; that’s the nature of control fraud.

And Hollywood filmmaking, as an industry, was particularly vulnerable to control frauds, because the relationship between lenders and borrowers is unusually skewed. Movies – even really bad ones – cost a lot of money to make, and once started they are hard to close down or take possession of. It is much more difficult to separate the borrower from the business; a half-built office block can be finished by another contractor, but it’s much more difficult to remove a half-shot film from its producer and director. And time is rarely on the lender’s side; films have shooting schedules and realease dates, so if you delay a decision you might destroy all hope of getting your money back. For this reason, film financing is a specialised business and not one for the faint of heart. In the 1980s, the king of Hollywood bankers was the man who invented the pre-selling distribution rights trick, the Californian representative of French bank Credit Lyonnais, a Dutchman called Frans Afman.

Hollywood is also a people business. Personal relationships are vital, and as the song goes, “There’s No People Like Show People” when it comes to having complicated relationships with one another. People give each other gifts all the time to demonstrate how important they are to one another. Sometimes quite valuable gifts, if they want to really emphasise a point. It is, or was in the 1980s, a place where things might happen that would be regarded as obvious and massive conflicts of interest anywhere else. As Michael Ovitz, the super-agent who founded CAA, used to put it, “no conflict, no interest”.

Paretti and Frans Afman first met at the Cannes Film Festival, where he immediately made an impression. First, by blatantly staring at Afman’s female colleague while pointing to his groin, and second by offering to “triple your salary” if Afman were to work for him while also remaining as head of film finance at Credit Lyonnais. It was an extraordinary approach; apparently it took Afman several minutes of conversation to realise that this was an attempt at a bribe. To his credit, he turned down both the offer and the proposal to finance Paretti’s acquisition of Cannon Films. This proved to be a good career move in the long term, but a bad one in the short term – Paretti met his boss the next day to overrule the decision and subsequently got Afman fired.

Georges Vigon, the global head of lending at Credit Lyonnais, had performed heroically during the Algerian War, but did not live up to the same standards as a banker. Despite commissioning a report into Paretti’s background which detailed his previous convictions for fraud (among other things, crudely adding zeros to a cheque to convert a $3,000 deposit into a $3m one), he allowed the Italian financier to take control of Cannon Films, a company which Lyonnais already had such a large exposure to that the regulators were telling them to reduce it.

Perhaps he saw something in Giancarlo Paretti that inspired trust. Perhaps that something was the copious flow of trips to Bora Bora and gifts of Picasso sketches which Paretti lavished on him. Perhaps it was the time that Paretti took him aside at a party, into a room with Jacques Griffault (Afman’s replacement) and gave them each a share certificate in “21^st Century Distribution”, a company he owned and whose debts were guaranteed by Cannon Pictures^[1]. Or perhaps it was the way in which Paretti hired Vigon’s ultimate boss, Jean Naville, and turned him into a lackey forced to negotiate with his former colleagues. Whatever it was, Vigon’s judgement was quickly and thoroughly corrupted.

Paretti’s first act as the new owner of Cannon Films was to change its name to Pathe Communications, in preparation to make a takeover bid for the venerable French company of that name. This move caused mighty and possibly intentional confusion^[2], as the bid was blocked by the government, probably more out of national chauvinism than any real appreciate of what the bidder was like.

Despite this failure, the now misleadingly-named Pathe was still in the mood to buy more assets. And after appointing Alan Ladd Jr as the local executive face of his operations (son of the cowboy actor famous for saying in the film Shane that “a gun is a tool ... as good or as bad as the man using it”), Paretti set his sights on one of the most famous studios in Hollywood, the roaring lion of MGM.

Why MGM? Partly because it was available; a marginal financial performer, it was known that Kirk Kerkorian, the corporate raider who owned it, was willing to part with it at the right price. But Paretti needed to buy something big and unprofitable. It had to be big because as we have seen, a control fraud relies on the size of the asset base you control. And it had to be unprofitable, because people don’t tend to sell big and profitable operations.

The lack of profitability did not concern Paretti because a control fraudster profits by draining the asset base and pumping a company full of debt, rather than by diverting its profits. And drain the asset base was what Giancarlo Paretti did; even when all he owned was the relatively small Cannon/Pathe, he had it pay for his family to live in a mansion in a wealthy LA suburb, and for three models on “acting” contracts whose theatrical talents extended only to pretending that they enjoyed sexual intercourse with a short, fat, vulgar criminal. He did business out of the former offices of legendary director Dino De Laurentiis whose troubled film company he helped; this friendship earned him an entry into the Hollywood social scene. It also earned him a punch in the balls from De Laurentiis’ daughter when he tried to grope her thigh at a party.

Credit Lyonnais was, from the outset, under pressure to reduce its lending to the film industry in general and to Cannon/Pathe in particular. Griffault and Vigon had already begun to use the accounting tricks discussed in our section on Parmalat in Chapter 5 to conceal debts, lending to companies like 21^st Century Distribution which were legally separate but actually controlled by the same party. They did not necessarily want to lend any major proportion of the US$1.25bn that Kirk Kerkorian wanted for MGM. So they encouraged Paretti to be creative, and to raise money elsewhere. This turned out to be a mistake.

Paretti did not have much of a way with investors. One of his deals fell through after he referred to the oil billionaire Marvin Davis as “a big, fat, rich, dumb Jew” while on a drive to the airport, unaware that Davis’ driver spoke Italian. But he managed to raise half of the purchase price by pre-selling the television rights to the MGM library to Time Warner, easing the passage of the deal by another gift of a Picasso sketch to Time Warner’s chairman, Steve Ross. The other half of the deal was to be financed with an investments from Paretti and Florio Fiorino’s other companies, including the giant hotel chain Melia.

There were numerous problems with this financing structure. For one part, although Paretti was fond of waving around a Melia brochure listing all its hotels and real estate to show what a rich man he was, he didn’t actually own them. Years before, Paretti and Fiorino had acquired the Melia company, but had financed the deal by pre-selling all the real estate to another hotel operator. Five minutes after the paperwork was signed, all they owned was the Melia name (which itself was licensed to the hotel purchaser), and a near-bankrupt travel agency. The money that Credit Lyonnais lent to Melia to finance its investment in the MGM deal was as good as gone the moment it was lent.

And for the other part, Steve Ross was a cautious man who liked to insure his valuable artwork. In the course of the insurance appraisal, he found out that the Picasso that Giancarlo Paretti had given him was a fake. So were most and possible all of the Miros, Picassos and other priceless artworks with which Paretti had bribed the Californian film and finance communities. Time Warner pulled out of the deal, so Credit Lyonnais had to pick up that half too. The total lending, including sums already advanced to finance Cannon/Pathe, was getting toward $2bn, well beyond the approval limit of any one manager or team, and so Vigon had to refer the loan to head office.

But Paretti, a lifelong bag-carrier for crooked PSI financiers, was also able to refer things to his own head office. It is not clear and has not been proven exactly what happened, or what money changed hands, but in later court testimony, Florio Fiorino stated unambiguously that the Italian foreign minister at the time, Gianni DeMichelis, made representations to the French government which linked Credit Lyonnais’ support for the MGM deal to a large purchase of equipment for the Italian railway system from a French manufacturer. Whatever happened, Credit Lyonnais financed the deal and its then chief executive, Jean-Yves Haberer, got himself significantly more personally involved in the transaction than was good for his subsequent career.

When the deal was completed in 1990, just as the walls were closing in on Charles Keating, Alan Ladd Jr^[3] was sent on a charm offensive to reassure Hollywood that the new Italian owner was a man of substance, and the matter of his having been found guilty in absentia and sentenced to three and a half years by a Naples court over frauds related to the Diario bankruptcy was in some way technical or trivial. This confidence building work was somewhat undone by the decision to appoint Valentina Paretti, the owner’s 21-year old daughter with no relevant experience, as the chief financial officer of MGM and to fire most of the audit and accounting staff responsible for monitoring the studio’s outgoings. It is not a difficult deduction to guess why Giancarlo might have done this; however, it breaks one of the key principles of the control fraud and he paid the price for failure to respect this principle.

That principle is that a control fraud needs to be a theft carried out by honest means. The payments themselves made to the fraudster are made through legitimate commercial transactions; salaries, bonuses, dividends, mansion rentals, contracts for actresses who never act, and so on. It is always the overall scheme, and particularly the financing and growth of the controlled entity, which constitutes the fraud. If you start simply grabbing the money, then two things happen. Firstly, it becomes easier to spot you, as you are behaving less like an executive and more like a criminal. And second, by removing your attention from the actual business, you start to lose the vital dimension of control.

It was the second issue that sunk him. As it went into the final phase of Paretti’s looting, MGM started getting sloppy with making its payments. Dustin Hoffman and Sean Connery failed to receive their fees on time. The film labs began to refuse to process the negatives of Thelma and Louise until their bills were settled. And eventually someone, probably an agent, decided it was time to get an aggressive lawyer involved.

As we said in Chapter 2, if you have a payment come due and you fail to make it, then you are bankrupt, in the legal sense, and somebody can take you to a bankruptcy court. Hollywood lawyers do this all the time, without necessarily meaning any great harm by it or intending to see the process through to its conclusion – in a town where people are always alternately kissing and screaming at each other, it’s one of the few ways to unambiguously indicate that you want to be taken seriously this time.

The trouble is that things tend to come out in bankruptcy filings; in particular, lists of creditors are compiled, and lists of assets, and lists of the previous and current fraud convictions of principal actors. It rapidly became apparent that Credit Lyonnais had lent many times the legal limit on its exposure to a single borrower, and had done so to a financier who was currently a fugitive from a prison sentence, and to a company which had made very large unauthorised payments to investment vehicles of doubtful provenance but suspicious ownership.

Credit Lyonnais realised that they had to do something, and called in their loans, effectively removing Paretti from control and repossessing his film studio. Alan Ladd Jr, presumably reflecting on his father’s famous line about a gun being as good or bad as the man that uses it, immediately turned on his old boss and denounced him, joining with enthusiasm the litigation which was to consume the next five years over who actually owned MGM and who had the rights to Cannon/Pathe’s global assets.

It ended badly for everyone; the sooner you got out of this story, the better your reputation survived. Paretti and Fiorino went to jail, Credit Lyonnais went bust and Kirk Kerkorian ended up owning MGM once more.As Michael Ovitz said, reflecting on the affair thirty years later, “In Hollywood, when somebody shows you a Picasso, it’s either worth a lot more than they say it is, or a whole lot less”.

^[1] In later litigation, Vigon and Griffault tried to claim that they thought the stock certificates were joke party favours.

^[2] There was at least some pretext for doing so. One of the reasons Cannon Films was in such bad financial trouble was that Golan and Globus had overextended themselves buying a chain of cinemas in England. One of the assets that came with the cinemas was the rights to the film library of Pathe’s English-language output over the previous century.

^[3] Having been a loyal employee at Cannon/Pathe, Ladd was made the chief executive of MGM, responsible for its operations and for maintaining a good facade about town. The job had previously been tentatively offered to Ronald Reagan, but was decisively turned down.

“If you had a choice between instantly receiving £50,000 or a 50% chance to win £1m, which would you pick?”

The debate has been split roughly between “five hundred thousand is more than fifty thousand, duh” and “actually fifty thousand pounds would make a huge difference to lots of people’s lives actually as you would know if you weren’t so privileged actually”. For what it’s worth, 73% of people polled by YouGov said they would take the certain money.

It seems that the Discourse, though, has focused on what it’s rational to do. From an economic perspective, what is the correct choice to make? What does the science of economics say about this dilemma?

[…]

Subscribe now

[…]

[…]

Ok I’m gonna stop you right there. Put down the graph, stop talking about utility, stop talking about risk aversion. What the science of economics tells you is that this will never happen. Nobody is ever going to put you in any situation where you’re stopped in the street as a stranger and offered either tens of thousands of pounds for nothing, or a risk free gamble worth half a million. If they do, you have long since left the world of economic reality behind.

Whatever else economics might say about rationality, it definitely says that the idea of “giving someone a choice between taking £50,000, or a 50/50 chance at £1m” is stochastically dominated by the feasible alternative strategy of “not doing that”. The best you might hope is that a bored behavioural economics prof might offer you 5p or a quid, and hope to extrapolate the results.

We have circumnavigated this mulberry bush before on the blog. Human beings evolved to make choices in real world situations, not completely bizarre ones that would never happen. Intuitions drawn from hypothetical cases stripped of all context and content are not necessarily insights into rationality. Here’s another hypothetical:

“You receive an inheritance of $1 million from a grandparent. A stranger claims to be a long-lost relative and contests the will, but offers to settle for $950k. Your lawyer tells you that you have a 50% chance of winning in court. Do you take the settlement or defend yourself?”

Strangely, people who claimed to have urgent needs for a certain £50k over a possible million in the first case start get very argumentative about the second. One of the things we know about choice under uncertainty is that framing effects matter.

But framing effects are all there are! Decisions happen in frames. They aren’t departures from rationality, they are the normal process of normal decision making operating normally.

As a matter of history the laws of probability we're first studied in the context of games of chance. But a casino is a very weird man-made environment, not a laboratory of rationality. The fact that the heuristics and choice principles that have kept humanity from dying out over thousands of years don’t necessarily give the right answers to some kinds of exam question isn’t a “bias” or “innumeracy”, it’s a failure of the model to describe reality.

The proverbs this time round are in a family, based on something a client said to me when we were discussing the question of which global banks were actually winning market share and which losing. As he put it, “Guidance is an opinion. Provisions for risks are an opinion. Even revenue is surprisingly often an opinion. Headcount is a fact and real estate is a fact”.

I later plagiarised this and adapted it for when I did training on fraud prevention. My spin was something like “It’s easy to create fake sales. It’s easy to create fake profits. It’s pretty easy to create fake cash. But it’s difficult to create fake buildings, and it’s very difficult to create fake people”.

These all relate to the concept of “ground truth”, the idea that in order to interpret “data” (in the sense of processed and systematised information) correctly, you often need to have unprocessed and unsystematised information to hand. And that the value of ground truth is not that it’s a more accurate picture of something, because it usually isn’t. It’s precisely that it’s unstructured, it doesn’t have any assumptions going into its creation. Something similar is behind my own “golden rule for fraud detection” from the last chapter of Lying for Money, which is “anything which is growing fast needs to be checked out, in a way in which it hasn’t been checked out before”.

All these could arguably be summarised as “listen to what they say, but look at what they do”. Actions speak louder than words.

And this is what’s on my mind at the moment. I think it’s really quite interesting that two of the big “hyperscalers” (SpaceX and Meta) are currently signing deals to sell their “excess compute”. Whatever they say, this looks to me like the sort of thing you’d do if you had basically given up on winning whatever the equivalent is of an arms race for frontier AI. Meanwhile, OpenAI and Anthropic are on the other side of the deal^[1].

There are two reasons why someone might give up. One would be that they don’t think their skills are up to the competition, and the other might be that they think that the prize isn’t worth winning. I have my own theories about which might be which, but in the meantime, and related to the “sensible scepticism” project, it does strike me that the scenario that things like “Europe 2031” are focusing on is one in which AI is hugely transformative to the economy, in which the rewards are captured by the developers of frontier models, but in which these rewards are not sufficiently tempting to Elon Musk and Mark Zuckerberg to stay at the table. That in turn would seem to imply an assessment of Meta and Grok’s technology which is quite negative, but also an assessment of both CEOs’ ego and ability to stop chasing a lost race which is quite positive. I would say that’s a possible scenario, but it’s quite a specific scenario.

^[1] For completeness, Google is buying compute while Amazon is selling it. I am less inclined to draw conclusions from these players because I think Google is probably a forced buyer in order to defend its existing monopoly, while Amazon has been a seller of all kinds of compute for a long time and hasn’t really been close to the game in frontier models anyway. Also Microsoft, where I’m not even able to work out if they are a net seller or a net buyer.

Subscribe now

There is a great danger of assuming that controls work in a predictable and consistent fashion. When you turn the steering wheel of a car, the wheels turn and the car moves in a consistent way. The steering doesn’t suddenly make a sharp turn in response to a small touch on the controls. And if you steer one way, then reversing your movements brings you back in exactly the same way – the steering doesn’t change its response depending on what direction you’re going in, or what you did previously.

Let’s describe this sort of stable and consistent relationship by saying that “the control surface is smooth”. The map which translates inputs from a steering wheel, or any other controller, is a map of a flat surface, or at least one of gradual and regular curves. Important properties of a smooth control surface are that fine-tuning is possible – you can home in as closely as you like to a desired position by making small adjustments to the controls. And the business of control is reversible – anything that you do, there is some sense in which you can undo it by moving the controls back to where you started from.

Smoothness of a control space is a very desirable situation to be in; there’s a reason that most machinery is designed in this way. For aircraft, smooth response is highly prized, and the extent to which reality approaches this ideal is a measure of how good the flying conditions are. In high winds, close to stall speed or with badly designed aerodynamics, the pilot might have to fight with discontinuous or volatile responses, but in a good aeroplane on a good day at cruising height and speed, the response to the stick should be pretty smooth.

Unfortunately, in social, political and management situations, if the “policy levers” exist at all, they operate on the basis of a very bad control surface.

[…]

What Professor Zeeman and his colleagues found, though, was that when it came to prison riots, the control surface was warped and twisted. They decided to model it in two dimensions, declaring that there were two variables that could be influenced, which they called “tension” and “alienation”. The first of these was a measure of how discontented the prisoners were, which could be measured by things like the population of the number of requests to see the governor, the number of men in the sick bay and the number of welfare visits. The second was a measure of how bad the prisoners’ ability to communicate with authorities was, measured by the population of the disciplinary wing and the number of requests to go into segregation. And these were related to a third measure of “disorder” which included fights, work strikes and other elements of protest as well as riots themselves.

The control surface that they eventually drew, however, had to reflect the reality of prison riots. Which is to say that a prison riot doesn’t develop gradually as tension increases – you start off in a position that appears OK, then a small increase in tension results in a large and immediate step-change in the amount of disorder. Added to which, once you’ve moved from the “not riot” state to the “riot” state, you can’t just undo the actions which caused the riot and move back. The situation is not reversible in that way; getting to a truce requires much larger movements in the positive directions than the negative moves which started the riot. And things are path-dependent; the order in which they happen matters. If you know that an increase in tension is on the way, because of staffing problems or overcrowding or political issues in the outside world, then improving communication ahead of time makes it much less likely that this will lead to a riot.

And so the surface is folded – it has what Zeeman called, following the French mathematician Rene Thom, a “cusp catastrophe”. At some points, a small move in the direction of higher tension or alienation takes you off a cliff edge, with a sharp increase in disorder. When that happens, you’re on another part of the surface, and the path which brought you to the cliff edge is inaccessible. You need to navigate another way back, possibly moving through another catastrophe that might be labelled “truce”. In principle, you can map out the shape of the cusp and get an idea where the control surface might respond to your policy interventions in a reasonably consistent manner, and where you might be vulnerable to sudden jumps. Zeeman and Shapland started to plot their data on a three dimensional graph, noting where they found sudden and sharp changes in the level of disorder.

[and, various consequences ensued …]

Subscribe now

“The Bono Complex” is a name I have given in my mind to a cultural phenomenon which I think is absolutely obvious – whenever someone is harsh about a popular band, particularly one which was popular when they were young, what they are actually telling you about is the things they are embarrassed about for themselves at the age when that band was in the charts.

So, U2 and their self-righteous political sloganizing? Oasis and their coarse and mindless behaviour? Sting, acting like he’s so much better than everyone else? Bruce Springsteen, all sweaty and earnest? Phil Collins, with that selfish streak. U2 again, but this time trying to be trendy. Blur, that absolute arse of a bass player getting into foodie stuff. The list isn’t difficult to extend.

“The Bono Corollary” is simply the observable truth that when you tell someone about the Bono Complex, they almost always go “oh, not me, I never liked them, always thought he was a wanker”, and are lying. If even half of the fiftysomething men who claimed to have hated Rattle & Hum the moment they first heard it were telling the truth, it would have sold about eight copies.

Particularly, Richard’s most recent papers have dragged out of the archives this wonderful piece by Ross Ashby, which seems to anticipate lots of the problems of LLMs and strategies for their solution, from the perspective of 1956, and from the perspective of someone who had built a “homeostat” which was miraculously able to stabilise a fluctuating voltage. The crucial insight that Ashby had – which the current AI industry is, to be fair, rediscovering extremely quickly – is that in his view, the key feature of intelligence is discrimination, being able to spot good from bad.

If you have a very efficient good-from-bad-spotting algorithm, then you can do a lot; particularly, you can make use of the “regulation by veto” method to use any external source of variety as a potential solution generating machine. And in this way, Ashby makes the case (in 1956!) that it’s obviously possible for human beings to make a machine that can solve problems that its designer couldn’t, as long as they’re able to recognise a solution when they see one. The problem is not really one of devising an algorithm to solve problems, but one that explores the space of solutions in an efficient way when it presents them to the evaluator.

Anyway, much more to come on this, but for the meantime, I think one of the consequences of Ashby’s argument, as set out by Richard, is that everything depends on how efficient your good-from-bad spotter is. And that this could be profoundly optimistic for the human race (or, I guess, pessimistic for that part of it which was hoping for a life of exorbitant riches gained from replacing human beings by robots).

The human brain is a really, really powerful wrongness detection engine, optimised by millennia of evolution. As far as we know, there’s a big general purpose neural network, with a bunch of specialised modules for detecting things like symmetry and cheating. And it’s contained in a surprisingly energy-efficient lump of biochemistry, mostly in your head but with help from a lot of nerve tissue elsewhere (“gut feel” is apparently a real thing).

We don’t know all that much about how you would write down what the human brain does as an algorithm, but it does look very much as if, considered as an algorithm, it’s significantly better than anything currently runnable on silicon. And when I say “significantly better”, I mean that in the sense used for comparing algorithms, in which the difference between a really good and really bad algo can be “half an hour versus the heat death of the universe”.

I think this matters, because as the number of variables and the size of the search space increases, the advantage of having a better algorithm tends to grow quickly. Since the starting point is one in which we are very close to maxed-out on our ability to fill that gap by pushing more electric power through expensive silicon, even for problems like writing computer code and drawing pictures of Taylor Swift in the style of Studio Ghibli, my guess is that we’re not far from the point at which people realise that it’s going to take another technological and mathematical leap at least as fundamental as the invention of neural networks themselves before they can start doing some of the things claimed for them.

So, I’m twisting the dial on “sensible skepticism” a little further in the direction of skeptical.There are going to be lots of really important problem domains where current AI technology can hugely improve its performance.But the ones I’m personally interested in – the solving of management problems in large organisations – are, I’m now prepared to bet, likely to maintain their intractability.

While I was growing up in North Wales, we all knew that there was one big piece of development policy that was crucial to the economy – the electrification of the coastal railway. This was the big thing that would have revolutionised transport links and allowed our little rural outpost to be dragged into the second half of the twentieth century. It was a shocking betrayal by Westminster that we never got it, almost as bad as (various grievances relating to the Welsh language and the flooding of Capel Celyn).

I ended up thinking, though, that the concentration of Plaid Cymru and the local Labour Party franchises on the Great Big Investment that we needed was a blight in itself.

For one thing, it sucked the oxygen out of local politics. It was always there, tantalisingly close, looking like it might just be one more feasibility study or round of grant applications away. And so everything concentrated on that – effort was diverted from things which might have had a more immediate return at a smaller scale, and which might actually have happened.

But for another, “planning blight” doesn’t just apply to negative externalities of construction. One of the things you learn at business school – but which, like most such things, is really a codification of the sensible instincts of business people – is that in periods of uncertainty, the option to wait and see is valuable. Electrifying the North Wales Coast railway would very much change the relative attractiveness of different investment projects, particularly in tourist infrastructure. So if it was just about to happen (or not happen), you could minimise your regret by keeping your money in your pocket for the meantime.

And that’s what people seemed to do. For years and years.

Later on, I got to thinking that something like this might have been at the root of the extraordinary divergence in outcomes between the two ends of the Holyhead – Dun Laoghire ferry trip. The Celtic Tiger economy began with the generation that David McWilliams calls “the Pope’s Children”, born around the time of the papal visit in 1978. Ireland didn’t have a baby boom, because it wasn’t involved in the second world war in the same way, so this was the generation that just decided that it wasn’t going to settle for rural mediocrity any more. That’s basically the story of Ireland – a huge increase in the labour force with women coming into the workforce, a generation of educated young people that was possibly a little too big to all emigrate, but really, the 1990s were the years when Ireland just basically sat up and decided something like “let’s go for it”.

I think one important factor in that decision was that Ireland understood that nobody else was going to do it for them. Irish governments have always been quite good at extracting money from EU development programmes (holding on to their Objective One status as an underdeveloped region well past the point of ridiculousness), but the thing about EU funding is that it always came on a matched basis; you get supported for things you are willing to do for yourselves. Welsh local government didn’t have the ability to do that because of the centralisation of infrastructure investment in the UK, so they were always diverting their energy into working the Westminster machine.

I’ve said before on this blog that one of the great benefits of devolution and independence is that it gets rid of one of the most insidious obstacles to development – that of “having someone else to blame”. If we’re going to do devolution, and I think we really should, we need to make sure that we’re doing it properly, in a way that facilitates Ireland-like outcomes and doesn’t create situations where local state capacity is purely focused on its relationship with the central funding source. So maybe not “tough love”, but a more sensible parenting strategy, aimed at building autonomy.

Subscribe now

Twenty years ago, a sociologist of science called Harry Collins tried out an experiment. He had been researching the community of physicists who worked on gravity waves for the preceding two decades, trying to get insight into standards of proof and how experimental results were reconciled and interpreted. In furtherance of this project, he had been attending gravity waves conferences, where he was a familiar figure and had lots of friends in the community (because he talked to lots of people, physicists often sought him out to get gossip and news on what other labs were doing and seeing).

The experiment was a sort of Turing test – one group of physicists came up with a list of half a dozen questions about gravity waves, and then sent them by email to both Collins and a physicist, with instructions just to answer them from general knowledge rather than looking anything up. A jury of other physicists were then given the job of deciding which set of answers had come from Collins, and which from the physicist.

And all but one of the judges identified the sociologist as the physicist.

Although extremely funny, this wasn’t actually meant as a practical joke; it was meant to test a serious hypothesis about expertise. Harry Collins theorised that there was a thing called “interactional expertise” (basically the ability to hold a sensible and useful conversation about a subject) which was distinct from “contributory expertise” (he didn’t actually even know enough maths to do gravity wave physics). Deep interaction with a knowledge base and literature could provide one, but it wasn’t the same thing as actually working within the field.

When I wrote this up at the time, I remember thinking – was there really a clear dividing line? Like, for example, if someone really tried and dedicated a lot of time and effort to it, might they be able to pass an even tougher version of the same test. For example, could you get someone to be a co-author on a published journal article, based on nothing more than interactional expertise? I think the answer might be “trivially obviously so if it was just a literature review, and quite possibly they could be seen as a helpful and valuable co-author even on quite substantial original research”.

Or in fact – how does transactional expertise of the kind that a sociologist of science can put together by attending conferences – how does this differ from the understanding of the cutting edge of research which might be possessed by a principal investigator who runs a lab, makes decisions about hiring and such but who hasn’t actively carried out an experiment for the last decade? If someone like that hires a professional grant application writer to describe their current research program, what is the level of expertise that’s gone into that piece of writing?

You can see where I’m going here, can’t you, particularly since I gave so many clues on Wednesday. Because it seems to me that there is an important distinction here, which is not any less important because the dividing line might be difficult to establish empirically, or even if that line turns out to be in a different place from where we guessed it was. As well as difficult cases where it’s not clear, I think we could also come up with cases where the distinction between interactional and contributory expertise would suddenly become very clear and important indeed – the ones where someone who was faking it got “found out”.

And so the question that I think is quite important is whether there is a similar kind of distinction between the kind of expertise that it’s possible for a machine[1] to get by industralised consumption and interaction with a much larger corpus of literature than any human being could inhale, and genuine contributory expertise that could apply to entirely new situations outside that literature.

I think there’s a few contradictory intuitions here, and I’ll put down a marker that I would probably guess that the answer would be different in mathematics, logic and similar fields where there’s an axiomatic path to extrapolate from things you’ve already read to interesting and important theorems and sentences that haven’t yet been uttered. At present, I think it looks like the very impressive results of LLMs in mathematics seem to resemble Thomas Royen’s proof of the Gaussian correlation inequality – the sudden making of a connection which allows interaction with one part of the literature to open up something that’s always been implicit in another part.

But if we start thinking about other fields where progress isn’t monotonic; where things can be disproved, authorities disagree with one another and results can be true at one time and false at another … as we move along that spectrum I think something like the interactional/contributory distinction becomes more important.I am still not sure whether it’s a sharp distinction between two different things – and I’ll be writing more about that when an important paper gets published next week that I will be citing a lot – but I think it matters.

[1] To continue with the project of “sensible skepticism”, I should say that there’s two questions – whether it’s possible in principle for any machine at all, and whether it’s possible for any machine which can be built subject to reasonable resource and economic constraints in the foreseeable future.

Basically, Thomas Royen was a retired statistician who had worked in the pharmaceutical industry. While brushing his teeth, he realised that a couple of formulas that were quite standard in that field could be put together to prove something about functions of correlated gamma distributions, and that this thing was equivalent to proving a general theorem that had as one of its special cases a famous unsolved problem called the Gaussian Correlation Inequality.

What’s interesting about this case is that everyone who had been working on the GCI had assumed that it would be solved by extremely tricky convex geometry or some such; as a result it was mainly worked on by different kinds of mathematicians who weren’t familiar with the same formulas as medical statisticians. The proof of the GCI was always there implicitly in the literature, but nobody who cared about one thing knew about the other and vice versa, until one person made the connection.

I think - let’s have a heated debate! - that this seems to be, from the press reports and comments from mathematicians close to the most recent LLM proofs, something like a model for how LLMs do it. If you’ve got an extrapolation engine, and you’ve interacted with lots of mathematical literature, then you are very well placed to industrialise the process of filling in those gaps.

On the one hand, this is really impressive, and absolutely gives the lie to any idea that “it’s just next token prediction” is an important thing to say about anything. If you’re pointing the vector in the right direction, next token prediction can be very useful!

But on the other hand, I am not yet convinced that this is the kind of thing which produced the real era-defining breakthroughs in mathematics, and more importantly, I don’t think it’s a trick which will transfer all that well to other sciences, even less well to the humanities and not at all to my personal area of interest, general management.

What do you think?

Subscribe now

I was talking the other week with a number of different groups of people who are working on the general issues in planning and state capacity which I refer to as “The Problem Factory”. There really does appear to be a lot of enthusiasm in YIMBY world for restructuring the planning system to make it more difficult to tie up projects in legal risk for years, and it is making me a bit worried, because I think they might be losing sight of the original vision.

What I mean by this is that NIMBYism is fundamentally a political problem, not a legal or technical one. Lots of people don’t want things to be built. Sometimes these reasons aren’t very good[1] reasons, but this is a democracy.

If you take away too many of the legal and process mechanisms but leave the fundamental lack of consensus and consent in place, I think the problem is likely to bubble straight back up as a political problem. Which means that it will arrive very directly at the door of the Secretaries of State who are currently making speeches about builders and blockers. The Problem Factory is an accountability sink, in my own terminology.

And when you dismantle the problem factory, you dismantle that accountability sink, which means that you have to face up to all of the unpleasantness which the sink was protecting you from. How brave do we think ministers are actually going to be, when asked to make decisions which will have dispersed benefits two or three election cycles in the future but create concentrated costs now?

This is only one form of political risk, by the way – in days of yore, it was not unknown for people to physically resist infrastructure construction by chaining themselves to things. And equally importantly, when it comes to making investment decisions, political risk has all of the bad characteristics which people hate about legal risk. The risk surface is still there, it’s just that there is no process for managing it now, no judge to limit the number of times questions can be relitigated and no barrier to entry.

I think this might end up being another entry in my table of “attempts to solve the problem of sclerosis which had disappointing results”. You can’t get around the need for consultation and consent.

Subscribe now

[1] Although – the infrastructure and building industry would do well to look in the mirror once in a while. One of the big reasons why people might have a reflexive knee-jerk negative reaction to building things might be well summarised as “the past behaviour of the infrastructure and construction industries”. And one of the reasons why people are often resistant to evidence that projects are safe is “the past behaviour of the infrastructure and construction industries when it comes to lying”. Trust has to be built (and rebuilt once lost) – you can’t assume everyone else’s belief in your own good faith, any more than you can assume a can opener.

What’s on my mind are two things. First, Ed Zitron having got hold of some audited financial data from OpenAI, which to my mind is open to a lot of different interpretations, but which seems to confirm the general picture that everyone had of the economics of the big AI firms – that they make a gross margin of about 40-60% when you compare the revenue they generate from people making queries (using tokens, employing agents, basically “buying a thing”) to the cost directly attributable to computing for those sales. And then they have huge costs of other things, mainly training and researching new models, plus all the capex and depreciation and whatnot.

The other thing is the rash of news stories about companies getting more and more concerned about their AI token bills. I’ll come back to this, but I think it’s actually quite important to the whole story – a lot of the predictions about the effect of AI on the economy really did depend on inference being too cheap to meter, or at least significantly less expensive than human judgement. But first, a bit of accountancy influencing:

In principle a 60% profit margin on inference looks good. But (I do this spiel at training courses a lot) gross margin is a tricky little ratio, and is the absolute poster child for my slogan that “understanding the numbers has to come from understanding the business, not the other way round”. Ten per cent margin for a supermarket is not bad, because you have very fast inventory turn and very predictable demand, which comes from customers who walk into your shop and pick things up. One hundred per cent gross margin for a luxury fashion house can be quite bad if you are paying a load of sales commission, Mayfair rents and holding a million pounds of inventory. Restaurants usually operate with gross margins north of 70% but they go bust all the time.

Venture capitalists who grew up in the era of software-as-a-service have the assumption at the back of their mind that gross margin is all that matters, because if your overhead costs are fixed, then as you grow bigger and bigger, your total margin tends toward your gross margin. And I think that back-of-mind assumption has been transferred to AI hyperscalers. But there are a few problems with doing so – partly in the accounting, and partly in the economics.

Basically, it’s easy to kid yourself that “overhead” and “fixed costs” are synonyms but they’re not. Lots of items don’t get classified as “cost of goods sold” but grow and shrink in line with sales. (And that’s with all actual accounting issues assumed away, which I would definitely not necessarily do in a company where a lot of the sales are bartered cloud credits with connected parties!). In the OpenAI numbers that Ed Zitron has, it is very noticeable that the sales and marketing expense makes the difference between positive and negative operating profit in both 2024 and 2025, and that it’s grown more or less exactly in line with revenues.

But assume all these things away, and let’s say that there is some equilibrium level where all the truly variable costs can be covered and the AI firms are profitable on a variable cost basis. OpenAI was originally set up to be a chatbot with an expensive hobby – the idea was always that the high-margin services would generate free cash flow which could be invested into research. Again, I think the back-of-mind assumption is that there is some steady state where the research budget can be right-sized and held steady, leaving either a residual surplus or, at least, making the thing self-financing.

I worry about whether that’s true. There is definitely some level of “R&D” expense that is actually the equivalent of necessary maintenance expenditure; the models suffer from drift and need to be retrained to keep up with a changing world. As far as I can tell, this can be done relatively cheaply, in principle. But does this expense grow as the models get bigger and more complicated? Does it grow as they are used more for more things? In fact, is the kind of drift and rot that we currently observe the only possible such problem, or will future generations of models develop even weirder and more complicated problems which require further R&D to keep on top of? I don’t know.

But set that aside too, and think about the competitive equilibrium. I think there’s another back-of-mind assumption here that the big AI companies will be able to settle down to a kind of implicit truce, keeping their respective R&D budgets at self-financing levels and not engaging in a destructive arms race to keep on developing and releasing frontier models to steal market share. Which … could happen. But maintaining a cartel is a tricky business, and looking at the personalities involved, I am not sure they are up to it.

We have seen from Claude Code that there’s a very strong tendency for the single best model to gain users, and therefore to set in place a virtuous spiral for themselves and a vicious one for the competition. So I think the incentives are always to be paranoid about slipping behind; unless the big AI companies can somehow find separate niches so that they’re not directly facing each other, there’s a big danger that they face the economics of airlines or reinsurers, where the greatest enemy of capitalism is a big player that didn’t like its market share last year.

Which then brings to mind another issue – how confident are we in the pricing power that underpins that 60% gross margin in the first place? In the last paragraph I was talking about the R&D equivalent of a price war, but the normal kind is also possible. The combination of price-sensitive B2B customers, big fixed costs and rewards going to the dominant player doesn’t suggest to me that pricing power is going to be sustainable indefinitely.

But, I think there’s a danger of missing the big picture here. Which is that, when large companies are telling their employees to be sensible and use AI tokens wisely, then the game is up. The race is over and John Henry won against the steam hammer. If you need a human being in the loop to decide on the allocation of AI tokens, then all those predictions of mass redundancy are gone.

Which throws a different light on the “price war” analysis. Even the best case above – that R&D spending can be kept under control in equilibrium, that all expenses other than COGS scale slower than revenues, that current bottlenecks in data centre capacity can be solved, finance remains available and that depreciation and replacement capex don’t upset anything ... even that case has to start from recognition that where we are right now is a point in which the ability to replace human beings with AI is limited. The story about mass unemployment only even looked like it might have worked when throwing tokens at a problem was cheaper than human beings.

And, of course, this is just for coding – the idea of making material use of AI for general management and governance is several generations of R&D, plus several multiples more token use intensity. It seems to me that we are quite a lot of unknowable technical advances (in model design, renewable energy availability, quite possibly orbital data centres) away from anything like this being possible. And that there is a very difficult business strategy problem of getting there, because the AI companies now have to manage their pricing to walk the tightrope between growth and cash burn.

Maybe it isn’t airlines I should be thinking of; maybe it’s something like nuclear fusion, where there’s a gap between what’s scientifically conceivable and in a sense possible, and what’s economically viable, given the amount of capital investment needed and the long term profitability of the equilibrium.

Does the media shape politics? Can it? Or does it only look that way because the media tries to reflect its audience’s already existing preferences back to it? It’s hard to believe that this was once something that was a matter which people seriously debated, back when we were worrying about the disastrous effect and huge unaccountable power of tabloid newspapers and talk radio. More innocent days.

When the feedback loop is accelerated, the process is obvious; the two options are not mutually exclusive and in fact one of them is an input into another. Most of the time, the media reflects people’s prejudices back toward them, developing a kind of feedback loop that’s somewhat bad in itself. From time to time, this feedback loop can be exploited by someone who wants to manipulate preferences by injecting their own material into the process – then the feedback takes over as people actually seem to like more and more of what they’ve already been shown.

I think what we previously missed was the idea that if preferences are in any way malleable at all, then the simple activity of “just” pandering to existing prejudices wasn’t in any way neutral – it’s actually potentially really bad, as we are seeing now in the algorithmic age. I am still worried by the William Burroughs quote I was talking about a few years ago.

“The junk merchant doesn’t sell his product to the consumer, he sells the consumer to his product. He does not improve and simplify his merchandise. He degrades and simplifies the client”.

And the other lesson of the algorithmic social media age is that industrialisation matters – the speeding up of the cycle doesn’t just clarify things analytically, it actually intensifies the damaging effects. Chris Hayes’ book “The Siren’s Call” is very good on this subject – it goes through a lot of evidence that people’s judgement is made a lot worse by the state of constant low-level cognitive exhaustion created by the attention economy.

I feel like this stuff is all going to have to be regulated some day, in ways that will look like very draconian restrictions on free speech. It’s happened before – the Cancer Act 1939 makes it absolutely illegal to make certain kinds of medical claims to the public in the UK, whether they can be scientifically backed or not.

Subscribe now

The question is: “Would you have built the Sydney Opera House? Show your working”.

As well as the questions I discuss below, I think this is a very interesting on in the field of aesthetics and even moral philosophy. The Opera House was funded by a lottery, and most of the people who bought tickets don’t go to the opera. It’s an absolute paradigm case of highbrow culture funded by taxes on the working class. But it’s very popular! The question here suggests that doing something which is basically unfair can, in principle, be justified retrospectively by artistic merit. Or does it? You tell me.

We are heading into the closing chapters of the book now, where I start talking about solutions and ways to tame the problem factory. There is a lot that can be done, I think, although first I need to talk about some ideas that won’t work or might be counterproductive. We’ve demonstrated our respect for the problem, at length, and we can now stop touching gloves with our adversary and start repeatedly punching it. But before we go into this, I need you to do a bit of a gut check.

In your heart of hearts, deep down, do you think it was a good idea to build the Sydney Opera House?

Looking through old textbooks on planning problems, the Opera House often shows up as an example of a “Great Planning Disaster”. In the book of that name, the theorist Peter Hall devotes a chapter to it, detailing how its costs ended up overrunning by more than ten times. The architect, Jørn Utzon, had never designed anything bigger than a medium-sized housing project. Ove Arup, the engineer responsible for realising his vision, described the cost forecast as having been “prepared, I believe, by some unfortunate Quantity Surveyors under duress in a few hours”.

This is something we might have to sit with, potentially a bit uncomfortably; whatever solution we come up with to all the social and organisational pathologies that stop things from being built, it is unlikely to be something which regularly facilitates things like this. And everyone knew this at the time. The project was started without proper planning, leading to the demolition of millions of dollars worth of wasted work, because as Arup put it “if a fait accompli were not established while Mr Cahill was Premier, the job would probably not go forward at all”.

But a lot of people, including most Sydneysiders, think now, and thought at the time, that it was worth it. There was even substantial support for decades afterward for continuing to finance the huge overrun and the underestimated running costs out of the proceeds of a lottery, meaning that they would largely be paid for by people who didn’t watch much opera.

Conversely, though, when I ask people the same heart-of-hearts question about whether the Concorde project should have gone ahead, they’re much more reluctant to say yes, even though it’s exactly the same sort of white elephant prestige project. And almost nobody will defend NASA’s decision to change their risk tolerance rule from “cancel a launch if there is any doubt about safety” to “proceed with a launch unless there is evidence it is unsafe” ahead of the 1986 Space Shuttle disaster.

I have consciously tried to avoid using the language of “trade-offs” so far in this book. There’s a number of reasons for this. First, it’s not an accurate way to describe most of the problems. Tradeoffs exist when you are at the efficient frontier, when it’s really not possible to get more of one desired thing without giving up another. We are nowhere near that frontier in most of our decision making; we can usually have more of everything we want, simply by spending less on doing stupid things which slow us down and annoy everyone. Removing resources from the problem factory and giving them to the solution factory isn’t a tradeoff, it’s just a gain.

Secondly, I find that the language of tradeoffs is often used in a rather bullying way. If you listen to people who are objecting to something, it’s rare that they don’t understand that there are tradeoffs in policy. They just don’t think it’s worth it. Or they think that the costs are falling disproportionately on them for benefits that go somewhere else. People think that they are sounding wise when they say that “the public want nice things but don’t want to pay for them”. But that’s just what the words “nice things” and “paying” mean. Everyone wants nice things, and nobody wants to pay, they used to teach you this when you did an economics degree.

Here is a real tradeoff, though. We are studying decision making, under conditions of uncertainty. That means that mistakes are inevitable. We can push toward the efficient frontier, to make as few mistakes as possible. But there comes a point at which it is too costly to reduce the error rate any more. Given the context is often that of vitally necessary investments, we cannot even help ourselves to the proverbial wisdom that the only way to avoid mistakes is to do nothing at all, because doing nothing may itself be a big mistake.

So there is going to be a decision to make; will our errors be those of commission, or of omission? It’s not an easy question, and I certainly can’t answer it on behalf of anyone else. I just want to warn you that even if you think you have come up with a solid set of principles to say “yes” to the Sydney Opera House, “no” to Concorde and “definitely not” to the Space Shuttle disaster, circumstances may find a way to make you look a fool, or worse.

The builders and blockers are a slightly different issue, because they’re intrinsically political rather than technical. They either need to be treated as problems to be solved with the same approach as other bottlenecks, or taken seriously as political stakeholders. And asking that question is the same thing as asking “has the decision to build this thing actually been taken yet?” In my view, a lot of the problems on the Abundance Agenda might have a negative answer to this as their root.

So, let’s call this the question of “the point of decision finality” – the moment when the answer to this question changes from no to yes. The thing that’s troubling me is – does this entity, the point of decision finality, does it exist? Is there any such thing? If there is, then it’s clearly an important question of institutional design to say where in the workflow the point should be placed.

But so far I’ve just raised philosophical question after philosophical conundrum for myself – so I hope the philosophers among my readers might be able to help. Clearly, there is a sense in which there is no finality to a decision to build something until the ribbon is cut and the doors opened; projects can be abandoned and often are. Does this mean that there is no such thing as a final decision?

I think the concept can be rescued. Let’s say that the “point of decision finality” is the moment before which not building the thing is correctly described as “we decided not to”, and after which not building the thing is correctly described as “we failed to build the thing”. Now the question is – will this definition support my intuition that the location in the workflow of the point of decision finality is both important and something which is itself part of the design of the system?

My answer would be “not without a bit more substance to it”. Simply changing what you call a project from “failed” to “decided not to” isn’t something that makes a difference in the real world, and it doesn’t meet the challenge. So, what does make that difference; what changes in the world and the organisation when a decision crosses the point of finality.

The answer has to be one of information architecture, because all organisational questions are. After the point of decision finality, there is a qualitative change in the kind of information which the system is allowed to pay attention to. Before some project P reaches decision finality, for example, people are allowed to suggest plans under which the organisation tries to achieve its higher level goals through a set of projects which doesn’t include P. After that point, P has been decided on; the set of alternative plans to be considered is shrunk.

That means that the analytical problem with respect to P is “how can this be achieved?” rather than “should we do this?”. There’s another conundrum here for me, though, because I am not sure that these two questions are sufficiently different – as long as the first question could potentially be answered “it can’t” (or “it can’t, except at excessive cost”), then the second is open. As I said earlier, nothing’s really “decided” until it’s finished.

Which means that there is a constraint on the placing of the point of decision finality. In order for the decision to acceptably approach the ideal of finality, we have to say that to declare a decision final is to assert that you are confident that it can be achieved within the resources allocated to it.

So there’s a limit on how early in the workflow the point of finality can be. I am going to assert that subject to this constraint, the point of decision should come as early in the workflow as possible; keeping decisions open when they ought to be closed seems to create a lot of opportunities for people to engage in strategic behaviour. (It also seems to give advisory bodies like the Planning Inspectorate delusions of being judges; when people feel like a decision is still open, they also seem to believe it’s incumbent on them to maintain the appearance of an open mind).

And that suggests another point of decision architecture. You can’t achieve decision finality until you are reasonably certain that a project can be achieved in an acceptable budget envelope (with “budget envelope” to be taken here as meaning the budget in terms of environmental cost and political goodwill as well as the financial). But of course, you can’t be anywhere near certain of that until you know what the budget actually is. So this suggests two stages of analysis; an initial broad-brush exercise establishing the order of magnitude of the benefits and costs and bringing the tradeoffs into the open, then the decision point, then a detailed planning exercise to see how much surplus can be created, in terms of doing better than the tradeoff which was accepted. Of course, if the second number is negative, then that’s a failure, but I think the idea of “the project has failed” now has more definite organisational and economic meaning.

Sorry, that was a bit dense wasn’t it, readers? I’ll try to come up with a joke by Friday.

Anyway, I was at a seminar this week and, as is my habit, made a silly joke which I now take seriously and think is my actual view. We were addressing the question of “Building Things”, and I proposed the following thought experiment:

What is the largest piece of public sector infrastructure that you could build, right now, if you were restricted to using no more than two contractors, getting permission from no more than two planning bodies and employing no more than two professional services firms?

The idea was to try and get a physical scale and scope for something we were certain we could achieve given the administrative and decision making resources at our disposal. My guess is that the answer might be “something really embarrassingly small, like possibly a bus shelter”. But, infrastructure projects like this have one important advantage over more serious and necessary projects, which is that we can actually do them. There’s only fifteen possible communication channels between six nodes (twenty-one if we imagine there’s a commissioning body that needs to talk to all of them), which ought to be possible for one person to keep track of with an email outbox and a telephone.

So, if we want to get building, let’s not start by being blinded with foolish pride. Declare open season for bus shelters and similar small scale projects. But while we’re building, we need to make sure we’re learning. So let’s also commission a few projects which are a bit bigger than the scale that we know we can manage – we need another thought experiment along the lines of “what’s the smallest thing that you think you couldn’t build if you were restricted to two contractors, two permissioning authorities and two professional services firms?”

Some of those projects will succeed and some will fail, because we’re not going to be able to precisely estimate our own capacity. But we can learn from the ones that fail – this will give us lessons about where the bottlenecks are. And we can learn even more from the ones which don’t exactly fail, but which take much longer than they ought to.

Now we have a, hopefully manageable, “snag list”. This will consist of communication bottlenecks, perceived legal risks and regulatory problems. It will also include some bad faith actors and “blockers” who are opposed to the aim. The first category of problems are the suitable raw material for Malcolm Sparrow’s “problem-oriented governance approach” – you recruit a problem team from all the areas the problem touches, agree on a definition, brainstorm and experiment solutions, then implement and communicate.

The builders and blockers are a slightly different issue, because they’re intrinsically political rather than technical. They either need to be treated as problems to be solved with the same approach as other bottlenecks, or taken seriously as political stakeholders. And asking that question is the same thing as asking “has the decision to build this thing actually been taken yet?” In my view, a lot of the problems on the Abundance Agenda might have a negative answer to this as their root.

And then you iterate; if the problem teams have concluded, communicated and wound up successfully, then the largest object that you can build within your administrative capacity is now a bit bigger. So, build a bunch of those, and then also commission a few more “learning” projects which feel like they’re a bit bigger than you can handle. You’ll be putting high speed trams across the Pennines before you know it.

Readers might notice that I haven’t defined “you” above, which was intentional, because I think this sort of approach can work at different levels from local authorities to departments to central government. The important thing, I think is to start with a realistic and honest assessment of current capacity. As Sun Tsu said, know the enemy and know yourself, and you need not fear the result of a hundred battles.

There is now a second edition available for pre-order, which is greatly revised from the first. The conclusion of the Grenfell Inquiry has made a lot of new information available, but more importantly, has made it much less legally dangerous to be specific about companies and building products which contributed to the disaster. I was lucky enough to blag a pre-publication copy and think anyone who is interested in the general subject matter of “Why don’t we learn?” and “How can we change?” will really enjoy the new edition. If you’re engaged with YIMBY adjacent topics in construction regulation, I would say it’s absolutely required reading.

Anyway, on with the show. I will try not to let the length of this one get out of control, because it’s one of my favourite subjects – the potential for AI to replace middle managers. I am going to do people like Matthew Prince and Jack Dorsey the courtesy of taking them seriously and assuming that they mean “actually replacing middle managers” here, rather than something anodyne like “increasing the productivity of middle managers so that each manager can supervise a larger unit”. Because if all they meant was “I have changed the capital/labour ratio in a company I run” then come on, that’s a bit of a waste of our attention.

Here are a few statements I regard as axiomatic:

1. Actually running a business is more difficult than buying and selling shares on a liquid market.

2. Picking stocks to outperform the market is something which a neural network can do, we know this because human brains do it.

3. However, most human beings, including most human beings who are paid to do so, can’t.

To me, this implies, straight off the bat, that trying to beat the equity market is an at-least AGI level task, and that trying to manage an organisation is potentially super-AGI level. (I’ve made a similar argument in the past, suggesting that a computer which was really capable of handling planning applications would have to be so smart that we would long since have handed over the whole of government to it). This seems to me at least to be either a refutation of the idea that “AI can replace 20% of my managers”, or a massive self-own on the part of people implicitly admitting that their business is so simple, uncompetitive and overstaffed that current generations of Claude can handle it.

[MASSIVE RABBIT HOLE WHICH I WILL POINT TO BUT NOT GO DOWN. Of course, part of the problem is that stock-picking is an adversarial process in which people compete with one another; doubling everyone’s cognitive capacity in the market would just make it more efficient without changing the proportion of people capable of generating outperformance. But so is managing a business, in most sectors.]

Anyway, I have about 500 words left, so I will sketch out why I think that LLMs, specifically, will have a hard time even solving the easier problem of stock picking. I start from the observation that there are a lot of very successful computer systems which beat the stock market using machine learning; they just don’t look very much like large language models.

In fact, quant systems get their edge over a market dominated by human beings precisely because they don’t reproduce human reasoning; they have rules and stick to them. There are two ways to underperform the stock market – by hanging on stubbornly to losers, or by getting the collywobbles and exiting from a good trade too quickly. (The proverb is that “amateurs go broke by taking big losses, professionals go broke by taking small profits”).

So imagine that we have an LLM with appropriate training and a system prompt that says “you are a really good stock picker. Pick stocks that will go up, hold on to them and when they are about to go down, sell them. Don’t make mistakes.” And it’s picked a good stock, that’s gone up a lot, but has now started to go down a bit. Will it get the collywobbles?

Yes! Because “getting the collywobbles in a good trade” is exactly the same activity as “not hanging on to a loser”, except that you’re wrong. Symmetrically, “hanging on stubbornly to a loser” is the same thing as “not getting shaken out of a good trade”, except you’re wrong. Stock market proverbs are massively contradictory; let your winners ride, but pigs get slaughtered. Cut your losses, but in the long term it’s a weighing machine. The trend is your friend, but be contrarian. Etc etc.

The trick is to understand what situation you’re in, and to apply the right one of those rules. And, of course, coping with a high-dimensional environment in which different rules are applicable is one thing that neural nets can handle very well.

It’s just that the neural nets which handle it well are the ones which have been specifically trained to do so, like the actual quant models which people actually use. A neural net which has been pre-trained on a massive corpus of human language by a transformer network is just going to get the collywobbles and double down like a human analyst, because that’s what happens in its training data.

The only way to stop it from doing so, or to bring it up to the standard of the best human analysts, would be to create a great big tagged dataset of good and bad trades along with the reasoning that led to them, so as to get better pre-training weights. And then you’d take on the huge additional task of updating the tagged dataset, so that it didn’t go stale as the market and the economy changed. (Stale knowledge and old habits are a huge problem for human stockpickers). But why would you do that, rather than just directly training the neural net to pick stocks?

And this matters, because when you shift focus back from stockpicking to general management, there’s no equivalent of directly training the model on the data. And it’s not even clear that “the data” has a stable referent – management is very much a game of “horses for courses”, in which the curse of dimensionality is very great indeed.

So you’re left with a model that has to be trained on the corpus of human decisions, and which will therefore be likely to reproduce human mistakes. Unless you can hire an army of offshore workers to tag management decisions as “good” and “bad”, I suppose. But that sounds expensive.

“…concerned not with abundance at all but rather its opposite, bankruptcy. Greece’s Odious Debt, by Jason Manolopoulos, was a polemic published in May 2011, in the very earliest days of the Greek debt crisis. While all three books discussed here so far are attempting to address the question of how we can stop our once-productive social and political institutions from becoming pathological, Manolopolous was forced to ask the same question shortly after the bill became due.

Greece’s Odious Debt contains a thoroughly unsentimental but nonetheless moving history of how the Hellenic Republic got to where it was, and importantly how the combination of financialization, political myopia and the economics of European Monetary Union caused it to spend a decade in a world of false abundance. For a few years, Greek citizens lived well beyond their means, importing Mercedes cars and enjoying a real estate boom. When the crisis happened, they were brought face to face, as few peacetime societies ever are, with two facts: Their system of governance was rotten, and their prosperity was fake.

The economy of Greece has still not recovered; GDP is about 20 percent below its 2008 peak. And its politics have by no means been transformed. So there is no solution here, but there is a lesson. Environmental debts will come due, just like financial ones. So will political and social problems that have been put off to the future because they didn’t appear to have engineering or lawyerly solutions.”

I still think this might be a serious challenge to the Abundance agenda; that achieving the goal of growth and human flourishing might occasionally involve taking a step back, and that we might be at one of those points right now. It doesn’t mean that the whole project is discredited - far from it, it implies that solving the problems which make it too difficult to construct renewable energy infrastructure is really very urgent.

But it does mean that we can’t brush off “degrowth” as easily as we might want to. Carbon dioxide emissions are very like debt; they provide a current benefit in exchange for a future liability, and the future liability might be significantly bigger than the initial benefit. It’s just that it’s a form of future obligation which doesn’t appear in the same balance sheet as financial debt.

I don’t think this is a difficult accounting problem; it’s tricky to quantify the environmental debt and to translate it into a precise number of pounds or dollars. But it’s surely not difficult to understand that it exists, or that it’s likely to be big - we’re already thinking about abandoning entire towns, for heaven’s sake. And yet, people say that Net Zero is unaffordable, who would never say the same thing about the 4.75% October 2035 Gilt.

When a country has an excessive debt burden of either kind, bad things are likely to happen, and the only solution is either a bailout or a period of austerity. I had, and have, many disagreements with Yanis Varoufakis and with Wolfgang Schaeuble, but they did, at least, start from recognising the facts. Who is going to tell us that our prosperity is fake?

And more worryingly, what is going to happen to them when they do? One of the key motivating factors behind the Abundance Agenda is that the politics of scarcity are incredibly bad right now. The main reason why we need policies capable of delivering economic growth is that without it, a lot of voters seem to develop a really unattractive and dangerous mean streak. I still think it’s possible to design some glide path which will maintain developed world consumption levels while financing investment and writing off a large part of the last twenty years’ malinvestment. Just don’t ask me to write it down right now.

Subscribe now

Lots of people are very much “gahh, enshittification, I just want my weblinks”, but I’m not - I do think this is potentially very useful. Being able to carry out a semantic search - something like “find me three examples of trade associations making outlandish claims about the economic impact of housing regulation” is a real time saver, and although you do have to do a lot of back-and-forth with the AI to get what you want, it’s less of a pain in the arse than filling your screen with browser tabs and reading the press releases yourself.

However, the question I want to ask is - if we put that kind of issue to one side and assume for the sake of argument that a really useful product will come out of this, how would we expect to see that reflected in the economic statistics?

I’m not sure that we would at all. I like using the intelligent search box, but that’s just unmeasured consumer surplus to me. I don’t like it so much that I’d pay for it, and Google know that if they were to charge for search, they would lose market share instantly.

It also doesn’t seem likely to me that Google could monetise that consumer surplus by raising its ad rates. It’s a monopoly already; it’s squeezing as much out of the advertisers as they have to give.

Google might be able to serve me more ads, if I spend more time in the intelligent search box and less time on other sites. But this is close to a zero sum game, in that those other sites will have fewer opportunities to sell me ads.

So, at best, the increment to Google’s revenue (and therefore, the only thing that has any chance at all of going into GDP) is the extent to which time spent on a Google site with ads is substituted for time spent on other sites which didn’t previously have ads. It feels pretty marginal, particularly since there’s a huge amount of capex needed to achieve it.

I think what’s going on here is that we’re experiencing a less than perfectly foreseen consequence of the big trend towards corporations trying to create “moats”. When a monopolist is earning monopoly profits, then they will invest to protect that monopoly, even if the investment is not one that would be justifiable as a stand-alone. If you are earning profits of 100, and you are faced with the need to make a big capital investment in order to ensure that those profits only drop to 90 rather than to 50, then you grit your teeth and make it.

If a lot of the capex going into AI is going into this kind of investment - negative sum games between monopolists trying to protect their rents - then we shouldn’t expect to see the big consequences that everyone’s been predicting. We’ll just get some more unmeasured consumer surplus, a few marginal players will go bust and the moat will keep getting filled.

This is a weird industry that almost no other country has in anything like the same size, made up of small consultancies and guys called Brian[1]. They have a great deal of specialised expertise and they sell it by the hour on a global market. This is about a fifth of Britain’s entire exports; it’s twice as big as financial services.

One point that I’ve been making since I started writing about it is that the Old Rectory Mittelstand is actually a lot more similar to its German namesake than you’d think. The British economy and the Germany economy are really in the business of selling knowledge and technical capability. It’s just that one set of companies sell the pure product straight from the teat, while the others sell a product bundle which includes a big steel box with a motor and some microprocessors in it.

This line of thinking comes from Peter Drucker, who famously said that the only two vital functions of a company were innovation and marketing. Coming up with new products and selling them are the only activities that directly lead to value creation; everything else is a cost. I’m not sure I agree with that, but it’s interesting to think about. In any case, you can see that although advising on how to bodge widgets is a service while making an automatic widget-bodger is manufacturing, they share a certain core of important things; it’s a matter of the boundary of the firm rather than something intrinsic to widget bodging as an activity.

What’s on my mind, though, is that the British version has some quite difficult political economy. At the level of society, the German model of bundling your high-value-added product with a bunch of steel and copper has the advantage that it gives something for people to do if they aren’t able to generate valuable quantities of knowhow and expertise themselves.

If you reorganise your highest value-added industries in such a way that the majority of the population can’t participate in them, this is already the sort of thing that has the seeds of future conflict. But there is also a transition problem from one state to the other.

Because the people who actually build the valuable things are psychologically unlikely to agree with Peter Drucker that they are a cost centre. They pick up a bunch of commodity components and put down a million dollar WidgetBodgeMatic; how is that not creating value? And the factory floor employees have a lot of skill of their own, which they are rightly proud of; it’s just not the right kind of skill to be productised.

So perhaps part of our national malaise is that we’ve got a macro-scale version of the problem that the Brompton factory had when it moved from piece-work craft assembly to a production line. People who previously considered themselves to be the heart and soul of the enterprise, with some justification for doing so, came face to face with one of the most intolerable things a human mind can consider – that they weren’t as important as they thought they were.

This actually holds out at least as much of a warning for Germany as for the UK though.Although the Brian of Britain model has this political economy problem, the alternative model has a business economy problem – it’s not so flexible and depends a lot on specialised capital equipment that can’t be repurposed easily.If you’re going through an economic or technological transition, particularly one which leads to the obsolescence of a previously very important class of metal boxes with motors in them, then you get all the same problems of political adjustment, plus a load of economic problems too.

[1] Or, indeed, if the particular field of expertise is newly published European financial regulation, called Dan.

Among other things, one of the key subjects of my current book is the concept of “career risk”. It’s one of those things that everyone instinctively knows what it is, but which doesn’t seem to have any rigorous theory of it. So the only thing to do is what I always do, which is to theorise it myself, at which point Sodd’s Law begins to operate and I suddenly discover all the famous people who have been writing about it since 1947.

I think career risk does explain a lot, both in terms of there being a lot of things in the modern world which happen because of it, and in terms of it being a very useful generalisable concept. Henry and Cosma talk about “coarse-graining” as an essential feature of bureaucracy, and below, I argue that one inevitable consequence of coarse-graining in organisations is that it provides the ground for career risk (and therefore, for certain kinds of friction and entropy). Obviously, for the same reason I was thinking about accounting systems while everyone in Santa Fe was talking about algorithms and hierarchies, I use the term “black box” instead of “coarse grain”, but we’re all looking at different parts of the same elephant.

Subscribe now

Black box actors are the condition for the existence of career risk

It seems, then, that our theory of career risk also ought to be built on the understanding that it is a problem of expertise. It arises when there are people who make decisions because they have a special role in the information processing system. This special role is that of being a “black box”.

I’m using the term “black box” here in a bit of a specialised way. There’s a bunch of theory in the background, for which I’m sorry. I learned it from Stafford Beer’s books on management cybernetics, as a way of modelling complex systems. When you’re doing that, there is an obvious need to reduce that complexity to a sensible level in order to explain what is going on. One way to reduce complexity is to take some part of the system, draw a box around it and then shade that box in to be completely black[1], visually declaring that you are no longer going to worry about the internal structure of that part of the system, but rather just treat it as something with inputs and outputs. The information going in can be seen, as can the decisions going out, but the process by which one is converted into the other is opaque. Not necessarily because it absolutely has to be mysterious, but rather because doing it any other way would be unrealistically expensive in terms of time and effort.

And although it’s best understood as an analytical technique for thinking and writing about organisations, the principle of the black box rings fairly at the level of ground truth as well. It’s a large part of what it means to delegate responsibility for something – to do so without also agreeing to treat the person or group that you’ve delegated to as a black box is the sin of “micromanagement[2]”. And there are people with particular decision making qualifications or legal certification – like social workers, or building inspectors – who are intrinsically black-boxed. For these kinds of workers, the nature of their position gives them the authority not to be second-guessed.

I worry that I might just have made it sound quite fun to be a black box. To some extent, it is. It’s one of the nice things about being a professional, that your judgement and opinions are respected in the way that those of hourly employees often aren’t. But there’s always a double edge. The same black-box property which stops you from being second guessed or overruled means that nobody is interested in your explanations for your decisions; it is definitional of being a black box that you are going to be judged by results[3].

And here we reach the payoff of our project of theorising career risk as a problem of information. If you are going to be judged by the results, and the results are uncertain (as they obviously are, in this naughty world we live in), then you are personally exposed to the risk. Furthermore, your risk exposure is exactly like that of the fund manager we were talking about at the beginning of this chapter; all your eggs are in one basket. This might be one decision among hundreds for your organisation, but it’s a much bigger part of your personal portfolio.

[1] I have actually done this once or twice, on a whiteboard when drawing management structure diagrams. It was unbelievably satisfying.

[2] In the jargon, there are also “muddy boxes”, which are treated as black most of the time, but which can be periodically opened up and examined when there is need to do so.

[3] If you’ll forgive even more jargon, the same management cybernetics books call this the problem of the “resource bargain”. Gaining the status of a black box is a kind of bargain, in which you gain autonomy in exchange for accepting accountability.