In the world of engineering materials, there's a thing called "low-background steel", which is steel that was made before widespread testing of nuclear explosions lofted radioactive material into the atmosphere, contaminating everything made after 1945 with trace amounts of it. Low-background steel, scavenged from pre-war shipwrecks, is important when building shielding or enclosures for sensitive scientific instruments that need to detect trace amounts of radiation from an experiment, and not from the environment.
I feel like the AI boom is going to have a similar effect, there will be "low-LLM" cultural artifacts that we can be sure are not products of AI -- books and articles from before the 2020s, and as described here, uncontaminated languages like Maltese and Irish and Welsh and Icelandic.
Alas, India has now fallen prey to the University of Whatsapp in terms of misinformation, so I wouldn't bank on it...
The thing about tea, jam, and sugar is an interesting point on both ends--the average British person's diet got less healthy, if more addictive, while the crucial ingredients were supplied to Britain by slave labor, indentured servitude, and colonial exploitation.
I was listening to this most recent Odd Lots with Paul Kedrosky (https://www.bloomberg.com/news/audio/2025-11-14/odd-lots-ai-is-like-every-bubble-all-rolled-into-one-podcast?srnd=phx-oddlots) and he said something interesting midway through about AI and what he called "intuition." I don't have the transcript in front of me, but the essence of his point was that once you hand something off to an AI model for parts of your job, you are necessarily giving up your intuition about that process. In his example, he discussed a NLP model that his fund uses to analyze language in earnings calls to predict changes in stock price; by using the model, they are necessarily giving up intuitions that might be gained if they actually went through the effort of listening for patterns in the earnings calls themselves.
This seems right, but the more I think about it, it's also a profoundly different prediction than the "augmentation" framework that people (mostly on LinkedIn, it seems) throw around about white-collar work. The idea there being that you can use AI to automate all the "rote" parts of your job and spend more time on the interesting bits. Even though he isn't commenting on this idea directly, if you pull the thread on Kedrosky's argument I think we would find these fundamentally opposed. By industrializing the rote work you are necessarily industrializing the "interesting" work too.
If the "craft" of knowledge work will disappear in many high-status professions, it makes me worry about a dark political turn along the lines of the Luddite rebellion, but much bigger.
Exactly. I hate writing "health of the business" documents as I consider myself a physicist or maybe engineer. But if I don't take time to write them myself I will never stop and reflect on what I am doing on a day to day basis.
"Stop and reflect" is what gets chainsawed out of the work environment when an obsession with cutting costs or any other single measure rules. But it is literally what keeps your view whole, and sane. How will you ever be effective if you never spend time wandering around making discoveries, or remembering things you had forgotten about? Your mental model of reality will become more and more detached and inaccurate; out of touch. A more subtle risk than flattery, but equally important. And pernicious because it happens slowly. Then one day, trouble. We've all seen what happens when the boss never leaves his office, or his comfort zone...and for some people, chaos is their comfort zone!
Learned and melodious Gaelic to the rescue! According to Flann O'Brien, his "An Beal Bocht" was the only book ever written in Gaelic to require a second printing and it's only about 100 pages long.
I’m going to take you up on footnote [1] and claim credit for the “surrounded by people who agree with you” effect — I wrote something earlier this year called The Yes Protocol, which looks at how professional environments already train people into obedience-by-agreement.
The point isn’t that AI creates that pattern, but that it industrialises it. Most workplaces already reward smoothness over challenge; RLHF just bakes that same bias into the model’s reward function. What was a cultural habit becomes a structural behaviour.
That’s why your “transition window” point hits so hard — we’re in a phase where AI isn’t strong enough to replace judgement, but it’s strong enough to degrade the feedback loops that protect it.
If anyone’s interested, here’s the piece: [The Yes Protocol: How Artificial Certainty Turns Human Bias into Systemic Law]
"According to a fragment first published in 1959 by Helmut Sembdner, Heinrich von Kleist proposed dividing humanity into two classes: “(1) those who are adept at metaphor, and (2) those who are adept at formula.” Those capable of both were too few to constitute a class to Kleist."
Maybe I'm kidding myself, but I find ChatGPT's default level of sycophancy to be so absurd that it's easy to discount. Looking at this a bit further, I found that I can prompt it to generate a range of personas from starry-eyed fan (even more enthusaistic than the default). to bored butler.
Porridge and beer? Never would have thought of GCSE, wow!
Personally, I was leaning toward the largely-forgotten "Ninety Nine Bottles of Beer" theory, but I am not a subject-matter expert on beer, so here is my only reference:
Memory is fickle, so I checked with legal (aka "the flunkies") and they came up with some additional plausible "origin stories," should they prove necessary. Herewith:
"Your honor: I am pretty sure about this, although there is a sly...sorry, "slight," chance an intern might have come up with it via his new bestie, "Addled Intelligence."
You could say it also might have come from:
1. A fortune cookie
2. Your brother-in-law
3. A dream
4. Google
5. A rumor in a chat room
6. You were in a Barnes and Noble in the 'Astrology" section and a book suddenly fell off the shelf, and somehow opened to that very page!
7. A bookie named named "Liar"
8. A college textbook you unfortunately threw out when you moved into the condo
9. Grammarly
10. The Onion
Because, who knows, right?
Or:
"Has Your Honor heard the one about the 'multiple universes' theory? Where steel could be like rubber, and rubber could be like steel? And past and future are one and the same? Kinda makes ya wonder, doesn't it?"
(Legal disclaimer: any representations made, or unmade, or will at any time in the future be made, or unmade, re this matter, or others, by any party whatsoever, before this jurisdiction, or elsewhere, whether by the first party, or any agents appointed on behalf of the first party, or any other party, or delegated to them, or known to them, or not, whether in English, Latin, or any other language, known or unknown, heretofore or not, ipso facto, de minimis, jargon jargon jargon...whereby conclusions might, or might not, be, reached, or not...all aforesaid representations, made or unmade, etc., may be denied, in perpetuity.)
This post is also a strong argument for Indigenous language preservation: not just the "famous" ones like Navajo, but ones at the lower end of the population scale, like Yolŋu Matha in the NT.
Accelerated evolution. People who habitually use AI as debating partners to strengthen their own thinking and provide new perspectives to keep their minds open will prosper. Those who use it to be validated and confirmed in their opinions will ... not. Self-administered Darwin Award in a few years.
To me the parallel is the introduction of cheap strong alcohol to naive populations. It caused much devastation, which is still going on in many cases.
>It’s basically a thought arising from the really clear and increasing danger of people cooking their brains with AI.<
I don't see this as a unique feature of AI. It has been happening for as long as populations have become larger, and freedom to express views has become safer. It may have started with different tribal beliefs, then different religions. Then, the creation of small interest groups. Then, books that allowed groups of people to read select books supporting their beliefs. Partisan politics and associated political parties. Social media made this even easier, with human feedback in all cases strengthening the feedback, and in some cases, pushing the "weirdness" ever more towards the extremes. AI just cuts out the need to find "your group of like-minded individuals" and democratizes the feedback to groups of one, you and your AI. As wealthy and powerful people have found, sycophancy and "yes men/women" become something for some to guard against (e.g., JFK after the Cuba standoff) and narcissists to encourage (e.g., Trump).
If "cooking their brains" includes overloading brains with bad memes, exacerbated by the stress of modern life, then I suggest that AI is just the latest means to do this, as are the examples above through history. As for "frying their brains", alcohol and drug addiction have a long history, which is problematic for certain individuals easily addicted.
AIs could even be helpful. Does sycophancy reduce the need to demand that other people agree with you? Do AI "girlfriends" reduce the need for incel violence? [It has been claimed that violent imagery in Japanese manga and violent computer games reduces the need to act out in the real world, as opposed to claims that these worsen real-world violence.) IDK which claim is correct, I suspect it could go either way depending on the circumstances.
> you would probably also be backing an Indian future, as it’s a large country where the main everyday language isn’t the business language of the IT professionals and which has the additional protection of Unicode being annoying to work with.
I always thought Unicode was uniquely bad in Japan (due to Han unification), is there something similar going on in India?
In the world of engineering materials, there's a thing called "low-background steel", which is steel that was made before widespread testing of nuclear explosions lofted radioactive material into the atmosphere, contaminating everything made after 1945 with trace amounts of it. Low-background steel, scavenged from pre-war shipwrecks, is important when building shielding or enclosures for sensitive scientific instruments that need to detect trace amounts of radiation from an experiment, and not from the environment.
I feel like the AI boom is going to have a similar effect, there will be "low-LLM" cultural artifacts that we can be sure are not products of AI -- books and articles from before the 2020s, and as described here, uncontaminated languages like Maltese and Irish and Welsh and Icelandic.
Someone is already doing this, and using the same low-background steel metaphor, to boot: https://arstechnica.com/ai/2025/06/why-one-man-is-archiving-human-made-content-from-before-the-ai-explosion/
Alas, India has now fallen prey to the University of Whatsapp in terms of misinformation, so I wouldn't bank on it...
The thing about tea, jam, and sugar is an interesting point on both ends--the average British person's diet got less healthy, if more addictive, while the crucial ingredients were supplied to Britain by slave labor, indentured servitude, and colonial exploitation.
I was listening to this most recent Odd Lots with Paul Kedrosky (https://www.bloomberg.com/news/audio/2025-11-14/odd-lots-ai-is-like-every-bubble-all-rolled-into-one-podcast?srnd=phx-oddlots) and he said something interesting midway through about AI and what he called "intuition." I don't have the transcript in front of me, but the essence of his point was that once you hand something off to an AI model for parts of your job, you are necessarily giving up your intuition about that process. In his example, he discussed a NLP model that his fund uses to analyze language in earnings calls to predict changes in stock price; by using the model, they are necessarily giving up intuitions that might be gained if they actually went through the effort of listening for patterns in the earnings calls themselves.
This seems right, but the more I think about it, it's also a profoundly different prediction than the "augmentation" framework that people (mostly on LinkedIn, it seems) throw around about white-collar work. The idea there being that you can use AI to automate all the "rote" parts of your job and spend more time on the interesting bits. Even though he isn't commenting on this idea directly, if you pull the thread on Kedrosky's argument I think we would find these fundamentally opposed. By industrializing the rote work you are necessarily industrializing the "interesting" work too.
If the "craft" of knowledge work will disappear in many high-status professions, it makes me worry about a dark political turn along the lines of the Luddite rebellion, but much bigger.
Exactly. I hate writing "health of the business" documents as I consider myself a physicist or maybe engineer. But if I don't take time to write them myself I will never stop and reflect on what I am doing on a day to day basis.
"Stop and reflect" is what gets chainsawed out of the work environment when an obsession with cutting costs or any other single measure rules. But it is literally what keeps your view whole, and sane. How will you ever be effective if you never spend time wandering around making discoveries, or remembering things you had forgotten about? Your mental model of reality will become more and more detached and inaccurate; out of touch. A more subtle risk than flattery, but equally important. And pernicious because it happens slowly. Then one day, trouble. We've all seen what happens when the boss never leaves his office, or his comfort zone...and for some people, chaos is their comfort zone!
Learned and melodious Gaelic to the rescue! According to Flann O'Brien, his "An Beal Bocht" was the only book ever written in Gaelic to require a second printing and it's only about 100 pages long.
I’m going to take you up on footnote [1] and claim credit for the “surrounded by people who agree with you” effect — I wrote something earlier this year called The Yes Protocol, which looks at how professional environments already train people into obedience-by-agreement.
The point isn’t that AI creates that pattern, but that it industrialises it. Most workplaces already reward smoothness over challenge; RLHF just bakes that same bias into the model’s reward function. What was a cultural habit becomes a structural behaviour.
That’s why your “transition window” point hits so hard — we’re in a phase where AI isn’t strong enough to replace judgement, but it’s strong enough to degrade the feedback loops that protect it.
If anyone’s interested, here’s the piece: [The Yes Protocol: How Artificial Certainty Turns Human Bias into Systemic Law]
https://open.substack.com/pub/jimgrafton/p/the-yes-protocol-how-artificial-certainty?utm_source=share&utm_medium=android&r=5pz184
Is Finnish safe?
Good shout on Finnish, that probably hasn't produced enough tokens to train a model even if you include every day conversation
I know some of the people from CSC who built the "Finnish LLM" so unfortunately this is not true.
Yes, but Gibberish is not.
I saw this on twitter recently.
"According to a fragment first published in 1959 by Helmut Sembdner, Heinrich von Kleist proposed dividing humanity into two classes: “(1) those who are adept at metaphor, and (2) those who are adept at formula.” Those capable of both were too few to constitute a class to Kleist."
Ben Recht has a book coming out next year which I think will deal with the idea of "math becomes metaphor" and it is going to be fantastic
Around the time I was writing Models.Behaving.Badly I came across this by Yuri Manin: Mathematics as Metaphor: Selected Essays of Yuri I. Manin
Originally saw the intro by Dyson
https://bookstore.ams.org/cworks-20/
Maybe I'm kidding myself, but I find ChatGPT's default level of sycophancy to be so absurd that it's easy to discount. Looking at this a bit further, I found that I can prompt it to generate a range of personas from starry-eyed fan (even more enthusaistic than the default). to bored butler.
Is there a handy reference on the porridge + small beer story? I can’t place where I heard it before.
To be honest I think I remember it from GCSE history so it might be very distorted!
Porridge and beer? Never would have thought of GCSE, wow!
Personally, I was leaning toward the largely-forgotten "Ninety Nine Bottles of Beer" theory, but I am not a subject-matter expert on beer, so here is my only reference:
https://en.wikipedia.org/wiki/99_Bottles_of_Beer
Memory is fickle, so I checked with legal (aka "the flunkies") and they came up with some additional plausible "origin stories," should they prove necessary. Herewith:
"Your honor: I am pretty sure about this, although there is a sly...sorry, "slight," chance an intern might have come up with it via his new bestie, "Addled Intelligence."
You could say it also might have come from:
1. A fortune cookie
2. Your brother-in-law
3. A dream
4. Google
5. A rumor in a chat room
6. You were in a Barnes and Noble in the 'Astrology" section and a book suddenly fell off the shelf, and somehow opened to that very page!
7. A bookie named named "Liar"
8. A college textbook you unfortunately threw out when you moved into the condo
9. Grammarly
10. The Onion
Because, who knows, right?
Or:
"Has Your Honor heard the one about the 'multiple universes' theory? Where steel could be like rubber, and rubber could be like steel? And past and future are one and the same? Kinda makes ya wonder, doesn't it?"
(Legal disclaimer: any representations made, or unmade, or will at any time in the future be made, or unmade, re this matter, or others, by any party whatsoever, before this jurisdiction, or elsewhere, whether by the first party, or any agents appointed on behalf of the first party, or any other party, or delegated to them, or known to them, or not, whether in English, Latin, or any other language, known or unknown, heretofore or not, ipso facto, de minimis, jargon jargon jargon...whereby conclusions might, or might not, be, reached, or not...all aforesaid representations, made or unmade, etc., may be denied, in perpetuity.)
See, I've got you covered.
This post is also a strong argument for Indigenous language preservation: not just the "famous" ones like Navajo, but ones at the lower end of the population scale, like Yolŋu Matha in the NT.
Accelerated evolution. People who habitually use AI as debating partners to strengthen their own thinking and provide new perspectives to keep their minds open will prosper. Those who use it to be validated and confirmed in their opinions will ... not. Self-administered Darwin Award in a few years.
To me the parallel is the introduction of cheap strong alcohol to naive populations. It caused much devastation, which is still going on in many cases.
>It’s basically a thought arising from the really clear and increasing danger of people cooking their brains with AI.<
I don't see this as a unique feature of AI. It has been happening for as long as populations have become larger, and freedom to express views has become safer. It may have started with different tribal beliefs, then different religions. Then, the creation of small interest groups. Then, books that allowed groups of people to read select books supporting their beliefs. Partisan politics and associated political parties. Social media made this even easier, with human feedback in all cases strengthening the feedback, and in some cases, pushing the "weirdness" ever more towards the extremes. AI just cuts out the need to find "your group of like-minded individuals" and democratizes the feedback to groups of one, you and your AI. As wealthy and powerful people have found, sycophancy and "yes men/women" become something for some to guard against (e.g., JFK after the Cuba standoff) and narcissists to encourage (e.g., Trump).
If "cooking their brains" includes overloading brains with bad memes, exacerbated by the stress of modern life, then I suggest that AI is just the latest means to do this, as are the examples above through history. As for "frying their brains", alcohol and drug addiction have a long history, which is problematic for certain individuals easily addicted.
AIs could even be helpful. Does sycophancy reduce the need to demand that other people agree with you? Do AI "girlfriends" reduce the need for incel violence? [It has been claimed that violent imagery in Japanese manga and violent computer games reduces the need to act out in the real world, as opposed to claims that these worsen real-world violence.) IDK which claim is correct, I suspect it could go either way depending on the circumstances.
> you would probably also be backing an Indian future, as it’s a large country where the main everyday language isn’t the business language of the IT professionals and which has the additional protection of Unicode being annoying to work with.
I always thought Unicode was uniquely bad in Japan (due to Han unification), is there something similar going on in India?