a master at work
our message, their massage
I promised this earlier in the week, and it’s mainly a fun post; maybe there will be something heavier on Friday. Over the bank holiday, my wife was bemoaning the extent to which her Vinted (international readers – it’s a British second-hand and vintage clothes website) favourites list had got out of hand, and sent my oldest daughter a message asking what she should do. She got this reply:
At first glance, this looks like ChatGPT output, and indeed, our first reaction was to assume that our beloved child was exploring new frontiers in online rudeness. However, it turned out that this is something a bit more special. Poppy likes to tease her parents, and she knows our views on the use of chatbots, so she in fact responded with a hand-written piece of pastiche ChatGPT output.
I was impressed. After getting her permission to reproduce it, I asked her to describe the thought process; this is my reconstruction of the short interview we had over a glass of local cider.
Whoa! That’s a lot! It’s good you’re being critical about what to buy
“I wrote the first sentence and thought God, that looks like Chat GPT. So I thought let’s see if I can really do it. I started by praising, because it always praises.”
I think we need to FaceTime about this because the job needs a detailed conversation, there’s definitely things that need to be improved to achieve the look you want
“The FaceTime thing is a bit off, but that’s what I wanted to actually do. I thought I’d slightly misunderstand the request, it often does that. Went straight into passive voice.”
Not a text chat
Not a five minute convo
But a real deep dive FaceTime
“I was pleased with this, the rhythm is just right, which distracts you from noticing that the FaceTime offer isn’t. These lists are better than doing “not this but that”.”
Overall it’s a good start, but we’re not there yet.
“It repeats itself a lot and puts in way more useless words than a real text would”
If you need my help condensing this list down to ten or so items, I can help
“Just repeating Mum’s text back to her”
The good news is, you’re already half way there by chucking out the ones you don’t like
“I realised I hadn’t given any supportive messages for a couple of sentences so it had to be done – again, the language is completely off here, but you don’t notice that because it’s doing the thing Chat always does.”
If you need my help with this, I am ready to FaceTime when you are xx
“The more you repeat things the more it looks right, even if I slipped up and added the x’s”.
My daughter is, as you can see, an extremely clever and amusing young woman. She has also always been a natural mimic; she’s gone to a couple of different schools from London to Devon to New Zealand, and it never takes her more than a week or so to have a perfect local accent and master the particular slang. I think she has intuitively extracted a number of elements of the modern chatbot style. I find myself wondering, though, where have they been extracted from?
The OpenAI explanation of why their Codex tool has to be restrained from talking about goblins gives us a clue. The chatbot style isn’t a result of the transformer neural network picking up latent features of the entire training set; it’s a feature set at a much higher level, in the “system prompts” and the optimisation of the actual product in testing and manual tweaking, rather than anything happening at the level of vector spaces.
In other words, the chatbot has been developed in order to chat, so it’s been forced to learn the rules of “sounding chatty”. There are lots of areas of human communication where the content is less important than the mastery and reproduction of a small number of stylistic tropes – I can highly recommend the book “Our Masters’ Voices” by Max Atkinson, which sets out the features and cliches of political speeches. (It was published to accompany a television series in which half a dozen absolute neophytes were trained in these techniques, going from never having spoken to a large audience to getting ovations at party conferences).
This suggests to me that at some point in the future, when we decide to take the problems of AI psychosis seriously, it is not going to be that difficult to regulate them. The chatbot seems convincing and persuasive because it’s been trained to use a small number of techniques which have been known for ages to have this effect on unguarded humans. (As well as political speeches, after all, there are books on selling over the telephone and pick-up artistry). We just need to say to anyone making an AI for public interaction that they aren’t allowed to do this.
(By the way, Poppy’s card for internships this summer is full, but she will be studying abroad next year and then coming back for the final year of her sociology and management course. If anyone has any interesting opportunities and thinks they might find this sort of capability useful, you can always get in touch with me by replying to one of these emails!).
Not a text chat
Not a five minute convo
But a real deep dive over an actual beverage
"The OpenAI explanation of why their Codex tool has to be restrained from talking about goblins gives us a clue. The chatbot style isn’t a result of the transformer neural network picking up latent features of the entire training set; it’s a feature set at a much higher level, in the “system prompts” and the optimisation of the actual product in testing and manual tweaking, rather than anything happening at the level of vector spaces."
This is wrong, I'm afraid, there's a very large human-feedback reinforcement component in all these models where annotators have labelled massive response corpuses as racist, helpful, cheerful, pornographic, difficult-to-understand etc. etc. and that's incorporated directly into the training process via e.g. https://cameronrwolfe.substack.com/p/ppo-llm. Much of the "chatbot style" is actually baked into the network weights.