corey and the robots
skim reading artificial intelligence
Sorry, it seems that we’re in summer holidays now, so this post is a day late for Wednesday. Or a day early for Friday I guess. This week, Corey Robin, a former co-blogger of mine, became the latest to discover that Large Language Models can actually produce output that really seems not bad, if you know what you’re doing and put a bit of effort into tweaking them.
Which is, as the rather thoughtful and interesting piece linked above shows, extremely inconvenient if you’ve got a method of teaching that’s dependent on getting people to understand a subject by writing and redrafting essays based on feedback. Corey’s independently discovered one of the key principles of variety amplification; that recognising whether a piece of work is dud or not is a really cheap cognitive operation compared to creating something yourself.
Consequently, you can go through a dozen iterations quite quickly, tuning and improving your prompts every time, and end up with something that looks like a more than passable answer to even a quite obscure essay question. And the ability to rate whether answers look better or worse requires very little in the way of domain-specific knowledge about the subject or even writing skill – as Corey says, the reasonable-looking assumption that “if a student knows enough about paper-writing to make ChatGPT work for them … without detection by a minimally alert instructor, that student has probably already mastered the skills of essay-writing” isn’t true.
To an extent, we shouldn’t be surprised by this; in many ways, the fact that it’s much easier to judge others than to create something yourself is the basis of democracy itself. We organise trials on the basis that people who don’t know anything in particular are nonetheless able to say which expert explanation of a set of facts is more convincing. So although it’s surprising to see that Corey’s daughter was able to tune up a ChatGPT version to produce an essay (and presumably would have found it even easier to do so if he’d been giving her detailed notes), it maybe shouldn’t be.
But there’s an offhand remark in Corey’s essay which really set me thinking:
Good work was never about writing good papers. It was about being able to order your world, to take the confusion that one is confronted with, and turn it into something meaningful and coherent. And to know that that doesn’t just happen spontaneously or instinctively; it’s a practice, requiring, well, work.
That’s not simply a skill for college classes. That’s a life-long practice, of being able to see a situation, pick out those elements that matter and lend it significance, and bring clarity out of chaos.
That’s critical to being a good friend, a good parent, a good citizen, a good neighbor, and having a good life. I really, firmly believe that. I wouldn’t spend as much time as I do on student papers if I didn’t.
And … maybe it’s not?
It certainly feels like the ability to synthesise and see the key underlying structure in a mass of confusing text, then re-express the ideas clearly is an essential skill. It feels like part of the definition of what it is to be intelligent. But what if it’s not? Once upon a time, adding up columns of figures was an intellectual achievement, or multiplying and dividing large numbers.
Or what if there is some essential skill of the sort Corey describes, but it stands in the same relation to essay writing that mathematics does to the pocket calculator? Once upon a time, being able to do long multiplication was an important skill for mathematics – now that we have machines to do it for us, we realise that there are deeper and more abstract levels of reasoning which really matter. Perhaps the humanities classes need to reinvent the equivalent of “with-calculator” and “no-calculator” tests.
I’m not sure what I believe here. Notoriously, my own degree subject was famous for mainly training people to digest a load of material quickly, form snap judgements and explain them in confident and articulate terms. That’s why we used to dominate politics, banking and the civil service. I will still stand up for that as an important ability, and we can have the argument another day about how generalist managers ought to interact with specialists and scientists. But what if it’s just the equivalent of adding up a long column of figures? How do we reorganise society and management in a world where skimming for content is something everyone just does with their phone?
I think Corey is right here. Also I think his argument is more general than yours: it's not just being able to find the core of an argument in a text, it's the ability to find the core element of what's going on in the world around us.
I am not an actual AI guy, at all, but I have digested some work on the subject, including some of the modern arguments, as well as quite a bit from pioneer Marvin Minsky. (Some of Minsky's later classes at MIT are viewable on youtube, which is amazing as fuck.) Minsky said that one of the core difficulties of AI is taking an undifferentiated input and extracting the elements that matter for the task at hand -- in other words, what Corey said.
Notice what Corey says about psychoanalysis, at the end of that piece. The question is how do we order our own thinking vis-a-vis the world. I don't see how one could possibly summarize this as "skimming for content" ... or as something anyone could ever hope to do with their phone.
Are you sure that you're picking up what Corey is putting down?
This reminds me of similar arguments about spreadsheets. Prior to spreadsheets some boss would ask a questions and a week later get back an answer. They might ask about a different value for one of the variables and get another week. With spreadsheets, they could get the results in seconds. Often they would try a dozen alternatives because it took so little time to see the results. (e.g. What is we reduced our error rate by 5%)
Of course this only worked for certain kinds of problems, which now seemed trivial.