faking versus making
recalling a twenty year old experiment
Something came to mind while reading this excellent essay by Tom Rachman, passed on by Conor Griffin. It’s not directly relevant to Tom’s specific subject, the role of human writing in the age of AI, but I think it’s in an area that sort of rhymes, and which is very important to me because it’s relevant to the subject of my next book, the one I guess I’m writing now, with short breaks to revise the manuscript I sent to my editor on Monday. (cheers cheers thank you).
Twenty years ago, a sociologist of science called Harry Collins tried out an experiment. He had been researching the community of physicists who worked on gravity waves for the preceding two decades, trying to get insight into standards of proof and how experimental results were reconciled and interpreted. In furtherance of this project, he had been attending gravity waves conferences, where he was a familiar figure and had lots of friends in the community (because he talked to lots of people, physicists often sought him out to get gossip and news on what other labs were doing and seeing).
The experiment was a sort of Turing test – one group of physicists came up with a list of half a dozen questions about gravity waves, and then sent them by email to both Collins and a physicist, with instructions just to answer them from general knowledge rather than looking anything up. A jury of other physicists were then given the job of deciding which set of answers had come from Collins, and which from the physicist.
And all but one of the judges identified the sociologist as the physicist.
Although extremely funny, this wasn’t actually meant as a practical joke; it was meant to test a serious hypothesis about expertise. Harry Collins theorised that there was a thing called “interactional expertise” (basically the ability to hold a sensible and useful conversation about a subject) which was distinct from “contributory expertise” (he didn’t actually even know enough maths to do gravity wave physics). Deep interaction with a knowledge base and literature could provide one, but it wasn’t the same thing as actually working within the field.
When I wrote this up at the time, I remember thinking – was there really a clear dividing line? Like, for example, if someone really tried and dedicated a lot of time and effort to it, might they be able to pass an even tougher version of the same test. For example, could you get someone to be a co-author on a published journal article, based on nothing more than interactional expertise? I think the answer might be “trivially obviously so if it was just a literature review, and quite possibly they could be seen as a helpful and valuable co-author even on quite substantial original research”.
Or in fact – how does transactional expertise of the kind that a sociologist of science can put together by attending conferences – how does this differ from the understanding of the cutting edge of research which might be possessed by a principal investigator who runs a lab, makes decisions about hiring and such but who hasn’t actively carried out an experiment for the last decade? If someone like that hires a professional grant application writer to describe their current research program, what is the level of expertise that’s gone into that piece of writing?
You can see where I’m going here, can’t you, particularly since I gave so many clues on Wednesday. Because it seems to me that there is an important distinction here, which is not any less important because the dividing line might be difficult to establish empirically, or even if that line turns out to be in a different place from where we guessed it was. As well as difficult cases where it’s not clear, I think we could also come up with cases where the distinction between interactional and contributory expertise would suddenly become very clear and important indeed – the ones where someone who was faking it got “found out”.
And so the question that I think is quite important is whether there is a similar kind of distinction between the kind of expertise that it’s possible for a machine[1] to get by industralised consumption and interaction with a much larger corpus of literature than any human being could inhale, and genuine contributory expertise that could apply to entirely new situations outside that literature.
I think there’s a few contradictory intuitions here, and I’ll put down a marker that I would probably guess that the answer would be different in mathematics, logic and similar fields where there’s an axiomatic path to extrapolate from things you’ve already read to interesting and important theorems and sentences that haven’t yet been uttered. At present, I think it looks like the very impressive results of LLMs in mathematics seem to resemble Thomas Royen’s proof of the Gaussian correlation inequality – the sudden making of a connection which allows interaction with one part of the literature to open up something that’s always been implicit in another part.
But if we start thinking about other fields where progress isn’t monotonic; where things can be disproved, authorities disagree with one another and results can be true at one time and false at another … as we move along that spectrum I think something like the interactional/contributory distinction becomes more important.I am still not sure whether it’s a sharp distinction between two different things – and I’ll be writing more about that when an important paper gets published next week that I will be citing a lot – but I think it matters.
[1] To continue with the project of “sensible skepticism”, I should say that there’s two questions – whether it’s possible in principle for any machine at all, and whether it’s possible for any machine which can be built subject to reasonable resource and economic constraints in the foreseeable future.
This connects to something I've been thinking about since your last post. In my math olympiad days, we talked about the concept of "mathematical maturity" (or "maturing")
First level is simply remembering a proof. You can reproduce it, even talk about it convincingly, so its the interactional expertise. Then there's the level where you can re-enact it: you're not relying on memory but on understanding the mechanics well enough to reconstruct it. You're not faking it, but the subjective experience can still feel like you're an automaton.
Then there's this more elusive level, where universal truths about the subject start becoming clear. This is what "maturing" then means. You can extrapolate the theory, you "really get it." I think there's yet a level beyond that, where you see the structure of the representation itself and can go beyond it. That's where creativity lives. Feels like contributory can be at either of these two levels, depending on the contribution.
What strikes me is how long it can take to "mature," how slow the process is. There are concepts where I didn't really feel subjectively that I "got it" until decades later. And so many more that I never "got."
There's a a Quanta profile of Fields Medalist June Huh https://www.quantamagazine.org/june-huh-high-school-dropout-wins-the-fields-medal-20220705/ that also touches on this. His collaborator describes him as "very slow," so slow you'd think he couldn't pass a qualifying exam, but he was learning simple things in a deeper way that later proved essential.
It also makes me think of how hard it is to teach that the last mile of autonomous driving, while any 18 year old can pick it up in just a few months. The 18 year old, of course, has spent 18 years building a model of the physical world. Is there something to the slowness that scale cannot replicate?
I should have known you were also a member of the Harry Collins Superfan Club.
There's so much discourse in academia about AI eliminating the need for graduate students, but through the lens of Collins' interactional expertise, you see the PIs of big science--whose work devolves into management, grant-writing, and self-promotion--are even more automatable.