There’s a whole knot of problems with LLMs around the fact that they don’t have expectations and thus can’t think “hang on, that’s not the expected result”, and I’d anticipate a similar problem applying to purely passive reviewers: if you’re *just* reviewing, and not also actively creating similar work yourself, your mental models are very likely to atrophy and you lose the ability to catch mistakes that are semantic rather than merely syntactic. If this bears out in reality, the number you’re looking for tends towards zero fairly rapidly for any individual who’s relegated to “just reviews”.
(On judgment, it gets worse: if your judgment goes, you also lose the ability to notice that your legs have stopped working.
On transduction, the two differences I see vs “lost in translation” are 1) the implication that the loss of information deliberate rather than a side effect, and 2) the implication that the loss is 100% within a demarcated range and 0% otherwise, rather than intermittent across the full range of input. Don’t know if those differences are material, but that’s the separation I read into the two terms.)
Reviewing someone else's LLM generated text feels much more difficult than human generated text. Whereas I find human errors are normally either poorly written or give the wrong answer, LLM errors are very fluidly written, and so I can feel it's wrong, but it's quite hard to identify which sentences need to be re-written.
Will work out with time if that's structural or if I just need to learn how to be a better editor.
I don’t know Stafford Beer’s work hugely well and wouldn’t dream of saying what he meant. But as a former biochemist, signal transduction is the process whereby a signal from outside a cell is detected and acted on. Most transduction pathways start with signalling molecules outside the cell binding to receptors on the cell surface, which in turn then trigger a response inside the cell, typically a cascade of reactions. In the eye the external signal starts not with a molecule binding to a receptor but a photon of light falling on a cone or rod cell, which eventually triggers a nerve impulse.
So I have always understood the organisational metaphor to be the process whereby one entity detects and then acts on signals that come from outside. And soft-systems style you can look at transduction at multiple granularities/levels of system.
(Relatedly, “transduction” without the leading “signal” also means the process of inserting DNA into a target cell - which can be done eg by viruses or biologists wanting to affect a cell to make it do something it doesn’t currently do. But I doubt that’s the one in this context since it doesn’t often happen in the eye.)
As I say, I have never really understood why SB insisted on this one as distinct from "translation" and the cybernetics crowd in general were always a bit wavy with neurological metaphors; I'll check up.
Speculating here, but in cellular signal transduction many different stimuli (which ones depending on the cell type) give rise to the same kind of signal, and specificity is conferred by the current state of the cell more than by the content of the signal, much of which is lost. So I can see how that might appeal to cyberneticians where the concept of translation (in the ordinary language sense) might seem to suggest more like a 1:1 mapping.
You could expand the boss's gnomic remarks to a full policy document with one LLM, and the reduce that down to three bullet points with another LLM. That this is what happens in many large organizations already is not, I think, the argument for LLMs that many seem to think it is.
I've done a lot of IT consulting on what are essentially information transmission systems for large corporations - and the conclusion I've come to is that most large organizations somehow function despite being very inefficient, extremely poor at transmitting information, while that information is at best misleading and often inaccurate (for reasons similar to what you've discussed in your articles about accounting). I'm not sure what the solution to this is, but I do know that the preferred solution of bolting on another piece of software, internal process or report - only makes things worse. And I really don't see how LLMs are supposed to solve the problem of Garbage In.
My way of looking at this is from the perspective of the present. Classic 80/20 rules suggests that 80% of the work is done by 20% of the employees. For a moment if we believe this to be true, perhaps anyone saying AI will allow for smaller organisations needs to answer the question of why is there so much bloat in the first place?
Flatter hierarchies etc work for smaller organisations but don't scale up. These organisations already had resource constraints so maybe AI makes it less acute. Doubt the become smaller but speed up the process to become bigger.
It sounds like we'd end up with people checking AI output. But they end up having limits, so you need a checker for the checkers. At what point do you end up with nobody doing any work but checking LLM output?
Firstly, thanks putting LLM or machines under variety management lenses. I do see that it can work as amplifier. Attenuation is more tricky, as some details are crucial like a word or tone in sentence giving out sarcasm.
Maybe AI can be put more into variety imbalance and time series anomaly detection like cybersyn envisioned.
There’s a whole knot of problems with LLMs around the fact that they don’t have expectations and thus can’t think “hang on, that’s not the expected result”, and I’d anticipate a similar problem applying to purely passive reviewers: if you’re *just* reviewing, and not also actively creating similar work yourself, your mental models are very likely to atrophy and you lose the ability to catch mistakes that are semantic rather than merely syntactic. If this bears out in reality, the number you’re looking for tends towards zero fairly rapidly for any individual who’s relegated to “just reviews”.
(On judgment, it gets worse: if your judgment goes, you also lose the ability to notice that your legs have stopped working.
On transduction, the two differences I see vs “lost in translation” are 1) the implication that the loss of information deliberate rather than a side effect, and 2) the implication that the loss is 100% within a demarcated range and 0% otherwise, rather than intermittent across the full range of input. Don’t know if those differences are material, but that’s the separation I read into the two terms.)
"Which ought to be worrying, because we know..."
I thought you were going to say, "because we know that air traffic control is one of the most mentally exhausting and difficult professions."
Reviewing someone else's LLM generated text feels much more difficult than human generated text. Whereas I find human errors are normally either poorly written or give the wrong answer, LLM errors are very fluidly written, and so I can feel it's wrong, but it's quite hard to identify which sentences need to be re-written.
Will work out with time if that's structural or if I just need to learn how to be a better editor.
I don’t know Stafford Beer’s work hugely well and wouldn’t dream of saying what he meant. But as a former biochemist, signal transduction is the process whereby a signal from outside a cell is detected and acted on. Most transduction pathways start with signalling molecules outside the cell binding to receptors on the cell surface, which in turn then trigger a response inside the cell, typically a cascade of reactions. In the eye the external signal starts not with a molecule binding to a receptor but a photon of light falling on a cone or rod cell, which eventually triggers a nerve impulse.
So I have always understood the organisational metaphor to be the process whereby one entity detects and then acts on signals that come from outside. And soft-systems style you can look at transduction at multiple granularities/levels of system.
(Relatedly, “transduction” without the leading “signal” also means the process of inserting DNA into a target cell - which can be done eg by viruses or biologists wanting to affect a cell to make it do something it doesn’t currently do. But I doubt that’s the one in this context since it doesn’t often happen in the eye.)
As I say, I have never really understood why SB insisted on this one as distinct from "translation" and the cybernetics crowd in general were always a bit wavy with neurological metaphors; I'll check up.
Speculating here, but in cellular signal transduction many different stimuli (which ones depending on the cell type) give rise to the same kind of signal, and specificity is conferred by the current state of the cell more than by the content of the signal, much of which is lost. So I can see how that might appeal to cyberneticians where the concept of translation (in the ordinary language sense) might seem to suggest more like a 1:1 mapping.
You could expand the boss's gnomic remarks to a full policy document with one LLM, and the reduce that down to three bullet points with another LLM. That this is what happens in many large organizations already is not, I think, the argument for LLMs that many seem to think it is.
I've done a lot of IT consulting on what are essentially information transmission systems for large corporations - and the conclusion I've come to is that most large organizations somehow function despite being very inefficient, extremely poor at transmitting information, while that information is at best misleading and often inaccurate (for reasons similar to what you've discussed in your articles about accounting). I'm not sure what the solution to this is, but I do know that the preferred solution of bolting on another piece of software, internal process or report - only makes things worse. And I really don't see how LLMs are supposed to solve the problem of Garbage In.
My way of looking at this is from the perspective of the present. Classic 80/20 rules suggests that 80% of the work is done by 20% of the employees. For a moment if we believe this to be true, perhaps anyone saying AI will allow for smaller organisations needs to answer the question of why is there so much bloat in the first place?
Flatter hierarchies etc work for smaller organisations but don't scale up. These organisations already had resource constraints so maybe AI makes it less acute. Doubt the become smaller but speed up the process to become bigger.
It sounds like we'd end up with people checking AI output. But they end up having limits, so you need a checker for the checkers. At what point do you end up with nobody doing any work but checking LLM output?
I incautiously took sides back in 2022 https://crookedtimber.org/2022/10/08/ai-is-coming-for-bullsht-jobs/
Firstly, thanks putting LLM or machines under variety management lenses. I do see that it can work as amplifier. Attenuation is more tricky, as some details are crucial like a word or tone in sentence giving out sarcasm.
Maybe AI can be put more into variety imbalance and time series anomaly detection like cybersyn envisioned.