You want the dog to always fetch the newspaper that usually lands in your driveway, but which sometimes ends up in your neighbor's driveway, and also you do not want the dog to attack the neighbor who thinks the paper belongs to them, and also you don't want the dog to get in the habit of stealing your neighbor's Amazon deliveries, and also you want the dog to do the task eagerly and enthusiastically, and also you want the dog to love you, and also you don't want to get sued by your neighbor, a lawyer (and your brother-in-law), and also you forgot that your subscription to the paper ran out and it was actually your neighbor's copy of the paper, and also you were half asleep when you mindlessly shouted "Fetch!"
And you totally forgot that Robbie was a Robo-dog, and he didn't get the urgent software update that came out last night because you hadn't checked your email yet. Not that it would have mattered, because the email labelled "Emergency" had gone straight into SPAM. Too bad, it would also have fixed the occasional problem where the software goes crazy and starts telling the Robo-dog to attack and kill its owner.
Whose idea was it to get the Robo-dog, anyway? Not yours. The children really wanted a dog, but you are allergic to dogs, so you got this Robo-thingy instead, then somehow became emotionally attached to it...Robbie is really cute, and reminds you of that doggie you had as child, but your parents got rid of when they found out you were allergic to dog hair. You never really recovered from the loss of your playmate, but had repressed the memory...until today.
Is this how slave owners fall in love with their slaves and have children with them?
Then you realize that you have foolishly been loving a bag of silicon and graphite that only simulates loving you back and won't be giving you any adorable puppies either, and you collapse into a heap of self recrimination and humiliation.
Why, oh why?
And all this because of a stupid newspaper! And a literal-minded Robo-dog! You never liked the dog anyway. It was always leaving a trail of little lithium-ion batteries that you have to pick up and recharge...
Then the alarm went off and you woke up, to begin another perfect day in your actual life.
I am extremely worried about the combination of LLMs/outsourcing -- as our vendors start relying on it, it'll be much harder to detect cases where they didn't properly understand the requirements. Instead of failing upfront, we'll get failures nobody understands six months down the line.
Yes, this is definitely going to happen, particularly at the lower end of KPO. We've spent ages on trying to give our offshore staff a sense of career progression and pride in their work, and I feel like this is where it is going to pay dividends in that they might be more inclined to take warnings about GenAI seriously
«Feb 21, 2025 [...] What exactly is AGI, and how close are we to achieving it? "Whatever AGI is and how we define it, I think in every definition that is reasonable, we are still far away," DeepL CEO Jarek Kutylowski tells me during our recent conversation.»
LLMs currently are in the essence summarizers of web searches and for that they do moderately well and sometimes they appear to "reason".
With respect, in the AI industry, the term "AGI" does in fact stand for "artificial general intelligence." This is distinguished from "Gen AI," which is short for generative artificial intelligence. Gen AI itself is different from other forms of AI, for example machine learning and expert systems.
The LLMs like ChatGPT, etc., are all Gen AI, meaning they generate stuff--text, images, etc. They ARE NOT yet AGI, much as the hype artists (sorry, CEOs) of the companies like Open AI and Anthropic claim that it is just around the corner. True AGI, meaning an artificial version of the kind of general intelligence that humans possess, is unreachable via the current technical approaches available.
But then how can it help offshoring/outsourcing? It takes AGI to understand requirements and organize code (or legal briefs etc.). LLMs do not "hallucinate" as they are not "lucid" either as they just makes a summary of a set of web search results.
If the requirements are not part of existing documents the result will be off, and it will be on target only if the requirements and code (or legal briefs etc.) in effect are part of the same document set, and if they are that means the product already exists (or the legal document is in effect a standard template of which all lawyers have a stack).
I just wanted to clarify Dan's probable meaning for you in the spirit of encouraging good communication! I'm not implying it makes his claim any more or less plausible and I have no interest in debating the point.
«I am extremely worried about the combination of LLMs/outsourcing»
The purpose of LLMs or outsourcing is to boost the bonuses/options of executives. Any "unexpected" side effects matter a lot less, and somebody else's problem.
It would be so wonderful to have a way to say that sentence in the last paragraph convincingly. And to convince one’s self of it as well, which is part of the problem.
I think that just due to, like, neurochemistry or whatever, you’re probably never going to get it into a sentence, or even the same paragraph. The need to let the first signal take effect and then dissipate before communicating the second is likely unavoidable.
Your last comment isn't quite true. Some of the greatest recent improvements in coding assistance AI have been exactly in handling feedback. The new coding AI tools can turn user instructions into rules that are kept in-context all the time (even in very long conversations), and Gemini 2.5 Pro is also far better at stopping in the middle of a chain of thought to realize that it doesn't, in fact, know what to do and asking the user for clarifications. IMO this really points at the need to train LLMs specifically for work.
«“Whose job is it to handle communication with the offshore team”? It is classic cybernetics stuff – part of the cost of setting up a system is the cost of setting up information infrastructure to make sure that information flows in and out of the black box»
Ah, I have recently thought why I find our author/blogger's book and post here to be a bit too sophisticated or abstract for me:
* My impression is that our author/blogger seems to base his examples and perhaps his reasoning on information flow based on communication via language, whether written or spoken, and he seems to envisage that the best understanding of (management, mostly) systems via cybernetics is to look at written or spoken communication flow and feedback.
* In my simplistic and naive view I am used to assume that money or bullets are far more powerful carriers of information and feedback and a much more common basis for decisions than pondering over communication via written documents or oral conversations.
So for example for me far from being the signal of the end of management cybernetics the bullets that hit Allende in the Kissinger-Pinochet "feedback" mentioned in his book were the veritable triumph of cybernetics (just like many other episodes of the same type). They proved to be far more effective feedback than a button or a dial in every chilean home :-(.
Not all information and feedback are carried by words and some other means are far better at adding "requisite variety" (which may be an euphemism for "leverage"). Just as a picture is worth a thousands words, money (and less so the opportunity to make money) or bullets (and less so the fear of them) seem to me worth a thousand reports or briefings as carriers of information and feedback and a basis for decisions in a cybernetic system involving human beings, and I simple-mindedly reckon that in many cases flows reports and briefings are just covers for flows of money or bullets.
This said I can imagine that there are situations where written or spoken words are more important carriers of information and feedback than money or bullets but my impression is that usually happens when the stakes are not that important to the people who have much money or bullets at "level 5", or when those people who have much money or bullets are divided.
I'm suprised there is no "Actually Indians" joke for AI. After all thats what it turned out Amazon was doing with its stores and saying the magic word AI.
I'm in the process of writing something about legal AI and billable hours - basically why genAI will not "kill" the billable hour by magically making lawyers more productive. As part of that research, right before seeing this post, I had read this article:
The article itself isn't noteworthy - there are a million that say the same thing - but the genre is one of "purchasing people at law firms purchased some AI product, set it loose, and are now realizing that people don't just pick it up and magically become more productive." So now everyone is "retooling," and as far as I can tell, doing more work than they did before to figure out how to get the frontline staff to consistently use the product they already bought.
“We don’t have the human language to make a sentence like “That is bad news and I am annoyed, specifically I am annoyed with you for doing something wrong but thank you for telling me, I am pleased with that bit” sound convincing”—not at all sure this is true! Isn’t this the basis of most parenting? “Thank you for telling me, I really appreciate your honesty. Next time…” (and variations thereon)
Asimove made controlling robots so easy. All teh Spacers with their many robots were adept at giving the correct orders to get the desired task done. IRL, it is much harder to get a GenAI to provide the answer to the task you want done. When those AIs get embedded in robots...
Humans can often use the theory of mind to ensure that their communication works. "Common sense" and experience help too. But machine intelligence doesn't have any theory of mind, AFAIK. One needs to be far more explicit with requests, and even then, it starts to get to the "It will be faster if I do it myself".
When we had servants, the master or mistress would summon the head servant to convey the tasks to be done, who then in turn, conveyed those orders to the lower staff. Experience of the household and past orders informed the head servant how best to convey those orders. Perhaps a solution is to have a personalized AI act as an intermediary between you and the specialty AIs?
I'll add the obvious from my time running an outsourced IT team (as the PoC on the company side.)
Even when you have the communications / delegation / general info sorted, people massively underestimate how much context there is for the outsourced team to learn. (And of course, just as they have learned it, the best of them find jobs elsewhere and you're back to the learning phase.)
The LLM angle here is potentially that the LLMs work best for things where their training set included a lot of the context in forms not easily muddled by the training process. So coding, college essays, churnalism and a relatively few other things. Bigger context windows help, but it's slow going. I suspect we'll get some kind of noticeable step forward when various memory functions are put in place - but AIUI from some people working on this stuff, that's not so easy under the current economic model, so it may require someone to take a real sidestep first.
How much context there is to learn and/or how slowly context is learned over the phone/internet. It’s painful even if you’re in the same timezone and visit regularly!
My Facebook feed has become cluttered with ads for AI training jobs, "starting at $40 an hour!" Training an LLM requires humans to rank LLM responses using human judgement, which is how you bend the arc of AI to reliability (in human terms, at least). The higher-rated responses get fed back into the hopper, are used as starting points for the new round, etc. etc.
I'm wondering, re: "reinforcing but negative feedback"--doesn't that require human expertise? Wouldn't a path to an effective specialized LLM use, say, experienced lawyers or humanities Ph.D.s or cybernetics experts to train the particular speciality by ranking its responses? Not at $40 an hour, I'd think.
I'd never realised the point about offshoring and people being bad a delegation.
But my wife almost never delegates work to people who should be helping her, yet I'll happily do that for her.
Often for me I'll delegate what I can, but as the technical Co-Founder of a SaaS startup who only has 1 other developer I can delegate to, and he's a mid level programmer in the Philippines, there's only so much I can delegate.
Also there's plenty I want to do myself because it's exciting and fun and likely to get me into a Flow state.
So delegation based on likelyhood of someone being excited for the work, aka they'll be in the Zone, seems like an ideal tactic we should be applying on a larger scale.
Does working with an AI agent help that flow state? Is your ability to be in flow affected by using AI vs delegation vs doing it yourself?
I don't know and it obviously depends on the work and how you are using the AI.
E.g there's Vibe coding and Augmented Coding.
I'll Vibe code on a prototype project in a programming language I know barely anything about.
I'll manually code really important stuff where I know exactly what the code should be.
I'll do augmented coding when there's something I know I want done but can't be bothered putting in ALL the effort. Like when trying to deal with a Typescript issue. The code and logic is there it's just that telling the compiler what type of data to expect and is somewhat arcane in how to write certain types.
Or I'll ask Cursor, Claude or whatever AI to write a first pass then spend a while cleaning it up and making it work as needed and to spec.
I think this mode is most like an offshore delegation. I'd get back code that's technically doing what was asked but doesn't do it with real care to the whole system nor really considering edge cases well enough.
Sounds like the classic "dog training" problem:
You want the dog to always fetch the newspaper that usually lands in your driveway, but which sometimes ends up in your neighbor's driveway, and also you do not want the dog to attack the neighbor who thinks the paper belongs to them, and also you don't want the dog to get in the habit of stealing your neighbor's Amazon deliveries, and also you want the dog to do the task eagerly and enthusiastically, and also you want the dog to love you, and also you don't want to get sued by your neighbor, a lawyer (and your brother-in-law), and also you forgot that your subscription to the paper ran out and it was actually your neighbor's copy of the paper, and also you were half asleep when you mindlessly shouted "Fetch!"
And you totally forgot that Robbie was a Robo-dog, and he didn't get the urgent software update that came out last night because you hadn't checked your email yet. Not that it would have mattered, because the email labelled "Emergency" had gone straight into SPAM. Too bad, it would also have fixed the occasional problem where the software goes crazy and starts telling the Robo-dog to attack and kill its owner.
Whose idea was it to get the Robo-dog, anyway? Not yours. The children really wanted a dog, but you are allergic to dogs, so you got this Robo-thingy instead, then somehow became emotionally attached to it...Robbie is really cute, and reminds you of that doggie you had as child, but your parents got rid of when they found out you were allergic to dog hair. You never really recovered from the loss of your playmate, but had repressed the memory...until today.
Is this how slave owners fall in love with their slaves and have children with them?
Then you realize that you have foolishly been loving a bag of silicon and graphite that only simulates loving you back and won't be giving you any adorable puppies either, and you collapse into a heap of self recrimination and humiliation.
Why, oh why?
And all this because of a stupid newspaper! And a literal-minded Robo-dog! You never liked the dog anyway. It was always leaving a trail of little lithium-ion batteries that you have to pick up and recharge...
Then the alarm went off and you woke up, to begin another perfect day in your actual life.
I am extremely worried about the combination of LLMs/outsourcing -- as our vendors start relying on it, it'll be much harder to detect cases where they didn't properly understand the requirements. Instead of failing upfront, we'll get failures nobody understands six months down the line.
Yes, this is definitely going to happen, particularly at the lower end of KPO. We've spent ages on trying to give our offshore staff a sense of career progression and pride in their work, and I feel like this is where it is going to pay dividends in that they might be more inclined to take warnings about GenAI seriously
«take warnings about GenAI seriously»
LLMs are very far away from general AI (if "Gen" stands for general).
https://www.forbes.com/sites/charlestowersclark/2025/02/21/deepl-ceo-challenges-microsoft--openais-100-billion-agi-definition/
«Feb 21, 2025 [...] What exactly is AGI, and how close are we to achieving it? "Whatever AGI is and how we define it, I think in every definition that is reasonable, we are still far away," DeepL CEO Jarek Kutylowski tells me during our recent conversation.»
LLMs currently are in the essence summarizers of web searches and for that they do moderately well and sometimes they appear to "reason".
It stands for generative not general.
With respect, in the AI industry, the term "AGI" does in fact stand for "artificial general intelligence." This is distinguished from "Gen AI," which is short for generative artificial intelligence. Gen AI itself is different from other forms of AI, for example machine learning and expert systems.
The LLMs like ChatGPT, etc., are all Gen AI, meaning they generate stuff--text, images, etc. They ARE NOT yet AGI, much as the hype artists (sorry, CEOs) of the companies like Open AI and Anthropic claim that it is just around the corner. True AGI, meaning an artificial version of the kind of general intelligence that humans possess, is unreachable via the current technical approaches available.
"GenAI" is the term I was clarifying for blissex. I agree it standardly means generative artificial intelligence :)
«It stands for generative not general.»
But then how can it help offshoring/outsourcing? It takes AGI to understand requirements and organize code (or legal briefs etc.). LLMs do not "hallucinate" as they are not "lucid" either as they just makes a summary of a set of web search results.
If the requirements are not part of existing documents the result will be off, and it will be on target only if the requirements and code (or legal briefs etc.) in effect are part of the same document set, and if they are that means the product already exists (or the legal document is in effect a standard template of which all lawyers have a stack).
I just wanted to clarify Dan's probable meaning for you in the spirit of encouraging good communication! I'm not implying it makes his claim any more or less plausible and I have no interest in debating the point.
«I am extremely worried about the combination of LLMs/outsourcing»
The purpose of LLMs or outsourcing is to boost the bonuses/options of executives. Any "unexpected" side effects matter a lot less, and somebody else's problem.
It would be so wonderful to have a way to say that sentence in the last paragraph convincingly. And to convince one’s self of it as well, which is part of the problem.
I think that just due to, like, neurochemistry or whatever, you’re probably never going to get it into a sentence, or even the same paragraph. The need to let the first signal take effect and then dissipate before communicating the second is likely unavoidable.
Your last comment isn't quite true. Some of the greatest recent improvements in coding assistance AI have been exactly in handling feedback. The new coding AI tools can turn user instructions into rules that are kept in-context all the time (even in very long conversations), and Gemini 2.5 Pro is also far better at stopping in the middle of a chain of thought to realize that it doesn't, in fact, know what to do and asking the user for clarifications. IMO this really points at the need to train LLMs specifically for work.
«“Whose job is it to handle communication with the offshore team”? It is classic cybernetics stuff – part of the cost of setting up a system is the cost of setting up information infrastructure to make sure that information flows in and out of the black box»
Ah, I have recently thought why I find our author/blogger's book and post here to be a bit too sophisticated or abstract for me:
* My impression is that our author/blogger seems to base his examples and perhaps his reasoning on information flow based on communication via language, whether written or spoken, and he seems to envisage that the best understanding of (management, mostly) systems via cybernetics is to look at written or spoken communication flow and feedback.
* In my simplistic and naive view I am used to assume that money or bullets are far more powerful carriers of information and feedback and a much more common basis for decisions than pondering over communication via written documents or oral conversations.
So for example for me far from being the signal of the end of management cybernetics the bullets that hit Allende in the Kissinger-Pinochet "feedback" mentioned in his book were the veritable triumph of cybernetics (just like many other episodes of the same type). They proved to be far more effective feedback than a button or a dial in every chilean home :-(.
Not all information and feedback are carried by words and some other means are far better at adding "requisite variety" (which may be an euphemism for "leverage"). Just as a picture is worth a thousands words, money (and less so the opportunity to make money) or bullets (and less so the fear of them) seem to me worth a thousand reports or briefings as carriers of information and feedback and a basis for decisions in a cybernetic system involving human beings, and I simple-mindedly reckon that in many cases flows reports and briefings are just covers for flows of money or bullets.
This said I can imagine that there are situations where written or spoken words are more important carriers of information and feedback than money or bullets but my impression is that usually happens when the stakes are not that important to the people who have much money or bullets at "level 5", or when those people who have much money or bullets are divided.
I'm suprised there is no "Actually Indians" joke for AI. After all thats what it turned out Amazon was doing with its stores and saying the magic word AI.
https://www.goingconcern.com/when-ai-stands-for-actually-indians/
I'm in the process of writing something about legal AI and billable hours - basically why genAI will not "kill" the billable hour by magically making lawyers more productive. As part of that research, right before seeing this post, I had read this article:
https://www.thomsonreuters.com/en-us/posts/technology/ilta-evolve-fitting-genai-into-current-systems/
The article itself isn't noteworthy - there are a million that say the same thing - but the genre is one of "purchasing people at law firms purchased some AI product, set it loose, and are now realizing that people don't just pick it up and magically become more productive." So now everyone is "retooling," and as far as I can tell, doing more work than they did before to figure out how to get the frontline staff to consistently use the product they already bought.
“We don’t have the human language to make a sentence like “That is bad news and I am annoyed, specifically I am annoyed with you for doing something wrong but thank you for telling me, I am pleased with that bit” sound convincing”—not at all sure this is true! Isn’t this the basis of most parenting? “Thank you for telling me, I really appreciate your honesty. Next time…” (and variations thereon)
"Walking from Hotspot to Hotspot like the mfking wolf from Pulp Fiction". Yeah, that tracks.
Asimove made controlling robots so easy. All teh Spacers with their many robots were adept at giving the correct orders to get the desired task done. IRL, it is much harder to get a GenAI to provide the answer to the task you want done. When those AIs get embedded in robots...
Humans can often use the theory of mind to ensure that their communication works. "Common sense" and experience help too. But machine intelligence doesn't have any theory of mind, AFAIK. One needs to be far more explicit with requests, and even then, it starts to get to the "It will be faster if I do it myself".
When we had servants, the master or mistress would summon the head servant to convey the tasks to be done, who then in turn, conveyed those orders to the lower staff. Experience of the household and past orders informed the head servant how best to convey those orders. Perhaps a solution is to have a personalized AI act as an intermediary between you and the specialty AIs?
I'll add the obvious from my time running an outsourced IT team (as the PoC on the company side.)
Even when you have the communications / delegation / general info sorted, people massively underestimate how much context there is for the outsourced team to learn. (And of course, just as they have learned it, the best of them find jobs elsewhere and you're back to the learning phase.)
The LLM angle here is potentially that the LLMs work best for things where their training set included a lot of the context in forms not easily muddled by the training process. So coding, college essays, churnalism and a relatively few other things. Bigger context windows help, but it's slow going. I suspect we'll get some kind of noticeable step forward when various memory functions are put in place - but AIUI from some people working on this stuff, that's not so easy under the current economic model, so it may require someone to take a real sidestep first.
How much context there is to learn and/or how slowly context is learned over the phone/internet. It’s painful even if you’re in the same timezone and visit regularly!
My Facebook feed has become cluttered with ads for AI training jobs, "starting at $40 an hour!" Training an LLM requires humans to rank LLM responses using human judgement, which is how you bend the arc of AI to reliability (in human terms, at least). The higher-rated responses get fed back into the hopper, are used as starting points for the new round, etc. etc.
I'm wondering, re: "reinforcing but negative feedback"--doesn't that require human expertise? Wouldn't a path to an effective specialized LLM use, say, experienced lawyers or humanities Ph.D.s or cybernetics experts to train the particular speciality by ranking its responses? Not at $40 an hour, I'd think.
I'd never realised the point about offshoring and people being bad a delegation.
But my wife almost never delegates work to people who should be helping her, yet I'll happily do that for her.
Often for me I'll delegate what I can, but as the technical Co-Founder of a SaaS startup who only has 1 other developer I can delegate to, and he's a mid level programmer in the Philippines, there's only so much I can delegate.
Also there's plenty I want to do myself because it's exciting and fun and likely to get me into a Flow state.
So delegation based on likelyhood of someone being excited for the work, aka they'll be in the Zone, seems like an ideal tactic we should be applying on a larger scale.
Does working with an AI agent help that flow state? Is your ability to be in flow affected by using AI vs delegation vs doing it yourself?
I don't know and it obviously depends on the work and how you are using the AI.
E.g there's Vibe coding and Augmented Coding.
I'll Vibe code on a prototype project in a programming language I know barely anything about.
I'll manually code really important stuff where I know exactly what the code should be.
I'll do augmented coding when there's something I know I want done but can't be bothered putting in ALL the effort. Like when trying to deal with a Typescript issue. The code and logic is there it's just that telling the compiler what type of data to expect and is somewhat arcane in how to write certain types.
Or I'll ask Cursor, Claude or whatever AI to write a first pass then spend a while cleaning it up and making it work as needed and to spec.
I think this mode is most like an offshore delegation. I'd get back code that's technically doing what was asked but doesn't do it with real care to the whole system nor really considering edge cases well enough.