Our Servants Will Do That For Us

Source: Fernando Borretti · https://borretti.me/article/our-servants-will-do-that-for-us
URL: https://borretti.me/article/our-servants-will-do-that-for-us
Original date: 2026-08-20
Captured: 2026-08-22 (AAIF daily intake, opencli web read)

摘要（AAIF）

Fernando Borretti 用任务复杂度×时间的 sigmoid 图拆开 AGI 的隐含许诺：250 年自动化史只推进可形式化任务的边界线，而 y 轴上并不存在 drudgery 界限——drudgery 与 meaningful work 同属一个复杂度类，能自动化的技术必然同时吞掉两者；AGI 一旦把曲线瞬间推到 100%，经济激励反向运转，人类唯一逃逸口是关系经济（必须在场的人脸工作：播客主、客户接口活人）。第二论点是 revealed preference：对自己工作谈意义、对他人工作只谈结果（Waymo 例），若机器能产出同等审美价值的文本图像，受众流失多少是尚未揭晓的经验问题。数学家正在重演软件工程师十二个月前的过程。

为什么值得读：在 AI 取代人类的乐观/悲观情绪辩论之外，给出一个用任务图与复杂度类就能验证或证伪的清晰框架。

Summary (EN)

Fernando Borretti dissects the implicit promise of modernity with a task-complexity-vs-time sigmoid: 250 years of automation pushed the limit of formalizability, but there is no 'limit of drudgery' on the complexity axis — drudgery and meaningful work sit in the same complexity class, so whatever automates one automates the other. Once AGI shoots the curve to 100 percent automation, economic incentives turn against humanity and the only niche left is the relational economy where a human face is part of the job description. His second argument is revealed preference: we describe our own work in meaningful terms and everyone else's instrumentally (his Waymo example), and whether audiences stay loyal to human-made art once machines produce equal aesthetic value is an open empirical question. Mathematicians, he notes, are rerunning software engineering's trajectory twelve months later.

原文（full text capture）

The implicit promise of modernity is: we have to work really hard for a few centuries, but eventually we’ll get to the _Star Trek_ future where we’ve automated all the drudgery, and all work is meaningful, heroic, and dignifies the spirit. We will be artists, scholars, captains of the starship _Enterprise_, etc. There’s a few problems with this idea:

1. “Drudgery” and “meaningful work” are in the same complexity class, so the technology to automates the former also automates the latter. 2. Our revealed preference seems to be that we prefer solipsistic convenience over human relationships.

The rest of this post elaborates the argument.

Task Complexity and AGI

We’ve spent the last 250 years building machines to automate one task at a time, and the result has been overwhelmingly positive, for the simple reason that if a human can design a machine or an effective procedure to perform some task, then by definition the task is drudgery. And so the economic incentive to automate work has led to more human flourishing: over time work becomes safer, higher-paying, less monotonous, healthier, more intellectually and socially stimulating.

You can think of it as a graph where the xxx axis is time and the yyy axis is task complexity. Somewhere along the yyy axis there’s a limit: tasks below the limit are those which we can design machines or programs to perform, tasks above the limit are too complex, we have tried and failed (and failed and failed) and failed) to automate them with software. The industrial revolution, and later the computer revolution, automated most tasks under this limit:

The human-complete tasks are those you need a human to perform. Some are meaningful, like writing essays, some are drudgery, but human-level drudgery, like life admin.

But note that nowhere in the yyy axis is there a “Limit of Drudgery”: the line we draw between “good” and “bad” work is not the line nature draws between easy and hard to automate. The invention of AGI—“highly autonomous systems that outperform humans at most economically valuable work”, as per the OpenAI charter—creates an obvious discontinuity: suddenly we shoot up to 100% automation, both of the meaningful work and the drudgery:

And at this point AGI the economic incentives work against humanity. The only niche to escape to is the relational economy, where having a human face is an intrinsic part of the job description, like being a podcaster or a human liability crumple zone.

Solipsism

The second problem has to do with people’s preferences. We think of our own work as meaningful, but we think of most everyone else’s work in instrumental terms: we care about the ends rather than the human means. Thus when selling we emphasize human qualities like experience, reputation, trust, relationships, etc., and when buying we want cheap, fast, and convenient. And, all else being equal, we prefer the transactional and faceless over the human and relational. Waymo is a thousand times better than Uber, even though it’s slower and more expensive, because there’s no human.

The flip side of this is: whatever work you do, most people would rather an alternative that gave them the same outcome or better1, without having to deal with a human person. That is, if you ask everyone on Earth what jobs they’d like to see automated, and take the union of that, it’s everything. There’s nothing left.

We see this with ordinary consumer products and services, where often the advertising seems premised on you being some kind of stylite who lives in a little human terrarium, probably in a building called something like “The Soho”, food arrives through a hole in the wall, and you have this sovereign, solipsistic existence that leaves no physical trace in the world.

We see this, increasingly, with knowledge work. Personally, I think software engineering is a very meaningful and intellectually-rewarding activity. But users just want working software. They’re not paying me to sit around polishing the database API into a jewel of flawless, geometric logic. They don’t care about my opinions on the Standard ML module system. They just want the software. Why deal with some opinionated maniac when, for $20 a month, you get access to this little machine ghost that writes reams and reams of mostly working code, tirelessly, instantly?

The revealed preference of many software firms is: let AI write the code, and software engineers move one level up, to wrangle the AIs and complement the things they can’t do. But this change in job description is not because firms love employing humans, it’s because today’s AI models have serious limitations: no online learning, no mutable long-term memory, and while they can write code that works, they’re not good at organizing and structuring that code. If, in the future, these limitations are overcome (and trillions of dollars are being spent on overcoming them), then software engineering as a profession will disappear.

Mathematicians are going through this process now, too, and it’s an exact parallel of what happened to software engineering, only twelve months later. The difference is that most mathematicians hate the de-intellectualization of their field, while software engineering is full of narrow-minded, hustle culture-brained, incurious, anti-intellectual hucksters who are overjoyed that they no longer have to read or write another line of code, or think using formal systems at all. Why reason at all? Our servants will do that for us.

What about art? Art is the _ne plus ultra_ of meaning, so artists should be safe. But every work of art is two things: there’s a concrete thing, a text or a painting or a song; and there’s a constellation of human aspects: who made it, and why, what madness they were trying to exorcise in making it, what minor deity breathed the work into them, the discourse around the work, later art inspired by it, and so on. And these things are separable: you can enjoy a work of art for its aesthetic value, as a sequence of sensory impressions, and ignore every other dimension.

Now say that a machine could generate texts or images of equivalent aesthetic value2 to your own, instantly and at ~zero cost. How much of your audience would desert you? Relational jobs have an analogous problem: you might like being a podcaster, but how much of your audience would prefer an infinite stream of AI-generated speech on whatever topic they want to hear about?

This is an empirical question, and we can’t answer it, because AI is not there, yet. But we might discover that our preferences are a lot less humanist and a lot more solipsistic that we want to believe.

Acknowledgements

Emmy Noether once said that “everything is already in Dedekind”. As for me, everything I write about AI was already said by J. D. Pressman.

Footnotes

1. This is a horrendous oversimplification, because I’ve stuffed all the complexity into the notion of “same outcome or better”. So let’s try it again, _more geometrico_ and with unnecessary LaTeX. You want to buy a good. You look around, assemble a set of options, sort them by utility and pick the winner. And you can think of the utility as like a weighted sum:

u(x)\=∑a∈Aαaxa\\mathfrak{u}(x) = \\sum\\limits\_{a \\in A} \\alpha\_a x\_au(x)\=a∈A∑αaxa

Where the a’s are the aspects you are weighing (price, shipping speed, trust in the brand, how much social status I get from this thing, etc.), the variables quantify these aspects, and the alphas are the coefficients. So if you’re a hypebeast your αclout\\alpha\_\\text{clout}αclout is much higher than your αprice\\alpha\_\\text{price}αprice, and if you’re a neckbeard the xcloutx\_\\text{clout}xclout of a busted ThinkPad is higher than that of a MacBook.

Now, humans are slow, because biology; and because of Baumol we get more expensive over time. Computers are fast and cheap. On top of that, the pace of improvement in AI is vastly greater than the pace of improvement in humans. So the utility of automated production gets pulled up, and the utility of human production gets pulled down, and eventually they are so far apart that you have to be an extremist with a very high αhuman\\alpha\_\\text{human}αhuman coefficient to choose the human alternative. ↩

2. I’m being careful to say “texts or images” rather than “art”, and “aesthetic value” rather than “quality”, because you might object that machine-made art is not art. And what I’m trying to say is that is doesn’t have to be art. It can just be a pleasing image, a pleasing text, and so on. ↩

Source: https://borretti.me/article/our-servants-will-do-that-for-us

Captured: 2026-08-22 (AAIF daily-intake-evening)