Discernment

Source: https://pluralistic.net/2026/07/28/hitl-ers/

Platform: blog

Author: Cory Doctorow

Original date: 2026-07-28

Summary (zh)

Doctorow 用「看不懂陶哲轩与 ChatGPT 讨论 Jacobian 猜想」说明：可靠使用 AI 依赖用户已有技能与经验；没有领域辨别力，就分不出严谨推理和数学味词沙拉。HITL 的前提是审查者懂行，而不是有人点确认。他把这一点落到教学：学生定义上缺乏辨别力，用 chatbot 当老师在逻辑上就不成立。

Summary (en)

Doctorow argues human-in-the-loop only works when the human already has domain discernment; without it, users cannot tell rigorous reasoning from plausible nonsense, especially when AI is used as a teacher.

One-liner

没有辨别力，AI 教不了你：HITL 需要的是判断力，不是确认按钮。

Fetched / evidence body

Discernment：没有辨别力，AI 教不了你

url: https://pluralistic.net/2026/07/28/hitl-ers/
score: 8.4
feed: Pluralistic / Cory Doctorow
archived_at: 2026-07-29T20:14:57.277238+08:00

Pluralistic: Discernment (28 Jul 2026) – Pluralistic: Daily links from Cory Doctorow

发布时间: October 29, 2025

原文链接: https://pluralistic.net/2026/07/28/hitl-ers/

https://pluralistic.net/2026/07/28/hitl-ers/

Today's links

Discernment: How can you fact-check AI when you're asking it to teach you something you don't understand?
Hey look at this: Delights to delectate.
Object permanence: How to help with computers; Batman equation; Fuck and the law; NC's racist voting law; Pregancy app is full of spyware; Stiglitz v Apple's "tax fraud"; Accelerometer fingerprinting; Sacklers v bankruptcy; Boss-politics antitrust.
Upcoming appearances: Edinburgh, Sydney, Melbourne, Brighton, London, South Bend.
Recent appearances: Where I've been.
Latest books: You keep readin' em, I'll keep writin' 'em.
Upcoming books: Like I said, I'll keep writin' 'em.
Colophon: All the rest.

Discernment (permalink)

As far as I can tell, this dialog between MacArthur prize-winning mathematician Terrence Tao and Chatgpt about "the Jacobian conjecture counterexample" is very impressive:

https://chatgpt.com/share/6a5fdc7a-d6f8-83e8-bbea-8deb42cfed56

Now, the clause "as far as I can tell" is doing a _lot_ of work in that sentence. I am reasonably math literate, up to first-year calculus and a lifetime spent around my father (a mathematician). However, I have never heard of "the Jacobian conjecture," and while I know what all the words in the first paragraph of the relevant Wikipedia entry mean, I can't parse any of the sentences they form:

https://en.wikipedia.org/wiki/Jacobian\_conjecture

In other words, I lack the discernment to evaluate the output of the chatbot that Tao exchanged theories with. If you showed me an equally opaque transcript of a "conversation" between a chatbot and a crank with AI psychosis whose math made no sense whatsoever, I couldn't make an _a priori_ judgment about which one was a solid piece of mathematical theorizing and which one was a math-flavored word-salad.

As many skilled programmers can attest, chatbots can produce very useful output – but as even the most ardent AI-assisted coder will admit, chatbot-written code is also full of baffling, obvious errors (and subtle, hard-to-spot ones):

https://pluralistic.net/2025/08/04/bad-vibe-coding/#maximally-codelike-bugs

These errors (which the industry wants us to refer to as "hallucination" – a whimsical, obscuring, anthropomorphizing euphemism) are the reason that reliable AI use requires the discernment that comes from skill and expertise. I use a local chatbot to spellcheck these posts. Chatbots spot all kinds of typos that regular spellcheckers miss:

https://pluralistic.net/2026/02/19/now-we-are-six/#stock-buyback

There's a reactionary group of strangers who seek me out to tell me that I'm a bad person for doing this. These are pointless conversations, mostly because I can barely make out a word over the scraping sounds of all the goalpost-moving these scolding strangers engage in.

They start by insisting that I'm burning down the planet by running a low-CPU load piece of software on my own computer. After I explain that running a chatbot on my machine uses no more carbon than, say, applying a blur effect to an image in my image editor, they tell me I'm unwisely giving my private data to the AI companies. Then I show them the network logs that demonstrate that my local chatbot doesn't send or receive _any_ network data.

Then they turn to the supposed cognitive effects of using a chatbot to find typos in an essay. I explain that I'm not asking an AI to write things for me or explain them to me – I'm asking it to point out where I've forgotten to put a period at the end of a paragraph, or fatfingered a word like "ever" as "every." I even send them the "before" and "after" of an essay after I've corrected some chatbot-identified typos in it:

https://craphound.com/before.txt

https://craphound.com/after.txt

This is when things get increasingly pointless. My interlocutors come up with farcical reasons why it's immoral or dangerous to use this LLM-based spellchecker. They say I'm using too much compute and that I could use a simpler piece of software to do the same thing (which is both untrue and silly – I also run a journaling filesystem on my computer that is vastly overpowered for editing a textfile – who cares?). Or they insist that the mere act of making copies of published works in order to count their elements and the relationships between them is a sin, despite the fact that this standard would kill search engines, the Internet Archive, and the Oxford English Dictionary:

https://pluralistic.net/2023/09/17/how-to-think-about-scraping/

I mean, by all means let's hate the AI companies and work to end their disgusting campaign to pauperize creative workers, but let's not fall into the trap of siding with the media bosses who insist that the salvation of creative labor will arrive when Sam Altman pays David Zaslav for the right to cram the entire Warner catalog into Openai's chatbots:

https://pluralistic.net/2026/03/03/its-a-trap-2/#inheres-at-the-moment-of-fixation

Above all, my interlocutors continue to insist that my LLM-powered, local, open source chatbot spellchecker will make me a worse writer. It's a very strange insistence. My first word processor was a program listing published in a magazine I bought at a corner store and laboriously typed into my Apple \]\[+. In the 40+ years since, word processors have gotten _lots_ of new features, many of which I thought were useful and many more that I found annoying. There were even some of these features that made the writers who used them worse at writing, in my (expert) judgment.

But from the very start, I knew that you couldn't just trust a spellchecker to correct your documents. I mean, I'm a _science fiction writer_. I started making up silly words _decades_ before coining "enshittification." I've been telling spellcheckers to fuck off since I learned to type. If you aren't a good writer, spellcheckers are _dangerous_, and the more "advanced" the spellchecker is, the more dangerous it is.

A few of my collaborators insist that I use Office 365's AI-enabled version of Word to work on documents with them. It's _maddening_. I estimate the ratio of good suggestions to bad ones that M365 insists on shoving into my face at about 1:100. It's practically unusable – so much so that I often copy the block of text we're working on into a text editor, make my changes, then paste it back into the Word window.

If I were to accept even 10% of these suggestions, my work would be made _significantly_ worse. Putting chatbots into Word pushed it from "annoying" into "enshittening." I certainly understand how relying on a chatbot to make edits to your work could make it worse.

That's where _discernment_ comes in. I have written more than 30 books over the past 25 years. I have _lots_ of experience defending my word choices, and not just against the mechanical judgments of a high-handed spellchecker, but also against overreaching copyeditors and paranoid publisher's lawyers. I _know_ which words I want to write, and I know _why_ I want to write them – and I know when a suggested fix is a good one and when it's wrong or stupid or just plain clunky. When it comes to writing, I have discernment.

That's not true when it comes to higher math. I would no more ask a chatbot to explain "the Jacobian conjecture counterexample" than I would tell my writing students to get a chatbot to suggest ways to fix their stories:

https://pluralistic.net/2026/01/07/delicious-pizza/#hold-the-gravel

I don't know nearly enough about math to ask a chatbot to explain it, or check my work, or even assemble a bibliography of human-authored works I should work my way through if I want to learn about it. If I wanted to understand "the Jacobian conjecture counterexample," I would set aside several days and work my way through that gnarly Wikipedia entry and its references and blue links to related concepts. If I _really_ wanted to understand it, I'd enroll in a course at the Open University or Khan Academy.

All of this has been obvious to me since I first encountered LLM-powered bots. If you understand a subject really well – well enough to discern useful bot output from defective bot output – then bots can be useful. Sometimes very useful, mostly ordinarily useful. For example, I've been writing Pluralistic for about 6.5 years now. I've written 1,683 posts now (1,684 after I hit publish on this one), and the corpus is now getting large enough that I sometimes struggle to find a post I'm trying to reference, even with all my careful tagging and my extensive knowledge of WordPress's URL-line options for searching the database with tag and keyword combos.

I've been toying with the idea of exporting my whole corpus and shoveling it into a local chatbot, so that I can type, "Which post did I talk about the evils of showing people your chatbot output in?" and get a link to the correct essay:

https://pluralistic.net/2026/03/02/nonconsensual-slopping/#robowanking

(Don't follow this link! I will be referencing the essay it goes to shortly; I struggled to find it when I sat down to write today; I'd accidentally tagged it with "at" instead of "ai" and missed the typo when I published it.)

There are very few subjects I have more discernment over than "essays I have written." If I ask a chatbot to tell me which post I'm thinking of, I will _instantly_ know which of its guesses are correct and which ones aren't. No one in the universe is better qualified than me to perform this task. No one ever will be.

Now, as it happens, I know exactly how badly a chatbot can screw up when it comes to my own work, because strangers insist on asking chatbots about me and then, for reasons I find baffling, they send me the output. Please don't show anyone your chatbot transcripts unless they ask to see them. It's embarrassing at best and annoying at worst: