Reading Every AI Output Doesn't Scale Anymore. Here's What Does.

If your check on AI output is still "read the whole thing, then decide," that habit is the bottleneck now, not the safety net. Four things that happened in the past six weeks all point at the same gap: AI is producing more output, in more formats, faster than any one person can read line by line — and reading faster isn't the fix.

Four signals, one shape

In early October, arXiv capped submissions at two per author per month after AI-assisted papers pushed monthly volume to roughly double 2024's rate — the first major research repository to hard-limit output specifically because production outran what its moderators could check.

Days later, OpenAI published 722 AI-generated math manuscripts in one batch, most without independent verification. The stack read as credible because it was large. Size isn't proof; it's 722 copies of the same unverified status.

The same week, GPT-6 and Claude both started choosing their own output format — dashboards, calculators, animations — instead of defaulting to prose. Andrej Karpathy's framing of why stuck: AI now produces output faster than anyone can review it, so the industry's answer is formats that look checked at a glance, whether or not they are.

And in late September, a widely-read guide from Hamel Husain and Shreya Shankar named the step most teams skip entirely: reading real failures by hand before picking a metric to track them. A score built before anyone read a failure is a guess wearing a number.

The skill implication

None of these are stories about AI getting worse. They're the same story from four angles: volume and format both scaled past what a full read-through can absorb, and "I read it" stopped being a check you can actually run on everything. Three habits replace it. Decide what "checked" means for a specific claim before you generate anything — a named source, a reproducible method, a number you can trace. Read a real sample of outputs, including the ones that were accepted, before you trust any score or dashboard built on top of them. And when volume is high, check the one output you're about to act on instead of skimming the pile and assuming the rest match it.

A compliance analyst at a 90-person fintech lender

Priti's team gets roughly 200 AI-flagged transaction-risk alerts a day, each one a short writeup with a recommended action. For months the team spot-checked a handful each morning and called the rest fine if the format looked right — sourced, formatted, confident. A quarterly audit found six false clears in one week, all citing the correct regulation but misreading which clause applied, the kind of error a glance at formatting never catches. She now writes down, before she opens a single alert, exactly what "cleared" has to mean that day — which source it must cite, which log it must match — and checks every alert she's about to act on against that line, not against how finished it looks.

An engineering manager at a 35-person logistics software company

Dev's team started using AI-generated dashboards to summarize pull-request risk instead of reading diffs line by line — faster, and the charts looked authoritative. Three weeks in, a dashboard marked a migration PR "low risk" based on lines changed, a number that said nothing about whether the migration was reversible. He now keeps a short list of what "low risk" has to actually verify for his team — reversibility, test coverage on the changed path, a rollback plan — written down once, checked against every dashboard before anyone trusts the color.

The takeaway

Volume and format both got away from the old habit of reading everything start to finish — the fix isn't a faster read, it's a narrower, written-down definition of checked that you can actually run on everything that matters.