One Agent Taught You to Supervise. Five Is a Different Job.

The advice you've absorbed about supervising an AI agent — scope it, set a checkpoint, review what matters — was written for one agent. Once you're running three or four at once, following that advice for each one separately is itself the mistake. The scarce resource was never your rules. It's your attention, and attention doesn't scale the way a checklist does.

What's forcing this now

OpenAI disclosed this month that its own researchers now run 3.1 agent-workdays for every human workday — each person effectively directing three agents at once, not one (three incidents, one week). Microsoft's Work IQ is built on the same premise from the other direction: it turns an org chart into an agent platform, so a manager who used to assign work to people is now provisioning agents against the same chart, several at a time (what can it access). Neither of these is a story about one agent getting more capable. They're both about one person's supervision load multiplying without anyone redesigning how it works.

Ethan Mollick's "twilight factory" framing names half of what has to change: an agent should route a decision to you only at the moment that decision would actually change the outcome, not on a fixed schedule (the question isn't autonomy, it's when to stop). That's the right idea for one agent. It doesn't say what happens when five agents are each deciding, independently, that this is the moment they need you — because nothing coordinates those requests, and the one thing you can't do is be interrupted five times as often and still do the job well.

This isn't the same problem as agents that hand off to each other in a chain, where the fix is a checkpoint at each handoff (supervising a system of agents). A portfolio of independent agents doesn't have handoffs to check — it has you, dividing a fixed amount of attention across tasks that don't know about each other and don't take turns.

What actually has to change

Tier the portfolio before you tier any single agent's checkpoints. Decide which of your agents are running a task where a slow or missed check costs real money or trust, and which are running something recoverable. That ranking, not each agent's individual scope, decides where your attention goes first when three things need you at once. A tiered review system built for one high-volume workflow works the same way at portfolio scale — the categories just span agents instead of output types within one (the supervision problem is yours too).

Batch your check-ins instead of answering each agent the moment it asks. An agent that pings you the second it hits a decision point is optimizing for its own run, not for your day. Set fixed windows — twice a day, once an hour, whatever the stakes justify — where you clear every pending decision across the whole portfolio at once, rather than context-switching each time one agent wants an answer.

Name what happens when two agents want you at once. With one agent this never comes up. With several, it will, and "whichever pinged first" is not a decision — it's an accident. Decide in advance which of your agents' tasks wins your attention when they collide, based on the tier you already set.

A finance director at a 70-person manufacturing firm

She runs four agents: one reconciling vendor invoices, one drafting board-deck commentary from the monthly numbers, one monitoring a cash-flow dashboard for anomalies, and one drafting routine vendor correspondence. For the first month, she reviewed each one's output as it landed, in whatever order it arrived — which meant the vendor correspondence, harmless if wrong, was getting the same real-time attention as the cash-flow anomaly agent, whose misses were expensive and slow to notice.

She ranked the four by what a missed check would cost: cash-flow anomalies and board commentary get reviewed the moment they flag anything; invoice reconciliation and vendor correspondence get batched into a single afternoon pass. The total review time didn't change much. Which agent got her attention first, when more than one needed it, did.

A district communications officer running three agents for different schools

He set up separate agents to draft parent newsletters, monitor a shared inbox for urgent messages, and summarize board meeting minutes — each scoped correctly on its own. The trouble showed up only when two flagged him in the same ten minutes: an urgent parent message and a newsletter draft needing a factual check. He'd been treating "whoever asks first" as the rule, which meant a routine newsletter question once beat out a parent's time-sensitive complaint by four minutes, purely because of arrival order.

He wrote one line into how he works, not into any agent's settings: the inbox-monitoring agent always wins a collision, full stop, because nothing else he runs is time-sensitive in the same way. Everything else waits for his next scheduled pass.

The one thing

A checklist for supervising one agent tells you what to check. It says nothing about what to do when three agents need you in the same five minutes — and that collision, not any single agent's mistake, is where a portfolio actually breaks down. Rank the agents before you rank their checkpoints, and decide who wins a tie before you're in one.