What to Log When Agents Do Your Work

Records feel like an IT concern until the first time someone asks a question you can't answer.

Which version of the figures went to the client on the eleventh? Did anyone check this before it went out? Has this agent been doing that the whole time, or since the update?

Each has an answer or it doesn't, and which one is decided months earlier by whether anybody kept a record. The gap here is rarely capability — most tools keep more history than people realise. It's that nobody has decided what's worth being able to reconstruct.

What you're actually recording for

Three different needs, and they want different things:

Reconstruction. Something went wrong and you need to know what happened. Wants detail, needs to survive months, gets read rarely and urgently.

Drift detection. Is it still doing what it did in March? Wants comparability over time — the same measure, taken repeatedly. Detail is unhelpful here.

Demonstrating oversight. Showing a client, an auditor, or a regulator that a human was involved. Wants proof of the check, not the content of the work.

Most people accidentally optimise for the first and end up with a large pile that answers none of the three, because reconstruction detail without comparability tells you nothing about drift.

The five worth keeping

What it was told. The brief or configuration, versioned. Agent behaviour changes when the instructions change, and the single most common answer to "why did it start doing that?" is that someone edited the brief six weeks ago. Without versions, that's unanswerable.

What it read. Not the content — the sources. Which files, which records, which date range. This is what tells you a summary of "last quarter's tickets" was a summary of twelve of them.

What it changed or sent. Every action that touched the world outside the run, with a timestamp. This is the non-negotiable one, and it's the one people most often assume the tool is keeping when it isn't.

Who checked it, and against what. A name and a standard. Not "reviewed" — reviewed against these three criteria. This is the difference between evidence of oversight and a tick.

What it couldn't do. Failures, stops, empty results, permission errors. This list is the most informative thing you'll have and the least likely to be collected, because failures don't produce output and nothing prompts you to record an absence.

What not to keep

Worth stating, because over-collection is the more common failure and it carries its own risk.

Don't keep full content of everything the agent read. It's an enormous retention liability, it duplicates data that already lives in a system of record, and it will contain personal data you now have obligations about. Keep references, not copies.

Don't keep more than your retention policy allows just because storage is cheap. A log is discoverable, and a thorough record of six years of everything is a genuine exposure, not a precaution.

Doing it without an engineer

None of this needs a build.

Most tools have a history or activity view. Find out how long it keeps things — often shorter than you'd guess, sometimes thirty days — and whether it can export.

For the parts the tool doesn't cover, a shared sheet is a legitimate answer. Date, task, who ran it, what was checked, anything odd. It takes twenty seconds per run and it answers the oversight question, which is the one most likely to be asked by someone external.

Version the brief by keeping it in a document with revision history rather than pasted into a settings box. That single move solves the "when did it start doing that?" question permanently.

Anders — the question he couldn't answer

Anders is operations manager at a professional services firm. A client asked whether an AI-assisted report had been reviewed before it went out, and by whom.

It had been. He knew the person, and there was nothing anywhere to say so — the tool's history covered thirty days, the review had happened in a conversation, and the report itself carried no record of it.

He now keeps a four-column sheet: date, output, reviewer, criteria checked. It takes about twenty seconds per item and it turned an uncomfortable exchange into a link.

Priyanka — the edit nobody remembered

Priyanka runs a data team at a media company. A categorisation agent started producing subtly different output, and the team spent a week hunting for a model change or a data-quality issue.

Neither. Someone had edited the brief five weeks earlier — a small clarification, entirely well-intentioned, with a side effect nobody predicted. The brief lived in the tool's settings box, so there was no history and no author.

It now lives in a versioned document that the configuration points at. Her comment: the answer had been sitting there for five weeks in a form nobody could see.

The one thing

Keep the brief with versions, what it read, what it changed, who checked it against what, and what it couldn't do. Keep references rather than copies, and don't hoard beyond your retention policy.

None of it needs an engineer. It needs someone to decide, in advance, which questions you want to be able to answer.