"Should the AI do this automatically?" is the wrong shape of question, because it only has two answers and both are wrong.
The useful version has four. We use a ladder, and we set the rung explicitly for every workflow an agent touches — written down, agreed with the client, enforced in code.
Level 0 — Observe
The agent watches and records. It does not produce output anyone acts on.
This sounds useless. It isn't. A week at level 0 tells you how often the workflow actually fires, what the inputs really look like versus what everyone said they look like, and where the model would have struggled. It's how you build an eval set from reality instead of imagination.
Cheap, zero risk, and it prevents the most expensive mistake — automating a workflow that turns out to run four times a month.
Level 1 — Suggest
The agent produces an answer. A human reads it, then does the work themselves.
The output is advice: "this looks like a pricing enquiry from an existing client, they last bought in March." The human still writes the reply.
Level 1 is a good diagnostic and a bad destination. It genuinely helps — context is delivered instead of hunted for. But the human is still doing every keystroke, so time saved is modest, and teams that park here usually conclude AI is overrated. They're right about their setup.
Level 2 — Draft and hold
The agent does the entire job, produces the finished artefact, and stops one inch short of the consequence.
The reply is written, addressed, and formatted. The booking is chosen, the slot held. The assessment is complete with its sources attached. Nothing has left the building. A human looks at it and hits approve, edit, or reject.
This is where almost all the value is, and it's the rung most teams skip past on their way to arguing about full autonomy.
Why it works:
- The work is genuinely done. Approving is seconds; writing is minutes. That ratio is the whole return.
- Errors are free. A bad draft costs one rejection. A bad send costs a customer.
- It generates labelled data automatically. Every approve, every edit, every reject is a training signal about what "good" means here. Six weeks of level 2 gives you the eval set you'd otherwise have to invent — and it's the only honest evidence that you're ready for level 3.
- Accountability stays put. A named person approved it. For anything regulated, that isn't a nice-to-have.
The failure mode of level 2 is the rubber stamp. If your approval queue is a hundred items deep, nobody is reviewing anything — they're clicking. Which is why the queue design matters as much as the model does, and why we treat it as a first-class interface rather than an admin screen.
Level 3 — Act, with a trail
The agent acts on its own. The human sees what happened afterwards, and can undo it.
You earn this rung per workflow, never globally. The criteria we use:
- The workflow has run at level 2 long enough to produce a meaningful approval rate.
- That rate is high and stable — not "mostly fine," but consistently high across weeks and edge cases.
- The action is reversible, or its worst outcome is survivable.
- Every action is traced, and there's a kill switch that drops the workflow back to level 2 instantly.
Note what's not on that list: how good the model is. Rung three is a decision about consequences and evidence, not about intelligence.
Some things stay at level 2 permanently. Anything that moves money, makes a legal representation, or fires a customer stays behind a human, no matter how good the numbers get. That's not caution — it's just correct.
Using the ladder
Three things make it work in practice.
Set the level per workflow, not per agent. The same agent can be at level 3 for scheduling, level 2 for outbound replies, and level 1 for anything involving a complaint. Autonomy is a property of the job, not of the software.
Make the level visible. It should be legible in the product, in the config, and to the client. "Which of these can it do without asking me?" is the first question every operator asks, and the answer should be a screen, not an email thread.
Make it reversible in one action. Down a level should be immediate and non-technical. Up a level should require evidence.
Why this matters more than the model conversation
Most stalled AI projects aren't stalled on capability. They're stalled because nobody could agree on how much the thing was allowed to do, so it stayed in a pilot where it was allowed to do nothing.
Naming the rung ends that argument in an afternoon. It turns an unbounded fear — "what if it goes rogue" — into a specific, answerable question: what's the worst outcome of this exact action, and who sees it before it happens?
Start everything at level 2. Move down when it's dangerous. Move up when you have the numbers.
Part of an ongoing series on production agent systems. Monday: why the pilot worked and the rollout didn't.