Guardrails as code

Most organizations that deploy agents have a policy about what those agents may do. It is a reasonable document. It says agents should not spend more than a certain amount, should not contact customers without approval, should escalate when unsure. And it has one flaw: nothing enforces it.

The agent does not read the policy. The system that runs the agent either implements the limits or it does not. Where it does not, the policy is a description of hope. We call the alternative guardrails as code, and it is the difference between an agent you can leave running and one you have to watch.

The four families

In our experience the limits that matter fall into four families. Each one is a mechanism, not a rule.

Spend limits. Every agent has a budget per run and per period, enforced by the layer that makes the model and tool calls. When the budget is hit, the agent stops and files for a human, rather than continuing on credit. This is the guardrail that prevents the surprising invoice, and it is the one most often left as a policy line.

Action limits. Each agent has an allow-list of the tools it may call and the operations it may perform with them. Reading the CRM is not the same permission as writing to it; writing a note is not the same as changing a status. The list is explicit, and anything not on it is refused by the system rather than avoided by the prompt. An agent that can technically do something will eventually do it.

Iteration and time limits. Agents that loop, retry, or plan can run away. A cap on iterations and a cap on wall-clock time per task, with a stop-and-ask when either is hit, turns a runaway into a queued question. This one is cheap and skipped constantly.

Approval gates. For actions with consequences, the system pauses and waits for a person. The gate is placed by consequence and reversibility, and the person sees what is about to happen, what it is based on, and one button. The gate is in the system, so the agent cannot route around it.

The rule that makes the rest work

Underneath the four families is one behavioural rule, and it is the most important guardrail of all: when uncertain, ask; never guess.

An agent that queues a question when it does not know what to do can be trusted with more, because its failure mode is a delay. An agent that fills the gap with its best guess has a failure mode of confident, wrong, and possibly expensive. The question channel has to be built, cheap for the agent to use, and cheap for a person to answer. Once it exists, most of what looked like a need for heavy approval gates turns out to be a need for a good question.

A second behavioural rule sits beside it: report honestly. Telemetry that records real iterations, real costs, and real outcomes, not the tidy version. Guardrails are only as good as the data that shows whether they fired.

Verification that does not trust the author

For agents that produce work rather than take actions, code, documents, changes to records, the guardrail is a verification chain. Ordered gates that the output must pass before it counts: automated checks, an independent review that did not write the thing, and a human walk-through for anything that ships. The principle is that the entity that produced the work is never the only one that judged it.

On our own platform, where agents run a portfolio of products, this shows up as a fixed sequence of gates per change, an independent AI reviewer among them, and a merge policy where only small, clean, low-risk changes proceed on their own. Production promotion is always human-gated. The agents are productive precisely because the limits are firm.

Governance you can prove

The reason to do this as code rather than as policy is not only safety. It is provability. When leadership, a customer, or a regulator asks what the agents can and cannot do, the answer is a configuration you can show, an audit trail that demonstrates the limits fired, and a set of override records that show the gates were real.

A policy document answers the question with a promise. Guardrails as code answer it with evidence. The second one is the only answer that gets more convincing over time.