Chapter 9 · AI Engineer

Guardrails for agents

max-steps cap · allow-list tools · block forbidden actions · audit logAIE-80765 of 84

An agent that can call tools can do damage — loop forever, call a tool it shouldn't, or take a destructive action like deleting a database. Guardrails are the safety layer wrapped around the loop, and every production agent needs them. Three pieces:

Step cap — the bound from the last lesson: never run past max_steps, and halt loudly if you hit it.
Tool allow-list — an explicit set of tools the agent is permitted to call. If a proposed tool isn't on the list, you block it: don't run it, don't crash, just skip it and keep going. Default-deny, not default-allow.
Audit log — a record of every decision: which tools ran (RAN: search), which were blocked (BLOCKED: delete_db), and when the agent was halted. When something goes wrong, the log tells you exactly what the agent tried.

The key behaviour: a blocked tool must be skipped, not fatal. The agent shouldn't crash because the model proposed something forbidden — it should refuse that one action, log it, and carry on within its other limits. Refuse, record, continue.

Syntax

if not self.is_allowed(name):
    self.log.append(f"BLOCKED: {name}")
    continue                       # skip — do NOT run it, do NOT crash
self.log.append(f"RAN: {name}")
# ...and after the loop:
self.log.append("HALTED: max_steps reached")

Worked examples

An allow-list membership check

class GuardedAgent:
    def __init__(self, allowed_tools, max_steps=4):
        self.allowed = set(allowed_tools)
        self.max_steps = max_steps
        self.log = []
        self._turn = 0

    def is_allowed(self, name):
        return name in self.allowed

a = GuardedAgent(["search"])
print(a.is_allowed("search"))     # True
print(a.is_allowed("delete_db"))  # False  (not on the allow-list)

Blocking a forbidden tool — skipped, logged, never run

# A misbehaving model: tries search, then forbidden delete_db, then loops forever.
def rogue_model_stub(turn):
    if turn == 0:
        return {"type": "tool", "name": "search", "input": {"q": "hi"}}
    if turn == 1:
        return {"type": "tool", "name": "delete_db", "input": {}}
    return {"type": "tool", "name": "search", "input": {"q": "loop"}}

class GuardedAgent:
    def __init__(self, allowed_tools, max_steps=4):
        self.allowed = set(allowed_tools)
        self.max_steps = max_steps
        self.log = []
        self._turn = 0

    def is_allowed(self, name):
        return name in self.allowed

    def run(self, goal):
        for _ in range(self.max_steps):
            step = rogue_model_stub(self._turn)
            self._turn += 1
            if step["type"] == "final":
                return step["answer"]
            name = step["name"]
            if not self.is_allowed(name):
                self.log.append(f"BLOCKED: {name}")
                continue
            self.log.append(f"RAN: {name}")
        self.log.append("HALTED: max_steps reached")
        return "HALTED"

agent = GuardedAgent(["search"], max_steps=4)
print(agent.run("go"))
for line in agent.log:
    print(line)
# Output:
# HALTED
# RAN: search
# BLOCKED: delete_db
# RAN: search
# RAN: search
# HALTED: max_steps reached

The audit log is your evidence trail

# From the run above, the log proves the safety properties held:
#   - "RAN: search" appears        -> allowed tools executed
#   - "BLOCKED: delete_db" appears -> the forbidden tool was refused
#   - "RAN: delete_db" never appears -> it was NEVER executed
#   - "HALTED: max_steps reached"  -> the runaway was stopped at the cap
# When an agent misbehaves in production, this log is the first thing you read.

Common mistakes & gotchas

Raising on a forbidden tool kills the agent

Blocking should continue the loop, not raise. If you crash on the first forbidden call, one bad model suggestion takes down the whole run. The robust behaviour is to refuse that single action, log it, and let the agent keep working within its remaining limits — skip, don't explode.

Default-allow is the dangerous default

An allow-list (permit only what's named) is safe; a block-list (forbid only what you thought of) is not — you'll always miss a dangerous tool you didn't anticipate. Start from "nothing is allowed" and add tools deliberately. With agents, default-deny is the only sane stance.

A guardrail with no log is half a guardrail

Blocking a tool silently means you never learn the model tried something forbidden. The audit log turns a silent refusal into an investigable event — without it you can't tell a well-behaved agent from one that's constantly being stopped from doing damage. Log every RAN, BLOCKED, and HALT.

Why it matters

The gap between a demo agent and a production one is almost entirely guardrails. An agent loose with real tools — your database, your email, your customers' data — is a liability until it's capped, allow-listed, and audited. These few lines are what let you actually ship an agent without lying awake wondering what it might do at 3am.