Guardrails and Approval
An agent can call any tool you give it, as often as the loop allows. Guardrails are the controls that decide what it may do, how long it may run, and when a person must step in. Anthropic's guidance on agents calls for stopping conditions such as a maximum number of iterations, and for extensive testing in sandboxed environments with appropriate guardrails.
Four kinds of guardrail
- Budget. Limits on steps, tokens, time, and money per run.
- Permission. Which tools exist for this run and which arguments they accept.
- Approval. A person must confirm an action before it runs.
- Validation. Checks on tool inputs and final outputs before they reach a system of record.
The model should never be the only enforcer. A system prompt that says "never delete records" is a request, not a control. The control is a tool that does not exist for that run, or an approval gate that stops the call.
How an approval pauses a run
Microsoft's approval pattern shows the mechanics clearly. A tool marked to require approval does not execute when the model requests it. The run returns early with a request for user input instead of a final answer. The caller shows the pending call to a person, collects the decision, and resumes the run with an approval or rejection. The caller checks every response until no requests remain.
# Framework-neutral sketch of an approval gate
APPROVAL_REQUIRED = {"delete_customer_record", "issue_refund"}
def run_tool(call):
if call.name in APPROVAL_REQUIRED:
return {"status": "pending_approval", "call": call} # do not execute
return execute(call)
# The caller resumes only with a recorded decision:
decision = ask_human(pending_call) # True or False
if decision:
result = execute(pending_call)
else:
result = {"status": "rejected_by_user"}
messages.append(tool_result(pending_call.id, result))
Practice
- List every tool your agent could call and mark each as read or write. Mark any write action that is hard to reverse.
- Add an approval gate to one of those write actions, using the sketch above or your framework's equivalent.
- Test the rejection path. Confirm that a rejected call returns a clear result to the model and does not run. Then confirm the step limit still stops a looping run.
Retrieval check
An agent is about to delete a customer record. Which control is strongest?
Correct. A person confirms the action before it runs. Instructions are requests, while an approval gate is an enforced control.
Not quite. Instructions and logs do not stop the action. Only a gate that holds the call until a person approves it is an enforced control.
When a run pauses for approval, what must the caller do next?
Correct. The run is paused, not finished. The caller gathers the decision and resumes the run with that response.
Not quite. A paused run needs a decision from a person. Retrying or raising limits does not supply one, and the call should not run without it.