Decide when your AI feature needs human approval

An AI assistant drafts a customer email and asks “Proceed?” The user agrees. Before sending, the assistant rewrites the message, adds a recipient, and attaches a document retrieved afterward. The user approved one proposal, but the system executed another.

Human approval is useful only when it is concrete and enforceable. It should correspond to a specific action whose consequences the reviewer can understand. This article uses an illustrative support assistant that drafts replies and may issue account credits, showing how to choose approval boundaries without interrupting every harmless step.

Classify actions by consequence

Separate reading, drafting, reversible internal changes, external communication, and consequential writes. Searching permitted tickets is usually lower consequence than emailing a customer. Suggesting a credit differs from applying it to an account balance.

Consider sensitivity, reversibility, financial effect, and who can be harmed by a mistake. A small internal label change may be easy to undo. A message sent outside the organization cannot be unsent reliably. The relevant boundary follows the consequence, not whether the tool name contains “write.”

Also consider uncertainty. If the model infers a recipient from ambiguous text, review may be useful even for an otherwise routine action. Deterministic business rules and known inputs can support more automation, while unclear intent should lead to clarification or a constrained proposal.

Keep preparation autonomous where appropriate

Approval should come after the system has done the authorized preparation. The assistant can retrieve permitted context, draft the reply, validate required fields, and present the exact recipient and attachments. Asking permission before every read creates friction without making the final action easier to assess.

Define which preparation is allowed. Drafting should not secretly send, publish, or commit a change. A tool named prepareCredit should create a proposal rather than modify the ledger. The preparation contract must be enforced by the server.

If a proposal cannot yet be made concrete, ask for missing intent rather than presenting a vague approval. “Do you want to help this customer?” does not authorize a particular financial action. The system needs enough information to show what it will actually do.

Present a reviewable proposal

Show the exact action, target, content, and material consequences. For an email, include recipients, subject, body, and attachments. For a credit, include account, amount, currency, reason, and relevant policy limit.

Make important differences visible. A changed recipient or a larger amount should not be buried in prose. Use a compact comparison when updating an existing proposal. Avoid hiding consequential details behind a general “looks good” button.

Keep source evidence available where it affects the decision. If the proposed reply promises a resolution date, show the ticket or policy supporting that promise. A fluent draft should not make unsupported commitments appear authoritative.

Bind approval to the exact proposal

Store an immutable proposal identifier or digest covering the fields that matter. Approval should reference that proposal and trusted user identity. Execution should retrieve the approved proposal rather than accept fresh arbitrary arguments from the model.

If the proposal changes, require new approval according to the product policy. Adding a recipient, attachment, amount, or destructive target changes the action. Minor presentation changes can follow a documented rule, but the system should not improvise what counts as minor.

Give approval an appropriate lifetime. Account state, permissions, and the underlying document can change. A credit approved yesterday may no longer be valid after another adjustment. Revalidate important preconditions at execution rather than treating approval as permanent authority.

Recheck authorization and state when executing

Human approval does not replace permission checks. The approving user must be allowed to perform the action, and the execution service must enforce that rule. A model cannot use a manager's name in text as proof of authority.

Check state transitions atomically where needed. If the credit proposal assumes an unpaid balance, verify that assumption before writing. If the ticket has already closed, determine whether sending the draft remains appropriate. Changed state should produce a clear stale-proposal outcome.

Make execution retry-safe. A network failure after sending may leave the UI uncertain. Use operation identity and provider capabilities to avoid repeating the external action. Show the actual outcome or an explicit unknown state rather than encouraging blind retries.

Use AI to identify weak approval points

A coding assistant can inspect the workflow and ask where the approved data can change. Give it proposal creation, approval storage, and execution code. Require it to trace exact fields through the lifecycle.

An illustrative prompt is: “Find whether the recipient, attachment, or credit amount can differ between approval and execution. Identify any model-controlled arguments accepted after approval. Propose a narrow binding and stale-state check.” This is more useful than asking whether the workflow has a human in the loop.

Test approval bypass, expired approval, changed proposal, unauthorized approver, and duplicate execution. The guide on narrow agent tools explains how capabilities should support these checks independently of model behavior.

Keep approval meaningful over time

Too many prompts can train users to click automatically. Reserve approval for meaningful boundaries and make each proposal worth reviewing. Routine read-only steps can often proceed within the established task, while external or consequential actions deserve a clear checkpoint.

Monitor cancelled, changed, and rejected proposals using safe metadata. These outcomes can reveal confusing interfaces or systematic model mistakes. Do not treat approval rate alone as a quality measure; users may approve because the system makes review difficult.

A robust approval flow preserves human control over the actual consequence. It prepares useful work, presents a concrete proposal, binds consent to that proposal, and rechecks the conditions that can change. The result is a system that can act efficiently without turning a vague “yes” into unlimited authority.

Provide a clear rejection path. The reviewer should be able to cancel or edit a proposal without accidentally executing it, and the assistant should understand that rejection is not permission to try another tool. Preserve useful draft work when appropriate, but mark cancelled proposals as non-executable. If the user asks for a revision, create a new proposal with visible changes. This helps the interface distinguish collaboration on a draft from authority to carry out its external consequence.

Test the interface when the reviewer is interrupted. A proposal may remain open while the user switches accounts, loses their session, or receives updated ticket data. The final confirmation should identify the current actor and target clearly, and execution should reject stale authority. Do not allow a remembered button state to carry approval across a changed account context. These ordinary interruptions exercise the same binding guarantees as adversarial attempts, making the approval flow safer without requiring users to learn the implementation details.

Sources

Anthropic: Building effective agents discusses human checkpoints and environmental feedback in agent systems.

Anthropic: Writing effective tools for agents discusses explicit tool contracts and evaluation.