Governed action pattern for an AI customer-support agent

Problem

Support teams want an AI agent to do more than answer questions: they want it to carry out a customer's request. Some of those requests trigger an action that is high-risk and hard to undo, and every request arrives as untrusted free text that can carry instructions aimed at the model. An agent that both reads the message and decides whether to act is one persuasive message away from a wrong action. And an offline evaluation can look healthy while hiding exactly the errors that matter, because a single quality score blends harmless mistakes with dangerous ones.

Goal

Add the capability so that a wrong action cannot happen by construction, every action is recorded and recoverable, and launch readiness is judged on the errors that matter rather than on an average.

My role

Sole author: design, implementation, tests and the evaluation method.

Solution

The work is split so that each part does only what it can be trusted with.

The model extracts. It turns the message into a structured request, with the supporting text quoted from the message. It never decides.

Deterministic code authorises. A pure policy function looks at the request and the known facts, and either approves automatically, sends the case to a person or refuses, always with a named reason.

One gateway acts. It is the only code that can perform the action, and it records its intent in a ledger before it calls anything.

Before launch, the whole decision path is run over past cases whose correct outcome is already known, and two numbers come out instead of one average.

How it works

Extraction. The model returns a structured request: the fields the policy needs and, for each one, the evidence quoted from the message.

Authorisation. The policy walks a fixed order of checks, hard refusals first. Each check either passes or stops with a named reason; there is no fall-through and no default that allows. A test proves that every reason can actually be reached, so no rule is dead code.

Action. One guarded executor is the only code that can perform the action, and it accepts only decisions the policy approved automatically. It writes down what it is about to do before calling anything and refuses to act if that record cannot be written. Operators can stop all actions at once, volume is capped, and a retry can never perform the same action twice. When a call ends without a clear answer, the record says so and the case is settled afterwards rather than guessed.

Replay before launch. The whole path runs over historical cases. One number counts actions that should never have happened (the target is zero); the other counts cases sent to a person that did not need one (a cost, not a danger). They are never averaged, so a better-looking score can never hide a dangerous miss.

Technical challenges

Making a hard-to-undo action recoverable: a record written before the call, and unclear outcomes settled afterwards instead of guessed.

Resisting prompt injection in untrusted inbound text: the model's output can never authorise, and every extracted field carries the text it was taken from, so a person can see why.

Proving the decision table is complete: every reason reachable, no silent default.

Capabilities

Structured request extraction with evidence quoted from the message
Deterministic authorisation with named reasons and no default allow
A single guarded executor with a write-ahead record
An operator stop switch, volume caps and retries that cannot act twice
Settling unclear outcomes after the fact
Replay over past cases before launch, with two separate error counts

Technology

Impact

The capability is safe by construction rather than by prompt wording, and it is backed by a test suite that found real defects in the author's own code before release. The replay gives a launch decision two honest numbers instead of one flattering one. No metrics are claimed.

Skills demonstrated

Governance of agentic AI
Separating model judgement from deterministic authorisation
Evaluation design and asymmetric error reporting
Ledger and reconciliation design for uncertain outcomes
Prompt-injection resistance
Fail-closed engineering
Plain risk communication