Demo
A test harness that refuses what it was not allowed
You play a coding agent testing a chat agent for an invented plant nursery. The harness types one message at a time, reads the flow, counts how often a reply says something and asks the platform’s assistant for changes. Before any of that, one policy check decides whether it may touch the agent at all.
This is a simulation. No AI model runs on this page: the agent builder, both agents, their replies and the assistant are scripted, and a seeded random choice stands in for the way a real agent’s answers vary. The nursery, the agents and every reply are invented, and no phone number is shown or used. Nothing you do leaves your browser or is kept, and reloading the page starts again.
A path through it
- Press the Status button to see what the harness may touch.
- Send the line Where is my order? and see which block of the flow answered.
- Run the order question 10 times and count how often a reply says email address. The order block says to ask for it every time, yet the replies vary from run to run: the count shows how often the agent actually asked.
- Ask the assistant to change the plant care block. It is refused: tick the two assistant boxes in the allowlist, ask again, then apply it and read the before and after.
- Send the line My fern has yellow leaves and see the changed block offer the watering guide.
- Point the harness at nursery-live. It is refused, even once you put it on the allowlist.
- Tick the box that takes the phone number field out of the platform’s replies, and try the test copy again.
The harness
In real use a coding agent decides what to test and judges the answers. The harness only types, reads and refuses, and each command answers with a single JSON result for the agent to read.
Look
Chat, one message at a time
Send a suggested line as the customer, or type your own.
Repeat and count
Runs the lines "Hi" then "Where is my order?" in fresh conversations, and counts the runs whose replies say the words you pick. Replies vary from run to run, so the count says how often the words came up. The harness only counts; you choose what to count.
Change the agent through its assistant
Output
Nothing yet. Each command you run appears here as a numbered entry: the command line, what it did in plain words, and the JSON it printed.
The allowlist
A file on the operator’s own machine. An agent gets only what is ticked here, and each capability has its own tick.
The file as saved
{
"agents": {
"nursery-test": {
"granted": [
"talk"
]
}
}
}The agent builder
Invented platform
Agents
Nursery helper, test copy
Nursery helper, live
The flow of nursery-test
Fingerprint 2f5e4b
Greeting
Welcome the customer and offer help with orders, plant care and opening hours.
Plant care
Answer care questions from the care sheet.
- Care sheet: on
Opening hours
Give the opening hours from the settings.
- Hours: 8:00 to 17:00, every day
Order status
Ask for the email address on the order, then look the order up.
- Ask for: email address
- Look up orders: on
Wrap up
Thank the customer and end the chat.
Fallback
Say what the helper can do and ask again.
Test chat with nursery-test
No messages yet.
What the demo shows
- Policy in code, checked before every command. The agent must be on the allowlist, the capability the command needs must be ticked for it, and the platform must say at that moment that no phone number is attached. Every reason for a refusal is listed, and nothing is touched.
- Capabilities kept apart. Chatting, asking the assistant for a proposal and applying one each need their own tick, because applying one changes the agent. Chatting is not treated as harmless either: a test message can still set off whatever the agent is wired to do.
- It fails closed. If the platform’s reply has no phone number field, the harness counts that as a refusal: a missing field never means there is no number.
- One message at a time, with the block that answered. A reply on its own does not show an agent answering from the wrong part of its flow; the block name does.
- Counting, not guessing. An agent’s replies differ between runs, so a single try settles little. The harness repeats the same lines and counts; the calling agent decides what the count means.
- Changes proved, not claimed. An applied proposal is checked by reading the flow again and comparing fingerprints and fields, before and after.
- Output for an agent to read. Each command answers with a single JSON result laid out the same way every time, so a coding agent can act on it without reading a screen.