Evidence-first workflow skills for an AI coding agent

Problem

AI coding sessions start from a stale memory of the project, stall on one blocked task, call work done without proof and hand back status reports nobody can check. A single assistant reviewing its own work agrees with itself, research drifts toward whatever the first search returned, and status numbers get guessed. Meeting follow-up is worse: requested changes are scattered through an hour of talk, AI summaries drop and invent details, and changes get built on interpretations nobody confirmed.

Goal

A small set of repeatable commands that make an AI assistant work like a careful lead engineer: find out where things really stand, bring in independent second opinions, prove claims, keep every quote and number traceable to its source, and stop for a human at every decision that matters.

My role

Sole designer and author of the whole set: the methods, guardrails, report formats and supporting scripts, iterated through real use and tested with throwaway repositories, prompt regression checks and real read-only runs.

Solution

Four groups of assistant skills and commands built on one idea: an AI assistant's output is trusted only when it is backed by evidence a machine can check, and people approve every step that matters.

A build-session skill runs a fixed procedure from catch-up to a plain-language report.

A review and research command set calls other vendors' command-line agents for independent review and runs research with a built-in refutation step.

A meeting-to-change-list skill turns a recorded meeting into quote-backed changes and works through them one approved change at a time.

A scoping skill turns a request into a costed, evidence-based proposal.

How it works

Build session. A shell script compares local and remote history. A fetch is its only network action; it scrubs credentials from remote addresses before printing anything and handles diverged, mid-merge, detached, remote-less and multi-worktree states, pulling fast-forward only when that is safe. An audit step compares what was asked for with what is built. Every task is sorted into ready, blocked, needs a decision, probably done but unproven, done and proven, no longer needed, or later phase; a blocked task records what is in the way, who holds it, when they were last chased and what would clear it. For big decisions, two models from other vendors propose designs blind, before seeing the agent's own plan, and the choice is made on evidence. Ready work is built, proven with the project's tests and then by real use, and reviewed in up to three rounds, where later rounds see only a snapshot diff of the fixes and fixed stop rules end the loop. Only proven items are ticked, and the report ends with the single best next step. It never deploys, pushes, force-writes or writes to outside tools without asking.

Second opinions and research. A review command first runs checks (resolve the absolute project path, confirm it is a real repository, refuse to fall back to the current folder), calls the other vendor's agent non-interactively with a mode-specific prompt, and the host assistant writes a merged view marking agreements, disagreements and single-model points. Research splits a question into distinct angles, deduplicates sources, extracts falsifiable claims with their sources attached, tries to refute each material claim against more authoritative sources, then ranks findings by corroboration and lists what was dropped. A status command derives file lists and counts from version history, session transcripts or file times, never from memory.

Meeting follow-up. The full transcript is fetched (never the recorder's own summary) and written one line per speaker turn with a clickable time. Each extracted change carries a quote and a time. Before the list is shown, a verifier normalises the text, matches every quote of four or more words piece by piece (handling ellipses and bracketed insertions), checks that every cited time is within a few seconds of a real line, can flag a quote attributed to the wrong speaker, and fails the run on any fabricated quote. Each change then moves through a small state machine: understanding, one recommendation, a before and after preview, approval, the change, and a self-test with evidence. Approval covers making and testing only; going live needs a separate yes. Progress persists, so work resumes across sessions, and a private demo sheet closes the cycle.

Scoping. A skill builds a costed proposal from everything already on record, compares options on a like-for-like basis, and checks its own redaction before any outside model reviews it.

Technical challenges

Making every status claim traceable to evidence rather than to the model's recollection.

Avoiding model groupthink with blind proposals made before the host agent's plan is visible.

Review loops that converge instead of arguing forever: snapshot diffs and fixed stop rules.

Handling someone else's uncommitted work and half-finished merges safely, in portable shell for an old system shell, without ever printing a credential embedded in a remote address.

Pinning an explicit working directory so a headless review never runs against the wrong project.

Preferring vendor model aliases over pinned versions to avoid name rot.

Fuzzy but safe quote matching across punctuation, curly quotes, ellipses and edits.

Redaction that is checked rather than best-effort.

Testing prompt-driven tools at all: byte-identical prompt regression checks and throwaway repositories in over a dozen states.

Capabilities

Grounded project catch-up
Seven-state task triage with blocker fields
Blind multi-model design proposals
Proof by tests and by real use
Bounded multi-round review
Independent second-opinion review from other vendors' agents
Cited research with a refutation pass
Evidence-derived status updates
Quote-backed meeting change extraction with deterministic verification
One-change-at-a-time approval with self-testing
Evidence-based scoping with like-for-like option comparison
Self-checking redaction
Self-contained PDF output

Technology

Impact

Open-ended AI sessions became repeatable, auditable routines. In its first real read-only run the build skill completed the catch-up, ran the project's full test suite, produced its phase report, built and pushed nothing, and held to the house rules over a conflicting injected instruction. Meeting follow-up became traceable to what was actually said, second opinions and fact-checked research became a routine step before shipping, guessed numbers left status reporting, and proposals came from the full evidence base. No metrics are claimed.

Skills demonstrated

AI-assisted engineering process design
Agent guardrails and human-in-the-loop control
Multi-model orchestration and evaluation
Hallucination controls and provenance verification
Privacy engineering (checked redaction)
Git internals and defensive shell scripting
API integration
Test design for prompt-driven tools
Solution scoping and estimation
Technical writing