Governed AI action platform: architecture, data platform and advisory agents

Problem

An online retailer had AI requests arriving from several departments at once. Leadership wanted a clear view of where to focus, the advertising team wanted help deciding which changes to make, a buyer researched suppliers by hand, and daily reporting was a manual grind. The data behind all of it (sales, advertising, inventory, orders, fees and costs) sat across several third-party reporting APIs, each with its own authentication, strict per-endpoint rate limits, scarce report quotas and slow asynchronous reports, plus spreadsheets. There was no trustworthy single place to analyse it, and the team's AI analysis routines ran from manual exports on each person's machine, so instructions drifted and data went stale.

Each request could easily have become a one-off tool with no reuse, no audit trail and no proof of return. Two common first instincts were unsafe once real money moves: a spreadsheet as the system of record, and autonomous agents acting directly on live advertising spend. On top of that, a language model summarising business data can invent or misstate numbers, which makes its advice unusable for decisions.

Goal

Turn scattered requests into a repeatable program: one governed platform built once and reused by every use case, where AI proposes and explains while rules and people decide and execute. In practice that meant nightly, hands-off data loads that stay correct when jobs fail and rerun and never overrun an API's limits; advisory agents that back every figure with an executed query and put every change through a person and coded limits; and costed, terms-compliant designs with clear phase gates before any further agent was built.

My role

I led the program end to end as its architect and lead engineer, working as a solo builder alongside AI coding assistants. I ran the discovery that mapped each department's recurring work to candidate automations, framed the program for leadership, and wrote the engineering build specification, the phased roadmap and the production architecture, putting the design through independent multi-model reviews and resolving their disagreements.

I then designed and built the data platform (ingest jobs, API client and rate limiting, orchestration, the final merge job, warehouse views, infrastructure as code, the security baseline and alerting), the connector library with its guarded write tool, and the three advisory systems (the verifier and control plane, the rules engine and safety gate, the chat approval and question-answering services, the versioned skill releases and the capped executor). I worked with the advertising specialist to turn their process into testable rules. I also led the feasibility research and design for two further agents, which are designs and were not built.

Solution

One story in four parts: designed, built, extended.

Architecture (designed). A single governed-action chassis on managed cloud services. Each domain contributes only connectors and domain rules; the chassis supplies ingestion, a typed recommendation model, an approval flow surfaced in team chat, one guarded execution path, an append-only audit ledger, a configuration-driven policy engine, scheduling and notifications. The doctrine is simple: AI proposes and explains; rules and people decide and execute. A companion principle: build deterministic, rule-based skills first, then put an AI router on top whose only jobs are to interpret a request, route it to the right skill and explain the result, so skill failures and routing failures can be debugged separately.

Data platform (built). A rate-limit-aware connector library for the commerce and advertising reporting APIs, and a serverless ingest platform: scheduled orchestration fans out sharded container jobs, each stages its results in object storage, and one idempotent merge job writes them into partitioned warehouse tables. A relational schema with per-workload roles and an append-only action ledger enforced by database triggers is in place for agents that will act.

Advisory agents (built). A verified executive-brief agent that posts a ranked brief only after a fail-closed verifier approves every figure; an advisory advertising-optimisation agent whose proposals all pass one safety gate and a signed human approval; and centrally versioned analysis skills with a guarded, capped write-back path.

Next agents (design, not built). A measurement-only monitor for how AI answer engines mention the business, and a procurement assistant that researches and drafts but never contacts a supplier without human approval.

How it works

Discovery and architecture (design). Each department's recurring work was mapped to narrowly scoped sub-agents with one job and a small tool set, so prompts stay tight and failures stay contained. Anything that leaves the company defaults to "the agent drafts, a person sends", argued from how much damage each kind of action can do and from review costing less than repairing a wrong external action, with named conditions under which auto-execution could later be earned. Every domain follows one pipeline: ingest, normalise, analyse, recommend, validate against policy, stage for approval, execute, measure. Rules are versioned configuration evaluated deterministically; each recommendation carries its evidence, risk and an expiry; chat clicks are inputs only; and execution is guarded by independent deterministic checks, dry runs and reversibility among them. The language model is confined to explanations, summaries, outreach drafts and exception triage, and never owns calculations, eligibility, payloads or approval state.

Data platform (built). Every call passes through a client that keeps a token bucket per account and endpoint, seeded from the published limits and adjusted from each response's rate headers; it honours retry hints and backs off with jitter. Before an asynchronous report is created, a ledger keyed on a hash of the request checks for an identical existing report, so reruns reuse it instead of burning quota. Personal-data fields are stripped at the edge. A scheduler starts all ingest jobs under one shared run id; jobs write typed rows to object storage rather than to the warehouse; one merge job deletes exactly the slices this run owns and re-inserts rows deduplicated to one per business key, so any rerun converges to the same result, and a freeze window protects settled history. Failures alert to chat, and a preventive organisation-level security and cost baseline managed as code (identity restrictions, central audit logging, quota limits and per-environment budgets with an automatic billing cut-off) wraps the estate. A separate tool makes exactly one live advertising change at a time, defaults to a dry run, and is guarded by locking, verification, an audit record and a revert path.

Advisory agents (built). The brief agent builds sourced metric packets so every number carries the query that produced it, and ranks proposals by a fixed formula so the ranking reproduces. A separate runner rebuilds the verification context and fails closed: every number must equal an executed result, no unknown entities, no digits in free prose, a reproducible ranking and schema-valid output. In chat, the model only picks an intent from a catalogue of cost-capped queries tested in advance; it never writes SQL. The advertising agent reads performance data read-only, checks whether there is enough settled data to act, and lets independent rule modules propose; one gate applies a kill switch, sufficiency thresholds, a per-run exposure cap, a runaway guard and idempotency keys. Approvals are signed, allowlisted and re-gated at click time, and every recommendation is logged and later graded against what happened. The skills load immutable releases through a "current release" pointer with a cached fallback, read data through a freshness-checked adapter, and hand approved changes to an executor that is dry-run first, capped and audited.

Next agents (design, not built). The monitor sends a frozen, versioned panel of buyer questions, drawn from real demand, to several AI answer engines through sanctioned APIs, repeats each query because answers vary, and flags thin samples instead of reporting zero; engines with no sanctioned API go to a manual lane. The procurement assistant fans out across supplier sources, drafts requests for quote from approved templates, and sends nothing until a batch approval card is approved.

Technical challenges

Replacing a spreadsheet-as-database and an agent swarm with a durable, auditable design, without losing the decision gates and approval model of the earlier functional design.

Limits that differ per endpoint and shift at runtime, scarce report quotas and slow asynchronous reports: adaptive token buckets, an idempotent report ledger and a resumable pipeline.

Exactly-once results from at-least-once jobs: staging, owned-slice delete and deduplicated reinsert, with late and restated data weighed against protected history.

Making language model output trustworthy for executives: verification that recomputes every figure independently and fails closed, and a model that selects intents but never writes queries.

Translating a specialist's judgement into deterministic, testable rules instead of letting a model guess, and refusing to act on fresh but incomplete data.

Safe first writes to a live advertising account: layered guards, a read before each write, a check after it, revert and audit, plus a click-time re-check because conditions change between plan and approval.

Exactly-once chat posting under retries: content-hash deduplication, idempotency keys, leases and a compare-and-swap run lock.

Distributing prompt-based tooling so every user runs the same version without reinstalling.

Approval fatigue, answered with batching, risk tiers and prioritisation; right-sizing the stack when one reviewer pushed heavy orchestration and another minimalism.

Designing a measurement that stays inside each provider's terms (at least one major provider's terms rule out analysing its grounded results) and stays meaningful despite non-deterministic answers.

Capabilities

Discovery-led mapping of departmental work to narrowly scoped sub-agents
A drafts-not-sends principle for anything leaving the company
Reusable governed-action chassis with a configuration-driven policy engine (designed)
Multi-account authentication and adaptive per-endpoint rate limiting
Report deduplication and asynchronous report pipelines
Parallel, sharded ingest with an idempotent merge step and a freeze window
Partitioned fact tables and derived business views
Verified, ranked executive brief with grounded chat questions and answers
Rules-based advertising recommendations behind one safety gate
Signed chat approvals with a click-time re-check
Decision audit table with later self-grading
Centrally published, versioned AI skills with an offline fallback
Guarded single-change write tool and a capped, dry-run-first executor, both audited
Organisation-level security and cost guardrails with an automatic billing cut-off
Designs for an answer-engine visibility monitor and a batch-approved procurement assistant

Technology

Impact

Leadership adopted the architecture and the program framing at the design review, and the specification became the reference for the ingest, warehouse and advisory-agent builds that followed. The business moved from scattered reports and manual exports to nightly-refreshed data that analytics and agents were built on, with reruns safe by design and cloud spend under a hard ceiling. Leadership got a ranked view of where to focus in which every figure traces to an executed query; the advertising team got a consistent, explainable action plan with every recommendation recorded and graded; and nothing touches live spend without a person's approval and coded limits. The two further agents exist as costed, terms-checked designs with phase gates. No metrics are claimed.

Skills demonstrated

AI systems architecture and program framing
Governance and human-in-the-loop design
Data engineering and API client engineering
Idempotent, distributed batch design
LLM reliability and fail-closed verification
Rules-engine design and translating domain expertise into software
Safe automation of financial actions
Cloud architecture, infrastructure as code, security and cost governance
Release engineering for AI tooling
Technical feasibility research and terms-of-service analysis
Executive and engineering communication
Multi-model design review and reconciliation