Three AI agents, three written promises, and records anyone can check
When a customer's internet drops, three AI agents (customer, IT and network) work the fault together on the MoDaaS platform, using NVIDIA Nemotron models. Before testing, we wrote down three promises they must keep. This site shows each part of how we hold them to it, and lets you check every answer yourself.
What was already running, and what we built
The workshop gave every team the left column. The right column is ours, each with its proof.
Every model, tool and agent call goes through the MoDaaS gateway, which writes one line per call. We use that line as the figure of record, and every asset carries a named approver.
We kept them as the comparison, then patched our copy: a spending cap inside the agent, a record of each step, and real IT-network negotiation.
The agents' diary. We join it to the gateway's lines, so an entry that never landed is visible, and seal both per run.
We froze our quality baseline on it (control 9), and found and wrote down where it is too lenient (AGX-9-v2).
| Scenario (organisers' grader) | Reference agents | Our agents | What each agent decided: reference → ours |
|---|
How it fits together
What we use from AWS, ServiceNow, NVIDIA and TM Forum
Each line names the record that shows it. Shown: a record proves it. Built: the code is ready and a team member runs it. Not yet: written in the gap list.
| Partner | Tool | Status | What we do with it | Record |
|---|
The platform's layers: what we use, and the record that shows it
The workshop's architecture: CRs declare intent (governance plane), operators enforce it through AWS services (enforcement layer), agents run on AgentCore (runtime layer), CloudWatch observes. Open any record to check.
| Layer | Part | Use | What we do with it | Record |
|---|
The two-minute tour
- Control register: the three rules, their thresholds, owners and frozen versions.
- Runs: press Play on AGX-16-BREACH and watch the spending cross the limit.
- Control query: ask whether a promise held between two times.
- Try to break it: hide the overspend and see the seal catch it.
- Live: run agx against the live platform and watch its output stream in.
How to read the colours
- Metthe promise was kept
- Brokenit was not, and we show it
- Can't tellnot enough evidence, so we make no claim. Missing evidence is never a pass.
People appear by name, as the rule book names them; email addresses are not shown. Links leave this site only for AWS and ServiceNow, which need the team's login.
Controls, thresholds, owners, versions
The rule book (register.yaml). Each control has a plain rule, a threshold, a tolerance, an owner and a different reviewer. A change is a new version with its own date; a frozen version is never edited. When asked, it hands out the version in force at a given time. It is never in the agents' path: agents and the gateway never call it; the watchdog and the evaluator read it.
Which version was in force?
The same question the evaluator asks before judging a run.
Check the register
Checks every entry has an owner, a different reviewer, a dated version, a threshold, a measured basis and an enforcement point. How the register picks a version
Compares a value with its threshold and returns a verdict
At the decision point, the evaluator takes one measured value from the run's records and the rule version the register hands out. It compares them and returns Met, Broken or Can't tell. It is stateless: it keeps nothing between calls, so the same records and the same rule always give the same verdict. That is why anyone can recompute it.
Try the comparison
Type a value and see the verdict. Clear a value to see what missing evidence does.
The evaluator on every assessed run
Computed by our code () on each run's sealed records. Click a row to watch the run.
| Run | Value | Threshold | Version | Verdict |
|---|
Every verdict, bound to its control, version and run
Each line of ledger/verdicts.jsonl binds one verdict to its control, the rule version that judged it and the run, with the number and record ids behind it. Lines are only ever added. Sealed: each run's records carry sha256 fingerprints in its manifest, written when collected. Chained: every version of the ledger is in git and agx/agx verify walks that history to show no line was changed. Notes about past lines go in amendments.jsonl, never into the lines. Click a line to see what it binds.
Evidence, not logs
“A readable audit trail that links intent, control decisions and results. Activity logs alone do not qualify.” (the organisers' bar)
A log is what a program says about itself, kept until it rotates. Our evidence is written by parties that cannot edit each other, sealed when collected, tied to a rule frozen before the run, and re-checkable by a stranger.
agx spans).agx verify names the file. Try it.agx verify walks every committed version of the ledger and the run book.tools/redact.py), before sealing.agx/agx verify on a copy, offline, or Re-check on AWS below.The life of one record
- Rule frozencommitted to git with its date
- Run declaredrow in the run book, pushed before it starts
- Rungateway writes its line; agents write their diary
- Collectedcopied, secrets removed, sha256 sealed in manifest.json
- Committedgit records when
- Judgedverdict line added to the ledger, bound to rule version and run
- Re-checkedagx verify, any time, by anyone
What we do not claim: the seal is sha256 plus git history, not write-once storage or a signed chain. The platform's audit ledger write token is readable by every participant (in the gap list), which is why every verdict also stands on the gateway's own line.
| Line | Recorded (UTC) | Run | Control | Rule version | Value / threshold | Verdict | Seal |
|---|
Declared before they ran
The first git commit of each assessed run's row in the run book, against the time the run started.
| Run | Declared | Started |
|---|
Re-check the whole ledger
Every recorded verdict recomputed from scratch, every seal checked, every rule compared with its freeze commit, every past version of the ledger and run book compared.
“Was control X satisfied between T1 and T2?”
Answered on demand from the runs and their verdicts. Every run that started in the window is one observation. If more runs broke the rule than the tolerance allows, the window is broken. If any run cannot be judged, we make no claim. A window with no runs is can't tell, never met.
Every run, and a replay of any one
Each promise was tested on purpose twice: once in a run meant to keep it, once in a run meant to break it, both declared before they ran. Practice runs from before the rules were frozen are can't tell: no rule existed yet to judge them.
Promise by promise
How often each promise held
AI spending per run against the 50,000 limit
Drop in answer quality against the 10% limit
Watch a run
Played back from the run's own records: every request through the gateway, every diary step, every watchdog decision.
Scored by the organisers' grader once the run ends. See below.
Did this run keep its promises?
Try to fake the results. See what catches you.
Pick a run and a trick. A copy of that run's records is changed, the same checks run again, and you see what changed and what caught it. Every record was sealed with a fingerprint when it was collected, so any edit leaves a mark. The originals are never touched.
Run agx against the live platform, now
Press Run live and the command runs for real: the team's worker, in our IDE inside the AWS account, takes it from our service and runs the actual agx against the live cluster, AWS and the repository with its full git history. The output streams back here as it is written. This list only reads. Starting a run, changing the cluster or asking a model stays with a person at the IDE (team rule 6), and you can watch that happen in the diary below.
not connected (offline copy)
unknown
Read only. Each command is checked twice, by the service and again by the worker, against a fixed list.
Pick a command and press Run live.
The agents' diary as it is written
Shown on our public AWS link, where the service reads the agents' audit ledger every 10 seconds.
| Time (UTC) | Agent | Step | What it wrote |
|---|
The commands behind every answer
Everything on this site comes from our tool, agx. On our AWS link, Re-check on AWS asks our service, which holds the same code and sealed records, to run the command now and show its output. On your own computer, the same commands run on a copy of our repository (in our hand-in package), with Python 3.9+ and git and no network. Each card also shows the real output captured when this site was built.
On our live cluster (team access)
| Command | What it does | Effect |
|---|
The same records where they live
Links go only to what was shared for this project: our ServiceNow instance's AI Control Tower, AWS CloudWatch traces, the AWS workshop pages and ServiceNow's documentation. They need the team's login; ask us during judging and we open them live. Each entry names the record on this site that proves it.