WTWilly Tai
Case studyAI Governance & Assurance

An audit is a photograph. Controls drift the next day.

The audit bot tests the whole ledger once. This runs every day, and instead of re-screaming the same 46 exceptions every morning, it diffs each run against the last and surfaces only what's new since the previous run. That delta is the part a one-shot audit can't give you. This page walks through how the diff works, why "just re-run the audit daily" fails, and where the dial sits.

01 / The pain

Sign-off is true for a day. The breach arrives on day two.

An internal audit certifies the books as of a date. The morning after sign-off, a new invoice lands, an approver changes, or a duplicate slips across a period boundary, and the control that was clean yesterday is breached today, invisibly, until the next quarterly review or a loss makes it visible. Point-in-time assurance scales to a calendar, not to a business that transacts every day. What you actually need is a control that re-tests continuously and tells you the moment the picture changes, rather than a sharper version of the same photograph.

02 / Why the naive approach fails

"Just re-run the full audit every day" trains the team to stop looking.

The obvious continuous version is a cron job around the audit bot: run all six checks over the whole population, every morning, and email the results. It demos beautifully and dies in week two, because re-running a stable audit over a slow-moving ledger re-surfaces the same exceptions every single day:

So the real job is not to "run the audit more often." It is to snapshot each run, diff against the previous one, and alert only on the delta, while tracking a per-control trend so drift and re-opens become visible across time, instead of lost inside a daily wall of identical rows.

NAIVE · re-run the whole audit daily Mon · 46 Tue · 46 Wed · 46 same list → filtered to spam the new breachdrowns here ✗ REFINED · snapshot, diff, alert on the delta only snapshot[t-1]seen ids · 27 difftoday vs seen new → alert persisting → suppress 4 NEW reach the queue48 already-seen suppressed trend (exceptions / run) ↑ drift
fig.1 · why a daily re-run fatigues, and the diff that replaces it
03 / How it works

The diff is arithmetic in code. The model only narrates the digest.

This is the load-bearing decision, so it's worth drawing precisely. Counts, the set-difference against the prior snapshot, the value-at-risk delta, the per-control trend and the hash-chain are all computed in Python from the joined records, reproducible to the cent, re-runnable, and comparable run over run. The language model is downstream and read-only: it writes the daily digest paragraph about a delta the engine already computed, and it must cite the exact new record IDs or it writes nothing. The model never decides what counts as new.

DETERMINISTIC · computed, reproducible, comparable MODEL · language only, read-only, cited Today's pop.full ledger Match enginetools.py · 6 checks Snapshot[t]52 exception ids Diff vssnapshot[t-1]set difference Digest draftLLM · new only · cited Humantriage → {new, cleared, persisting} · only `new` flows right Append-only, hash-chained ledger · every run & decisionintegrity verified · replay 52/52 · tamper-evident
fig.2 · the trust boundary: the diff is computed, the digest is narrated

Inside a run, each finding carries a stable invoice_id. The monitor keeps a seen set of every id that has ever been raised, and today's ids minus seen is the alert set. The trend is simply the exception count appended to the run history. The parameters are explicit, not a matter of judgment:

This run ids[t] = 52value_at_riskper-control countsigma · dpmo set diff ids[t] − seennew → alert ids[t] ∩ seenpersisting → suppress seen − ids[t]cleared / re-open watch runs.append(count)trend over time seen[t-1] every id everraised · 27 idsincl. dated keysINV-20260624-02 + dedup window: a re-seen id within the window stays suppressed; outside it, re-opens fire again
fig.3 · the hard part: a set-difference snapshot diff plus a per-control trend
04 / One worked example

The day a new breach appeared, and the 48 the monitor stayed quiet on.

The run on 2026-06-24 is the clean case. That day's generator dropped fresh transactions, and the engine raised 52 exceptions, up from 48. The diff against the prior snapshot put 4 of them in ids[t] − seen, including INV-20260624-02 and INV-20260624-03. Those four alerted, and the 48 already in seen were correctly suppressed. Only then did the model draft the digest, quoting those exact new IDs.

InvoiceIn prior snapshot?ObservationValue-at-risk deltaEvidenceResult
INV-20260624-02no · first seen 06-24new three-way-match breach in today's batch+S$11,401INV-20260624-02 · PO · GRNalerted · NEW
INV-00001yes · seen since 06-13same quantity-over-PO exception, unchangedn/aINV-00001 · PO-00001 · GRN-00001suppressed · already-seen
INV-20260625n/a · 06-25 run0 ids in ids[t] − seen the next morningn/arun 2026-06-25quiet · nothing new
The suppression is the point. A daily re-run would have re-reported all 52 on 06-24 and all 52 again on 06-25, burying the four that mattered and then crying wolf the next morning. The monitor alerted on exactly the 4 new on 06-24, then went silent on 06-25 because new_exceptions = 0: no fresh breach, no alert. A control that knows how to say nothing is what keeps the team reading the digest when it does speak.
05 / Numbers & honest trade-offs

What the run history shows, and what it does and doesn't prove.

14
runs logged
2026-06-13 → 06-25, append-only history.
52
open exceptions, latest
Drifted up from 40 over the window.
0
new since last run
06-25 was quiet, with nothing alerted.
S$495.5k
value at risk, latest
Up from S$481.3k as drift accrued.

Where these come from, honestly. Origin Company is a synthetic enterprise whose books grow with seeded transactions every day. So these figures prove the mechanism, that the daily delta is computed correctly run over run, rather than that it survives a specific real ledger. The exceptions, value-at-risk and sigma (~3.1 to 3.2σ) are illustrative on synthetic data, and the engine is the audit bot's tools.py unchanged. What's real here is the diff: the 40→42→43→45→46→48→52 trend and the per-run new_exceptions count (5, 2, 1, 2, 1, then a 4-spike on 06-24, then 0) come straight from the run history, not a mock.

The trade-off the monitor actually buys on is what counts as "new." The dial is the dedup window: how long a cleared exception's id stays in seen before a re-appearance is treated as new again. Set it too wide and a genuine re-open (a breach that cleared, then came back) is silently suppressed as "already seen", which is a missed event. Set it too narrow and ordinary day-to-day churn re-alerts on ids you've already triaged, which is back to fatigue. There's no free setting. Tightening recall on re-opens costs you false alarms, and the right point depends on how fast the control's population actually turns over. The monitor doesn't pretend a delta equals a problem. It produces a cited, deduplicated candidate list a human triages, and the window is the knob that decides how much lands on their desk each morning.

06 / Governance, shown

The monitoring itself is audited. Here's a row of the trail.

Every run and every governed decision is written to a hash-chained ledger, so the monitor can be audited, not just the transactions. Each entry's hash covers its contents and the previous hash, so altering any record after the fact breaks the chain at exactly that row. The decision fields are the control plane's structured output, and the model never writes them.

AI Control Ledger · monitoring entryintegrity verified · replay 52/52
seq        : 312
agent      : Continuous Controls Monitoring
action     : raise_rfi
subject    : INV-20260624-02     run_date : 2026-06-24
delta      : NEW (ids[t] − seen) · run exceptions 48 → 52 · new 4
tier       : low          decision : escalate → control owner
evidence   : [INV-20260624-02, PO-…, GRN-…]
prev_hash  : 7b14…e3a9
hash       : c5d0…91ff   ← = sha256(prev_hash + canonical(entry))
Threat model: append-only; a flipped decision or a doctored new-count changes the hash → chain breaks at the altered seq. Tamper test runs live on the home page.
07 / Old vs new

The same control, re-engineered from a photograph to a feed.

PHOTOGRAPH · quarterly, point-in-time audit a sample, Q1 …silence until Q2… drift & breaches hide between FEED · daily, diffed, delta-only full pop · every day diff vs last → only NEW alerts trend exposes drift & re-opens
fig.4 · from a quarterly snapshot to a governed daily delta

What this says about how I work.

The audit bot and this monitor share an engine, the same tools.py and the same population, but they are answering different questions. The distinct judgment here is temporal. A one-shot audit answers "what's wrong now," and the moment you run it on a schedule, the real problem becomes telling signal from noise over time. So I draw the trust boundary first: the diff and the trend are computed in code, reproducible run over run, and the model sits strictly downstream, narrating only the new set and citing it. Then I confront the hard part out loud. A re-run that can't say what changed trains people to stop looking, and the dedup window is a real trade between recall and fatigue with no free setting. Re-engineering a control to run continuously and stay quiet until something genuinely changes is the judgment I'd bring to leading an AI transformation.

See it run.

The numbers above are real output from the engine's run history over Origin's growing ledger. The full interactive console runs as a working application: the run timeline, the drift trend, the new-since-last-run delta, and a live tamper test on the ledger.

See the live governance demo →The full console runs on my machine. Ask me for a 90-second walkthrough or a screen-share.