WTWilly Tai
Use case · AI governance & assuranceLive demoPublic dataset

The two questions JE testing keeps failing, answered in code.

When an inspector reviews journal-entry fraud testing, the file has to answer two documentary questions. Can you prove the population you tested was complete? And for every entry that met your own fraud criteria, can you show why it was tested or why it was skipped? The work usually got done; the evidence usually did not survive. The JE Forensics Bench makes both answers a property of the tooling: a trial-balance roll-forward that must land on zero to the cent, and a selection log that refuses to close while any flagged entry is undocumented. It runs live over 7,664 real general-ledger lines from the State of Oklahoma's open books.

$0.00
roll-forward difference
opening TB through 7,704 entries to closing TB, exact
100%
of the population scored
6 criteria × 7,704 entries · no sampling at this stage
145
entries flagged · 1.9%
all 40 labelled synthetic injections caught
0
undocumented skips at finalise
the gate refuses until every flag carries a disposition

Real data: State of Oklahoma General Ledger, FY2026 Q4, CC-BY · posting dates, users, sources synthesised · 40 synthetic suspicious entries injected and labelled

01 · The inspection finding

The work happens. The evidence does not.

PCAOB's January 2025 Audit Focus on journal entries reads like a checklist of things everyone believes they already do. Under AS 2401, teams must test journal entries for management override; under AS 1215, the workpapers must let an experienced auditor understand what was done and who did it. The recurring deficiencies sit exactly there. Inspections keep finding engagements that cannot demonstrate the completeness of the journal-entry population they extracted, and engagements whose files do not show the rationale for which flagged entries were examined. Neither finding says the auditor tested badly. Both say the file cannot prove the testing happened properly.

The workflow explains why. A GL dump arrives from the client and a completeness check is attempted by comparing totals, then quietly abandoned when two systems disagree by an amount nobody can explain. Excel or IDEA filters for weekend postings and round amounts. Someone picks entries to examine, and the reason entry X was tested while entry Y was not lives in that person's head. The memo gets written weeks later from memory. At inspection, the two questions arrive, and the honest answer to both is a shrug with formatting.

02 · Where AI makes this worse

A model summarising a broken population writes confident fiction.

The tempting move in 2026 is to hand the GL dump to a language model and ask for "unusual journal entries with explanations". That fails on both inspection questions at once. A model cannot prove completeness, because completeness is an arithmetic property of the dataset, and a model given a truncated ledger will happily analyse the truncation as if it were the population. And a model asked to explain a selection invents fluent rationale after the fact, which is worse than no rationale, because it reads like judgement while being unattached to any decision anyone made.

The failure is not the model's fluency. It is the placement of the model in the decision path. Completeness, criteria scoring and the decision to finalise are computations with exact answers, so they belong in deterministic code. What the model is genuinely good for is the last step, phrasing why a specific entry met specific criteria, and even there it needs a fence: it may cite only fields that exist on the record, and a checker verifies every cited value before the sentence reaches the file.

03 · The architecture

Code proves and decides. The model phrases. A named human closes.

The bench imports 7,704 journal lines: 7,664 real GL lines from Oklahoma's published FY2026 Q4 ledger, carrying their real amounts, agencies, funds and accounts, plus 40 synthetic suspicious entries that are individually labelled in the interface, because a government ledger published for transparency contains no fraud texture to catch. Posting dates, users and source systems are synthesised deterministically and the page says so; the state does not publish them at entry level. Then five steps run, in this order, on this side of the trust boundary:

JE FORENSICS BENCH · THE ORDER OF OPERATIONS IS THE CONTROL 1 · DETERMINISTIC · PROVE opening TB + every entry must equal closing TB · to the cent 2 · DETERMINISTIC · SCORE 6 fraud criteria × 100% of entries · counts published 3 · HUMAN · DISPOSITION every flagged entry: tested, or a named, reasoned exception 4 · MODEL (STUB HERE) · PHRASE ONLY narrative cites only fields on the record · field checker verifies each value · abstains otherwise 5 · hash-chained run ledger agent id + human id · replays to the same hash FINALISATION GATE · refuses while the population does not reconcile, no reviewer is named, or any flagged entry lacks a disposition · the refusal itself is ledgered
The order matters: nothing downstream runs over a population that has not proved itself first.

Here is the deterministic core from the shipped file, the completeness proof and the gate that everything else defers to:

// The completeness proof: opening TB + every entry must land exactly on the
// closing TB. Returns the difference to the cent; any gap is a loud failure.
function completeness(removedIdx) {
  const bal = new Map(POP.opening);
  POP.entries.forEach((e, i) => {
    if (i === removedIdx) return;
    bal.set(e[1], (bal.get(e[1]) || 0) + e[2]);   // integer cents, no floats
  });
  let totalAbsDiff = 0;
  const gaps = [];
  for (const [ac, stated] of POP.closing) {
    const diff = (bal.get(ac) || 0) - stated;
    if (diff !== 0) { gaps.push({ ac, diff }); totalAbsDiff += Math.abs(diff); }
  }
  return { pass: gaps.length === 0, totalAbsDiff, gaps };
}

// The finalisation gate. Refuses while any flagged entry has neither a test
// mark nor a named, reasoned exception, or while the population does not prove.
function gateDecision() {
  const undisp = DERIVED.flaggedIdx.filter(i => !STATE.dispositions.has(i));
  const comp = completeness(STATE.removedIdx);
  if (!comp.pass) return { ok: false, why: "population does not reconcile", undisp };
  if (!STATE.reviewer) return { ok: false, why: "no named reviewer", undisp };
  if (undisp.length > 0) return { ok: false, why: undisp.length +
    " flagged entries have neither a test mark nor an exception reason", undisp };
  return { ok: true, undisp: [] };
}
04 · Break it yourself

Delete one entry and watch completeness stop being a sentence.

The first button on the page removes one journal entry from the import: JE-00245, a real $662,536,537.00 appropriated transfer posted to account 621000 by the State of Oklahoma's central agency. The roll-forward immediately fails, and it fails specifically: short by $662,536,537.00 at account 621000 Appropriated Transfers Out, which is the removed entry to the cent. Restore it, and the run recomputes to the identical hash it had before, 110ed18f, because the engine is deterministic and the population is the population.

The rest of the run follows the same arc. The six criteria (weekend or holiday posting, round amounts, manual source, rare agency-account pairings, values within 3% below the $250,000 approval limit, rare users) score all 7,704 entries and flag 145, including every one of the 40 labelled injections. Try to finalise straight away and the gate refuses, writes the refusal to the ledger, and tells you why: 145 flagged entries carry neither a test mark nor an exception reason. Name yourself as reviewer, disposition the worklist (the 55 entries meeting two or more criteria in one pass, the remainder with a reason from a controlled list), and the gate opens. Tamper with any ledger record afterwards and verification locates the exact record that no longer hashes.

Why the gap is the demo: completeness deficiencies survive because the assertion and the evidence live in different places. Here they are the same object. If the population is short, the number on screen is red and names the account, and no amount of drafting talent can write around it.
05 · What is real, what is synthetic, what this does not do

The numbers, their provenance, and the limits stated flatly.

The amounts, agencies, funds and accounts on 7,664 of the 7,704 entries are real lines from the State of Oklahoma General Ledger open dataset (FY2026 quarter 4, Creative Commons Attribution licence, data.ok.gov). The extract is small by design, about 174 KB across 26 agencies, pulled by a fetch script with a stated deterministic selection rule so anyone can rebuild it. Posting dates, user ids and source systems are synthesised, because the state does not publish them at entry level; the synthesis is seeded and reproducible, and the interface discloses it. The 40 suspicious entries are wholly synthetic and individually labelled, because a transparency dataset is clean by design and the criteria need something real to catch.

Three limits are worth stating plainly. First, a state government ledger lacks private-company fraud texture even with injections, so the 145-flag worklist demonstrates the mechanism rather than any client's risk profile. Second, the criteria set and its thresholds ($10,000 roundness, the $250,000 approval limit, the rarity floors) are illustrative engagement judgements and no substitute for an engagement-specific fraud risk assessment under AS 2401. Third, the bench does no monetary-unit sampling and no statistical projection. It is about population integrity and selection documentation, the two things inspections keep flagging, and it stops where sampling methodology begins.

What the mechanism does prove: a completeness assertion can be an arithmetic identity a reviewer re-runs in one click, and a selection rationale can be something a tool refuses to lose. Both are properties of the control plane, and they transfer to any ledger you point the bench at.

06 · See it run

Break the population. Get refused. Then close it properly.

Remove JE-00245 and watch the proof fail by $662,536,537.00 at the named account, restore it to the identical run hash, click Finalise early to collect the refusal, then work the 145-entry worklist to a workpaper with zero undocumented skips. Every step you take lands on the hash-chained ledger, including the ones the bench declined.

▶ Open the live demo Runs entirely in your browser · real Oklahoma GL lines (CC-BY) · deterministic engine, identical replay

The auditor's reflex, kept.

I spent years on the other side of this file, being asked the two questions and watching good teams reach for evidence that never got written down. The engineering habit this site demonstrates, deterministic decisions, fenced models, ledgers that replay, is really the audit habit transplanted: never assert what you can compute, never compute what you cannot re-perform. The bench is what JE testing looks like when the workpaper is a running program instead of a memory of one. If your firm is deciding what AI-assisted fieldwork should leave behind for the inspector, this is my answer, and you can break it yourself before you believe it.