The model-risk gate that never stops watching.

The findings live where the checklist never looks.

Examiners describe two recurring weak spots in model risk management, and AI/ML makes both worse because it's now roughly half of a large bank's model inventory. The first is the inventory itself. A model that turns up during an exam, running in production, never recorded, never validated, is one of the most common MRAs (Matters Requiring Attention). The second is ongoing outcomes monitoring: SR 11-7's guidance on watching a live model for degradation is among the least-implemented sections, so models drift quietly until a bad quarter exposes them. MAS's December 2024 AI Model Risk paper names the same two gaps for AI specifically.

A static deployment checklist, even a perfect one, addresses neither. It can't catch a model that was never entered to be checked, and it goes silent the moment the model passes. DBS, under MAS's Project MindForge, runs 430+ AI use cases across 2,000+ models; at that scale, "we reviewed it at launch" is not a control.

Asking an LLM "is this model OK to deploy?" is the one thing you must not do.

The tempting shortcut is to hand the model's documentation to an LLM and let it judge readiness. That fails for structural reasons, and no better prompt can fix them:

• Non-determinismA deploy/no-deploy decision must be reproducible. The same model on the same day must yield the same verdict for the same reason. An LLM's judgement drifts from run to run, which is itself an audit finding.
• No citation"Looks fine" is not a control. An examiner needs the verdict tied to a specific control (C3 validation expired), not a paragraph of plausible prose.
• Silent absenceAn LLM can't flag what it was never shown. The undocumented model isn't in the pack you pasted. Catching it is a reconciliation of telemetry against inventory, not a reading task.
• Drift is arithmeticPSI past 0.25 for two months is a number, not an opinion. Forcing re-validation on threshold breach is a deterministic rule; narrating it is the only safe job for the model.

So the real system is a deterministic rules engine over a published control set, plus a deterministic drift monitor over each model's live performance series. The model is allowed exactly one job. It phrases the gap memo and board note, and it does so only by citing a control the engine already failed, or by abstaining where there's no evidence.

The decision is made in code. The model only writes it up, and only with a citation.

This is the load-bearing line of the system. Every verdict and every drift trigger is computed in code from the model's inventory record and its performance series. Each one is reproducible, re-runnable, and tied to a named control. The language layer is strictly downstream and read-only: it drafts the gap memo and the board paragraph about decisions the engine already made, and each clause must cite a failing control or it isn't written. A human, the MRM Head, is required for the one consequential action, which is waiving a CONDITIONAL.

Six controls resolve to one verdict, with no model in the loop.

Here is the actual gate. It joins a model's inventory fields against the published control set, scores six controls, and resolves PASS / CONDITIONAL / BLOCKED by an explicit policy: a Tier-1 model failing a hard control (inventory, validation, monitoring) is blocked; any model failing two or more is blocked; a single soft gap is conditional and waivable. The parameters are published in thresholds.json, not buried in prose.

function evaluateGate(m) {
  const checks = [];
  checks.push({id:"C1", met: m.in_inventory === true});                    // inventoried
  checks.push({id:"C2", met: m.tier != null});                            // risk-tiered
  let c3 = m.validated === true && m.validation_date &&                    // validated, in date
           monthsSince(m.validation_date) <= T.validation_validity_months[m.tier];
  checks.push({id:"C3", met: c3});
  checks.push({id:"C4", met: m.monitoring_configured === true});           // monitoring on
  checks.push({id:"C5", met: !!m.owner && !!m.approver});                  // owner + approver
  let c6 = m.last_review_date &&                                          // review in date
           monthsSince(m.last_review_date) <= T.review_cadence_months[m.tier];
  checks.push({id:"C6", met: c6});

  const fails = checks.filter(c => !c.met).map(c => c.id);
  if (fails.length === 0) return {verdict:"PASS", fails, checks};
  const hardFailed = fails.some(id => T.hard_controls.includes(id));       // C1, C3, C4
  if ((m.tier === 1 && hardFailed) || fails.length >= 2)
    return {verdict:"BLOCKED", fails, checks};
  return {verdict:"CONDITIONAL", fails, checks};                           // 1 soft gap → waivable
}

One blocked model and one drifting model, each traced to its evidence.

Two cases show the two halves of the system. The first is a static gate verdict. The second is a model that passed the gate and was forced back into validation months later by the drift monitor, which is the failure mode a launch checklist can never see.

What the suite produced, and exactly what it does not prove.

Run over the synthetic suite of 200 inventoried models plus 8 telemetry-only ones, the engine produces: 7 blocked for SR 11-7 / MAS gaps, 23 conditional, 170 passed; inventory completeness 96.2% once the 8 undocumented models are surfaced; 10 live models auto-flagged for forced re-validation on drift; and 100% of gate and drift decisions cited to a control or metric and reproduced on replay. The 30 seeded-deficient models resolve cleanly to 7 blocked plus 23 conditional, which is the policy working, not a fudge.

Where these come from, flatly. This is a synthetic suite, a seeded inventory whose deficiencies and degradations are deliberately injected, so these figures show what a governed control plane looks like rather than a real-world outcome on a real model book. The inputs (models.json, thresholds.json) are published and inspectable, and the gate and drift engine is fully deterministic, so a third party gets the same verdicts and the same hashes on every run. That reproducibility is the point, but it is reproducibility of a constructed world.

The honest limits, stated plainly:

• The hard part of real MRM is independent-validation quality, a judgement-heavy assessment of whether a model is conceptually sound. A synthetic boolean validated? flag sidesteps exactly that. Custodian proves a control architecture and continuous-monitoring discipline, not validation correctness.
• The drift thresholds (PSI 0.25, AUC 0.70, two consecutive months) are illustrative defaults, not calibrated to any real model or business line. In production each is a per-model parameter set with validation evidence behind it.
• Because I author both the data and the rules, the headline numbers are true by construction. The defensible claim is the mechanism (published thresholds, a deterministic decision function, cited verdicts, a replayable ledger), not the specific counts.

A row of the ledger, and what breaks it.

Every gate verdict, drift trigger and MRM sign-off is appended to a hash-chained ledger. Each entry's hash covers its own fields and the previous hash, so rewriting any decision after the fact breaks the chain at exactly that row, which the live Tamper button demonstrates. Replay recomputes every hash from the recorded fields and confirms they reproduce, which is the same guarantee the seeded generator gives upstream.

seq        : 2
action     : drift_revalidation
subject    : MDL-0113        decision : FORCED RE-VALIDATION
evidence   : [month:2026-03, "PSI 0.406 > 0.25; AUC 0.685 < 0.7"]
prev_hash  : 7a1c4e09
hash       : b1b3f3d5   ← = fnv1a(prev_hash + canonical(entry))

See it run.

The numbers above are live output from the engine over the synthetic suite. The working application gives you the 200-row inventory, the animated control checklist with its cited gap memo, the 24-month PSI/AUC lifecycle timeline with the breach marked, the MRM-Head waiver, and the tamper-evident ledger, all client-side with no network.