A technician or tenant photographs a defect and a work order is drafted on the spot: fault class proposed by a model, asset resolved from the register, priority and SLA clock read off a deterministic matrix, permits attached by rule, and a named human on every dispatch. The model must clear a confidence threshold; below it, the photo goes to a person instead of a guess. This one runs live in your browser, with a clickable link below.
Walk any managed building and fault reporting looks the same: someone photographs a stained ceiling tile, drops it into a chat group, and a coordinator retypes it into the CMMS at the end of the day. Between the photo and the work order, the details decay. The location becomes "level 3 somewhere", the asset is guessed, the priority is whatever the coordinator was in the mood for, and the response clock starts when the ticket is keyed, not when the fault was seen. Then the wrong trade turns up, photographs the same fault again, and the loop restarts. Across ten years of FM technology programmes the pattern repeats: the capture is instant, and everything after it is manual, lossy and unaccountable.
The obvious build is to let a multimodal model read the photo and create the work order end to end. It demos beautifully and fails operationally, because the model is being asked to make four decisions it has no business making alone:
None of these are model-quality problems. Bigger models make the tickets more fluent, not more governed. The fix is architectural: let the model propose, and let rules and people decide.
The pipeline tags every step with who did it: SYSTEM, AI or HUMAN. The photo is hashed at capture, a quality gate rejects unusable shots while the reporter is still standing at the fault, and only then does the model classify. Its proposal must clear a 0.75 confidence threshold or the photo escalates to a person with the shortlist attached. From the accepted class onward, nothing is generative: the asset comes from the register via the location QR, the priority is read off the severity-by-criticality matrix, SLA clocks and permits follow from the same tables, and a named supervisor approves the dispatch. Overriding the matrix is allowed, but only with a recorded reason.
Pick the ceiling-stain capture and the pipeline runs end to end: classified as an FCU condensate leak at 0.93 confidence, resolved to FCU-L3-018 (criticality C2), priced off the matrix as S2 × C2 → P2 with a 2-hour response clock, and dispatched by a named supervisor. Pick the blurred one and the quality gate rejects it before anyone wastes a callout on it. Pick the faint ring mark and the model returns 0.54, under the threshold, so it refuses to guess and hands a person the shortlist: live leak, or historic stain to monitor.
| Capture | Pipeline outcome | Result |
|---|---|---|
| Ceiling stain and drip under FCU | 0.93 → FCU-L3-018 · S2×C2 → P2 · respond 2 h | dispatched |
| Tripped MCB on DB-L2-04 | 0.97 → S1×C1 → P1 · isolation permit attached | dispatched · permit |
| Blurred ponding photo | quality gate: sharpness fail | retake requested |
| Faint ring mark on tile | 0.54 < 0.75 → human triage with shortlist | refused · escalated |
Where this stands, honestly. The captures are simulated scenes over a synthetic estate, and the classifier's proposals are precomputed: a static page ships no vision model, and the demo says so on the page. Upload your own photo and it takes the honest fallback path, which every governed pipeline needs anyway: hashed custody, a person supplying the classification, the same deterministic rules downstream. What the demo proves is the governance architecture: threshold-gated proposals, rule-resolved consequences, named approval, and a hash chain computed live in your browser. What it does not prove is vision-model accuracy on real faults; in deployment that model runs on the capture device and its accuracy is measured against a labelled fault library before the threshold is set, not after.
The trade-off you tune is the threshold. Raise it and more photos route to people (safer, slower); lower it and more dispatch straight through (faster, and the wrong-dispatch rate is now a measured number, not a surprise). Because every classification and its confidence land in the trail, that tuning is an evidence-based decision made from the ledger, and the override log tells you when supervisors stop trusting the matrix.
Every step appends to a chain computed in the browser: capture hash, gate verdict, the model's confidence against the threshold, the register and matrix versions used, the named approval, and any override with its reason. When a contractor disputes an SLA breach, or an auditor asks why a P1 was downgraded, the answer is a replay, not a recollection.
01 SYSTEM Capture cap1 ingested · sha-256 9f41c2… e1a09b3c 02 SYSTEM Quality gate passed (sharpness, exposure, framing) 77d2f04a 03 AI Class "ACMV condensate leak" · 0.93 ≥ 0.75 b8c11e97 04 SYSTEM Resolved FCU-L3-018 (C2) · register v2026-07 03d97a5e 05 AI Draft WO-2607-101 · P2 from S2×C2 · matrix v2026-07 5f6b20cd 06 HUMAN Approved and dispatched · Ops Supervisor (named) a94e17f8
I spent ten years putting FM technology into buildings, and the gap was never the capture: phones are excellent at photographs. The gap is everything between the photograph and a defensible work order. So the build puts the intelligence where it pays (classification at the lens, quality gating while the reporter can still retake) and refuses to let it wander into decisions that belong to rules and people: priority, SLA, permits, dispatch. One model proposal, behind a threshold, inside a hash-chained trail. That is the shape I would put in front of a facilities director, because it is the shape their auditor will eventually demand.
Run the six captures, including the one the gate bounces and the one the model refuses. Override a priority and watch the pipeline demand a reason. Then upload any photo of your own and see the honest fallback path: hashed custody, your classification, the same rules downstream.