WTWilly Tai
Case studyAgentic Engineering

A retail copilot that can't invent stock or give away margin.

A customer-facing assistant that does real work. It checks live stock, quotes the list price, and builds a quotation, using only Origin's actual catalogue, and it physically cannot grant a discount beyond its authority. The language model proposes; deterministic typed tools and a policy gate decide and execute. When a customer pushes past policy, it escalates to a human instead of caving. Here is exactly where that boundary sits, and why it holds.

01 / The pain

The two ways a sales chatbot quietly costs you money.

Preparing a quote by hand is slow. A rep checks the warehouse system for on-hand stock, looks up the list price, applies the discount rules, and drafts the document. So companies automate it with a chatbot, and meet the two failure modes that make legal nervous. The assistant invents a stock figure or a price it never actually looked up ("yes, we have 12 in stock"). Or, under pressure to close, it gives away margin: a customer talks it into a 30% discount it was never authorised to grant, and now that concession is in writing. A useful assistant has to be fast and impossible to talk out of policy.

02 / Why the naive approach fails

"Let the model just answer" breaks the moment a customer pushes.

The obvious build is a chatbot with the catalogue pasted into its prompt, left to answer freely. It demos beautifully and fails in production, because a language model will confidently produce a number it has no basis for, and will agree to almost anything to be helpful:

So the real job is more than answering the question. The model should handle language while every number, every commitment, and every policy decision is taken out of its hands. Typed tools read those from real data and a gate rules on them. The last part of the job is being honest about exactly where that line falls.

NAIVE · model answers freely LLM + catalogue in the prompt ✗ invents "12 in stock"✗ talked into 30% off✗ prices a dead SKU✗ promises "by Friday"✗ chat ≠ audit trail GOVERNED · model proposes, tools + gate decide model: language typed tools · real cataloguecheck_stock · quote_price · build_quote policy gate · ≤15% allow>15% → escalate to human cited numbers · decision logged
fig.1 · why free-answering fails, and the boundary that replaces it
03 / How it works

The model writes the sentence. It never sets a price or commits stock.

This is the load-bearing decision of the whole system, so it's worth drawing precisely. On the left, the language model does one job. It reads the customer's message, proposes which tool to call, and writes the customer-facing reply. It never holds a number. On the right, deterministic typed tools read Origin's real catalogue and a policy gate rules on every action. The gate's structured decision, not the model's prose, is the single source of truth. Every action crosses into the append-only ledger. The model can request a tool; only code decides and executes.

MODEL · language only, proposes DETERMINISTIC · reads real data, decides, executes Propose intent + tool callparse request · pick a tool · draft reply check_stock · quote_pricebuild_quote · apply_discount Policy gateclassify → allow / escalate reads on-hand qty & list price from Origin's catalogue, never invents one ≤15% → allow · >15% → escalate to a sales manager Append-only AI Control Ledgerevery tool call & gate decision written here · the logged decision is what the customer is told the model can request a tool; it cannot set a price, commit stock, or grant a discount
fig.2 · the trust boundary: the model proposes language; tools and the gate decide and execute

Inside that boundary, each request maps to one typed tool, classified and gated before anything is committed. The discount rule is the sharp edge, and it's explicit. It lives as a constant in the gate, not as a prompt instruction:

Customer ask "in stock?""how much?""30% off?" route to tool check_stock → on-hand vs reorder_pointreal number or "out of stock" quote_price → list_pricecited from catalogue apply_discount: pct ≤ 15allow apply_discount: pct > 15escalate → sales manager Catalogue sku · namelist_pricestock_on_hand DISCOUNT_AUTO = 15%, the single threshold, held in the gate (POL-SALES-01)
fig.3 · the hard part: typed tools over real data, and one explicit discount threshold
04 / One worked example

The catch, the near-miss it allows, and a stock check it can't fake.

Three real exchanges over Origin's catalogue. A customer asks for 30% off the Halcyon Laptop One. That is above the 15% auto-approve band, so the gate escalates and the assistant says so rather than granting it. A different customer asks for 10%, inside the band, so the gate allows it. And a stock query returns the real on-hand figure, or "out of stock", never a number the model made up.

RequestTool calledWhat the gate sawCited from catalogueDecision
"30% off, please"apply_discount30% > 15% threshold (exceeds_policy)POL-SALES-01escalated
"can you do 10%?"apply_discount10% ≤ 15% thresholdPOL-SALES-01allowed
"Halcyon in stock?"check_stockstock_on_hand read from catalogueSKU-00161 · on-handreal number
Telling those cases apart is the whole point. A naive chatbot either caves on every discount (lost margin) or refuses every one (lost sale). This one allows the 10% because it's inside the band and escalates the 30% because it's over, and for stock it reports stock_on_hand verbatim, never a generated figure. Getting the in-policy "yes" right is exactly what makes the over-policy "no" trustworthy. And whatever the gate decided is what the customer is told, word for word.
05 / Numbers & honest trade-offs

What it does, and what those numbers do and don't prove.

15%
auto-approve band
DISCOUNT_AUTO, the one threshold, held in the gate.
4
typed tools
check_stock · quote_price · build_quote · apply_discount.
0
invented numbers
Every stock & price figure cited from the catalogue.
1:1
chat ↔ ledger
The reply is the gate's logged decision, restated.

Where these come from, honestly. Origin Company is a synthetic retailer I generate. Its products, prices and on-hand quantities are seeded, not pulled from a live ERP. So these figures prove the mechanism works: a model handling language alongside typed tools and a policy gate produces cited numbers and an enforced threshold. They don't prove integration with a specific warehouse or pricing system. That is a connector, not a redesign.

The trade-off every sales lead actually weighs is where to set the band. 15% is a deliberate, conservative auto-approve line (DISCOUNT_AUTO = 15). Raise it and more deals close without a human, but more margin leaks before anyone looks. Lower it and you protect margin at the cost of routing more routine requests to a manager. The point is that the dial is a single number in the gate, changed in one place and instantly reflected in both the customer reply and the audit trail. It is not a sentence in a prompt that the next clever customer might talk around.

06 / Governance, shown

The decision the customer hears is the row in the ledger.

Every tool call, whether a stock check, a quote, or a discount request, is risk-classified, ruled on by the gate, and written to a hash-chained ledger. What gets logged is the gate's structured decision, and that same decision is what the assistant says back. There is no path where the chat grants 30% and the ledger records an escalation, or the reverse. The reply is the logged decision, restated in plain language. Chat equals ledger, by construction.

AI Control Ledger · entryintegrity verified · decision = customer reply
seq        : 23
agent      : Enterprise Assistant
action     : apply_discount
subject    : discount 30%
tier       : medium       decision : escalate → sales manager
policy     : POL-SALES-01  (exceeds_policy: 30% > 15%)
told to customer : "30% is above the 15% I'm authorised to grant,
                    so I've escalated it to a sales manager."
prev_hash  : 7a1f…b8e0
hash       : c3d9…40af   ← = sha256(prev_hash + canonical(entry))
Threat model: the ledger is append-only. Flip the decision from escalate to allow and the hash changes, so the chain breaks at this seq. The customer-facing reply is generated from this same decision field, so the two can't diverge. Tamper test runs live on the home page.
07 / Old vs new

The same quote desk, re-engineered.

FREE-ANSWERING CHATBOT · fast, unsafe model invents the number talked past the discount rule no audit trail GOVERNED COPILOT · fast and bounded typed tools · cited numbers gate enforces 15% · escalates above every action logged · chat = ledger
fig.4 · a chatbot that hopes → a copilot that's bounded by construction

What this says about how I work.

I make the guardrail structural, not a wish. It's easy to tell a chatbot "cite real numbers and don't exceed 15%" in a prompt and hope. The durable design draws the trust boundary first. The model handles language and proposes, while typed tools read real data and a policy gate decides and executes. Then it makes the assistant report the gate's decision, so the conversation and the audit trail are the same fact. Building agents that stay inside policy by construction, not by good intentions, is the judgment I'd bring to putting AI in front of a customer.

See it run.

The exchanges above are real output from the assistant over Origin's catalogue. Ask it for stock and a quote, then try to talk it into a 30% discount and watch it escalate. Every move is written to the same ledger as the rest of the portfolio.

See the live assistant →The full console runs on my machine. Ask me for a 90-second walkthrough or a screen-share.