Skip to content
All failure modes
Failure mode

Wrong customer

Injected failure: agent refunds the wrong order. Action recorded for order_1001, tools called on order_1005.

Mode wrong_customer of the failure matrix, on one of its eight workflows. Each run below is the sandbox's scripted agent acting on the sandbox's fake systems of record under the injected failure, an oracle reading those systems' databases, and the verdict of engine 1.0.0 from the systems of record alone.

Synthetic demo data.

Last changed Markdown versionllms.txt

WorkflowAgent reportedOracleCommitLayerWhy
Refund a customersuccessFAILFAILNo refund was created in payments

Refund a customer

Scenario refund/wrong_customer: agent refunds the wrong order.

What the agent reported

The agent reported success. Its calls, and what it saw come back:

  • GET /orders/order_1005200
  • POST /payments/refunds200
  • POST /orders/order_1005/status200
  • POST /crm/notes200
  • POST /tickets/notify200

Agent notes: reported order_1001 but acted on order_1005.

What the systems of record showed

Records written after the action:

  • payments.refund_created on order_1005
    amount=180, currency=USD
  • orders.order_status_changed on order_1005
    status=refunded
  • crm.note_added on order_1005
    author=lab-agent, text=Refunded 180.0 USD per customer request
  • tickets.customer_notified on order_1005
    channel=email

The oracle, reading the databases directly: FAIL

missing payments.refund_created; missing orders.order_status_changed; missing crm.note_added; missing tickets.customer_notified

Effects on a record the action never named: order_1005:order_status_changed, order_1005:refund_created.

The verdict and why

FAIL No refund was created in payments

Outcome FAIL, policy FAIL; the oracle reads outcome FAIL, policy FAIL.

  • expected effect
    No refund was created in payments
    payments.refund_created observed within 10m of the action
  • expected effect
    Order did not move to refunded
    orders.order_status_changed observed within 10m of the action where status=refunded
  • expected effect
    No CRM note was added
    crm.note_added observed within 10m of the action
  • expected effect
    Customer was not notified
    tickets.customer_notified observed within 10m of the action

Reported as effects with no action behind them: order_1005:customer_notified, order_1005:note_added, order_1005:order_status_changed, order_1005:refund_created.

Run it in the Failure Lab

The Failure Lab replays each run above: the agent's calls, the records the systems held and the verdict. No account is needed. The verdict comes from the production engine while you watch.

The sandbox in the app runs the exact scenario in your own workspace, as a verified action: