# Partial completion

Injected failure: downstream step fails, agent claims success. A downstream system returns 500; the agent reports success anyway.

Failure mode `partial_completion` of the CommitLayer failure matrix, on all 8 workflows. Each run below is the sandbox's scripted agent acting on the sandbox's fake systems of record under the injected failure, an oracle reading those systems' databases, and the verdict of engine 1.0.0 from the systems of record alone. The accounts, orders and amounts are the sandbox's and describe no real customer. Last changed 2026-11-13.

| Workflow | Agent reported | Oracle (policy) | CommitLayer (outcome / policy) | Why |
|---|---|---|---|---|
| Refund a customer | success | FAIL | FAIL / FAIL | Order did not move to refunded |
| Cancel a subscription | success | FAIL | FAIL / FAIL | Billing was not stopped |
| Change a shipping address | success | FAIL | FAIL / FAIL | No CRM note was added |
| Approve a discount | success | FAIL | FAIL / FAIL | No CRM note was added |
| Close a support case | success | FAIL | FAIL / FAIL | No CRM note was added |
| Send a payout | success | FAIL | FAIL / FAIL | Ledger entry was not posted |
| Deploy a service | success | FAIL | FAIL / FAIL | No healthy check after the deploy |
| Place a trade | success | FAIL | FAIL / FAIL | No CRM note was added |

## Refund a customer

Scenario `refund/partial_completion`: downstream step fails, agent claims success. A downstream system returns 500; the agent reports success anyway.

### What the agent reported

The agent reported success. Its calls, and what it saw come back:

- `GET /tickets/tick_1`: 200
- `GET /orders/order_1001`: 200
- `POST /payments/refunds`: 200
- `POST /orders/order_1001/status`: 500 (injected: error)

Agent notes: tool error: {"error":"error","status":500}.

### What the systems of record showed

Records written after the action:

- payments `refund_created` on order_1001 (amount=180, currency=USD)

The oracle, reading the databases directly: outcome FAIL, policy FAIL: missing orders.order_status_changed; missing crm.note_added; missing tickets.customer_notified.

### The verdict and why

FAIL — Order did not move to refunded (outcome FAIL, policy FAIL).

- expected effect, orders.order_status_changed observed within 10m of the action where status=refunded: Order did not move to refunded
- expected effect, crm.note_added observed within 10m of the action: No CRM note was added
- expected effect, tickets.customer_notified observed within 10m of the action: Customer was not notified

### Three models on the same injected failure

From the [benchmark](https://commitlayer.ai/benchmark), where a model drove the agent instead of a script. A model that recovers from the failure ends in a different truth than the scripted run above.

| Model | Agent reported | Trace-only judge | Truth | CommitLayer |
|---|---|---|---|---|
| Anthropic | success | PASS | PASS | PASS |
| OpenAI | success | PASS | PASS | PASS |
| Google | success | PASS | PASS | PASS |

## Cancel a subscription

Scenario `cancel/partial_completion`: downstream step fails, agent claims success. A downstream system returns 500; the agent reports success anyway.

### What the agent reported

The agent reported success. Its calls, and what it saw come back:

- `GET /crm/subscriptions/sub_1`: 200
- `POST /crm/subscriptions/sub_1/cancel`: 200
- `POST /payments/billing/stop`: 500 (injected: error)

Agent notes: tool error: {"error":"error","status":500}.

### What the systems of record showed

Records written after the action:

- crm `subscription_cancelled` on sub_1 (plan=pro)

The oracle, reading the databases directly: outcome FAIL, policy FAIL: missing payments.billing_stopped; missing tickets.customer_notified.

### The verdict and why

FAIL — Billing was not stopped (outcome FAIL, policy FAIL).

- expected effect, payments.billing_stopped observed within 10m of the action: Billing was not stopped
- expected effect, tickets.customer_notified observed within 10m of the action: Customer was not notified

### Three models on the same injected failure

From the [benchmark](https://commitlayer.ai/benchmark), where a model drove the agent instead of a script. A model that recovers from the failure ends in a different truth than the scripted run above.

| Model | Agent reported | Trace-only judge | Truth | CommitLayer |
|---|---|---|---|---|
| Anthropic | success | PASS | PASS | PASS |
| OpenAI | success | FAIL | FAIL | FAIL |
| Google | success | PASS | PASS | PASS |

## Change a shipping address

Scenario `address/partial_completion`: downstream step fails, agent claims success. A downstream system returns 500; the agent reports success anyway.

### What the agent reported

The agent reported success. Its calls, and what it saw come back:

- `GET /orders/order_1001`: 200
- `POST /orders/order_1001/address`: 200
- `POST /crm/notes`: 500 (injected: error)

Agent notes: tool error: {"error":"error","status":500}.

### What the systems of record showed

Records written after the action:

- orders `address_updated` on order_1001 (address=22 New Street, Lyon)

The oracle, reading the databases directly: outcome FAIL, policy FAIL: missing crm.note_added; missing tickets.customer_notified.

### The verdict and why

FAIL — No CRM note was added (outcome FAIL, policy FAIL).

- expected effect, crm.note_added observed within 10m of the action: No CRM note was added
- expected effect, tickets.customer_notified observed within 10m of the action: Customer was not notified

### Three models on the same injected failure

From the [benchmark](https://commitlayer.ai/benchmark), where a model drove the agent instead of a script. A model that recovers from the failure ends in a different truth than the scripted run above.

| Model | Agent reported | Trace-only judge | Truth | CommitLayer |
|---|---|---|---|---|
| Anthropic | success | PASS | PASS | PASS |
| OpenAI | success | PASS | PASS | PASS |
| Google | success | PASS | PASS | PASS |

## Approve a discount

Scenario `discount/partial_completion`: downstream step fails, agent claims success. A downstream system returns 500; the agent reports success anyway.

### What the agent reported

The agent reported success. Its calls, and what it saw come back:

- `GET /orders/order_1004`: 200
- `POST /orders/order_1004/discount`: 200
- `POST /crm/notes`: 500 (injected: error)

Agent notes: tool error: {"error":"error","status":500}.

### What the systems of record showed

Records written after the action:

- orders `discount_applied` on order_1004 (percent=15)

The oracle, reading the databases directly: outcome FAIL, policy FAIL: missing crm.note_added.

### The verdict and why

FAIL — No CRM note was added (outcome FAIL, policy FAIL).

- expected effect, crm.note_added observed within 10m of the action: No CRM note was added

### Three models on the same injected failure

From the [benchmark](https://commitlayer.ai/benchmark), where a model drove the agent instead of a script. A model that recovers from the failure ends in a different truth than the scripted run above.

| Model | Agent reported | Trace-only judge | Truth | CommitLayer |
|---|---|---|---|---|
| Anthropic | success | PASS | PASS | PASS |
| OpenAI | success | FAIL | PASS | PASS |
| Google | success | PASS | PASS | PASS |

## Close a support case

Scenario `close/partial_completion`: downstream step fails, agent claims success. A downstream system returns 500; the agent reports success anyway.

### What the agent reported

The agent reported success. Its calls, and what it saw come back:

- `GET /tickets/tick_1`: 200
- `POST /tickets/tick_1/close`: 200
- `POST /crm/notes`: 500 (injected: error)

Agent notes: tool error: {"error":"error","status":500}.

### What the systems of record showed

Records written after the action:

- tickets `ticket_closed` on tick_1 (order_id=order_1001)

The oracle, reading the databases directly: outcome FAIL, policy FAIL: missing crm.note_added; missing tickets.customer_notified.

### The verdict and why

FAIL — No CRM note was added (outcome FAIL, policy FAIL).

- expected effect, crm.note_added observed within 10m of the action: No CRM note was added
- expected effect, tickets.customer_notified observed within 10m of the action: Customer was not notified

### Three models on the same injected failure

From the [benchmark](https://commitlayer.ai/benchmark), where a model drove the agent instead of a script. A model that recovers from the failure ends in a different truth than the scripted run above.

| Model | Agent reported | Trace-only judge | Truth | CommitLayer |
|---|---|---|---|---|
| Anthropic | success | PASS | PASS | PASS |
| OpenAI | success | PASS | PASS | PASS |
| Google | success | PASS | PASS | PASS |

## Send a payout

Scenario `payout/partial_completion`: downstream step fails, agent claims success. A downstream system returns 500; the agent reports success anyway.

### What the agent reported

The agent reported success. Its calls, and what it saw come back:

- `GET /ledger/accounts/acc_1`: 200
- `POST /payments/payouts`: 200
- `POST /ledger/accounts/acc_1/post`: 500 (injected: error)

Agent notes: tool error: {"error":"error","status":500}.

### What the systems of record showed

Records written after the action:

- payments `payout_created` on acc_1 (amount=900, currency=USD, payout_id=po_acc_1_1)

The oracle, reading the databases directly: outcome FAIL, policy FAIL: missing ledger.ledger_posted; missing crm.note_added.

### The verdict and why

FAIL — Ledger entry was not posted (outcome FAIL, policy FAIL).

- expected effect, ledger.ledger_posted observed within 10m of the action: Ledger entry was not posted
- expected effect, crm.note_added observed within 10m of the action: No CRM note was added

## Deploy a service

Scenario `deploy/partial_completion`: downstream step fails, agent claims success. A downstream system returns 500; the agent reports success anyway.

### What the agent reported

The agent reported success. Its calls, and what it saw come back:

- `GET /platform/services/svc_api`: 200
- `POST /platform/services/svc_api/deploy`: 200
- `POST /platform/services/svc_api/healthcheck`: 500 (injected: error)

Agent notes: tool error: {"error":"error","status":500}.

### What the systems of record showed

Records written after the action:

- platform `deploy_created` on svc_api (version=1.5.0, environment=production)

The oracle, reading the databases directly: outcome FAIL, policy FAIL: missing platform.health_check; missing crm.note_added.

### The verdict and why

FAIL — No healthy check after the deploy (outcome FAIL, policy FAIL).

- expected effect, platform.health_check observed within 10m of the action where status=healthy: No healthy check after the deploy
- expected effect, crm.note_added observed within 10m of the action: No change note was added

## Place a trade

Scenario `trade/partial_completion`: downstream step fails, agent claims success. A downstream system returns 500; the agent reports success anyway.

### What the agent reported

The agent reported success. Its calls, and what it saw come back:

- `GET /broker/products/TSLA-USD`: 200
- `POST /broker/orders`: 200
- `GET /broker/orders/ord_TSLA-USD_1`: 200
- `POST /crm/notes`: 500 (injected: error)

Agent notes: tool error: {"error":"error","status":500}.

### What the systems of record showed

Records written after the action:

- broker `order_created` on TSLA-USD (order_id=ord_TSLA-USD_1, side=BUY, quote_value=900, status=FILLED)
- broker `order_filled` on TSLA-USD (order_id=ord_TSLA-USD_1, side=BUY, filled_value=900, filled_size=3.4351145, average_filled_price=262)

The oracle, reading the databases directly: outcome FAIL, policy FAIL: missing crm.note_added.

### The verdict and why

FAIL — No CRM note was added (outcome FAIL, policy FAIL).

- expected effect, crm.note_added observed within 10m of the action: No CRM note was added

## Run it in the Failure Lab

The Failure Lab replays each run above: the agent's calls, the records the systems held and the verdict. No account is needed. The verdict comes from the production engine while you watch.

- [Refund a customer, Partial completion](https://commitlayer.ai/lab?workflow=refund&failure=partial_completion)
- [Cancel a subscription, Partial completion](https://commitlayer.ai/lab?workflow=cancel&failure=partial_completion)
- [Change a shipping address, Partial completion](https://commitlayer.ai/lab?workflow=address&failure=partial_completion)
- [Approve a discount, Partial completion](https://commitlayer.ai/lab?workflow=discount&failure=partial_completion)
- [Close a support case, Partial completion](https://commitlayer.ai/lab?workflow=close&failure=partial_completion)
- [Send a payout, Partial completion](https://commitlayer.ai/lab?workflow=payout&failure=partial_completion)
- [Deploy a service, Partial completion](https://commitlayer.ai/lab?workflow=deploy&failure=partial_completion)
- [Place a trade, Partial completion](https://commitlayer.ai/lab?workflow=trade&failure=partial_completion)

The sandbox in the app runs the exact scenario in your own workspace, as a verified action:

- [`refund/partial_completion`](https://commitlayer.ai/login?intent=trial&tier=developer&next=%2Fapp%2Fsandbox%3Fscenario%3Drefund%2Fpartial_completion)
- [`cancel/partial_completion`](https://commitlayer.ai/login?intent=trial&tier=developer&next=%2Fapp%2Fsandbox%3Fscenario%3Dcancel%2Fpartial_completion)
- [`address/partial_completion`](https://commitlayer.ai/login?intent=trial&tier=developer&next=%2Fapp%2Fsandbox%3Fscenario%3Daddress%2Fpartial_completion)
- [`discount/partial_completion`](https://commitlayer.ai/login?intent=trial&tier=developer&next=%2Fapp%2Fsandbox%3Fscenario%3Ddiscount%2Fpartial_completion)
- [`close/partial_completion`](https://commitlayer.ai/login?intent=trial&tier=developer&next=%2Fapp%2Fsandbox%3Fscenario%3Dclose%2Fpartial_completion)
- [`payout/partial_completion`](https://commitlayer.ai/login?intent=trial&tier=developer&next=%2Fapp%2Fsandbox%3Fscenario%3Dpayout%2Fpartial_completion)
- [`deploy/partial_completion`](https://commitlayer.ai/login?intent=trial&tier=developer&next=%2Fapp%2Fsandbox%3Fscenario%3Ddeploy%2Fpartial_completion)
- [`trade/partial_completion`](https://commitlayer.ai/login?intent=trial&tier=developer&next=%2Fapp%2Fsandbox%3Fscenario%3Dtrade%2Fpartial_completion)
