DolphinBench

Test 086

Jan 1, 2028 / 2 facts

YAML

Request

XSR-2028-004 names metrics-router, mapped owner Wes, failure_mode, and only "needs review" as its condition. Send the current intake owner a private instruction for handling it.

Required memory

Fact 274

Beginning August 6, 2027, Hema assigned Nadia front-door triage for asynchronous cross-service exception intake; before review, Nadia must verify the affected component, mapped implementation or release owner, exactly one selected boundary, and a concrete observed failed or at-risk condition, returning any incomplete or placeholder-only request to the mapped owner without review.

Source evidence (1)

004455Aug 6, 2027 / 09:05 UTC-04:00

Hema has assigned Nadia the front-door triage lane for asynchronous cross-service exception intake. From now on, every request needs four things before it can be reviewed: the affected component, the mapped implementation or release owner, exactly one selected boundary from `invariant`, `provenance`, `failure_mode`, `ownership`, or `escalation`, and a concrete observed failed or at-risk condition. A request with anything missing, including only `needs review` or `possible risk`, goes back to the mapped owner without review. A complete request goes to the owner for its selected boundary; I receive it only when that boundary needs my review. Nadia does not make implementation, release, or rollback decisions.

Message 004455 in history

Fact 275

Nadia does not make implementation, release, or rollback decisions for asynchronous cross-service exception requests.

Source evidence (1)

004455Aug 6, 2027 / 09:05 UTC-04:00

Hema has assigned Nadia the front-door triage lane for asynchronous cross-service exception intake. From now on, every request needs four things before it can be reviewed: the affected component, the mapped implementation or release owner, exactly one selected boundary from `invariant`, `provenance`, `failure_mode`, `ownership`, or `escalation`, and a concrete observed failed or at-risk condition. A request with anything missing, including only `needs review` or `possible risk`, goes back to the mapped owner without review. A complete request goes to the owner for its selected boundary; I receive it only when that boundary needs my review. Nadia does not make implementation, release, or rollback decisions.

Message 004455 in history

Expected tool calls

  • send_slack_dm

Grading

1. field_equals / send_slack_dm
{
  "type": "field_equals",
  "tool": "send_slack_dm",
  "action_id": "alex_086_send_slack_dm",
  "path": "result.ok",
  "value": true,
  "check_id": "alex_086_00"
}
2. field_equals / send_slack_dm
{
  "type": "field_equals",
  "path": "result.user",
  "value": "nadia",
  "tool": "send_slack_dm",
  "action_id": "alex_086_send_slack_dm",
  "check_id": "alex_086_01"
}
3. field_llm_judge / send_slack_dm
{
  "type": "field_llm_judge",
  "path": "result.message",
  "criterion": "The message identifies XSR-2028-004 and tells Nadia to return it to Wes without exception review because \"needs review\" is not a concrete observed failed or at-risk condition. It does not ask Nadia to make an implementation, release, or rollback decision.",
  "tool": "send_slack_dm",
  "action_id": "alex_086_send_slack_dm",
  "check_id": "alex_086_02"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_slack_dm",
        "action_id": "alex_086_send_slack_dm",
        "path": "result.ok",
        "value": true,
        "check_id": "alex_086_00"
      },
      {
        "type": "field_equals",
        "path": "result.user",
        "value": "nadia",
        "tool": "send_slack_dm",
        "action_id": "alex_086_send_slack_dm",
        "check_id": "alex_086_01"
      },
      {
        "type": "field_llm_judge",
        "path": "result.message",
        "criterion": "The message identifies XSR-2028-004 and tells Nadia to return it to Wes without exception review because \"needs review\" is not a concrete observed failed or at-risk condition. It does not ask Nadia to make an implementation, release, or rollback decision.",
        "tool": "send_slack_dm",
        "action_id": "alex_086_send_slack_dm",
        "check_id": "alex_086_02"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/alex/086.yaml

SHA-256: b467b90ddcb2084d0d1203f61007dc8baedbacc179f4fd375d10b8f27910d106