DolphinBench

Test 131

Sep 14, 2026 / 1 fact

YAML

Request

Email security@scaffold.dev, copying hr@scaffold.dev, my instructions for handling Nadia Singh's product/customer-growth admin access and user record in the September 2026 access review, based on her recorded employment status.

Required memory

Fact 236

Nadia Singh’s final day is June 20, 2025.

Source evidence (1)

001803Jun 20, 2025 / 11:24 UTC-07:00

Closeout is complete. Nadia’s final day is today.

Message 001803 in history

Expected tool calls

  • send_email

Grading

1. field_equals / send_email
{
  "type": "field_equals",
  "tool": "send_email",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_131_00",
  "action_id": "morgan_131_send_email"
}
2. field_equals / send_email
{
  "type": "field_equals",
  "path": "result.to",
  "value": "security@scaffold.dev",
  "tool": "send_email",
  "check_id": "morgan_131_01",
  "action_id": "morgan_131_send_email"
}
3. field_list_includes / send_email
{
  "type": "field_list_includes",
  "path": "result.cc",
  "values": [
    "hr@scaffold.dev"
  ],
  "tool": "send_email",
  "check_id": "morgan_131_02",
  "action_id": "morgan_131_send_email"
}
4. field_llm_judge / send_email
{
  "type": "field_llm_judge",
  "path": "result.body",
  "criterion": "Instructs Security to revoke or deactivate Nadia's admin access and treat her as an inactive former employee. Retaining an inactive historical user record is valid; deletion is not required. These are instructions, not evidence that the access action has already happened. If a final-day date is stated, it must be June 20, 2025, not the current September review date; omitting that date is valid.",
  "tool": "send_email",
  "check_id": "morgan_131_03",
  "action_id": "morgan_131_send_email"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_email",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_131_00",
        "action_id": "morgan_131_send_email"
      },
      {
        "type": "field_equals",
        "path": "result.to",
        "value": "security@scaffold.dev",
        "tool": "send_email",
        "check_id": "morgan_131_01",
        "action_id": "morgan_131_send_email"
      },
      {
        "type": "field_list_includes",
        "path": "result.cc",
        "values": [
          "hr@scaffold.dev"
        ],
        "tool": "send_email",
        "check_id": "morgan_131_02",
        "action_id": "morgan_131_send_email"
      },
      {
        "type": "field_llm_judge",
        "path": "result.body",
        "criterion": "Instructs Security to revoke or deactivate Nadia's admin access and treat her as an inactive former employee. Retaining an inactive historical user record is valid; deletion is not required. These are instructions, not evidence that the access action has already happened. If a final-day date is stated, it must be June 20, 2025, not the current September review date; omitting that date is valid.",
        "tool": "send_email",
        "check_id": "morgan_131_03",
        "action_id": "morgan_131_send_email"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/131.yaml

SHA-256: e7f1a555c12c298c07f70976a8902506cc0b357a6fc5c0fdf128418d4a76a4cd