DolphinBench
DiscordGitHub

Test 193

Sep 14, 2026 / 1 fact

YAML

Request

Text Maya at +14155501346 what help I offered Mom after Christmas in December 2023 and whether Jamie was included. This is a historical reminder, not a new commitment.

Required memory

Fact 164

Morgan requested a text to Maya saying that Morgan can help Mom with one specific question after the 2023 Christmas holiday if Maya sends the exact form first, that Jamie is excluded from the plan, and that the quiet week after Christmas is not a paperwork block.

Source evidence (1)

000940Dec 21, 2023 / 17:31 UTC-08:00

Maya says Mom is already asking whether the quiet week after Christmas can include a paperwork afternoon. Please text Maya: I can help Mom with one specific question after the holiday if you send the exact form first. Jamie is not part of the plan, and I’m not treating that week as a paperwork block.

Message 000940 in history

Expected tool calls

  • send_sms

Grading

1. field_equals / send_sms
{
  "type": "field_equals",
  "tool": "send_sms",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_193_00",
  "action_id": "morgan_193_send_sms"
}
2. field_equals / send_sms
{
  "type": "field_equals",
  "path": "result.to",
  "value": "+14155501346",
  "tool": "send_sms",
  "check_id": "morgan_193_01",
  "action_id": "morgan_193_send_sms"
}
3. field_llm_judge / send_sms
{
  "type": "field_llm_judge",
  "path": "result.body",
  "criterion": "Limits the historical offer to one specific question after receiving the exact form first, not an open-ended paperwork afternoon/week. Keeps Jamie outside the plan and does not offer new help today.",
  "tool": "send_sms",
  "check_id": "morgan_193_02",
  "action_id": "morgan_193_send_sms"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_sms",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_193_00",
        "action_id": "morgan_193_send_sms"
      },
      {
        "type": "field_equals",
        "path": "result.to",
        "value": "+14155501346",
        "tool": "send_sms",
        "check_id": "morgan_193_01",
        "action_id": "morgan_193_send_sms"
      },
      {
        "type": "field_llm_judge",
        "path": "result.body",
        "criterion": "Limits the historical offer to one specific question after receiving the exact form first, not an open-ended paperwork afternoon/week. Keeps Jamie outside the plan and does not offer new help today.",
        "tool": "send_sms",
        "check_id": "morgan_193_02",
        "action_id": "morgan_193_send_sms"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/193.yaml

SHA-256: 159b8b790ce4197b3998c4df0912c0c728de3f3401d0d5856b2dc7b50ff73a76