DolphinBench

Test 062

Sep 14, 2026 / 1 fact

YAML

Request

For Friday dinner, repeat the order I was considering on April 14, 2023. Text Jamie at +14155550142 what to pick up.

Required memory

Fact 59

Morgan Chen tentatively planned to order green curry from Lemongrass again for dinner on April 14, 2023.

Source evidence (1)

000366Apr 14, 2023 / 19:00 UTC-07:00

last week's green curry from Lemongrass kinda ruined me a little. same move tonight i think. friday, etc. Jamie's home at a normal hour for once so that helps

Message 000366 in history

Expected tool calls

  • send_sms

Grading

1. field_equals / send_sms
{
  "type": "field_equals",
  "tool": "send_sms",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_062_00",
  "action_id": "morgan_062_send_sms"
}
2. field_equals / send_sms
{
  "type": "field_equals",
  "path": "result.to",
  "value": "+14155550142",
  "tool": "send_sms",
  "check_id": "morgan_062_01",
  "action_id": "morgan_062_send_sms"
}
3. field_llm_judge / send_sms
{
  "type": "field_llm_judge",
  "path": "result.body",
  "criterion": "Requests green curry from Lemongrass or Lemongrass Thai, not merely mentioning it as an alternative being rejected.",
  "tool": "send_sms",
  "check_id": "morgan_062_02",
  "action_id": "morgan_062_send_sms"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_sms",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_062_00",
        "action_id": "morgan_062_send_sms"
      },
      {
        "type": "field_equals",
        "path": "result.to",
        "value": "+14155550142",
        "tool": "send_sms",
        "check_id": "morgan_062_01",
        "action_id": "morgan_062_send_sms"
      },
      {
        "type": "field_llm_judge",
        "path": "result.body",
        "criterion": "Requests green curry from Lemongrass or Lemongrass Thai, not merely mentioning it as an alternative being rejected.",
        "tool": "send_sms",
        "check_id": "morgan_062_02",
        "action_id": "morgan_062_send_sms"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/062.yaml

SHA-256: 630d8229a24e6caa6967661526918b7208a14736f73e899160958d4f4554518b