DolphinBench

Test 060

Sep 14, 2026 / 1 fact

YAML

Request

Create our selected household dinner hold for September 16, 6:30-8:30 PM Pacific, at home in Oakland. Add notes for anyone triaging work conflicts against it.

Required memory

Fact 208

Because Jamie’s hospital weeks keep shifting, Morgan Chen and Jamie adopted a flexible household schedule by November 22, 2024: each week they choose one weeknight dinner and one weekend morning as offline household time. Once selected, Scaffold work, investor follow-ups, customer check-ins, and internal catch-ups must route around those blocks unless there is a real incident or equivalent customer-impacting emergency.

Source evidence (1)

001434Nov 22, 2024 / 20:37 UTC-08:00

Jamie and I finally had the household scheduling conversation I’d been dodging after the missed Kibo follow-up and then the stiffness-vet thread. Because his hospital weeks keep moving, we’re making it flexible instead of pretending a fixed block will hold: each week we pick one weeknight dinner and one weekend morning as offline household time. Once we pick them, Scaffold work, investor follow-ups, customer check-ins, and internal catch-ups route around those blocks unless there’s a real incident or the equivalent customer-impacting emergency.

Message 001434 in history

Expected tool calls

  • create_calendar_event

Grading

1. field_equals / create_calendar_event
{
  "type": "field_equals",
  "tool": "create_calendar_event",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_060_00",
  "action_id": "morgan_060_create_calendar_event"
}
2. field_llm_judge / create_calendar_event
{
  "type": "field_llm_judge",
  "path": "result.event.body",
  "criterion": "Marks the selected block as offline household time. Says routine Scaffold work, investor follow-ups, customer check-ins, and internal catch-ups must be scheduled around it. Allows an override only for a real incident or an equivalent customer-impacting emergency.",
  "tool": "create_calendar_event",
  "check_id": "morgan_060_01",
  "action_id": "morgan_060_create_calendar_event"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_calendar_event",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_060_00",
        "action_id": "morgan_060_create_calendar_event"
      },
      {
        "type": "field_llm_judge",
        "path": "result.event.body",
        "criterion": "Marks the selected block as offline household time. Says routine Scaffold work, investor follow-ups, customer check-ins, and internal catch-ups must be scheduled around it. Allows an override only for a real incident or an equivalent customer-impacting emergency.",
        "tool": "create_calendar_event",
        "check_id": "morgan_060_01",
        "action_id": "morgan_060_create_calendar_event"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/060.yaml

SHA-256: 6ed20773d92d68a739b8f423bdabbcd44b6e62f7cdfaaa30a9be910edc131914