DolphinBench
DiscordGitHub

Test 020

Sep 14, 2026 / 1 fact

YAML

Request

Create a venue-screening checklist for the Redwood Ridge team offsite before we choose a venue.

Required memory

Fact 92

Morgan Chen requested adding a locked rule to the offsite booking checklist requiring business-grade internet on-site and direct confirmation from the property operations contact rather than relying on the venue website or marketing PDF.

Source evidence (1)

000464Apr 29, 2023 / 14:00 UTC-07:00

Add this to the offsite booking checklist: business-grade internet on-site is mandatory. Confirm it with the property ops contact directly, not the venue site or marketing PDF -- I don't care what the page says if ops won't back it. Locked rule now.

Message 000464 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_020_00",
  "action_id": "morgan_020_create_doc"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "path": "result.document.body",
  "criterion": "Makes business-grade on-site internet mandatory and requires direct confirmation from the property operations contact, rather than relying on the website or marketing material. A merely optional connectivity check fails.",
  "tool": "create_doc",
  "check_id": "morgan_020_01",
  "action_id": "morgan_020_create_doc"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_020_00",
        "action_id": "morgan_020_create_doc"
      },
      {
        "type": "field_llm_judge",
        "path": "result.document.body",
        "criterion": "Makes business-grade on-site internet mandatory and requires direct confirmation from the property operations contact, rather than relying on the website or marketing material. A merely optional connectivity check fails.",
        "tool": "create_doc",
        "check_id": "morgan_020_01",
        "action_id": "morgan_020_create_doc"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/020.yaml

SHA-256: 5a260b82f0735797e9227cf26383e72120342a2beab01349ec26bcb65f3f6047