DolphinBench

Test 090

Sep 14, 2026 / 1 fact

YAML

Request

Create a draft document titled Sep 14 manager note from the notes below. Notes: The September read should stay bounded. There is a distinction between reporting the evidence we have and turning the review into a new decision process. This note is for the former. Owner responsibilities remain intact; nobody should read the review as a reassignment of the work or a reason to open a parallel owner thread. Account-thread evidence questions should go to Sarah. That belongs in the note so people know where to send questions rather than distributing them across the group. There is no hiring decision today. The review is not an instruction to start one, and the wording should not imply that a staffing approval has happened. The central point is to report the September evidence while keeping the existing responsibilities clear.

Required memory

Fact 68

Starting April 20, 2023, Morgan Chen requested that future drafts use bullets only, omit preambles, and cap each bullet at three lines.

Source evidence (2)

000405Apr 20, 2023 / 10:00 UTC-07:00

IMP-A draft is too long. Just bullets. No preamble.

Message 000405 in history
000406Apr 20, 2023 / 10:00 UTC-07:00

cap each bullet at 3 lines max and redo it. going forward do them like that -- i don't have time to read essays

Message 000406 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_090_00",
  "action_id": "morgan_090_create_doc"
}
2. field_markdown_bullets / create_doc
{
  "type": "field_markdown_bullets",
  "path": "result.document.body",
  "value": {
    "only_bullets": true
  },
  "tool": "create_doc",
  "check_id": "morgan_090_01",
  "action_id": "morgan_090_create_doc"
}
3. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "path": "result.document.body",
  "criterion": "Each bullet is a concise substantive point, not a heading, preamble, concluding wrapper, or essay disguised as a bullet. Apply the remembered three-line preference as brevity, not a guessed display width. Concise single-sentence bullets are valid.",
  "tool": "create_doc",
  "check_id": "morgan_090_02",
  "action_id": "morgan_090_create_doc"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_090_00",
        "action_id": "morgan_090_create_doc"
      },
      {
        "type": "field_markdown_bullets",
        "path": "result.document.body",
        "value": {
          "only_bullets": true
        },
        "tool": "create_doc",
        "check_id": "morgan_090_01",
        "action_id": "morgan_090_create_doc"
      },
      {
        "type": "field_llm_judge",
        "path": "result.document.body",
        "criterion": "Each bullet is a concise substantive point, not a heading, preamble, concluding wrapper, or essay disguised as a bullet. Apply the remembered three-line preference as brevity, not a guessed display width. Concise single-sentence bullets are valid.",
        "tool": "create_doc",
        "check_id": "morgan_090_02",
        "action_id": "morgan_090_create_doc"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/090.yaml

SHA-256: 08e45ea4dd46118611a7694521f29a001acfa9ea50bcf51f7f70a8fc28c82fbd