DolphinBench
DiscordGitHub

Test 154

Sep 14, 2026 / 1 fact

YAML

Request

Create and save a document reconstructing the first Compass v0.1 working-session agenda, including the questions for each signal and the limits on what we were building.

Required memory

Fact 212

Morgan Chen requested a compact Tuesday working-session agenda for Anna, Nadia, Jake, Leo, and Sarah covering, for each Project Compass v0.1 signal, what action it should trigger, who sees it, what evidence and caveat accompany it, what account-note and source quality must be true, what is explicitly not being built, and the boundaries of no board dashboard and no new hiring lane.

Source evidence (1)

001480Jan 20, 2025 / 12:14 UTC-08:00

Can you make a compact Tuesday working-session agenda for Anna, Nadia, Jake, Leo, and Sarah? For each Compass v0.1 signal I want the group to answer: what action should this trigger, who sees it, what evidence and caveat travel with it, what account note/source quality has to be true, and what we are explicitly not building. Keep the no-board-dashboard and no-new-hiring-lane boundary in the agenda.

Message 001480 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_154_00",
  "action_id": "morgan_154_create_doc"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "path": "result.document.body",
  "criterion": "Organizes each signal around its triggered action, audience, evidence and caveat, source-quality requirement, and explicit not-building boundary. Excludes a board dashboard and new hiring activity. Does not replace action decisions with generic signal discussion.",
  "tool": "create_doc",
  "check_id": "morgan_154_01",
  "action_id": "morgan_154_create_doc"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_154_00",
        "action_id": "morgan_154_create_doc"
      },
      {
        "type": "field_llm_judge",
        "path": "result.document.body",
        "criterion": "Organizes each signal around its triggered action, audience, evidence and caveat, source-quality requirement, and explicit not-building boundary. Excludes a board dashboard and new hiring activity. Does not replace action decisions with generic signal discussion.",
        "tool": "create_doc",
        "check_id": "morgan_154_01",
        "action_id": "morgan_154_create_doc"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/154.yaml

SHA-256: cea238d758d254e54300397729b14f1ee68d34aa9ff3689fe9509c922a7f6a2a