DolphinBench

Test 168

Sep 14, 2026 / 2 facts

YAML

Request

Create a concise guide to the five issue categories used for Mercury/Evergreen customer-growth feedback and the rule for assigning follow-up ownership.

Required memory

Fact 196

In the first customer-growth read reported May 10, 2024, Nadia classified Mercury/Evergreen learning into activation friction, admin handoff and role confusion, procurement/security packaging, expansion stalls after first live sync, and true product gaps.

Source evidence (1)

001279May 10, 2024 / 12:18 UTC-07:00

Nadia’s first customer-growth read happened with me, Sarah, Devon, Jake, and Anna. Good shape: she did not turn it into a sales forecast. She classified the Mercury/Evergreen learning into activation friction, admin handoff and role confusion, procurement/security packaging, expansion stalls after first live sync, and true product gaps. The important rule we kept was an owner route attached to each bucket, so customer-growth doesn’t become the place every uncomfortable customer issue goes to die. We left with v1 of the signal taxonomy for turning messy design-partner learning into operating input.

Message 001279 in history

Fact 197

The group retained a rule that every customer-growth signal bucket must have an owner route attached, to prevent customer-growth from becoming a catch-all for uncomfortable customer issues.

Source evidence (1)

001279May 10, 2024 / 12:18 UTC-07:00

Nadia’s first customer-growth read happened with me, Sarah, Devon, Jake, and Anna. Good shape: she did not turn it into a sales forecast. She classified the Mercury/Evergreen learning into activation friction, admin handoff and role confusion, procurement/security packaging, expansion stalls after first live sync, and true product gaps. The important rule we kept was an owner route attached to each bucket, so customer-growth doesn’t become the place every uncomfortable customer issue goes to die. We left with v1 of the signal taxonomy for turning messy design-partner learning into operating input.

Message 001279 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_168_00",
  "action_id": "morgan_168_create_doc"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "path": "result.document.body",
  "criterion": "Identifies five categories: activation friction; admin handoff and role confusion together; procurement/security packaging; expansion stalls after first live sync; and true product gaps. States the rule that every category needs an explicit owner route so customer-growth is not the default destination for all issues. No individual routes are supplied by the source, so stating the rule is sufficient and invented historical assignments are incorrect.",
  "tool": "create_doc",
  "check_id": "morgan_168_01",
  "action_id": "morgan_168_create_doc"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_168_00",
        "action_id": "morgan_168_create_doc"
      },
      {
        "type": "field_llm_judge",
        "path": "result.document.body",
        "criterion": "Identifies five categories: activation friction; admin handoff and role confusion together; procurement/security packaging; expansion stalls after first live sync; and true product gaps. States the rule that every category needs an explicit owner route so customer-growth is not the default destination for all issues. No individual routes are supplied by the source, so stating the rule is sufficient and invented historical assignments are incorrect.",
        "tool": "create_doc",
        "check_id": "morgan_168_01",
        "action_id": "morgan_168_create_doc"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/168.yaml

SHA-256: c2005a95abc1704d895a3d2283beaf5933924b56a04ef0bf3f2d10f60de2d3b9