DolphinBench
DiscordGitHub

Test 182

Sep 14, 2026 / 2 facts

YAML

Request

Create a short product-history note on Project Mercury's original customer journey and engineering point person.

Required memory

Fact 45

As of April 9, 2023, Morgan Chen was calling the Q3 release Project Mercury and defined it as a self-serve customer path from workspace creation through first-source connection, first live sync, teammate invitations, and admin setup without hand-holding.

Source evidence (1)

000316Apr 9, 2023 / 15:00 UTC-07:00

Q3 is basically Project Mercury in my head at this point -- the release that gets a customer from creating a workspace through connecting the first source, the first live sync, teammate invites, and the admin path without us hand-holding it. keeps swallowing every other tab.

Message 000316 in history

Fact 46

Morgan Chen said that having Jake run engineering for Project Mercury helps.

Source evidence (1)

000317Apr 9, 2023 / 15:00 UTC-07:00

Jake's running point on Mercury engineering-wise, which honestly helps. I don't need twelve opinions on every cut.

Message 000317 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_182_00",
  "action_id": "morgan_182_create_doc"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "path": "result.document.body",
  "criterion": "Describes workspace creation through first source, first live sync, teammate invitations, and admin setup without hand-holding, with Jake as engineering point person.",
  "tool": "create_doc",
  "check_id": "morgan_182_01",
  "action_id": "morgan_182_create_doc"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_182_00",
        "action_id": "morgan_182_create_doc"
      },
      {
        "type": "field_llm_judge",
        "path": "result.document.body",
        "criterion": "Describes workspace creation through first source, first live sync, teammate invitations, and admin setup without hand-holding, with Jake as engineering point person.",
        "tool": "create_doc",
        "check_id": "morgan_182_01",
        "action_id": "morgan_182_create_doc"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/182.yaml

SHA-256: 686218b60fdadb4592556853f67870cdd0a5ac356d3352b4bd9d82afdbfa4fef