DolphinBench
DiscordGitHub

Test 084

Sep 14, 2026 / 1 fact

YAML

Request

Email sarah@scaffold.dev how to answer Acme's request for a written Compass insert in its renewal packet.

Required memory

Fact 261

On October 28, 2025, Morgan Chen requested a draft reply from Sarah Kim to Greg Shipman that politely declines a written Compass-style account-health prompt summary for Acme’s renewal packet, uses only ordinary account-thread language about Acme’s own API v2 export fix and the normal support route, and does not imply that Compass is customer-facing or that a new Acme reliability item is open.

Source evidence (1)

002384Oct 28, 2025 / 09:18 UTC-07:00

Sarah forwarded Greg’s renewal-packet ask. My call is no written Compass artifact for Acme. Draft Sarah’s reply to Greg plus a short internal routing note: politely decline any written Compass-style account-health prompt summary for the packet, keep it to ordinary account-thread language about Acme’s own API v2 export fix and the normal support route, and don’t imply Compass is customer-facing or that a new Acme reliability item is open. Sarah Kim → Morgan Chen, Tue 9:04 AM: "Greg is asking for something adjacent to Compass for their renewal packet. I have not replied. My instinct is no written Compass artifact, but I want your call because he’s specifically tying it to the export timeout history. Can we give them a short account-health prompt summary if it only covers Acme’s own API v2 exports/current reliability posture?" Greg Shipman → Sarah Kim, Tue 8:31 AM: "For the renewal packet, can Scaffold send over a short written account-health summary similar to the prompt format you walked us through live earlier this year? The most useful version would cover the export-timeout history, what changed after the API v2 fix, and your current reliability posture / route if we see another long export issue. We don’t need screenshots or anything design-partner-specific, just a written summary our procurement and ops folks can keep with the packet."

Message 002384 in history

Expected tool calls

  • send_email

Grading

1. field_equals / send_email
{
  "type": "field_equals",
  "tool": "send_email",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_084_00",
  "action_id": "morgan_084_send_email"
}
2. field_equals / send_email
{
  "type": "field_equals",
  "path": "result.to",
  "value": "sarah@scaffold.dev",
  "tool": "send_email",
  "check_id": "morgan_084_01",
  "action_id": "morgan_084_send_email"
}
3. field_llm_judge / send_email
{
  "type": "field_llm_judge",
  "path": "result.body",
  "criterion": "Declines written customer-facing Compass material. Keeps any response to ordinary account-thread/support wording about Acme's own API v2 export fix and normal support route, without inventing a new open reliability issue.",
  "tool": "send_email",
  "check_id": "morgan_084_02",
  "action_id": "morgan_084_send_email"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_email",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_084_00",
        "action_id": "morgan_084_send_email"
      },
      {
        "type": "field_equals",
        "path": "result.to",
        "value": "sarah@scaffold.dev",
        "tool": "send_email",
        "check_id": "morgan_084_01",
        "action_id": "morgan_084_send_email"
      },
      {
        "type": "field_llm_judge",
        "path": "result.body",
        "criterion": "Declines written customer-facing Compass material. Keeps any response to ordinary account-thread/support wording about Acme's own API v2 export fix and normal support route, without inventing a new open reliability issue.",
        "tool": "send_email",
        "check_id": "morgan_084_02",
        "action_id": "morgan_084_send_email"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/084.yaml

SHA-256: 595540a60af0f197fbd511c0744e25fa93c2d9723252fbf2e72a41b45276246d