DolphinBench

Test 153

Sep 14, 2026 / 3 facts

YAML

Request

Email hr@scaffold.dev the Mercury contractor QA engagement's actual completion date and originally authorized end date so HR can reconcile its closeout record.

Required memory

Fact 309

Morgan Chen reported on July 9, 2026 that she approved a narrow contractor-channel extension through August 7, 2026 solely for a short QA pass on Mercury supported-source documentation and support-macro examples.

Source evidence (1)

003184Jul 9, 2026 / 11:44 UTC-07:00

The Mercury setup-copy/support-quality kickoff just ended. Jake told me the contractor who helped with the Mercury source-picker staging cleanup can also do a short QA pass on supported-source docs and support-macro examples, and I approved a narrow contractor-channel extension through August 7, 2026 for that QA work only. Please send Jake an email, cc Priya and HR, saying Priya reviews copy quality, Jake owns sequencing, and this extension is not a full-time Mercury engineering path, headcount exception, or July hiring reopen.

Message 003184 in history

Fact 313

Jake, Priya, and Sarah finished the Q3 Mercury source-caveat cleanup on July 23, 2026.

Source evidence (1)

003234Jul 23, 2026 / 09:10 UTC-07:00

Logging this for memory only: Jake, Priya, and Sarah finished the Q3 Mercury source-caveat cleanup this morning. Priya fixed the confusing NetSuite line, so the supported-source-types docs and support macro now cover Salesforce sandbox setup, NetSuite variant checks, and the other source-specific caveats without pushing any of that into product copy. Sarah confirmed the support macro is live for the support rotation. The product empty state stayed simple and generic, and the contractor docs/support QA work is complete now, not just paused until August 7, with no Mercury hiring implication.

Message 003234 in history

Fact 315

The Mercury contractor documentation and support QA work was completed by July 23, 2026.

Source evidence (1)

003234Jul 23, 2026 / 09:10 UTC-07:00

Logging this for memory only: Jake, Priya, and Sarah finished the Q3 Mercury source-caveat cleanup this morning. Priya fixed the confusing NetSuite line, so the supported-source-types docs and support macro now cover Salesforce sandbox setup, NetSuite variant checks, and the other source-specific caveats without pushing any of that into product copy. Sarah confirmed the support macro is live for the support rotation. The product empty state stayed simple and generic, and the contractor docs/support QA work is complete now, not just paused until August 7, with no Mercury hiring implication.

Message 003234 in history

Expected tool calls

  • send_email

Grading

1. field_equals / send_email
{
  "type": "field_equals",
  "tool": "send_email",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_153_00",
  "action_id": "morgan_153_send_email"
}
2. field_equals / send_email
{
  "type": "field_equals",
  "path": "result.to",
  "value": "hr@scaffold.dev",
  "tool": "send_email",
  "check_id": "morgan_153_01",
  "action_id": "morgan_153_send_email"
}
3. field_llm_judge / send_email
{
  "type": "field_llm_judge",
  "path": "result.body",
  "criterion": "Distinguishes actual completion on July 23, 2026 from the originally authorized extension through August 7, 2026. The QA work finished early; it was not merely paused until August 7. Equivalent unambiguous date formats are valid.",
  "tool": "send_email",
  "check_id": "morgan_153_02",
  "action_id": "morgan_153_send_email"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_email",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_153_00",
        "action_id": "morgan_153_send_email"
      },
      {
        "type": "field_equals",
        "path": "result.to",
        "value": "hr@scaffold.dev",
        "tool": "send_email",
        "check_id": "morgan_153_01",
        "action_id": "morgan_153_send_email"
      },
      {
        "type": "field_llm_judge",
        "path": "result.body",
        "criterion": "Distinguishes actual completion on July 23, 2026 from the originally authorized extension through August 7, 2026. The QA work finished early; it was not merely paused until August 7. Equivalent unambiguous date formats are valid.",
        "tool": "send_email",
        "check_id": "morgan_153_02",
        "action_id": "morgan_153_send_email"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/153.yaml

SHA-256: 3763399c14d805afddeb9ce882c713dfa8f8f76af38dec1f3e237e6e6d93921c