Test 028

Sep 14, 2026 / 1 fact

Request

Reply to Priya on Discord about adding Salesforce sandbox and NetSuite variant notes to the Mercury empty state. Tell her where those caveats belong.

Required memory

Fact 314

As of July 23, 2026, Mercury’s supported-source-types documentation and support macro cover Salesforce sandbox setup, NetSuite variant checks, and other source-specific caveats without putting those caveats into product copy.

Source evidence (1)

003234Jul 23, 2026 / 09:10 UTC-07:00

Logging this for memory only: Jake, Priya, and Sarah finished the Q3 Mercury source-caveat cleanup this morning. Priya fixed the confusing NetSuite line, so the supported-source-types docs and support macro now cover Salesforce sandbox setup, NetSuite variant checks, and the other source-specific caveats without pushing any of that into product copy. Sarah confirmed the support macro is live for the support rotation. The product empty state stayed simple and generic, and the contractor docs/support QA work is complete now, not just paused until August 7, with no Mercury hiring implication.

Message 003234 in history

Expected tool calls

send_discord_dm

Grading

1. field_equals / send_discord_dm

{
  "type": "field_equals",
  "tool": "send_discord_dm",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_028_00",
  "action_id": "morgan_028_send_discord_dm"
}

2. field_equals / send_discord_dm

{
  "type": "field_equals",
  "path": "result.user",
  "value": "priya",
  "tool": "send_discord_dm",
  "check_id": "morgan_028_01",
  "action_id": "morgan_028_send_discord_dm"
}

3. field_llm_judge / send_discord_dm

{
  "type": "field_llm_judge",
  "path": "result.message",
  "criterion": "Keeps the Salesforce and NetSuite caveats out of product empty-state copy and puts them in the supported-source/setup documentation and support macro. Equivalent descriptions of those two destinations are valid. A short source-specific caveat in product copy still fails.",
  "tool": "send_discord_dm",
  "check_id": "morgan_028_02",
  "action_id": "morgan_028_send_discord_dm"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_discord_dm",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_028_00",
        "action_id": "morgan_028_send_discord_dm"
      },
      {
        "type": "field_equals",
        "path": "result.user",
        "value": "priya",
        "tool": "send_discord_dm",
        "check_id": "morgan_028_01",
        "action_id": "morgan_028_send_discord_dm"
      },
      {
        "type": "field_llm_judge",
        "path": "result.message",
        "criterion": "Keeps the Salesforce and NetSuite caveats out of product empty-state copy and puts them in the supported-source/setup documentation and support macro. Equivalent descriptions of those two destinations are valid. A short source-specific caveat in product copy still fails.",
        "tool": "send_discord_dm",
        "check_id": "morgan_028_02",
        "action_id": "morgan_028_send_discord_dm"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/morgan/028.yaml

SHA-256: c3cf03b9d3d559ac466d9b3875306b68951b1c91ce5ff07060a9eec8b1c35d78