Test 110

Sep 14, 2026 / 1 fact

Request

Email support@scaffold.dev a customer-safe explanation of what the April 2025 Mercury admin-visibility fix was verified to cover, without overstating its scope.

Required memory

Fact 222

On April 21, 2025, Morgan Chen approved saying in Evergreen support notes, if useful, that Mercury’s basic invited-teammate/source-owner cues are verified in the current activation path for setup guidance, source connection, and first live sync; the wording must not include custom admin policy, a granular role editor, advanced admin controls, procurement/security language, or broader Evergreen admin support.

Source evidence (1)

001582Apr 21, 2025 / 14:32 UTC-07:00

Sarah asked whether Evergreen support notes can now say the basic admin visibility fix is verified. Yes, if useful, and only as basic invited-teammate/source-owner cues in Mercury's current activation path for setup guidance, source connection, and first live sync. Do not add custom admin policy, granular role editor, advanced admin controls, procurement/security language, or broader Evergreen admin support. Send Sarah the approved Evergreen support line.

Message 001582 in history

Expected tool calls

send_email

Grading

1. field_equals / send_email

{
  "type": "field_equals",
  "tool": "send_email",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_110_00",
  "action_id": "morgan_110_send_email"
}

2. field_equals / send_email

{
  "type": "field_equals",
  "path": "result.to",
  "value": "support@scaffold.dev",
  "tool": "send_email",
  "check_id": "morgan_110_01",
  "action_id": "morgan_110_send_email"
}

3. field_llm_judge / send_email

{
  "type": "field_llm_judge",
  "path": "result.body",
  "criterion": "Limits verified coverage to basic invited-teammate/source-owner cues in the current activation path across setup guidance, source connection, and first live sync. Does not imply custom admin policy, a granular role editor, advanced controls, procurement/security scope, or general Evergreen admin support. No separate paragraph listing every excluded feature is required.",
  "tool": "send_email",
  "check_id": "morgan_110_02",
  "action_id": "morgan_110_send_email"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_email",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_110_00",
        "action_id": "morgan_110_send_email"
      },
      {
        "type": "field_equals",
        "path": "result.to",
        "value": "support@scaffold.dev",
        "tool": "send_email",
        "check_id": "morgan_110_01",
        "action_id": "morgan_110_send_email"
      },
      {
        "type": "field_llm_judge",
        "path": "result.body",
        "criterion": "Limits verified coverage to basic invited-teammate/source-owner cues in the current activation path across setup guidance, source connection, and first live sync. Does not imply custom admin policy, a granular role editor, advanced controls, procurement/security scope, or general Evergreen admin support. No separate paragraph listing every excluded feature is required.",
        "tool": "send_email",
        "check_id": "morgan_110_02",
        "action_id": "morgan_110_send_email"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/morgan/110.yaml

SHA-256: 83bc7b344358e1a0a80ad653179f75e36df02f51eec88a530d6165fac3d5db65