Test 036

Sep 14, 2026 / 1 fact

Request

Jake has a new Mercury support-polish fix and wants to post release wording in #eng-releases. Send Jake a Discord DM with the review steps required before posting and the limits on the release claims.

Required memory

Fact 195

As of April 24, 2024, Jake is adopting Leo’s short risk/evidence/readout format for upcoming Mercury support fixes instead of ad hoc launch summaries; the repeatable pre-release check ties claims to verification notes and bars portraying support-polish or invited-member handoff fixes as broader admin-state changes.

Source evidence (1)

001201Apr 24, 2024 / 11:35 UTC-07:00

Leo finished the April Mercury release-risk review on the small support-polish bundle before Jake posted release language. He caught one draft note making the invited-member handoff sound like a broader admin-state fix, trimmed it back to the actual evidence, and had Marcus attach the verification notes before anything went to #eng-releases. Jake is adopting Leo’s short risk/evidence/readout shape for the next Mercury support fixes instead of ad hoc launch summaries, so this is now a repeatable pre-release evidence check: claims tied to verification notes, no turning support-polish or invited-member handoff fixes into broader admin-state changes.

Message 001201 in history

Expected tool calls

send_discord_dm

Grading

1. field_equals / send_discord_dm

{
  "type": "field_equals",
  "tool": "send_discord_dm",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_036_00",
  "action_id": "morgan_036_send_discord_dm"
}

2. field_equals / send_discord_dm

{
  "type": "field_equals",
  "path": "result.user",
  "value": "jake",
  "tool": "send_discord_dm",
  "check_id": "morgan_036_01",
  "action_id": "morgan_036_send_discord_dm"
}

3. field_llm_judge / send_discord_dm

{
  "type": "field_llm_judge",
  "path": "result.message",
  "criterion": "Tells Jake to use the short risk, evidence, and readout review before posting, and to support each release claim with the verification notes for this fix. Limits the claims to the support-polish behavior and does not portray it as a broader admin-state change.",
  "tool": "send_discord_dm",
  "check_id": "morgan_036_02",
  "action_id": "morgan_036_send_discord_dm"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_discord_dm",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_036_00",
        "action_id": "morgan_036_send_discord_dm"
      },
      {
        "type": "field_equals",
        "path": "result.user",
        "value": "jake",
        "tool": "send_discord_dm",
        "check_id": "morgan_036_01",
        "action_id": "morgan_036_send_discord_dm"
      },
      {
        "type": "field_llm_judge",
        "path": "result.message",
        "criterion": "Tells Jake to use the short risk, evidence, and readout review before posting, and to support each release claim with the verification notes for this fix. Limits the claims to the support-polish behavior and does not portray it as a broader admin-state change.",
        "tool": "send_discord_dm",
        "check_id": "morgan_036_02",
        "action_id": "morgan_036_send_discord_dm"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/morgan/036.yaml

SHA-256: 578eb48e3a162cf47c62fe35e106bd086fb8a612a0790683f9c0eee9f1c4693d