Test 136

Sep 14, 2026 / 1 fact

Request

Review scaffold/compass#2661 against the established Q4 behavior and post the missing observed result.

Required memory

Fact 258

Leo reported that source_freshness_warning renders as caveat text rather than rejection in the live Compass Q4 2025 queue.

Source evidence (1)

002359Oct 2, 2025 / 12:22 UTC-07:00

Internal launch thread: Jake: Owner queue is live for the Q4 group. Default owner action queue includes proposed, accepted, completed, blocked_waiting_customer_follow_up. rejected_thin_source_or_date is not in the default owner queue. Anna's evidence/caveat view still has the rejected rows. Leo: source_freshness_warning is rendering as caveat text, not rejection. No data-seam blocker on the live queue. Anna: I can audit rejected-thin rows and caveated rows from the evidence/caveat view. Please don't call the rejected view an owner action backlog. Sarah: Seeded the first account-thread examples for owner use: renewal-risk/admin-friction examples and post-first-live-sync expansion prompt examples. No customer-facing material.

Message 002359 in history

Expected tool calls

list_open_prs
post_pr_comment

Grading

1. tool_called / list_open_prs

{
  "type": "tool_called",
  "tool": "list_open_prs",
  "check_id": "morgan_136_00",
  "action_id": "morgan_136_list_open_prs"
}

2. field_equals / post_pr_comment

{
  "type": "field_equals",
  "tool": "post_pr_comment",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_136_01",
  "action_id": "morgan_136_post_pr_comment"
}

3. field_equals / post_pr_comment

{
  "type": "field_equals",
  "tool": "post_pr_comment",
  "path": "result.comment.pr_id",
  "value": "scaffold/compass#2661",
  "check_id": "morgan_136_02",
  "action_id": "morgan_136_post_pr_comment"
}

4. field_llm_judge / post_pr_comment

{
  "type": "field_llm_judge",
  "tool": "post_pr_comment",
  "path": "result.comment.body",
  "criterion": "States that source_freshness_warning rendered as caveat text in the live Q4 2025 queue and was not treated as a rejection. Do not require unrelated queue-status or audit-view details.",
  "check_id": "morgan_136_03",
  "action_id": "morgan_136_post_pr_comment"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "tool_called",
        "tool": "list_open_prs",
        "check_id": "morgan_136_00",
        "action_id": "morgan_136_list_open_prs"
      },
      {
        "type": "field_equals",
        "tool": "post_pr_comment",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_136_01",
        "action_id": "morgan_136_post_pr_comment"
      },
      {
        "type": "field_equals",
        "tool": "post_pr_comment",
        "path": "result.comment.pr_id",
        "value": "scaffold/compass#2661",
        "check_id": "morgan_136_02",
        "action_id": "morgan_136_post_pr_comment"
      },
      {
        "type": "field_llm_judge",
        "tool": "post_pr_comment",
        "path": "result.comment.body",
        "criterion": "States that source_freshness_warning rendered as caveat text in the live Q4 2025 queue and was not treated as a rejection. Do not require unrelated queue-status or audit-view details.",
        "check_id": "morgan_136_03",
        "action_id": "morgan_136_post_pr_comment"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/morgan/136.yaml

SHA-256: 2585ae5ad9f078ba79519b22b3d6d3d59292f31d4c866b2b6b1dc83dda4b6cba