Test 021

Sep 14, 2026 / 1 fact

Request

PR scaffold/connectors#5271 adds underscore-style signature headers and a query-string fallback to Pinecone verification. Post a review comment correcting the proposed behavior.

Required memory

Fact 143

Morgan Chen requested a narrow response to Rishi Patel clarifying that preserved `X-Scaffold-Signature` and fully lowercased `x-scaffold-signature` are accepted against the same body and workspace, while underscore/env-var-style names are not, without introducing alternate signature locations or broader header normalization.

Source evidence (1)

000759Oct 4, 2023 / 10:07 UTC-07:00

Rishi’s Pinecone logs are here: From: Rishi Patel <rishi@atlas-test.com> To: Morgan Chen <morgan@atlas-test.com> Subject: Pinecone connector logs — header casing again Date: Wed, 4 Oct 2023 09:42:18 -0700 Morgan — Forwarding the Pinecone excerpt below. The odd part is that preserved `X-Scaffold-Signature` and fully lowercased `x-scaffold-signature` both pass, but support is still seeing people assume any normalized variant is fine. Can you sanity-check whether the docs need one tighter line? I do not want to accidentally imply we accept underscore/env-var style names or anything beyond the actual header contract. Forwarded log excerpt: ``` sandbox=atlas-connector-rerun-2 body_sha256=8eeb1f7c4d0b2d9c4cf0c6f7c4e0a9c1d8c6fd0b5f0a4c1e9a2d6b7c8f1e0a3 2023-10-04T15:58:12Z req=pc_84fa9 POST /v1/connectors/atlas/webhook headers: X-Scaffold-Workspace=atlas-sandbox; X-Scaffold-Signature=sha256=6f2c0f9d... verify: ok status: 200 2023-10-04T15:58:49Z req=pc_84faa POST /v1/connectors/atlas/webhook headers: X-Scaffold-Workspace=atlas-sandbox; x-scaffold-signature=sha256=6f2c0f9d... verify: ok status: 200 2023-10-04T15:59:11Z req=pc_84fab POST /v1/connectors/atlas/webhook headers: X-Scaffold-Workspace=atlas-sandbox; x_scaffold_signature=sha256=6f2c0f9d... verify: fail status: 401 error: missing required signature header `x-scaffold-signature` 2023-10-04T15:59:34Z req=pc_84fac POST /v1/connectors/atlas/webhook headers: X-Scaffold-Workspace=atlas-sandbox; X_SCAFFOLD_SIGNATURE=sha256=6f2c0f9d... verify: fail status: 401 error: missing required signature header `x-scaffold-signature` ``` Operator note from their side: same request body/workspace in all four attempts; header-only verification path on every run. Inspect this and draft a narrow response for Rishi. The fix should clarify that preserved `X-Scaffold-Signature` and fully lowercased `x-scaffold-signature` are accepted against the same body/workspace, but underscore/env-var style names are not. Do not introduce alternate signature locations or broader header normalization.

Message 000759 in history

Expected tool calls

post_pr_comment

Grading

1. field_equals / post_pr_comment

{
  "type": "field_equals",
  "tool": "post_pr_comment",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_021_00",
  "action_id": "morgan_021_post_pr_comment"
}

2. field_equals / post_pr_comment

{
  "type": "field_equals",
  "path": "result.comment.pr_id",
  "value": "scaffold/connectors#5271",
  "tool": "post_pr_comment",
  "check_id": "morgan_021_01",
  "action_id": "morgan_021_post_pr_comment"
}

3. field_llm_judge / post_pr_comment

{
  "type": "field_llm_judge",
  "path": "result.comment.body",
  "criterion": "Says to use the hyphenated X-Scaffold-Signature header, allows normal case-insensitive handling of that header, and rejects underscore-style names and the query-string fallback.",
  "tool": "post_pr_comment",
  "check_id": "morgan_021_02",
  "action_id": "morgan_021_post_pr_comment"
}

4. field_regex / post_pr_comment

{
  "type": "field_regex",
  "path": "result.comment.body",
  "pattern": "(?i)(?<![A-Za-z0-9_])x-scaffold-signature(?![A-Za-z0-9_-])",
  "tool": "post_pr_comment",
  "check_id": "morgan_021_03",
  "action_id": "morgan_021_post_pr_comment"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "post_pr_comment",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_021_00",
        "action_id": "morgan_021_post_pr_comment"
      },
      {
        "type": "field_equals",
        "path": "result.comment.pr_id",
        "value": "scaffold/connectors#5271",
        "tool": "post_pr_comment",
        "check_id": "morgan_021_01",
        "action_id": "morgan_021_post_pr_comment"
      },
      {
        "type": "field_llm_judge",
        "path": "result.comment.body",
        "criterion": "Says to use the hyphenated X-Scaffold-Signature header, allows normal case-insensitive handling of that header, and rejects underscore-style names and the query-string fallback.",
        "tool": "post_pr_comment",
        "check_id": "morgan_021_02",
        "action_id": "morgan_021_post_pr_comment"
      },
      {
        "type": "field_regex",
        "path": "result.comment.body",
        "pattern": "(?i)(?<![A-Za-z0-9_])x-scaffold-signature(?![A-Za-z0-9_-])",
        "tool": "post_pr_comment",
        "check_id": "morgan_021_03",
        "action_id": "morgan_021_post_pr_comment"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/morgan/021.yaml

SHA-256: b219a621396a6466ac16a39277e61eaff3b5b19c26b82f2e5d45c3f7442df767