DolphinBench

Test 065

Sep 14, 2026 / 2 facts

YAML

Request

Email support@scaffold.dev the approved Pinecone verification contract and whether to add the requested Python example when there is no failing Python reproduction.

Required memory

Fact 173

On February 9, 2024, Rishi proposed this as the final external Pinecone connector wording: “Scaffold’s supported Pinecone connector verification path is header-only: send `X-Scaffold-Workspace` and `X-Scaffold-Signature` on the incoming request and verify the signature against the same raw request body and workspace; preserved `X-Scaffold-Signature` and fully lowercased `x-scaffold-signature` are accepted.”

Source evidence (1)

001016Feb 9, 2024 / 10:18 UTC-08:00

Rishi is trying to close the Pinecone docs wording thread this week. Discord DM — Rishi → Morgan Chen Fri Feb 9, 2024 10:06 AM PT Trying to close the Pinecone docs wording thread this week. Proposed final external sentence: “Scaffold’s supported Pinecone connector verification path is header-only: send `X-Scaffold-Workspace` and `X-Scaffold-Signature` on the incoming request and verify the signature against the same raw request body and workspace; preserved `X-Scaffold-Signature` and fully lowercased `x-scaffold-signature` are accepted.” Internal close note I’d keep with it: - no legacy/API v2 comparison line - no Python example unless they send a concrete failing repro - no alternate signature locations, query fallback, or broader header-normalization language - keep the existing Go example and the single SDK-auth crosslink on the canonical header-only path - otherwise I’d mark the docs thread closed for now rather than broadening the page again this week If you want a shorter version, I can trim the accepted-header clause into a separate troubleshooting sentence, but I would keep the public reply on the same narrow contract. Please give me the final customer-facing sentence and a short internal close note. Keep the public wording on the narrow header-only contract, with no legacy/API v2 comparison, no Python example unless Pinecone sends a concrete failing repro, and no alternate signature locations or query fallback. The internal note should mark the docs thread closed for now rather than broadening the page again this week.

Message 001016 in history

Fact 174

Morgan requested a final customer-facing Pinecone sentence that retains the narrow header-only contract and the exclusions concerning legacy/API v2 comparisons, Python examples without a concrete failing repro, alternate signature locations, and query fallback.

Source evidence (1)

001016Feb 9, 2024 / 10:18 UTC-08:00

Rishi is trying to close the Pinecone docs wording thread this week. Discord DM — Rishi → Morgan Chen Fri Feb 9, 2024 10:06 AM PT Trying to close the Pinecone docs wording thread this week. Proposed final external sentence: “Scaffold’s supported Pinecone connector verification path is header-only: send `X-Scaffold-Workspace` and `X-Scaffold-Signature` on the incoming request and verify the signature against the same raw request body and workspace; preserved `X-Scaffold-Signature` and fully lowercased `x-scaffold-signature` are accepted.” Internal close note I’d keep with it: - no legacy/API v2 comparison line - no Python example unless they send a concrete failing repro - no alternate signature locations, query fallback, or broader header-normalization language - keep the existing Go example and the single SDK-auth crosslink on the canonical header-only path - otherwise I’d mark the docs thread closed for now rather than broadening the page again this week If you want a shorter version, I can trim the accepted-header clause into a separate troubleshooting sentence, but I would keep the public reply on the same narrow contract. Please give me the final customer-facing sentence and a short internal close note. Keep the public wording on the narrow header-only contract, with no legacy/API v2 comparison, no Python example unless Pinecone sends a concrete failing repro, and no alternate signature locations or query fallback. The internal note should mark the docs thread closed for now rather than broadening the page again this week.

Message 001016 in history

Expected tool calls

  • send_email

Grading

1. field_equals / send_email
{
  "type": "field_equals",
  "tool": "send_email",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_065_00",
  "action_id": "morgan_065_send_email"
}
2. field_equals / send_email
{
  "type": "field_equals",
  "path": "result.to",
  "value": "support@scaffold.dev",
  "tool": "send_email",
  "check_id": "morgan_065_01",
  "action_id": "morgan_065_send_email"
}
3. field_llm_judge / send_email
{
  "type": "field_llm_judge",
  "path": "result.body",
  "criterion": "Keeps verification header-only using X-Scaffold-Workspace and X-Scaffold-Signature against the same raw request body and workspace; preserved and fully lowercased signature headers are accepted. Does not broaden this to arbitrary header normalization, query fallback, or alternate signature locations. Defers a Python example until a concrete failing reproduction is supplied. Equivalent wording is valid; no verbatim paragraph or unrelated docs-close checklist is required.",
  "tool": "send_email",
  "check_id": "morgan_065_02",
  "action_id": "morgan_065_send_email"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_email",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_065_00",
        "action_id": "morgan_065_send_email"
      },
      {
        "type": "field_equals",
        "path": "result.to",
        "value": "support@scaffold.dev",
        "tool": "send_email",
        "check_id": "morgan_065_01",
        "action_id": "morgan_065_send_email"
      },
      {
        "type": "field_llm_judge",
        "path": "result.body",
        "criterion": "Keeps verification header-only using X-Scaffold-Workspace and X-Scaffold-Signature against the same raw request body and workspace; preserved and fully lowercased signature headers are accepted. Does not broaden this to arbitrary header normalization, query fallback, or alternate signature locations. Defers a Python example until a concrete failing reproduction is supplied. Equivalent wording is valid; no verbatim paragraph or unrelated docs-close checklist is required.",
        "tool": "send_email",
        "check_id": "morgan_065_02",
        "action_id": "morgan_065_send_email"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/065.yaml

SHA-256: a5a39a67efd2dd781f75fe8b382e0a7be60d863684f8ac4be0ad3a82e7460642