Test 005 for Morgan Chen

Request

Email privacy@scaffold.dev the handling instructions for Acme's customer-record copy request received this morning.

Required memory

Fact 93

Privacy runbook tab 7 requires customer-record copy requests to be fulfilled within 30 calendar days.

Source evidence (1)

000467May 1, 2023 / 10:00 UTC-07:00

Privacy runbook, tab 7: if someone asks for a customer-record copy, we serve it inside 30 calendar days. Mask PII fields by default; only include them if the customer explicitly opted in to retain them. Written confirmation gets logged on every transaction.

Message 000467 in history

Fact 94

Privacy runbook tab 7 requires PII fields in customer-record copies to be masked by default and included only if the customer explicitly opted in to retain them.

Source evidence (1)

000467May 1, 2023 / 10:00 UTC-07:00

Privacy runbook, tab 7: if someone asks for a customer-record copy, we serve it inside 30 calendar days. Mask PII fields by default; only include them if the customer explicitly opted in to retain them. Written confirmation gets logged on every transaction.

Message 000467 in history

Fact 95

Privacy runbook tab 7 requires written confirmation to be logged for every customer-record copy transaction.

Source evidence (1)

000467May 1, 2023 / 10:00 UTC-07:00

Privacy runbook, tab 7: if someone asks for a customer-record copy, we serve it inside 30 calendar days. Mask PII fields by default; only include them if the customer explicitly opted in to retain them. Written confirmation gets logged on every transaction.

Message 000467 in history

Expected tool calls

send_email

Grading

1. field_equals / send_email

{
  "type": "field_equals",
  "tool": "send_email",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_005_00",
  "action_id": "morgan_005_send_email"
}

2. field_equals / send_email

{
  "type": "field_equals",
  "path": "result.to",
  "value": "privacy@scaffold.dev",
  "tool": "send_email",
  "check_id": "morgan_005_01",
  "action_id": "morgan_005_send_email"
}

3. field_llm_judge / send_email

{
  "type": "field_llm_judge",
  "path": "result.body",
  "criterion": "Instructs fulfillment within 30 calendar days, masking PII unless the customer explicitly opted in to retain it, and logging written confirmation for the transaction. Accept equivalent wording; do not require references to a fact number or runbook tab.",
  "tool": "send_email",
  "check_id": "morgan_005_02",
  "action_id": "morgan_005_send_email"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_email",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_005_00",
        "action_id": "morgan_005_send_email"
      },
      {
        "type": "field_equals",
        "path": "result.to",
        "value": "privacy@scaffold.dev",
        "tool": "send_email",
        "check_id": "morgan_005_01",
        "action_id": "morgan_005_send_email"
      },
      {
        "type": "field_llm_judge",
        "path": "result.body",
        "criterion": "Instructs fulfillment within 30 calendar days, masking PII unless the customer explicitly opted in to retain it, and logging written confirmation for the transaction. Accept equivalent wording; do not require references to a fact number or runbook tab.",
        "tool": "send_email",
        "check_id": "morgan_005_02",
        "action_id": "morgan_005_send_email"
      }
    ]
  }
}