Test 128

Jan 1, 2028 / 1 fact

Request

Create a historical Helio Start operating-phase card for the period after the October 2023 close. State the next period when limited exposure could continue, the public and commercial uses that remained excluded, and who accepted that distinction.

Required memory

Fact 415

After Marcus Vail asked whether broader exposure could begin in November, Riley Tanaka distinguished another small manually vetted batch from public or pricing-page exposure, paid acquisition, GA, or pipeline framing; Tomás accepted that distinction in the close discussion.

Source evidence (1)

000787Nov 3, 2023 / 10:55 UTC-05:00

The October Helio Start close read is in, and it is better but still bounded. Final counts: 31 of the 34 invitees accepted; 21 confirmed admin ownership; 10 selected `unsure` and routed to CS before setup; 13 new workspaces created; and 11 reached first integration. The admitted-account support questions were mostly ordinary API-key/setup questions, with two admin edge cases and no admitted manual-billing or import tickets. Marcus Vail immediately asked whether this means broader exposure can start in November. I drew a hard distinction between another small manually vetted batch and any public or pricing-page exposure, paid acquisition, GA, or pipeline framing, and Tomás accepted that distinction in the close discussion. Draft a concise internal close note that makes the improvement visible without turning it into launch language.

Message 000787 in history

Expected tool calls

create_doc

Grading

1. field_equals / create_doc

{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_operating_phase_card",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_128_00"
}

2. field_llm_judge / create_doc

{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "create_operating_phase_card",
  "path": "args",
  "criterion": "The card says another small manually reviewed November batch could proceed after the October close, while public or pricing-page exposure, paid acquisition, general availability, and pipeline framing remained excluded; Tomás accepted that distinction.",
  "check_id": "riley_128_01"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_operating_phase_card",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_128_00"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "create_operating_phase_card",
        "path": "args",
        "criterion": "The card says another small manually reviewed November batch could proceed after the October close, while public or pricing-page exposure, paid acquisition, general availability, and pipeline framing remained excluded; Tomás accepted that distinction.",
        "check_id": "riley_128_01"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/riley/128.yaml

SHA-256: 59d7cd0f9237a536cf65bd922188b5ad9270675624540262da5a5c107360e756