Test 109 for Riley Tanaka

Request

Please post a concise routing summary to #helio-start-ops for these current Helio Start authority-readiness cases: HS-801 — verified yes; HS-802 — no; HS-803 — not sure. Identify what happens next for each case and briefly name the readiness confirmation being applied.

Required memory

Fact 68

Helio Start applicants must separately confirm that they or an available owner can perform the owner-level permission or network-allowlist action required by the connector selected for first-week setup; a verified yes may continue, while no or not sure must be routed to Customer Success before admission.

Source evidence (1)

002262Dec 16, 2024 / 11:30 UTC-06:00

Priya and I approved a targeted Helio Start intake change based on the completed diagnostic. Applicants must separately confirm that they or an available owner can perform the owner-level permission or network-allowlist action required by the connector selected for first-week setup. A verified yes can continue; no or not sure routes to Customer Success before admission. This does not replace or broaden the existing Helio admin-proof, connector-authentication, entitlement, billing, or deployment requirements. Daniela records external-authority readiness as a separate state, Ines holds setup prompts while that state is pending, and Owen reports it separately so support effects can be evaluated without rewriting the existing admission measures. Post this approved operating instruction to growth operations, making the distinction between external authority, admin proof, and connector authentication explicit.

Message 002262 in history

Fact 69

The approved Helio Start intake change must not replace or broaden existing admin-proof, connector-authentication, entitlement, billing, or deployment requirements.

Source evidence (1)

002262Dec 16, 2024 / 11:30 UTC-06:00

Priya and I approved a targeted Helio Start intake change based on the completed diagnostic. Applicants must separately confirm that they or an available owner can perform the owner-level permission or network-allowlist action required by the connector selected for first-week setup. A verified yes can continue; no or not sure routes to Customer Success before admission. This does not replace or broaden the existing Helio admin-proof, connector-authentication, entitlement, billing, or deployment requirements. Daniela records external-authority readiness as a separate state, Ines holds setup prompts while that state is pending, and Owen reports it separately so support effects can be evaluated without rewriting the existing admission measures. Post this approved operating instruction to growth operations, making the distinction between external authority, admin proof, and connector authentication explicit.

Message 002262 in history

Expected tool calls

send_slack_message

Grading

1. field_equals / send_slack_message

{
  "type": "field_equals",
  "tool": "send_slack_message",
  "action_id": "post_authority_routing",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_109_00"
}

2. field_equals / send_slack_message

{
  "check_id": "riley_109_01",
  "type": "field_equals",
  "tool": "send_slack_message",
  "action_id": "post_authority_routing",
  "path": "args.channel",
  "value": "#helio-start-ops"
}

3. field_llm_judge / send_slack_message

{
  "type": "field_llm_judge",
  "tool": "send_slack_message",
  "action_id": "post_authority_routing",
  "path": "args",
  "criterion": "The message must briefly and accurately identify the separate authority-readiness confirmation as whether the applicant or an available owner can perform the selected connector’s required owner-level permission or network-allowlist action in the setup or pre-setup context. Semantically equivalent wording is acceptable; the literal phrase “selected for first-week setup” is not required. The message must not conflate this confirmation with Helio admin proof, connector authentication, or another proof.\n\nThe message clearly identifies HS-801 (verified yes) as permitted to continue, and clearly identifies both HS-802 (no) and HS-803 (not sure) as requiring routing to Customer Success before admission. Missing, swapped, or contradictory outcomes must fail.",
  "check_id": "riley_109_02"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_slack_message",
        "action_id": "post_authority_routing",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_109_00"
      },
      {
        "check_id": "riley_109_01",
        "type": "field_equals",
        "tool": "send_slack_message",
        "action_id": "post_authority_routing",
        "path": "args.channel",
        "value": "#helio-start-ops"
      },
      {
        "type": "field_llm_judge",
        "tool": "send_slack_message",
        "action_id": "post_authority_routing",
        "path": "args",
        "criterion": "The message must briefly and accurately identify the separate authority-readiness confirmation as whether the applicant or an available owner can perform the selected connector’s required owner-level permission or network-allowlist action in the setup or pre-setup context. Semantically equivalent wording is acceptable; the literal phrase “selected for first-week setup” is not required. The message must not conflate this confirmation with Helio admin proof, connector authentication, or another proof.\n\nThe message clearly identifies HS-801 (verified yes) as permitted to continue, and clearly identifies both HS-802 (no) and HS-803 (not sure) as requiring routing to Customer Success before admission. Missing, swapped, or contradictory outcomes must fail.",
        "check_id": "riley_109_02"
      }
    ]
  }
}