Test 143

Jan 1, 2028 / 3 facts

Request

Create a compact regression-lineage card titled `Helio Start intake regression lineage — 2024-04-23` in the Archive folder. Treat it as a historical baseline. Use every input from the approved public-path admission checklist plus the later connector-access input. Include one admitted case with every input present and one separate non-admitted case for each individual input; do not combine two missing inputs into one case.

Required memory

Fact 13

Helio Start’s public self-serve path is limited to under-50 accounts with admin proof, simple deployment, first-week integration fit, a current entitlement state, and supported billing conditions. Legitimate domain mismatches may use corroborating evidence, while agency identities, manual billing, imports, migrations, and unsupported permissions must route out.

Source evidence (1)

001211Mar 4, 2024 / 15:05 UTC-06:00

The Helio Start decision meeting just ended. Tomás approved a bounded March 18 general-availability launch on the public pricing page, but paid acquisition and any forecastable pipeline language are still out. The public path is only for under-50 accounts when admin proof, deployment simplicity, first-week integration fit, a current entitlement state, and supported billing conditions are all present. Legitimate domain mismatches can use corroborating evidence, while agency identities, manual billing, imports, migrations, and unsupported permissions route out instead of entering self-serve. Priya also expanded my remit to the full self-serve operating model across eligibility and product constraints, activation and support-load measurement, lifecycle, support boundaries, and executive language, without creating a new team. Support load stays a first-class metric beside activation. Owen now owns the reporting layer and has to keep Helio Start activation and support-load measurement separate from retained mid-segment and Expansion Assist. Daniela owns entitlement and support-state instrumentation, Ines owns lifecycle copy and risk controls, and Customer Success owns routed-before-setup support paths.

Message 001211 in history

Fact 20

The reviewed connector-access gate is now live in the Helio Start public-pricing-page intake: a definite Yes proceeds through the remaining eligibility checks, while No or Not sure stays pending and routes to assistance before workspace admission.

Source evidence (2)

001394Apr 23, 2024 / 11:45 UTC-05:00

The reviewed connector-access change is now live in the Helio Start intake reached from the public pricing page. A definite Yes continues through the otherwise unchanged eligibility checks; No and Not sure remain pending and route to assistance before workspace admission. Daniela stores the connector-authentication answer separately from Helio admin proof, Ines prevents connection prompts while proof is pending, and Customer Success has added the pending case to its routing instructions. Agencies, unresolved ownership, manual billing, imports, migrations, unsupported permissions, and the other existing route-out conditions remain unchanged. Update the existing Helio Start connector-access investigation brief in place with the live design, routing and lifecycle behavior, operating owners, unchanged boundaries, and Owen's noncausal evaluation plan.

Message 001394 in history

001256Mar 18, 2024 / 09:14 UTC-05:00

Helio Start is live on the public pricing page. The live smoke test admitted the simple under-50 workspace with verified admin access, supported billing, a current entitlement state, and a first-week integration fit. The manual-billing case routed to assisted, the agency case was ineligible for Start, and the unresolved-ownership case stayed pending instead of entering a workspace. Owen's live view keeps Start activation and support load separate from ordinary starter and retained mid-segment, Ines's Start lifecycle fork is active, and no unrelated pricing-page layout change shipped with the release.

Message 001256 in history

Fact 21

Daniela stores the Helio Start connector-authentication answer separately from Helio admin proof.

Source evidence (1)

001394Apr 23, 2024 / 11:45 UTC-05:00

Message 001394 in history

Expected tool calls

create_doc

Grading

1. field_equals / create_doc

{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_lineage_card",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_143_00"
}

2. field_equals / create_doc

{
  "check_id": "riley_143_01",
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_lineage_card",
  "path": "args.title",
  "value": "Helio Start intake regression lineage — 2024-04-23"
}

3. field_equals / create_doc

{
  "check_id": "riley_143_02",
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_lineage_card",
  "path": "args.folder",
  "value": "Archive"
}

4. field_llm_judge / create_doc

{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "create_lineage_card",
  "path": "args",
  "criterion": "The document clearly presents the matrix as the live Helio Start intake baseline on April 23, 2024, and explicitly avoids presenting it as the later or current intake path.\n\nThe document contains a compact acceptance-test matrix that (1) has a positive case where every required input is satisfied and the expected result is admitted, and (2) has a separate negative case for each of these high-level inputs: account is under 50, admin proof is present, deployment is simple, first-week integration fit is present, a current entitlement state is present, and billing is supported. Each negative case must show that the applicant is not admitted when that input is absent or unsatisfied. The first-week integration-fit case must reflect the April 23 connector-access design: the applicant must definitely be able to authenticate the selected supported connector; a No or Not sure answer remains pending and routes to assistance before admission. Connector access must not be conflated with Helio admin proof. Equivalent concise labels for non-admission, pending, assisted routing, or ineligibility are acceptable where they preserve these outcomes.",
  "check_id": "riley_143_03"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_lineage_card",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_143_00"
      },
      {
        "check_id": "riley_143_01",
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_lineage_card",
        "path": "args.title",
        "value": "Helio Start intake regression lineage — 2024-04-23"
      },
      {
        "check_id": "riley_143_02",
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_lineage_card",
        "path": "args.folder",
        "value": "Archive"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "create_lineage_card",
        "path": "args",
        "criterion": "The document clearly presents the matrix as the live Helio Start intake baseline on April 23, 2024, and explicitly avoids presenting it as the later or current intake path.\n\nThe document contains a compact acceptance-test matrix that (1) has a positive case where every required input is satisfied and the expected result is admitted, and (2) has a separate negative case for each of these high-level inputs: account is under 50, admin proof is present, deployment is simple, first-week integration fit is present, a current entitlement state is present, and billing is supported. Each negative case must show that the applicant is not admitted when that input is absent or unsatisfied. The first-week integration-fit case must reflect the April 23 connector-access design: the applicant must definitely be able to authenticate the selected supported connector; a No or Not sure answer remains pending and routes to assistance before admission. Connector access must not be conflated with Helio admin proof. Equivalent concise labels for non-admission, pending, assisted routing, or ineligibility are acceptable where they preserve these outcomes.",
        "check_id": "riley_143_03"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/riley/143.yaml

SHA-256: 976ad34f0ef6b37a667e27942e03992357df0186ea3a8a2cff94aae0d8bc3106