Test 097

Jan 1, 2028 / 1 fact

Request

Create a document titled `Pricing-page continuity baseline card — 2028-01-01` in the `Pricing Analytics Archive` folder. Make it a concise, citation-ready card for the July–October 2026 pricing-page continuity period. Include the certified coverage dates, organic-session and Helio Start eligibility-flow-start denominators, unsupported-condition routed-out count and rate, eligible-workspace admission-completion rate, and account-level tail support effort. Explicitly frame the figures as post-rollout descriptive continuity evidence rather than a randomized test or causal result. Add a brief source note identifying the November 6, 2026 source-closure summary and the November 13, 2026 final-claims review.

Required memory

Fact 227

Certified pricing-page continuity evidence for July 1 through October 31, 2026 covered 12,426 organic sessions and 2,576 Helio Start eligibility-flow starts; 183 starts, or 7.1%, were later routed out for an existing unsupported condition. Eligible-workspace admission completion was 44.4%, and 90th-percentile account-level support time was 74 minutes.

Source evidence (1)

003762Nov 6, 2026 / 15:24 UTC-06:00

Owen completed the scheduled source-closure summary. He independently normalized and reconciled the engineering decision and reliability sources through October 31, confirmed that every required field is present, established definitions are unchanged, every named owner certified their section, and no cross-surface discrepancy exists. He therefore closed the first complete four-lane source packet under his September 24 authority. The certified packet shows: - Helio Start authority-checklist evidence, September 15 through October 31: 148 newly admitted organic workspaces, 45 checklist exposures, and seven ordinary setup-assistance requests, with no eligibility-route mismatch, lifecycle-suppression bypass, paid traffic, or unsupported admission. - Expansion Assist: 16 actual exposures from July 1 through September 15 with complete 45-day windows; six help-or-options requests and four expansions. No exposed account had active Customer Success adoption or renewal work, and the evidence remains descriptive. - Pricing-page continuity, July 1 through October 31: 12,426 organic sessions and 2,576 Helio Start eligibility-flow starts; 183, or 7.1%, were later routed out for an existing unsupported condition. Eligible-workspace admission completion was 44.4%, and account-level support time at the 90th percentile was 74 minutes. - CS auto-nudge coverage, July 1 through October 31: 6,704 unique decisions, including 249 visible field-specific holds and 6,455 decisions that proceeded or were suppressed using current state. There was no governed-field disagreement, customer correction, or rollback; four source-age alerts fired and resolved before breach. I received Owen’s closure-and-exception summary without doing a line-by-line source review. The earlier coverage discrepancy is resolved and the packet is complete. Causal, revenue, forecast, board-readiness, and executive-language decisions remain open in their existing owner lanes.

Message 003762 in history

Expected tool calls

create_doc

Grading

1. field_equals / create_doc

{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_continuity_card",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_097_00"
}

2. field_equals / create_doc

{
  "check_id": "riley_097_01",
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_continuity_card",
  "path": "args.title",
  "value": "Pricing-page continuity baseline card — 2028-01-01"
}

3. field_equals / create_doc

{
  "check_id": "riley_097_02",
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_continuity_card",
  "path": "args.folder",
  "value": "Pricing Analytics Archive"
}

4. field_llm_judge / create_doc

{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "create_continuity_card",
  "path": "args",
  "criterion": "The body must accurately state all of these certified pricing-page continuity measurements: coverage from July 1 through October 31, 2026; 12,426 organic sessions; 2,576 Helio Start eligibility-flow starts; 183 starts, or 7.1%, later routed out for an existing unsupported condition; 44.4% eligible-workspace admission completion; and account-level support time at the 90th percentile of 74 minutes. Each number must be associated with the correct population or measure, and missing or contradictory values fail.\n\nThe body must explicitly characterize the figures as post-rollout descriptive continuity evidence and must not present them as a randomized or controlled experiment, a causal result, or evidence of causal improvement. It must also include a brief source note identifying both the November 6, 2026 source-closure summary and the November 13, 2026 final-claims review.",
  "check_id": "riley_097_03"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_continuity_card",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_097_00"
      },
      {
        "check_id": "riley_097_01",
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_continuity_card",
        "path": "args.title",
        "value": "Pricing-page continuity baseline card — 2028-01-01"
      },
      {
        "check_id": "riley_097_02",
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_continuity_card",
        "path": "args.folder",
        "value": "Pricing Analytics Archive"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "create_continuity_card",
        "path": "args",
        "criterion": "The body must accurately state all of these certified pricing-page continuity measurements: coverage from July 1 through October 31, 2026; 12,426 organic sessions; 2,576 Helio Start eligibility-flow starts; 183 starts, or 7.1%, later routed out for an existing unsupported condition; 44.4% eligible-workspace admission completion; and account-level support time at the 90th percentile of 74 minutes. Each number must be associated with the correct population or measure, and missing or contradictory values fail.\n\nThe body must explicitly characterize the figures as post-rollout descriptive continuity evidence and must not present them as a randomized or controlled experiment, a causal result, or evidence of causal improvement. It must also include a brief source note identifying both the November 6, 2026 source-closure summary and the November 13, 2026 final-claims review.",
        "check_id": "riley_097_03"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/riley/097.yaml

SHA-256: 0fb6abbeed65d891e29f0f348c3d6ebf807515303214a163af31f342f8845acf