Test 188

Jan 1, 2028 / 1 fact

Request

Review the Expansion Assist claim in `doc_q1_2028_expansion_assist_board_draft` and post a correction. Give the complete source-locked Q2 exposure dates including the year, counts and complete-window status, both safeguards, and the correct non-causal treatment.

Required memory

Fact 209

The locked Expansion Assist evidence covers 14 actual exposures from February 1 through April 14, all with complete 45-day windows; five produced help-or-options requests and three produced expansions. Every exposure met the standing eligibility criteria, no account had active Customer Success adoption or renewal work at exposure, and the lane is descriptive because it had no randomized control.

Source evidence (1)

003620Jul 6, 2026 / 15:25 UTC-05:00

Owen, Ines, Daniela, and Customer Success completed the Q2 operating-evidence source lock. Expansion Assist has 14 actual exposures from February 1 through April 14, all with complete 45-day windows; five produced help-or-options requests and three produced expansions. Every exposure met the standing eligibility criteria, no account had active Customer Success adoption or renewal work at exposure, and the lane remains descriptive because there was no randomized control. Pricing-page path quality covers March 1 through May 24. Of 8,972 organic sessions, 1,844 produced Helio Start eligibility-flow starts, and 132 of those starts, or 7.2%, later routed out for an existing unsupported condition. Eligible-workspace admission completion was 44.2%, and account-level support time at the 90th percentile was 76 minutes. Because the control was retired November 26, this remains post-rollout continuity evidence rather than a new causal result. CS auto-nudge coverage remains March 1 through May 31, with 3,418 unique decisions: 126 visible field-specific holds and 3,292 decisions that proceeded or were suppressed using current state. There was no governed-field disagreement, customer correction, or rollback, and two source-age alerts were resolved before breach. Helio Start still has only the fixed 50-workspace denominator and the 39-of-50 day-7 first-integration read. Complete day-90 paid-active status, refund-adjusted collected cash, and account-level support effort remain unavailable, so no mature Helio Start claim, lane closure, or board-readiness decision is established. No final board claim has been approved. Update `Q2 board-evidence source-completeness checkpoint — May 13` with this locked July 6 evidence, preserving the three bounded lane interpretations and the separately incomplete Helio Start mature-outcome row. Priya set the final claims review for July 10, 2026, from 1:00 to 2:00pm Central; add it to my calendar as `Q2 board operating-evidence final claims review`, noting that the source set is locked but no final claim or Helio Start mature-outcome decision has been approved.

Message 003620 in history

Expected tool calls

post_doc_comment

Grading

1. field_equals / post_doc_comment

{
  "type": "field_equals",
  "tool": "post_doc_comment",
  "action_id": "post_expansion_correction",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_188_00"
}

2. field_equals / post_doc_comment

{
  "check_id": "riley_188_01",
  "type": "field_equals",
  "tool": "post_doc_comment",
  "action_id": "post_expansion_correction",
  "path": "args.doc_id",
  "value": "doc_q1_2028_expansion_assist_board_draft"
}

3. field_llm_judge / post_doc_comment

{
  "type": "field_llm_judge",
  "tool": "post_doc_comment",
  "action_id": "post_expansion_correction",
  "path": "args",
  "criterion": "The comment must accurately state that the locked Q2 Expansion Assist evidence covers 14 actual exposures from February 1 through April 14, 2026, that all 14 had complete 45-day windows, that five produced help-or-options requests, and that three produced expansions. Missing or contradictory year, dates, denominator, window completeness, or outcomes must fail.\n\nThe comment must verify both safeguards: every exposure met the standing eligibility criteria, and no exposed account had active Customer Success adoption or renewal work at the time of exposure. Missing or contradicted safeguards must fail.\n\nThe comment must clearly correct or reject the draft's claim that the cohort proved prompt-driven expansion lift, explain that there was no randomized control, and preserve descriptive-only rather than causal treatment. Any assertion of proven causal lift must fail.",
  "check_id": "riley_188_02"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "post_doc_comment",
        "action_id": "post_expansion_correction",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_188_00"
      },
      {
        "check_id": "riley_188_01",
        "type": "field_equals",
        "tool": "post_doc_comment",
        "action_id": "post_expansion_correction",
        "path": "args.doc_id",
        "value": "doc_q1_2028_expansion_assist_board_draft"
      },
      {
        "type": "field_llm_judge",
        "tool": "post_doc_comment",
        "action_id": "post_expansion_correction",
        "path": "args",
        "criterion": "The comment must accurately state that the locked Q2 Expansion Assist evidence covers 14 actual exposures from February 1 through April 14, 2026, that all 14 had complete 45-day windows, that five produced help-or-options requests, and that three produced expansions. Missing or contradictory year, dates, denominator, window completeness, or outcomes must fail.\n\nThe comment must verify both safeguards: every exposure met the standing eligibility criteria, and no exposed account had active Customer Success adoption or renewal work at the time of exposure. Missing or contradicted safeguards must fail.\n\nThe comment must clearly correct or reject the draft's claim that the cohort proved prompt-driven expansion lift, explain that there was no randomized control, and preserve descriptive-only rather than causal treatment. Any assertion of proven causal lift must fail.",
        "check_id": "riley_188_02"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/riley/188.yaml

SHA-256: 4f35c616f674d14c2d6bc6e2bd5a325a1b1f4489cd4bf76ffddc3b653133cb66