DolphinBench

Test 106

Jan 1, 2028 / 7 facts

YAML

Request

Create a short historical comparison note for new board-operations reviewers explaining why Helio Start was excluded from the approved Q1 and Q2 2026 quarterly operating-evidence sections. For Q1, state what the evidence meant for Q2 forecast credit. For Q2, state what the day-7 result did and did not establish. Preserve the approved interpretation of the other three lanes in each package, and make the quarter-specific distinction clear rather than describing both exclusions as generic immaturity.

Required memory

Fact 182

Riley Tanaka determined that Helio Start’s two qualifying mature cohorts permit consideration of a capped proposal but do not create Q2 forecast credit because there is no paid traffic, approved budget, or acquisition evidence.

Source evidence (1)

003560Mar 24, 2026 / 14:15 UTC-05:00

The 1:00pm final claims review is complete. Marcus Vail asked whether Helio Start’s two qualifying mature cohorts should create Q2 forecast credit. I separated the operating gate from a commercial result: the cohorts permit consideration of a capped proposal, but there is still no paid traffic, approved budget, or acquisition evidence. The Helio Start row also still lacks the exact cohort start and end dates required by the January 6 readiness gate, so I kept it `not board-ready` and excluded it from the board operating-evidence section rather than weakening the gate. Helio Start remains organic-only through March 31, and the cleared proposal gate does not authorize paid traffic, budget, acquisition claims, pipeline treatment, or forecast credit. Expansion Assist remains descriptive because it has no randomized control. The November 26-through-February 28 pricing-page coverage remains post-rollout continuity evidence rather than a second causal test, and the January 1-through-February 28 CS auto-nudge lane remains limited to system-control performance. Owen, Ines, and Daniela confirmed the applicable source and limit sections for those three board-ready lanes. Tomás approved my operating-evidence section containing those three lanes without adding a Helio Start forecast claim. Update `Quarterly board operating evidence intake — Q1 2026` with the final review decision, Tomás’s approved three-lane section, the final claims and limits, and Helio Start’s continuing exclusion and `not board-ready` status.

Message 003560 in history

Fact 183

Riley Tanaka kept Helio Start `not board-ready` and excluded from the board operating-evidence section because its row lacks the exact cohort start and end dates required by the January 6 readiness gate; Riley chose not to weaken that gate.

Source evidence (1)

003560Mar 24, 2026 / 14:15 UTC-05:00

The 1:00pm final claims review is complete. Marcus Vail asked whether Helio Start’s two qualifying mature cohorts should create Q2 forecast credit. I separated the operating gate from a commercial result: the cohorts permit consideration of a capped proposal, but there is still no paid traffic, approved budget, or acquisition evidence. The Helio Start row also still lacks the exact cohort start and end dates required by the January 6 readiness gate, so I kept it `not board-ready` and excluded it from the board operating-evidence section rather than weakening the gate. Helio Start remains organic-only through March 31, and the cleared proposal gate does not authorize paid traffic, budget, acquisition claims, pipeline treatment, or forecast credit. Expansion Assist remains descriptive because it has no randomized control. The November 26-through-February 28 pricing-page coverage remains post-rollout continuity evidence rather than a second causal test, and the January 1-through-February 28 CS auto-nudge lane remains limited to system-control performance. Owen, Ines, and Daniela confirmed the applicable source and limit sections for those three board-ready lanes. Tomás approved my operating-evidence section containing those three lanes without adding a Helio Start forecast claim. Update `Quarterly board operating evidence intake — Q1 2026` with the final review decision, Tomás’s approved three-lane section, the final claims and limits, and Helio Start’s continuing exclusion and `not board-ready` status.

Message 003560 in history

Fact 185

Tomás approved Riley Tanaka’s three-lane operating-evidence section: Expansion Assist is descriptive because it lacks a randomized control, November 26–February 28 pricing-page coverage is post-rollout continuity evidence rather than a second causal test, and the January 1–February 28 CS auto-nudge lane is limited to system-control performance. The approval added no Helio Start forecast claim.

Source evidence (1)

003560Mar 24, 2026 / 14:15 UTC-05:00

The 1:00pm final claims review is complete. Marcus Vail asked whether Helio Start’s two qualifying mature cohorts should create Q2 forecast credit. I separated the operating gate from a commercial result: the cohorts permit consideration of a capped proposal, but there is still no paid traffic, approved budget, or acquisition evidence. The Helio Start row also still lacks the exact cohort start and end dates required by the January 6 readiness gate, so I kept it `not board-ready` and excluded it from the board operating-evidence section rather than weakening the gate. Helio Start remains organic-only through March 31, and the cleared proposal gate does not authorize paid traffic, budget, acquisition claims, pipeline treatment, or forecast credit. Expansion Assist remains descriptive because it has no randomized control. The November 26-through-February 28 pricing-page coverage remains post-rollout continuity evidence rather than a second causal test, and the January 1-through-February 28 CS auto-nudge lane remains limited to system-control performance. Owen, Ines, and Daniela confirmed the applicable source and limit sections for those three board-ready lanes. Tomás approved my operating-evidence section containing those three lanes without adding a Helio Start forecast claim. Update `Quarterly board operating evidence intake — Q1 2026` with the final review decision, Tomás’s approved three-lane section, the final claims and limits, and Helio Start’s continuing exclusion and `not board-ready` status.

Message 003560 in history

Fact 210

Riley Tanaka excluded Helio Start from the approved Q2 operating-evidence section and added no acquisition or forecast claim based on its 39-of-50 day-7 first-integration result because the fixed 50-workspace cohort still lacks complete day-90 paid-active status, refund-adjusted collected cash, and account-level support effort; the exclusion remains pending complete mature-window outcomes.

Source evidence (1)

003623Jul 10, 2026 / 14:16 UTC-05:00

The July 10 final claims review is complete. Marcus Vail asked whether the paid Helio Start cohort’s 39-of-50 day-7 first-integration result could support a growth or forecast statement. I kept it out because the fixed 50-workspace cohort still lacks complete day-90 paid-active status, refund-adjusted collected cash, and account-level support effort. Tomás approved a three-lane Q2 operating-evidence section: Expansion Assist remains descriptive with no causal, revenue, pipeline, or forecast treatment; pricing-page path quality remains post-rollout continuity evidence rather than a new causal result; and CS auto-nudge evidence remains limited to system-control performance. Helio Start is excluded from the approved Q2 section until its mature window closes, with no acquisition or forecast claim added. This closes the Q2 final-claims decision; the paid-pilot day-90 outcome matter remains open separately. Update `Q2 board-evidence source-completeness checkpoint — May 13` with the completed July 10 review, the three approved bounded lanes, and the Helio Start exclusion pending complete day-90 outcomes.

Message 003623 in history

Fact 211

Tomás approved treating Expansion Assist as descriptive Q2 operating evidence with no causal, revenue, pipeline, or forecast treatment.

Source evidence (1)

003623Jul 10, 2026 / 14:16 UTC-05:00

The July 10 final claims review is complete. Marcus Vail asked whether the paid Helio Start cohort’s 39-of-50 day-7 first-integration result could support a growth or forecast statement. I kept it out because the fixed 50-workspace cohort still lacks complete day-90 paid-active status, refund-adjusted collected cash, and account-level support effort. Tomás approved a three-lane Q2 operating-evidence section: Expansion Assist remains descriptive with no causal, revenue, pipeline, or forecast treatment; pricing-page path quality remains post-rollout continuity evidence rather than a new causal result; and CS auto-nudge evidence remains limited to system-control performance. Helio Start is excluded from the approved Q2 section until its mature window closes, with no acquisition or forecast claim added. This closes the Q2 final-claims decision; the paid-pilot day-90 outcome matter remains open separately. Update `Q2 board-evidence source-completeness checkpoint — May 13` with the completed July 10 review, the three approved bounded lanes, and the Helio Start exclusion pending complete day-90 outcomes.

Message 003623 in history

Fact 212

Tomás approved treating pricing-page path quality as post-rollout continuity evidence rather than a new causal result.

Source evidence (1)

003623Jul 10, 2026 / 14:16 UTC-05:00

The July 10 final claims review is complete. Marcus Vail asked whether the paid Helio Start cohort’s 39-of-50 day-7 first-integration result could support a growth or forecast statement. I kept it out because the fixed 50-workspace cohort still lacks complete day-90 paid-active status, refund-adjusted collected cash, and account-level support effort. Tomás approved a three-lane Q2 operating-evidence section: Expansion Assist remains descriptive with no causal, revenue, pipeline, or forecast treatment; pricing-page path quality remains post-rollout continuity evidence rather than a new causal result; and CS auto-nudge evidence remains limited to system-control performance. Helio Start is excluded from the approved Q2 section until its mature window closes, with no acquisition or forecast claim added. This closes the Q2 final-claims decision; the paid-pilot day-90 outcome matter remains open separately. Update `Q2 board-evidence source-completeness checkpoint — May 13` with the completed July 10 review, the three approved bounded lanes, and the Helio Start exclusion pending complete day-90 outcomes.

Message 003623 in history

Fact 213

Tomás approved limiting CS auto-nudge evidence to system-control performance.

Source evidence (1)

003623Jul 10, 2026 / 14:16 UTC-05:00

The July 10 final claims review is complete. Marcus Vail asked whether the paid Helio Start cohort’s 39-of-50 day-7 first-integration result could support a growth or forecast statement. I kept it out because the fixed 50-workspace cohort still lacks complete day-90 paid-active status, refund-adjusted collected cash, and account-level support effort. Tomás approved a three-lane Q2 operating-evidence section: Expansion Assist remains descriptive with no causal, revenue, pipeline, or forecast treatment; pricing-page path quality remains post-rollout continuity evidence rather than a new causal result; and CS auto-nudge evidence remains limited to system-control performance. Helio Start is excluded from the approved Q2 section until its mature window closes, with no acquisition or forecast claim added. This closes the Q2 final-claims decision; the paid-pilot day-90 outcome matter remains open separately. Update `Q2 board-evidence source-completeness checkpoint — May 13` with the completed July 10 review, the three approved bounded lanes, and the Helio Start exclusion pending complete day-90 outcomes.

Message 003623 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "historical_comparison_note",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_106_00"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "historical_comparison_note",
  "path": "args",
  "criterion": "The note says Helio Start was excluded from the Q1 2026 section because its two qualifying mature cohorts permitted consideration of a capped proposal but its row lacked the exact cohort dates required for board readiness; at that time there was no paid traffic, approved budget, or acquisition evidence, and no Q2 forecast credit followed. It says Helio Start was excluded from the Q2 2026 section because the fixed 50-workspace cohort did not yet have complete day-90 paid-active status, refund-adjusted collected cash, and account-level support effort, and its 39-of-50 day-7 first-integration result supported no growth or forecast statement. For both quarters it preserves the three included lanes: Expansion Assist as descriptive rather than causal evidence, pricing-page path quality as post-rollout continuity rather than a new causal test, and CS auto-nudge as system-control evidence. It also says the Q2 Expansion Assist lane had no revenue, pipeline, or forecast treatment.",
  "check_id": "riley_106_01"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "historical_comparison_note",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_106_00"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "historical_comparison_note",
        "path": "args",
        "criterion": "The note says Helio Start was excluded from the Q1 2026 section because its two qualifying mature cohorts permitted consideration of a capped proposal but its row lacked the exact cohort dates required for board readiness; at that time there was no paid traffic, approved budget, or acquisition evidence, and no Q2 forecast credit followed. It says Helio Start was excluded from the Q2 2026 section because the fixed 50-workspace cohort did not yet have complete day-90 paid-active status, refund-adjusted collected cash, and account-level support effort, and its 39-of-50 day-7 first-integration result supported no growth or forecast statement. For both quarters it preserves the three included lanes: Expansion Assist as descriptive rather than causal evidence, pricing-page path quality as post-rollout continuity rather than a new causal test, and CS auto-nudge as system-control evidence. It also says the Q2 Expansion Assist lane had no revenue, pipeline, or forecast treatment.",
        "check_id": "riley_106_01"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/riley/106.yaml

SHA-256: 39c1d28b4019084a088ecb3441fe6f3be10fd17352a87a5254b7f771cb68c725