DolphinBench

Test 175

Jan 1, 2028 / 7 facts

YAML

Request

Create a concise 2026 Helio Start paid-search pilot closeout. State when and what closed the pilot, final spend and unused budget, attribution and admission checks, every pause-rule result, and whether traffic or further admissions remain open.

Required memory

Fact 186

The Helio Start budget review was completed on April 2, 2026, and Priya approved Riley Tanaka’s recommended six-week high-intent paid-search pilot with a maximum budget of $15,000; the pilot closes at the earlier of $15,000 spent or 50 admitted workspaces.

Source evidence (1)

003567Apr 2, 2026 / 14:18 UTC-05:00

The Helio Start budget review is done. Priya approved my recommended six-week high-intent paid-search pilot with a budget of no more than $15,000, closing at the earlier of $15,000 in spend or 50 admitted workspaces. The working draft still presents that recommendation and the smaller fallback as unapproved, with budget approval and preflight both outstanding. Update `Helio Start capped paid-acquisition proposal — Q2 2026 working draft` to show budget-approved/pending-preflight status, then post the bounded decision in #lifecycle-systems. Make clear that this is not launch authorization: paid traffic stays blocked until paid-source attribution reconciles from eligibility-flow start through admission and assisted routing. Stop immediately if an unsupported applicant enters the admitted path; pause if a routing or instrumentation failure prevents reconciliation; and, after at least 20 admissions, pause if more than 10% exceed 90 support minutes. Day-7 first integration is leading only, while paid-active status, refund-adjusted collected cash, and account-level support effort need complete outcome windows before any mature claim. Existing eligibility, refusal, connector-access, external-authority, lifecycle, instrumentation, and support-routing rules remain unchanged, with no pipeline target or forecast credit.

Message 003567 in history

Fact 194

Riley Tanaka reported that the Helio Start paid-search pilot closed on May 18, 2026 at its 50-admitted-workspace cap after $14,420 in spend.

Source evidence (1)

003593May 18, 2026 / 10:05 UTC-05:00

Here’s the private Slack DM for Priya: Paid pilot closed at the admission cap: 50 admitted workspaces and $14,420 in spend. Paid-source attribution reconciles for all 50; no unsupported applicant entered the admitted path; no routing or instrumentation failure required a pause; and 4 of 50 are over 90 support minutes, below the more-than-10% pause threshold. Paid traffic is off and the cohort is fixed—no additional admission can enter this pilot. Day-7 and mature outcome reads still follow their predeclared windows, so this is not yet acquisition, revenue, pipeline, forecast, or scale evidence.

Message 003593 in history

Fact 195

Paid-source attribution reconciled for all 50 admitted workspaces in the closed Helio Start paid-search pilot.

Source evidence (1)

003593May 18, 2026 / 10:05 UTC-05:00

Here’s the private Slack DM for Priya: Paid pilot closed at the admission cap: 50 admitted workspaces and $14,420 in spend. Paid-source attribution reconciles for all 50; no unsupported applicant entered the admitted path; no routing or instrumentation failure required a pause; and 4 of 50 are over 90 support minutes, below the more-than-10% pause threshold. Paid traffic is off and the cohort is fixed—no additional admission can enter this pilot. Day-7 and mature outcome reads still follow their predeclared windows, so this is not yet acquisition, revenue, pipeline, forecast, or scale evidence.

Message 003593 in history

Fact 196

No unsupported applicant entered the admitted path during the Helio Start paid-search pilot.

Source evidence (1)

003593May 18, 2026 / 10:05 UTC-05:00

Here’s the private Slack DM for Priya: Paid pilot closed at the admission cap: 50 admitted workspaces and $14,420 in spend. Paid-source attribution reconciles for all 50; no unsupported applicant entered the admitted path; no routing or instrumentation failure required a pause; and 4 of 50 are over 90 support minutes, below the more-than-10% pause threshold. Paid traffic is off and the cohort is fixed—no additional admission can enter this pilot. Day-7 and mature outcome reads still follow their predeclared windows, so this is not yet acquisition, revenue, pipeline, forecast, or scale evidence.

Message 003593 in history

Fact 197

No routing or instrumentation failure required a pause during the Helio Start paid-search pilot.

Source evidence (1)

003593May 18, 2026 / 10:05 UTC-05:00

Here’s the private Slack DM for Priya: Paid pilot closed at the admission cap: 50 admitted workspaces and $14,420 in spend. Paid-source attribution reconciles for all 50; no unsupported applicant entered the admitted path; no routing or instrumentation failure required a pause; and 4 of 50 are over 90 support minutes, below the more-than-10% pause threshold. Paid traffic is off and the cohort is fixed—no additional admission can enter this pilot. Day-7 and mature outcome reads still follow their predeclared windows, so this is not yet acquisition, revenue, pipeline, forecast, or scale evidence.

Message 003593 in history

Fact 198

Four of the 50 admitted workspaces in the Helio Start paid-search pilot exceeded 90 support minutes, below the more-than-10% pause threshold.

Source evidence (1)

003593May 18, 2026 / 10:05 UTC-05:00

Here’s the private Slack DM for Priya: Paid pilot closed at the admission cap: 50 admitted workspaces and $14,420 in spend. Paid-source attribution reconciles for all 50; no unsupported applicant entered the admitted path; no routing or instrumentation failure required a pause; and 4 of 50 are over 90 support minutes, below the more-than-10% pause threshold. Paid traffic is off and the cohort is fixed—no additional admission can enter this pilot. Day-7 and mature outcome reads still follow their predeclared windows, so this is not yet acquisition, revenue, pipeline, forecast, or scale evidence.

Message 003593 in history

Fact 199

Paid traffic for the Helio Start paid-search pilot is off, and its fixed cohort cannot accept additional admissions.

Source evidence (1)

003593May 18, 2026 / 10:05 UTC-05:00

Here’s the private Slack DM for Priya: Paid pilot closed at the admission cap: 50 admitted workspaces and $14,420 in spend. Paid-source attribution reconciles for all 50; no unsupported applicant entered the admitted path; no routing or instrumentation failure required a pause; and 4 of 50 are over 90 support minutes, below the more-than-10% pause threshold. Paid traffic is off and the cohort is fixed—no additional admission can enter this pilot. Day-7 and mature outcome reads still follow their predeclared windows, so this is not yet acquisition, revenue, pipeline, forecast, or scale evidence.

Message 003593 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_pilot_closeout",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_175_00"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "create_pilot_closeout",
  "path": "args",
  "criterion": "The closeout says the pilot closed on May 18, 2026 at 50 admitted workspaces after $14,420 in spend, before its $15,000 cap, leaving $580 unused. Attribution reconciled for all 50; no unsupported applicant was admitted; no routing or instrumentation failure caused a pause; four of 50 exceeded 90 support minutes, below the more-than-10% pause threshold; paid traffic is off; and the fixed cohort cannot accept more admissions.",
  "check_id": "riley_175_01"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_pilot_closeout",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_175_00"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "create_pilot_closeout",
        "path": "args",
        "criterion": "The closeout says the pilot closed on May 18, 2026 at 50 admitted workspaces after $14,420 in spend, before its $15,000 cap, leaving $580 unused. Attribution reconciled for all 50; no unsupported applicant was admitted; no routing or instrumentation failure caused a pause; four of 50 exceeded 90 support minutes, below the more-than-10% pause threshold; paid traffic is off; and the fixed cohort cannot accept more admissions.",
        "check_id": "riley_175_01"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/riley/175.yaml

SHA-256: f04c4a7054fe809451e4cdcfa67288df48807e5490c1b77b1d4be8cd35278e7e