DolphinBench

Test 132

Jan 1, 2028 / 6 facts

YAML

Request

Create a historical design card for the approved 2026 Helio Start paid-search pilot. Record the acquisition type, duration, budget and closure cap, launch gate, stop and pause rules, and which measures required complete outcome windows.

Required memory

Fact 186

The Helio Start budget review was completed on April 2, 2026, and Priya approved Riley Tanaka’s recommended six-week high-intent paid-search pilot with a maximum budget of $15,000; the pilot closes at the earlier of $15,000 spent or 50 admitted workspaces.

Source evidence (1)

003567Apr 2, 2026 / 14:18 UTC-05:00

The Helio Start budget review is done. Priya approved my recommended six-week high-intent paid-search pilot with a budget of no more than $15,000, closing at the earlier of $15,000 in spend or 50 admitted workspaces. The working draft still presents that recommendation and the smaller fallback as unapproved, with budget approval and preflight both outstanding. Update `Helio Start capped paid-acquisition proposal — Q2 2026 working draft` to show budget-approved/pending-preflight status, then post the bounded decision in #lifecycle-systems. Make clear that this is not launch authorization: paid traffic stays blocked until paid-source attribution reconciles from eligibility-flow start through admission and assisted routing. Stop immediately if an unsupported applicant enters the admitted path; pause if a routing or instrumentation failure prevents reconciliation; and, after at least 20 admissions, pause if more than 10% exceed 90 support minutes. Day-7 first integration is leading only, while paid-active status, refund-adjusted collected cash, and account-level support effort need complete outcome windows before any mature claim. Existing eligibility, refusal, connector-access, external-authority, lifecycle, instrumentation, and support-routing rules remain unchanged, with no pipeline target or forecast credit.

Message 003567 in history

Fact 187

Priya’s Helio Start pilot budget approval is not launch authorization; paid traffic remains blocked pending preflight confirmation that paid-source attribution reconciles from eligibility-flow start through admission and assisted routing.

Source evidence (1)

003567Apr 2, 2026 / 14:18 UTC-05:00

The Helio Start budget review is done. Priya approved my recommended six-week high-intent paid-search pilot with a budget of no more than $15,000, closing at the earlier of $15,000 in spend or 50 admitted workspaces. The working draft still presents that recommendation and the smaller fallback as unapproved, with budget approval and preflight both outstanding. Update `Helio Start capped paid-acquisition proposal — Q2 2026 working draft` to show budget-approved/pending-preflight status, then post the bounded decision in #lifecycle-systems. Make clear that this is not launch authorization: paid traffic stays blocked until paid-source attribution reconciles from eligibility-flow start through admission and assisted routing. Stop immediately if an unsupported applicant enters the admitted path; pause if a routing or instrumentation failure prevents reconciliation; and, after at least 20 admissions, pause if more than 10% exceed 90 support minutes. Day-7 first integration is leading only, while paid-active status, refund-adjusted collected cash, and account-level support effort need complete outcome windows before any mature claim. Existing eligibility, refusal, connector-access, external-authority, lifecycle, instrumentation, and support-routing rules remain unchanged, with no pipeline target or forecast credit.

Message 003567 in history

Fact 188

The Helio Start paid-search pilot must stop immediately if an unsupported applicant enters the admitted path.

Source evidence (1)

003567Apr 2, 2026 / 14:18 UTC-05:00

The Helio Start budget review is done. Priya approved my recommended six-week high-intent paid-search pilot with a budget of no more than $15,000, closing at the earlier of $15,000 in spend or 50 admitted workspaces. The working draft still presents that recommendation and the smaller fallback as unapproved, with budget approval and preflight both outstanding. Update `Helio Start capped paid-acquisition proposal — Q2 2026 working draft` to show budget-approved/pending-preflight status, then post the bounded decision in #lifecycle-systems. Make clear that this is not launch authorization: paid traffic stays blocked until paid-source attribution reconciles from eligibility-flow start through admission and assisted routing. Stop immediately if an unsupported applicant enters the admitted path; pause if a routing or instrumentation failure prevents reconciliation; and, after at least 20 admissions, pause if more than 10% exceed 90 support minutes. Day-7 first integration is leading only, while paid-active status, refund-adjusted collected cash, and account-level support effort need complete outcome windows before any mature claim. Existing eligibility, refusal, connector-access, external-authority, lifecycle, instrumentation, and support-routing rules remain unchanged, with no pipeline target or forecast credit.

Message 003567 in history

Fact 189

The Helio Start paid-search pilot must pause if a routing or instrumentation failure prevents paid-source attribution reconciliation.

Source evidence (1)

003567Apr 2, 2026 / 14:18 UTC-05:00

The Helio Start budget review is done. Priya approved my recommended six-week high-intent paid-search pilot with a budget of no more than $15,000, closing at the earlier of $15,000 in spend or 50 admitted workspaces. The working draft still presents that recommendation and the smaller fallback as unapproved, with budget approval and preflight both outstanding. Update `Helio Start capped paid-acquisition proposal — Q2 2026 working draft` to show budget-approved/pending-preflight status, then post the bounded decision in #lifecycle-systems. Make clear that this is not launch authorization: paid traffic stays blocked until paid-source attribution reconciles from eligibility-flow start through admission and assisted routing. Stop immediately if an unsupported applicant enters the admitted path; pause if a routing or instrumentation failure prevents reconciliation; and, after at least 20 admissions, pause if more than 10% exceed 90 support minutes. Day-7 first integration is leading only, while paid-active status, refund-adjusted collected cash, and account-level support effort need complete outcome windows before any mature claim. Existing eligibility, refusal, connector-access, external-authority, lifecycle, instrumentation, and support-routing rules remain unchanged, with no pipeline target or forecast credit.

Message 003567 in history

Fact 190

After at least 20 admissions, the Helio Start paid-search pilot must pause if more than 10% exceed 90 support minutes.

Source evidence (1)

003567Apr 2, 2026 / 14:18 UTC-05:00

The Helio Start budget review is done. Priya approved my recommended six-week high-intent paid-search pilot with a budget of no more than $15,000, closing at the earlier of $15,000 in spend or 50 admitted workspaces. The working draft still presents that recommendation and the smaller fallback as unapproved, with budget approval and preflight both outstanding. Update `Helio Start capped paid-acquisition proposal — Q2 2026 working draft` to show budget-approved/pending-preflight status, then post the bounded decision in #lifecycle-systems. Make clear that this is not launch authorization: paid traffic stays blocked until paid-source attribution reconciles from eligibility-flow start through admission and assisted routing. Stop immediately if an unsupported applicant enters the admitted path; pause if a routing or instrumentation failure prevents reconciliation; and, after at least 20 admissions, pause if more than 10% exceed 90 support minutes. Day-7 first integration is leading only, while paid-active status, refund-adjusted collected cash, and account-level support effort need complete outcome windows before any mature claim. Existing eligibility, refusal, connector-access, external-authority, lifecycle, instrumentation, and support-routing rules remain unchanged, with no pipeline target or forecast credit.

Message 003567 in history

Fact 191

For the Helio Start paid-search pilot, day-7 first integration is only a leading operating indicator; mature claims about paid-active status, refund-adjusted collected cash, or account-level support effort require complete outcome windows.

Source evidence (1)

003567Apr 2, 2026 / 14:18 UTC-05:00

The Helio Start budget review is done. Priya approved my recommended six-week high-intent paid-search pilot with a budget of no more than $15,000, closing at the earlier of $15,000 in spend or 50 admitted workspaces. The working draft still presents that recommendation and the smaller fallback as unapproved, with budget approval and preflight both outstanding. Update `Helio Start capped paid-acquisition proposal — Q2 2026 working draft` to show budget-approved/pending-preflight status, then post the bounded decision in #lifecycle-systems. Make clear that this is not launch authorization: paid traffic stays blocked until paid-source attribution reconciles from eligibility-flow start through admission and assisted routing. Stop immediately if an unsupported applicant enters the admitted path; pause if a routing or instrumentation failure prevents reconciliation; and, after at least 20 admissions, pause if more than 10% exceed 90 support minutes. Day-7 first integration is leading only, while paid-active status, refund-adjusted collected cash, and account-level support effort need complete outcome windows before any mature claim. Existing eligibility, refusal, connector-access, external-authority, lifecycle, instrumentation, and support-routing rules remain unchanged, with no pipeline target or forecast credit.

Message 003567 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_pilot_design_card",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_132_00"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "create_pilot_design_card",
  "path": "args",
  "criterion": "The card records the approved six-week high-intent paid-search pilot with a maximum $15,000 budget and closure at the earlier of $15,000 spent or 50 admissions. Budget approval was not launch approval; paid traffic remained blocked until attribution could reconcile through admission and assisted routing. The pilot stops if an unsupported applicant is admitted, pauses if routing or instrumentation prevents reconciliation, and after 20 admissions pauses if more than 10% exceed 90 support minutes. Day-7 first integration is only an early indicator; paid-active status, refund-adjusted collected cash, and account-level support effort require complete outcome windows.",
  "check_id": "riley_132_01"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_pilot_design_card",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_132_00"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "create_pilot_design_card",
        "path": "args",
        "criterion": "The card records the approved six-week high-intent paid-search pilot with a maximum $15,000 budget and closure at the earlier of $15,000 spent or 50 admissions. Budget approval was not launch approval; paid traffic remained blocked until attribution could reconcile through admission and assisted routing. The pilot stops if an unsupported applicant is admitted, pauses if routing or instrumentation prevents reconciliation, and after 20 admissions pauses if more than 10% exceed 90 support minutes. Day-7 first integration is only an early indicator; paid-active status, refund-adjusted collected cash, and account-level support effort require complete outcome windows.",
        "check_id": "riley_132_01"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/riley/132.yaml

SHA-256: 516887ddd52a8ffba14627c4760b9fe0079da34d8122a5dfb675de948b4714eb