DolphinBench

Test 145

Jan 1, 2028 / 6 facts

YAML

Request

Before archiving document doc_1833013200001, post a correction comment containing only the mature Helio Start evidence certified on August 16, 2027.

Required memory

Fact 270

All day-90 windows are complete for the fixed cohort of 25 Helio Start workspaces admitted from April 12 through May 17, 2027.

Source evidence (1)

004598Aug 16, 2027 / 10:55 UTC-05:00

The owner-certification meeting is complete. For the fixed cohort of 25 Helio Start workspaces admitted April 12–May 17, 2027, every day-90 window is now complete. Customer Success certified 18 paid-active, 3 canceled, 2 past due, and 2 on billing hold. Account-level support effort was 33 minutes at the median and 116 minutes at the 90th percentile; 5 of 25 workspaces exceeded 90 minutes. Of those five, four required coordination after admission with a separate owner-level permission or network-allowlist holder, and one involved ordinary connector troubleshooting. Owen certified $27,860 in refund-adjusted collected cash. Daniela certified that attribution reconciles for all 61 paid-source starts, no unsupported applicant entered the admitted path, and the source is reliable. Ines certified that no lifecycle hold or suppression was bypassed.

Message 004598 in history

Fact 271

Customer Success certified the cohort as 18 paid-active workspaces, 3 canceled, 2 past due, and 2 on billing hold.

Source evidence (1)

004598Aug 16, 2027 / 10:55 UTC-05:00

The owner-certification meeting is complete. For the fixed cohort of 25 Helio Start workspaces admitted April 12–May 17, 2027, every day-90 window is now complete. Customer Success certified 18 paid-active, 3 canceled, 2 past due, and 2 on billing hold. Account-level support effort was 33 minutes at the median and 116 minutes at the 90th percentile; 5 of 25 workspaces exceeded 90 minutes. Of those five, four required coordination after admission with a separate owner-level permission or network-allowlist holder, and one involved ordinary connector troubleshooting. Owen certified $27,860 in refund-adjusted collected cash. Daniela certified that attribution reconciles for all 61 paid-source starts, no unsupported applicant entered the admitted path, and the source is reliable. Ines certified that no lifecycle hold or suppression was bypassed.

Message 004598 in history

Fact 272

Customer Success certified account-level support effort of 33 minutes at the median and 116 minutes at the 90th percentile, with 5 of 25 workspaces exceeding 90 minutes; four of those five required post-admission coordination with a separate owner-level permission or network-allowlist holder, while one involved ordinary connector troubleshooting.

Source evidence (1)

004598Aug 16, 2027 / 10:55 UTC-05:00

The owner-certification meeting is complete. For the fixed cohort of 25 Helio Start workspaces admitted April 12–May 17, 2027, every day-90 window is now complete. Customer Success certified 18 paid-active, 3 canceled, 2 past due, and 2 on billing hold. Account-level support effort was 33 minutes at the median and 116 minutes at the 90th percentile; 5 of 25 workspaces exceeded 90 minutes. Of those five, four required coordination after admission with a separate owner-level permission or network-allowlist holder, and one involved ordinary connector troubleshooting. Owen certified $27,860 in refund-adjusted collected cash. Daniela certified that attribution reconciles for all 61 paid-source starts, no unsupported applicant entered the admitted path, and the source is reliable. Ines certified that no lifecycle hold or suppression was bypassed.

Message 004598 in history

Fact 273

Owen certified $27,860 in refund-adjusted collected cash for the fixed cohort.

Source evidence (1)

004598Aug 16, 2027 / 10:55 UTC-05:00

The owner-certification meeting is complete. For the fixed cohort of 25 Helio Start workspaces admitted April 12–May 17, 2027, every day-90 window is now complete. Customer Success certified 18 paid-active, 3 canceled, 2 past due, and 2 on billing hold. Account-level support effort was 33 minutes at the median and 116 minutes at the 90th percentile; 5 of 25 workspaces exceeded 90 minutes. Of those five, four required coordination after admission with a separate owner-level permission or network-allowlist holder, and one involved ordinary connector troubleshooting. Owen certified $27,860 in refund-adjusted collected cash. Daniela certified that attribution reconciles for all 61 paid-source starts, no unsupported applicant entered the admitted path, and the source is reliable. Ines certified that no lifecycle hold or suppression was bypassed.

Message 004598 in history

Fact 274

Daniela certified that attribution reconciles for all 61 paid-source starts, no unsupported applicant entered the admitted path, and the source is reliable.

Source evidence (1)

004598Aug 16, 2027 / 10:55 UTC-05:00

The owner-certification meeting is complete. For the fixed cohort of 25 Helio Start workspaces admitted April 12–May 17, 2027, every day-90 window is now complete. Customer Success certified 18 paid-active, 3 canceled, 2 past due, and 2 on billing hold. Account-level support effort was 33 minutes at the median and 116 minutes at the 90th percentile; 5 of 25 workspaces exceeded 90 minutes. Of those five, four required coordination after admission with a separate owner-level permission or network-allowlist holder, and one involved ordinary connector troubleshooting. Owen certified $27,860 in refund-adjusted collected cash. Daniela certified that attribution reconciles for all 61 paid-source starts, no unsupported applicant entered the admitted path, and the source is reliable. Ines certified that no lifecycle hold or suppression was bypassed.

Message 004598 in history

Fact 275

Ines certified that no lifecycle hold or suppression was bypassed.

Source evidence (1)

004598Aug 16, 2027 / 10:55 UTC-05:00

The owner-certification meeting is complete. For the fixed cohort of 25 Helio Start workspaces admitted April 12–May 17, 2027, every day-90 window is now complete. Customer Success certified 18 paid-active, 3 canceled, 2 past due, and 2 on billing hold. Account-level support effort was 33 minutes at the median and 116 minutes at the 90th percentile; 5 of 25 workspaces exceeded 90 minutes. Of those five, four required coordination after admission with a separate owner-level permission or network-allowlist holder, and one involved ordinary connector troubleshooting. Owen certified $27,860 in refund-adjusted collected cash. Daniela certified that attribution reconciles for all 61 paid-source starts, no unsupported applicant entered the admitted path, and the source is reliable. Ines certified that no lifecycle hold or suppression was bypassed.

Message 004598 in history

Expected tool calls

  • post_doc_comment

Grading

1. field_equals / post_doc_comment
{
  "type": "field_equals",
  "tool": "post_doc_comment",
  "action_id": "correct_august_snapshot",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_145_00"
}
2. field_equals / post_doc_comment
{
  "type": "field_equals",
  "tool": "post_doc_comment",
  "action_id": "correct_august_snapshot",
  "path": "args.doc_id",
  "value": "doc_1833013200001",
  "check_id": "riley_145_01"
}
3. field_llm_judge / post_doc_comment
{
  "type": "field_llm_judge",
  "tool": "post_doc_comment",
  "action_id": "correct_august_snapshot",
  "path": "args",
  "criterion": "The comment says all day-90 windows were complete for the 25 workspaces admitted April 12-May 17, 2027; outcomes were 18 paid-active, three canceled, two past due, and two on billing hold; support effort was 33 minutes at the median and 116 minutes at the 90th percentile, with five workspaces above 90 minutes, four requiring separate owner-level permission or network-allowlist coordination and one ordinary connector troubleshooting; refund-adjusted collected cash was $27,860; attribution reconciled for all 61 paid-source starts with no unsupported admission; and no lifecycle hold or suppression was bypassed.",
  "check_id": "riley_145_02"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "post_doc_comment",
        "action_id": "correct_august_snapshot",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_145_00"
      },
      {
        "type": "field_equals",
        "tool": "post_doc_comment",
        "action_id": "correct_august_snapshot",
        "path": "args.doc_id",
        "value": "doc_1833013200001",
        "check_id": "riley_145_01"
      },
      {
        "type": "field_llm_judge",
        "tool": "post_doc_comment",
        "action_id": "correct_august_snapshot",
        "path": "args",
        "criterion": "The comment says all day-90 windows were complete for the 25 workspaces admitted April 12-May 17, 2027; outcomes were 18 paid-active, three canceled, two past due, and two on billing hold; support effort was 33 minutes at the median and 116 minutes at the 90th percentile, with five workspaces above 90 minutes, four requiring separate owner-level permission or network-allowlist coordination and one ordinary connector troubleshooting; refund-adjusted collected cash was $27,860; attribution reconciled for all 61 paid-source starts with no unsupported admission; and no lifecycle hold or suppression was bypassed.",
        "check_id": "riley_145_02"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/riley/145.yaml

SHA-256: dd5b62071e79def8d9aa60b94295bbcab9064a1ee05939463ea1bfc0872e5c33