DolphinBench

Test 012

Jan 1, 2028 / 1 fact

YAML

Request

Create a compact lifecycle-design checklist for future Helio Start touches. Before reviewers approve copy or automation, require them to record the approved lifecycle-fork choice and the approved decision on wholesale reuse of the retained-mid onboarding and lifecycle path. State the approved answers; do not leave either item as an open choice.

Required memory

Fact 401

Riley Tanaka accepted Ines's recommendation to use a conservative initial lifecycle fork for Helio Start rather than borrowing the retained-mid onboarding and lifecycle path wholesale.

Source evidence (1)

000342Jun 23, 2023 / 08:35 UTC-05:00

Ines finished the Helio Start lifecycle-fork map, and the big takeaway is that we can't borrow the retained-mid onboarding and lifecycle path wholesale. She found three concrete risk areas: the existing sequence assumes a multi-admin invite path, assumes the integration owner already has enough permission context, and pushes upgrade or team-use language before simple self-serve workspaces understand imports and billing. I accepted her recommendation to keep the initial fork conservative. We're not rewriting the whole lifecycle program at once; heavier QA should apply only to state-changing lifecycle touches, while lower-risk mapping and non-state-changing education can stay lighter. No deliverable right now. I'm noting the decision and ownership.

Message 000342 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_helio_start_lifecycle_checklist",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_012_00"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "create_helio_start_lifecycle_checklist",
  "path": "args",
  "criterion": "The body is a compact, actionable lifecycle-design checklist for future Helio Start touches rather than an essay. Before copy or automation can be approved, it must require reviewers to confirm a conservative Helio Start-specific lifecycle fork and explicitly disallow wholesale borrowing of the retained-mid onboarding or lifecycle path. Equivalent wording and checklist formats are acceptable.",
  "check_id": "riley_012_01"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_helio_start_lifecycle_checklist",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_012_00"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "create_helio_start_lifecycle_checklist",
        "path": "args",
        "criterion": "The body is a compact, actionable lifecycle-design checklist for future Helio Start touches rather than an essay. Before copy or automation can be approved, it must require reviewers to confirm a conservative Helio Start-specific lifecycle fork and explicitly disallow wholesale borrowing of the retained-mid onboarding or lifecycle path. Equivalent wording and checklist formats are acceptable.",
        "check_id": "riley_012_01"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/riley/012.yaml

SHA-256: e6052dce00d99f0027783c00edb519a900a0ae380d80db606cbd9144a1bd907f