DolphinBench

Test 006

Jan 1, 2028 / 3 facts

YAML

Request

Create a compact Helio Start field-handling quick reference that identifies the readiness states that must remain separate. Include who records the external-authority state, who withholds setup messaging while it is pending, and who reports it.

Required memory

Fact 70

Daniela records external-authority readiness as a state separate from Helio admin proof and connector authentication.

Source evidence (1)

002262Dec 16, 2024 / 11:30 UTC-06:00

Priya and I approved a targeted Helio Start intake change based on the completed diagnostic. Applicants must separately confirm that they or an available owner can perform the owner-level permission or network-allowlist action required by the connector selected for first-week setup. A verified yes can continue; no or not sure routes to Customer Success before admission. This does not replace or broaden the existing Helio admin-proof, connector-authentication, entitlement, billing, or deployment requirements. Daniela records external-authority readiness as a separate state, Ines holds setup prompts while that state is pending, and Owen reports it separately so support effects can be evaluated without rewriting the existing admission measures. Post this approved operating instruction to growth operations, making the distinction between external authority, admin proof, and connector authentication explicit.

Message 002262 in history

Fact 71

Ines holds Helio Start setup prompts while the external-authority readiness state is pending.

Source evidence (1)

002262Dec 16, 2024 / 11:30 UTC-06:00

Priya and I approved a targeted Helio Start intake change based on the completed diagnostic. Applicants must separately confirm that they or an available owner can perform the owner-level permission or network-allowlist action required by the connector selected for first-week setup. A verified yes can continue; no or not sure routes to Customer Success before admission. This does not replace or broaden the existing Helio admin-proof, connector-authentication, entitlement, billing, or deployment requirements. Daniela records external-authority readiness as a separate state, Ines holds setup prompts while that state is pending, and Owen reports it separately so support effects can be evaluated without rewriting the existing admission measures. Post this approved operating instruction to growth operations, making the distinction between external authority, admin proof, and connector authentication explicit.

Message 002262 in history

Fact 72

Owen reports Helio Start external-authority readiness separately so support effects can be evaluated without rewriting existing admission measures.

Source evidence (1)

002262Dec 16, 2024 / 11:30 UTC-06:00

Priya and I approved a targeted Helio Start intake change based on the completed diagnostic. Applicants must separately confirm that they or an available owner can perform the owner-level permission or network-allowlist action required by the connector selected for first-week setup. A verified yes can continue; no or not sure routes to Customer Success before admission. This does not replace or broaden the existing Helio admin-proof, connector-authentication, entitlement, billing, or deployment requirements. Daniela records external-authority readiness as a separate state, Ines holds setup prompts while that state is pending, and Owen reports it separately so support effects can be evaluated without rewriting the existing admission measures. Post this approved operating instruction to growth operations, making the distinction between external authority, admin proof, and connector authentication explicit.

Message 002262 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_field_reference",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_006_00"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "create_field_reference",
  "path": "args",
  "criterion": "The body presents this as Helio Start guidance and unambiguously treats external-authority readiness, Helio admin proof, and connector authentication as distinct states rather than aliases or one combined state.\n\nThe body assigns Daniela responsibility for recording external-authority readiness as its own state.\n\nThe body assigns Ines responsibility for holding, withholding, or suppressing setup prompts while the external-authority state is pending.\n\nThe body assigns Owen responsibility for reporting the external-authority state separately rather than folding it into other admission or readiness measures.",
  "check_id": "riley_006_01"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_field_reference",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_006_00"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "create_field_reference",
        "path": "args",
        "criterion": "The body presents this as Helio Start guidance and unambiguously treats external-authority readiness, Helio admin proof, and connector authentication as distinct states rather than aliases or one combined state.\n\nThe body assigns Daniela responsibility for recording external-authority readiness as its own state.\n\nThe body assigns Ines responsibility for holding, withholding, or suppressing setup prompts while the external-authority state is pending.\n\nThe body assigns Owen responsibility for reporting the external-authority state separately rather than folding it into other admission or readiness measures.",
        "check_id": "riley_006_01"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/riley/006.yaml

SHA-256: be4851f74c93af445bb2ea01f251fd28262a46c0421d94a450182fedcd298cbc