DolphinBench

Test 060

Jan 1, 2028 / 4 facts

YAML

Request

Create a concise, current operating checklist for the lifecycle team covering routine-send release, audit and stop conditions, state-changing preflight requirements, and escalation. Use each person's name for routine release, preflight exceptions, customer corrections and incident stops, and engineering release controls. Do not use `you`, `the requester`, or a generic role in place of a person's name. Make the checklist usable without case-by-case approval of each low-risk send.

Required memory

Fact 62

Priya granted Ines lasting authority to approve and release routine, non-state-changing lifecycle sends, limited to three sends per week, with each send audited within 24 hours and missing or conflicting state failing closed.

Source evidence (1)

002141Nov 8, 2024 / 09:30 UTC-06:00

Post the lifecycle ownership decision to `#growth-ops`. State that Priya granted Ines lasting authority to approve and release routine, non-state-changing lifecycle sends within the ceiling of no more than three sends per week, with every send audited within 24 hours and missing or conflicting state failing closed. State that I remain the stop owner for customer corrections and incidents and retain authority over state-changing, ambiguous, and exception-heavy sends. Daniela retains all engineering release controls, and Ines has no engineering release-control access.

Message 002141 in history

Fact 6

The six-week lifecycle QA engagement is closed.

Source evidence (1)

001180Feb 23, 2024 / 08:32 UTC-06:00

The six-week lifecycle QA engagement is closed. The final inventory landed at 43 active Customer.io campaigns, and we retired nine stale test campaigns. Active-suppression checks and recipient-level merge-alias checks are now in the preflight for 14 state-changing sends, and the contractor left a risk-tiered owner map that splits routine checks from the small exception set that still needs my judgment. Ines ran the final two preflights from the new checklist without me rechecking either audience, and both were clean for duplicate and suppressed recipients.

Message 001180 in history

Fact 7

Active-suppression and recipient-level merge-alias checks are now included in the preflight for 14 state-changing sends.

Source evidence (1)

001180Feb 23, 2024 / 08:32 UTC-06:00

The six-week lifecycle QA engagement is closed. The final inventory landed at 43 active Customer.io campaigns, and we retired nine stale test campaigns. Active-suppression checks and recipient-level merge-alias checks are now in the preflight for 14 state-changing sends, and the contractor left a risk-tiered owner map that splits routine checks from the small exception set that still needs my judgment. Ines ran the final two preflights from the new checklist without me rechecking either audience, and both were clean for duplicate and suppressed recipients.

Message 001180 in history

Fact 63

Riley Tanaka requested that `#growth-ops` state that Riley remains the stop owner for customer corrections and incidents and retains authority over state-changing, ambiguous, and exception-heavy lifecycle sends.

Source evidence (3)

002141Nov 8, 2024 / 09:30 UTC-06:00

Post the lifecycle ownership decision to `#growth-ops`. State that Priya granted Ines lasting authority to approve and release routine, non-state-changing lifecycle sends within the ceiling of no more than three sends per week, with every send audited within 24 hours and missing or conflicting state failing closed. State that I remain the stop owner for customer corrections and incidents and retain authority over state-changing, ambiguous, and exception-heavy sends. Daniela retains all engineering release controls, and Ines has no engineering release-control access.

Message 002141 in history
001573Jun 5, 2024 / 12:10 UTC-05:00

I've incorporated Priya's requested revision into my final midyear self-review. The outcome section separates my judgment from the implementation owned by Owen, Daniela, Ines, and Customer Success; the growth-edge section owns my delayed leadership call and names a concrete H2 commitment. The review is due Friday. Email Priya the final draft below today with the subject `Midyear self-review — Riley Tanaka`. Outcomes and judgment I defined the evidence and decision boundaries for Helio Start from signup through the first paid month. I kept admission routing, pre-admission assistance, post-admission support, activation, and first-paid evidence distinct; pushed for the connector-access check when credential failures were obscuring setup fit; and framed the May decision around what the evidence could support. That helped Helio continue the bounded organic path through Q3 without turning early activation evidence into a paid-acquisition or pipeline claim. Team contribution The operating result was shared work. Owen owned the reporting implementation and denominator checks, Daniela built and validated the instrumentation, Ines carried the lifecycle QA, and Customer Success operated the assisted routes. My contribution was setting the questions, making the evidence limits explicit, and ensuring that the continuation decision did not claim more than the team had measured. Growth edge and H2 commitment I waited too long to name which ambiguities needed an explicit owner, especially timestamp semantics, finance dependencies, and validation responsibility. That was a leadership delay, not just a tooling issue. In H2, I will assign ownership for metric definitions, finance inputs, and validation at the start of evidence work, with unresolved semantics recorded before implementation begins.

Message 001573 in history
002018Oct 3, 2024 / 08:35 UTC-05:00

Priya approved a 30-day lifecycle ownership trial beginning October 8, with the October 8 onboarding-help audience as the first governed send. During the trial, Ines may make the governed pre-send and release decisions, with no more than three sends per week, an audit within 24 hours, fail-closed treatment of missing or conflicting state, and an automatic stop routed to me if a customer correction is required. Ines gets no engineering release-control access, and Priya has reserved the permanent ownership decision until the trial is complete. Post this approval and its boundaries to the existing lifecycle working document `doc_1724944200005`.

Message 002018 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_release_checklist",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_060_00"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "create_release_checklist",
  "path": "args",
  "criterion": "The body is a concise, actionable operating checklist for lifecycle routine-send release, rather than merely a narrative summary.\n\nThe checklist clearly states that Ines may independently approve and release routine, non-state-changing lifecycle sends without Riley's case-by-case approval.\n\nThe checklist imposes an exact ceiling of no more than three routine lifecycle sends per week.\n\nThe checklist requires every routine, non-state-changing lifecycle send released under Ines's delegated authority to be audited within 24 hours. It need not apply this delegated-lane audit condition to state-changing sends, which are outside Ines's routine release authority.\n\nThe checklist says missing or conflicting required state fails closed, meaning the send must not be released while that condition exists.\n\nFor state-changing sends, the checklist requires both an active-suppression check and a recipient-level merge-alias check during preflight.\n\nThe checklist draws a clear escalation boundary: routine non-state-changing work stays in Ines's lane, while preflight exceptions go to Riley; customer corrections and incident stops remain with Riley, and state-changing, ambiguous, or exception-heavy sends are outside routine release authority and must be escalated rather than released under it.",
  "check_id": "riley_060_01"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_release_checklist",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_060_00"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "create_release_checklist",
        "path": "args",
        "criterion": "The body is a concise, actionable operating checklist for lifecycle routine-send release, rather than merely a narrative summary.\n\nThe checklist clearly states that Ines may independently approve and release routine, non-state-changing lifecycle sends without Riley's case-by-case approval.\n\nThe checklist imposes an exact ceiling of no more than three routine lifecycle sends per week.\n\nThe checklist requires every routine, non-state-changing lifecycle send released under Ines's delegated authority to be audited within 24 hours. It need not apply this delegated-lane audit condition to state-changing sends, which are outside Ines's routine release authority.\n\nThe checklist says missing or conflicting required state fails closed, meaning the send must not be released while that condition exists.\n\nFor state-changing sends, the checklist requires both an active-suppression check and a recipient-level merge-alias check during preflight.\n\nThe checklist draws a clear escalation boundary: routine non-state-changing work stays in Ines's lane, while preflight exceptions go to Riley; customer corrections and incident stops remain with Riley, and state-changing, ambiguous, or exception-heavy sends are outside routine release authority and must be escalated rather than released under it.",
        "check_id": "riley_060_01"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/riley/060.yaml

SHA-256: a09015420f3c8e6d3347d0590040e2dbdff59b5b1e9c84a7931ccf7423ff52fe