DolphinBench

Test 017

Jan 1, 2028 / 3 facts

YAML

Request

Create a short ownership-map document for maintaining the standing investigation framework so teams can route standard readout-shape and example updates, and lifecycle-risk content, without sending every change to me.

Required memory

Fact 15

Owen will maintain the investigation framework’s standard readout shape and examples beyond Q1.

Source evidence (1)

001251Mar 15, 2024 / 14:12 UTC-05:00

We finished the beyond-Q1 decision meeting, and Priya and I are keeping the investigation framework as the standing readout practice for Growth and Customer Success. It stays lightweight: denominator and effect size come before the conclusion, first significance still doesn't justify rollout until the result stabilizes, and lifecycle audience risk stays in the review. We explicitly did not turn it into a companywide mandatory template or a queue where every readout has to run through me, Owen, or Ines. Owen will maintain the standard readout shape and examples, Ines will maintain the lifecycle-risk portion, and that Q1 audit decision is done.

Message 001251 in history

Fact 16

Ines will maintain the investigation framework’s lifecycle-risk portion beyond Q1.

Source evidence (1)

001251Mar 15, 2024 / 14:12 UTC-05:00

We finished the beyond-Q1 decision meeting, and Priya and I are keeping the investigation framework as the standing readout practice for Growth and Customer Success. It stays lightweight: denominator and effect size come before the conclusion, first significance still doesn't justify rollout until the result stabilizes, and lifecycle audience risk stays in the review. We explicitly did not turn it into a companywide mandatory template or a queue where every readout has to run through me, Owen, or Ines. Owen will maintain the standard readout shape and examples, Ines will maintain the lifecycle-risk portion, and that Q1 audit decision is done.

Message 001251 in history

Fact 422

Ines owns the Q1 investigation framework's lifecycle-risk review language.

Source evidence (1)

000917Dec 8, 2023 / 15:24 UTC-06:00

The retro landed. Priya asked me to make the investigation framework the default Q1 readout practice for Growth and Customer Success, and I agreed, but not as a fully companywide mandatory process. My condition was that the practice be taught and maintained by the team rather than turning into a centralized approval step: Owen owns the standard readout shape, including denominator, effect size, and conclusion order, and Ines owns lifecycle-risk review language. We also kept effect size visible and preserved the rule that first significance is followed by a stabilization cohort before rollout.

Message 000917 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "ownership_map_doc",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_017_00"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "ownership_map_doc",
  "path": "args",
  "criterion": "The document clearly routes maintenance of the standing investigation framework's standard readout shape and examples to Owen.\n\nThe document clearly routes maintenance of the standing investigation framework's lifecycle-risk content to Ines.",
  "check_id": "riley_017_01"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "ownership_map_doc",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_017_00"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "ownership_map_doc",
        "path": "args",
        "criterion": "The document clearly routes maintenance of the standing investigation framework's standard readout shape and examples to Owen.\n\nThe document clearly routes maintenance of the standing investigation framework's lifecycle-risk content to Ines.",
        "check_id": "riley_017_01"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/riley/017.yaml

SHA-256: ac549e04fb1062ee4099aaab0476627446fe5cf5149dae161752c06fc55c700c