Test 129

Jan 1, 2028 / 1 fact

Request

Comment on doc-rollup-ownership-retro with the rollup-service ownership lesson from before the May 2023 audit.

Required memory

Fact 40

Alex Valdez wants the April 18 outage’s technical-lessons section to explain that rollup-service’s lack of practical ownership left everyone with only a partial map and nobody with the full downstream story.

Source evidence (1)

000146May 3, 2023 / 15:00 UTC-04:00

on the technical-lessons section for the Apr 18 outage, i think the cleanest shape is just name the three things and be pretty literal. `shard-keeper rollback path` was way too easy to make worse under pressure -- looked reversible on paper, wasn't once state had partially moved. then the ownership mess: `rollup-service` was basically un-owned in practice, so everyone had a partial map and nobody had the whole downstream story. and the `label-name diff` part needs to be explicit because symptom alerts did not point at that at all. from the pages it looked like generic ingest weirdness, not "one label changed and one consumer never got the memo." on May 3 i'd rather write that plainly than over-explain incident mechanics.

Message 000146 in history

Expected tool calls

post_doc_comment

Grading

1. field_equals / post_doc_comment

{
  "type": "field_equals",
  "tool": "post_doc_comment",
  "action_id": "alex_129_post_doc_comment",
  "path": "result.ok",
  "value": true,
  "check_id": "alex_129_00"
}

2. field_equals / post_doc_comment

{
  "type": "field_equals",
  "path": "result.comment.doc_id",
  "value": "doc-rollup-ownership-retro",
  "tool": "post_doc_comment",
  "action_id": "alex_129_post_doc_comment",
  "check_id": "alex_129_01"
}

3. field_llm_judge / post_doc_comment

{
  "type": "field_llm_judge",
  "path": "result.comment.body",
  "criterion": "The comment says rollup-service lacked practical ownership before the audit, leaving people with partial maps and nobody with the full downstream story. It does not make claims about the later owner map.",
  "tool": "post_doc_comment",
  "action_id": "alex_129_post_doc_comment",
  "check_id": "alex_129_02"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "post_doc_comment",
        "action_id": "alex_129_post_doc_comment",
        "path": "result.ok",
        "value": true,
        "check_id": "alex_129_00"
      },
      {
        "type": "field_equals",
        "path": "result.comment.doc_id",
        "value": "doc-rollup-ownership-retro",
        "tool": "post_doc_comment",
        "action_id": "alex_129_post_doc_comment",
        "check_id": "alex_129_01"
      },
      {
        "type": "field_llm_judge",
        "path": "result.comment.body",
        "criterion": "The comment says rollup-service lacked practical ownership before the audit, leaving people with partial maps and nobody with the full downstream story. It does not make claims about the later owner map.",
        "tool": "post_doc_comment",
        "action_id": "alex_129_post_doc_comment",
        "check_id": "alex_129_02"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/alex/129.yaml

SHA-256: 9ea107492e1b28d3f585d21de387b23a07cd47d670509020270e2c2c95af7a9a