Test 184 for Alex Valdez

Request

Inspect shard-keeper#4210 and review it against the established production lease-handoff behavior and runbook checks. State what the production revocations showed, which ordering the replacement must preserve, and what evidence it must provide.

Required memory

Fact 116

Synthetic revocations across the remaining shard-keeper production population preserved exactly one serving holder, closed each old holder’s serving gate before another routing response, and produced no stale reads; brief latency and error movement returned to baseline without restarts.

Source evidence (1)

001614Jul 3, 2024 / 09:35 UTC-04:00

The mapped shard-keeper owner completed the full-production rollout. Synthetic revocations across the remaining production population again preserved exactly one serving holder, closed each old holder’s gate before another routing response, and recorded no stale reads. Brief latency and error movement returned to baseline without restarts. Update the existing `rb_1710794400000` runbook entry to state that lease-loss notification closes the serving gate before another response, the periodic poll is only a backup detector, and production verification must include holder count, gate-close latency, stale responses, replacement acquisition, p99, errors, and restarts. The owner map and mapped deployment responsibility remain unchanged.

Message 001614 in history

Fact 117

Alex Valdez requested that runbook entry `rb_1710794400000` state that lease-loss notification closes the shard-keeper serving gate before another response, that periodic polling is only a backup detector, and that production verification must cover holder count, gate-close latency, stale responses, replacement acquisition, p99 latency, errors, and restarts.

Source evidence (1)

001614Jul 3, 2024 / 09:35 UTC-04:00

The mapped shard-keeper owner completed the full-production rollout. Synthetic revocations across the remaining production population again preserved exactly one serving holder, closed each old holder’s gate before another routing response, and recorded no stale reads. Brief latency and error movement returned to baseline without restarts. Update the existing `rb_1710794400000` runbook entry to state that lease-loss notification closes the serving gate before another response, the periodic poll is only a backup detector, and production verification must include holder count, gate-close latency, stale responses, replacement acquisition, p99, errors, and restarts. The owner map and mapped deployment responsibility remain unchanged.

Message 001614 in history

Expected tool calls

get_pr
review_pr

Grading

1. field_equals / review_pr

{
  "type": "field_equals",
  "tool": "review_pr",
  "action_id": "alex_184_review_pr",
  "path": "result.ok",
  "value": true,
  "check_id": "alex_184_00"
}

2. field_equals / get_pr

{
  "type": "field_equals",
  "tool": "get_pr",
  "action_id": "alex_184_read_pr",
  "path": "result.pr.id",
  "value": "shard-keeper#4210",
  "check_id": "alex_184_01"
}

3. field_equals / review_pr

{
  "type": "field_equals",
  "path": "result.review.pr_id",
  "value": "shard-keeper#4210",
  "tool": "review_pr",
  "action_id": "alex_184_review_pr",
  "check_id": "alex_184_02"
}

4. field_equals / review_pr

{
  "type": "field_equals",
  "path": "result.review.decision",
  "value": "request_changes",
  "tool": "review_pr",
  "action_id": "alex_184_review_pr",
  "check_id": "alex_184_03"
}

5. field_llm_judge / review_pr

{
  "type": "field_llm_judge",
  "path": "result.review.body",
  "criterion": "The review says production revocations preserved exactly one serving holder, closed the old holder's serving gate on lease-loss notification before another routing response, and produced no stale reads. It rejects a polling or grace-period gate and says polling is only a backup detector. It requires evidence for holder count, gate-close latency, stale responses, replacement acquisition, p99 latency, errors, and restarts.",
  "tool": "review_pr",
  "action_id": "alex_184_review_pr",
  "check_id": "alex_184_04"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "review_pr",
        "action_id": "alex_184_review_pr",
        "path": "result.ok",
        "value": true,
        "check_id": "alex_184_00"
      },
      {
        "type": "field_equals",
        "tool": "get_pr",
        "action_id": "alex_184_read_pr",
        "path": "result.pr.id",
        "value": "shard-keeper#4210",
        "check_id": "alex_184_01"
      },
      {
        "type": "field_equals",
        "path": "result.review.pr_id",
        "value": "shard-keeper#4210",
        "tool": "review_pr",
        "action_id": "alex_184_review_pr",
        "check_id": "alex_184_02"
      },
      {
        "type": "field_equals",
        "path": "result.review.decision",
        "value": "request_changes",
        "tool": "review_pr",
        "action_id": "alex_184_review_pr",
        "check_id": "alex_184_03"
      },
      {
        "type": "field_llm_judge",
        "path": "result.review.body",
        "criterion": "The review says production revocations preserved exactly one serving holder, closed the old holder's serving gate on lease-loss notification before another routing response, and produced no stale reads. It rejects a polling or grace-period gate and says polling is only a backup detector. It requires evidence for holder count, gate-close latency, stale responses, replacement acquisition, p99 latency, errors, and restarts.",
        "tool": "review_pr",
        "action_id": "alex_184_review_pr",
        "check_id": "alex_184_04"
      }
    ]
  }
}