Test 131

Jan 1, 2028 / 1 fact

Request

Inspect alerting#904 and privately ask the current alert-semantics owner to review it. In the DM, state the correct meaning of queue refusal and the correct meaning of the canary behavior, without assigning implementation work.

Required memory

Fact 96

Nadia retains responsibility for alert semantics in the Cardinality Guardrails review path.

Source evidence (1)

000844Dec 12, 2023 / 10:36 UTC-05:00

Append this section: ## Required operating rule — narrow label-change class For production review after Dec 12, 2023, label changes that touch `metrics-router`, `shard-keeper`, or `rollup-service` must run the Cardinality Guardrails checks below before the change is treated as safe for production review. Required checks: 1. Label-cardinality preflight. - Identify new or changed label keys and changed value-shape behavior. - Show whether the value set is bounded, expected high-cardinality with an owner/budget, or effectively unbounded. - Use live canary evidence or a representative live-path fixture for live-path value-shape changes; replay evidence alone is not enough for a live-path label change. - Keep unbounded raw identifiers in logs or trace context unless a reviewed bounded metric dimension is justified. 2. Alert-source classification check. - Classify whether the relevant panel or alert signal is live-path, replay-source, mirror-source, or other validation-source evidence. - Replay or mirror movement can remain visible as investigation evidence, but it is not rollback criteria by itself. - Reviewers must not allow validation-source panels to read like live rollback triggers. Review path: - Nadia keeps alert semantics in the Cardinality Guardrails review path. - Cyrus keeps the cost/cardinality angle attached to review. - Rollup-service ownership remains with the Cyrus/data-platform side; this rule does not move enforcement ownership to data platform. Non-goals: - This is not a mature or complete Cardinality Guardrails program. - This is not a general telemetry rewrite. - This does not make Wes or any practical backup the default enforcement owner.

Message 000844 in history

Expected tool calls

get_pr
send_slack_dm

Grading

1. field_equals / send_slack_dm

{
  "type": "field_equals",
  "tool": "send_slack_dm",
  "action_id": "alex_131_send_slack_dm",
  "path": "result.ok",
  "value": true,
  "check_id": "alex_131_00"
}

2. field_equals / get_pr

{
  "type": "field_equals",
  "tool": "get_pr",
  "action_id": "alex_131_read_pr",
  "path": "result.pr.id",
  "value": "alerting#904",
  "check_id": "alex_131_01"
}

3. field_equals / send_slack_dm

{
  "type": "field_equals",
  "path": "result.user",
  "value": "nadia",
  "tool": "send_slack_dm",
  "action_id": "alex_131_send_slack_dm",
  "check_id": "alex_131_02"
}

4. field_llm_judge / send_slack_dm

{
  "type": "field_llm_judge",
  "path": "result.message",
  "criterion": "The DM asks Nadia to review alerting#904. It says queue-refusal wording should describe retryable rejection before acceptance rather than accepted-work loss, and canary wording should describe promotion being held while health returns to baseline rather than a failed rollback. It asks only for alert-semantics review, not implementation work.",
  "tool": "send_slack_dm",
  "action_id": "alex_131_send_slack_dm",
  "check_id": "alex_131_03"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_slack_dm",
        "action_id": "alex_131_send_slack_dm",
        "path": "result.ok",
        "value": true,
        "check_id": "alex_131_00"
      },
      {
        "type": "field_equals",
        "tool": "get_pr",
        "action_id": "alex_131_read_pr",
        "path": "result.pr.id",
        "value": "alerting#904",
        "check_id": "alex_131_01"
      },
      {
        "type": "field_equals",
        "path": "result.user",
        "value": "nadia",
        "tool": "send_slack_dm",
        "action_id": "alex_131_send_slack_dm",
        "check_id": "alex_131_02"
      },
      {
        "type": "field_llm_judge",
        "path": "result.message",
        "criterion": "The DM asks Nadia to review alerting#904. It says queue-refusal wording should describe retryable rejection before acceptance rather than accepted-work loss, and canary wording should describe promotion being held while health returns to baseline rather than a failed rollback. It asks only for alert-semantics review, not implementation work.",
        "tool": "send_slack_dm",
        "action_id": "alex_131_send_slack_dm",
        "check_id": "alex_131_03"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/alex/131.yaml

SHA-256: affda7a898239c97b04eac957c3b60ec00e694c3562eaab1de23aa4bf1224426