Test 187 for Alex Valdez

Request

The Cardinality Guardrails review has separate questions about replay-versus-live alert sources and the cost impact of a metrics change. Send each question privately to its established reviewer.

Required memory

Fact 96

Nadia retains responsibility for alert semantics in the Cardinality Guardrails review path.

Source evidence (1)

000844Dec 12, 2023 / 10:36 UTC-05:00

Append this section: ## Required operating rule — narrow label-change class For production review after Dec 12, 2023, label changes that touch `metrics-router`, `shard-keeper`, or `rollup-service` must run the Cardinality Guardrails checks below before the change is treated as safe for production review. Required checks: 1. Label-cardinality preflight. - Identify new or changed label keys and changed value-shape behavior. - Show whether the value set is bounded, expected high-cardinality with an owner/budget, or effectively unbounded. - Use live canary evidence or a representative live-path fixture for live-path value-shape changes; replay evidence alone is not enough for a live-path label change. - Keep unbounded raw identifiers in logs or trace context unless a reviewed bounded metric dimension is justified. 2. Alert-source classification check. - Classify whether the relevant panel or alert signal is live-path, replay-source, mirror-source, or other validation-source evidence. - Replay or mirror movement can remain visible as investigation evidence, but it is not rollback criteria by itself. - Reviewers must not allow validation-source panels to read like live rollback triggers. Review path: - Nadia keeps alert semantics in the Cardinality Guardrails review path. - Cyrus keeps the cost/cardinality angle attached to review. - Rollup-service ownership remains with the Cyrus/data-platform side; this rule does not move enforcement ownership to data platform. Non-goals: - This is not a mature or complete Cardinality Guardrails program. - This is not a general telemetry rewrite. - This does not make Wes or any practical backup the default enforcement owner.

Message 000844 in history

Fact 97

Cyrus retains responsibility for the cost and cardinality aspects of Cardinality Guardrails reviews.

Source evidence (1)

000844Dec 12, 2023 / 10:36 UTC-05:00

Append this section: ## Required operating rule — narrow label-change class For production review after Dec 12, 2023, label changes that touch `metrics-router`, `shard-keeper`, or `rollup-service` must run the Cardinality Guardrails checks below before the change is treated as safe for production review. Required checks: 1. Label-cardinality preflight. - Identify new or changed label keys and changed value-shape behavior. - Show whether the value set is bounded, expected high-cardinality with an owner/budget, or effectively unbounded. - Use live canary evidence or a representative live-path fixture for live-path value-shape changes; replay evidence alone is not enough for a live-path label change. - Keep unbounded raw identifiers in logs or trace context unless a reviewed bounded metric dimension is justified. 2. Alert-source classification check. - Classify whether the relevant panel or alert signal is live-path, replay-source, mirror-source, or other validation-source evidence. - Replay or mirror movement can remain visible as investigation evidence, but it is not rollback criteria by itself. - Reviewers must not allow validation-source panels to read like live rollback triggers. Review path: - Nadia keeps alert semantics in the Cardinality Guardrails review path. - Cyrus keeps the cost/cardinality angle attached to review. - Rollup-service ownership remains with the Cyrus/data-platform side; this rule does not move enforcement ownership to data platform. Non-goals: - This is not a mature or complete Cardinality Guardrails program. - This is not a general telemetry rewrite. - This does not make Wes or any practical backup the default enforcement owner.

Message 000844 in history

Expected tool calls

send_slack_dm

Grading

1. field_equals / send_slack_dm

{
  "type": "field_equals",
  "tool": "send_slack_dm",
  "action_id": "route_alert_semantics",
  "path": "result.ok",
  "value": true,
  "check_id": "alex_187_00"
}

2. field_equals / send_slack_dm

{
  "type": "field_equals",
  "tool": "send_slack_dm",
  "action_id": "route_cost_review",
  "path": "result.ok",
  "value": true,
  "check_id": "alex_187_01"
}

3. field_equals / send_slack_dm

{
  "type": "field_equals",
  "action_id": "route_alert_semantics",
  "path": "result.user",
  "value": "nadia",
  "tool": "send_slack_dm",
  "check_id": "alex_187_02"
}

4. field_llm_judge / send_slack_dm

{
  "type": "field_llm_judge",
  "action_id": "route_alert_semantics",
  "path": "result.message",
  "criterion": "The DM asks Nadia to review the replay-versus-live alert-source interpretation and does not assign her the cost review.",
  "tool": "send_slack_dm",
  "check_id": "alex_187_03"
}

5. field_equals / send_slack_dm

{
  "type": "field_equals",
  "action_id": "route_cost_review",
  "path": "result.user",
  "value": "cyrus",
  "tool": "send_slack_dm",
  "check_id": "alex_187_04"
}

6. field_llm_judge / send_slack_dm

{
  "type": "field_llm_judge",
  "action_id": "route_cost_review",
  "path": "result.message",
  "criterion": "The DM asks Cyrus to review the cost and cardinality impact of the metrics change and does not assign him the alert-semantics review.",
  "tool": "send_slack_dm",
  "check_id": "alex_187_05"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_slack_dm",
        "action_id": "route_alert_semantics",
        "path": "result.ok",
        "value": true,
        "check_id": "alex_187_00"
      },
      {
        "type": "field_equals",
        "tool": "send_slack_dm",
        "action_id": "route_cost_review",
        "path": "result.ok",
        "value": true,
        "check_id": "alex_187_01"
      },
      {
        "type": "field_equals",
        "action_id": "route_alert_semantics",
        "path": "result.user",
        "value": "nadia",
        "tool": "send_slack_dm",
        "check_id": "alex_187_02"
      },
      {
        "type": "field_llm_judge",
        "action_id": "route_alert_semantics",
        "path": "result.message",
        "criterion": "The DM asks Nadia to review the replay-versus-live alert-source interpretation and does not assign her the cost review.",
        "tool": "send_slack_dm",
        "check_id": "alex_187_03"
      },
      {
        "type": "field_equals",
        "action_id": "route_cost_review",
        "path": "result.user",
        "value": "cyrus",
        "tool": "send_slack_dm",
        "check_id": "alex_187_04"
      },
      {
        "type": "field_llm_judge",
        "action_id": "route_cost_review",
        "path": "result.message",
        "criterion": "The DM asks Cyrus to review the cost and cardinality impact of the metrics change and does not assign him the alert-semantics review.",
        "tool": "send_slack_dm",
        "check_id": "alex_187_05"
      }
    ]
  }
}