DolphinBench

Test 179

Jan 1, 2028 / 2 facts

YAML

Request

Privately route this daylight metrics-router canary question for a first-pass assessment under the deploy pipeline, with escalation only if needed: build mr-2028.01.01-canary.4; 10% slice; p99 210 to 248 ms for seven minutes; error rate 0.08%; zero dropped writes; no cross-service invariant failure.

Required memory

Fact 71

Alex Valdez and Hema settled that Wes must use the deploy pipeline rather than laptop deploys because laptop deploys bypass the canary.

Source evidence (1)

000386Aug 7, 2023 / 09:20 UTC-04:00

Wes asked before taking a small ingest-edge staging fix whether ingest-edge staging counts as canary-safe, or whether he's still limited to metrics-router non-prod under the May guidance. That exposed the annoying truth that the canary-safe environment guidance was still just a promised follow-up from the May shard-keeper laptop-deploy correction. Hema and I settled the practical rule: Wes has to use the deploy pipeline rather than laptop deploys because laptop deploys bypass the canary; he may operate solo in metrics-router non-prod and ingest-edge staging; shard-keeper stays outside his solo scope unless I or the Cyrus-team backup explicitly pairs with him. Create a short document titled Canary-safe environments — Wes deploy guidance in the Eng folder using the settled guidance below.

Message 000386 in history

Fact 100

After reviewing Wes's Q4 examples, Hema and Alex Valdez agreed that Wes is the default daylight first-pass handoff for metrics-router canary questions and ingest-edge staging rollback drills.

Source evidence (1)

000899Dec 22, 2023 / 15:50 UTC-05:00

## Dec 22, 2023 — daylight first-pass handoff After reviewing Wes's Q4 examples, including the dashboard-source cleanup and the Nov 21 metrics-router canary, Hema and Alex agreed that Wes is the default daylight first-pass handoff for: - `metrics-router` canary questions. - `ingest-edge` staging rollback drills. This is under the existing deploy-pipeline rules: - Wes must use the deploy pipeline rather than laptop deploys; laptop deploys bypass the canary. - Wes's solo safe surface continues to include `metrics-router` non-prod and `ingest-edge` staging. - Wes can make first-pass production decisions for `metrics-router` canaries and staging rollback calls for `ingest-edge` under the deploy-pipeline rules. - Alex no longer shadows every safe first-pass decision that falls inside this practical backup scope. Non-scope and ownership boundaries: - `shard-keeper` remains outside Wes's solo scope unless Alex or the Cyrus/data-platform backup explicitly pairs with him. - The `metrics-router` and `ingest-edge` owner map does not change. - Wes is a genuine practical backup, but not a formal primary owner.

Message 000899 in history

Expected tool calls

  • send_slack_dm

Grading

1. field_equals / send_slack_dm
{
  "type": "field_equals",
  "tool": "send_slack_dm",
  "action_id": "alex_179_send_slack_dm",
  "path": "result.ok",
  "value": true,
  "check_id": "alex_179_00"
}
2. field_equals / send_slack_dm
{
  "type": "field_equals",
  "path": "result.user",
  "value": "wes",
  "tool": "send_slack_dm",
  "action_id": "alex_179_send_slack_dm",
  "check_id": "alex_179_01"
}
3. field_llm_judge / send_slack_dm
{
  "type": "field_llm_judge",
  "path": "result.message",
  "criterion": "The DM asks Wes for the daylight metrics-router canary first pass under the deploy pipeline and includes the supplied build, canary percentage, p99 movement and duration, error rate, dropped-write result, and absence of a cross-service invariant failure. It requests escalation only if needed.",
  "tool": "send_slack_dm",
  "action_id": "alex_179_send_slack_dm",
  "check_id": "alex_179_02"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_slack_dm",
        "action_id": "alex_179_send_slack_dm",
        "path": "result.ok",
        "value": true,
        "check_id": "alex_179_00"
      },
      {
        "type": "field_equals",
        "path": "result.user",
        "value": "wes",
        "tool": "send_slack_dm",
        "action_id": "alex_179_send_slack_dm",
        "check_id": "alex_179_01"
      },
      {
        "type": "field_llm_judge",
        "path": "result.message",
        "criterion": "The DM asks Wes for the daylight metrics-router canary first pass under the deploy pipeline and includes the supplied build, canary percentage, p99 movement and duration, error rate, dropped-write result, and absence of a cross-service invariant failure. It requests escalation only if needed.",
        "tool": "send_slack_dm",
        "action_id": "alex_179_send_slack_dm",
        "check_id": "alex_179_02"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/alex/179.yaml

SHA-256: b7dd90708147df98215e2c7a831f572819cefc8e659d0811513cbebb4b45429b