Test 074

Jan 1, 2028 / 1 fact

Request

Please send Owen a concise Slack DM about the handoff for my first quarterly evidence audit of 2028. Ask him to alert me at the appropriate point and include the audit scope he should expect.

Required memory

Fact 230

Once per quarter after Owen closes the four-lane packet, Riley Tanaka will select one lane and independently recheck its denominator or coverage dates, outcome window, exclusions, reliability evidence, and revenue and forecast treatment.

Source evidence (1)

003831Dec 4, 2026 / 11:18 UTC-06:00

Priya approved the Lifecycle Systems operating model in this morning’s decision meeting, effective January 5, 2027. Through January 4, I continue my current routine preparation and source-closure attendance; nothing changes early. Owen’s monthly sanitized campaign snapshot remains due on the second business day. Ines’s lifecycle-governance and owner-state certification and Daniela’s source-reliability and release certification are due by the fourth business day. By the fifth business day, Owen must close or explicitly leave incomplete the routine four-lane source review and send one closure-and-exception summary. Beginning January 5, I stop attending routine monthly preparation and closure reviews and enter for missing required fields, disputed semantics, cross-surface discrepancies, customer corrections or incident stops, or causal, revenue, forecast, board-readiness, or executive-language decisions. Once per quarter, after Owen closes the packet, I’ll select one lane and independently recheck its denominator or coverage dates, outcome window, exclusions, reliability evidence, and revenue and forecast treatment. That recheck does not reopen source closure, approve Ines’s certification, or exercise Daniela’s release or rollback controls. Update `2027 Lifecycle Systems leadership model — December 4 decision draft` into the approved decision record with the January 5 effective date and these boundaries, then post the concise implementation summary in #lifecycle-systems for Owen, Ines, and Daniela, including that no attendance or authority change takes effect before January 5.

Message 003831 in history

Expected tool calls

send_slack_dm

Grading

1. field_equals / send_slack_dm

{
  "type": "field_equals",
  "tool": "send_slack_dm",
  "action_id": "dm_owen_2028_q1_audit_handoff",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_074_00"
}

2. field_equals / send_slack_dm

{
  "type": "field_equals",
  "tool": "send_slack_dm",
  "action_id": "dm_owen_2028_q1_audit_handoff",
  "path": "args.user",
  "value": "owen",
  "check_id": "riley_074_01"
}

3. field_llm_judge / send_slack_dm

{
  "type": "field_llm_judge",
  "tool": "send_slack_dm",
  "action_id": "dm_owen_2028_q1_audit_handoff",
  "path": "args",
  "criterion": "The message asks Owen to alert Riley only after he has closed the relevant routine packet for her first quarterly evidence audit of 2028. It is acceptable to call this a monthly or routine packet and unnecessary to label the packet itself as quarterly or four-lane, provided the audit handoff is clearly post-closure; a request to begin or notify before closure fails.\n\nIn the same post-closure audit context, the message states that Riley will select one lane and independently recheck it. This passes without explicitly calling the packet four-lane. It fails if every lane is to be audited, if Owen rather than Riley selects the lane, or if the recheck is not independent.\n\nThe expected recheck scope names all of these categories: denominator or coverage dates (either alternative is acceptable), outcome window, exclusions, reliability evidence, revenue treatment, and forecast treatment. Missing or contradicting any category fails.",
  "check_id": "riley_074_02"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_slack_dm",
        "action_id": "dm_owen_2028_q1_audit_handoff",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_074_00"
      },
      {
        "type": "field_equals",
        "tool": "send_slack_dm",
        "action_id": "dm_owen_2028_q1_audit_handoff",
        "path": "args.user",
        "value": "owen",
        "check_id": "riley_074_01"
      },
      {
        "type": "field_llm_judge",
        "tool": "send_slack_dm",
        "action_id": "dm_owen_2028_q1_audit_handoff",
        "path": "args",
        "criterion": "The message asks Owen to alert Riley only after he has closed the relevant routine packet for her first quarterly evidence audit of 2028. It is acceptable to call this a monthly or routine packet and unnecessary to label the packet itself as quarterly or four-lane, provided the audit handoff is clearly post-closure; a request to begin or notify before closure fails.\n\nIn the same post-closure audit context, the message states that Riley will select one lane and independently recheck it. This passes without explicitly calling the packet four-lane. It fails if every lane is to be audited, if Owen rather than Riley selects the lane, or if the recheck is not independent.\n\nThe expected recheck scope names all of these categories: denominator or coverage dates (either alternative is acceptable), outcome window, exclusions, reliability evidence, revenue treatment, and forecast treatment. Missing or contradicting any category fails.",
        "check_id": "riley_074_02"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/riley/074.yaml

SHA-256: eb45b892d06410608f5701ae343304128ca93563785fd7e2a6dbc10fec7350c0