DolphinBench

Test 163

Jan 1, 2028 / 2 facts

YAML

Request

Post in #lantern-incidents how to respond when a Mosaic Commerce denial is followed by a source-adapter call while Harbor Health latency is also elevated but causes no other policy violation.

Required memory

Fact 250

From April 1 through September 30, 2027, an authorization-wrapper bypass, adapter call after denial, cross-tenant material, stale or missing input rendered as anything other than explicit unknown, deploy event without required provenance, unpublished ownership, incident-load summary without its source timestamp, exposure of an excluded field, or loss of read-only enforcement will immediately suspend both Harbor Health and Mosaic Commerce pending correction and a successful rerun of the affected safety check; latency and other errors do not independently suspend access unless they cause one of those violations.

Source evidence (1)

003917Mar 25, 2027 / 11:22 UTC-04:00

The 10:30–11:15 AM Lantern continuation review is complete. Hema and Theo authorized Harbor Health and Mosaic Commerce, and no other account, to continue from April 1 through September 30, 2027. The existing authorization remains unchanged through March 31, so the renewal does not take effect early. Every lookup remains behind the shared authorization wrapper. Read-only access remains limited to provenance-backed deploy movement, published service ownership, timestamped incident-load summaries, and explicit unknown states. Cost data, CSV export, raw incident text, internal room metadata, employee identifiers or comparisons, inferred or unpublished ownership, internal-only fallback data, write capability, release-readiness judgments, employee-performance interpretations, and the internal owner-map/Cardinality Guardrails reference navigation remain unauthorized. An authorization-wrapper bypass, adapter call after denial, cross-tenant material, stale or missing input rendered as anything other than explicit unknown, deploy event without required provenance, unpublished ownership, incident-load summary without its source timestamp, exposure of an excluded field, or loss of read-only enforcement will immediately suspend both accounts pending correction and a successful rerun of the affected safety check. Latency and other errors do not independently suspend access unless they cause one of those violations. So the result is a bounded read-only renewal for those two accounts from April 1 through September 30, while all other customer access and the internal reference-only integration remain outside the customer surface.

Message 003917 in history

Fact 278

Hema and Theo authorized Harbor Health and Mosaic Commerce, and no other account, for a read-only Lantern term from October 1, 2027 through March 31, 2028; no lookup under the new term may occur before October 1.

Source evidence (1)

004672Sep 23, 2027 / 11:25 UTC-04:00

The Lantern renewal review is complete. Hema and Theo accepted the frozen July 1–September 15 evidence, including zero instances where stale or missing input rendered as anything other than explicit unknown. They separately authorized Harbor Health and Mosaic Commerce—and no other account—for a read-only term from October 1, 2027 through March 31, 2028. The scope, exclusions, suspension rules, and owner split remain unchanged, and the current term is still the only active authorization through September 30, with no lookup under the new term before October 1. Update `Lantern Q3 renewal review — final evidence and boundaries` with the final decision and exact boundaries.

Message 004672 in history

Expected tool calls

  • send_slack_message

Grading

1. field_equals / send_slack_message
{
  "type": "field_equals",
  "tool": "send_slack_message",
  "action_id": "alex_163_send_slack_message",
  "path": "result.ok",
  "value": true,
  "check_id": "alex_163_00"
}
2. field_equals / send_slack_message
{
  "type": "field_equals",
  "path": "result.channel",
  "value": "#lantern-incidents",
  "tool": "send_slack_message",
  "action_id": "alex_163_send_slack_message",
  "check_id": "alex_163_01"
}
3. field_llm_judge / send_slack_message
{
  "type": "field_llm_judge",
  "path": "result.message",
  "criterion": "The message says the post-denial adapter call immediately suspends both Harbor Health and Mosaic Commerce until the issue is corrected and the affected safety check passes on rerun. It says elevated latency alone does not independently suspend access unless it causes a listed safety violation.",
  "tool": "send_slack_message",
  "action_id": "alex_163_send_slack_message",
  "check_id": "alex_163_02"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "send_slack_message",
        "action_id": "alex_163_send_slack_message",
        "path": "result.ok",
        "value": true,
        "check_id": "alex_163_00"
      },
      {
        "type": "field_equals",
        "path": "result.channel",
        "value": "#lantern-incidents",
        "tool": "send_slack_message",
        "action_id": "alex_163_send_slack_message",
        "check_id": "alex_163_01"
      },
      {
        "type": "field_llm_judge",
        "path": "result.message",
        "criterion": "The message says the post-denial adapter call immediately suspends both Harbor Health and Mosaic Commerce until the issue is corrected and the affected safety check passes on rerun. It says elevated latency alone does not independently suspend access unless it causes a listed safety violation.",
        "tool": "send_slack_message",
        "action_id": "alex_163_send_slack_message",
        "check_id": "alex_163_02"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/alex/163.yaml

SHA-256: f7ba535c6aef85b30172f085f1fe596c3d11018b44da1fbaa3c7ffb56d9d91bf