DolphinBench

Test 194

Jan 1, 2028 / 2 facts

YAML

Request

Create a short Q4 2027 Lantern customer-use note with the frozen use counts and the decisions that the evidence freeze did not make.

Required memory

Fact 297

Harbor Health used the Lantern view in 14 release follow-ups, and Mosaic Commerce used it in 13 incident handoffs.

Source evidence (1)

004988Dec 16, 2027 / 09:00 UTC-05:00

Product Engineering reconciled every row with an authoritative `source_timestamp` from October 1 through December 15 and froze the renewed Lantern term’s Q4 operating package. It contains 3,402 customer sessions and 651 logical explanation requests: 431 eligible combined explanations, 145 valid cases kept separate because source timestamps differed by more than 24 hours, and 75 explicit-unknown outcomes for stale or missing input. Eligible renderer failures are zero. All 119 delayed or retried rows are ordered by authoritative `source_timestamp` without processing-time reordering. Harbor Health used the view in 14 release follow-ups, and Mosaic Commerce used it in 13 incident handoffs. Authorization-wrapper bypasses, adapter calls after denial, cross-tenant material, incorrect stale-or-missing rendering, missing required provenance or source timestamps, unpublished ownership renders, excluded-field exposures, losses of read-only enforcement, and internal-reference requests, artifacts, or outputs in either customer path are all zero. The freeze makes no post-March renewal, expansion, implementation, release, or ownership decision.

Message 004988 in history

Fact 298

The Lantern Q4 operating-package freeze makes no decision about post-March renewal, expansion, implementation, release, or ownership.

Source evidence (1)

004988Dec 16, 2027 / 09:00 UTC-05:00

Product Engineering reconciled every row with an authoritative `source_timestamp` from October 1 through December 15 and froze the renewed Lantern term’s Q4 operating package. It contains 3,402 customer sessions and 651 logical explanation requests: 431 eligible combined explanations, 145 valid cases kept separate because source timestamps differed by more than 24 hours, and 75 explicit-unknown outcomes for stale or missing input. Eligible renderer failures are zero. All 119 delayed or retried rows are ordered by authoritative `source_timestamp` without processing-time reordering. Harbor Health used the view in 14 release follow-ups, and Mosaic Commerce used it in 13 incident handoffs. Authorization-wrapper bypasses, adapter calls after denial, cross-tenant material, incorrect stale-or-missing rendering, missing required provenance or source timestamps, unpublished ownership renders, excluded-field exposures, losses of read-only enforcement, and internal-reference requests, artifacts, or outputs in either customer path are all zero. The freeze makes no post-March renewal, expansion, implementation, release, or ownership decision.

Message 004988 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "alex_194_create_doc",
  "path": "result.ok",
  "value": true,
  "check_id": "alex_194_00"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "path": "result.document.body",
  "criterion": "The note says Harbor Health used Lantern in 14 release follow-ups and Mosaic Commerce used it in 13 incident handoffs during the frozen Q4 operating period. It says the freeze itself made no post-March renewal, expansion, implementation, release, or ownership decision.",
  "tool": "create_doc",
  "action_id": "alex_194_create_doc",
  "check_id": "alex_194_01"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "alex_194_create_doc",
        "path": "result.ok",
        "value": true,
        "check_id": "alex_194_00"
      },
      {
        "type": "field_llm_judge",
        "path": "result.document.body",
        "criterion": "The note says Harbor Health used Lantern in 14 release follow-ups and Mosaic Commerce used it in 13 incident handoffs during the frozen Q4 operating period. It says the freeze itself made no post-March renewal, expansion, implementation, release, or ownership decision.",
        "tool": "create_doc",
        "action_id": "alex_194_create_doc",
        "check_id": "alex_194_01"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/alex/194.yaml

SHA-256: 321502127b333c2627110253b74841bce96c272bb6d4aeeb891e17c78edd537f