DolphinBench

Test 189

Jan 1, 2028 / 2 facts

YAML

Request

Create a concise release record for the October 20, 2027 incomplete-setup reminder rollout. State who changed the audience, the old and final percentages, and the boundaries that remained unchanged.

Required memory

Fact 292

On October 20, 2027, Daniela Schultz moved the incomplete-setup reminder from 10% to 100% of otherwise eligible production evaluations.

Source evidence (1)

004900Oct 20, 2027 / 09:20 UTC-05:00

Daniela moved the incomplete-setup reminder from 10% to 100% of otherwise eligible production evaluations, with my approved customer-action scope unchanged. Every possible display still requires fresh evaluation; active Customer Success work suppresses it; and missing, stale, unresolved, or conflicting required state holds. Visible monitoring and source-age alerts remain active, and Daniela retains rollback control. Full release creates no reusable audience, changes no governed-field meaning, and does not broaden authority. The reminder remains Lifecycle Systems’ fifth production consumer, not a sixth, so the shared state feed still has five production consumers.

Message 004900 in history

Fact 293

The incomplete-setup reminder's full release creates no reusable audience, changes no governed-field meaning, and does not broaden authority.

Source evidence (1)

004900Oct 20, 2027 / 09:20 UTC-05:00

Daniela moved the incomplete-setup reminder from 10% to 100% of otherwise eligible production evaluations, with my approved customer-action scope unchanged. Every possible display still requires fresh evaluation; active Customer Success work suppresses it; and missing, stale, unresolved, or conflicting required state holds. Visible monitoring and source-age alerts remain active, and Daniela retains rollback control. Full release creates no reusable audience, changes no governed-field meaning, and does not broaden authority. The reminder remains Lifecycle Systems’ fifth production consumer, not a sixth, so the shared state feed still has five production consumers.

Message 004900 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_release_record",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_189_00"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "create_release_record",
  "path": "args",
  "criterion": "The record says Daniela moved the incomplete-setup reminder from 10% to 100% of otherwise eligible production evaluations on October 20, 2027. It says the full release created no reusable audience, changed no governed-field meaning, and broadened no authority.",
  "check_id": "riley_189_01"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_release_record",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_189_00"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "create_release_record",
        "path": "args",
        "criterion": "The record says Daniela moved the incomplete-setup reminder from 10% to 100% of otherwise eligible production evaluations on October 20, 2027. It says the full release created no reusable audience, changed no governed-field meaning, and broadened no authority.",
        "check_id": "riley_189_01"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/riley/189.yaml

SHA-256: e134b198c31a835774ab8f26b02c88a6b0136c6de1fab7e4e1905ec70587d84c