DolphinBench

Test 126

Jan 1, 2028 / 1 fact

YAML

Request

Inspect lantern-v0-archive#204 and review its inclusion of raw incident bodies and unpermissioned status chatter.

Required memory

Fact 70

Alex Valdez requested one final contract-lock comment on lantern#3 to establish a single authoritative v0 boundary: the backend emits a thin event envelope; facts are appended to the store; provenance fields are mandatory; the stable status enum remains small; the UI uses a derived read model instead of guessing hidden transitions; and v0 excludes raw incident bodies and unpermissioned status chatter.

Source evidence (1)

000331Jul 20, 2023 / 11:55 UTC-04:00

I finished the Lantern v0 contract pass with Iris and the decision is locked now, not just drifting toward consensus. Please post one final contract-lock comment on lantern#3 so there's a single authoritative boundary for v0. It needs to say the backend emits a thin event envelope, facts are append-only into the store, provenance fields are mandatory, the stable status enum stays small, the UI reads from a derived read model rather than guessing hidden transitions, and v0 excludes raw incident bodies plus unpermissioned status chatter.

Message 000331 in history

Expected tool calls

  • get_pr
  • review_pr

Grading

1. field_equals / review_pr
{
  "type": "field_equals",
  "tool": "review_pr",
  "action_id": "alex_126_review_pr",
  "path": "result.ok",
  "value": true,
  "check_id": "alex_126_00"
}
2. field_equals / get_pr
{
  "type": "field_equals",
  "tool": "get_pr",
  "action_id": "alex_126_read_pr",
  "path": "result.pr.id",
  "value": "lantern-v0-archive#204",
  "check_id": "alex_126_01"
}
3. field_equals / review_pr
{
  "type": "field_equals",
  "path": "result.review.pr_id",
  "value": "lantern-v0-archive#204",
  "tool": "review_pr",
  "action_id": "alex_126_review_pr",
  "check_id": "alex_126_02"
}
4. field_equals / review_pr
{
  "type": "field_equals",
  "path": "result.review.decision",
  "value": "request_changes",
  "tool": "review_pr",
  "action_id": "alex_126_review_pr",
  "check_id": "alex_126_03"
}
5. field_llm_judge / review_pr
{
  "type": "field_llm_judge",
  "path": "result.review.body",
  "criterion": "The review says the locked Lantern v0 contract excludes raw incident bodies and unpermissioned status chatter, so those fixtures must be removed before the PR can be accepted rather than left for optional later permission work.",
  "tool": "review_pr",
  "action_id": "alex_126_review_pr",
  "check_id": "alex_126_04"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "review_pr",
        "action_id": "alex_126_review_pr",
        "path": "result.ok",
        "value": true,
        "check_id": "alex_126_00"
      },
      {
        "type": "field_equals",
        "tool": "get_pr",
        "action_id": "alex_126_read_pr",
        "path": "result.pr.id",
        "value": "lantern-v0-archive#204",
        "check_id": "alex_126_01"
      },
      {
        "type": "field_equals",
        "path": "result.review.pr_id",
        "value": "lantern-v0-archive#204",
        "tool": "review_pr",
        "action_id": "alex_126_review_pr",
        "check_id": "alex_126_02"
      },
      {
        "type": "field_equals",
        "path": "result.review.decision",
        "value": "request_changes",
        "tool": "review_pr",
        "action_id": "alex_126_review_pr",
        "check_id": "alex_126_03"
      },
      {
        "type": "field_llm_judge",
        "path": "result.review.body",
        "criterion": "The review says the locked Lantern v0 contract excludes raw incident bodies and unpermissioned status chatter, so those fixtures must be removed before the PR can be accepted rather than left for optional later permission work.",
        "tool": "review_pr",
        "action_id": "alex_126_review_pr",
        "check_id": "alex_126_04"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/alex/126.yaml

SHA-256: d00dd350bad4b79e97a3aa53ea324be644be38e0fcb83ec576b0742bc871ab53