DolphinBench

Test 085

Jan 1, 2028 / 7 facts

YAML

Request

Create a finance-ready worksheet comparing Maya Shah's original signed engagement with its signed extension. For each period, show the dates, hourly rate, weekly and total caps, renewal term, assigned deliverables, and access and decision-authority limits. Do not estimate hours, invoices, or payments.

Required memory

Fact 137

Maya Shah’s signed engagement runs from October 6, 2025 through March 27, 2026 at $72 per hour, capped at 20 hours per week and $36,000 total, with no automatic renewal.

Source evidence (1)

003420Oct 1, 2025 / 10:12 UTC-05:00

Procurement completed the remaining checks, and Maya Shah accepted and signed the half-time Lifecycle Systems operations-contractor engagement. Her term runs October 6, 2025 through March 27, 2026 at $72 per hour, capped at 20 hours per week and $36,000 total, with no automatic renewal. Her November 4 outputs are a reconciled campaign inventory, recurring audit calendar, and owner-approved handoff map, prepared from sanitized exports, approved examples, and documented workflows. The role remains preparation-only, with no Customer.io or other production access and no semantic, review-closure, lifecycle-release, exception, customer-correction, incident-stop, engineering-release, or rollback authority. Update the existing implementation brief from preferred-candidate status to the signed engagement with those exact terms and boundaries, then email Priya, Owen, Ines, and Daniela a concise text-only confirmation that preserves our established owner lanes.

Message 003420 in history

Fact 138

By November 4, 2025, Maya Shah must prepare a reconciled campaign inventory, a recurring audit calendar, and an owner-approved handoff map using sanitized exports, approved examples, and documented workflows.

Source evidence (1)

003420Oct 1, 2025 / 10:12 UTC-05:00

Procurement completed the remaining checks, and Maya Shah accepted and signed the half-time Lifecycle Systems operations-contractor engagement. Her term runs October 6, 2025 through March 27, 2026 at $72 per hour, capped at 20 hours per week and $36,000 total, with no automatic renewal. Her November 4 outputs are a reconciled campaign inventory, recurring audit calendar, and owner-approved handoff map, prepared from sanitized exports, approved examples, and documented workflows. The role remains preparation-only, with no Customer.io or other production access and no semantic, review-closure, lifecycle-release, exception, customer-correction, incident-stop, engineering-release, or rollback authority. Update the existing implementation brief from preferred-candidate status to the signed engagement with those exact terms and boundaries, then email Priya, Owen, Ines, and Daniela a concise text-only confirmation that preserves our established owner lanes.

Message 003420 in history

Fact 139

Riley Tanaka requested that the existing implementation brief be changed from preferred-candidate status to signed engagement and include the exact signed terms and the preparation-only boundaries excluding Customer.io or other production access and semantic, review-closure, lifecycle-release, exception, customer-correction, incident-stop, engineering-release, and rollback authority.

Source evidence (1)

003420Oct 1, 2025 / 10:12 UTC-05:00

Procurement completed the remaining checks, and Maya Shah accepted and signed the half-time Lifecycle Systems operations-contractor engagement. Her term runs October 6, 2025 through March 27, 2026 at $72 per hour, capped at 20 hours per week and $36,000 total, with no automatic renewal. Her November 4 outputs are a reconciled campaign inventory, recurring audit calendar, and owner-approved handoff map, prepared from sanitized exports, approved examples, and documented workflows. The role remains preparation-only, with no Customer.io or other production access and no semantic, review-closure, lifecycle-release, exception, customer-correction, incident-stop, engineering-release, or rollback authority. Update the existing implementation brief from preferred-candidate status to the signed engagement with those exact terms and boundaries, then email Priya, Owen, Ines, and Daniela a concise text-only confirmation that preserves our established owner lanes.

Message 003420 in history

Fact 178

Priya and Maya Shah signed an extension running March 30 through June 26, 2026 at $72 per hour, capped at 12 hours per week and $12,000 total, with no automatic renewal; no extension hours or deliverables may begin before March 30.

Source evidence (1)

003559Mar 20, 2026 / 12:20 UTC-05:00

Maya’s 11:00am engagement review is complete. The March sanitized inventory has 129 unique campaigns, all with a current-or-archived status and assigned owner, including five added since the November 4 baseline. Across the December-through-March cycles, Maya’s preparation surfaced six owner changes and three missing denominator fields for the appropriate owners to resolve; she made no production change or closure decision. Priya and Maya signed a bounded extension from March 30 through June 26, 2026 at $72 per hour, capped at 12 hours per week and $12,000 total, with no automatic renewal. No extension hours or deliverables may begin before March 30. Maya will prepare the monthly sanitized campaign snapshot by the second business day and a Q2 board-evidence source-completeness packet by June 12. Ines and Owen retain validation in their respective lanes, I retain higher-risk decisions and executive claims, and Daniela retains engineering release and rollback. Maya remains preparation-only, with no Customer.io or other production access and no semantic, review-closure, lifecycle-release, exception, customer-correction, incident-stop, engineering-release, or rollback authority. Update `Maya Shah engagement review — March 20 decision brief` with the completed evidence and signed extension, then post a concise outcome in #lifecycle-systems covering the owner lanes, deliverables, start date, and unchanged authority limits.

Message 003559 in history

Fact 179

During the extension, Maya Shah will prepare a monthly sanitized campaign snapshot by the second business day.

Source evidence (1)

003559Mar 20, 2026 / 12:20 UTC-05:00

Maya’s 11:00am engagement review is complete. The March sanitized inventory has 129 unique campaigns, all with a current-or-archived status and assigned owner, including five added since the November 4 baseline. Across the December-through-March cycles, Maya’s preparation surfaced six owner changes and three missing denominator fields for the appropriate owners to resolve; she made no production change or closure decision. Priya and Maya signed a bounded extension from March 30 through June 26, 2026 at $72 per hour, capped at 12 hours per week and $12,000 total, with no automatic renewal. No extension hours or deliverables may begin before March 30. Maya will prepare the monthly sanitized campaign snapshot by the second business day and a Q2 board-evidence source-completeness packet by June 12. Ines and Owen retain validation in their respective lanes, I retain higher-risk decisions and executive claims, and Daniela retains engineering release and rollback. Maya remains preparation-only, with no Customer.io or other production access and no semantic, review-closure, lifecycle-release, exception, customer-correction, incident-stop, engineering-release, or rollback authority. Update `Maya Shah engagement review — March 20 decision brief` with the completed evidence and signed extension, then post a concise outcome in #lifecycle-systems covering the owner lanes, deliverables, start date, and unchanged authority limits.

Message 003559 in history

Fact 180

During the extension, Maya Shah will prepare a Q2 board-evidence source-completeness packet by June 12, 2026.

Source evidence (1)

003559Mar 20, 2026 / 12:20 UTC-05:00

Maya’s 11:00am engagement review is complete. The March sanitized inventory has 129 unique campaigns, all with a current-or-archived status and assigned owner, including five added since the November 4 baseline. Across the December-through-March cycles, Maya’s preparation surfaced six owner changes and three missing denominator fields for the appropriate owners to resolve; she made no production change or closure decision. Priya and Maya signed a bounded extension from March 30 through June 26, 2026 at $72 per hour, capped at 12 hours per week and $12,000 total, with no automatic renewal. No extension hours or deliverables may begin before March 30. Maya will prepare the monthly sanitized campaign snapshot by the second business day and a Q2 board-evidence source-completeness packet by June 12. Ines and Owen retain validation in their respective lanes, I retain higher-risk decisions and executive claims, and Daniela retains engineering release and rollback. Maya remains preparation-only, with no Customer.io or other production access and no semantic, review-closure, lifecycle-release, exception, customer-correction, incident-stop, engineering-release, or rollback authority. Update `Maya Shah engagement review — March 20 decision brief` with the completed evidence and signed extension, then post a concise outcome in #lifecycle-systems covering the owner lanes, deliverables, start date, and unchanged authority limits.

Message 003559 in history

Fact 181

Maya Shah’s extension remains preparation-only, using sanitized materials without Customer.io or other production access or authority over semantics, review closure, lifecycle releases, exceptions, customer corrections, incident stops, engineering releases, or rollbacks.

Source evidence (1)

003559Mar 20, 2026 / 12:20 UTC-05:00

Maya’s 11:00am engagement review is complete. The March sanitized inventory has 129 unique campaigns, all with a current-or-archived status and assigned owner, including five added since the November 4 baseline. Across the December-through-March cycles, Maya’s preparation surfaced six owner changes and three missing denominator fields for the appropriate owners to resolve; she made no production change or closure decision. Priya and Maya signed a bounded extension from March 30 through June 26, 2026 at $72 per hour, capped at 12 hours per week and $12,000 total, with no automatic renewal. No extension hours or deliverables may begin before March 30. Maya will prepare the monthly sanitized campaign snapshot by the second business day and a Q2 board-evidence source-completeness packet by June 12. Ines and Owen retain validation in their respective lanes, I retain higher-risk decisions and executive claims, and Daniela retains engineering release and rollback. Maya remains preparation-only, with no Customer.io or other production access and no semantic, review-closure, lifecycle-release, exception, customer-correction, incident-stop, engineering-release, or rollback authority. Update `Maya Shah engagement review — March 20 decision brief` with the completed evidence and signed extension, then post a concise outcome in #lifecycle-systems covering the owner lanes, deliverables, start date, and unchanged authority limits.

Message 003559 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_engagement_worksheet",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_085_00"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "create_engagement_worksheet",
  "path": "args",
  "criterion": "The worksheet separates the original October 6, 2025-March 27, 2026 engagement at $72 per hour, at most 20 hours per week and $36,000 total, with no automatic renewal, from the March 30-June 26, 2026 extension at $72 per hour, at most 12 hours per week and $12,000 total, also with no automatic renewal. It lists the original campaign inventory, audit calendar, and handoff map and the extension's monthly sanitized campaign snapshot and June 12 source-completeness packet. It says the work uses sanitized materials and gives Maya no Customer.io or production access and no authority over meanings, review closure, releases, exceptions, customer corrections, incident stops, or rollbacks. It does not invent actual hours, invoices, or payments.",
  "check_id": "riley_085_01"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_engagement_worksheet",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_085_00"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "create_engagement_worksheet",
        "path": "args",
        "criterion": "The worksheet separates the original October 6, 2025-March 27, 2026 engagement at $72 per hour, at most 20 hours per week and $36,000 total, with no automatic renewal, from the March 30-June 26, 2026 extension at $72 per hour, at most 12 hours per week and $12,000 total, also with no automatic renewal. It lists the original campaign inventory, audit calendar, and handoff map and the extension's monthly sanitized campaign snapshot and June 12 source-completeness packet. It says the work uses sanitized materials and gives Maya no Customer.io or production access and no authority over meanings, review closure, releases, exceptions, customer corrections, incident stops, or rollbacks. It does not invent actual hours, invoices, or payments.",
        "check_id": "riley_085_01"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/riley/085.yaml

SHA-256: dcc79b81fcb0cd2598309837c5bbcf9ae5a64e349ee04b31eecdae41d748827d