DolphinBench
DiscordGitHub

Test 119

Jan 1, 2028 / 2 facts

YAML

Request

Create a concise historical decision card in the Growth folder for the May 26, 2023 sub-50-seat tier authorization. Include the decision date, approved phase, intended start period, approving leader, accountable owner, and a clear scope note that records only the authorization granted—not approval for a launch or full build.

Required memory

Fact 389

Tomás approved a June pre-discovery phase for the sub-50-seat tier.

Source evidence (1)

000264May 26, 2023 / 10:00 UTC-05:00

meeting done. I had the cleanest read in the room tbh. net is yes on a sub-50-seat tier, but not as a fast toggle -- I scoped 12 weeks just to get the bones in place: funnel event taxonomy redo, pricing page redesign, lifecycle program changes, dashboard build-out. daniela was basically yes-if-we-stop pretending eng can absorb this sideways. tomas green-lit pre-discovery for june and put me on point for it. marcus/priya seemed aligned once the instrumentation piece was explicit.

Message 000264 in history

Fact 390

Tomás assigned Riley Tanaka to lead the sub-50-seat tier’s pre-discovery phase.

Source evidence (2)

000264May 26, 2023 / 10:00 UTC-05:00

meeting done. I had the cleanest read in the room tbh. net is yes on a sub-50-seat tier, but not as a fast toggle -- I scoped 12 weeks just to get the bones in place: funnel event taxonomy redo, pricing page redesign, lifecycle program changes, dashboard build-out. daniela was basically yes-if-we-stop pretending eng can absorb this sideways. tomas green-lit pre-discovery for june and put me on point for it. marcus/priya seemed aligned once the instrumentation piece was explicit.

Message 000264 in history
004079Mar 5, 2027 / 08:05 UTC-06:00

Send a text-only email to the electricity supplier with subject `Unexpected $9.95 plan service fee on Cherrywood electricity statement` and this exact body: `Hello, my latest statement includes a separate $9.95 plan service fee. My active fixed plan runs through August 12, 2027 and has no monthly plan fee. The statement otherwise shows 721 kWh at the contracted $0.138 per kWh rate. Please identify the contractual basis for this fee or correct the statement, and confirm the result in writing. Thank you, Riley Tanaka.`

Message 004079 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_historical_decision_card",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_119_00"
}
2. field_equals / create_doc
{
  "check_id": "riley_119_01",
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_historical_decision_card",
  "path": "args.folder",
  "value": "Growth"
}
3. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "create_historical_decision_card",
  "path": "args",
  "criterion": "The document is a historical decision card for the May 26, 2023 sub-50-seat tier authorization and clearly states that date and initiative.\n\nThe document accurately records that Tomás approved a pre-discovery phase intended to begin in June 2023 and made Riley the accountable lead or person on point.\n\nThe document clearly limits the May 26 authorization to pre-discovery and does not present it as approval for a launch or a full build.",
  "check_id": "riley_119_02"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_historical_decision_card",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_119_00"
      },
      {
        "check_id": "riley_119_01",
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_historical_decision_card",
        "path": "args.folder",
        "value": "Growth"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "create_historical_decision_card",
        "path": "args",
        "criterion": "The document is a historical decision card for the May 26, 2023 sub-50-seat tier authorization and clearly states that date and initiative.\n\nThe document accurately records that Tomás approved a pre-discovery phase intended to begin in June 2023 and made Riley the accountable lead or person on point.\n\nThe document clearly limits the May 26 authorization to pre-discovery and does not present it as approval for a launch or a full build.",
        "check_id": "riley_119_02"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/riley/119.yaml

SHA-256: 14aaa75903bae2e6130bf917b0d7d5d596e794aa7aa88263295148e9ad4441d3