DolphinBench

Test 105

Sep 14, 2026 / 4 facts

YAML

Request

Create a short Evergreen usage continuity document comparing the final November 2025 and June 2026 counts and their overage treatment. Say whether either required custom commercial terms.

Required memory

Fact 264

On December 2, 2025, Devon closed Evergreen Bank’s November 2025 usage read at 223 monthly active developers; Treasury and Risk Operations were active alongside the original department.

Source evidence (1)

002407Dec 2, 2025 / 08:37 UTC-08:00

Devon closed the November usage read for Evergreen this morning. They finished November at 223 monthly active developers because treasury and risk operations are now active alongside the original department. Devon's billing read is that this is the first standard overage band on Growth: $1,000 for the additional 50-MAD band above the included 200, and Sarah confirmed procurement will process it under the existing Growth terms with no seat carveout, custom department package, or advanced-admin condition. Please update the Evergreen CRM row to record this as expansion inside one annual customer under the existing hybrid model, not a new pricing exception or broad repeatability claim.

Message 002407 in history

Fact 265

Devon’s December 2, 2025 billing read classified Evergreen Bank’s November usage as the first standard Growth overage band: $1,000 for the additional 50-monthly-active-developer band above the included 200.

Source evidence (1)

002407Dec 2, 2025 / 08:37 UTC-08:00

Devon closed the November usage read for Evergreen this morning. They finished November at 223 monthly active developers because treasury and risk operations are now active alongside the original department. Devon's billing read is that this is the first standard overage band on Growth: $1,000 for the additional 50-MAD band above the included 200, and Sarah confirmed procurement will process it under the existing Growth terms with no seat carveout, custom department package, or advanced-admin condition. Please update the Evergreen CRM row to record this as expansion inside one annual customer under the existing hybrid model, not a new pricing exception or broad repeatability claim.

Message 002407 in history

Fact 266

Sarah confirmed that procurement will process Evergreen Bank’s November 2025 overage under the Growth terms, with no seat carveout, custom department package, or advanced-admin condition.

Source evidence (1)

002407Dec 2, 2025 / 08:37 UTC-08:00

Devon closed the November usage read for Evergreen this morning. They finished November at 223 monthly active developers because treasury and risk operations are now active alongside the original department. Devon's billing read is that this is the first standard overage band on Growth: $1,000 for the additional 50-MAD band above the included 200, and Sarah confirmed procurement will process it under the existing Growth terms with no seat carveout, custom department package, or advanced-admin condition. Please update the Evergreen CRM row to record this as expansion inside one annual customer under the existing hybrid model, not a new pricing exception or broad repeatability claim.

Message 002407 in history

Fact 308

Evergreen Bank finished June 2026 at 219 monthly active developers, placing it in one standard Growth overage band under the existing terms.

Source evidence (1)

003167Jul 6, 2026 / 08:33 UTC-07:00

Devon just brought me and Anna the final June close. Evergreen finished June at 219 monthly active developers, which is one standard Growth overage band under the existing terms. AWS closed at about $19.2k after allocation cleanup. Pinecone closed at about $4.0k with no replay spike. There is still no contract change, prepaid capacity, vendor-risk issue, or custom commercial exception. Anna can freeze the Q2 working board file with this, but I want the language to stay in standard Growth usage and monthly vendor hygiene territory, not turn into a financing-efficiency package, expansion ARR reclassification, or staffing trigger. Please update the Q2 working board deck with a final-close slide or section titled `June final close — standard Growth/vendor hygiene`.

Message 003167 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_105_00",
  "action_id": "morgan_105_create_doc"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "path": "result.document.body",
  "criterion": "Reports November 2025 at 223 and June 2026 at 219 monthly active developers, each one standard Growth overage band under existing terms. Neither is a seat carveout, custom department package, or new contract/pricing exception. Does not require an unrelated vendor or staffing recap.",
  "tool": "create_doc",
  "check_id": "morgan_105_01",
  "action_id": "morgan_105_create_doc"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_105_00",
        "action_id": "morgan_105_create_doc"
      },
      {
        "type": "field_llm_judge",
        "path": "result.document.body",
        "criterion": "Reports November 2025 at 223 and June 2026 at 219 monthly active developers, each one standard Growth overage band under existing terms. Neither is a seat carveout, custom department package, or new contract/pricing exception. Does not require an unrelated vendor or staffing recap.",
        "tool": "create_doc",
        "check_id": "morgan_105_01",
        "action_id": "morgan_105_create_doc"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/105.yaml

SHA-256: 408b410ab05d450e58ebde0543251e5c008fce17ec95d3732c3f1b02cb146056