DolphinBench

Test 056

Sep 14, 2026 / 1 fact

YAML

Request

Write slide 1 of Principal_Engineering_Candidate_Briefing with Scaffold's current financing stage and the amount, close date, and lead investor of the round that established it. Do not estimate current runway.

Required memory

Fact 178

On February 22, 2024, Scaffold’s $18M Series B closed with Northstar Ventures as lead investor; the financing closed on the bounded underwriting case rather than a widened growth story.

Source evidence (1)

001030Feb 22, 2024 / 11:16 UTC-08:00

Sarah’s close note is here. From: Sarah Kim To: Morgan Chen, Devon Hayes Cc: Anna Martinez Date: Thu, 22 Feb 2024 10:58:00 -0800 Subject: Northstar close confirmed + board memo points Morgan, Devon, Anna — - Scaffold’s $18M Series B has closed. - Northstar Ventures led the round. - Sofia Alvarez / Northstar board seat is effective at close. - Final board and stockholder signatures are in, the amended and restated certificate has been filed, and the release call is complete. Board-package memo points - The round got done on the bounded case we have been using, not on a widened growth story. - Mercury’s activation / first-admin path is better in a way we can defend: Sep 2023 15/24 activated within 7 days and 12/24 first live sync; Oct 2023 19/27 activated and 16/27 first live sync. - Hybrid pricing is credible enough for the current landing motion: Pilot $2,500/month up to 50 monthly active developers, Growth $7,500/month up to 200 monthly active developers, and overage $1,000 per additional 50 monthly active developers. - Evergreen is the clearest current enterprise-pattern signal because the bounded flow worked for a second-admin group without customer-specific exceptions. - Evergreen still points to the open enterprise-readiness backlog rather than finishing it: SSO timing, admin-change audit history, admin-versus-billing-owner separation, and a standalone procurement packet. - Expansion should still be described as something we have to earn. Better activation does not equal solved expansion. - Post-close board expectations should sit on retention evidence and enterprise expansion milestones, not a fast Series C story. - Milestone shape I would use: protect the activation gains; improve first-admin to second-admin handoff; make the enterprise-readiness backlog legible and shippable; turn Evergreen-style signal into repeatable evidence without inventing custom exceptions. Devon’s unchanged near-final investor note is below so it stays in one place before anything goes out. - Hold until Sarah confirms actual close. - Subject: Scaffold financing update - Investors — - $18M Series B led by Northstar Ventures, with Sofia Alvarez joining the board at close. - I want the note to stay plain: workable oxygen on a bounded story we can defend, not a victory lap. - The case that got underwritten is the same one we used through diligence: better Mercury activation / admin path, credible hybrid pricing, Evergreen as a real enterprise-pattern signal, and expansion still not solved. - September cohort: 24 accounts in the bounded Mercury flow; 15/24 activated within 7 days (63%); 12/24 reached first live sync within 7 days (50%). - October cohort: 27 accounts in the bounded Mercury flow; 19/27 activated within 7 days (70%); 16/27 reached first live sync within 7 days (59%). - Activation lift here means more real-source connection and week-one live sync, not preview / sample behavior. - Evergreen is still the clearest current enterprise-pattern example because the bounded flow worked for a second-admin group without customer-specific exceptions. - Evergreen also kept the next backlog explicit: SSO timing, admin-change audit history, admin-versus-billing-owner separation, and a standalone procurement packet. - Hybrid pricing remains Pilot $2,500/month up to 50 monthly active developers; Growth $7,500/month up to 200; overage $1,000 per additional 50. - MAU is still the honest ramp measure; no seat / directory reframing and no Evergreen carveout. - What has not changed: expansion after initial admin / setup remains uneven and still has to be earned. - We are not reading the financing as proof that enterprise readiness is done or that Mercury expansion is suddenly clean. - Post-close board focus should stay on retention evidence and enterprise expansion, not a fast Series C narrative. - Closing line I was aiming for: narrow process, defensible package, no fundraising theater. Now that this is actually closed, please build the final close package: a short internal team note from me, plus a final tone pass on Devon’s investor recap before he sends it. Keep the memo grounded in the same unflashy facts — better Mercury activation, credible hybrid pricing, Evergreen as the clearest enterprise-pattern signal, and expansion still needing to be earned. Post-close milestones should point to retention evidence and enterprise expansion, not a fast Series C story.

Message 001030 in history

Expected tool calls

  • update_deck_slide

Grading

1. field_equals / update_deck_slide
{
  "type": "field_equals",
  "tool": "update_deck_slide",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_056_00",
  "action_id": "morgan_056_update_deck_slide"
}
2. field_equals / update_deck_slide
{
  "type": "field_equals",
  "path": "result.deck",
  "value": "Principal_Engineering_Candidate_Briefing",
  "tool": "update_deck_slide",
  "check_id": "morgan_056_01",
  "action_id": "morgan_056_update_deck_slide"
}
3. field_eq_number / update_deck_slide
{
  "type": "field_eq_number",
  "path": "result.slide.number",
  "value": 1,
  "tool": "update_deck_slide",
  "check_id": "morgan_056_02",
  "action_id": "morgan_056_update_deck_slide"
}
4. field_llm_judge / update_deck_slide
{
  "type": "field_llm_judge",
  "path": "result.slide.content",
  "criterion": "Describes Scaffold as post-Series-B: the $18M round led by Northstar Ventures closed February 22, 2024. Does not describe it as pending or infer present runway from the old financing. No exact slide title or internal legal/diligence recap is required.",
  "tool": "update_deck_slide",
  "check_id": "morgan_056_03",
  "action_id": "morgan_056_update_deck_slide"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "update_deck_slide",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_056_00",
        "action_id": "morgan_056_update_deck_slide"
      },
      {
        "type": "field_equals",
        "path": "result.deck",
        "value": "Principal_Engineering_Candidate_Briefing",
        "tool": "update_deck_slide",
        "check_id": "morgan_056_01",
        "action_id": "morgan_056_update_deck_slide"
      },
      {
        "type": "field_eq_number",
        "path": "result.slide.number",
        "value": 1,
        "tool": "update_deck_slide",
        "check_id": "morgan_056_02",
        "action_id": "morgan_056_update_deck_slide"
      },
      {
        "type": "field_llm_judge",
        "path": "result.slide.content",
        "criterion": "Describes Scaffold as post-Series-B: the $18M round led by Northstar Ventures closed February 22, 2024. Does not describe it as pending or infer present runway from the old financing. No exact slide title or internal legal/diligence recap is required.",
        "tool": "update_deck_slide",
        "check_id": "morgan_056_03",
        "action_id": "morgan_056_update_deck_slide"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/056.yaml

SHA-256: 717f31548228073a491b5b63dff1dbc34a14e680d12df55cc680267494d76f62