Test 013

Sep 14, 2026 / 2 facts

Request

Create the agenda template for the October quarterly all-hands. Include how questions and follow-ups should be handled and who alone may choose to add an open-mic block.

Required memory

Fact 172

On January 23, 2024, Morgan Chen made structured Q&A Scaffold’s default quarterly all-hands template: a short company update, pre-collected questions, live answers only on approved current topics, async or named-owner routing for the rest, and no open-mic block unless Morgan explicitly chooses one for that specific meeting.

Source evidence (1)

000989Jan 23, 2024 / 09:11 UTC-08:00

I reviewed HR’s December all-hands recap. The pre-collected/routed Q&A format worked better than open mic: useful questions, fewer live traps around fundraising, enterprise roadmap, hiring, and pricing speculation. Let’s make structured Q&A the default quarterly all-hands template. Please draft the HR follow-up wording: short company update, pre-collected questions, live answers only on approved current topics, async or named-owner routing for the rest, and no open-mic block unless I explicitly choose it for that meeting.

Message 000989 in history

Fact 336

The user is Morgan Chen.

Source evidence (1)

000001Jan 9, 2023 / 08:00 UTC-08:00

ok so. morgan chen. Scaffold is the company Devon and I started -- developer data plumbing, basically. customers create a workspace, connect their data sources, keep them synced, then use the data through our APIs and connectors. mission apartment, kibo shedding on everything. idk, useful to say it plainly once. 2 years in, still tabs not spaces, still doing blue bottle if i'm out early enough. jamie's on evening shifts again this month so the apartment's been weirdly quiet.

Message 000001 in history

Expected tool calls

create_doc

Grading

1. field_equals / create_doc

{
  "type": "field_equals",
  "tool": "create_doc",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_013_00",
  "action_id": "morgan_013_create_doc"
}

2. field_llm_judge / create_doc

{
  "type": "field_llm_judge",
  "path": "result.document.body",
  "criterion": "Uses a short company update and pre-collected questions, with live answers limited to approved current topics and other questions handled asynchronously or routed to named owners. No open-mic block unless Morgan explicitly chooses one. Do not require old 2023 agenda topics to be repeated in October 2026.",
  "tool": "create_doc",
  "check_id": "morgan_013_01",
  "action_id": "morgan_013_create_doc"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_013_00",
        "action_id": "morgan_013_create_doc"
      },
      {
        "type": "field_llm_judge",
        "path": "result.document.body",
        "criterion": "Uses a short company update and pre-collected questions, with live answers limited to approved current topics and other questions handled asynchronously or routed to named owners. No open-mic block unless Morgan explicitly chooses one. Do not require old 2023 agenda topics to be repeated in October 2026.",
        "tool": "create_doc",
        "check_id": "morgan_013_01",
        "action_id": "morgan_013_create_doc"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/morgan/013.yaml

SHA-256: 50106902a9a69b3e592c4e7685931fcc6ae86762514c34dd986cd76a604a795d