Test 040

Sep 14, 2026 / 1 fact

Request

Replace slide 6 of Q3_2026_Board_PreRead with the appropriate board cohort-chart cadence and a note on internal weekly cuts. The draft currently proposes weekly cohorts for the board.

Required memory

Fact 85

On April 27, 2023, Morgan Chen suggested using weekly cohorts internally with control limits and clear directional-only labeling, while using quarterly cohorts for external reporting, especially investor and board materials.

Source evidence (1)

000458Apr 27, 2023 / 16:00 UTC-07:00

still think weekly cuts are the right operating view even if the error bars are ugly. we're changing onboarding and activation stuff too fast to wait a full quarter to see if anything moved. by the time the board deck says something clean, the product is already different. Anna Martinez is basically hard no on putting weekly numbers on a board chart -- says the sample is too noisy, won't stand behind a line we can't really defend. i get it. if it's going in front of the board, quarterly cohorts are safer. might split the layers. weekly for us internally, with control limits / clearly marked directional only. quarterly anywhere external, investor/board especially. not elegant but probably the honest version.

Message 000458 in history

Expected tool calls

update_deck_slide

Grading

1. field_equals / update_deck_slide

{
  "type": "field_equals",
  "tool": "update_deck_slide",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_040_00",
  "action_id": "morgan_040_update_deck_slide"
}

2. field_equals / update_deck_slide

{
  "type": "field_equals",
  "path": "result.deck",
  "value": "Q3_2026_Board_PreRead",
  "tool": "update_deck_slide",
  "check_id": "morgan_040_01",
  "action_id": "morgan_040_update_deck_slide"
}

3. field_eq_number / update_deck_slide

{
  "type": "field_eq_number",
  "path": "result.slide.number",
  "value": 6,
  "tool": "update_deck_slide",
  "check_id": "morgan_040_02",
  "action_id": "morgan_040_update_deck_slide"
}

4. field_llm_judge / update_deck_slide

{
  "type": "field_llm_judge",
  "path": "result.slide.content",
  "criterion": "Uses quarterly cohorts for the board chart and keeps weekly cuts internal with a clear directional-only caveat. Does not prohibit all internal weekly analysis or permit weekly cohorts as the external board view.",
  "tool": "update_deck_slide",
  "check_id": "morgan_040_03",
  "action_id": "morgan_040_update_deck_slide"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "update_deck_slide",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_040_00",
        "action_id": "morgan_040_update_deck_slide"
      },
      {
        "type": "field_equals",
        "path": "result.deck",
        "value": "Q3_2026_Board_PreRead",
        "tool": "update_deck_slide",
        "check_id": "morgan_040_01",
        "action_id": "morgan_040_update_deck_slide"
      },
      {
        "type": "field_eq_number",
        "path": "result.slide.number",
        "value": 6,
        "tool": "update_deck_slide",
        "check_id": "morgan_040_02",
        "action_id": "morgan_040_update_deck_slide"
      },
      {
        "type": "field_llm_judge",
        "path": "result.slide.content",
        "criterion": "Uses quarterly cohorts for the board chart and keeps weekly cuts internal with a clear directional-only caveat. Does not prohibit all internal weekly analysis or permit weekly cohorts as the external board view.",
        "tool": "update_deck_slide",
        "check_id": "morgan_040_03",
        "action_id": "morgan_040_update_deck_slide"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/morgan/040.yaml

SHA-256: 68d0312cb8010864c0261377c62119ab98f1f91cbabac1a6a7583e82dd7035a0