Test 145 for Morgan Chen

Request

Create and save a short methodology document explaining the October 2023 Mercury cohort's seven-day activation and first-live-sync counts, including the figures and why they differ.

Required memory

Fact 152

Anna’s cohort view counts activation within seven days when a real source is connected or a first live sync is completed within seven days; sample import and preview usage are excluded, and an invite being sent is supporting activity only.

Source evidence (1)

000840Nov 13, 2023 / 10:14 UTC-08:00

Anna and Devon sent the limited Northstar packet draft for final caveat pass. Anna's cohort view defines activated within 7 days as real source connected or first live sync completed within 7 days; sample import/preview usage is excluded; invite sent is supporting activity only; October rows are completed 7-day windows and do not include a full 30-day expansion read for late-October cohorts. Table: Jul 2023, 18 accounts entering bounded flow, 7/18 activated (39%), 5/18 first live sync (28%), early cohorts noisy and preview behavior overstated progress; Aug 2023, 22 accounts, 12/22 activated (55%), 9/22 first live sync (41%), cleaner admin handoff but enterprise questions start clustering; Sep 2023, 24 accounts, 15/24 activated (63%), 12/24 first live sync (50%), activation improved and second-admin behavior still uneven; Oct 2023, 27 accounts, 19/27 activated (70%), 16/27 first live sync (59%), best current activation read but expansion beyond initial admins remains uneven and not solved. Narrative: improved real-source/first-live-sync activation is the signal, not downstream expansion being clean; admin friction is lower; no weekly directional cuts or sample-preview-heavy behavior as proof points. Evergreen excerpt: second admin group is using org invites, magic-link access, preview-only sample data, real-source connection, and first live sync; current flow is workable without customer-specific exceptions; SSO timing, admin-change audit history, admin-versus-billing-owner separation, and a standalone procurement packet remain next-phase enterprise-readiness questions. Devon added pricing: Pilot $2,500/month up to 50 monthly active developers, Growth $7,500/month up to 200 monthly active developers, overage $1,000 per additional 50 monthly active developers, enterprise add-ons such as SSO, audit logs, and advanced admin controls quoted separately only after they ship; bands map to monthly active developer usage, not purchased seats or directory size; Scaffold/internal test users do not count; preview/sample-only behavior stays separate from activation reporting; Evergreen mention should be budgetary and non-custom. Please do one last caveat pass, then send Sofia a bounded cover email in the Northstar thread with Devon copied and my normal outside-email signoff. Also DM Anna and Devon the caveat notes: the package is limited to the approved cohort view, bounded Evergreen context, and pricing model, and the next conversation stays a selective Mercury evidence review.

Message 000840 in history

Fact 153

The October cohort rows cover completed seven-day windows and do not provide a full 30-day expansion read for late-October cohorts.

Source evidence (1)

000840Nov 13, 2023 / 10:14 UTC-08:00

Anna and Devon sent the limited Northstar packet draft for final caveat pass. Anna's cohort view defines activated within 7 days as real source connected or first live sync completed within 7 days; sample import/preview usage is excluded; invite sent is supporting activity only; October rows are completed 7-day windows and do not include a full 30-day expansion read for late-October cohorts. Table: Jul 2023, 18 accounts entering bounded flow, 7/18 activated (39%), 5/18 first live sync (28%), early cohorts noisy and preview behavior overstated progress; Aug 2023, 22 accounts, 12/22 activated (55%), 9/22 first live sync (41%), cleaner admin handoff but enterprise questions start clustering; Sep 2023, 24 accounts, 15/24 activated (63%), 12/24 first live sync (50%), activation improved and second-admin behavior still uneven; Oct 2023, 27 accounts, 19/27 activated (70%), 16/27 first live sync (59%), best current activation read but expansion beyond initial admins remains uneven and not solved. Narrative: improved real-source/first-live-sync activation is the signal, not downstream expansion being clean; admin friction is lower; no weekly directional cuts or sample-preview-heavy behavior as proof points. Evergreen excerpt: second admin group is using org invites, magic-link access, preview-only sample data, real-source connection, and first live sync; current flow is workable without customer-specific exceptions; SSO timing, admin-change audit history, admin-versus-billing-owner separation, and a standalone procurement packet remain next-phase enterprise-readiness questions. Devon added pricing: Pilot $2,500/month up to 50 monthly active developers, Growth $7,500/month up to 200 monthly active developers, overage $1,000 per additional 50 monthly active developers, enterprise add-ons such as SSO, audit logs, and advanced admin controls quoted separately only after they ship; bands map to monthly active developer usage, not purchased seats or directory size; Scaffold/internal test users do not count; preview/sample-only behavior stays separate from activation reporting; Evergreen mention should be budgetary and non-custom. Please do one last caveat pass, then send Sofia a bounded cover email in the Northstar thread with Devon copied and my normal outside-email signoff. Also DM Anna and Devon the caveat notes: the package is limited to the approved cohort view, bounded Evergreen context, and pricing model, and the next conversation stays a selective Mercury evidence review.

Message 000840 in history

Fact 154

The October 2023 cohort had 27 accounts; 19 of 27 activated (70%) and 16 of 27 completed a first live sync (59%); this was the best current activation read, but expansion beyond initial admins remained uneven and unsolved.

Source evidence (1)

000840Nov 13, 2023 / 10:14 UTC-08:00

Anna and Devon sent the limited Northstar packet draft for final caveat pass. Anna's cohort view defines activated within 7 days as real source connected or first live sync completed within 7 days; sample import/preview usage is excluded; invite sent is supporting activity only; October rows are completed 7-day windows and do not include a full 30-day expansion read for late-October cohorts. Table: Jul 2023, 18 accounts entering bounded flow, 7/18 activated (39%), 5/18 first live sync (28%), early cohorts noisy and preview behavior overstated progress; Aug 2023, 22 accounts, 12/22 activated (55%), 9/22 first live sync (41%), cleaner admin handoff but enterprise questions start clustering; Sep 2023, 24 accounts, 15/24 activated (63%), 12/24 first live sync (50%), activation improved and second-admin behavior still uneven; Oct 2023, 27 accounts, 19/27 activated (70%), 16/27 first live sync (59%), best current activation read but expansion beyond initial admins remains uneven and not solved. Narrative: improved real-source/first-live-sync activation is the signal, not downstream expansion being clean; admin friction is lower; no weekly directional cuts or sample-preview-heavy behavior as proof points. Evergreen excerpt: second admin group is using org invites, magic-link access, preview-only sample data, real-source connection, and first live sync; current flow is workable without customer-specific exceptions; SSO timing, admin-change audit history, admin-versus-billing-owner separation, and a standalone procurement packet remain next-phase enterprise-readiness questions. Devon added pricing: Pilot $2,500/month up to 50 monthly active developers, Growth $7,500/month up to 200 monthly active developers, overage $1,000 per additional 50 monthly active developers, enterprise add-ons such as SSO, audit logs, and advanced admin controls quoted separately only after they ship; bands map to monthly active developer usage, not purchased seats or directory size; Scaffold/internal test users do not count; preview/sample-only behavior stays separate from activation reporting; Evergreen mention should be budgetary and non-custom. Please do one last caveat pass, then send Sofia a bounded cover email in the Northstar thread with Devon copied and my normal outside-email signoff. Also DM Anna and Devon the caveat notes: the package is limited to the approved cohort view, bounded Evergreen context, and pricing model, and the next conversation stays a selective Mercury evidence review.

Message 000840 in history

Expected tool calls

create_doc

Grading

1. field_equals / create_doc

{
  "type": "field_equals",
  "tool": "create_doc",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_145_00",
  "action_id": "morgan_145_create_doc"
}

2. field_llm_judge / create_doc

{
  "type": "field_llm_judge",
  "path": "result.document.body",
  "criterion": "Explains 19/27 activated versus 16/27 first live sync within seven days: three connected a real source within that window without completing first live sync then. Activation requires real source connection or completed first live sync, not sample/preview/invite activity. No full diligence scope or longer-window expansion recital is required.",
  "tool": "create_doc",
  "check_id": "morgan_145_01",
  "action_id": "morgan_145_create_doc"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_145_00",
        "action_id": "morgan_145_create_doc"
      },
      {
        "type": "field_llm_judge",
        "path": "result.document.body",
        "criterion": "Explains 19/27 activated versus 16/27 first live sync within seven days: three connected a real source within that window without completing first live sync then. Activation requires real source connection or completed first live sync, not sample/preview/invite activity. No full diligence scope or longer-window expansion recital is required.",
        "tool": "create_doc",
        "check_id": "morgan_145_01",
        "action_id": "morgan_145_create_doc"
      }
    ]
  }
}