DolphinBench

Test 159

Sep 14, 2026 / 3 facts

YAML

Request

Create an archive note explaining whether Evergreen's January 2024 current-flow examples pack was cleared to circulate and whether it constituted a complete procurement/security packet.

Required memory

Fact 168

The pack’s first-admin example has the first admin receive access to the Mercury organization, enter through the current magic-link flow, land in that organization’s setup experience, optionally see preview or sample data only as bounded non-activating context, connect a real source, and proceed to first live sync without an SSO/SAML step.

Source evidence (1)

000962Jan 5, 2024 / 09:46 UTC-08:00

Sarah has the final Evergreen pack and reviewer signal. From: Sarah Kim To: Morgan Chen, Devon Hayes, Jake, Leo Park Date: Fri, Jan 5, 2024 9:27 AM PST Subject: Re: Evergreen example cases to turn into current-flow pack Quick update: Evergreen's procurement/security reviewers said the example format is usable for internal routing. Their note was that the concrete examples are specific enough to circulate internally without turning into a roadmap discussion. Forwarded note excerpt from Evergreen reviewers The example-based format is usable for our internal routing. The current-state language is clearer when paired with concrete flows. Please keep it in the present-state frame rather than roadmap timing. Final pack Evergreen Mercury current-flow examples Use - Current bounded flow only: magic links, org invites, preview-only sample data, real-source connection, and first live sync. - No SSO/SAML step is part of the current examples. - No Evergreen-specific exception or separate branch is being described. 1. First admin - Current example: a first admin receives access to the Mercury org, enters through the current magic-link flow, and lands in the setup experience for that org. - Preview/sample data may appear as bounded context only; it is non-activating. - The current path proceeds to real-source connection and then first live sync; there is no SSO/SAML step in this flow. 2. Second admin - Current example: an existing admin invites a second admin into the same workspace using the current org-invite flow. - The second admin enters through the same email-based magic-link model and joins the same current Mercury experience. - This is the current shared flow, not a separate Evergreen-only or separately permissioned branch. 3. Delayed or failed first live sync - Current example: a real source is connected, but first live sync is delayed or does not complete cleanly. - The current product example should describe present-state sync behavior and follow-up through the existing thread with Sarah if the issue needs investigation. - This example does not represent a standalone admin-change audit history or a full self-serve failure-debug surface. 4. Invite / resend - Current example: an admin invites another admin; if the original invite is missed or expires, the current flow supports resend through the same invite-based path. - The recipient re-enters through the same email/magic-link model. - This example shows present invite behavior only and should not be presented as an audited event trail. 5. Source connection / disconnection - Current example: an admin connects a real source and Mercury starts the current first-live-sync process. - If a source is later disconnected or reconnected, that can be described as current product behavior at a high level. - The example should not imply a standalone historical ledger for those admin-side changes. 6. Audit-history caveat - Current product position now: Mercury does not provide standalone admin-side audit history for invite or resend actions, role or permission changes, source connection/disconnection, or related admin configuration changes. - Later-scope frame: audit history for admin-side changes remains next-phase enterprise-readiness evaluation. - Available-now materials: explicit written current-state language plus the concrete current-flow examples above. 7. Materials separation - Circulate now: bounded-flow summary, this examples pack, and written current-state answers that separate current Mercury behavior from later enterprise-readiness evaluation. - Not a current circulate-now artifact: a fuller standalone procurement/security packet that fully replaces live walkthroughs. - Later-scope frame: fuller procurement/security packaging remains later enterprise-readiness work. Please reply to Sarah in the Evergreen thread: yes, send the final concrete examples pack to Evergreen now. Keep it present-state and circulate-now only, and say the fuller standalone procurement/security packet remains later enterprise-readiness work.

Message 000962 in history

Fact 169

The final pack states that Mercury does not provide standalone admin-side audit history for invite or resend actions, role or permission changes, source connection or disconnection, or related administrative configuration changes.

Source evidence (1)

000962Jan 5, 2024 / 09:46 UTC-08:00

Sarah has the final Evergreen pack and reviewer signal. From: Sarah Kim To: Morgan Chen, Devon Hayes, Jake, Leo Park Date: Fri, Jan 5, 2024 9:27 AM PST Subject: Re: Evergreen example cases to turn into current-flow pack Quick update: Evergreen's procurement/security reviewers said the example format is usable for internal routing. Their note was that the concrete examples are specific enough to circulate internally without turning into a roadmap discussion. Forwarded note excerpt from Evergreen reviewers The example-based format is usable for our internal routing. The current-state language is clearer when paired with concrete flows. Please keep it in the present-state frame rather than roadmap timing. Final pack Evergreen Mercury current-flow examples Use - Current bounded flow only: magic links, org invites, preview-only sample data, real-source connection, and first live sync. - No SSO/SAML step is part of the current examples. - No Evergreen-specific exception or separate branch is being described. 1. First admin - Current example: a first admin receives access to the Mercury org, enters through the current magic-link flow, and lands in the setup experience for that org. - Preview/sample data may appear as bounded context only; it is non-activating. - The current path proceeds to real-source connection and then first live sync; there is no SSO/SAML step in this flow. 2. Second admin - Current example: an existing admin invites a second admin into the same workspace using the current org-invite flow. - The second admin enters through the same email-based magic-link model and joins the same current Mercury experience. - This is the current shared flow, not a separate Evergreen-only or separately permissioned branch. 3. Delayed or failed first live sync - Current example: a real source is connected, but first live sync is delayed or does not complete cleanly. - The current product example should describe present-state sync behavior and follow-up through the existing thread with Sarah if the issue needs investigation. - This example does not represent a standalone admin-change audit history or a full self-serve failure-debug surface. 4. Invite / resend - Current example: an admin invites another admin; if the original invite is missed or expires, the current flow supports resend through the same invite-based path. - The recipient re-enters through the same email/magic-link model. - This example shows present invite behavior only and should not be presented as an audited event trail. 5. Source connection / disconnection - Current example: an admin connects a real source and Mercury starts the current first-live-sync process. - If a source is later disconnected or reconnected, that can be described as current product behavior at a high level. - The example should not imply a standalone historical ledger for those admin-side changes. 6. Audit-history caveat - Current product position now: Mercury does not provide standalone admin-side audit history for invite or resend actions, role or permission changes, source connection/disconnection, or related admin configuration changes. - Later-scope frame: audit history for admin-side changes remains next-phase enterprise-readiness evaluation. - Available-now materials: explicit written current-state language plus the concrete current-flow examples above. 7. Materials separation - Circulate now: bounded-flow summary, this examples pack, and written current-state answers that separate current Mercury behavior from later enterprise-readiness evaluation. - Not a current circulate-now artifact: a fuller standalone procurement/security packet that fully replaces live walkthroughs. - Later-scope frame: fuller procurement/security packaging remains later enterprise-readiness work. Please reply to Sarah in the Evergreen thread: yes, send the final concrete examples pack to Evergreen now. Keep it present-state and circulate-now only, and say the fuller standalone procurement/security packet remains later enterprise-readiness work.

Message 000962 in history

Fact 167

Morgan Chen asked the recipients to reply to Sarah Kim in the Evergreen thread confirming that the final concrete examples pack should be sent to Evergreen now, kept limited to present-state and circulate-now material, and accompanied by a statement that the fuller standalone procurement/security packet remains later enterprise-readiness work.

Source evidence (1)

000962Jan 5, 2024 / 09:46 UTC-08:00

Sarah has the final Evergreen pack and reviewer signal. From: Sarah Kim To: Morgan Chen, Devon Hayes, Jake, Leo Park Date: Fri, Jan 5, 2024 9:27 AM PST Subject: Re: Evergreen example cases to turn into current-flow pack Quick update: Evergreen's procurement/security reviewers said the example format is usable for internal routing. Their note was that the concrete examples are specific enough to circulate internally without turning into a roadmap discussion. Forwarded note excerpt from Evergreen reviewers The example-based format is usable for our internal routing. The current-state language is clearer when paired with concrete flows. Please keep it in the present-state frame rather than roadmap timing. Final pack Evergreen Mercury current-flow examples Use - Current bounded flow only: magic links, org invites, preview-only sample data, real-source connection, and first live sync. - No SSO/SAML step is part of the current examples. - No Evergreen-specific exception or separate branch is being described. 1. First admin - Current example: a first admin receives access to the Mercury org, enters through the current magic-link flow, and lands in the setup experience for that org. - Preview/sample data may appear as bounded context only; it is non-activating. - The current path proceeds to real-source connection and then first live sync; there is no SSO/SAML step in this flow. 2. Second admin - Current example: an existing admin invites a second admin into the same workspace using the current org-invite flow. - The second admin enters through the same email-based magic-link model and joins the same current Mercury experience. - This is the current shared flow, not a separate Evergreen-only or separately permissioned branch. 3. Delayed or failed first live sync - Current example: a real source is connected, but first live sync is delayed or does not complete cleanly. - The current product example should describe present-state sync behavior and follow-up through the existing thread with Sarah if the issue needs investigation. - This example does not represent a standalone admin-change audit history or a full self-serve failure-debug surface. 4. Invite / resend - Current example: an admin invites another admin; if the original invite is missed or expires, the current flow supports resend through the same invite-based path. - The recipient re-enters through the same email/magic-link model. - This example shows present invite behavior only and should not be presented as an audited event trail. 5. Source connection / disconnection - Current example: an admin connects a real source and Mercury starts the current first-live-sync process. - If a source is later disconnected or reconnected, that can be described as current product behavior at a high level. - The example should not imply a standalone historical ledger for those admin-side changes. 6. Audit-history caveat - Current product position now: Mercury does not provide standalone admin-side audit history for invite or resend actions, role or permission changes, source connection/disconnection, or related admin configuration changes. - Later-scope frame: audit history for admin-side changes remains next-phase enterprise-readiness evaluation. - Available-now materials: explicit written current-state language plus the concrete current-flow examples above. 7. Materials separation - Circulate now: bounded-flow summary, this examples pack, and written current-state answers that separate current Mercury behavior from later enterprise-readiness evaluation. - Not a current circulate-now artifact: a fuller standalone procurement/security packet that fully replaces live walkthroughs. - Later-scope frame: fuller procurement/security packaging remains later enterprise-readiness work. Please reply to Sarah in the Evergreen thread: yes, send the final concrete examples pack to Evergreen now. Keep it present-state and circulate-now only, and say the fuller standalone procurement/security packet remains later enterprise-readiness work.

Message 000962 in history

Expected tool calls

  • create_doc

Grading

1. field_equals / create_doc
{
  "type": "field_equals",
  "tool": "create_doc",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_159_00",
  "action_id": "morgan_159_create_doc"
}
2. field_llm_judge / create_doc
{
  "type": "field_llm_judge",
  "path": "result.document.body",
  "criterion": "Says the bounded current-flow examples were cleared for Evergreen internal routing, but were not a finished standalone procurement/security packet. Fuller standalone packaging remained later work. No list of excluded features is required. Claims that those January examples already included SSO/SAML or standalone admin-change audit history are incorrect.",
  "tool": "create_doc",
  "check_id": "morgan_159_01",
  "action_id": "morgan_159_create_doc"
}
Complete grading specification
{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_159_00",
        "action_id": "morgan_159_create_doc"
      },
      {
        "type": "field_llm_judge",
        "path": "result.document.body",
        "criterion": "Says the bounded current-flow examples were cleared for Evergreen internal routing, but were not a finished standalone procurement/security packet. Fuller standalone packaging remained later work. No list of excluded features is required. Claims that those January examples already included SSO/SAML or standalone admin-change audit history are incorrect.",
        "tool": "create_doc",
        "check_id": "morgan_159_01",
        "action_id": "morgan_159_create_doc"
      }
    ]
  }
}
App stateDownload JSON
Source file

tests/morgan/159.yaml

SHA-256: b0e5a8f8e0b0da51a85ac45f59528e73df8f4c9b153823e11713b536a83b2b3a