Test 023 for Riley Tanaka

Request

Create an audit worksheet for a historical review of Q4 2023 Expansion Assist shadow decisions on retained mid-segment accounts, applying the Customer Success suppression check and the evidence combination required before a commercial prompt could be considered at that time.

Required memory

Fact 413

For Q4 Expansion Assist, open Customer Success adoption work suppresses commercial prompts.

Source evidence (1)

000749Oct 24, 2023 / 10:45 UTC-05:00

We ran the first Q4 Expansion Assist shadow pass, and the edge cases were exactly the ones I expected: open Customer Success adoption work and usage-ceiling-only rows. Customer Success was clear that they can help Growth review false positives, but they cannot become a weekly pre-clear queue for Sales alerts, and Marcus asked whether the useful-looking Sales-alert state can become a Q4 pipeline signal. I said no and kept the review narrow. Please post this boundary note directly on the `Expansion Assist state matrix` thread so the rules don't drift: open CS adoption work suppresses commercial prompts; usage-ceiling evidence is not enough without active-admin behavior; Growth and Customer Success will review false positives monthly while the state rules are still being proven; Customer Success is not a weekly pre-clear queue for Sales alerts; and this remains Q4 design work, not a launch spec, Sales queue, Q4 forecast lever, revenue-credit source, or live replacement for the simple day-60 expansion nudge.

Message 000749 in history

Fact 414

For Q4 Expansion Assist, usage-ceiling evidence is insufficient without active-admin behavior.

Source evidence (1)

000749Oct 24, 2023 / 10:45 UTC-05:00

We ran the first Q4 Expansion Assist shadow pass, and the edge cases were exactly the ones I expected: open Customer Success adoption work and usage-ceiling-only rows. Customer Success was clear that they can help Growth review false positives, but they cannot become a weekly pre-clear queue for Sales alerts, and Marcus asked whether the useful-looking Sales-alert state can become a Q4 pipeline signal. I said no and kept the review narrow. Please post this boundary note directly on the `Expansion Assist state matrix` thread so the rules don't drift: open CS adoption work suppresses commercial prompts; usage-ceiling evidence is not enough without active-admin behavior; Growth and Customer Success will review false positives monthly while the state rules are still being proven; Customer Success is not a weekly pre-clear queue for Sales alerts; and this remains Q4 design work, not a launch spec, Sales queue, Q4 forecast lever, revenue-credit source, or live replacement for the simple day-60 expansion nudge.

Message 000749 in history

Expected tool calls

create_doc

Grading

1. field_equals / create_doc

{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_expansion_assist_q4_audit",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_023_00"
}

2. field_llm_judge / create_doc

{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "create_expansion_assist_q4_audit",
  "path": "args",
  "criterion": "The document is a usable audit worksheet for reviewing individual shadow decisions and makes open Customer Success adoption work a suppressing condition: when such work is open, a commercial prompt must not be considered or allowed, regardless of otherwise positive evidence.\n\nThe worksheet does not treat usage-ceiling evidence alone as enough for commercial-prompt consideration; it requires active-admin behavior alongside usage-ceiling evidence before a prompt may be considered, without implying that this pair overrides suppression conditions.",
  "check_id": "riley_023_01"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_expansion_assist_q4_audit",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_023_00"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "create_expansion_assist_q4_audit",
        "path": "args",
        "criterion": "The document is a usable audit worksheet for reviewing individual shadow decisions and makes open Customer Success adoption work a suppressing condition: when such work is open, a commercial prompt must not be considered or allowed, regardless of otherwise positive evidence.\n\nThe worksheet does not treat usage-ceiling evidence alone as enough for commercial-prompt consideration; it requires active-admin behavior alongside usage-ceiling evidence before a prompt may be considered, without implying that this pair overrides suppression conditions.",
        "check_id": "riley_023_01"
      }
    ]
  }
}