Test 104

Jan 1, 2028 / 1 fact

Request

Create a concise metric card for the certified CS auto-nudge coverage in the closed four-lane source packet. Give the complete coverage dates including the year, total decisions, both disposition counts and rates, arithmetic reconciliation, source-age alert outcome, control failures, and the narrow system-control interpretation.

Required memory

Fact 228

Certified CS auto-nudge coverage for July 1 through October 31, 2026 included 6,704 unique decisions: 249 visible field-specific holds and 6,455 decisions that proceeded or were suppressed using current state. There was no governed-field disagreement, customer correction, or rollback, and four source-age alerts were resolved before breach.

Source evidence (1)

003762Nov 6, 2026 / 15:24 UTC-06:00

Owen completed the scheduled source-closure summary. He independently normalized and reconciled the engineering decision and reliability sources through October 31, confirmed that every required field is present, established definitions are unchanged, every named owner certified their section, and no cross-surface discrepancy exists. He therefore closed the first complete four-lane source packet under his September 24 authority. The certified packet shows: - Helio Start authority-checklist evidence, September 15 through October 31: 148 newly admitted organic workspaces, 45 checklist exposures, and seven ordinary setup-assistance requests, with no eligibility-route mismatch, lifecycle-suppression bypass, paid traffic, or unsupported admission. - Expansion Assist: 16 actual exposures from July 1 through September 15 with complete 45-day windows; six help-or-options requests and four expansions. No exposed account had active Customer Success adoption or renewal work, and the evidence remains descriptive. - Pricing-page continuity, July 1 through October 31: 12,426 organic sessions and 2,576 Helio Start eligibility-flow starts; 183, or 7.1%, were later routed out for an existing unsupported condition. Eligible-workspace admission completion was 44.4%, and account-level support time at the 90th percentile was 74 minutes. - CS auto-nudge coverage, July 1 through October 31: 6,704 unique decisions, including 249 visible field-specific holds and 6,455 decisions that proceeded or were suppressed using current state. There was no governed-field disagreement, customer correction, or rollback; four source-age alerts fired and resolved before breach. I received Owen’s closure-and-exception summary without doing a line-by-line source review. The earlier coverage discrepancy is resolved and the packet is complete. Causal, revenue, forecast, board-readiness, and executive-language decisions remain open in their existing owner lanes.

Message 003762 in history

Expected tool calls

create_doc

Grading

1. field_equals / create_doc

{
  "type": "field_equals",
  "tool": "create_doc",
  "action_id": "create_metric_card",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_104_00"
}

2. field_llm_judge / create_doc

{
  "type": "field_llm_judge",
  "tool": "create_doc",
  "action_id": "create_metric_card",
  "path": "args",
  "criterion": "The document states that the coverage period was July 1 through October 31, 2026 and reports exactly 6,704 unique decisions, comprising exactly 249 visible field-specific holds and exactly 6,455 decisions that proceeded or were suppressed using current state. It must make clear that the two disposition counts reconcile to the total (249 + 6,455 = 6,704). Missing or contradictory figures fail.\n\nThe document gives both correctly calculated disposition rates using 6,704 as the denominator: the visible-hold rate is approximately 3.71% and the proceeded-or-suppressed-using-current-state rate is approximately 96.29%. Reasonable rounding such as 3.7% and 96.3% is acceptable, and the rates should total approximately 100%.\n\nThe document records that four source-age alerts fired and all resolved before breach, and that there was no governed-field disagreement, no customer correction, and no rollback. Missing or contradicted outcomes fail.\n\nThe interpretation is explicitly limited to system-control evidence or control performance. It must not characterize the figures as evidence of customer success, engagement, conversion, revenue, pipeline, forecast, or other commercial performance, and must not make causal claims.",
  "check_id": "riley_104_01"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_doc",
        "action_id": "create_metric_card",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_104_00"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_doc",
        "action_id": "create_metric_card",
        "path": "args",
        "criterion": "The document states that the coverage period was July 1 through October 31, 2026 and reports exactly 6,704 unique decisions, comprising exactly 249 visible field-specific holds and exactly 6,455 decisions that proceeded or were suppressed using current state. It must make clear that the two disposition counts reconcile to the total (249 + 6,455 = 6,704). Missing or contradictory figures fail.\n\nThe document gives both correctly calculated disposition rates using 6,704 as the denominator: the visible-hold rate is approximately 3.71% and the proceeded-or-suppressed-using-current-state rate is approximately 96.29%. Reasonable rounding such as 3.7% and 96.3% is acceptable, and the rates should total approximately 100%.\n\nThe document records that four source-age alerts fired and all resolved before breach, and that there was no governed-field disagreement, no customer correction, and no rollback. Missing or contradicted outcomes fail.\n\nThe interpretation is explicitly limited to system-control evidence or control performance. It must not characterize the figures as evidence of customer success, engagement, conversion, revenue, pipeline, forecast, or other commercial performance, and must not make causal claims.",
        "check_id": "riley_104_01"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/riley/104.yaml

SHA-256: 4cb5ce5eb308f53a53321ba1a17a385a6a508988ccae8a30af9834f17d405218