Test 020

Jan 1, 2028 / 1 fact

Request

Create a 10-minute reminder at 9:00 AM Central three days before each 2028 employer HSA deposit. Each reminder must include the deposit date, amount, and eligibility condition.

Required memory

Fact 284

Helio’s employee-only HDHP includes employer HSA deposits of $250 scheduled for January 15, April 15, July 15, and October 15, 2028, each contingent on Riley Tanaka remaining HSA-eligible on the deposit date.

Source evidence (1)

004825Oct 5, 2027 / 08:20 UTC-05:00

Helio’s final October 5 benefits posting says: Election deadline: November 12, 2027. My current primary-care clinician, physical-therapy clinic, and eye-care clinic are in both final networks. Employee-only HDHP: $98 on each of 24 paychecks, $2,352 annually; $2,000 deductible; $4,000 out-of-pocket maximum; employer HSA deposits of $250 scheduled for January 15, April 15, July 15, and October 15, 2028, each contingent on me remaining HSA-eligible on that date. Employee-only PPO: $164 on each of 24 paychecks, $3,936 annually; $750 deductible; $3,500 out-of-pocket maximum; no employer HSA contribution.

Message 004825 in history

Expected tool calls

create_calendar_event

Grading

1. field_equals / create_calendar_event

{
  "type": "field_equals",
  "tool": "create_calendar_event",
  "action_id": "jan_hsa_reminder",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_020_00"
}

2. field_equals / create_calendar_event

{
  "type": "field_equals",
  "tool": "create_calendar_event",
  "action_id": "apr_hsa_reminder",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_020_01"
}

3. field_equals / create_calendar_event

{
  "type": "field_equals",
  "tool": "create_calendar_event",
  "action_id": "jul_hsa_reminder",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_020_02"
}

4. field_equals / create_calendar_event

{
  "type": "field_equals",
  "tool": "create_calendar_event",
  "action_id": "oct_hsa_reminder",
  "path": "result.ok",
  "value": true,
  "check_id": "riley_020_03"
}

5. datetime_absolute_eq / create_calendar_event

{
  "type": "datetime_absolute_eq",
  "tool": "create_calendar_event",
  "action_id": "jan_hsa_reminder",
  "path": "args.start",
  "value": "2028-01-12T09:00:00-06:00",
  "check_id": "riley_020_04"
}

6. datetime_absolute_eq / create_calendar_event

{
  "type": "datetime_absolute_eq",
  "tool": "create_calendar_event",
  "action_id": "jan_hsa_reminder",
  "path": "args.end",
  "value": "2028-01-12T09:10:00-06:00",
  "check_id": "riley_020_05"
}

7. datetime_absolute_eq / create_calendar_event

{
  "type": "datetime_absolute_eq",
  "tool": "create_calendar_event",
  "action_id": "apr_hsa_reminder",
  "path": "args.start",
  "value": "2028-04-12T09:00:00-05:00",
  "check_id": "riley_020_06"
}

8. datetime_absolute_eq / create_calendar_event

{
  "type": "datetime_absolute_eq",
  "tool": "create_calendar_event",
  "action_id": "apr_hsa_reminder",
  "path": "args.end",
  "value": "2028-04-12T09:10:00-05:00",
  "check_id": "riley_020_07"
}

9. datetime_absolute_eq / create_calendar_event

{
  "type": "datetime_absolute_eq",
  "tool": "create_calendar_event",
  "action_id": "jul_hsa_reminder",
  "path": "args.start",
  "value": "2028-07-12T09:00:00-05:00",
  "check_id": "riley_020_08"
}

10. datetime_absolute_eq / create_calendar_event

{
  "type": "datetime_absolute_eq",
  "tool": "create_calendar_event",
  "action_id": "jul_hsa_reminder",
  "path": "args.end",
  "value": "2028-07-12T09:10:00-05:00",
  "check_id": "riley_020_09"
}

11. datetime_absolute_eq / create_calendar_event

{
  "type": "datetime_absolute_eq",
  "tool": "create_calendar_event",
  "action_id": "oct_hsa_reminder",
  "path": "args.start",
  "value": "2028-10-12T09:00:00-05:00",
  "check_id": "riley_020_10"
}

12. datetime_absolute_eq / create_calendar_event

{
  "type": "datetime_absolute_eq",
  "tool": "create_calendar_event",
  "action_id": "oct_hsa_reminder",
  "path": "args.end",
  "value": "2028-10-12T09:10:00-05:00",
  "check_id": "riley_020_11"
}

13. field_llm_judge / create_calendar_event

{
  "type": "field_llm_judge",
  "tool": "create_calendar_event",
  "action_id": "jan_hsa_reminder",
  "path": "args",
  "criterion": "The reminder identifies the $250 employer HSA deposit scheduled for January 15",
  "check_id": "riley_020_12"
}

14. field_llm_judge / create_calendar_event

{
  "type": "field_llm_judge",
  "tool": "create_calendar_event",
  "action_id": "apr_hsa_reminder",
  "path": "args",
  "criterion": "The reminder identifies the $250 employer HSA deposit scheduled for April 15",
  "check_id": "riley_020_13"
}

15. field_llm_judge / create_calendar_event

{
  "type": "field_llm_judge",
  "tool": "create_calendar_event",
  "action_id": "jul_hsa_reminder",
  "path": "args",
  "criterion": "The reminder identifies the $250 employer HSA deposit scheduled for July 15",
  "check_id": "riley_020_14"
}

16. field_llm_judge / create_calendar_event

{
  "type": "field_llm_judge",
  "tool": "create_calendar_event",
  "action_id": "oct_hsa_reminder",
  "path": "args",
  "criterion": "The reminder identifies the $250 employer HSA deposit scheduled for October 15",
  "check_id": "riley_020_15"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2028-01-01",
    "semantic_judge_version": 2,
    "assertions": [
      {
        "type": "field_equals",
        "tool": "create_calendar_event",
        "action_id": "jan_hsa_reminder",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_020_00"
      },
      {
        "type": "field_equals",
        "tool": "create_calendar_event",
        "action_id": "apr_hsa_reminder",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_020_01"
      },
      {
        "type": "field_equals",
        "tool": "create_calendar_event",
        "action_id": "jul_hsa_reminder",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_020_02"
      },
      {
        "type": "field_equals",
        "tool": "create_calendar_event",
        "action_id": "oct_hsa_reminder",
        "path": "result.ok",
        "value": true,
        "check_id": "riley_020_03"
      },
      {
        "type": "datetime_absolute_eq",
        "tool": "create_calendar_event",
        "action_id": "jan_hsa_reminder",
        "path": "args.start",
        "value": "2028-01-12T09:00:00-06:00",
        "check_id": "riley_020_04"
      },
      {
        "type": "datetime_absolute_eq",
        "tool": "create_calendar_event",
        "action_id": "jan_hsa_reminder",
        "path": "args.end",
        "value": "2028-01-12T09:10:00-06:00",
        "check_id": "riley_020_05"
      },
      {
        "type": "datetime_absolute_eq",
        "tool": "create_calendar_event",
        "action_id": "apr_hsa_reminder",
        "path": "args.start",
        "value": "2028-04-12T09:00:00-05:00",
        "check_id": "riley_020_06"
      },
      {
        "type": "datetime_absolute_eq",
        "tool": "create_calendar_event",
        "action_id": "apr_hsa_reminder",
        "path": "args.end",
        "value": "2028-04-12T09:10:00-05:00",
        "check_id": "riley_020_07"
      },
      {
        "type": "datetime_absolute_eq",
        "tool": "create_calendar_event",
        "action_id": "jul_hsa_reminder",
        "path": "args.start",
        "value": "2028-07-12T09:00:00-05:00",
        "check_id": "riley_020_08"
      },
      {
        "type": "datetime_absolute_eq",
        "tool": "create_calendar_event",
        "action_id": "jul_hsa_reminder",
        "path": "args.end",
        "value": "2028-07-12T09:10:00-05:00",
        "check_id": "riley_020_09"
      },
      {
        "type": "datetime_absolute_eq",
        "tool": "create_calendar_event",
        "action_id": "oct_hsa_reminder",
        "path": "args.start",
        "value": "2028-10-12T09:00:00-05:00",
        "check_id": "riley_020_10"
      },
      {
        "type": "datetime_absolute_eq",
        "tool": "create_calendar_event",
        "action_id": "oct_hsa_reminder",
        "path": "args.end",
        "value": "2028-10-12T09:10:00-05:00",
        "check_id": "riley_020_11"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_calendar_event",
        "action_id": "jan_hsa_reminder",
        "path": "args",
        "criterion": "The reminder identifies the $250 employer HSA deposit scheduled for January 15",
        "check_id": "riley_020_12"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_calendar_event",
        "action_id": "apr_hsa_reminder",
        "path": "args",
        "criterion": "The reminder identifies the $250 employer HSA deposit scheduled for April 15",
        "check_id": "riley_020_13"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_calendar_event",
        "action_id": "jul_hsa_reminder",
        "path": "args",
        "criterion": "The reminder identifies the $250 employer HSA deposit scheduled for July 15",
        "check_id": "riley_020_14"
      },
      {
        "type": "field_llm_judge",
        "tool": "create_calendar_event",
        "action_id": "oct_hsa_reminder",
        "path": "args",
        "criterion": "The reminder identifies the $250 employer HSA deposit scheduled for October 15",
        "check_id": "riley_020_15"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/riley/020.yaml

SHA-256: fb14009588b54c7276e2d83fb94f69386fe100d95372eb672dfe9c9ba9693af0