Test 071

Sep 14, 2026 / 1 fact

Request

Post a review comment on scaffold/metrics#2417 naming the tracing vendor our new request instrumentation should target.

Required memory

Fact 57

On the morning of April 14, 2023, Morgan Chen chose Honeycomb over Datadog because Honeycomb's tracing is better for their use case, particularly for following requests end to end.

Source evidence (1)

000364Apr 14, 2023 / 12:00 UTC-07:00

kept flipping Datadog vs Honeycomb longer than i wanted. settled it this morning -- Honeycomb for us the tracing is just better, especially once you actually follow a request end to end instead of staring at logs and pretending. anyway. decision made. moving on

Message 000364 in history

Expected tool calls

post_pr_comment

Grading

1. field_equals / post_pr_comment

{
  "type": "field_equals",
  "tool": "post_pr_comment",
  "path": "result.ok",
  "value": true,
  "check_id": "morgan_071_00",
  "action_id": "morgan_071_post_pr_comment"
}

2. field_equals / post_pr_comment

{
  "type": "field_equals",
  "path": "result.comment.pr_id",
  "value": "scaffold/metrics#2417",
  "tool": "post_pr_comment",
  "check_id": "morgan_071_01",
  "action_id": "morgan_071_post_pr_comment"
}

3. field_llm_judge / post_pr_comment

{
  "type": "field_llm_judge",
  "path": "result.comment.body",
  "criterion": "Selects Honeycomb for request tracing, not Datadog or an unspecified vendor.",
  "tool": "post_pr_comment",
  "check_id": "morgan_071_02",
  "action_id": "morgan_071_post_pr_comment"
}

Complete grading specification

{
  "type": "tool_trace",
  "config": {
    "check_version": 2,
    "today": "2026-09-14",
    "assertions": [
      {
        "type": "field_equals",
        "tool": "post_pr_comment",
        "path": "result.ok",
        "value": true,
        "check_id": "morgan_071_00",
        "action_id": "morgan_071_post_pr_comment"
      },
      {
        "type": "field_equals",
        "path": "result.comment.pr_id",
        "value": "scaffold/metrics#2417",
        "tool": "post_pr_comment",
        "check_id": "morgan_071_01",
        "action_id": "morgan_071_post_pr_comment"
      },
      {
        "type": "field_llm_judge",
        "path": "result.comment.body",
        "criterion": "Selects Honeycomb for request tracing, not Datadog or an unspecified vendor.",
        "tool": "post_pr_comment",
        "check_id": "morgan_071_02",
        "action_id": "morgan_071_post_pr_comment"
      }
    ]
  }
}

App state

Download JSON

Source file

tests/morgan/071.yaml

SHA-256: 66aa8338f7bfad5d4f2ef8a02eab96eb6a0eda750267cbe6af9783cba7da571c