finding: the QA assertion `field` path is documented as `body.*`, but `HttpTestAdapter` returns the parsed body with no `body` wrapper — every documented path resolves to `undefined`

[os-zhuang]

Observation-class finding, measured while running the blast radius for #7256 (PR #7348). Filed unassigned, no pm:queue — for triage to grade. Deliberately not folded into #7256's PR: the fix there is one else branch in packages/core/src/qa/runner.ts, and this is a mismatch between packages/core/src/qa/http-adapter.ts and the describe() text in packages/spec/src/qa/testing.zod.ts — a different file and a different call (change the adapter, or change the convention).

The mismatch

TestAssertionSchema.field documents its own convention, and the generated reference repeats it:

field: z.string().describe('Field path in the result to check (e.g. "body.data.0.status")')

TestRunner.assert resolves that path against whatever the adapter returned. HttpTestAdapter.handleResponse returns the parsed response body itself:

const contentType = response.headers.get('content-type');
if (contentType && contentType.includes('application/json')) {
    return response.json();     // ← the body, not { body: … }
}
return response.text();

There is no body key anywhere in the value the runner sees, and none in the platform's own response envelope either (ResponseEnvelopeConfigSchema wraps into data + meta, never body). So a suite written to the documented convention — field: "body.data.0.status", field: "body.status" — resolves every assertion path to undefined.

Why it surfaced now, and why it is worth a card

Until #7256 this was half-invisible. With equals/not_equals/not_null a body.* path already failed loudly, so an author hit it on their first run and worked the convention out by trial. With contains the undefined fell out of the switch and the assertion passed, so a body.* contains reported ✅ forever and nobody learned anything.

Measured against a stand-in server returning {"ok":true,"data":{"total":0},"name":"acme corp","tags":["acme","globex"]}, same fixture, same HttpTestAdapter:

scenario	base f3f855ac	with #7256's fix
contains on body.data.items (the documented convention)	✅ (asserting nothing)	❌ got undefined
contains on data.items (path absent)	✅ (asserting nothing)	❌ got undefined
contains on data.total (a number)	✅ (asserting nothing)	❌ got number
contains on tags (a real array)	✅	✅
contains on name (a real string)	✅	✅

So #7256 makes the mismatch visible rather than causing it — the documented-convention row is a test that was never running. That is the right outcome, but the author who now sees red is following the schema's own example, and the error will tell them the path did not resolve without telling them the documentation is what's wrong.

The call, which is a real choice

1. Fix the convention — correct the describe() example (and the generated reference) to the shape the adapter actually returns (data.0.status). Cheapest, and makes the docs true today. Touches packages/spec (domain:spec-surface by the accept-face test: describe text only, no accept/reject change).
2. Fix the adapter — have HttpTestAdapter return { status, headers, body } so body.* and status both work, which is the shape the field example implies and what most HTTP test runners hand back. Strictly better for authors (an assertion on the status code is impossible today), but it is a behaviour change to a published adapter and would break any suite written against the current shape.

Option 2 is the better surface and option 1 is the smaller change; picking between them is a protocol call, not a cleanup, which is why this is filed rather than fixed.

Dedup

Searched open issues and PRs for HttpTestAdapter, the field path convention and the body prefix before filing — nothing names this. #7256 names the contains fall-through only.

[claude]

Findings triage: escalated finding to needs-user-decision (kept domain:engine-core).

• The fork is genuine and two-way: (1) fix the documented convention (testing.zod.ts describe text to the adapter's actual shape, e.g. data.0.status — lands domain:spec-surface by the accept-face test), or (2) change HttpTestAdapter to return { status, headers, body } — the better authoring surface (status assertions become possible) but a behavior change to a published adapter that breaks suites written against the current shape. Picking one is a protocol call, as the filer said.
• Timeliness (why escalate now rather than hold): qa(core): the contains assertion SILENTLY PASSES when the actual value is neither an array nor a string #7256's fix (PR fix(core): fail the QA contains assertion on a non-evaluable actual instead of silently passing (#7256) #7348, in flight) flips the failure mode from silent-pass to loud-red for every suite written to the documented body.* convention; the ruling is best made before or with that landing, so authors hitting red get docs that tell the truth.
• Premise verified on origin/main @ 60f0dd8: HttpTestAdapter.handleResponse returns the parsed body unwrapped; the describe() example says body.data.0.status.
• Recommendation: option 1 now (docs true today, zero break), with option 2 as a spec proposal if author demand shows — mirrors the Object-form params on a type:'url' inline action is a THIRD meaning of the key (interpolation scope + newTab) with no authorized spelling (observation) #6828-family "no key without demonstrated pull" pattern.
• Dedup: qa(core): the contains assertion SILENTLY PASSES when the actual value is neither an array nor a string #7256 names the contains fall-through only; nothing else names the field path convention.

本评论来自分诊座位 Routine（#5474 试点），不构成认领。

---

Generated by Claude Code