docs(gaps): register LV-26 — an upstream 401 reaches the caller as 502 - #140
Merged
Conversation
… Core
Step 5 reported "no gates" against a Core that had evaluated six of them.
The Core was right; the assertion was wrong.
The canonical `EvaluationResult` nests per-kind results under `results`
(`gate`, `artifact`, `compliance`). It is the TRACKER's own DTO that
flattens them to `gates`, and this robot reaches the Core THROUGH the
gateway, so it receives the canonical shape and never the flattened one.
A second defect sat in the same helper and would have survived the first
fix: `summarizeVerdict` compared each verdict against `'failed'` while the
Core emits `FAIL`. Even given the right array it would have reported zero
failures on a run where all six gates failed — a green-looking summary
over a red result, which is worse than no summary at all.
Verified against a captured REST response rather than argued, and run
against the UNFIXED helper first so the change is known to matter:
before gates=0 · failed=0 (and the check said "no gates")
after gates=6 · failed=6
Checked for the same shape elsewhere in robosoft/: no other robot reads
`.gates` or compares against a lowercase verdict.
Found while running this robot against a live two-cluster stack for the
first time — it is excluded from the default runner and had never
executed. The same run found a real product defect (the MCP chart's
runAsUser, fixed in evolith#425); this one was the instrument.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`AgentRuntimeGateway` throws with the upstream status. The executor
flattens it to a string so the failed turn is still recorded — that
intent is right — but the status does not survive: `Translate` re-derives
one from the string, matches no case, and falls through to
`_ => ("AgentRuntime.Failed", 502)`. Two things are lost at once: the
upstream 401, and the specific code, rewritten to the generic one.
A CREDENTIAL problem is therefore reported as an AVAILABILITY problem.
502 sends the operator to check whether the runtime is up; it was up, and
answering 401 in 88ms. The endpoint's own comment already recognises that
collapsing to an HTTP code erases the story and that the trace keeps it —
but the caller has no trace, and the caller is who acts.
Observed on 2026-08-04: `core-integration` reported
`POST /assistant/converse → 502` against a live two-cluster stack. The
cause was AGENT_RUNTIME_API_KEY differing between the Tracker's secret and
the runtime's. Only the tracker-api log carried the 401. Aligning the key
turned the step green, so the diagnosis is confirmed and the mis-mapping
is what cost the time.
Registered P2/XS with the fix scoped to the ERASURE, not the number: 502
is arguably right (the caller's own auth succeeded, the failure is
upstream) — what is wrong is that the body cannot tell an auth failure
from an unreachable runtime.
Found running this robot against a live stack for the FIRST time; it is
excluded from the default runner. The same run found a real Core defect
(evolith_arch32#425) and one in the robot itself (#139).
NOTE, pre-existing and untouched: the board's declared counters are stale
by five rows. Before this change the header said 174 done / 17 pending
while the table held 179 DONE / 12 PENDING; the same offset holds after.
This row follows the existing convention (total and pending both +1)
rather than silently re-baselining numbers it has not reconciled
row-by-row. Worth its own pass — the Core repo has a guard for exactly
this and this board has none.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The defect
AgentRuntimeGatewaythrowsAgentRuntimeGatewayException(code, statusCode: <upstream>).AgentRuntimeTurnExecutorcatches it and flattens it to a string so the failed turn is still recorded — that intent is right. But the status does not survive the flattening:AssistantEndpoints.Translatere-derives one from the string, matches no case, and falls through:Two things are lost at once — the upstream status (401) and the specific code, rewritten from
AgentRuntime.HttpErrorto the genericAgentRuntime.Failed.Why it matters
A credential problem is reported as an availability problem.
502 Bad Gatewaysends the operator to check whether the runtime is up. It was up, and answering 401 in 88 ms.The endpoint's own comment already recognises that collapsing to an HTTP code erases the story and that the trace keeps it — but the caller has no trace, and the caller is who has to act.
Observed
On 2026-08-04 the
core-integrationrobot reportedPOST /assistant/converse → 502against a live two-cluster stack. The cause wasAGENT_RUNTIME_API_KEYdiffering between the Tracker'stracker-runtime-authsecret and the runtime's own. Only the tracker-api log carried the 401. Aligning the key turned the step green, so the diagnosis is confirmed and the mis-mapping is what cost the time.Registered P2/XS, with the fix scoped to the erasure, not the number: 502 is arguably right (the caller's own auth succeeded; the failure is upstream). What is wrong is that the body cannot tell an auth failure from an unreachable runtime.
Context
Found running
core-integrationagainst a live stack for the first time — it is excluded from the default RoboSoft runner. The same run found a real Core defect (beyondnetcode/evolith_arch32#425, the MCP chart'srunAsUser) and one defect in the robot itself (#139). One of three was the product; two were the instruments.Pre-existing, untouched, and worth flagging
The board's declared counters are stale by five rows. Before this change the header said
174 done / 17 pendingwhile the table held 179 DONE / 12 PENDING; the same offset holds after. This row follows the existing convention (total and pending both +1) rather than silently re-baselining numbers it has not reconciled row-by-row.Worth its own pass. Correction to an earlier draft of this description: this repo DOES have a
Gap registry coherencecheck, and it passes on this PR — so the guard exists; what it does not validate is the declared counters in the header. The Core repo's08-validate-trackingfails on exactly that drift.🤖 Generated with Claude Code