Conversation
Running out of Claude usage stops every session on a host at once. The web UI has had the failover since XERK-246; Android had neither control, which is a gap precisely when it matters most — usage runs out while you are away from a desk, and a phone was then the one surface that could do nothing about it. Two controls, matching the web: - A "run: <source>" chip in the chat compose bar, beside model and mode, that POSTs .../sessions/<id>/model-source. The agent relaunches with --resume, so the conversation, worktree and branch carry over. - A "Run against" row in the spawn composer, so new work can START on the local model. Without it you could fail existing sessions over from the phone but not begin anything once usage was gone. Both follow the HOST's localModel.available exactly as the composer's clip button follows uploadMaxBytes: an agent reporting nothing cannot do it and the hub 409s the command, so the control is hidden rather than offered and refused. The chip is shown anyway when the session is already local, so one whose host later lost its configuration keeps a visible way back. The chip paints from a memo until the heartbeat agrees — the relaunch takes several beats, and without it the value springs back and reads as a control that did nothing. The memo retires on the heartbeat reporting the switch, and ages out at 60s so one that never lands can't pin the chip on a lie; a refusal drops it at once and says why. The Claude model picker is hidden on a local session (a static chip states the model instead), as on the web: every alias it could offer — "default" included, since that resolves to the shared login's default — is one the self-hosted endpoint refuses. Pure half in core/ModelSource.kt so the Compose screens stay thin renderers. android/PARITY.md narrows the gap to the remaining 🏠 mark on session cards.
…composer QA drove the branch and found the memo did not survive leaving the chat screen, which is the one moment it exists for. - The memo lived in ChatViewModel, which is scoped to the chat's nav back-stack entry, so walking back to the session list mid-switch destroyed it and the chip sprang back to the old value — the exact "control that did nothing" reading the memo is there to prevent. It moves to AppContainer.modelSwitches, the same place and for the same reason as the compose drafts (XERK-122). The web keeps its memo at module scope across the same navigation. - The spawn composer hid the Model row and dropped the alias for a local spawn. The web does neither: sessions.html renders and sends it whatever the source. The agent drops --model for a local session itself, and the alias is what that session returns to if it is later switched back — so discarding it gave an Android-spawned session a different model from a web-spawned one. Only the CHAT bar fixes the model, because there the picker breaks a live session. - The chip now carries the web's ☁/🏠 glyph and its "Subscription" label. The amber tint alone can't answer "which model wrote this turn" for a colour-blind reader. The tooltip is not ported (a phone has no hover, and the model name is already the chip's text) and is recorded in PARITY.md. - A memo with a blank session id is no longer honoured, so it can never paint a record-less session; the web refuses it the same way. Five mutations escaped QA's mutation run because the UI/VM half had no coverage. The decisions move into pure, tested functions — ModelSource.settle / spawnValue / composerOffers / glyph and FleetViewModel.spawnRequest — and the tests now pin the settle rule, the composer gate, the chip glyph, and the exact spawn JSON (bare, subscription and local). AgentDecodeTest gains the null modelSourceAt that hub-agent sends on every session that never moved. 312 tests, 0 failures.
…found QA re-drove the branch: the memo fix, the composer parity fix and the chip glyph all held up on device, but it found the branch no longer merged and two mutations still escaping. - **Rebased onto origin/main** (XERK-251/252 rewrote the same files). The one real conflict was `observeFleet`'s `_state.update`, where main added `tunnelOnline` and this branch added `localModel` to the same `copy(...)`. Taking either side alone silently disables a whole feature — QA ran that exact resolution as a mutation and all tests stayed green. That mapping is now `ChatUiState.fromFleet(agent, session, host)`, one place, pinned by two tests, so a future merge cannot truncate it unnoticed. - **The drop-on-refusal was the last inline decision left in the ViewModel**, so deleting it kept the suite green. It is now `ModelSource.afterAttempt`, and both outcomes are asserted: holding a memo for a full minute when the hub has already refused is the same lie the TTL exists to bound. - **The TTL assertions derived their boundary from the constant under test**, so raising it from 60s to 16.7 hours passed. Both now use literals, plus a direct assertion on the constant. - `ModelSwitchStore.kt` used a literal NUL as its key separator, which made git treat the file as binary — the file that IS the memo fix would have rendered as "Binary file not shown" in the PR. It uses the escape now; same key. - The chip's missing tooltip moved to PARITY.md's "Deliberate differences", where the file's own preamble says justified differences belong. qa.md is resolved toward main — the branch had been editing a pre-XERK-251 copy and would have dropped ~180 lines of it, including the container Android recipe that is the only way to build Android on the TrueNAS host. Both QA passes' findings are re-applied on top of main's version: this workstation as a third box in "know which box you are on", the native build + emulator-driving traps, the logging-proxy recipe, and the two pre-existing behaviours QA confirmed against main (a chat restored after process death never starts the fleet poll; one malformed host silently freezes the whole fleet view while still claiming "N / N online"). 308 tests, 0 failures — 281 on main plus 27 added here, no deletions.
…calModel
Third QA pass. The five earlier fixes all held on device, but the guard I added
for the merge-conflict finding did not do what it claimed.
- **`fromFleet`'s test asserted four of its five fields and skipped
`tunnelOnline`** — the one the conflict was actually over. QA deleted it,
rebuilt, and watched the chat header say "· live" while the hub reported the
tunnel down, with the whole suite green. It is worse than a lost warning
because the field defaults TRUE: the header then asserts the tunnel is up.
Both tests now name every field, and say to add an assert whenever a field is
added there.
- **`ModelSwitchStore` had no test**, though it is a stated copy of `DraftStore`
and `.claude/rules/android.md` names `DraftStoreTest` as the convention for
exactly this class. Returning a fresh flow per call — i.e. restoring the whole
D1 bug — kept the suite green. `ModelSwitchStoreTest` mirrors DraftStoreTest's
three cases; both mutations are now caught.
- **The fixed-model chip dropped a tooltip that carried real information.** The
web's says why the chip is inert and how to get the picker back; the run
chip's only repeats the model name. That distinction was missing from
PARITY.md, which justified dropping both with the run chip's reason. The
static chip now carries the web's wording as a contentDescription — on a
MERGING wrapper, since Text writes its own semantics last and a description in
the same modifier chain never reaches the tree — and PARITY.md separates the
two cases.
- **`normalizeLocalModel` coerces the block at hub ingest**, the remedy this
repo already documents for `limits` and per-model usage and which `localModel`
shipped without. Android decodes /api/agents ATOMICALLY into typed fields, so
one host sending `available:"yes"` or a contextTokens past Int threw for the
whole array: every other host silently vanished from the phone while the tile
still read "N / N online". Verified live — a rogue host is now coerced to
`{available:false, model:null, contextTokens:null}` and the app shows 4/4.
The generalised rule is in CLAUDE.md's heartbeat contract, since it binds the
agent, the hub and three clients.
Left alone, with reasons: the three memo call sites in ChatViewModel are still
only covered by driving the app — no test in the project constructs a
ViewModel, so closing that means new test infrastructure, not a test. The pure
rules behind them are covered. Defects 7 and 8 from the second pass are
pre-existing and recorded in qa.md §6.2 rather than fixed here.
Android 311 tests, hub 928, agent 1258 — all green.
…n rule
Fourth QA pass came back PARTIAL. Three LOW findings, all fixed here.
- **`normalizeLocalModel` cut the model name with `slice(0, 60)`**, which counts
UTF-16 units: a name whose 60th unit is the high half of an astral pair
shipped a LONE SURROGATE to every client. Cosmetic in the app (Android draws
a replacement char) but genuinely unencodable — it kills `uiautomator dump`
outright, which is the tool a QA pass drives the app with. It cuts on code
points now, and the test pins the bound both ways; nothing pinned the length
at all before, so removing the cap entirely had kept the hub suite green.
- **The spawn composer's per-host lookup was inline in both screens**, so
`firstOrNull { it.key == host }` → `firstOrNull()` passed — the "fix placed in
the wrong loop" shape this repo has shipped before, and it would offer one
host's self-hosted model when spawning on another. It is
`ModelSource.hostLocalModel(agents, host)` now, tested, and the mutation fails.
- **The heartbeat rule I added to CLAUDE.md last commit was stated absolutely
and was already false.** "Any field Android decodes into a typed one needs a
normalize*" is not satisfied by the per-SESSION fields (`modelSource`,
`modelSourceAt`, `model`, `permissionMode`), which have no coercion and freeze
the fleet view the same way if an object lands in one — reproducible on main.
It is scoped to HOST-LEVEL blocks now, with the session-level gap named as
known, and it says that a `normalize*` is a whitelist that silently drops a
sub-key a newer agent adds.
Deliberately not fixed, and stated plainly rather than implied away: five
mutations still survive in the render/wiring layer — the two memo call sites in
`ChatViewModel`, the `StaticChip` description, and the two `if (…)` guards that
render the chip and the composer row. This repo has no instrumented or Compose
test source set (`android-ci.yml` runs `testDebugUnitTest` + `assembleDebug`
only), so closing them means new test infrastructure, not a test. Every one of
those behaviours was driven on the emulator by QA and by me; none of them is
protected by a test, and the PR says so.
Android 312 tests, hub 348 in server.test.js (928 across the suite) — all green.
… say why a spawn was refused Fifth QA pass. The feature held everywhere it was driven; these are the three things it found wrong, plus a rebase. - **Rebased onto b902de8.** main restructured `qa.md` and split `qa-findings.md` out of it, so the branch's copy conflicted. Resolved toward main and the five passes' notes re-applied onto the new structure — the WSL workstation as a third box in §0, a native-build + dedicated-AVD section under §2.5 (main's container recipe untouched), the rig/logging-proxy recipe, the merge-conflict-as-mutation and derived-boundary lessons in §5.6, and the two pre-existing Android behaviours in §6.2. - **The heartbeat rule I wrote two commits ago was still false**, and QA reproduced it: `capacity: {maxSessions: "eight"}` on one host drops that host from the phone while the tile still reads "N / N online". Only THREE blocks are coerced; every other host-level block Android types is raw. The rule now says that, as a live hazard rather than a solved one, instead of implying host-level is covered. - **`normalizeLocalModel` only stopped the hub MANUFACTURING a lone surrogate**; one arriving in a rogue agent's name was passed straight through, and the comment and test both read as though it were handled. Either direction kills `uiautomator dump` — the tool a QA pass drives the app with. Input surrogates are replaced now, and the test asserts both directions. - **A spawn the hub refuses said "hub unreachable".** `FleetViewModel.run` swallowed every exception into that one string, so choosing the local model on a host that lost its configuration reported a network fault for an 8ms 409. It prefers the hub's own message now, exactly as `ChatViewModel.setModelSource` already did; "hub unreachable" is left for a genuinely unanswered request. The Sessions screen still doesn't collect `messages` at all — pre-existing, and out of this branch's scope. Residual risk, stated rather than implied: 13 mutations survive, in two clusters — nine in Composable bodies and four at ViewModel call sites. Neither is an oversight; this project has no instrumented source set and no coroutine-test harness, so closing them is new infrastructure. Every one of those behaviours was driven on the emulator, including all five failure paths of the memo drop (409/500/401/HTML/socket-drop each revert the chip within ~1s and surface the hub's own text). Android 312, hub 348 in server.test.js — green. Merge with main clean.
…s, pin the wire Sixth QA pass. It found a real hole in the coercion, a fourth strike on the CLAUDE.md bullet, and two ungated wire-contract lines. - **`normalizeLocalModel` never ran on the `state.json` restore**, which is the hole that matters most: a hub restart is exactly when a new coercion ships, and the restore is the first thing it serves. A record holding a rogue block — or belonging to an OFFLINE host, where no beat will ever rewrite it — reached the phone raw and threw for the whole fleet, for up to the record's 7-day life. The loader now applies all three (`normalizeLimits` had the same hole and this branch had copied its precedent rather than `normalizeUsage`'s), with a test that fails if a coercion is added at ingest and not there. Proven by booting a hub on a state file holding `available:"yes"`: served coerced. - **The surrogate strip closed one case, not the class.** A C0 control in the model name kills `uiautomator dump` exactly as a lone surrogate does — the very failure the strip cites as its reason — and the app renders it fine, so it is invisible until the tooling dies. Both classes are stripped now. - **`CLAUDE.md`'s coercion bullet was wrong for the fourth time**: there is a FOURTH coercion, `sanitizeHeartbeat`'s `sanitizeLiveAgents`, and it is load-bearing — Android types `session.agents` too. It no longer states a count; it names the four, says to grep rather than trust the doc, and adds the restore requirement. The "every OTHER host vanishes" claim was also imprecise: that is the poll path, while per-agent SSE events decode individually, so with SSE healthy only the bad host is missing. - **The switch's own wire contract was ungated.** Renaming the `@POST` path or `ModelSourceRequest`'s field shipped green with the feature dead on the wire (404, or a 400 from the hub's enum check) — and unlike the render-layer survivors this needed no new infrastructure, since the sibling `SpawnRequest.modelSource` was already pinned. Both mutations now fail. - `qa.md`: the sixth pass's corrections, plus the two bullets this commit made stale (the restore hole and the surrogate-only claim) rewritten to match. Residual risk, corrected again: 14 mutations survive, in three locations — Composable bodies (8), ViewModel call sites (4), and `AppContainer`'s `modelSwitches` accessor. The two `HubApi` escapes are closed above. Every one of those behaviours was driven on the emulator; none is protected by a test, because the project has no instrumented source set and no coroutine-test harness. Android 313, hub 349 in server.test.js — green. No literal control bytes in either file (a NUL or a C0 makes git treat the source as binary).
…ranch typed
Seventh QA pass. Two of these are holes in my own previous fix.
- **The restore was throwing into its own `catch {}`.** `sanitizeLiveAgents`
reads two module `const`s declared ~1700 lines BELOW the restore, so at module
init they are in their temporal dead zone: the loop coerced `localModel`, hit
a ReferenceError on the session half, and the whole thing was swallowed —
half-coerced record, no log line, every suite green. The constants moved above
the restore with a comment saying why they can't live beside their function,
and a test asserts that ordering. This is the second time this branch has
shipped a coercion that didn't reach the restore; both were invisible.
- **The restore missed the fourth coercion**, `sanitizeLiveAgents`, while the
comment beside it claimed "every coercion the ingest path applies". Both paths
now go through one `normalizeRecord`, so adding a coercion covers both, and
the test asserts the two call sites rather than enumerating names — the old
test listed three and therefore could not notice the fourth missing, which is
exactly the shape that let the hole exist.
- **This branch made two session fields decode-fatal and coerced neither.**
Typing `modelSource`/`modelSourceAt` on `SessionInfo` is what did it: before
that, `ignoreUnknownKeys` skipped them and any value was harmless; after, an
object in either throws for the whole `/api/agents` array and every other host
vanishes from the phone. Measured at 7 of 12 hosts shown, no error. They are
coerced in `normalizeSessions` now, and CLAUDE.md states the general rule:
typing a field on a client and adding its hub-side coercion are one change.
- **The XML-illegal strip missed U+FFFE/U+FFFF**, the last two code points
excluded by XML 1.0's Char production, while the code and test both claimed
the whole class. They kill `uiautomator dump` exactly as a C0 does. Added,
with U+FDD0 and U+1FFFE asserted to SURVIVE so the strip doesn't overreach.
- CLAUDE.md's cold-start parenthetical was backwards (SSE carries the good hosts;
it is the poll-only case that loses the screen), and `qa-findings.md` still
claimed the split kept `qa.md` under a ceiling that does not apply to it.
Residual risk, with the seventh pass's measured numbers rather than my
arithmetic: **28 of 58 Android mutations survive**, in three locations —
Composable bodies (17), ViewModel call sites (10), and one `@Serializable` field
default. The hub half is 0 of 12. Nothing in `core/` survives. Those are all
render/wiring sites with no gate in this project (no instrumented source set, no
coroutine-test harness); each behaviour was driven on the emulator instead.
Android 313, hub 351, node 1031, agent python 1258 — green.
…-killing the hub
Eighth QA pass. The headline finding is in my own coercion, and it is the
worst thing this branch has produced.
- **`normalizeLocalModel` spread an agent-controlled string per code point
BEFORE bounding it**, and it runs BEFORE the `AGENT_RECORD_MAX` check that
exists to refuse an oversized beat. One agent-authed heartbeat with a 24 MiB
`localModel.model` OOM-killed the hub at its deployed `mem_limit: 256m` on
`node:24-alpine`, and `restart: unless-stopped` makes that a repeatable
outage loop of the fleet's whole control plane — every host's sessions,
terminals and in-flight migrations. Reproduced and A/B'd in a container at the
deployed shape: without the bound, `exit=137 oom=true` and the hub gone; with
it, 200 and still serving. The name is `slice(0, 512)`d first (far more than
the 60 code points that survive; a split astral pair is handled by the
surrogate replace immediately after), and a test holds it as a time+memory
budget rather than by reading the source.
- **`sessions: [null]` hid a host** — `normalizeSessions` skipped a non-object
element instead of dropping it, though `sessions` is typed `List<SessionInfo>`
on Android. Measured as a host silently absent while the tile still counted
it, and the whole screen replaced by the raw decoder exception with SSE also
down. Dropped now.
- A **non-array** `sessions` is deliberately still left alone: my first attempt
rewrote it to `[]`, which erased the amplifier `AGENT_RECORD_MAX` exists to
refuse — an 8 MiB string `sessions` went from 413 to 200 and the existing
XERK-235 test caught me. The rule, now written down: a coercion running
before that check may only ever SHRINK a record.
- **The TDZ test named two constants**, so a third would walk straight past it —
QA proved that with a new late-declared const that kept the suite green while
the restore silently threw again. It is behavioural now: load the real module
in a child process against a fixture holding every wrong shape at once, and
assert both the `loaded N agents` line and the coerced result.
- **A refused spawn from the Sessions screen was silent** — that pane collected
`vm.messages` from nothing, which was survivable while its routes only failed
on the network, but this branch gives it a first-class 409. It has a
SnackbarHost now; verified on device (`✗ hub unreachable` where there was
previously no feedback at all, and the hub's own text on a real refusal).
- Two `qa.md` statements my own last commit had falsified, corrected again.
Residual risk is unchanged and disclosed in the PR: no real agent/claude/gateway
in the loop, and 28 of 58 Android mutations survive in the render/wiring layer
(17 Composable bodies, 10 ViewModel call sites, 1 `@Serializable` default) —
nothing in `core/`, `data/` or `net/`, and the hub half is fully gated.
Android 313, hub 352, node 1032, agent python 1258 — green. Merge clean.
…stic
Ninth QA pass. The ordering insight in its second finding resolves three
things at once.
- **`normalizeRecord` now runs PAST the `AGENT_RECORD_MAX` gate**, not before
it. Before was wrong in both directions: a coercion there shrinks away the
amplifier the gate exists to refuse (my own attempt to rewrite a non-array
`sessions` turned an 8 MiB string from 413 into 200, caught by XERK-235's own
test), and it walks an oversized record field by field before throwing it out
— which is how a 24 MiB name reached a spread in the first place. Past the
gate the record is bounded, so a coercion is free to REWRITE. `sanitizeHeartbeat`
stays pre-gate and keeps only the shrink-only half.
That unlocks two fixes QA had filed as forced trade-offs:
- A non-array `sessions` is rewritten to `[]`. It doesn't merely hide that host
— measured, the app **cannot sign in at all**, because the login probe
decodes `/api/agents` and the throw reads as "Could not reach the hub".
- `sessions:{a:1}` in a `state.json` no longer aborts the whole restore.
`normalizeUsage` iterates `sessions || []`, so a non-iterable threw into the
restore's silent `catch {}`; `normalizeSessions` runs first now.
- **The OOM regression guard was nondeterministic** — 8 MiB against 50ms/64MB
caught the reintroduced bug only 5 runs in 8, decided by whether a GC landed
between two samples. A resource budget can't be made reliable at a 1.02x
margin, so it is now a structural assertion (the spread must be preceded by a
slice) PLUS a budget at 32 MiB with ~10x headroom. Measured: 6/6 catches with
the bug reintroduced, 4/4 clean.
- **The spawn composer kept a hidden `local` choice** after the row that shows
it disappeared, so a host losing its configuration mid-composer produced a
guaranteed 409 with nothing on screen to explain or change. It resets.
- Three more `qa.md` statements my own commits had falsified.
Residual risk, unchanged and disclosed in the PR: no real agent/claude/gateway
in the loop, and 30 render/wiring mutations with no gate (19 Composable bodies,
10 ViewModel call sites, 1 @serializable default) — nothing in core/, data/ or
net/, and the hub battery is now 12/12 caught.
Android 313, node 1068, agent python 1258 — green. Merge clean. Memo re-driven
on device after the rebuild: ☁ → 🏠 → survives HOME → survives leaving the chat
→ ☁ at t+65s.
Tenth QA pass. My previous commit's "coerce past the gate" was wrong in the
other direction, and it produced two HIGH regressions against main. Both
demonstrated end to end, both now closed and mutation-tested.
- **An expanding coercion escaped the ceiling.** `normalizeModelUsage` rewrites
`"m"` to `{model:"m"}` — ~3.5x — so measuring only the pre-coercion size let
an 8 MiB beat of bare model names park 28 MiB per host, for the record's whole
7-day life, in `state.json`, in every `/api/agents` response and every SSE
frame. Exactly the amplification XERK-235 added the ceiling to stop. Measured:
branch 200 and a 29 MB payload, main 413 and 139 bytes.
- **A throw inside the coercion left the RAW record installed**, because it now
ran after `agents[key] = next`. `normalizeUsage`'s bare `for (… of repoUsage
|| [])` throws on an object; the hub answered 400 and served the host anyway.
Worse than refusing: uncoerced `localModel.available:"yes"` is truthy, so
`localModelAvailable` handed out the switch a host cannot honour, and the
poison reached `state.json`, where the restore then aborted into its silent
`catch {}` and left every host after it uncoerced on every boot.
The fix is neither before nor after but BETWEEN: measure the raw size (the
amplifier check a shrinking coercion must not defeat), coerce, then measure the
stored size (which an expanding coercion must not escape). Coercion sits in the
middle, so it never walks an unbounded record — the property that kept the OOM
fix honest — and a throw is caught and rolls the record back, so "accepted" can
never mean "accepted and raw".
Also: `normalizeUsage` guards its iterations with `Array.isArray` instead of
`|| []`, and REWRITES a non-array `repoUsage` to `[]` rather than stepping
around it — it is `List<RepoUsage>` on Android, so serving `{a:1}` is
fleet-fatal for the phone. Guarding the loop alone would have turned main's
accidental 400 into a 200 serving that shape.
Five mutations now caught that were not: coerce-before-the-raw-check, drop the
coerced check, unguarded `repoUsage`, drop the non-array `sessions` rewrite, and
the ingest ordering itself. The order INSIDE `normalizeRecord` is deliberately
no longer load-bearing — each coercion guards its own input shape — and the
comment says so rather than justifying an order that no longer matters.
Two more `qa.md` statements corrected (the dropdown opens anywhere in the field,
not only its caret; a dump costs ~2s, not 30-60s — both measured this pass).
Residual risk unchanged: no real agent/claude/gateway in the loop, and the
render/wiring mutations with no gate in this project.
Android 313, node 1071, agent python 1258 — green. Merge clean.
Eleventh QA pass — the first run by a session other than the one that wrote this branch, against the pushed tip. Sixteen of its seventeen claims held, several with measurements. These are the three that did not, two of them inside this branch's own fixes. - **An ARRAY element in `sessions` survived the new filter and blocked sign-in entirely.** `normalizeSessions`' predicate was `!s || typeof s !== "object"`, and `typeof [] === "object"` — so the element it was written to drop was the one shape it passed through. The comment above it named `null` and a bare string; the fixture used `null` and a bare string; the third non-object shape was in neither, so the fix read as complete from every angle except running it. Not merely a hidden host: the login probe decodes `/api/agents`, so the throw reads as "Could not reach the hub" and the app cannot sign in at all. Predicate is one `objectish` helper now, the fixture carries every non-object shape (`null`, string, `[1,2]`, `[]`) and asserts the surviving ids rather than a count of drops. - **The switch memo's TTL was a timer that never fired.** It aged out inside `canSwitchModelSource()`, read from `System.currentTimeMillis()` at composition time, and Compose skips recomposition while the state compares equal — so on a quiet fleet nothing re-read the clock. Reproduced twice: switching away from `local` on a host that had lost its `localModel` removed the whole "run against" chip, still gone at t+120s, with the bar reading `model: default` while the session was in fact still on the self-hosted model and no way on screen to retry or see the truth. Expiry now RETIRES the memo from the store — a state change, which is what repaints — via `settle(…, now)` plus a bounded per-memo alarm armed off the store, so a memo carried in from another nav entry is covered too. This was also a parity break: the web recomputes `localModelOffered()` unconditionally in `onPoll` every beat. - **`setModel`/`setMode` reported every refusal as "✓ model queued"**, having discarded their `Result`. Survivable while those routes only failed on the network — and then this branch's own hub commit gave `/model` a first-class 409 for a session on the self-hosted model, added precisely so an out-of-parity client could not silently drop the command. Reachable through the memo state above. Both now report the hub's own words, exactly as `setModelSource` and `FleetViewModel.run` already do. All three are mutation-checked, and the two Android ones went to `core/` rather than staying at their call site for the reason this branch already extracted `afterAttempt`: a ViewModel call site has no gate in this project, so a decision left inline is one a mutation deletes unnoticed. Reverting each fix fails its test — `Array.isArray` dropped: 1 failure; the expiry drop deleted: 1 failure; `outcomeMessage` forced to its success arm: 1 failure. `qa-findings.md` §5.8 records the three shapes, since two are §5.3 and §5.4 recurring one round later inside the code that fixed them. Android 315, node 1071 — green. Instruction files under the 40k cap. Merges clean. Not addressed here, ticketed instead: the ungated render/wiring mutations and the unreachable rollback backstop (XERK-262), the hub OOM at two concurrent large heartbeats (XERK-258, pre-existing), uncoerced `repoUsage` elements (XERK-259, pre-existing), and the unbounded command queue (XERK-261).
Resolves one conflict, in `qa-findings.md`: main added its own §5.8 and §5.9 (XERK-254's thirteen rounds) while this branch appended a §5.8 of its own. Both are kept — main's keep their numbers, this branch's becomes §5.10 — since they record different rounds and neither supersedes the other. Everything else merged clean. Main's new `QA.md` is a separate file from this branch's `qa.md`, not a rename of it. Merging rather than rebasing: the eleven commits below carry ten QA passes' evidence in their messages, and rewriting them would detach that from the code it describes. Merged because CI could not run at all while the branch was conflicted — a `pull_request` workflow builds the merge commit, and GitHub cannot create one for a DIRTY branch, so zero checks were queued on 822ecf9. Suites on the merged tree: node 1073, agent python 1277, Android 315 — green. Instruction files all under the 40k cap.
Twelfth QA pass, on the merged tip. It confirmed all three of the eleventh
round's fixes with fresh evidence — including a decisive isolation for the memo
alarm, driven with the fleet's heartbeats fully PAUSED, so the control returning
at t+58-63s is the alarm doing the work and not fleet churn. Every finding below
is a defect in one of those fixes.
- **The `typeof [] === "object"` fix was one level too shallow.** The identical
predicate sat FIVE LINES below the one I fixed, on `s.session` — and
`"agents" in []` is false too, so an array fell through both halves of that
guard. One heartbeat of `sessions:[{id:"s1",session:[]}]` reproduced the
original symptom exactly: served raw, and the app could not SIGN IN, because
the login probe decodes /api/agents and reads the throw as "Could not reach
the hub". The predicate is now one hoisted `objectish()` helper used at all
three sites, so there is nothing left to grep — a `function` declaration
rather than a `const`, because `sanitizeHeartbeat` calls it ~70 lines above
its definition and this file has already shipped a coercion that died of
exactly that TDZ.
I audited the remaining `typeof x === "object"` predicates rather than
assuming: `normalizeLimits` and `sanitizeLiveAgents` both have it, and both
are SAFE, because each rebuilds a whitelist from scratch — an array falls out
at the next field read instead of reaching a client. Recorded, so the next
pass does not re-derive it.
- **My first version of that fix did not actually fix it**, and the test is what
caught me: guarding the SANITIZE with the right predicate leaves the raw array
in the record, which is the thing that gets served. A non-object `session` is
REWRITTEN to null now — the "can't tell you" value every client already reads
— which is legal here only because normalizeSessions runs past the record gate.
- **The memo alarm could sleep through a backward wall-clock jump.** It derived
its delay from `currentTimeMillis`, slept on `delay` (uptime), then re-read the
wall clock; a backward jump made the re-check false, `settle` returned the SAME
instance, `MutableStateFlow` did not emit an equal value, the collector never
ran, and no new alarm was armed. Measured at t+190s after a 10-minute jump,
chip claiming the subscription while the record said local. It retires by
IDENTITY now — `compareAndSet(pending, null)` — which cannot be undone by a
clock, and no-ops if a newer switch or a settle already replaced the memo.
- **`reportedOutcome` branched on the HTTP status alone**, ignoring
`OkResponse.error`, so a `200 {ok:false,error:…}` would read "✓ model queued".
Latent, not live — no route answers that shape for /model or /mode today — but
that is precisely the position /model was in before it grew its 409, which is
how the bug this replaces was born. `FleetViewModel.run` already checked it;
the two sides agree now.
- The TTL test asserted 60_999 and 61_001 and never 61_000, so `>=` → `>`
survived a mutation battery. The exact edge is asserted.
Mutation-checked, five new: deleting the session rewrite, dropping
`Array.isArray` from `objectish`, weakening the rewrite to a falsy test,
`expired`'s `>=` → `>`, and ignoring `bodyError` — each fails its test.
**Honestly ungated:** the `compareAndSet` change itself is a ViewModel call
site, so nothing in the suite can catch its deletion. That is XERK-262 (11 of 25
mutations escape in vm/ and ui/, no coroutine-test harness exists), not
something this commit can close without building that harness.
qa-findings.md §5.11 records the four shapes.
Suites: node 1073, agent python 1277, Android 316 — green.
The pure half of the Android client was fully gated and the call sites were not, so 11 of a 25-mutation battery escaped: the model-source memo rules could be deleted from ChatViewModel, `canSwitchModelSource` could be forced false (hiding the whole failover feature), `composerOffers` and `modelPickable` could be forced true, two effects could be deleted outright, a spawn site could be handed `localModel = null`, and `LocalModelInfo.available` could default to `true` — inverting the capability contract fleet-wide — all with a green suite. Pushing those decisions further down into `core/` cannot close this. They are already in `core/` and already tested; what had no gate is the wiring. Test infrastructure, in the `test/` source set: - Robolectric + compose-ui-test, so `createComposeRule` and a real Application run under `testDebugUnitTest` — the only Android gate CI runs. androidTest would need an emulator job this repo doesn't have, so tests there would gate nothing, which is the failure being fixed. - `harness/MainDispatcherRule` for `viewModelScope`, and `harness/HubHarness`, which points Config at a MockWebServer after startup (HubClient rebuilds its Retrofit on a base-URL change, so no production seam is needed) and drives the fleet by awaiting `FleetRepository.refresh()` rather than the 6s poll. - Tests use the real Retrofit/OkHttp stack: the refusals being covered are a 409 with a JSON error body, which is exactly what `hubErrorMessage` reads. All 11 escapes are caught; each was re-run as a mutation against the whole suite to prove it fails. Three further mutations are covered as a side effect (ChatViewModel's own hubErrorMessage, the Sessions spawn site's localModel, and rewiring `hostLocalModel` to the fleet's first host). Also in scope: the hub's coercion backstop. That try/catch could be deleted with the node suite green. It is unreachable from the heartbeat wire — every `normalize*` guards its input shape, and the one coercion that could throw is walked by `sanitizeHeartbeat` before any record is installed — so it is forced through a `recordCoercion` holder exported under TURMA_TEST. Unreachable is precisely why deleting it reads as safe: `agents[key] = next` has already run, so a throw would leave the raw record installed and served. Investigating it turned up a real hole, filed and fixed separately as XERK-278 (#438): the same value reaches `sanitizeLiveAgents` from the `/agent/control` WebSocket frame handler, where nothing catches it and the process dies. Nothing in this PR claims that path is safe. Two rounds of adversarial QA. The second found four defects in the first, all fixed here: the Sessions spawn test seeded one host so the wrong-loop shape still escaped; `the memo survives a heartbeat that has not caught up` was vacuous because FleetState is a data class in a StateFlow and a byte-identical re-seed emits nothing; MockWebServer's port-0 bind runs with SO_REUSEADDR off and flaked 2 runs in 10; and one absence assertion passed against a dialog that rendered nothing. CI: android-ci now caches Robolectric's ~190MB android-all jar, which this change is what makes it fetch. Tests pin `@Config(sdk = [35])` to match compileSdk so only one such jar is ever needed. Verified: 344 Android unit tests and 1077 node tests green; 16 mutations applied and caught; 8 consecutive full Android runs under CPU load with no flake.
main moved to v1.0.0 while this branch was open. Three textual conflicts and two
semantic ones; every side's intent is kept.
- CLAUDE.md — both sides added bullets under the heartbeat wire contract. Kept
both: this branch's atomic-decode/whitelist rules, and main's XERK-264 rule
that a hub refusal must reach the operator in the hub's own words.
- ChatViewModel — main (XERK-264) routed setModel/setMode through a generic
`report(ok){}`; this branch routes them through `reportedOutcome`, which does
the same thing AND reads the `{ok:false,error}` a hub can answer 200 with,
which `report` discards (its block returns Unit). Kept `reportedOutcome` — a
strict superset, so main's intent survives — and main's `report` stays for the
calls that use it.
- FleetViewModel — same line touched twice: this branch added the rationale for
preferring the hub's own reason, main added the optimistic-pending rollback on
a refusal. Kept both.
- SessionsScreen — both branches independently gave SessionsListPane a snackbar,
so the textual merge produced two of everything: duplicate imports, two
`val snackbar`, two `SnackbarHost` renders. Deduped to one, folding both
rationales into the surviving comment. The two differed in where the `Box`
closes — main closes it before the dialogs, this branch wrapped them — which
left the second render outside `BoxScope` and failed to compile. Main's shape
wins: a dialog is a popup, so its place in the layout tree is immaterial.
- FleetViewModel had a duplicated `hubErrorMessage` import from the same merge.
Verified after the merge: 351 Android unit tests and 1099 node tests green, and
the 8 mutations that land in merged files were re-applied and are still caught.
xerhab
added a commit
that referenced
this pull request
Aug 12, 2026
XERK-262 (#439) landed on main and touches the same two files, so both conflicts are with that work. Nothing from either side is dropped. - turma/server.js — main hoisted LIVE_AGENTS_MAX / LIVE_AGENT_FIELD_MAX to the top of the file while this branch added `safeString` next to their old position. Kept main's placement and only the new function; taking this side wholesale would have declared both constants twice. - turma/tests/server.test.js — a pure append conflict: both branches added tests at the end. Kept both, main's first. (The first splice truncated main's last test, whose closing brace sat past the conflict marker; restored.) One semantic interaction, deliberate on both sides. XERK-262's `heartbeat: a live-agent field with no primitive conversion never reaches the record` accepts EITHER a 200 or a 400 and asserts only the invariant, precisely because this fix was known to be coming: with the coercion no longer able to throw, that beat is now an ordinary 200 with the poisoned row dropped rather than a refusal. It still passes. This branch's ingest test pins the post-fix contract exactly, and now says so and cross-references the other. Verified after the merge: 1112 node tests green; both halves of the fix reverted independently still redden the suite; the original one-frame repro still leaves the hub answering.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes XERK-262.
The gap
The ticket's mutation battery at #430's tip:
core/10/10 caught,data/1/1, butmodel/1/2,vm/2/6,ui/0/6. The pure half is fully gated; the call sites are not.Pushing the decisions further down into
core/cannot close this. They are already incore/and already tested —hostLocalModel,composerOffers,modelPickable,afterAttempt,settleall have passing unit tests. What has no gate is the wiring:canSwitchModelSource = false,localModel = null, a deletedLaunchedEffect. Only a test that renders the screen and reads what is on it fails on those.So:
ui/gets Compose tests. I asked which harness you wanted; the question timed out, so I picked and am flagging it here to overrule.The harness
In the
test/source set, notandroidTest:createComposeRuleand a realApplicationrun undertestDebugUnitTest— the only Android gate CI runs.androidTestwould need an emulator job this repo doesn't have, so tests there would gate nothing, which is the exact failure being fixed.harness/MainDispatcherRule—UnconfinedTestDispatcherforviewModelScope, so assertions need noadvanceUntilIdle()(a forgotten one is a test asserting on state nothing wrote yet).harness/HubHarness— pointsConfigat aMockWebServerafter startup.HubClientrebuilds its Retrofit on a base-URL change, so no production seam was needed. Drives the fleet by awaitingFleetRepository.refresh()directly rather than the 6s poll, and fails loud on a fixture that doesn't decode (/api/agentsdecodes atomically andrefresh()swallows the throw into an empty fleet — that cost me an hour).errorbody, which is exactly whathubErrorMessagereads; a fakeHubApiwould prove only that the fake was called.Rules that a reasonable change would otherwise undo are in
.claude/rules/android.md— notably that@Config(qualifiers = …)on a Compose test puts Robolectric into a composition loop that fails the whole class after a 60s timeout.Results
All 11 escapes caught. Each was applied as a mutation against the whole suite and confirmed to fail:
ChatViewModelafterAttemptdeletedChatModelSourceTest.a refused switch drops the memoChatViewModelsettledeleted…the memo retires once the heartbeat agreesChatViewModelPendingdeleted…an accepted switch arms the memo and keeps itFleetViewModel:70hubErrorMessagecollapsedFleetOutcomeTest.a refused spawn reports the reason the hub gaveChatScreen:273canSwitchModelSource = falseChatModelChipsTest.the run chip shows…ChatScreen:613modelPickableforced true…a local session states its fixed model…FleetDialogs:75reset effect deletedSpawnComposerTest.losing the host's local model resets…FleetDialogs:113composerOffersforced trueSessionsScreen:523snackbar collector deletedSessionsPaneSpawnTest.a refused spawn is shown to the operatorFleetScreen:158localModel = nullFleetSpawnLocalModelTest.the composer offers the row…Models.kt:151availabledefault →trueAgentDecodeTest.an empty localModel block is not availableThree more are covered as a side effect:
ChatViewModel's ownhubErrorMessage, and rewiring either spawn site'shostLocalModelto the fleet's first host.Models.ktwas the cheap one and the dangerous one: the existing tests all sendavailableexplicitly, so the default — the whole contract for alocalModel: {}block — was ungated. Flipped, a host reporting{}reads fleet-wide as "can fail over".The hub's coercion backstop
Also in scope. That
try/catchcould be deleted with the node suite green.It is genuinely unreachable from the heartbeat wire: every
normalize*guards its own input shape, and the one coercion that could throw is walked bysanitizeHeartbeatbefore any record is installed. So it is forced through arecordCoercionholder exported underTURMA_TEST— a property read in production, nothing more.Unreachable is precisely why deleting it reads as safe. It isn't:
agents[key] = nexthas already run by the time the coercion is called, so a throw would leave the raw, uncoerced record installed and served.QA
Two adversarial rounds. The second returned FAIL and found four defects in the first, all fixed here:
the memo survives a heartbeat that has not caught upwas vacuous:FleetStateis adata classin aStateFlow, so a byte-identical re-seed emits nothing and the collector wheresettlelives never ran. It passed against asettlethat cleared unconditionally. The fixture now variesnow, and the trap is documented onfleetJson.MockWebServer.start()usessetReuseAddress(port != 0)— SO_REUSEADDR off on port 0 — so aTIME_WAITport failed the bind: 2 failures in 10 loaded runs. Now retried. 0 failures in 8 consecutive loaded runs since.SpawnDialogthat rendered nothing; it has a positive anchor now.Also from QA, not acted on here:
api.github.comcalls per run (the in-app updater fires fromApplication.onCreate, and Robolectric builds a fresh Application per test). Verified harmless — with egress blackholed the suite is 344/344 green in 11s, because the check is fire-and-forget and nothing joins it. Filed as XERK-281 rather than fixed, since suppressing it cleanly needs a production seam.ui-test-manifestmerges an exportedComponentActivityinto the debug manifest. Release builds are unaffected, andPreviewActivitywas already there the same way.Verification
assembleDebugbuilds.