Skip to content

fix(chat): stop the context-usage percentage flickering during streaming - #122

Merged
setkyar merged 1 commit into
mainfrom
fix/context-usage-flicker
Sep 17, 2026
Merged

setkyar merged 1 commit into
mainfrom
fix/context-usage-flicker

Conversation

@setkyar

@setkyar setkyar commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Fixes #110.

Problem

The context-usage capsule derives its snapshot from the last assistant message carrying a usage object. During a streamed turn a half-written assistant entry lands with an empty usage object, so contextTokens fell to 0 — the percentage jumped to 0% (or the capsule hid entirely) until the next reload put a real value back.

Changes

  • collectContextUsage skips assistant messages whose usage totals zero when scanning backwards, staying on the last message that actually measured the context.
  • updateContextUsage takes the snapshot it last rendered (previous) and returns the new one. A transient zero reuses the previous value, so once a value is on screen the capsule never flickers to 0% or hides. An empty model label likewise reuses the previous limit instead of falling back to the 128k default.
  • createContextUsageController holds the first paint until /api/models settles — and paints anyway if that fetch fails — so the denominator does not visibly jump from the heuristic fallback to the registry value.

Tests

context-usage.test.js gains regression coverage: incremental streaming entries without usage don't move or hide the indicator; the limit survives a momentarily unknown model label; the first paint waits for the registry; and it still paints when the registry fetch rejects.

make check passes.

Known limitations

  • The registry is pinned before the first paint, but the model label is not. If /api/models settles before worker status reports the model, the first paint uses the 128k default and adjusts once when the label arrives.
  • Non-zero regressions are still possible if pi writes an intermediate assistant message with a small-but-positive usage. Monotonicity is deliberately not enforced — compaction legitimately drops the context.

The indicator picked its context snapshot from the last assistant message
carrying a `usage` object. A half-written streaming entry lands with an
empty usage object, so the percentage dropped to 0% (or the capsule hid)
until the next reload replaced it.

- collectContextUsage now skips assistant messages whose usage totals
  zero, staying on the last message that actually measured the context.
- updateContextUsage takes the snapshot it last rendered and returns the
  new one: a transient zero reuses the previous value, so once a value is
  on screen the capsule never flickers to 0% or hides. An empty model
  label likewise reuses the previous limit instead of the 128k default.
- The controller holds the first paint until /api/models settles (and
  paints anyway if that fetch fails) so the denominator does not jump
  from the heuristic fallback to the registry value.

Known limitation: the model label is not pinned the same way, so the
first paint can still adjust once when worker status reports the model.
Non-zero regressions are left alone on purpose — compaction legitimately
drops the context.

Fixes #110
@setkyar
setkyar merged commit 90d9106 into main Sep 17, 2026
6 checks passed
@setkyar
setkyar deleted the fix/context-usage-flicker branch September 17, 2026 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Context-window usage percentage fluctuates or flickers during streaming

1 participant