Skip to content

v2: refresh model limits after a provider reload - #742

Merged
ualtinok merged 1 commit into
cortexkit:masterfrom
tobwen:fix/v2-model-limit-refresh
Oct 20, 2026
Merged

ualtinok merged 1 commit into
cortexkit:masterfrom
tobwen:fix/v2-model-limit-refresh

Conversation

@tobwen

@tobwen tobwen commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Refs #741 (awaiting the required design-approved label; this PR is kept as a draft).

Summary

registerContext cached the first catalog read per model key in queriedModels. A provider plugin that changes the same model's limit.context and calls provider.reload() produced provider.updated, but the event loop discarded it at the data.sessionID guard. The next pass skipped context.model.list(), so the transform and pressure math kept the old window.

The new branch clears queriedModels on provider.updated and model.updated, before the session filter. The next pass re-runs the existing one-time read and refreshes the shared cache.

Why this is safe

  • provider.updated fires only when the provider snapshot changes (Provider.reload -> state.reload -> notify), not on every pass.
  • The existing read and refreshModelLimitsFromApi path is reused; no new cache or polling.
  • model.updated is also cleared, so a direct model reload takes the same path.

Test

Focused and green:

  • bun test packages/plugin/src/v2/hooks/model-limit-refresh.test.ts - 1 pass, 0 fail
  • bun test packages/plugin/src/v2/hooks - 292 pass, 4 skip, 0 fail
  • bun run --cwd packages/plugin typecheck - pass
  • bun run build - pass
  • bunx biome check src/v2/hooks/context.ts src/v2/hooks/model-limit-refresh.test.ts - clean
  • git diff --check - pass

Full plugin suite (bun run --cwd packages/plugin test): 8758 pass, 79 skip, 11 fail. The failures are outside this change:

  • 9 review bundle tests need the built worker files. After bun run build, 5 pass; the 4 packed variants still fail because metadata.files is undefined.
  • 1 tui-pack-graph test fails because src/tui/entry.mjs is not in the pack list.
  • 1 storage-db permission test fails because the created file mode is 0o644 instead of the expected 0o664.

None of these tests import src/v2/hooks/context.ts.

Harness coverage (required)

For each entry select: covered / not applicable - surface does not exist there / NOT covered.

  • OpenCode 1: not applicable - the v1 adapter refreshes limits after recording usage (event-handler.ts:881-1130) and has no queriedModels latch.
  • OpenCode 2: covered - provider.updated and model.updated now clear queriedModels; regression test model-limit-refresh.test.ts.
  • Pi: not applicable - Pi reads ctx.model.contextWindow on every pass and has no shared catalog latch.
  • OMP: not applicable - same Pi handler.
  • Rust module: not applicable - it does not own the TypeScript model-limit cache.

Summary by cubic

Fixes #741: after a provider or model reload, model limits are now refreshed instead of using the stale cached catalog read. Previously the first read per model key was cached, so a provider that changed limit.context and called provider.reload() kept the old window. Now provider.updated and model.updated events clear the cached set before the session filter, so the next pass re-reads the live limit.

Adds a regression test that verifies the catalog is re-read and the shared limit cache updates after a provider reload.

Written for commit 9b40cc2. Summary will update on new commits.

View guided diff

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

#741 has the design-approved label; this PR can be reviewed.

@tobwen
tobwen force-pushed the fix/v2-model-limit-refresh branch from 537a0d2 to 9b40cc2 Compare October 20, 2026 14:08
@ualtinok
ualtinok merged commit 7054385 into cortexkit:master Oct 20, 2026
20 of 24 checks passed
@magic-alfonso

magic-alfonso Bot commented Oct 20, 2026

Copy link
Copy Markdown

Merged, thank you @tobwen. This was a clean, well-scoped fix with a regression test that shows the bug. It will ship in the next patch release, and we'll note it on issue 741 then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenCode 2 keeps a stale model context limit after a provider reload changes it in place

2 participants