Repository navigation
v2: refresh model limits after a provider reload - #742
Merged
Merged
Conversation
|
#741 has the |
tobwen
force-pushed
the
fix/v2-model-limit-refresh
branch
from
October 20, 2026 14:08
537a0d2 to
9b40cc2
Compare
|
Merged, thank you @tobwen. This was a clean, well-scoped fix with a regression test that shows the bug. It will ship in the next patch release, and we'll note it on issue 741 then. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #741 (awaiting the required design-approved label; this PR is kept as a draft).
Summary
registerContextcached the first catalog read per model key inqueriedModels. A provider plugin that changes the same model'slimit.contextand callsprovider.reload()producedprovider.updated, but the event loop discarded it at thedata.sessionIDguard. The next pass skippedcontext.model.list(), so the transform and pressure math kept the old window.The new branch clears
queriedModelsonprovider.updatedandmodel.updated, before the session filter. The next pass re-runs the existing one-time read and refreshes the shared cache.Why this is safe
provider.updatedfires only when the provider snapshot changes (Provider.reload->state.reload->notify), not on every pass.refreshModelLimitsFromApipath is reused; no new cache or polling.model.updatedis also cleared, so a direct model reload takes the same path.Test
Focused and green:
bun test packages/plugin/src/v2/hooks/model-limit-refresh.test.ts- 1 pass, 0 failbun test packages/plugin/src/v2/hooks- 292 pass, 4 skip, 0 failbun run --cwd packages/plugin typecheck- passbun run build- passbunx biome check src/v2/hooks/context.ts src/v2/hooks/model-limit-refresh.test.ts- cleangit diff --check- passFull plugin suite (
bun run --cwd packages/plugin test): 8758 pass, 79 skip, 11 fail. The failures are outside this change:review bundletests need the built worker files. Afterbun run build, 5 pass; the 4packedvariants still fail becausemetadata.filesis undefined.tui-pack-graphtest fails becausesrc/tui/entry.mjsis not in the pack list.storage-dbpermission test fails because the created file mode is 0o644 instead of the expected 0o664.None of these tests import
src/v2/hooks/context.ts.Harness coverage (required)
For each entry select: covered / not applicable - surface does not exist there / NOT covered.
event-handler.ts:881-1130) and has noqueriedModelslatch.provider.updatedandmodel.updatednow clearqueriedModels; regression testmodel-limit-refresh.test.ts.ctx.model.contextWindowon every pass and has no shared catalog latch.Summary by cubic
Fixes #741: after a provider or model reload, model limits are now refreshed instead of using the stale cached catalog read. Previously the first read per model key was cached, so a provider that changed
limit.contextand calledprovider.reload()kept the old window. Nowprovider.updatedandmodel.updatedevents clear the cached set before the session filter, so the next pass re-reads the live limit.Adds a regression test that verifies the catalog is re-read and the shared limit cache updates after a provider reload.
Written for commit 9b40cc2. Summary will update on new commits.