Skip to content

feat(ext): budget routed compaction off the served model's context window - #970

Open
Symbiomancer wants to merge 3 commits into
mainfrom
evanusa/pi-compaction-served-context
Open

feat(ext): budget routed compaction off the served model's context window#970
Symbiomancer wants to merge 3 commits into
mainfrom
evanusa/pi-compaction-served-context

Conversation

@Symbiomancer

Copy link
Copy Markdown
Contributor

Summary

Pi 0.74 budgets automatic compaction off the requested model's static context window (installer default 200000), but the router routinely serves a different model — often one with a much larger context. The server side of this fix landed in #968 (x-router-context-window header, emitted with the served model's effective window at every routed response site, re-stamped fresh on semantic-cache hits). This PR is the client half: the pi extension now consumes that header.

Before: a session requesting claude-sonnet-4-6 compacted at ~200K even while a 1M-context model served it.

After: the extension tracks the served window per response and uses it as the compaction budget at agent_end, falling back to the requested model's static window when the header is absent.

Changes

File Change
src/config.ts add ROUTED_CONTEXT_WINDOW_HEADER constant
src/compaction.ts capture servedContextWindow from successful responses (ignores non-2xx, malformed values); reset per run; use it in the agent_end budget
test/compaction.test.ts harness now parameterizes the requested window; 3 new tests (large served window → no pre-compaction; small served window → mid-loop compaction; absent header → requested budget)
test/e2e.sh unit-suite pass count 22 → 25

Behavior

  • Served window > requested: no premature compaction (the reported bug).
  • Served window < requested: still compacts early enough (authoritative in both directions).
  • Header absent (e.g. older router): unchanged behavior via the requested-model fallback.

Validated with tsx --test (same class of TS loader pi uses): 25/25 unit tests pass.

…ndow

Pi budgets auto-compaction off the model the client requested, but the router
routinely serves a different (often larger-context) model. Without the served
window, a session requesting claude-sonnet-4-6 compacted at ~200K even while a
1M-context model served it.

Follow the router's x-router-context-window header (emitted by #968): capture
it per successful response and use it as the compaction budget at agent_end,
falling back to the requested model's static window when absent. The served
window is authoritative in both directions - a larger served model avoids
premature compaction; a smaller one still compacts early enough.

- config: add ROUTED_CONTEXT_WINDOW_HEADER
- compaction: track servedContextWindow per run (reset on start/compact) and
  apply it in the agent_end budget; ignore non-2xx responses and malformed values
- tests: 3 new cases (larger served window no pre-compaction; smaller served
  window mid-loop compaction; absent header keeps requested budget); the
  harness now parameterizes the requested model window
- e2e.sh: unit-suite pass count 22 -> 25

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 162c734. Configure here.

Comment thread install/pi-router/src/compaction.ts
@greptile-apps

greptile-apps Bot commented Aug 20, 2026

Copy link
Copy Markdown

T-Rex T-Rex Logs

What T-Rex did

  • T-Rex produced two finding-comment proofs for the posted P1 findings and linked them to the review comments for details.
  • Contract validation showed the packaging run produced @workweave/router@0.2.10 with pi-router/src/compaction.ts, and the registry run confirms npm already contains that version, so republishing is blocked; the authored executable check script is retained in the publish workflow.
  • Artifacts for both findings and contract-validation were prepared and labeled for reviewer access.

View all artifacts

T-Rex Ran code and verified through T-Rex

Comments Outside Diff (2)

  1. install/npm/package.json, line 3 (link)

    P1 Bundled extension cannot be published at the unchanged version

    The Pi extension is copied into @workweave/router, but this publishable package remains at version 0.2.10. npm already contains @workweave/router@0.2.10, so the configured release will be rejected and users cannot receive this compaction fix. Increment this package to an unused version and publish with the corresponding router-v<version> tag.

    Artifacts

    Targeted package and registry publication check source

    • Executable check that packages the current archive, verifies the compaction extension is included, and queries npm for the exact release version, ending with the takeaway that the unchanged version is already occupied.

    Package publication-path check at version 0.2.10

    • Captured clean-archive `npm pack --dry-run` check showing `@workweave/router@0.2.10` includes `pi-router/src/compaction.ts`, ending with the takeaway that the release workflow targets the unchanged version.

    npm registry check for version 0.2.10

    • Captured npm registry query showing `@workweave/router@0.2.10` already exists, ending with the takeaway that npm cannot publish the configured version again.

    Pi router publication configuration inventory

    • Captured manifest and workflow inventory identifying the npm publishing workflow and Pi extension references, ending with the takeaway that the extension is released through `@workweave/router`.

    View artifacts

    T-Rex Ran code and verified through T-Rex

  2. General comment

    P1 Changed Pi extension cannot be released at the unchanged published npm version

    • Bug
      • install/pi-router/src/compaction.ts is copied into the @workweave/router npm package during prepack. The package manifest remains at 0.2.10, and the publish workflow requires a matching router-v0.2.10 tag. npm already has @workweave/router@0.2.10, so the configured npm publish path rejects the release and users cannot obtain the changed extension.
    • Cause
      • The bundled package version was not incremented with the extension change; the private install/pi-router/package.json is not the published package manifest.
    • Fix
      • Increment install/npm/package.json to an unused semantic version and publish using the corresponding router-v<new-version> tag.

    T-Rex Ran code and verified through T-Rex

Reviews (1): Last reviewed commit: "feat(ext): budget routed compaction off ..." | Re-trigger Greptile

…requested budget

Pi's built-in compaction only fires when the requested model's static window
is exceeded. When the router serves a smaller window, a final turn above the
served threshold but below the requested budget previously fell through both
sides: agent_end returned early for Pi to own it, Pi never fired, and the
served model could overflow.

Only defer the over-threshold final turn to Pi when Pi's own budget is also
exceeded (or the served window is at least the requested window). Add a
regression test for the served-window-smaller-than-requested final-turn case.

Addresses Cursor Bugbot review on #970.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant