Repository navigation
Conversation
A measurement phase: the default endpoint family stays `modern`, so no plugin behaviour changes and no characterization row flips. - Name the default in one constant, AUTO_ENDPOINT_FAMILY in src/utils/plane-model.ts, which is what `auto` resolves to, and pin it in test/endpoint-family.test.ts. - Extend the keyless matrix to three URL shapes (legacy, root, modern) over four protocols and two planes, add LEGACY_DISPATCH, and add one keyless free-model probe with its own pure classifier. - Add minimal keyed probes (pure test/support/keyed-probes.ts and its e2e): one tiny request per plane, protocol and shape, sent with the plane's own key and the plugin's identity from one shared constant, PROBE_IDENTITY. Only the cells the default sends to are asserted; the rest are recorded. - Gate the live job by tier with OPENCODE_E2E_KEYED (none, probes, full; unset means full). CI arms `probes` on a pull request that touches what the probes send, `full` nightly and on dispatch, and passes the secrets only when the tier is not `none`. - Write a dated endpoint-family record from both live suites to the run summary and an `endpoint-families` artifact, and add §13 to docs/redirect-planes.md with every cell pending the first CI run. The live part runs in CI only; no live result is invented here. A failing keyed modern cell in CI is the evidence for a later `fix:` that flips the default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
|
This PR didn't cause it. No fix exists yet. Regenerating the table needs Generated by Claude Code |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
What the review found, and what changed: - The keyed flip rule read a default cell's retired-model answer as a retired model without checking the same model on the other shapes. planeOutcome now reads each default cell beside the legacy and root cells of the same plane and protocol. - A refused key counted as flip evidence even when every shape refused it. It is now "check the CI secret" when a sibling refused the key and none accepted it, and "nothing else measured the key" only when no sibling refused it; a path-gone, rate-limited or busy sibling no longer blocks the CI-secret reason. - Breaks no flip could fix (accepted on the root shape only, or Go's Google, which no flip moves) were called flip evidence, and the "protocol" and "gone" reasons named causes nothing measured. A wrong-protocol or path-gone cell is now decided by the cell that endpoint: legacy dispatches to: "refused" (a flip would not serve it) when that cell refused readably, and always for Go's Google; "unmeasured" (re-run) when it measured nothing. Neither is flip evidence. - An account answer counted as the flip target accepting a protocol or a model, though the account's gate answers before either is read. It now counts only beside a refused key or a gone path, and a retired model next to an account answer no longer fails the plane. - The keyless free probe failed on a busy upstream (503/529) that the keyed classifier treats as routed. It now moves on past a busy candidate, and its dated record no longer pairs a derived upstream-busy with the last retired candidate's model and status. - A busy upstream followed by no answer failed the keyed plane, against the verdict definitions in redirect-planes.md section 13. Section 13 now says what the code does (no answer on the last attempt is transport-error unless an earlier attempt was a 429), and a test pins that it fails as unreadable, not as flip evidence. - Comments and docs said P-R2 had already measured the families while section 13 is pending its first CI run; said models.opencode.ai declares the root shape, which the generated catalog contradicts; said legacy cells are asserted nowhere live; and said a keyed auth-refused means the plugin's own turns would be refused, though a default Go turn sends a different key. All corrected. - CONTRIBUTING's key and mirror bullets now cover the keyed probes. - The plan no longer claims P-R2 opens with a green pin, gives the conditional flip a phase, and says the keyed probes decide a plane from the default cells read beside their siblings, not alone, and that an unreadable answer is never flip evidence. - Tests now cover the tier wiring of the four keyed live suites, no probe following a redirect, the push-to-none arm and the publish and upload steps of the CI wiring, the free probe moving on past a busy upstream, and the record being rewritten after every cell. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
|
This phase's first reviews ran on the general model because the dedicated reviewer was unavailable at the time. It has now had the dedicated review: three rounds through three lenses (correctness, repository rules, test matrix). Rounds
Four pairs of upheld findings were the same defect reported by two lenses. Rounds 2 and 3 mostly reviewed the previous round's classifier fixes. Fixed
Declined
Check, unit suite, mock suite, build and bundle gate all pass on the commit. Commit: 3ccd70c Generated by Claude Code |
Ported from the stack's base (d13cbe2 on #105) so this pull request's own `catalog freshness` goes green now, rather than waiting for the base to be merged up. It no-ops once the merge from P-R1 brings the same change. It takes `src/catalog-data.ts` exactly as the nightly refresh produced it (#111). The counts that restate the catalog's size follow it, on this branch's own wording of those passages: - the Zen shim count in catalog.test.ts (82 -> 81); - redirect-planes.md: Zen served 82 -> 81, the no-SDK Zen cell 25 -> 24, and "0 of 82" -> "0 of 81"; - protocol-routing-and-merge.md: the per-protocol shares now read "of 81", with completions at 24; - both READMEs: the no-SDK row is 24, and the active total is 81. "127 listed" is unchanged, because a model the vendor retires stays listed as `status: "deprecated"`. (cherry picked from commit d13cbe2) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
…r1' into claude/vigilant-mendel-bctz9n-p-r2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
…t-mendel-bctz9n-p-r2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
Phase P-R2 of docs/improvement-plan.md, stacked on #113.
P-R2 is a measurement phase. The default endpoint family stays
modern, so the plugin's behaviour does not change and noS0_TODAYor characterization row flips. The title carries no!and the commit has no breaking footer.What changed
AUTO_ENDPOINT_FAMILY = "modern"is whatautoresolves to. Before this, the choice sat inside a ternary. A later flip is a one-word edit, and the constant is pinned.src/utils/plane-model.ts,test/endpoint-family.test.ts/zen(/go)/v1, a newroot/inference(/go)/v1, and modern/inference(/go)/{openai,anthropic,google}/…. That is 24 cells over 4 protocols and 2 planes. It addsLEGACY_DISPATCH(the cellsendpoint: legacyreally sends to) and one keyless free-model probe with its own pure classifier.classifyKeylessProbeis unchanged.test/support/keyless-matrix.ts,test/e2e/keyless-matrix.e2e.tsPROBE_IDENTITY, and an offline test pins it against the identity a real turn sends. Only the cells the default sends to (TODAY_DISPATCH) are asserted. Every other cell is recorded.test/support/keyed-probes.ts,test/e2e/keyed-probes.e2e.ts,test/keyed-probes.test.tsOPENCODE_E2E_KEYEDtakesnone,probesorfull. Unset meansfull, so local runs behave as before. CI picks the tier per event:probesfor a pull request touchingsrc/host/**,protocols.ts,plane-model.ts,identity.ts,request-shape.tsor the probes' own inputs;fullnightly and on dispatch;noneotherwise. Secrets are passed only when the tier is notnone. The full keyed suites read their keys througharmedKey(tier, "full", …)..github/workflows/ci.yml,test/e2e/*.e2e.ts$OPENCODE_E2E_RECORD_DIR. The job publishes it to the run summary and to anendpoint-familiesartifact.test/support/endpoint-record.ts,test/endpoint-record.test.tsdocs/redirect-planes.md§13 uses the same columns, and every cell reads "pending the first CI run". A unit test holds the doc's tables to the formatter's columns and row order. The tier scope, the skip reasons and the plan's status row are updated.docs/redirect-planes.md,docs/improvement-plan.md,CONTRIBUTING.md,AGENTS.mdDecision recorded in the plan:
endpoint: legacyis recorded, not guarded, in the live job. Unit tests pin its mapping of every protocol on both planes. If a keyed modern cell fails in CI, that is the evidence for a laterfix:that flips the default.Behaviour changes
Plugin: none. The changes below are to the test suite and CI only:
PROBE_IDENTITYintest/support/keyed-probes.ts. What they send live is unchanged: user-agentOPENCODE_UA,x-opencode-client,x-opencode-project, and the probe session id under both session headers. They deliberately leave outx-opencode-request, the per-request metric id.probeskeyed tier on a pull request that touchessrc/utils/request-shape.ts, because every Messages probe sends itsanthropic-version. This is in addition tosrc/host/**,protocols.ts,plane-model.ts,identity.tsand the probes' own inputs.PLUGIN_HEADERStoOPENCODE_UA. That user agent reaches the suite's keyless free-tier and unmetered cases on every tier, so on every pull request and every push to main. This PR's own e2e run is therefore the first live observation of the new user agent on keyless traffic, before the nightly run.docs/credentials.mdrecords that this user agent did not change the free-tier verdict.Review
Each round ran three reviewers (correctness, tests, rules) with none missing. Every upheld finding was fixed in the next round.
not_found_error; apipefailhazard in the tier steprequest-shape.tsRound-4 mutation checks were each run, watched fail, then restored:
PROBE_IDENTITYbound to the patch UA: 4 failed.UNAVAILABLEand "model is overloaded" alternatives: 2 failed.UNAVAILABLEmatch: 1 failed.Verification
pnpm run checkvp test runvp test run --config vitest.mock.config.tspnpm run buildscripts/check.tscheck passed, including "lib/ carries the current build, including the served client bundle"vp test run --coveragegit diff, the 6 new files andlib/sk-probe-0123456789abcdefThe first five Node 26 rows were re-run on the final tree right before the commit. The live keyed and keyless results are not in this PR: they come from this branch's CI run, and §13 stays "pending" until then.
Known red check:
catalog freshnessfails on every branch until the vendor-catalog refresh lands on main. It does not come from this change.Generated by Claude Code