diff --git a/AGENTS.md b/AGENTS.md index 123555b6..8569e7ee 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -92,7 +92,7 @@ It is the same trick `scripts/check.ts` already uses for the client bundle, at f - `src/stream-hook.ts` · `src/fetch-patch.ts` · `src/tool-fallback.ts` — the `llm/stream` hook, the fetch interceptor, and the free-tier `read`/`bash` fallback. - `src/utils/request-shape.ts` — the fetch patch's PURE decisions, extracted so each branch is unit-testable on its own: which plane a URL belongs to, which credential header a protocol reads, the URL and header source of a `fetch` input, the model id and tool names a JSON body carries, and the Anthropic version constant. Everything here is a function of its arguments — no context, no globals, no I/O. The rule it encodes: **a predicate that lives in a closure cannot be exercised alone**, so its branches survive until some integration path happens to take them. `hasSessionHeader` and `isOpenCodeRequest` were exported and tested; their six private siblings were not, and that was the difference. - `src/key-capture.ts` · `src/go-discovery.ts` · `src/usage.ts` · `src/usage-contract.ts` — credential extraction, quota discovery, the usage service, and the shared shape. -- `src/catalog-data.ts` · `src/models-catalog.ts` · `src/models-discovery.ts` — the generated static shim, the OpenCode-catalog parser with SWR refresh, and discovery decoration. `catalog-data.ts` is **generated**; never hand-edit it. +- `src/catalog-data.ts` · `src/models-catalog.ts` · `src/models-discovery.ts` — the generated static shim, the OpenCode-catalog parser with SWR refresh, and the Settings discovery wrapper (behind `enrichModels`; off, Fetch is pi's list and `NO_DISCOVERY` passes through): on a plane the plugin serves, Settings → Models → Fetch answers pi's list plus the catalog's servable, non-deprecated models pi lacks (D3, P3), and the catalog alone while pi-ai restarts. `catalog-data.ts` is **generated**; never hand-edit it. - A note on the sweep that found the two entries around here, because it nearly produced a false one: grepping for a symbol's callers while EXCLUDING the file that defines it makes every same-file call look like no call at all. `resolveGoKeyForRef` reads as unreferenced that way and is not — `resolveGoApiKey` calls it, and the usage service calls that. (An earlier version of this line said that is "how the Go endpoint keeps Zen keys away from itself"; there is no such exclusion, and the note below records why.) Check the defining file too before believing an export is dead. **The general form, because this shape recurred twice in one session**: a sweep that resolves a name in the wrong CONTEXT reports a clean tree as broken. Grepping for callers without the defining file makes same-file calls look absent; resolving a doc's relative links from the repository root instead of from the file's own directory makes every link look dead. Both produced a list of confident findings and both were entirely wrong. A third instance is reading a UTC timestamp as local time and concluding that a workflow had not run since the previous evening, when it had run twenty minutes earlier — same shape, wrong context, confident conclusion. Before acting on a sweep, check that the thing being searched is the thing the claim is about, including the clock it is written in. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d450a8ab..3d1698e5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -53,12 +53,12 @@ What the main files are for: ## The real-host harness -`test/support/host.ts` builds the host from the packages this repository installs, at the versions `test/support/host-versions.ts` names (pi 1.1.0 with the harness patch, the `@deepseek-ai` cluster at the published host's tag). The routing work is judged against it: the contract tests pin the host facts the design rests on that the host can answer in-process (`docs/improvement-plan.md`: F1, F2, F4, F5, F8, F10–F16, F18 and F20, each by number; F6 and F19's auth half (constructing a `PiAiAdapter`) are first exercised by P-R1; F19's export half is pinned in `host-shapes` (`recordKeyFor`); F17 is honoured, not pinned — tier B fixes the apply order and the binary e2e infers it; F9 is a fact about opencode's own client, outside this host), and the characterization rows record what today's plugin does. +`test/support/host.ts` builds the host from the packages this repository installs, at the versions `test/support/host-versions.ts` names (pi 1.1.0 with the harness patch, the `@deepseek-ai` cluster at the published host's tag). The routing work is judged against it: the contract tests pin the host facts the design rests on that the host can answer in-process (`docs/improvement-plan.md`: F1, F2, F4, F5, F8, F10–F16, F18 and F20, each by number; F6 and F19's auth half (constructing a `PiAiAdapter`) are first exercised by P-R1; F19's export half is pinned in `host-shapes` (`recordKeyFor`); F17 is honoured, not pinned — tier B fixes the apply order and the binary e2e infers it; F9 is a fact about opencode's own client, outside this host; F21 is pinned by P3 in `host-discovery`, the pi-ai restart and contract cases), and the characterization rows record what today's plugin does. Two tiers, both in-process: -- **Tier A — `mountTierA`.** A bare cordis `Context`, `LlmRuntime`, and `dsh-llm-pi-ai` mounted as a plugin with `{ providers }` (and optionally this plugin). No loader and no files. Use it for the `internal/config` waterfall and pi-ai's credential paths — anything that needs a fiber and the registry and nothing that reads a profile (`host-waterfall`, `host-shapes`, `host-credentials`). Mounted here (`plugin`), this plugin is the real one: tier A has no loader, so `mountTierA` swaps the plugin's host-module source for `hostModulesForTests` through the seam in `src/cordis-context.ts` (`replaceHostModuleSource`), restores it on dispose, and the route claim serves both planes as it does on the real host (`host-harness`, `host-planes`). pi-ai is mounted first there, so the claim restarts it once. Characterize the plugin as it ships in tier B, which keeps the default source. -- **Tier B — `bootTierB`.** dsh-app-boot's `boot()` with a temporary HOME, a test bundle and a profile patch, the way the harness's own composition tests boot one. Use it for the config editor, `settings.describe`, the loader's volatile commit and the characterization of the plugin as it ships (`host-editor-contracts`, `host-characterization`). Its rows carry **package names**, resolved by a stub module loader, not `cordis:` builtins: the plugin's default host-module source finds pi-ai's entry by `options.name === "@deepseek-ai/dsh-llm-pi-ai"` (`piAiEntry` in `src/host/host-modules.ts`), so a builtin-named row would leave the route claim without pi-ai's module. The loader starts rows in parallel, so tier B makes the order explicit: `order: "plugin-first"` (the default, and what the published binary does) or `"pi-ai-first"`. +- **Tier A — `mountTierA`.** A bare cordis `Context`, `LlmRuntime`, and `dsh-llm-pi-ai` mounted as a plugin with `{ providers }` (and optionally this plugin). No loader and no files. Use it for the `internal/config` waterfall and pi-ai's credential paths — anything that needs a fiber and the registry and nothing that reads a profile (`host-waterfall`, `host-shapes`, `host-credentials`). Mounted here (`plugin`), this plugin is the real one: tier A has no loader, so `mountTierA` swaps the plugin's host-module source for `hostModulesForTests` through the seam in `src/cordis-context.ts` (`replaceHostModuleSource`), restores it on dispose, and the route claim serves both planes as it does on the real host (`host-harness`, `host-planes`). pi-ai is mounted first there, so the claim restarts it once. Pass `hostModules: null` to keep the plugin's default source instead: with no loader it cannot find pi-ai's module, so the claim stands down and pi-ai keeps the planes (`host-discovery`'s stood-down case). Characterize the plugin as it ships in tier B, which keeps the default source. +- **Tier B — `bootTierB`.** dsh-app-boot's `boot()` with a temporary HOME, a test bundle and a profile patch, the way the harness's own composition tests boot one. Use it for the config editor, `settings.describe`, the loader's volatile commit and the characterization of the plugin as it ships (`host-editor-contracts`, `host-characterization`, `host-discovery`). Its rows carry **package names**, resolved by a stub module loader, not `cordis:` builtins: the plugin's default host-module source finds pi-ai's entry by `options.name === "@deepseek-ai/dsh-llm-pi-ai"` (`piAiEntry` in `src/host/host-modules.ts`), so a builtin-named row would leave the route claim without pi-ai's module. The loader starts rows in parallel, so tier B makes the order explicit: `order: "plugin-first"` (the default, and what the published binary does) or `"pi-ai-first"`. What every case does: @@ -78,7 +78,8 @@ What every case does: `.github/workflows/ci.yml`'s `e2e-host` job installs the published `@deepseek-ai/dsh` (at `HOST_VERSIONS.dsh`) into a throwaway prefix, builds this checkout, and runs `test/e2e/host-binary.e2e.ts`. That file creates a temporary HOME whose profile links this checkout and the probe bundle (`test/e2e/host/`) through relative `link:` paths, stores dummy keys in the temporary home's credentials document (`$DSH_HOME/.credentials.yaml`, shared by every profile under that home), boots `dsh --profile e2e --no-open --port 0`, and reads what `test/e2e/host/probe-plugin.mjs` measured. It asserts the rows `S0_TODAY` records, which P-R1 flipped to the plugin owning both planes. -- **Cells.** `pinned` forces pi, cordis and the loader to `HOST_VERSIONS` with npm `overrides` (the published host declares cordis and the loader as tilde ranges), applies the harness patch to pi with `patch -p1`, asserts all three installed versions (a cached prefix included), then asserts every row. `floating` takes the pi npm resolves for the published host. It skips only the per-model rows (a newer pi's own entry can move a model's metadata); the listings before and after a turn, the save rows (the catalog-unknown id is accepted on the first try) and the row that the plane provider is built from the host prefix's own pi are asserted in both cells, since the plugin owns both planes and none of them depends on which ids pi describes. Its probe JSON is the record. +- **Cells.** `pinned` forces pi, cordis and the loader to `HOST_VERSIONS` with npm `overrides` (the published host declares cordis and the loader as tilde ranges), applies the harness patch to pi with `patch -p1`, asserts all three installed versions (a cached prefix included), then asserts every row. `floating` takes the pi npm resolves for the published host. It skips the per-model rows (a newer pi's own entry can move a model's metadata) and the Settings fetch's order (pi's list first); the listings before and after a turn, the save rows (the catalog-unknown id is accepted on the first try) and the row that the plane provider is built from the host prefix's own pi are asserted in both cells, since the plugin owns both planes and none of them depends on which ids pi describes. Its probe JSON is the record. +- **Settings' fetch.** The probe also records what Settings → Models → Fetch offers on each plane, through the gateway's Remote face. Both cells assert the whole servable catalog, each id once, in the Remote's five fields; the pinned cell also asserts the order, pi's own list first, then the catalog models pi lacks. - **Triggers.** Dispatch and the nightly run take both cells. A pull request takes `pinned`, and only when it touches the host seam — `src/host/**` (the route claim, the plane adapter, the Settings graft, the host-module loaders) and the modules whose behaviour depends on the real loader and apply order beside it, `src/lifecycle.ts`, `src/identity.ts`, `src/cordis-context.ts` and `src/models-discovery.ts`, and the two modules that still install host-level patches until they move under `src/host/` (`src/fetch-patch.ts`, the global fetch wrapper whose order the probe reads, and `src/stream-hook.ts`) (every other row producer, such as the pure plane model and profile in `src/utils/`, `src/key-capture.ts`, `src/models-catalog.ts` or `src/config.ts`, is covered on each pull request by the in-process characterization of the same `S0_TODAY` rows, and by the nightly run) — `cordis.patch.yml`, `test/e2e/host/**`, `test/e2e/host-binary.e2e.ts`, `test/support/host.ts` (the rows the binary e2e asserts), `test/support/host-versions.ts` or `patches/` (what the job installs and patches), `.github/workflows/ci.yml`, `vitest.e2e.config.ts` or `vite.config.ts` (the job's own steps and runner config; `vite.config.ts` also holds the pack config that builds `lib/index.mjs`, the bundle only this job loads into a real host, and the test settings the e2e config inherits) (scoped by a step: a workflow-level path filter would silence every other job; a diff the step cannot compute fails it rather than skipping the job). `package.json` is left out because every dependency bump touches it, so a change to its `test:e2e:host` script alone is exercised only by the nightly run. It needs no secret. To run it locally, install the host into a prefix outside the checkout, then point the suite at it (placeholder paths; `pnpm` must be on `PATH`). The first line reads the versions from `test/support/host-versions.ts`, the same way the CI job does, so the snippet never names one itself: diff --git a/README.md b/README.md index 25df85fd..b8c7737e 100644 --- a/README.md +++ b/README.md @@ -170,7 +170,7 @@ Counts are the **82** active `opencode` models in `models.opencode.ai` as of 202 Four properties follow: -- **You keep choosing.** Each plane serves exactly the models its `llm-pi-ai` entry lists — and the whole catalog when the entry lists none. Any id you save is accepted and served: one neither the catalog nor pi describes is sent as Chat Completions, unless the plane's entry names a route-level `api`, which applies to exactly those ids. +- **You keep choosing.** Each plane serves exactly the models its `llm-pi-ai` entry lists — and the whole catalog when the entry lists none. With **Enrich listings from the OpenCode catalog** on (the default), Settings → Models → Fetch offers the whole catalog: pi's own list, plus every servable, non-retired model the catalog has that pi does not list. A row it offers carries the catalog's name, context window, output limit and modalities — what the composer serves — and pi's only where the catalog has none; one adopted from it saves like any other id and is served by the plugin. Any id you save is accepted and served: one neither the catalog nor pi describes is sent as Chat Completions, unless the plane's entry names a route-level `api`, which applies to exactly those ids. - **Your row wins.** A model's name, context window, output limit and input modalities come from your own model row first, then the catalog, then pi's installed data. Its reasoning efforts come from your row's `reasoningEfforts`, then the catalog. Its cost and its protocol are not row fields: the cost comes from the catalog, then pi, and the protocol from the catalog's SDK, then pi's entry. - **A model whose reasoning is only on or off is offered as Off / On.** Off appears only where the model's protocol can actually turn reasoning off; a model with a real effort list offers exactly its levels, and a Chat Completions model whose list names `none` also offers Off, sent as `reasoning_effort: "none"` the way pi's own entries carry it. - **One credential, by the host's own rule.** A plane authenticates the way `llm-pi-ai` would: the entry's `apiKeyEnv` through the credentials service (or the launch environment when no credentials service is mounted), a Sign-in record stored under `llm-pi-ai/opencode`, and otherwise pi's own `OPENCODE_API_KEY` lookup, which is the same variable on both planes: name `apiKeyEnv: OPENCODE_GO_API_KEY` on the Go entry. The installation's own OpenCode store is consulted only when no reference is named and the deployment opted in. A plane whose key does not resolve still lists its models, and the turn fails with `MISSING_CREDENTIAL` before any request is sent. @@ -197,7 +197,7 @@ The model list is assembled from four sources, each answering a different questi | Bundled shim | what can be offered before any network call — cold start, offline | | `models.opencode.ai` | the canonical spec, price and display name | | The gateway's `/models` | what **this account** actually has | -| Discovery decoration | what `discoverModels` should answer | +| Settings' fetch (`discoverModels`) | on `opencode`/`opencode-go`: pi's own list, then every catalog model pi lacks | Merging is **additive**: adapter rows are kept, canonical rows are appended when a model is missing, and a row is dropped only when the registry marks it `status: "deprecated"` — a fact, not a guess. Display names and prices come from the catalog (`models.opencode.ai`), because a gateway listing carries ids and little else. @@ -228,7 +228,7 @@ Nine controls in four sections — the decisions a user actually makes. Everythi | **Gateway Requests** | Inject User-Agent | `on` | Restores the official OpenCode CLI `User-Agent` so Cloudflare WAF checks pass | | **Gateway Requests** | Inject Origin Headers | `on` | Injects `x-opencode-client` (and the origin header set) on gateway traffic | | **Gateway Requests** | Attach Workspace Project | `on` | Tags `x-opencode-project` with the active folder name; off omits the header | -| **Models & Free Tier** | Enrich listings from the OpenCode catalog | `on` | Lets the gateway `/models` listing carry the full catalog, and offers it to Settings' model discovery. Does **not** add models to the picker — the picker shows declared models only; use the card's **Check catalog** row to declare more. A declared model's own info (context, modalities, thinking levels) follows your row and the catalog whatever this says | +| **Models & Free Tier** | Enrich listings from the OpenCode catalog | `on` | Lets the gateway `/models` listing carry the full catalog, and offers it to Settings' model discovery. Does **not** add models to the picker by itself — the picker shows the models you declared, or the plane's catalog when its entry declares none, and this switch changes neither. To declare more, use **Settings → Models → Fetch** on an OpenCode plane: it offers pi's list plus the catalog's models pi lacks, and adopting a row and saving declares it. A declared model's own info (context, modalities, thinking levels) follows your row and the catalog whatever this says | | **Models & Free Tier** | Inject Core Tools | `on` | Adds the `read` + `bash` schemas free-tier `/responses` bodies require | | **Quota Meter** | Enable Composer Usage Meter | `on` | Mounts the live meter in the composer dock — the Go quota ring, or the Zen session spend | | **Account & Endpoint** | API Endpoint | `auto` | Which endpoint family requests go to: **Automatic** (same as Modern) · **Legacy** (`zen/v1`, `zen/go/v1`) · **Modern** (per-protocol `inference/{openai,anthropic,google}/…`, `inference/go/…` on Go) | @@ -393,7 +393,7 @@ These exist in the schema but render no control — each is a literal, a marker, Disabling or removing the plugin hands both planes back to `llm-pi-ai`, with nothing to clean up first: the plane entries were never moved out of its `providers` block. - **`llm-pi-ai` serves the ids it knows again**, on its own adapter, endpoint family and reasoning rules. -- **Ids it cannot serve drop out of the composer.** That is an id its installed catalog does not describe — a catalog-unknown id, or a newer model than its pi carries. +- **Ids it cannot serve drop out of the composer.** That is an id its installed catalog does not describe — a catalog-unknown id, or a newer model than its pi carries — including a model adopted from Settings' fetch that pi does not list. - **The next Settings edit of that plane is refused until they are removed**, because `llm-pi-ai` validates the whole plane entry on every save. Delete those ids from the plane's model list, and the edit goes through. ## 🛠 Troubleshooting @@ -404,7 +404,7 @@ Disabling or removing the plugin hands both planes back to `llm-pi-ai`, with not | `400 MissingSessionID` | No session header attached | Ensure `dsh-opencode-patch` is listed in the profile's `dsh.profile.bundles` | | Auto Review fails with `TRANSPORT: Connection error` | Host process not restarted since the update | Stop and restart `dsh web` so the new `lib/index.mjs` loads | | Quota ring never appears for a Go model | **Restart `dsh web`, then read the browser console** — the plugin is loaded through a `link:`, so `lib/index.mjs` is read once at process start and a page reload alone keeps the old host code; the meter reports which of its paths returned nothing | The console names the cause: `no remote to mount on`, `did not mount` with the exception, `still waiting for remote.opencodeGoUsage`, `no opencodeGoUsage service on the remote` with the keys it does expose, or `the answer is not a usage shape` with what came back. A missing or unroutable credential is one cause of several, not the only one | -| A catalog model is missing from Settings → Fetch, or a retired one persists | Stale host module, enrichment off, or the adapter answered from its packaged catalog | `pnpm run build`, restart `dsh web`, keep **Enrich listings from the OpenCode catalog** on, fetch from the matching route, search the exact id (e.g. `space-bunny-free`) | +| A catalog model is missing from Settings → Fetch, or a retired one persists | Stale host module, enrichment off, the route is not a plane the plugin serves (only `opencode`/`opencode-go` on the served `llm-pi-ai` row are complemented), or the catalog has not been read yet (offline, or `models.opencode.ai` blocked), so pi's own list, retired ids included, passes through and models newer than the bundled catalog are absent | `pnpm run build`, restart `dsh web`, keep **Enrich listings from the OpenCode catalog** on, fetch from the matching route, search the exact id (e.g. `space-bunny-free`); if the catalog is unread, then once `models.opencode.ai` is reachable fetch once to start the catalog refresh (that answer is still the old list — the refresh runs behind the read, and nothing runs it on a timer), then fetch again a moment later | | Popover shows “Limit reached” in red | 100% of a rolling/monthly window reached | Open the [OpenCode Console](https://opencode.ai/console) and enable _Use balance_ to overflow into Zen credits | | Non-OpenCode models misbehaving | Unrelated to this patch | Traffic to other providers passes through untouched | @@ -423,6 +423,7 @@ Disabling or removing the plugin hands both planes back to `llm-pi-ai`, with not - **`modelOverrides`, `maxContextWindow` and `modelRoutes` were removed** (and the documented `hiddenModels`, which nothing ever read). A model's own row in the plane's entry carries its name, context window, output limit, input modalities and reasoning efforts, and the catalog fills the rest; a row's `contextWindow` is the stand-in for `maxContextWindow`. A row that still carries the removed keys validates, and the keys are ignored. - **No per-model protocol pin.** `modelRoutes` could send one model on a protocol of your choosing; nothing replaces that. A model the catalog or pi describes always speaks the protocol they name; the plane entry's route-level `api` applies only to ids neither describes, and `baseURL` is route-level only. Cost is not a row field either: it comes from the catalog, then pi. - **Disabling or reloading the plugin restores the host methods it wrapped.** Model discovery and the Settings graft come off on a disable, and a live reload replaces them instead of stacking another layer. The planes go back to `llm-pi-ai` — see [Removing or disabling the plugin](#removing-or-disabling-the-plugin). +- **Settings → Models → Fetch offers the catalog's models pi does not list.** With **Enrich listings from the OpenCode catalog** on (the default), on both planes it answers pi's list first, then every servable, non-retired catalog model pi lacks (e.g. Zen `gpt-5.3-codex-spark`, Go `minimax-m3.1-flash`), and while `llm-pi-ai` restarts it answers the catalog alone instead of an error. It used to answer pi's list only. A row it offers on a plane now carries the catalog's name, context window, output limit and modalities, and pi's only where the catalog has none, so a model adopted from it saves what the composer serves: adopting Zen `claude-sonnet-4-5` saves a 1,000,000-token context window where it used to save pi's 200,000. With the switch off, Fetch answers pi's list alone, and a restart shows `llm-pi-ai`'s error. - **`@viztor/dsh-opencode` is no longer published.** Its last version's "Renamed to @viztor/dsh-opencode-patch" deprecation stands as the final message; install `dsh-opencode-patch` or `@viztor/dsh-opencode-patch` instead. ## 📜 Compatibility & Verification diff --git a/README.zh-CN.md b/README.zh-CN.md index 089bda8c..937aa59b 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -168,7 +168,7 @@ Zen 的 provider 级 SDK 是 `@ai-sdk/openai-compatible`。models.dev **只在 由此有四条性质: -- **你的挑选仍然有效。** 每个平面服务的正是其 `llm-pi-ai` 条目列出的模型;条目没有列出任何模型时,服务整个目录。你保存的任何 id 都会被接受并服务:目录与 pi 都未描述的 id 按 Chat Completions 发送,除非平面条目指定了路由级的 `api`——它只作用于这类 id。 +- **你的挑选仍然有效。** 每个平面服务的正是其 `llm-pi-ai` 条目列出的模型;条目没有列出任何模型时,服务整个目录。在**用 OpenCode 官方目录补全列表**开启时(默认),Settings → Models → Fetch 提供整个目录:pi 自己的列表,加上目录里 pi 未列出的每个可服务、未下线的模型。它提供的每一行带的是目录的名称、上下文窗口、输出上限与模态——即输入框所服务的值——只有目录没有时才用 pi 的;从中采纳的模型与其他 id 一样保存,并由插件服务。你保存的任何 id 都会被接受并服务:目录与 pi 都未描述的 id 按 Chat Completions 发送,除非平面条目指定了路由级的 `api`——它只作用于这类 id。 - **你的模型行优先。** 模型的名称、上下文窗口、输出上限与输入模态,先取你自己的模型行,再取目录,最后取 pi 的内置数据。推理档位先取你行里的 `reasoningEfforts`,再取目录。价格与协议不是模型行的字段:价格先取目录、再取 pi,协议先取目录的 SDK、再取 pi 的条目。 - **只有开/关两种推理的模型提供“关 / 开”。** 只有该模型的协议确实能关闭推理时才提供“关”;有真正档位列表的模型只提供它自己的档位;档位列表中含 `none` 的 Chat Completions 模型同样提供“关”,按 pi 自己条目的写法发送 `reasoning_effort: "none"`。 - **一份凭据,按宿主自己的规则。** 平面的认证方式与 `llm-pi-ai` 相同:条目的 `apiKeyEnv` 经凭据服务解析(未挂载凭据服务时读启动环境),保存在 `llm-pi-ai/opencode` 下的登录记录,否则用 pi 自己的 `OPENCODE_API_KEY` 查找——两个平面读的是同一个变量:请在 Go 条目上写明 `apiKeyEnv: OPENCODE_GO_API_KEY`。只有在没有声明引用、且部署选择开启时,才会读取 OpenCode 安装自己的存储。密钥解析不到的平面仍列出其模型,回合在发出任何请求前以 `MISSING_CREDENTIAL` 失败。 @@ -190,12 +190,12 @@ Zen 的 provider 级 SDK 是 `@ai-sdk/openai-compatible`。models.dev **只在 模型清单由四个来源拼成,每个来源回答的是不同的问题: -| 来源 | 回答 | -| :------------------- | :--------------------------------------------- | -| 内置 shim | 一次网络请求都还没有时能提供什么——冷启动、离线 | -| `models.opencode.ai` | 规范参数、价格与显示名 | -| 网关的 `/models` | **这个账号**实际能用什么 | -| 发现装饰 | `discoverModels` 该回答什么 | +| 来源 | 回答 | +| :-- | :-- | +| 内置 shim | 一次网络请求都还没有时能提供什么——冷启动、离线 | +| `models.opencode.ai` | 规范参数、价格与显示名 | +| 网关的 `/models` | **这个账号**实际能用什么 | +| 设置的获取(`discoverModels`) | 在 `opencode`/`opencode-go` 上:先是 pi 自己的列表,再是 pi 缺少的每个目录模型 | 合并是**增量式**的:适配器给的行保留,规范行只在缺失时追加,只有当按 provider 记录的退役名单点名某个模型时才省略它——那是事实,不是猜测。显示名与价格取自目录(`models.opencode.ai`),因为网关列表只给 id,别的几乎没有。 @@ -226,7 +226,7 @@ Zen 的 provider 级 SDK 是 `@ai-sdk/openai-compatible`。models.dev **只在 | **网关请求** | 恢复 User-Agent(默认开启) | `on` | 恢复官方 OpenCode CLI 的 `User-Agent`,使 Cloudflare WAF 校验通过 | | **网关请求** | 注入客户端来源头(默认开启) | `on` | 在网关流量上注入 `x-opencode-client`(以及来源请求头集合) | | **网关请求** | 附带工作区项目标识(默认开启) | `on` | 用当前文件夹名标记 `x-opencode-project`;关闭则省略该请求头 | -| **模型与免费额度** | 用 OpenCode 官方目录补全列表(默认开启) | `on` | 让网关 `/models` 列表带上完整目录,并把目录提供给设置页的模型发现。**不会**往选择器里添加模型——选择器只显示已声明的;要添加请用卡片的**检查目录**。已声明模型自身的信息(上下文、模态、思考档位)始终来自你的模型行与目录,与此开关无关 | +| **模型与免费额度** | 用 OpenCode 官方目录补全列表(默认开启) | `on` | 让网关 `/models` 列表带上完整目录,并把目录提供给设置页的模型发现。本身**不会**往选择器里添加模型——选择器显示你声明的模型;条目未声明任何模型时显示该平面的目录,此开关不改变这两者。要声明更多,请在 OpenCode 平面上使用 **Settings → Models → Fetch**:它会提供 pi 的列表以及 pi 缺少的目录模型,选用一行并保存即完成声明。已声明模型自身的信息(上下文、模态、思考档位)始终来自你的模型行与目录,与此开关无关 | | **模型与免费额度** | 自动补全核心工具(默认开启) | `on` | 补上免费层 `/responses` 请求体所需的 `read` + `bash` schema | | **配额计量** | 开启输入区用量计量(默认开启) | `on` | 在输入框停靠栏挂载实时计量表——Go 额度环,或 Zen 会话消耗 | | **配额计量** | 显示会话消耗与模型费率(默认开启) | `on` | 追加显示本会话累计花费与当前模型每百万 Token 费率 | @@ -383,7 +383,7 @@ Zen 路线上不出现任何 Go 的数字:根本不画圆环(没有可量的 停用或移除插件会把两个平面交还给 `llm-pi-ai`,事先无需清理:平面条目从未离开它的 `providers` 块。 - **`llm-pi-ai` 重新服务它认识的 id**,使用它自己的适配器、端点家族与推理规则。 -- **它服务不了的 id 会从输入框的模型列表中消失。** 也就是它内置目录未描述的 id——目录未知的 id,或比它的 pi 更新的模型。 +- **它服务不了的 id 会从输入框的模型列表中消失。** 也就是它内置目录未描述的 id——目录未知的 id,或比它的 pi 更新的模型——包括从设置的获取中采纳、而 pi 未列出的模型。 - **在删除这些 id 之前,对该平面的下一次设置编辑会被拒绝**,因为 `llm-pi-ai` 每次保存都会校验整个平面条目。把这些 id 从平面的模型列表中删掉,编辑即可通过。 ## 🛠 故障排查 @@ -394,7 +394,7 @@ Zen 路线上不出现任何 Go 的数字:根本不画圆环(没有可量的 | `400 MissingSessionID` | 未附带会话请求头 | 确认 `dsh-opencode-patch` 已列入 profile 的 `dsh.profile.bundles` | | Auto Review 报 `TRANSPORT: Connection error` | 更新后宿主进程未重启 | 停止并重启 `dsh web`,让新的 `lib/index.mjs` 加载 | | Go 模型始终不显示额度环 | **重启 `dsh web`,然后查看浏览器控制台**——插件通过 `link:` 加载,`lib/index.mjs` 只在进程启动时读取一次,仅刷新页面仍运行旧的宿主代码;计量会报告它的哪条路径没有返回 | 控制台会指出原因:`no remote to mount on`、带异常的 `did not mount`、`still waiting for remote.opencodeGoUsage`、列出其实际暴露键的 `no opencodeGoUsage service on the remote`,或带返回内容的 `the answer is not a usage shape`。缺失或无法路由的凭据只是几个原因之一:在 `opencode-go` 条目上写明 `apiKeyEnv: OPENCODE_GO_API_KEY` 并在 DSH Credentials 中存储该密钥 | -| 目录中的模型在 Settings → Fetch 里缺失,或已下线的模型仍然存在 | 宿主模块过期、补全关闭,或适配器用自带目录作答 | `pnpm run build`、重启 `dsh web`、保持**用 OpenCode 官方目录补全列表**开启、从匹配的路由获取、搜索精确 id(如 `space-bunny-free`) | +| 目录中的模型在 Settings → Fetch 里缺失,或已下线的模型仍然存在 | 宿主模块过期、补全关闭、该路由不是插件服务的平面(只有所服务的 `llm-pi-ai` 行上的 `opencode`/`opencode-go` 会被补全),或目录尚未读取(离线,或 `models.opencode.ai` 被拦截)——此时 pi 自己的列表(包括已下线的 id)原样通过,比随包目录更新的模型也不会出现 | `pnpm run build`、重启 `dsh web`、保持**用 OpenCode 官方目录补全列表**开启、从匹配的路由获取、搜索精确 id(如 `space-bunny-free`);目录尚未读取时,待 `models.opencode.ai` 可访问后,先获取一次以触发目录刷新(这次仍是旧列表——刷新在读取之后于后台进行,且没有定时刷新),稍后再获取一次 | | 弹出面板以红色显示“已达限额” | 滚动 / 每月窗口已用到 100% | 打开 [OpenCode 控制台](https://opencode.ai/console)并启用 _使用余额(Use balance)_,以溢出到 Zen 额度 | | 非 OpenCode 模型行为异常 | 与本补丁无关 | 发往其他网关的流量原样通过 | @@ -413,6 +413,7 @@ Zen 路线上不出现任何 Go 的数字:根本不画圆环(没有可量的 - **`modelOverrides`、`maxContextWindow` 与 `modelRoutes` 已移除**(以及文档中写过、却从未被读取的 `hiddenModels`)。平面条目中模型自己的行可以写名称、上下文窗口、输出上限、输入模态与推理档位,其余由目录补齐;行里的 `contextWindow` 代替 `maxContextWindow`。仍带有这些已移除键的配置行可以通过校验,这些键会被忽略。 - **不再能按模型固定协议。** `modelRoutes` 可以让某个模型走你指定的协议;这一点没有替代。目录或 pi 描述过的模型总是使用它们指定的协议;平面条目的路由级 `api` 只作用于两者都未描述的 id,`baseURL` 也只在路由级。价格同样不是模型行的字段:先取目录,再取 pi。 - **停用或重载插件会恢复它包装过的宿主方法。** 停用时模型发现与设置嫁接会被撤下,热重载会替换它们而不是再叠一层。两个平面交还给 `llm-pi-ai`——见[移除或停用插件](#移除或停用插件)。 +- **Settings → Models → Fetch 会提供 pi 未列出的目录模型。** 在**用 OpenCode 官方目录补全列表**开启时(默认),它在两个平面上先回答 pi 的列表,再回答 pi 缺少的每个可服务、未下线的目录模型(例如 Zen 的 `gpt-5.3-codex-spark`、Go 的 `minimax-m3.1-flash`);`llm-pi-ai` 重启期间,它只回答目录,而不是报错。以前它只回答 pi 的列表。它在平面上提供的每一行现在带的是目录的名称、上下文窗口、输出上限与模态,只有目录没有时才用 pi 的,因此从中采纳的模型保存的正是输入框所服务的值:采纳 Zen 的 `claude-sonnet-4-5` 会保存 1,000,000 的上下文窗口,以前保存的是 pi 的 200,000。此开关关闭时,Fetch 只回答 pi 的列表,重启期间显示 `llm-pi-ai` 的错误。 - **`@viztor/dsh-opencode` 不再发布。** 其最后一个版本上的 "Renamed to @viztor/dsh-opencode-patch" 弃用提示即为最终信息;请改装 `dsh-opencode-patch` 或 `@viztor/dsh-opencode-patch`。 ## 📜 兼容性与验证 diff --git a/docs/deep-dive.md b/docs/deep-dive.md index e063e6b5..29390fe7 100644 --- a/docs/deep-dive.md +++ b/docs/deep-dive.md @@ -171,8 +171,7 @@ OpenCode's gateway `GET …/models` endpoints frequently return a truncated subs 1. Dual bundled shims (zero latency, offline): `OPENCODE_GO_CATALOG` carries every active Go subscription model with per-million-token rates, so session pricing works before the first refresh; `OPENCODE_ZEN_CATALOG` carries the flagships (`claude-sonnet-4-5`, `claude-opus-4-7`, `gpt-5.4`, `gemini-3.8-flash`, `qwen3.8-max`, `kimi-k3`). The counts are deliberately not restated here: they move with the vendor, they are visible in the generated file, and CI fails when the shim is stale. Retired models are excluded so a failed refresh can never resurrect a row the gateway no longer serves: the three Zen-route Muse Spark 1.2 ids are suppressed, while the paid Go 1.2 contributor entry stays (the CLI still lists it). Startup is instant: no cold-start delay, blocking network calls, or airplane-mode failures. 2. Background revalidation: both catalogs revalidate against [`https://models.opencode.ai/api.json`](https://models.opencode.ai/api.json) — the OpenCode catalog, which today serves the byte-identical object as `models.dev/api.json` — every **60 minutes** (the OpenCode CLI's canonical cycle), merging new models, deprecations and updated limits. Errors degrade gracefully and retain the active catalog. 3. Gateway models-endpoint enrichment: `patchFetch` intercepts `GET …/models` on OpenCode routes and merges the live Go or Zen catalog: human-friendly names (`DeepSeek V4.1 Flash`, `Qwen3.8 Flash`, `Grok 4.7`, `MiMo V2.6 Pro`), verified context windows (up to 1,000,000+ tokens) and max output tokens (up to 384,000), correct input modalities (`text`, `image`), with retired Muse Spark 1.2 rows omitted. -4. Settings “Fetch Available Models” decoration: Settings discovery is keyed by `llm-pi-ai`'s settings namespace, so for a plane id `llm-pi-ai`'s own discovery callback answers, from pi's packaged catalog without calling the gateway; no adapter is consulted, the plane adapter included. The plugin's `discoverModels` wrapper (`decorateModelDiscovery`) then decorates that hosted result: adapter rows and order are preserved, their metadata enriched from the catalog, and provider-retired rows removed. Nothing is dropped for its protocol: every SDK the catalog uses maps to one. It **does not append** catalog rows the adapter did not report: an earlier revision did, and that is how a deployment that had declared four models saw eighty-six in Settings. This is candidate metadata for the settings surface; it never rewrites saved route configuration. Widening a deployment's list is a separate, explicit action — the card's **Check catalog** row, which shows what the catalog offers and declares it only when asked. -5. Native model discovery registration: on the host runtime the plugin also registers with `ctx.llm.registerModelDiscovery` for `opencode-go` and `opencode`, which is what a _draft_ route's “discover” call answers from — not the picker, which reads the declared list. Its rows are the catalog's servable models (`toDiscovered`), which today is all of them. The gateway-listing enrichment, this registration and the discovery decoration all sit behind the **Enrich listings from the OpenCode catalog** switch (`enrichModels`). The model info a plane answers is not a decoration: the plane adapter builds every model from the user's row and the catalog (D1, D17), whatever the switch says. +4. Settings “Fetch Available Models” (`discoverModels`): Settings asks the namespace a provider is configured under — pi-ai's, for both planes — and pi-ai answers a plane from pi's packaged catalog without calling the gateway. On a plane the plugin serves (`opencode`/`opencode-go`, asked on the served `llm-pi-ai` row while the route claim is live), the plugin's wrapper keeps pi's rows and order — rows the OpenCode catalog has retired removed once a live catalog read has landed (offline, pi's list passes through), and each row's fields read from the catalog first and from pi only where the catalog has none, which is the composer's own order (D17), so a row adopted from the fetch carries what the composer already serves rather than pi's older capacities — and appends every servable, non-deprecated catalog model pi lacks, each id once, in the Remote's row shape (`id`, `name`, `contextWindow`, `maxTokens`, `inputModalities`; nothing else, and nothing `undefined`). While pi-ai restarts, its discovery is withdrawn and the host answers `NO_DISCOVERY`; the wrapper answers the plane's catalog alone instead. Nothing is written until the user adopts a row and saves (D8), and the composer still shows the declared list, or the plane's catalog for an entry that declares none. A route the plugin does not serve — another pi-ai row's planes, a custom relay, a draft whose base is an OpenCode gateway — keeps the enrich-only merge (pi's values first, the catalog filling only what a row omits): pi-ai serves those, and it cannot serve an id it does not list. The gateway-listing enrichment and this wrapper both sit behind the **Enrich listings from the OpenCode catalog** switch (`enrichModels`). The model info a plane answers is not a decoration: the plane adapter builds every model from the user's row and the catalog (D1, D17), whatever the switch says. ### 8. Session spend & model rate diff --git a/docs/engineering-notes.md b/docs/engineering-notes.md index 63d2d03a..9c39a01b 100644 --- a/docs/engineering-notes.md +++ b/docs/engineering-notes.md @@ -88,7 +88,7 @@ The report was `400 ModelProtocolUnsupported` from a session, and it is reproduc All three were listed on the same route, which declares `api: openai-completions`. The last two do not speak that protocol, so choosing either one cannot work — and the routing this plugin exists to do is what would have moved them to a route that does. -**It cannot, because those routes are not registered.** Measured against a real deployment's config: `opencode-responses` is present, `opencode-anthropic` and `opencode-google` are not. The plugin registers model discovery for `opencode` and `opencode-go` only, so the two routes the extra protocols need are the deployment's to write — which is the thing the core model says a user should never have to do. +**It cannot, because those routes are not registered.** Measured against a real deployment's config: `opencode-responses` is present, `opencode-anthropic` and `opencode-google` are not. At the time the plugin registered model discovery for `opencode` and `opencode-go` only (P3 removed those registrations; Settings' fetch now goes through pi-ai's namespace and the plugin's wrapper), so the two routes the extra protocols needed were the deployment's to write — which is the thing the core model says a user should never have to do. So the fix belongs here rather than in anyone's profile: the routes for the other protocols have to come from the plugin, the way the completions route already does. Until they do, a model that names an SDK other than completions is offered on a route that cannot serve it, and the answer is the `400` above. diff --git a/docs/improvement-plan.md b/docs/improvement-plan.md index 944d53c0..49d3cc6a 100644 --- a/docs/improvement-plan.md +++ b/docs/improvement-plan.md @@ -28,7 +28,7 @@ Phases land as **stacked pull requests**. Each one is based on the previous phas | P-R1 | `feat:` the plugin owns the plane routes | in review | | P-R2 | `test:` the endpoint families, measured | in review | | P-R2b | `fix:` flip the default on §13's evidence | if needed | -| P3 | `feat:` Settings complements pi's list | planned | +| P3 | `feat:` Settings complements pi's list | in review | | P-T2 | `test:` suite structure | planned | | P-T3 | `ci:` CI and scripts | planned | | P5 | `refactor:` installers | planned | @@ -89,7 +89,7 @@ Phases land as **stacked pull requests**. Each one is based on the previous phas - the SDK server (`sdk/server/src/server.ts:423`); - the translator (`experimental/translator/src/deepseek.ts:69`). - With no hidden routes (P-R1), **no listing patch is needed**; today's exists only to hide them from these readers. (Instance patches do reach Remote calls, because the gateway resolves methods with `Reflect.get` at call time — `api/gateway/src/index.ts:841-843` — which the `discoverModels` wrapper relies on.) -- **Settings discovery is keyed by the pi-ai entry id** (`llm-pi-ai/src/index.ts:138,156`; `ProviderEditor.tsx:254`; `ModelListEditor.tsx:167,244`). The plugin's own `registerModelDiscovery` namespaces are never asked. +- **Settings discovery is keyed by the pi-ai entry id** (`llm-pi-ai/src/index.ts:138,156`; `ProviderEditor.tsx:254`; `ModelListEditor.tsx:167,244`). The plugin's own `registerModelDiscovery` namespaces were never asked; P3 deleted them. - The host's normalization runs inside the original `discoverModels` (`index.ts:618-631`), so the wrapper must dedupe and whitelist the fields itself. - `LlmDiscoveredModel` carries only id, name, contextWindow, maxTokens and inputModalities, with no reasoning field. - **D1 representation.** Effort ids must be pi levels (`adapter.ts:170-181`), and two vocabularies mean opposite things by `null`: @@ -104,7 +104,7 @@ Phases land as **stacked pull requests**. Each one is based on the previous phas | Defect | Where | | :-- | :-- | | The deprecated wrapper re-exports a `default` that does not exist, an ESM link error. It is published every release. | `scripts/publish-scoped.ts:137`, `release.yml:124` | -| Settings shows pi's snapshot, not the catalog | `models-discovery.ts:142-198,532-539` | +| ~~Settings shows pi's snapshot, not the catalog~~ — fixed by P3 (`complementDiscoveredModels`) | `models-discovery.ts` (pre-P3 lines) | | Mode merging is done only for Go→Gemini. For the other modes the picker shows catalog levels while dispatch validates against a different profile: toggle models show Minimal…Max with no Off, and Go `minimax-m3` + Xhigh/Max is refused on the internal Anthropic route. | `models-discovery.ts:607-616,735-760,1063-1203`; `pi-provider.ts:351-386`; `responses-provider.ts:111-150` | | `isZenProvider("opencode-google")` is false | `utils/request-shape.ts:265-271` | | `enrichModelsResponse` turns a 401/403/5xx into a 200 catalog. It must still fill the Go plane's 404. | `models-catalog.ts:800-804` | @@ -124,7 +124,7 @@ Phases land as **stacked pull requests**. Each one is based on the previous phas **The problem, in the owner's words.** A provider defines one backend. Supporting every protocol and the newer catalog meant un-registering the original `opencode`/`opencode-go` provider and registering the plugin's own. Deleting another plugin's registration is unsupported, so the plugin mounted extra routes per protocol instead, merged their models back into the user's provider, and redirected dispatch. This section shows that the host **does** allow the original intent, through supported paths (F11–F13): keep pi-ai from claiming the two ids, and register the plugin's provider under them. A spike on the real host confirmed it (run S0). -**Facts from the host's own code and data.** F1–F5 and F10 are measured: the data files were counted and the resolver read. F6–F9 and F11–F13 are read from the code, and run S0 exercised F11–F13 end to end. P-T1's real-host harness pins the ones the host can answer in-process: P-T1b pins F1, F2, F4, F5, F8, F10–F16, F18 and F20 by number (F3 through its F4 consequence, F7 through F13's register and DUPLICATE halves); F6 (dispatch through a non-pi adapter) and F19's auth half (constructing a `PiAiAdapter`) are first exercised by P-R1, F6 through its mock gateway and the real `LlmRuntime`; F19's export half is pinned in `host-shapes` (`recordKeyFor`); F17 is honoured, not pinned — tier B fixes the apply order and the binary e2e infers it; F9 is a fact about opencode's own client, outside this host. A fact that fails where it is pinned sends the design back to this section. +**Facts from the host's own code and data.** F1–F5 and F10 are measured: the data files were counted and the resolver read. F6–F9 and F11–F13 are read from the code, and run S0 exercised F11–F13 end to end. P-T1's real-host harness pins the ones the host can answer in-process: P-T1b pins F1, F2, F4, F5, F8, F10–F16, F18 and F20 by number (F3 through its F4 consequence, F7 through F13's register and DUPLICATE halves); F6 (dispatch through a non-pi adapter) and F19's auth half (constructing a `PiAiAdapter`) are first exercised by P-R1, F6 through its mock gateway and the real `LlmRuntime`; F19's export half is pinned in `host-shapes` (`recordKeyFor`); F17 is honoured, not pinned — tier B fixes the apply order and the binary e2e infers it; F9 is a fact about opencode's own client, outside this host. F21 is pinned by P3 in `host-discovery` (the pi-ai restart and contract cases). A fact that fails where it is pinned sends the design back to this section. | # | Fact | Evidence | | :-- | :-- | :-- | @@ -148,6 +148,7 @@ Phases land as **stacked pull requests**. Each one is based on the previous phas | F18 | **`PiAiAdapter.listModels` answers `[]` on `MISSING_CREDENTIAL`.** | `llm-pi-ai/src/adapter.ts:295-311` | | F19 | **`PiAiAdapterOptions.auth` is required.** pi-ai does not export its credential-store or auth-context builders; it does export `PiAiAdapter`, `Config` and `recordKeyFor`. | `llm-pi-ai/src/adapter.ts:98-106`; `index.ts:77-92` | | F20 | **The Settings page reads pi-ai's resolved config.** `describe()` builds `value` from `fiber.config`, so once the claim strips the planes they read as unconfigured. The gateway resolves service methods per call (`Reflect.get`), so an instance wrapper on `describe` is seen by `settingsController.describe` and by `write()`. | `settings/settings/src/index.ts:319`; `api/gateway/src/index.ts:841-843`; `api/settings-controller/src/index.ts:98-103,217` | +| F21 | **A pi-ai restart withdraws its discovery.** Between UNLOADING and the reload, `discoverModels(piNs, …)` answers `NO_DISCOVERY`. The Remote face is `remoteDiscoverModels`, which reads `this.discoverModels` per call. | `llm/src/index.ts:569-661`; measured in tier B (`test/host-discovery.test.ts`, the pi-ai restart case) | Three corrections to the facts above: @@ -231,10 +232,10 @@ Three corrections to the facts above: #### `src/host/patch-method.ts`: the two host wrappers that remain -`patch-method.ts` restores a wrapper only if it is still the current one, deletes it when the original was inherited, and re-wraps on `llm/adapters-updated`. It installs two wrappers: +`patch-method.ts` restores a wrapper only if it is still the current one, deletes it when the original was inherited, and can re-wrap on `llm/adapters-updated` (no site needs it yet). It installs two wrappers: - **`settings.describe` (D18, F20).** It runs inside `ctx.inject(["settings"])`, and touches only the `llm-pi-ai` descriptor. When `value.providers[plane]` is missing, it sets it to the interpolated, `Config`-normalized committed entry; `base`, `user` and the revision are untouched. A contract test pins that no node under `providers` in pi-ai's `Config` carries `role("secret")`. Today only `apiKeyEnv` has a role, and it is `credential-ref`. -- **`discoverModels` (P3).** It answers pi's rows plus the catalog's missing ones, deduped and whitelisted. A `NO_DISCOVERY` while pi-ai restarts yields catalog-only rows. +- **`discoverModels` (P3).** It answers pi's rows plus the catalog's missing ones, deduped and whitelisted. A `NO_DISCOVERY` while pi-ai restarts yields catalog-only rows. Landed as: the complement applies only to a plane route asked on the namespace of the pi-ai row the claim serves, and only while the claim is live (`RouteClaim.claiming()`); every other request the wrapper classifies (another pi-ai row's planes, a custom relay, a gateway-URL draft) keeps the enrich-only merge, now whitelisted, since pi-ai serves those and refuses an id it does not list. No `rewrapOnAdaptersUpdated`: the `ctx.inject(["llm"])` scope already follows the service, and a pi-ai restart never touches the `llm` instance. #### Deleted @@ -291,8 +292,9 @@ src/ lifecycle.ts apply(): installers only host/ patch-method.ts wrap a host method: restore only if still current; delete when the original - was inherited; re-wrap on llm/adapters-updated (settings.describe, discoverModels) - discovery.ts discoverModels: pi's answer + the catalog's missing models (D3, D8) + was inherited; can re-wrap on llm/adapters-updated (no site needs it yet) + discovery.ts discoverModels: pi's answer + the catalog's missing models (D3, D8); + in src/models-discovery.ts until P10a route-claim.ts prepended internal/config filter: pi-ai never claims the planes; reconcile on commit; mount and teardown ordering (F14-F17) plane-provider.ts PlaneAdapter (extends the host's PiAiAdapter), registered as the declared planes @@ -404,10 +406,10 @@ What stays private is pinned by contract tests against the real host packages: - **The flip rule.** A keyed modern cell that answers path-gone, wrong-protocol or auth-refused with the CI key turns the job red, and so does a modern model-gone beside another shape of the same plane × protocol (same model, same key) that accepted it (`served` or `path-exists`). Flip evidence is a wrong-protocol, path-gone, auth-refused or model-gone on the modern cell while the cell a flip would send the same plane × protocol to (`LEGACY_DISPATCH`'s, when it is not the modern cell itself — never Go's Google) accepted the request (`served` or `path-exists`; beside a refused key or a gone path `account` counts too, as it shows the key read and the path there, but not beside a refused protocol or model, as the account's gate answers before either is read): the evidence for P-R2b, the `fix:` that flips `AUTO_ENDPOINT_FAMILY`, with the copy, `TODAY_DISPATCH` and the re-tagged wire facets. A refusal only the root or Go Google's `/zen/go` cell accepted is red but not flip evidence (no flip reaches either); so are a key refused on every shape that read it (the CI secret), a key no other shape read, and a protocol or a path the flip's cell refused too (a flip would not serve it). Each sibling counts only for what it measured: a protocol or a path the flip's cell answered nothing about (a rate limit, a busy upstream, the account's gate, no answer) is red, not evidence, and a re-run. A busy upstream never counts as a shape accepting the request. The job also fails when a default cell answers nothing readable (`other`, a redirect, a refused session, `transport-error`, `no-model`), which is not flip evidence, and skips a plane none of whose default cells was served (a busy upstream — a 503, a 529 or a 5xx naming an overload — skips like a rate limit; a 500 or 502 stays `other`). - **Measured results** live in `docs/redirect-planes.md` §13, pending the first CI run. - **The follow-up is P-R2b**, conditional: it pastes the first CI run's tables into §13, and flips `AUTO_ENDPOINT_FAMILY` only if that run (or a later one) shows flip evidence; with none, it closes as a `docs:` change once §13 holds a real run. -9. **P3 `feat:` Settings complements pi's list.** - - The `discoverModels` wrapper answers servable, non-deprecated models, deduped and whitelisted. - - Delete `installModelDiscovery` and `toDiscovered`. - - Rewrite the "adds nothing" tests, and the docs that say the same. +9. **P3 `feat:` Settings complements pi's list.** In review. Landed as: + - the `discoverModels` wrapper (`src/models-discovery.ts`, through `host/patch-method.ts`): on a plane the plugin serves, pi's rows first in pi's order (ids the OpenCode catalog has retired dropped once a live catalog read has landed — offline, pi's list passes through; each field read from the catalog first and from pi only where the catalog has none, the composer's own D17 order, so a row adopted from the fetch carries what the composer serves), then every servable, non-deprecated catalog model pi lacks (`complementDiscoveredModels`), each id once and whitelisted to `LlmDiscoveredModel`'s five fields; `NO_DISCOVERY` from pi-ai's namespace answers the catalog alone (F21); + - `installModelDiscovery`, `toDiscovered`, the `registerModelDiscovery` namespaces and their lifecycle tests deleted; + - the "adds nothing" tests and docs rewritten; `test/host-discovery.test.ts` carries the real-host and contract rows, and the binary e2e the Settings fetch row. Refinements: the `RouteClaim.claiming()` seam, so gap rows are offered only while the claim is live and only on the served row's namespace; the whitelist omits a field nobody supplied, unknown modalities included (absent means unknown in the Remote's schema), rather than defaulting it, and narrows a supplied modality list to text/image, omitting the field when nothing survives (as the composer's `catalogInput` does); `verify-catalog` check 3 retargeted from `mergeDiscoveredModels` to `planeProfileFor`, the composer's own source; and the wrapper kept behind `enrichModels`, as the listing enrichment and the old discovery registration were — with it off, Settings' fetch is pi's list and a restart's `NO_DISCOVERY` passes through, the user's opt-out (D8), a reading of D3 for the owner to confirm. 10. **P-T2 `test:` suite structure.** - Move the mock-booting unit files into `test/mock/`. - Retire the ad-hoc contexts, `createMockContext` and `HarnessSimulator`. @@ -597,7 +599,7 @@ P0 → P1a → P1b → P2 → P-T1a → P-T1b → P-H → P-R1 → P-R2 → (P-R **Cells:** - (a) pi 1.1.0, pinned and patched, on PRs, nightly and on dispatch; it asserts every row; -- (b) whatever npm installs, nightly and on dispatch. Since P-R1 it skips only the per-model rows (a newer pi's own entry can move a model's metadata); the listings before and after a turn, the save rows and the row that the plane provider is built from the host prefix's own pi are asserted in both cells, since none of them depends on which ids pi describes. Its probe JSON is the record. +- (b) whatever npm installs, nightly and on dispatch. Since P-R1 it skips the per-model rows (a newer pi's own entry can move a model's metadata) and, since P3, the Settings fetch's order (pi's list first); the listings before and after a turn, the save rows, the row that the plane provider is built from the host prefix's own pi, and (since P3) the Settings fetch row — the whole servable catalog, each id once, in the Remote's five fields — are asserted in both cells, since none of them depends on which ids pi describes. Its probe JSON is the record. ### The live gateway job (`OPENCODE_API_KEY` / `OPENCODE_GO_API_KEY`) diff --git a/docs/protocol-routing-and-merge.md b/docs/protocol-routing-and-merge.md index 6dfd1fb0..e373da00 100644 --- a/docs/protocol-routing-and-merge.md +++ b/docs/protocol-routing-and-merge.md @@ -86,7 +86,7 @@ They are different checks, and none stands in for another: - The **live suite** asks the gateway which endpoint recognises each shipped model, with a hand-written body per protocol. It proves the endpoint answers and what the gateway rejects. - The **mock-gateway suite** runs `llm.stream` through the real `LlmRuntime` and the plane adapter, with the plugin's fetch patch, against a local gateway that refuses a request whose shape its endpoint would not accept. It proves the request the plugin builds is one the endpoint accepts, for every characterized model on both planes and both families. -- The **real-host suites** (`test/host-characterization.test.ts`, `test/host-planes.test.ts`) run the plugin beside `llm-pi-ai` on the real host, in both apply orders, and pin ownership, restarts, the Settings flow, credentials and isolation. +- The **real-host suites** (`test/host-characterization.test.ts` and `test/host-planes.test.ts`) run the plugin beside `llm-pi-ai` on the real host, in both apply orders, and pin ownership, restarts, the Settings flow, credentials and isolation. `test/host-discovery.test.ts` covers Settings' fetch on tier B in the published binary's order (`plugin-first`), an explicit pi-ai restart that exercises the `NO_DISCOVERY` window a `pi-ai-first` mount would cause, and a stood-down claim on tier A, which mounts pi-ai first; with the claim stood down no restart follows, so the order is immaterial there. ### How we support API endpoints in different formats diff --git a/docs/redirect-planes.md b/docs/redirect-planes.md index 5e44e792..b0456409 100644 --- a/docs/redirect-planes.md +++ b/docs/redirect-planes.md @@ -56,7 +56,7 @@ These are the rules a correct implementation satisfies. Each one has a test in | | Invariant | What it requires | | :-- | :-- | :-- | -| I1 | Every offered model is routable | If a model appears in the picker, its plane's route serves it on the shape its SDK names. This is the plugin's headline promise; the Go gap broke it once, and one route per plane closed it for good. | +| I1 | Every offered model is routable | If a model appears in the picker, or in Settings → Models' fetch on a plane, its plane's route serves it on the shape its SDK names. This is the plugin's headline promise; the Go gap broke it once, and one route per plane closed it for good. | | I2 | A route carries one plane | Its base URLs, its catalog and its credential all come from the same plane. Mixing them resolves to "model not found" against the route's own endpoint. | | I3 | The endpoint is chosen from (plane, SDK) | Never from the SDK alone: the same SDK appears on both planes, on each plane's own base. | | I4 | No SDK means completions | A model naming no SDK speaks `openai-completions`, and so does an id no catalog describes (D15), unless the plane entry names an `api` for it. | @@ -75,6 +75,7 @@ These are the rules a correct implementation satisfies. Each one has a test in | `keyless-matrix.e2e.ts` · live, keyless | I3 | a cell the default sends to answers 404, or a free model is not parsed on the default completions path | | `keyed-probes.e2e.ts` · live, keyed | I1, I3 | a default-family cell answers path-gone, wrong-protocol or auth-refused with the plane's own key, a retired model another shape of the same protocol accepted, or nothing readable: `other`, a redirect, a refused session, `transport-error`, `no-model`. Flip evidence only when the cell `endpoint: legacy` dispatches to, a different one, accepted what the default cell refused (§13) | | `endpoint-family.test.ts` | I3 | `auto` stops meaning the pinned family, or legacy stops mapping a protocol on a plane | +| `host-discovery.test.ts` · Settings' fetch on the real host | I1 | a plane's fetch misses a servable catalog model, or offers one the plane adapter does not list, validate and send once saved | | coverage ratchet | — | a new branch of the plane model or profile is added without a test (both are held at 100%) | ## 6. Refactor (done, then superseded) @@ -102,7 +103,7 @@ One model, every stage. The left column is what a user can see; the right column | # | The user sees | What runs | Where | | --: | :-- | :-- | :-- | -| 1 | a model in the picker | the generated shim answers before any network call, `models.dev` refreshes it, and the gateway's own listing is merged in additively, so a row is only dropped when the registry marks it deprecated | `catalog-data.ts` → `models-catalog.ts` → `models-discovery.ts` | +| 1 | a model in the picker | the picker lists the plane's declared models, or its servable catalog when the entry declares none (D3), with the generated shim answering cold and `models.opencode.ai` refreshing it. Settings' fetch offers pi's list plus the catalog models pi lacks | `catalog-data.ts` → `models-catalog.ts` → `utils/plane-profile.ts` (picker) · `models-discovery.ts` (Settings' fetch) | | 2 | _(invisible)_ | each row carries the SDK the vendor ships (`provider_npm`). Nothing in the UI names it, and nothing the user sets can contradict it | the catalog | | 3 | one model selected | the composer sets `provider` + `model`; no route, protocol or endpoint appears anywhere in the interface | the host | | 4 | a turn starts | `llm/stream` fires; the hook claims the turn only for the routes this plugin declares, and leaves every other provider alone | `stream-hook.ts` | diff --git a/docs/upstream-findings.md b/docs/upstream-findings.md index 6f71627d..ce7ac03a 100644 --- a/docs/upstream-findings.md +++ b/docs/upstream-findings.md @@ -421,7 +421,7 @@ The row's `providers` list REPLACES the code default, so the two drifting apart Most of this document exists because four separate bugs had the same invisible shape: a model that is dropped is not filtered, not deprecated and not erroring — it is simply absent. Each was found with a hand-written script that was then thrown away. That script is now kept: ```sh -pnpm run catalog:verify # offline: shim, join, routing, free-tier tools +pnpm run catalog:verify # offline: shim, declared list, routing, free-tier tools pnpm run catalog:verify:probe # adds a live probe of every offered model ``` @@ -429,7 +429,7 @@ It checks, and exits non-zero on any failure: 1. **Shim freshness** — the bundled `catalog-data.ts` equals the live registry exactly (both come from the same parser over the same payload). What the registry should have flagged but did not is the probe's business, not this check's. 2. **Dead rows** (`--probe`) — no offered model answers `400 Model is unavailable` or `410 ModelDeprecated`. -3. **The picker's join** — every servable model reaches the parent route's list. +3. **The declared list** — the composer's source (`planeProfileFor`) publishes exactly the ids a plane declares: nothing added and nothing missing. Settings' fetch offers the catalog; the composer never widens. 4. **Routing** — every servable model resolves to a route this plugin mounts. 5. **Free-tier tools** — every free model gets the core-tool fallback. diff --git a/scripts/verify-catalog.ts b/scripts/verify-catalog.ts index 3e9d06fc..d62b31b0 100644 --- a/scripts/verify-catalog.ts +++ b/scripts/verify-catalog.ts @@ -1,5 +1,5 @@ /** - * Verify the catalog, the picker's join, and the routing table against each other. + * Verify the catalog, the declared list, and the routing table against each other. * * This exists because the failures it catches are all INVISIBLE from the outside: a * model that is dropped is not filtered, not deprecated and not erroring — it is @@ -12,8 +12,9 @@ * 1. **Shim freshness** — the bundled `catalog-data.ts` must match the live registry. * 2. **Dead rows** — no offered model may answer `400 Model is unavailable` or * `410 ModelDeprecated` (needs `--probe`; otherwise skipped). - * 3. **The picker's join** — every servable model must reach `listModels(parentRoute)`, - * which is `listProviders() × listModels(id)` and is where 55 models went missing. + * 3. **The declared list** — the composer's source (`planeProfileFor`) publishes + * exactly the ids a plane declares: Settings' fetch offers the catalog (D3), + * the composer never widens. * 4. **Routing** — every servable model must resolve, on its own plane's route, * to a protocol and a base on that plane (`planeModelFor`, D13). * 5. **Free-tier tools** — every free model must get the core-tool fallback. @@ -37,10 +38,10 @@ import { MODELS_OPENCODE_URL, parseModelsDevCatalog, } from "../src/models-catalog.ts"; -import { mergeDiscoveredModels } from "../src/models-discovery.ts"; import { openCodeSessionIdFor } from "../src/session.ts"; import { isCoreToolModel } from "../src/tool-fallback.ts"; import { planeModelFor } from "../src/utils/plane-model.ts"; +import { planeProfileFor } from "../src/utils/plane-profile.ts"; import { isServableSdk, protocolForSdk } from "../src/utils/protocols.ts"; const PROBE = process.argv.includes("--probe"); @@ -172,21 +173,46 @@ for (const [route, catalog, plane] of [ ["opencode", getLiveZenCatalog(), "zen"], ["opencode-go", getLiveGoCatalog(), "go"], ] as const) { - // A deployment declares some ids; the picker must publish exactly those. An - // earlier revision merged the whole catalog in, which silently overrode every - // `models:` array. This asserts the opposite: declared in, declared out. + // A deployment declares some ids; the composer must publish exactly those. + // An earlier revision merged the whole catalog in, which silently overrode + // every `models:` array. This asserts the opposite: declared in, declared + // out. The composer's list is the plane profile's provider, built here with + // a stub that hands back the models it is given; Settings' fetch is a + // different surface, and offers the catalog on purpose (D3). const declared = catalog .filter((m) => isServableSdk(m.provider_npm)) .slice(0, 3) .map((m) => ({ id: m.id, name: m.name })); - const listed = mergeDiscoveredModels(declared, plane); - const listedIds = new Set(listed.map((m) => m.id)); + const { profile } = planeProfileFor({ + buildProvider: ({ models }) => ({ getModels: () => models }), + catalog, + endpoint: "auto", + entry: { models: declared }, + familyApi: OPENCODE_FAMILY_API[plane], + piModels: [], + resolveRetryPolicy: () => ({}), + route, + }); + const provider = profile.piProvider as + | { getModels?: () => readonly { id?: unknown }[] } + | undefined; + if (typeof provider?.getModels !== "function") { + fail( + `${route}: the plane profile did not build: ${profile.catalogError ?? "no provider"}` + ); + continue; + } + const listedIds = new Set(provider.getModels().map((m) => String(m.id))); const declaredIds = declared.map((m) => m.id); const added = [...listedIds].filter((id) => !declaredIds.includes(id)); + // Declared OUT too: an empty provider would otherwise pass as "nothing added". + const missing = declaredIds.filter((id) => !listedIds.has(id)); if (added.length > 0) { fail( `${route}: ${added.length} undeclared models were added: ${added.slice(0, 5).join(", ")}` ); + } else if (missing.length > 0) { + fail(`${route}: declared models were not published: ${missing.join(", ")}`); } else { ok( `${route}: exactly the ${declaredIds.length} declared model(s), nothing added` @@ -264,5 +290,7 @@ if (failures.length > 0) { console.log(`FAILED: ${failures.length} check(s)`); process.exitCode = 1; } else { - console.log("PASS: catalog, join, routing and free-tier tools agree"); + console.log( + "PASS: catalog, declared list, routing and free-tier tools agree" + ); } diff --git a/src/cordis-context.ts b/src/cordis-context.ts index 7194505d..9ea56dee 100644 --- a/src/cordis-context.ts +++ b/src/cordis-context.ts @@ -67,15 +67,6 @@ export interface CordisContext { */ loader?: LoaderService; llm?: { - /** - * Offer to interrogate provider endpoints on behalf of one settings - * namespace. Uniqueness is by namespace, so a second registration under the - * same one is a conflict rather than a replacement. - */ - registerModelDiscovery?: ( - ns: string, - discover: (request?: unknown, signal?: AbortSignal) => Promise - ) => (() => void) | undefined; /** Declare routes a plugin can activate through configuration. */ registerConfigurableProviders?: ( entries: readonly unknown[] diff --git a/src/host/patch-method.ts b/src/host/patch-method.ts index 2e6cc471..fed3af76 100644 --- a/src/host/patch-method.ts +++ b/src/host/patch-method.ts @@ -1,9 +1,9 @@ /** * The one way this plugin wraps a method on a host object. * - * Every instance patch in `src/` goes through {@link patchMethod}: the listing, - * discovery and model-info wrappers on the `llm` service and the plane adapters' - * `resolveModel`, `prepareCall` and `modelOf` (all in `models-discovery.ts`). + * Every instance patch in `src/` goes through {@link patchMethod}; today two + * sites: model discovery (`models-discovery.ts`) and the `settings.describe` + * graft (`host/describe-graft.ts`). * Wrapping an INSTANCE is enough because the host resolves a service method per * call (F20: the gateway reads it with `Reflect.get` at dispatch), so a wrapper * set on the instance is what every later caller runs. diff --git a/src/host/route-claim.ts b/src/host/route-claim.ts index 8cef75aa..2a048bc5 100644 --- a/src/host/route-claim.ts +++ b/src/host/route-claim.ts @@ -112,6 +112,14 @@ export interface RouteClaim { >; /** The settings namespace pi-ai's row is described under. */ readonly piAiNamespace: () => string; + /** + * Whether the claim has neither stood down (the host's modules could not be + * read) nor torn down. True from install, so before `mounted` settles the + * plane routes may still be pi-ai's: a plane save is already validated by + * the plugin, and a fetch complemented in that window assumes the claim goes + * live, which it does unless the modules are unavailable. + */ + readonly claiming: () => boolean; /** Settles once the claim is mounted; never rejects. */ readonly mounted: Promise; } @@ -240,6 +248,7 @@ export const installRouteClaim = ( `the plane routes stay pi-ai's: this composition offers no ${missing.join(", ")}` ); return { + claiming: () => false, committedEntries: () => ({}), mounted: Promise.resolve(), piAiNamespace: () => PI_AI_RUNTIME, @@ -830,6 +839,7 @@ export const installRouteClaim = ( })(); return { + claiming: () => state.active, committedEntries: () => (state.active ? state.normalized : {}), mounted, piAiNamespace: () => livePiAi()?.entry?.options?.id ?? PI_AI_RUNTIME, diff --git a/src/lifecycle.ts b/src/lifecycle.ts index 816c943e..10061f84 100644 --- a/src/lifecycle.ts +++ b/src/lifecycle.ts @@ -12,7 +12,7 @@ * | {@link installUsageService} | register the Go quota service + remotes | * | {@link installFetchPatch} | wrap `globalThis.fetch` (returns `false` if none) | * | {@link installStreamHook} | capture turn state on `llm/stream` | - * | {@link installModelDiscovery} | publish the canonical catalog to the picker | + * | {@link installDiscovery} | complement Settings' model discovery with the catalog (D3) | * * Order is load-bearing and documented at the call site in {@link apply}. * @@ -32,14 +32,11 @@ import { patchFetch } from "./fetch-patch.ts"; import { resolveGoBaseURL } from "./go-discovery.ts"; import { installDescribeGraft } from "./host/describe-graft.ts"; import { installRouteClaim } from "./host/route-claim.ts"; -import { LEGACY_NAME, LEGACY_PKG, name } from "./identity.ts"; +import { LEGACY_NAME, LEGACY_PKG } from "./identity.ts"; import { - type CatalogModelSpec, - getLiveGoCatalog, - getLiveZenCatalog, - sanitizeModalities, -} from "./models-catalog.ts"; -import { decorateModelDiscovery } from "./models-discovery.ts"; + decorateModelDiscovery, + type DiscoveryClaim, +} from "./models-discovery.ts"; import { STORE_NAMESPACE } from "./opencode-store-contract.ts"; import { OpencodeStoreService } from "./opencode-store-service.ts"; import { ModelPurgeService } from "./purge-service.ts"; @@ -47,24 +44,6 @@ import { createStreamHook } from "./stream-hook.ts"; import type { ActiveTurnState } from "./turn-store.ts"; import { GoUsageService } from "./usage.ts"; import { isFetchFunction, isFunctionLike, isRecord } from "./utils/guards.ts"; -import { isServableSdk } from "./utils/protocols.ts"; - -/** - * One catalog row in the shape DSH's model-discovery surface expects. - * - * Models whose SDK no route serves are dropped: offering them would be offering - * a model that cannot work, and the user has no way to tell that from the row. - */ -const toDiscovered = (specs: readonly CatalogModelSpec[]) => - specs - .filter((m) => isServableSdk(m.provider_npm)) - .map((m) => ({ - contextWindow: m.context_window, - id: m.id, - inputModalities: sanitizeModalities(m.input_modalities), - maxTokens: m.max_output_tokens, - name: m.name, - })); /** Tell a row still configured under the pre-rename id/package to update. */ const logRenameNotice = (ctx: CordisContext): void => { @@ -378,94 +357,21 @@ const installStreamHook = ( }; /** - * The settings the LATEST activation on a root resolved, which is what every - * discovery callback answers from. - * - * Keyed on the root for the same reason the services are: `apply` runs once per - * live reload, and the host refuses a second registration under a namespace that - * is still held. The registrations therefore belong to the process, and only the - * settings they read move with each activation. - */ -const discoveryConfigOn = new WeakMap(); - -/** The namespaces this plugin publishes, and the catalog each one answers with. */ -const DISCOVERY_NAMESPACES: readonly { - readonly ns: string; - readonly catalog: () => readonly CatalogModelSpec[]; -}[] = [ - { catalog: () => [...getLiveGoCatalog(), ...getLiveZenCatalog()], ns: name }, - { catalog: getLiveGoCatalog, ns: "opencode-go" }, - { catalog: getLiveZenCatalog, ns: "opencode" }, -]; - -/** - * Publish the canonical Go/Zen catalog to DSH's model discovery. - * - * Model discovery is part of the same "canonical catalog" feature as the - * gateway listing enrichment, so both sit behind the one `enrichModels` - * toggle: turning it off leaves DSH's own catalog and the raw gateway - * listings untouched. - * - * Registered once per root and never withdrawn, like the usage service. The - * earlier shape registered on every activation and swallowed the host's - * duplicate refusal, so the FIRST activation's settings answered forever: a - * reload that changed `enrichModels` changed nothing the picker saw. Each - * namespace is registered on its own, so one refusal no longer skips the - * others. - */ -const installModelDiscovery = ( - scope: CordisContext, - config: ResolvedPluginConfig -): void => { - const root: CordisContext = scope.root ?? scope; - const registered = discoveryConfigOn.has(root); - if (!registered && !config.enrichModels) { - return; - } - discoveryConfigOn.set(root, config); - if (registered) { - return; - } - const { llm } = scope; - if (typeof llm?.registerModelDiscovery !== "function") { - return; - } - const current = (): ResolvedPluginConfig => - discoveryConfigOn.get(root) ?? config; - for (const { ns, catalog } of DISCOVERY_NAMESPACES) { - try { - llm.registerModelDiscovery(ns, () => { - const settings = current(); - return Promise.resolve( - settings.enrichModels ? toDiscovered(catalog()) : [] - ); - }); - } catch (error) { - // Non-fatal, and said: a picker missing a plane is otherwise a silence. - scope.logger?.warn?.( - `[dsh-opencode-patch] model discovery for ${ns} did not register`, - error - ); - } - } -}; - -/** - * Discovery for one `llm` scope: the catalog namespaces, and the wrapper that - * merges the catalog into a discovery answer, tied to the scope so an `llm` - * restart or a disable undoes it. + * Settings' model discovery for one `llm` scope: a fetch on a plane the + * plugin serves is complemented by the catalog (D3). Tied to the scope, so an + * `llm` restart wraps the new service and a disable undoes it. */ const installDiscovery = ( ctx: CordisContext, - config: ResolvedPluginConfig + config: ResolvedPluginConfig, + claim: DiscoveryClaim ): void => { ctx.inject?.(["llm"], (scope: unknown) => { if (!isRecord(scope)) { return; } const llmScope = scope as CordisContext; - installModelDiscovery(llmScope, config); - const stop = decorateModelDiscovery(llmScope, config); + const stop = decorateModelDiscovery(llmScope, config, claim); if (isFunctionLike(llmScope.effect)) { llmScope.effect(() => () => stop?.(), "dsh-opencode-patch.discovery"); } @@ -487,7 +393,8 @@ const installDiscovery = ( * 4. the fetch patch, which bails out (warning) when there is nothing to * wrap — the stream hook must not be attached then, or turns would run * without session affinity; - * 5. the stream hook and model discovery, which only make sense once the + * 5. the stream hook, and the Settings discovery wrapper that complements a + * plane's fetch with the catalog (D3), which only make sense once the * request path is patched. * * The returned promise is the claim's mount: cordis waits for it, which is @@ -513,7 +420,7 @@ export const apply = async ( if (installFetchPatch(ctx, config, als)) { installStreamHook(ctx, config, als); - installDiscovery(ctx, config); + installDiscovery(ctx, config, claim); } await claim.mounted; }; diff --git a/src/models-catalog.ts b/src/models-catalog.ts index db934519..03727b93 100644 --- a/src/models-catalog.ts +++ b/src/models-catalog.ts @@ -786,12 +786,14 @@ export const enrichModelsResponse = async ( const catalogById = new Map(catalog.map((m) => [m.id, m])); const merged = new Map>(); - // **This is the CATALOG surface, so it is the one place that seeds.** - // `discoverModels` in `dsh-llm-pi-ai` returns its installed catalog for a - // provider it ships and only fetches an endpoint for one it does not — so a - // `/models` answer is "what this endpoint offers", the list a user browses, not - // the list a picker shows. Widening is right here and wrong anywhere that - // decides what a deployment may pick. + // **This is a CATALOG surface, so it seeds.** So does Settings' fetch on a + // plane the plugin serves (`complementDiscoveredModels`): both offer + // candidates to browse or adopt. `discoverModels` in `dsh-llm-pi-ai` returns + // its installed catalog for a provider it ships and only fetches an endpoint + // for one it does not — so a `/models` answer is "what this endpoint offers", + // the list a user browses, not the list a picker shows. Widening is wrong + // anywhere that decides what a deployment may pick: the composer lists the + // declared models alone. // // The seed also covers the ordinary case of a gateway answering with a subset: // the vendor's own listing is narrower than its catalog. diff --git a/src/models-discovery.ts b/src/models-discovery.ts index 4123e2de..1cb17c05 100644 --- a/src/models-discovery.ts +++ b/src/models-discovery.ts @@ -1,18 +1,17 @@ /** - * Provider-aware decoration for DSH model discovery. + * Settings' model discovery, complemented by the plugin's catalog (D3). * - * The settings surface asks the adapter that owns a route for candidates — - * this plugin's plane adapter for `opencode` and `opencode-go`, which answers - * the way `llm-pi-ai` does, from pi's installed catalog without calling the - * gateway, so response enrichment never runs. Decorating its returned - * candidates is therefore the in-plugin way to fill a capacity the installed - * catalog omits, or to remove a model OpenCode CLI retired, from a “fetch - * available models” answer. (P3 replaces this wrapper with a discovery - * answer of the plugin's own.) + * Settings → Models → Fetch asks the namespace a provider is configured under + * — pi-ai's, for both planes — and pi-ai answers a plane from pi's packaged + * catalog without calling the gateway. On a plane the plugin serves, this + * wrapper answers pi's list plus every servable, non-deprecated catalog model + * pi lacks, so a model newer than pi's vintage can be adopted from the fetch + * rather than typed in; while pi-ai restarts it answers the catalog alone. * - * The decorator only rewrites results for an explicitly claimed OpenCode - * provider route. Nothing is stored: as with the underlying discovery call, - * the answer remains candidate metadata for the surface to adopt. + * It is candidate metadata, as the host's own answer is: nothing is written + * until the user adopts a row and saves (D8), and the composer still shows + * the declared list (or the plane's catalog for an entry that declares none, + * which this wrapper does not decide). * * @module dsh-opencode-patch/models-discovery */ @@ -20,13 +19,18 @@ import type { ResolvedPluginConfig } from "./config.ts"; import type { CordisContext } from "./cordis-context.ts"; import { patchMethod } from "./host/patch-method.ts"; +import type { RouteClaim } from "./host/route-claim.ts"; import { + type CatalogModelSpec, + type CatalogProvider, + DSH_SUPPORTED_MODALITIES, findModelSpecOn, + getLiveGoCatalog, + getLiveZenCatalog, isDeprecatedModelOn, - sanitizeModalities, - type CatalogProvider, } from "./models-catalog.ts"; import { isRecord } from "./utils/guards.ts"; +import { isPlaneRoute, isServableSdk } from "./utils/protocols.ts"; import { isGoPlaneUrl } from "./utils/request-shape.ts"; /** @@ -46,6 +50,23 @@ export interface DiscoveryCandidate { name?: string; } +/** + * One row of a discovery answer: exactly the host's `LlmDiscoveredModel`. + * + * The host normalizes inside the method this module wraps, so what the wrapper + * returns reaches the Remote as it is; its codec strips an unknown key and + * refuses a modality other than text/image. A field nobody supplied is ABSENT, + * never `undefined`: absent modalities mean "unknown", not "text" — and so + * does a supplied list in which no text or image entry survives. + */ +export interface DiscoveredRow { + readonly id: string; + readonly name?: string; + readonly contextWindow?: number; + readonly maxTokens?: number; + readonly inputModalities?: ("text" | "image")[]; +} + /** Whether a discovery result has the shape this decorator can preserve. */ const isDiscoveryCandidate = (value: unknown): value is DiscoveryCandidate => { if (!isRecord(value)) { @@ -74,6 +95,74 @@ const isDiscoveryCandidate = (value: unknown): value is DiscoveryCandidate => { return true; }; +/** + * A supplied modality list narrowed to what the Remote carries, or + * `undefined` when nothing survives: an audio-only list says nothing about + * text, so the row leaves the field absent ("unknown"), as the composer's own + * plane model does for the same spec. + */ +const remoteModalities = ( + modalities: readonly string[] +): ("text" | "image")[] | undefined => { + const kept = modalities.filter((modality): modality is "text" | "image" => + DSH_SUPPORTED_MODALITIES.has(modality) + ); + return kept.length > 0 ? kept : undefined; +}; + +/** + * Which source a row's fields are read from first. + * + * - `pi-first`: a route pi-ai serves. pi-ai reads pi's row (after the user's), + * so the catalog only fills a field pi's row omits. + * - `catalog-first`: a plane the plugin serves. The composer reads the catalog + * before pi (D17), and a row adopted from the fetch is saved as the user's + * row, which the composer reads before both — so the row carries what the + * composer already serves, and pi fills only what the catalog lacks. + */ +type FieldOrder = "catalog-first" | "pi-first"; + +/** A supplied modality list narrowed, or `undefined` when none was supplied. */ +const modalitiesOf = ( + supplied: readonly string[] | undefined +): ("text" | "image")[] | undefined => + supplied === undefined ? undefined : remoteModalities(supplied); + +/** + * One row in the Remote's shape: each field from the first source `order` + * names that has it, and only when one of them has it. + */ +const rowFrom = ( + candidate: DiscoveryCandidate | undefined, + spec: CatalogModelSpec | undefined, + id: string, + order: FieldOrder = "pi-first" +): DiscoveredRow => { + const catalogFirst = order === "catalog-first"; + const name = catalogFirst + ? (spec?.name ?? candidate?.name) + : (candidate?.name ?? spec?.name); + const contextWindow = catalogFirst + ? (spec?.context_window ?? candidate?.contextWindow) + : (candidate?.contextWindow ?? spec?.context_window); + const maxTokens = catalogFirst + ? (spec?.max_output_tokens ?? candidate?.maxTokens) + : (candidate?.maxTokens ?? spec?.max_output_tokens); + // Catalog-first mirrors the composer's `input`: the catalog's list when any + // text or image entry survives it, else pi's. + const modalities = catalogFirst + ? (modalitiesOf(spec?.input_modalities) ?? + modalitiesOf(candidate?.inputModalities)) + : modalitiesOf(candidate?.inputModalities ?? spec?.input_modalities); + return { + id, + ...(name === undefined ? {} : { name }), + ...(contextWindow === undefined ? {} : { contextWindow }), + ...(maxTokens === undefined ? {} : { maxTokens }), + ...(modalities === undefined ? {} : { inputModalities: modalities }), + }; +}; + /** * Decide which canonical catalog, if any, may supplement one discovery answer. * @@ -119,25 +208,16 @@ export const resolveDiscoveryProvider = ( }; /** - * Enrich an adapter’s discovery answer from the catalog, honoring provider-scoped - * retirements. - * - * The adapter’s own rows keep their capacities and order; a missing capacity is - * filled from the catalog. **Catalog rows are NOT appended when the adapter - * answered at all** — the deployment’s list is the list, and a refresh must not - * widen it. They are used only as a cold-start fallback, when the adapter - * reported nothing, so a route is never empty. - * - * An earlier version always appended the catalog. On the picker that was - * invisible until someone counted the rows; on a refresh it is the same mistake a - * second time — the settings model list would grow past what was configured. + * The adapter's own rows, each id once, retired ids dropped, and every row cut + * to the Remote's fields with its values read in `order`. */ -export const mergeDiscoveredModels = ( +const adapterRows = ( discovered: unknown, - provider: CatalogProvider -): DiscoveryCandidate[] => { + provider: CatalogProvider, + order: FieldOrder +): DiscoveredRow[] => { const seen = new Set(); - const merged: DiscoveryCandidate[] = []; + const merged: DiscoveredRow[] = []; if (Array.isArray(discovered)) { for (const candidate of discovered) { @@ -153,54 +233,126 @@ export const mergeDiscoveredModels = ( continue; } seen.add(candidate.id); - const spec = findModelSpecOn(provider, candidate.id); - merged.push({ - ...candidate, - contextWindow: candidate.contextWindow ?? spec?.context_window, - inputModalities: sanitizeModalities( - candidate.inputModalities ?? spec?.input_modalities - ), - maxTokens: candidate.maxTokens ?? spec?.max_output_tokens, - }); + merged.push( + rowFrom( + candidate, + findModelSpecOn(provider, candidate.id), + candidate.id, + order + ) + ); } } - // **Nothing is ever added.** An earlier revision seeded the whole catalog when - // the adapter reported no models ("cold start"), and that is how a deployment - // that had declared four models saw eighty-six: the fallback could not see the - // declaration, so it treated "no answer" as "no preference". A list this plugin - // is not the owner of is never widened — the same rule that withdrew the picker - // join, applied to the empty case. - // - // The cost is honest: a cold start with nothing discovered shows nothing until - // the adapter answers. Showing models the deployment did not select is the worse - // failure, because it looks like a working picker. - // **Discovery does NOT get the catalog either.** - // - // This briefly joined the whole catalog into `discoverModels`, reasoning that - // "browse to pick" is a different surface from "what this route serves". The - // owner's instruction had been explicit — *user can select them from the list, - // but we shouldn't override the user's config for what is usable* — and the - // join violated it: Settings listed models the deployment never declared, which - // is the same failure the picker join was withdrawn for. - // - // The original problem it was reaching for was narrower: pi's installed - // snapshot is a fixed vintage and can miss a model the ENDPOINT reports. That - // is answered by the adapter's own answer, not by this plugin's bundled copy. + // **Offering is not overriding.** An earlier revision seeded the whole + // catalog into the PICKER, and that is how a deployment that had declared + // four models saw eighty-six. The picker still lists a plane's declared + // models alone when it declares any (D3). What widens is Settings' fetch on + // a plane the plugin serves, which offers candidates for the user to adopt: + // nothing is saved until they do (D8), and that widening is + // {@link complementDiscoveredModels}, not this. return merged; }; /** - * Wrap the Host’s model-discovery method so “fetch available models” answers - * for claimed OpenCode routes include the canonical catalog’s missing rows - * and omit its provider-scoped retired rows. + * Enrich an adapter’s discovery answer from the catalog, honoring provider-scoped + * retirements. * - * The original method keeps ownership of errors, cancellation, and every - * unclaimed route. The returned disposer restores it. + * The adapter’s own rows keep their values and order; a field a row lacks is + * filled from the catalog, and every row is cut to the Remote's five fields. + * **Nothing is appended**: this is what a route the plugin does NOT serve gets + * — another pi-ai row's planes, a custom relay, a gateway-URL draft — and + * pi-ai, which serves those, cannot serve an id it does not list. A plane the + * plugin serves is complemented by {@link complementDiscoveredModels} instead. + */ +export const mergeDiscoveredModels = ( + discovered: unknown, + provider: CatalogProvider +): DiscoveredRow[] => adapterRows(discovered, provider, "pi-first"); + +/** + * Settings' fetch on a plane the plugin serves: pi's rows first, in pi's + * order, then every catalog model of the plane that is servable, not + * deprecated and missing from pi's answer, in catalog order (D3). + * + * Every row reads the catalog before pi, as the composer does (D17): the host + * editor's adopt copies a fetched row's capacities into the user's row, which + * the composer reads first, so a pi-first row would save pi's older numbers + * over the catalog's (Zen `claude-sonnet-4-5`: pi 200000, catalog 1000000). + * + * @param discovered - pi-ai's answer, or `[]` when it has none. + * @param plane - the catalog plane the route serves. + * @returns the rows, each id once, in the Remote's shape. + */ +export const complementDiscoveredModels = ( + discovered: unknown, + plane: CatalogProvider +): DiscoveredRow[] => { + const rows = adapterRows(discovered, plane, "catalog-first"); + const seen = new Set(rows.map((row) => row.id)); + const catalog = plane === "go" ? getLiveGoCatalog() : getLiveZenCatalog(); + for (const spec of catalog) { + if ( + seen.has(spec.id) || + !isServableSdk(spec.provider_npm) || + isDeprecatedModelOn(plane, spec.id) + ) { + continue; + } + seen.add(spec.id); + rows.push(rowFrom(undefined, spec, spec.id, "catalog-first")); + } + return rows; +}; + +/** What the wrapper reads from the route claim. */ +export type DiscoveryClaim = Pick; + +/** + * The catalog plane a discovery request is complemented from, or `undefined`. + * + * Only a plane route asked on the namespace of the pi-ai row the claim serves, + * and only while the claim has not stood down (from install on, before the + * mount settles, too — see {@link RouteClaim.claiming}): there, the plugin's + * adapter validates and serves a saved id whatever pi lists (D15). Anywhere + * else — after a stand-down, or on a second pi-ai row serving its own planes — + * pi-ai refuses at save an id it does not list, so offering one would offer a + * refusal. + */ +const servedPlane = ( + settingsNs: unknown, + request: unknown, + claim: DiscoveryClaim +): CatalogProvider | undefined => { + if ( + !claim.claiming() || + settingsNs !== claim.piAiNamespace() || + !isRecord(request) || + !isPlaneRoute(request.provider) + ) { + return undefined; + } + return request.provider === "opencode-go" ? "go" : "zen"; +}; + +/** + * Wrap the Host’s model-discovery method so Settings' fetch on a plane the + * plugin serves offers the catalog's models pi does not list (D3), and answers + * the catalog alone while pi-ai restarts instead of `NO_DISCOVERY`. + * + * Any other request the plugin can classify keeps the enrich-only merge, and + * everything else passes through. The original method keeps ownership of every + * other error and of cancellation. The returned disposer restores it. + * + * The wrapper never reads `signal`. On a plane route neither does pi-ai, which + * answers from its installed catalog, and the host throws `NO_DISCOVERY` before + * it looks at the signal; so a cancelled fetch resolves during the restart + * window exactly as it does outside it, and the Remote discards the answer. */ export const decorateModelDiscovery = ( ctx: CordisContext, - config: ResolvedPluginConfig + config: ResolvedPluginConfig, + claim: DiscoveryClaim ): (() => void) | undefined => { if (!config.enrichModels) { return undefined; @@ -215,21 +367,42 @@ export const decorateModelDiscovery = ( llm, "discoverModels", (discover) => - async ( - settingsNs: string, + async function discoverModels( + this: unknown, + settingsNs: unknown, request?: unknown, - signal?: AbortSignal - ): Promise => { - const discovered: unknown = await Reflect.apply(discover, llm, [ - settingsNs, - request, - signal, - ]); - const provider = resolveDiscoveryProvider(request, config); - if (provider === undefined) { - return discovered; + signal?: unknown + ): Promise { + const plane = servedPlane(settingsNs, request, claim); + let discovered: unknown; + try { + // The receiver is the caller's, as the host's own method would get + // it: the gateway calls through a view of the service. + discovered = await Reflect.apply(discover, this, [ + settingsNs, + request, + signal, + ]); + } catch (error) { + // pi-ai's discovery is withdrawn while it restarts. The catalog + // still answers for the plane, so the fetch does not fail on a + // window the user cannot see. + if ( + plane !== undefined && + isRecord(error) && + error.code === "NO_DISCOVERY" + ) { + return complementDiscoveredModels([], plane); + } + throw error; + } + if (plane !== undefined) { + return complementDiscoveredModels(discovered, plane); } - return mergeDiscoveredModels(discovered, provider); + const other = resolveDiscoveryProvider(request, config); + return other === undefined + ? discovered + : mergeDiscoveredModels(discovered, other); }, { id: PATCH_SITE.discovery } ); diff --git a/src/settings-copy.ts b/src/settings-copy.ts index e63d8007..699d1408 100644 --- a/src/settings-copy.ts +++ b/src/settings-copy.ts @@ -11,7 +11,7 @@ export const en = { "OpenCode Zen gateway origin headers, session affinity, free-tier compatibility, and live Go quota display.", enrichModels: "Enrich listings from the OpenCode catalog (default on)", enrichModelsHint: - "Lets the gateway's /models listing carry the full OpenCode catalog (models.opencode.ai — the same registry models.dev serves) with names, context limits and modalities, and offers that catalog to Settings' model discovery. It does NOT add models to the picker — the picker shows only what you declared; use \"Check catalog\" above for that. A declared model's own info (context, modalities, thinking levels) always comes from your model row and the catalog, whatever this says. Off leaves the gateway listing as the gateway sent it and offers no catalog to discovery.", + "Lets the gateway's /models listing carry the full OpenCode catalog (models.opencode.ai — the same registry models.dev serves) with names, context limits and modalities, and offers that catalog to Settings' model discovery. It does NOT add models to the picker by itself — the picker shows the models you declared, or the plane's catalog when its entry declares none, and this switch changes neither. To declare more, use Settings → Models → Fetch on an OpenCode plane: it offers pi's list plus the catalog's models pi lacks, and adopting a row and saving declares it. A declared model's own info (context, modalities, thinking levels) always comes from your model row and the catalog, whatever this says. Off leaves the gateway listing as the gateway sent it and offers no catalog to discovery.", endpoint: "API Endpoint", endpointAuto: "Automatic (same as Modern)", endpointHint: @@ -182,7 +182,7 @@ export const zh: Record = { "OpenCode Zen 网关来源头恢复、会话保持、免费模型兼容与 Go 实时额度显示。", enrichModels: "用 OpenCode 官方目录补全列表(默认开启)", enrichModelsHint: - "让网关 /models 列表带上 OpenCode 官方目录(models.opencode.ai,与 models.dev 是同一份注册表)里的完整目录,含名称、上下文上限与模态,并把这份目录提供给设置页的模型发现。它不会往选择器里添加模型——选择器只显示你声明的那些,要添加请用上面的「检查目录」。已声明模型自身的信息(上下文、模态、思考档位)始终来自你的模型行与目录,与此开关无关。关闭后网关列表保持原样,也不向模型发现提供目录。", + "让网关 /models 列表带上 OpenCode 官方目录(models.opencode.ai,与 models.dev 是同一份注册表)里的完整目录,含名称、上下文上限与模态,并把这份目录提供给设置页的模型发现。它本身不会往选择器里添加模型——选择器显示你声明的模型;条目未声明任何模型时显示该平面的目录,此开关不改变这两者。要声明更多,请在设置的模型页对 OpenCode 平面使用「获取可用模型」:它会提供 pi 的列表以及 pi 缺少的目录模型,选用一行并保存即完成声明。已声明模型自身的信息(上下文、模态、思考档位)始终来自你的模型行与目录,与此开关无关。关闭后网关列表保持原样,也不向模型发现提供目录。", injectCoreTools: "自动补全核心工具(默认开启)", injectCoreToolsHint: "在免费模型请求中自动注入 read 和 bash 工具声明以满足网关校验。留空沿用默认值。", diff --git a/test/catalog.test.ts b/test/catalog.test.ts index 6126dd17..9ba952fd 100644 --- a/test/catalog.test.ts +++ b/test/catalog.test.ts @@ -302,8 +302,10 @@ describe("OpenCode Model Catalog & Enrichment", () => { // **The catalog surface seeds, and that is correct here.** `discoverModels` // returns its installed catalog for a provider pi ships and only fetches an // endpoint for one it does not, so this listing is what the endpoint offers — - // the surface a user browses, not the list a picker shows. The picker's own - // path (`mergeDiscoveredModels`) still never widens. + // the surface a user browses, not the list a picker shows. The picker lists + // the plane profile's declared models (`planeProfileFor`, never widened); + // Settings' fetch on a served plane is what offers the catalog + // (`complementDiscoveredModels`). expect(json.data.length).toBe(OPENCODE_GO_CATALOG.length + 1); const deepseek = json.data.find((m) => m.id === "deepseek-v4.1-flash"); diff --git a/test/composition.test.ts b/test/composition.test.ts index ee7832f8..6fb9f5a1 100644 --- a/test/composition.test.ts +++ b/test/composition.test.ts @@ -26,16 +26,19 @@ import { Context } from "@deepseek-ai/cordis"; import { describe, expect, it } from "vitest"; -import { type CordisContext, apply, inject } from "./support/src.ts"; +import { + type CordisContext, + apply, + inject, + inspectPatch, +} from "./support/src.ts"; -/** The `llm` service, shaped as the plugin reads it, logging discovery registrations. */ -const llmService = (registered: string[] = []) => ({ +/** The `llm` service, shaped as the plugin reads it, with its own discovery. */ +const llmService = () => ({ + discoverModels: async (): Promise => [], listModels: async () => [], listProviders: () => [], registerAdapter: () => () => {}, - registerModelDiscovery: (ns: string) => { - registered.push(ns); - }, }); /** @@ -50,12 +53,11 @@ const mount = async ( options: { readonly withLlm?: boolean; readonly config?: Record; - readonly registered?: string[]; } = {} ) => { const ctx = new Context(); if (options.withLlm !== false) { - ctx.provide("llm", llmService(options.registered) as never); + ctx.provide("llm", llmService() as never); } let activated = false; const fiber = ctx.plugin({ @@ -79,15 +81,26 @@ describe("composition · a real cordis Context", () => { // need `llm` is reached through `ctx.inject`, and runs only once cordis // provides it. expect(inject).toEqual([]); - const registered: string[] = []; - const without = await mount({ registered, withLlm: false }); + const service = llmService(); + const original = service.discoverModels; + const without = await mount({ withLlm: false }); expect(without.activated, "the root waits for nothing").toBe(true); - expect(registered).toEqual([]); - without.ctx.provide("llm", llmService(registered) as never); + without.ctx.provide("llm", service as never); await new Promise((resolve) => { setTimeout(resolve, 10); }); - expect(registered).toContain("opencode"); + // The `llm` scope ran: Settings' discovery is wrapped on the service. + const installed = Reflect.getOwnPropertyDescriptor( + service, + "discoverModels" + )?.value; + expect(installed).not.toBe(original); + expect(inspectPatch(installed, "discoverModels")).toEqual([ + expect.objectContaining({ + id: "dsh-opencode-patch/model-discovery", + live: true, + }), + ]); }); it("leaves the purge service reachable through ctx.get", async () => { diff --git a/test/e2e/host-binary.e2e.ts b/test/e2e/host-binary.e2e.ts index cd2cd253..46c39632 100644 --- a/test/e2e/host-binary.e2e.ts +++ b/test/e2e/host-binary.e2e.ts @@ -15,15 +15,17 @@ * cannot drift apart unnoticed. Since P-R1 the plugin owns both plane routes * (D13): no row depends on which of pi-ai and the plugin applies first, so the * per-model rows are `S0_TODAY.rows` in either order, and the listing and save - * rows no longer depend on which ids pi describes. + * rows no longer depend on which ids pi describes. The Settings fetch rows (P3) + * are checked against this checkout's servable shim catalog, and against pi's + * own list in the pinned cell. * * Gated: it runs only with `DSH_HOST_E2E=1`, and needs `DSH_HOST_PREFIX`, a * directory whose `node_modules/@deepseek-ai/dsh` is the installed host. * `DSH_HOST_PI_CELL=pinned` (the default) also requires the pi * `host-versions.ts` names, with the harness patch, and asserts every row; * `floating` accepts whatever pi the host resolved, and skips the per-model - * rows (a newer pi's own entry can move a model's metadata); its probe JSON is - * the record. + * rows (a newer pi's own entry can move a model's metadata) and the Settings + * fetch's order (pi's list first); its probe JSON is the record. * * `DSH_HOST_PROBE_COPY` names a file to keep the probe's JSON in, for the CI * artifact; it is written whenever the probe wrote one, a failed run included. @@ -48,6 +50,7 @@ import { tmpdir } from "node:os"; import { join, relative, sep } from "node:path"; import { fileURLToPath } from "node:url"; +import { getBuiltinModels } from "@earendil-works/pi-ai/providers/all"; import { afterAll, beforeAll, describe, expect, it } from "vitest"; import { HOST_VERSIONS } from "../support/host-versions.ts"; @@ -55,11 +58,17 @@ import { CHARACTERIZED_MODELS, CHARACTERIZED_PROFILE, PI_AI_PACKAGE, + type Plane, PLANES, S0_TODAY, SAVE_CANDIDATE, ZEN_UNDESCRIBED, } from "../support/host.ts"; +import { + isServableSdk, + OPENCODE_GO_CATALOG, + OPENCODE_ZEN_CATALOG, +} from "../support/src.ts"; const ENABLED = process.env.DSH_HOST_E2E === "1"; const PREFIX = process.env.DSH_HOST_PREFIX ?? ""; @@ -127,8 +136,43 @@ interface ProbeOutput { dispatch: { sent: string[]; finish: string }; listed: unknown; }; + /** Settings → Models → Fetch per plane, through the gateway's Remote face. */ + readonly discovery: Record< + string, + | { ids: string[]; fields: string[]; undefinedFields: string[] } + | { error: { code?: string; message: string } } + >; } +/** The Remote's five `LlmDiscoveredModel` fields. */ +const REMOTE_FIELDS = [ + "contextWindow", + "id", + "inputModalities", + "maxTokens", + "name", +]; + +/** The plane's servable catalog ids, in catalog order, from this checkout. */ +const servable = (plane: Plane): string[] => + (plane === "opencode" ? OPENCODE_ZEN_CATALOG : OPENCODE_GO_CATALOG) + .filter((spec) => isServableSdk(spec.provider_npm)) + .map((spec) => spec.id); + +/** pi's own ids for a plane, once each, from the checkout's (pinned) pi. */ +const piIds = (plane: Plane): string[] => [ + ...new Set( + getBuiltinModels(plane as never).map((model: { id: string }) => model.id) + ), +]; + +/** The ids Settings' fetch offered on a plane; fails by name on an error. */ +const fetchedIds = (plane: Plane): string[] => { + const answered = out.discovery[plane]; + expect(answered, plane).toHaveProperty("ids"); + return answered !== undefined && "ids" in answered ? answered.ids : []; +}; + /** The entries of a directory, or none when it cannot be read. */ const entriesOf = (dir: string): string[] => { try { @@ -528,4 +572,42 @@ describe.skipIf(!ENABLED)("e2e · the published host binary (dsh)", () => { sent: ["https://opencode.ai/inference/openai/v1/chat/completions"], }); }); + + it("[BASELINE] Settings' fetch on each plane offers the whole servable catalog, each id once, in the Remote's row shape (D3)", () => { + for (const plane of PLANES) { + const ids = fetchedIds(plane); + expect(new Set(ids).size, plane).toBe(ids.length); + expect(ids, plane).toEqual(expect.arrayContaining(servable(plane))); + const answered = out.discovery[plane]; + const fields = + answered !== undefined && "fields" in answered ? answered.fields : []; + expect(fields, plane).toContain("id"); + expect(REMOTE_FIELDS, plane).toEqual(expect.arrayContaining(fields)); + // No key is present with an `undefined` value: the codec would hide it. + expect( + answered !== undefined && "undefinedFields" in answered + ? answered.undefinedFields + : undefined, + plane + ).toEqual([]); + } + expect(fetchedIds("opencode")).toContain("gpt-5.3-codex-spark"); + expect(fetchedIds("opencode-go")).toContain("minimax-m3.1-flash"); + expect(fetchedIds("opencode-go")).toContain("gemini-4-argon"); + }); + + // A newer pi lists gap ids itself and orders its own list differently, so + // only the pinned cell pins the order. + it.skipIf(!PINNED)( + "[BASELINE] Settings' fetch answers pi's own list first, then the catalog models pi lacks (pinned pi)", + () => { + for (const plane of PLANES) { + const pi = piIds(plane); + expect(fetchedIds(plane), plane).toEqual([ + ...pi, + ...servable(plane).filter((id) => !pi.includes(id)), + ]); + } + } + ); }); diff --git a/test/e2e/host/probe-plugin.mjs b/test/e2e/host/probe-plugin.mjs index f5ed3664..3e608e5b 100644 --- a/test/e2e/host/probe-plugin.mjs +++ b/test/e2e/host/probe-plugin.mjs @@ -4,9 +4,11 @@ * A bundle plugin with one job: once the published `dsh` binary has booted a * profile holding this plugin and `dsh-opencode-patch`, ask the host what it * now serves — the registry, the listings, the validation verdicts, where one - * turn per model is sent, what the settings page reads, and whether a - * catalog-unknown id can be saved — write that as JSON to `PROBE_OUT`, and exit - * the process. `test/e2e/host-binary.e2e.ts` reads the file. + * turn per model is sent, what the settings page reads, what Settings → + * Models → Fetch offers on each plane (through the gateway's Remote face), and + * whether a catalog-unknown id can be saved — write that as JSON to + * `PROBE_OUT`, and exit the process. `test/e2e/host-binary.e2e.ts` reads the + * file. * * It never reaches the network: `fetch` is replaced at module load, so every * request to `opencode.ai` or `models.opencode.ai` is recorded (URL and model @@ -308,12 +310,45 @@ const measure = async (ctx) => { registryRoutes: [...registry().keys()], }; const asked = await askAbout(llm, ids); - const zenNs = registered.configurable.find( - (entry) => entry.provider === "opencode" - )?.settingsNs; - const discovered = await settle(() => - llm.discoverModels(zenNs, { provider: "opencode" }) - ); + // Settings → Models → Fetch, as the Models page reaches it: the gateway + // reads the service from a fresh view and calls the descriptor's + // implementation, `remoteDiscoverModels` (the in-process contract pins that + // name), which reads `this.discoverModels` per call. Ids and field names + // only, never a value beyond the ids. + const llmView = ctx.extend({}).get("llm"); + const remote = Reflect.get(llmView, "remoteDiscoverModels"); + const discovery = {}; + for (const plane of PLANES) { + const settingsNs = registered.configurable.find( + (entry) => entry.provider === plane + )?.settingsNs; + const answered = await settle(() => + Reflect.apply(remote, llmView, [ + settingsNs, + { provider: plane }, + AbortSignal.timeout(15_000), + ]) + ); + discovery[plane] = answered.ok + ? { + fields: [ + ...new Set(answered.ok.flatMap((row) => Object.keys(row))), + ].toSorted((a, b) => a.localeCompare(b)), + ids: answered.ok.map((row) => row.id), + // A key present with no value is a shape the Remote's codec hides, + // so the record names it: the whitelist must never emit one. + undefinedFields: [ + ...new Set( + answered.ok.flatMap((row) => + Object.entries(row) + .filter(([, value]) => value === undefined) + .map(([key]) => key) + ) + ), + ].toSorted((a, b) => a.localeCompare(b)), + } + : answered; + } const dispatch = {}; for (const [route, models] of Object.entries(ids)) { for (const model of models) { @@ -385,7 +420,7 @@ const measure = async (ctx) => { value: Object.keys(piDescriptor.value?.providers ?? {}), }, describeEntries, - discovery: discovered.ok ? { count: discovered.ok.length } : discovered, + discovery, dispatch, modelsJoin, listModelsAfter, diff --git a/test/host-discovery.test.ts b/test/host-discovery.test.ts new file mode 100644 index 00000000..c042058d --- /dev/null +++ b/test/host-discovery.test.ts @@ -0,0 +1,573 @@ +/** + * Settings' model discovery on the real host (P3, D3, D8, D15, D21). + * + * The P3 rows of the plan's test matrix: what Settings → Models → Fetch + * answers on each plane, while pi-ai restarts, for a plane with no `models` + * list, and for a gap id adopted from it and saved; and the contract that the + * wrapper is what the gateway's own dispatch reaches, before and after a + * pi-ai restart and an `llm` restart. The fetch, restart, no-models-list, + * gap-save and contract cases each boot their own tier B and read the host's + * answers, the profile file and the wire — never the plugin's own state. The + * stood-down case mounts tier A with `hostModules: null`, which keeps the + * plugin's default host-module source: it cannot find pi-ai's module, so the + * route claim stands down, pi-ai keeps the planes, and the wrapper is shown + * reading that real claim. + * + * Every tier B case boots the published binary's apply order + * (`plugin-first`); the pi-ai restart that a `pi-ai-first` mount causes is the + * same withdrawal the restart case drives directly. The tier A stood-down case + * mounts pi-ai first, as tier A always does; with the claim stood down no + * restart follows (the claim restarts only a pi-ai lacking a declared plane, + * and there pi-ai resolved holding both), so the order is immaterial. + * + * The gateway (`@deepseek-ai/dsh-api-gateway`) is not installed here, so its + * dispatch is restated from dsh-llm's own typert descriptor: the service and + * the implementation it names, read from a fresh view of the context the way + * the gateway resolves a method per call, and the result codec the Remote + * encodes the answer with. + * + * @module test/host-discovery.test + */ +import { readFileSync } from "node:fs"; + +import type { Context } from "@deepseek-ai/cordis"; +import { TYPERT } from "@deepseek-ai/dsh-llm/typert"; +import { getBuiltinModels } from "@earendil-works/pi-ai/providers/all"; +import { afterEach, describe, expect, it, vi } from "vitest"; + +import { + bootTierB, + CHARACTERIZED_MODELS, + characterizedProviders, + drainStream, + fakeCredentials, + type HostLlm, + mountTierA, + ownerOf, + type Plane, + PLANES, + piKnows, + recordingFetch, + type RecordingFetch, + s0Providers, + type TierA, + type TierB, + waitForPlaneOwner, +} from "./support/host.ts"; +import { + inspectPatch, + isDeprecatedModelOn, + isRecord, + isServableSdk, + OPENCODE_GO_CATALOG, + OPENCODE_ZEN_CATALOG, + sanitizeModalities, +} from "./support/src.ts"; + +let tier: TierB | undefined; +let tierA: TierA | undefined; +let wire: RecordingFetch; + +afterEach(async () => { + try { + await tier?.dispose(); + await tierA?.dispose(); + } finally { + tier = undefined; + tierA = undefined; + vi.unstubAllGlobals(); + } +}); + +/** One discovered row, in the Remote's shape. */ +interface Row { + readonly id: string; + readonly name?: string; + readonly contextWindow?: number; + readonly maxTokens?: number; + readonly inputModalities?: string[]; +} + +/** One typert invocation descriptor, as far as the gateway reads it. */ +interface Invocation { + readonly id: string; + readonly service: string; + readonly implementation: string; + readonly result: { + readonly create: () => { parse: (value: unknown) => unknown }; + }; +} + +/** dsh-llm's descriptor for `llm/discoverModels`, the one the Models page calls. */ +const DISCOVER = ( + TYPERT as { readonly invocations: readonly Invocation[] } +).invocations.find( + (invocation) => invocation.id === "@deepseek-ai/dsh-llm#llm/discoverModels" +); + +const descriptor = (): Invocation => { + if (DISCOVER === undefined) { + throw new Error("dsh-llm's typert names no llm/discoverModels"); + } + return DISCOVER; +}; + +/** pi's own ids for a plane, once each, in pi's order. */ +const piIds = (plane: Plane): string[] => [ + ...new Set( + getBuiltinModels(plane as never).map((model: { id: string }) => model.id) + ), +]; + +/** The plane's servable catalog ids, in catalog order. */ +const servable = (plane: Plane): string[] => + (plane === "opencode" ? OPENCODE_ZEN_CATALOG : OPENCODE_GO_CATALOG) + .filter((spec) => isServableSdk(spec.provider_npm)) + .map((spec) => spec.id); + +/** + * pi's ids the wrapper keeps on a plane: it drops an id the live catalog has + * retired, on every path. Offline nothing is retired (the shim ships active + * models and the catalog refresh is refused), so this filter keeps them all; + * it states the wrapper's rule, not a difference this suite can observe. + */ +const piKept = (plane: Plane): string[] => + piIds(plane).filter( + (id) => !isDeprecatedModelOn(plane === "opencode" ? "zen" : "go", id) + ); + +/** What Settings' fetch on a served plane answers: pi's ids, then the catalog's others. */ +const expected = (plane: Plane): string[] => { + const pi = piKept(plane); + return [...pi, ...servable(plane).filter((id) => !pi.includes(id))]; +}; + +/** The settings namespace a plane is configured under, as the Models page reads it. */ +const nsOf = (llm: HostLlm, plane: Plane): string | undefined => + llm.listConfigurableProviders().find((entry) => entry.provider === plane) + ?.settingsNs; + +const idsOf = (rows: readonly { readonly id: string }[]): string[] => + rows.map((row) => row.id); + +/** + * The gateway's dispatch of `llm/discoverModels`, restated: the descriptor's + * service read from a fresh view of the context, and its implementation read + * from that view per call — which is how an instance wrapper is reached. + */ +const viaGateway = async ( + ctx: Context, + ns: string, + request: Record +): Promise<{ + answer: Row[]; + schema: { parse: (value: unknown) => unknown }; +}> => { + const invocation = descriptor(); + const view: unknown = ctx.extend({}).get(invocation.service as never); + if (typeof view !== "object" || view === null) { + throw new Error(`no ${invocation.service} service on the view`); + } + const method: unknown = Reflect.get(view, invocation.implementation); + if (typeof method !== "function") { + throw new TypeError(`no ${invocation.implementation} on the service`); + } + const answer = (await Reflect.apply(method, view, [ + ns, + request, + new AbortController().signal, + ])) as Row[]; + return { answer, schema: invocation.result.create() }; +}; + +/** Poll until `predicate` holds, or fail by name. */ +const until = async ( + predicate: () => boolean | Promise, + what: string, + deadlineMs = 10_000 +): Promise => { + const end = Date.now() + deadlineMs; + while (!(await predicate())) { + if (Date.now() > end) { + throw new Error(`never happened: ${what}`); + } + await new Promise((resolve) => { + setTimeout(resolve, 10); + }); + } +}; + +/** Boot tier B offline with a fresh wire recorder (stubbed before the boot). */ +const boot = async ( + providers: Record, + credentials = fakeCredentials({ + refs: { OPENCODE_API_KEY: "zen-k", OPENCODE_GO_API_KEY: "go-k" }, + }) +): Promise => { + wire = recordingFetch(); + vi.stubGlobal("fetch", wire.fetch); + tier = await bootTierB({ credentials, providers }); + return tier; +}; + +/** cordis `FiberState.UNLOADING`. */ +const UNLOADING = 5; + +describe( + "host discovery · Settings' fetch (tier B)", + { timeout: 30_000 }, + () => { + it("on each plane answers pi's own list, then every servable catalog model pi does not list, each id once, in the Remote's own result shape (D3)", async () => { + const { llm } = await boot(characterizedProviders()); + await waitForPlaneOwner(llm); + const schema = descriptor().result.create(); + const answers: Partial> = {}; + for (const plane of PLANES) { + const ns = nsOf(llm, plane); + expect(ns, plane).toBe("llm-pi-ai"); + const rows = (await llm.discoverModels(ns ?? "", { + provider: plane, + })) as Row[]; + expect(idsOf(rows), plane).toEqual(expected(plane)); + // Non-vacuous: the catalog has models pi does not list on this plane. + expect( + servable(plane).filter((id) => piKnows(plane, id) === undefined) + .length, + plane + ).toBeGreaterThan(0); + // The Remote's codec keeps every row as it is: no unknown key, and + // modalities are text/image only. + expect(schema.parse(rows), plane).toStrictEqual(rows); + // Every row the catalog describes carries the CATALOG's values, pi's + // rows included: the composer serves a plane model catalog-first + // (D17), and a row adopted from the fetch is saved as the user's row, + // which the composer reads before the catalog. + const catalog = + plane === "opencode" ? OPENCODE_ZEN_CATALOG : OPENCODE_GO_CATALOG; + for (const row of rows) { + const spec = catalog.find((model) => model.id === row.id); + if (spec === undefined) { + continue; + } + expect(row, `${plane}/${row.id}`).toStrictEqual({ + contextWindow: spec.context_window, + id: row.id, + inputModalities: sanitizeModalities(spec.input_modalities), + maxTokens: spec.max_output_tokens, + name: spec.name, + }); + } + answers[plane] = rows; + } + // Non-vacuous: pi knows a Zen model at a capacity the catalog does not + // (measured on pi 1.1.0: claude-sonnet-4-5 at 200000, catalog 1000000). + const differing = ( + getBuiltinModels("opencode" as never) as { + contextWindow?: number; + id: string; + }[] + ).filter((model) => { + const spec = OPENCODE_ZEN_CATALOG.find((each) => each.id === model.id); + return ( + spec !== undefined && model.contextWindow !== spec.context_window + ); + }); + expect(differing.length).toBeGreaterThan(0); + const zen = idsOf(answers.opencode ?? []); + const go = idsOf(answers["opencode-go"] ?? []); + expect(zen).toContain("gpt-5.3-codex-spark"); + expect(go).toContain("minimax-m3.1-flash"); + expect(go).toContain("gemini-4-argon"); + // A gap row is the CATALOG's row — never the user's saved + // `My Spark`/400000, which is configuration, not discovery. + const spec = OPENCODE_ZEN_CATALOG.find( + (model) => model.id === "gpt-5.3-codex-spark" + ); + expect( + answers.opencode?.find((row) => row.id === "gpt-5.3-codex-spark") + ).toStrictEqual({ + contextWindow: spec?.context_window, + id: "gpt-5.3-codex-spark", + inputModalities: sanitizeModalities(spec?.input_modalities), + maxTokens: spec?.max_output_tokens, + name: spec?.name, + }); + }); + + it("F21 — while pi-ai restarts answers the plane's catalog instead of NO_DISCOVERY, directly and through the gateway, and the full answer is back once it settles", async () => { + const current = await boot(characterizedProviders()); + await waitForPlaneOwner(current.llm); + const request = { provider: "opencode" }; + type Settled = { ok: unknown } | { error: unknown }; + const settled = (promise: Promise): Promise => + promise.then( + (ok) => ({ ok }), + (error: unknown) => ({ error }) + ); + let calls: + | { + control: Promise; + direct: Promise; + gateway: Promise<{ answer: Row[] }>; + } + | undefined; + const prototypeDiscover = ( + Reflect.getPrototypeOf(current.llm) as { + discoverModels: (...args: unknown[]) => Promise; + } + ).discoverModels; + const off = current.ctx.on("internal/status", (fiber, old) => { + // pi-ai's discovery is withdrawn between UNLOADING and the reload: the + // window is read synchronously, on the transition OUT of UNLOADING. + if ( + fiber !== current.piFiber || + old !== UNLOADING || + calls !== undefined + ) { + return; + } + calls = { + control: settled( + Reflect.apply(prototypeDiscover, current.llm, [ + "llm-pi-ai", + request, + ]) as Promise + ), + direct: current.llm.discoverModels( + "llm-pi-ai", + request + ) as unknown as Promise, + gateway: viaGateway(current.ctx, "llm-pi-ai", request), + }; + }); + try { + await current.piFiber.restart(); + } finally { + off(); + } + expect(calls, "the restart passed through UNLOADING").toBeDefined(); + // The host's own method, unwrapped, answers NO_DISCOVERY in this window: + // the window was hit. + const control = await calls?.control; + expect(control).toMatchObject({ error: { code: "NO_DISCOVERY" } }); + const direct = (await calls?.direct) ?? []; + expect(idsOf(direct)).toEqual(servable("opencode")); + const gateway = await calls?.gateway; + expect(idsOf(gateway?.answer ?? [])).toEqual(servable("opencode")); + await waitForPlaneOwner(current.llm); + expect( + idsOf((await current.llm.discoverModels("llm-pi-ai", request)) as Row[]) + ).toEqual(expected("opencode")); + }); + + it("a plane with no models list: the composer lists the plane's servable catalog keyless, Settings' fetch offers every id the composer lists, and the profile file is never written (D3, D8, D21)", async () => { + const current = await boot( + { + deepseek: s0Providers().deepseek, + opencode: { apiKeyEnv: "OPENCODE_API_KEY" }, + }, + fakeCredentials() + ); + await waitForPlaneOwner(current.llm, ["opencode"]); + const before = readFileSync(current.profile.patchPath); + const composer = idsOf(await current.llm.listModels("opencode")); + expect(composer).toEqual(servable("opencode")); + const fetched = idsOf( + (await current.llm.discoverModels(nsOf(current.llm, "opencode") ?? "", { + provider: "opencode", + })) as Row[] + ); + expect(fetched).toEqual(expect.arrayContaining(composer)); + expect(fetched).toEqual(expected("opencode")); + expect(readFileSync(current.profile.patchPath)).toEqual(before); + }); + + it("a gap row adopted from Settings' fetch saves like any other, and the composer lists, validates and sends it on the plugin's adapter (D15)", async () => { + const current = await boot(characterizedProviders()); + await waitForPlaneOwner(current.llm); + const rows = (await current.llm.discoverModels("llm-pi-ai", { + provider: "opencode", + })) as Row[]; + const gap = rows.find( + (row) => + piKnows("opencode", row.id) === undefined && + !CHARACTERIZED_MODELS.opencode.includes(row.id) + ); + expect(gap, "the fetch offers a model pi does not list").toBeDefined(); + if (gap === undefined) { + return; + } + // ModelListEditor's `adopt()`, restated: the row as the editor drafts it. + const adopted = { + id: gap.id, + ...(gap.name === undefined ? {} : { name: gap.name }), + ...(gap.contextWindow === undefined + ? {} + : { contextWindow: gap.contextWindow }), + ...(gap.maxTokens === undefined ? {} : { maxTokens: gap.maxTokens }), + ...(gap.inputModalities === undefined + ? {} + : { input: [...gap.inputModalities] }), + }; + // Before the save the composer does not list it: what lists it below is + // the save, not a composer widened to the catalog (D3, D8). + expect(idsOf(await current.llm.listModels("opencode"))).not.toContain( + gap.id + ); + expect(readFileSync(current.profile.patchPath, "utf8")).not.toContain( + gap.id + ); + await current.ctx.configEditor.edit(current.piEntry, (raw) => { + const providers = { + ...(raw.providers as Record>), + }; + const entry = providers.opencode ?? {}; + providers.opencode = { + ...entry, + models: [...((entry.models as unknown[]) ?? []), adopted], + }; + return { ...raw, providers }; + }); + expect(readFileSync(current.profile.patchPath, "utf8")).toContain(gap.id); + await until( + async () => + idsOf(await current.llm.listModels("opencode")).includes(gap.id), + "the adopted id is listed" + ); + await expect( + current.llm.resolveCallConfig({ + model: gap.id, + provider: "opencode", + } as never) + ).resolves.toBeDefined(); + expect(ownerOf(current.llm, "opencode")).toBe("PlaneAdapter"); + const sentBefore = wire.requests.length; + const drained = await drainStream(current.llm, "opencode", gap.id); + const sent = wire.requests + .slice(sentBefore) + .filter( + (request) => !request.url.startsWith("https://models.opencode.ai/") + ); + expect(sent).toHaveLength(1); + expect(sent[0]?.model).toBe(gap.id); + expect(sent[0]?.url.startsWith("https://opencode.ai/inference/")).toBe( + true + ); + expect(drained.finish?.code).toBe("AUTH"); + }); + } +); + +describe("host discovery · contract (tier B)", { timeout: 30_000 }, () => { + it("F21 — the wrapper is seen through the gateway's own dispatch and survives a pi-ai restart and an llm restart", async () => { + const current = await boot(characterizedProviders()); + await waitForPlaneOwner(current.llm); + // The descriptor names what the gateway calls: if a host renames it, this + // fails here before the binary probe, which calls it by name. + expect(descriptor().implementation).toBe("remoteDiscoverModels"); + expect(descriptor().service).toBe("llm"); + const request = { provider: "opencode" }; + const first = await viaGateway(current.ctx, "llm-pi-ai", request); + expect(idsOf(first.answer)).toEqual(expected("opencode")); + expect(first.schema.parse(first.answer)).toStrictEqual(first.answer); + + const site = (llm: HostLlm) => + inspectPatch( + Reflect.getOwnPropertyDescriptor(llm, "discoverModels")?.value, + "discoverModels" + ); + const live = [ + { + holdsWrapper: true, + id: "dsh-opencode-patch/model-discovery", + live: true, + }, + ]; + + await current.piFiber.restart(); + await waitForPlaneOwner(current.llm); + expect(site(current.llm)).toEqual(live); + const afterPiAi = await viaGateway(current.ctx, "llm-pi-ai", request); + expect(idsOf(afterPiAi.answer)).toEqual(expected("opencode")); + + const llmEntry = current.ctx.configEditor + .entries() + .find((entry) => entry.options.id === "llm"); + expect(llmEntry?.fiber).toBeDefined(); + await llmEntry?.fiber?.restart(); + await waitForPlaneOwner(current.ctx.llm); + expect(site(current.ctx.llm)).toEqual(live); + // Wait on pi-ai's own discovery, not on the wrapper: during NO_DISCOVERY + // the wrapper answers the catalog alone, which is non-empty, so a wait on + // the gateway's answer could stop before pi-ai re-registers. The host's + // unwrapped method rejects NO_DISCOVERY until it has. + const unwrapped = ( + Reflect.getPrototypeOf(current.ctx.llm) as { + discoverModels: (...args: unknown[]) => Promise; + } + ).discoverModels; + await until(async () => { + try { + await Reflect.apply(unwrapped, current.ctx.llm, ["llm-pi-ai", request]); + } catch (error) { + if (isRecord(error) && error.code === "NO_DISCOVERY") { + return false; + } + throw error; + } + return true; + }, "pi-ai's discovery is registered on the new llm"); + const viaNewLlm = await viaGateway(current.ctx, "llm-pi-ai", request); + expect(idsOf(viaNewLlm.answer)).toEqual(expected("opencode")); + }); +}); + +describe( + "host discovery · a stood-down claim (tier A)", + { timeout: 30_000 }, + () => { + it("with the claim stood down, Settings' fetch on a plane answers pi's own list, and no catalog model pi lacks (the real claim reaches the wrapper)", async () => { + wire = recordingFetch(); + vi.stubGlobal("fetch", wire.fetch); + // No loader and the plugin's default host-module source: it cannot find + // pi-ai's module, so the route claim stands down and pi-ai keeps the + // planes. The wrapper is still installed; it must read THIS claim. + tierA = await mountTierA({ + credentials: fakeCredentials({ + refs: { OPENCODE_API_KEY: "zen-k", OPENCODE_GO_API_KEY: "go-k" }, + }), + hostModules: null, + plugin: {}, + providers: characterizedProviders(), + }); + const { llm } = tierA; + await waitForPlaneOwner(llm, PLANES, "PiAiAdapter"); + expect( + inspectPatch( + Reflect.getOwnPropertyDescriptor(llm, "discoverModels")?.value, + "discoverModels" + ) + ).toEqual([ + { + holdsWrapper: true, + id: "dsh-opencode-patch/model-discovery", + live: true, + }, + ]); + for (const plane of PLANES) { + const pi = piKept(plane); + const rows = (await llm.discoverModels("llm-pi-ai", { + provider: plane, + })) as Row[]; + expect(idsOf(rows), plane).toEqual(pi); + // Non-vacuous: a live claim would have offered these. + const gap = servable(plane).filter((id) => !pi.includes(id)); + expect(gap.length, plane).toBeGreaterThan(0); + expect( + idsOf(rows).filter((id) => gap.includes(id)), + plane + ).toEqual([]); + } + }); + } +); diff --git a/test/lifecycle.test.ts b/test/lifecycle.test.ts index 5a0f86bf..16d6d966 100644 --- a/test/lifecycle.test.ts +++ b/test/lifecycle.test.ts @@ -18,6 +18,7 @@ import { clearSessionUsageStore, findModelSpecOn, getSessionUsage, + inspectPatch, } from "./support/src.ts"; import { collectUnknown, @@ -46,24 +47,36 @@ const withScope = (ctx: T): T => }); describe("apply (plugin lifecycle)", () => { - it("registers model discovery for both OpenCode routes by default", () => { - const registered: string[] = []; - const ctx = { + it("installs the Settings discovery wrapper in the llm scope by default, and none with enrichModels off", () => { + const original = async () => []; + const llm = { discoverModels: original }; + const ctx = withScope({ effect: () => {}, - llm: { - registerModelDiscovery: (ns: string) => { - registered.push(ns); - }, - }, + llm, on: () => {}, - } as unknown as CordisContext; + }) as unknown as CordisContext; + + void apply(ctx); + const installed = Reflect.getOwnPropertyDescriptor( + llm, + "discoverModels" + )?.value; + expect(installed).not.toBe(original); + expect(inspectPatch(installed, "discoverModels")?.[0]).toMatchObject({ + id: "dsh-opencode-patch/model-discovery", + live: true, + }); - void apply(withScope(ctx)); - // The component's own namespace plus one registration per gateway route, - // so DSH's picker can enumerate Go and Zen independently. - expect(registered).toContain("dsh-opencode-patch"); - expect(registered).toContain("opencode-go"); - expect(registered).toContain("opencode"); + const offLlm = { discoverModels: original }; + const off = withScope({ + effect: () => {}, + llm: offLlm, + on: () => {}, + }) as unknown as CordisContext; + void apply(off, { enrichModels: false }); + expect( + Reflect.getOwnPropertyDescriptor(offLlm, "discoverModels")?.value + ).toBe(original); }); it("registers the usage service only while the meter is on", () => { @@ -225,110 +238,24 @@ describe("apply (plugin lifecycle)", () => { expect(warned.join(" ")).toContain("neither ctx.plugin nor ctx.provide"); }); - it("answers discovery from the LATEST activation's settings after a reload", async () => { - // The host refuses a namespace that is still held, the way cordis does. The - // earlier shape re-registered on every apply, swallowed that refusal, and so - // kept answering with the first activation's settings for the process's life. - const discover = new Map Promise>(); - const ctx = { - effect: () => {}, - llm: { - registerModelDiscovery: (ns: string, fn: () => Promise) => { - if (discover.has(ns)) { - throw new Error(`DUPLICATE_DISCOVERY: ${ns}`); - } - discover.set(ns, fn); - return () => discover.delete(ns); - }, - }, - on: () => {}, - } as unknown as CordisContext; - const contextOf = async (id: string): Promise => { - const rows = (await discover.get("opencode")!()) as { - id: string; - contextWindow: number; - }[]; - return rows.find((row) => row.id === id)?.contextWindow; - }; - - void apply(withScope(ctx), { enrichModels: true }); - // The catalog's own window: the row carries no override of its own (D19). - const catalogWindow = findModelSpecOn( - "zen", - "claude-sonnet-4-5" - )?.context_window; - expect(catalogWindow).toBeGreaterThan(0); - expect(await contextOf("claude-sonnet-4-5")).toBe(catalogWindow); - - // Turning the feature off on a reload empties the answer rather than - // leaving the old catalog published. - void apply(withScope(ctx), { enrichModels: false }); - expect(await discover.get("opencode")!()).toEqual([]); - - // And on again: the second reload is answered too, not only the first. - void apply(withScope(ctx), { enrichModels: true }); - expect(await contextOf("claude-sonnet-4-5")).toBeGreaterThan(0); - expect([...discover.keys()].toSorted()).toEqual( - ["dsh-opencode-patch", "opencode", "opencode-go"].toSorted() - ); - }); - - it("registers each discovery namespace on its own, and says which one failed", () => { - // One `try` around all three meant a refusal of the first skipped the rest. - const registered: string[] = []; - const warn = vi.fn(); - const ctx = { - effect: () => {}, - llm: { - registerModelDiscovery: (ns: string) => { - if (ns === "dsh-opencode-patch") { - throw new Error("refused"); - } - registered.push(ns); - }, - }, - logger: { info: () => {}, warn }, - on: () => {}, - } as unknown as CordisContext; - - void apply(withScope(ctx)); - expect(registered.toSorted()).toEqual(["opencode", "opencode-go"]); - expect(warn).toHaveBeenCalledWith( - expect.stringContaining("model discovery for dsh-opencode-patch"), - expect.any(Error) - ); - }); - - it("registers no model discovery when enrichModels is off", () => { - const registered: string[] = []; - const ctx = { - effect: () => {}, - llm: { - registerModelDiscovery: (ns: string) => { - registered.push(ns); - }, - }, - on: () => {}, - } as unknown as CordisContext; - - void apply(withScope(ctx), { enrichModels: false }); - expect(registered).toEqual([]); - }); - - it("survives a host whose llm service has no model discovery", () => { + it("survives a host whose llm service has no discoverModels", () => { // Discovery is optional: an older host must still get the header patch - // and the stream hook rather than failing to mount. + // and the stream hook rather than failing to mount. The llm scope runs + // (`withScope`), so the discovery installer reads this service. let streamHandler: unknown; - const ctx = { + const llm = {}; + const ctx = withScope({ effect: () => {}, - llm: {}, + llm, on: (_event: string, handler: unknown) => { streamHandler = handler; }, - } as unknown as CordisContext; + }) as unknown as CordisContext; expect(() => apply(ctx)).not.toThrow(); expect(typeof streamHandler).toBe("function"); + // Nothing was installed on a service with no method to wrap. + expect(Reflect.ownKeys(llm)).toEqual([]); }); it("patches globalThis.fetch and restores it on disposer call", () => { diff --git a/test/models-discovery.test.ts b/test/models-discovery.test.ts index e177a73a..9b0c81b1 100644 --- a/test/models-discovery.test.ts +++ b/test/models-discovery.test.ts @@ -1,15 +1,15 @@ /** - * Provider-aware model-discovery decoration. + * Settings' model discovery, complemented by the catalog (P3, D3). * - * A configured OpenCode route can be answered from its adapter’s installed - * catalog without a gateway request. In that path, response enrichment never - * runs, so decorating the discovery answer is the only way to add a missing - * catalog row or remove a provider-retired row from “fetch available models.” + * Settings → Models → Fetch asks pi-ai's namespace, and pi-ai answers a plane + * from pi's packaged catalog. On a plane the plugin serves, the wrapper keeps + * pi's rows and order, then offers every servable, non-deprecated catalog model + * pi does not list, each id once and in the Remote's five fields; while pi-ai + * restarts (`NO_DISCOVERY`) it answers the catalog alone. Every other request + * it classifies keeps the enrich-only merge: pi-ai serves those routes, and it + * cannot serve an id it does not list. Nothing is written (D8). * - * The listing, model-info and plane-adapter decorations this file used to - * cover are gone (P-R1): the plane adapter answers those itself, and - * `test/host-characterization.test.ts` and `test/host-planes.test.ts` pin - * what it answers on the real host. + * `test/host-discovery.test.ts` pins the same answers on the real host. * * @module test/models-discovery.test */ @@ -17,13 +17,21 @@ import { afterEach, describe, expect, it, vi } from "vitest"; import { + complementDiscoveredModels, type CordisContext, decorateModelDiscovery, + type DiscoveryClaim, + findModelSpecOn, isDeprecatedModelOn, + isServableSdk, mergeDiscoveredModels, + OPENCODE_GO_CATALOG, + OPENCODE_ZEN_CATALOG, parseModelsDevCatalog, + refreshCatalog, resolveConfig, resolveDiscoveryProvider, + sanitizeModalities, } from "./support/src.ts"; afterEach(() => { @@ -35,6 +43,59 @@ const providers = new Set(["opencode", "opencode-go"]); const gatewayUrls = ["opencode.ai/zen"]; const config = { ...resolveConfig(), gatewayUrls, providers }; +/** A live claim on pi-ai's default row: the plugin serves both planes. */ +const SERVED: DiscoveryClaim = { + claiming: () => true, + piAiNamespace: () => "llm-pi-ai", +}; + +/** The claim after a stand-down: pi-ai serves the planes again. */ +const STOOD_DOWN: DiscoveryClaim = { + claiming: () => false, + piAiNamespace: () => "llm-pi-ai", +}; + +/** The five fields of the Remote's `LlmDiscoveredModel`, sorted. */ +const REMOTE_FIELDS = [ + "contextWindow", + "id", + "inputModalities", + "maxTokens", + "name", +]; + +/** The servable catalog ids of a plane's bundled shim, in catalog order. */ +const servableIds = ( + catalog: readonly { id: string; provider_npm?: string }[] +): string[] => + catalog.filter((spec) => isServableSdk(spec.provider_npm)).map((s) => s.id); + +/** Every row carries only the Remote's fields, and no value is `undefined`. */ +const expectWhitelisted = (rows: readonly object[]): void => { + for (const row of rows) { + for (const [key, value] of Object.entries(row)) { + expect(REMOTE_FIELDS, key).toContain(key); + expect(value, key).not.toBe(undefined); + } + } +}; + +/** Install the wrapper on a plain `llm` and hand back the installed method. */ +const install = ( + original: (...args: never[]) => unknown, + claim: DiscoveryClaim = SERVED +) => { + const ctx = { + llm: { discoverModels: original }, + } as unknown as CordisContext; + const stop = decorateModelDiscovery(ctx, resolveConfig(), claim); + const wrapped = ctx.llm?.discoverModels; + if (typeof wrapped !== "function") { + throw new TypeError("discovery decorator was not installed"); + } + return { ctx, stop, wrapped }; +}; + describe("resolveDiscoveryProvider", () => { it("prefers an explicitly claimed route ID", () => { expect(resolveDiscoveryProvider({ provider: "opencode" }, config)).toBe( @@ -123,7 +184,7 @@ describe("mergeDiscoveredModels", () => { expect(ids).toContain("adapter-only"); }); - it("keeps adapter rows, drops an empty id, and does NOT append the catalog", () => { + it("the enrich-only merge (a route the plugin does not serve) keeps adapter rows, drops an empty id, and appends nothing", () => { const merged = mergeDiscoveredModels( [ { @@ -139,15 +200,17 @@ describe("mergeDiscoveredModels", () => { "zen" ); - // **Exactly what the adapter reported.** The catalog supplies metadata, not - // rows: a refresh must not widen the deployment's list, so `beam-free` - // — catalogued, servable, and NOT declared here — stays out. + // **Exactly what the adapter reported.** This merge is what a route the + // plugin does NOT serve gets — another pi-ai row's planes, a custom relay, + // a gateway-URL draft — and pi-ai serves those, so it cannot serve an id it + // did not list. `mimo-v2.6-flash-free`, catalogued and servable, stays out. expect(merged.map((model) => model.id)).toEqual([ "adapter-model", "beam-free", ]); - // The adapter’s own capacities win over the canonical catalog. - expect(merged[0]).toEqual({ + // The adapter’s own capacities win over the canonical catalog, and the + // row is the Remote's shape: nothing added, nothing `undefined`. + expect(merged[0]).toStrictEqual({ contextWindow: 1, id: "adapter-model", inputModalities: ["text"], @@ -162,43 +225,94 @@ describe("mergeDiscoveredModels", () => { "go" ); - // Kept — it is not retired — and the catalog appends nothing. + // Kept — it is not retired — and the enrich-only merge appends nothing. expect(merged.map((model) => model.id)).toEqual([ "muse-spark-1.2-contributor", ]); }); - it("sanitizes candidate modalities to DSH-supported text and image only", () => { - const merged = mergeDiscoveredModels( - [ - { - id: "adapter-multimodal", - inputModalities: ["text", "image", "video", "audio", "pdf"], - }, - ], - "go" - ); - - // Unsupported modalities are stripped from the adapter row, and the catalog - // row is not appended — the adapter answered. - expect(merged).toEqual([ - { - contextWindow: undefined, - id: "adapter-multimodal", - inputModalities: ["text", "image"], - maxTokens: undefined, - name: undefined, - }, + it("a row is whitelisted to the Remote's five fields: no extra key, no undefined value", () => { + // Unsupported modalities are stripped, and a field nobody supplied is + // ABSENT rather than `undefined`: the Remote reads an absent modality list + // as "unknown", which is not the same as "text". + expect( + mergeDiscoveredModels( + [ + { + id: "adapter-multimodal", + inputModalities: ["text", "image", "video", "audio", "pdf"], + }, + ], + "go" + ) + ).toStrictEqual([ + { id: "adapter-multimodal", inputModalities: ["text", "image"] }, ]); + // A key the Remote does not carry never leaves the plugin. + expect( + mergeDiscoveredModels( + [ + { + api: "openai-responses", + cost: { input: 1 }, + id: "adapter-x", + reasoning: true, + }, + ], + "zen" + ) + ).toStrictEqual([{ id: "adapter-x" }]); }); - it("adds nothing when the adapter reported nothing", () => { - // An empty answer is an ABSENCE, and the previous reading of it — "no - // preference, so the catalog stands in" — is how a deployment that had - // declared four models saw eighty-six. The merge enriches what it is given and - // never widens it, in this case as in every other. - const merged = mergeDiscoveredModels([], "zen"); - expect(merged).toEqual([]); + it("a supplied modality list with no text or image entry leaves the field absent, not text", () => { + // An audio-only list says nothing about text: the row reads "unknown", as + // the composer's plane model does for the same spec, and the Remote's codec + // accepts the absent field. + expect( + mergeDiscoveredModels( + [{ id: "adapter-audio", inputModalities: ["audio", "video"] }], + "zen" + ) + ).toStrictEqual([{ id: "adapter-audio" }]); + expect( + mergeDiscoveredModels( + [{ id: "adapter-empty", inputModalities: [] }], + "zen" + ) + ).toStrictEqual([{ id: "adapter-empty" }]); + }); + + it("the enrich-only merge of an empty answer is empty; a served plane's complement of it is the servable catalog", () => { + // The enrich-only merge enriches what it is given and never widens it: an + // empty answer from a route the plugin does not serve stays empty. + expect(mergeDiscoveredModels([], "zen")).toEqual([]); + // A plane the plugin serves is complemented instead (D3): with pi's list + // empty, the answer is the catalog's servable models. + expect(complementDiscoveredModels([], "zen").map((row) => row.id)).toEqual( + servableIds(OPENCODE_ZEN_CATALOG) + ); + }); + + it("drops a row whose fields are not the Remote's types, and a repeated id; a non-list answer is no rows", () => { + expect( + mergeDiscoveredModels( + [ + { id: "kept" }, + { id: "kept", name: "Second" }, + { id: "bad-name", name: "" }, + { contextWindow: "1", id: "bad-window" }, + { id: "bad-max", maxTokens: "2" }, + { id: "bad-modalities", inputModalities: "text" }, + { id: "bad-modality", inputModalities: [1] }, + { id: 7 }, + null, + ], + "zen" + ).map((row) => row.id) + ).toEqual(["kept"]); + expect(mergeDiscoveredModels({ models: [{ id: "kept" }] }, "zen")).toEqual( + [] + ); }); it("enriches candidate rows missing contextWindow or maxTokens from the catalog spec", () => { @@ -230,7 +344,11 @@ describe("decorateModelDiscovery", () => { } as unknown as CordisContext; expect( - decorateModelDiscovery(ctx, resolveConfig({ enrichModels: false })) + decorateModelDiscovery( + ctx, + resolveConfig({ enrichModels: false }), + SERVED + ) ).toBe(undefined); expect(ctx.llm?.discoverModels).toBe(original); }); @@ -241,7 +359,9 @@ describe("decorateModelDiscovery", () => { llm: Object.freeze({ discoverModels: original }), } as unknown as CordisContext; - expect(decorateModelDiscovery(ctx, resolveConfig())).toBe(undefined); + expect(decorateModelDiscovery(ctx, resolveConfig(), SERVED)).toBe( + undefined + ); expect(ctx.llm?.discoverModels).toBe(original); }); @@ -251,7 +371,7 @@ describe("decorateModelDiscovery", () => { const ctx = { llm: { discoverModels: original }, } as unknown as CordisContext; - const stop = decorateModelDiscovery(ctx, resolveConfig()); + const stop = decorateModelDiscovery(ctx, resolveConfig(), SERVED); const wrapped = ctx.llm?.discoverModels; expect(typeof wrapped).toBe("function"); if (typeof wrapped !== "function") { @@ -270,51 +390,265 @@ describe("decorateModelDiscovery", () => { expect(ctx.llm?.discoverModels).toBe(original); }); - it("keeps the 1.2 row the zen plane serves and adds nothing on refresh", async () => { + it("Settings' fetch on a plane the plugin serves answers pi's rows first, then every servable, non-deprecated catalog model pi does not list, each id once", async () => { vi.stubGlobal( "fetch", vi.fn().mockRejectedValue(new Error("offline catalog test")) ); - const original = vi - .fn() - .mockResolvedValue([{ id: "beam-free", name: "Beam Free" }]); - const ctx = { - llm: { discoverModels: original }, - } as unknown as CordisContext; - const stop = decorateModelDiscovery(ctx, resolveConfig()); - const wrapped = ctx.llm?.discoverModels; - expect(typeof wrapped).toBe("function"); - if (typeof wrapped !== "function") { - throw new TypeError("discovery decorator was not installed"); - } + const original = vi.fn().mockResolvedValue([ + { id: "beam-free", name: "Beam Free" }, + { + contextWindow: 7, + id: "mimo-v2.6-flash-free", + inputModalities: ["text"], + maxTokens: 3, + name: "pi's MiMo", + }, + { + contextWindow: 11, + id: "pi-only-model", + inputModalities: ["text", "image"], + maxTokens: 5, + name: "pi's Own", + }, + ]); + const { ctx, stop, wrapped } = install(original); - const merged = (await wrapped("llm-pi-ai", { + const rows = (await wrapped("llm-pi-ai", { baseURL: "https://opencode.ai/zen/v1", provider: "opencode", - })) as Array<{ id: string }>; - const ids = merged.map((model) => model.id); + })) as { id: string }[]; expect(original).toHaveBeenCalledWith( "llm-pi-ai", { baseURL: "https://opencode.ai/zen/v1", provider: "opencode" }, undefined ); - // **Discovery does NOT join the catalog.** It briefly did — reasoning that - // "browse to pick" is a different surface from "what this route serves" — and - // the owner's instruction had been explicit: *user can select them from the - // list, but we shouldn't override the user's config for what is usable*. The - // join made Settings list models the deployment never declared. - // - // The refresh adds nothing the adapter did not answer. The free model the - // catalog knows is `mimo-v2.6-flash-free` — a DIFFERENT one from the row the - // adapter answered, so the assertion is about the join and not about a name. - expect(ids).not.toContain("mimo-v2.6-flash-free"); - expect(ids).toHaveLength(1); - // Kept, not removed: it answers on zen, so it is not retired. - expect(ids).toContain("beam-free"); + // pi's three rows in pi's order, then the catalog's servable rows pi + // lacks, in catalog order — `mimo-v2.6-flash-free` once, not twice (D3). + expect(rows.map((row) => row.id)).toEqual([ + "beam-free", + "mimo-v2.6-flash-free", + "pi-only-model", + ...servableIds(OPENCODE_ZEN_CATALOG).filter( + (id) => id !== "beam-free" && id !== "mimo-v2.6-flash-free" + ), + ]); + // The CATALOG's values win on a plane the plugin serves, in the composer's + // own order (D17: the user's row, then the catalog, then pi): a row + // adopted from the fetch is saved as the user's row, and it must carry + // what the composer already serves, not pi's older capacities. + const spec = findModelSpecOn("zen", "mimo-v2.6-flash-free"); + expect(spec).toBeDefined(); + expect(spec?.context_window).not.toBe(7); + expect(rows[1]).toStrictEqual({ + contextWindow: spec?.context_window, + id: "mimo-v2.6-flash-free", + inputModalities: sanitizeModalities(spec?.input_modalities), + maxTokens: spec?.max_output_tokens, + name: spec?.name, + }); + // pi fills what the catalog lacks: an id it does not describe keeps pi's. + expect(findModelSpecOn("zen", "pi-only-model")).toBeUndefined(); + expect(rows[2]).toStrictEqual({ + contextWindow: 11, + id: "pi-only-model", + inputModalities: ["text", "image"], + maxTokens: 5, + name: "pi's Own", + }); + expectWhitelisted(rows); stop?.(); expect(ctx.llm?.discoverModels).toBe(original); }); + it("NO_DISCOVERY on the served pi-ai row's namespace (pi-ai mid-restart) answers the plane's catalog alone", async () => { + const failure = Object.assign( + new Error('no model discovery is registered for "llm-pi-ai"'), + { code: "NO_DISCOVERY" } + ); + const original = vi.fn().mockRejectedValue(failure); + const { wrapped } = install(original); + + const rows = (await wrapped("llm-pi-ai", { + provider: "opencode-go", + })) as { id: string }[]; + expect(original).toHaveBeenCalledTimes(1); + expect(rows.map((row) => row.id)).toEqual(servableIds(OPENCODE_GO_CATALOG)); + expect(rows.length).toBeGreaterThan(0); + expectWhitelisted(rows); + }); + + it("NO_DISCOVERY with an already-aborted signal still answers the catalog: the wrapper never reads the signal", async () => { + // The host throws NO_DISCOVERY before it looks at the signal, and pi-ai's + // plane discovery answers from its installed catalog without reading it + // either; the restart window resolves as the steady state does. + const signal = AbortSignal.abort(); + const original = vi + .fn() + .mockRejectedValue( + Object.assign(new Error("none"), { code: "NO_DISCOVERY" }) + ); + const { wrapped } = install(original); + + const rows = (await wrapped( + "llm-pi-ai", + { provider: "opencode" }, + signal + )) as { id: string }[]; + expect(original).toHaveBeenCalledTimes(1); + expect(original).toHaveBeenCalledWith( + "llm-pi-ai", + { provider: "opencode" }, + signal + ); + expect(rows.map((row) => row.id)).toEqual( + servableIds(OPENCODE_ZEN_CATALOG) + ); + }); + + it("NO_DISCOVERY anywhere the plugin does not serve a plane still rejects as the host sent it", async () => { + const failure = Object.assign( + new Error('no model discovery is registered for "other-ns"'), + { code: "NO_DISCOVERY" } + ); + const original = vi.fn().mockRejectedValue(failure); + const { wrapped } = install(original); + + // Another namespace, a route that is not a plane, and a draft with no + // route: none of them is the served row's plane. + await expect(wrapped("other-ns", { provider: "opencode" })).rejects.toBe( + failure + ); + await expect(wrapped("llm-pi-ai", { provider: "deepseek" })).rejects.toBe( + failure + ); + await expect( + wrapped("llm-pi-ai", { baseURL: "https://opencode.ai/zen/v1" }) + ).rejects.toBe(failure); + // After a stand-down pi-ai serves the planes again: its error is its own. + const { wrapped: stoodDown } = install(original, STOOD_DOWN); + await expect(stoodDown("llm-pi-ai", { provider: "opencode" })).rejects.toBe( + failure + ); + }); + + it("a route the plugin does not serve keeps pi's own values over the catalog's: pi-ai serves it, in its own order", async () => { + // pi-ai serves this route and reads pi's row, never the catalog: the + // enrich-only merge fills only what pi's row omits. + const spec = findModelSpecOn("go", "minimax-m3"); + expect(spec?.context_window).not.toBe(7); + const original = vi + .fn() + .mockResolvedValue([ + { contextWindow: 7, id: "minimax-m3", maxTokens: 3 }, + ]); + const { wrapped } = install(original, STOOD_DOWN); + expect( + await wrapped("llm-pi-ai", { provider: "opencode-go" }) + ).toStrictEqual([ + { + contextWindow: 7, + id: "minimax-m3", + inputModalities: sanitizeModalities(spec?.input_modalities), + maxTokens: 3, + name: spec?.name, + }, + ]); + }); + + it("a plane on a pi-ai row the plugin does not serve keeps pi's own list: enriched, never widened", async () => { + const original = vi.fn().mockResolvedValue([{ id: "minimax-m3" }]); + // A second pi-ai row serves its own planes, and pi-ai refuses at save an + // id it does not list: offering the catalog there would offer a refusal. + const { wrapped } = install(original); + // Enriched: the row pi listed, its missing fields filled from the Go + // catalog's spec, and nothing appended. + const spec = findModelSpecOn("go", "minimax-m3"); + expect(spec).toBeDefined(); + const enriched = [ + { + contextWindow: spec?.context_window, + id: "minimax-m3", + inputModalities: sanitizeModalities(spec?.input_modalities), + maxTokens: spec?.max_output_tokens, + name: spec?.name, + }, + ]; + const second = await wrapped("llm-pi-ai:second", { + provider: "opencode-go", + }); + expect(second).toStrictEqual(enriched); + // The same after a stand-down, on the default row. + const { wrapped: stoodDown } = install(original, STOOD_DOWN); + const after = await stoodDown("llm-pi-ai", { + provider: "opencode-go", + }); + expect(after).toStrictEqual(enriched); + }); + + it("reads the claim per call: a mid-life stand-down or a renamed pi-ai row moves the complement with it", async () => { + // One install, one mutable claim: the wrapper must not sample the claim + // at install, nor assume pi-ai's default namespace. + let live = true; + let ns = "llm-pi-ai:custom"; + const claim: DiscoveryClaim = { + claiming: () => live, + piAiNamespace: () => ns, + }; + const original = vi.fn().mockResolvedValue([{ id: "minimax-m3" }]); + const { wrapped } = install(original, claim); + const idsOn = async (namespace: string): Promise => + ( + (await wrapped(namespace, { provider: "opencode-go" })) as { + id: string; + }[] + ).map((row) => row.id); + const complement = [ + "minimax-m3", + ...servableIds(OPENCODE_GO_CATALOG).filter((id) => id !== "minimax-m3"), + ]; + + // The served row is the claim's namespace, not the host default. + expect(await idsOn("llm-pi-ai:custom")).toEqual(complement); + expect(await idsOn("llm-pi-ai")).toEqual(["minimax-m3"]); + ns = "llm-pi-ai"; + expect(await idsOn("llm-pi-ai")).toEqual(complement); + expect(await idsOn("llm-pi-ai:custom")).toEqual(["minimax-m3"]); + // The claim stands down: the same wrapper now answers pi's list only. + live = false; + expect(await idsOn("llm-pi-ai")).toEqual(["minimax-m3"]); + }); + + it("on a served plane the original keeps every error but NO_DISCOVERY, and cancellation, as it threw them", async () => { + const signal = AbortSignal.abort(); + const invalid = Object.assign(new Error("boom"), { + code: "INVALID_DISCOVERY", + }); + const aborted = new DOMException("aborted", "AbortError"); + const thrown = "a thrown string"; + for (const failure of [invalid, aborted, thrown]) { + const original = vi.fn().mockRejectedValue(failure); + const { wrapped } = install(original); + await expect( + wrapped("llm-pi-ai", { provider: "opencode" }, signal) + ).rejects.toBe(failure); + expect(original).toHaveBeenCalledTimes(1); + expect(original).toHaveBeenCalledWith( + "llm-pi-ai", + { provider: "opencode" }, + signal + ); + } + }); + + it("a call with no request passes the original's answer through unchanged", async () => { + const answer = [{ api: "x", id: "adapter-model" }]; + const original = vi.fn().mockResolvedValue(answer); + const { wrapped } = install(original); + await expect(wrapped("llm-pi-ai")).resolves.toBe(answer); + expect(original).toHaveBeenCalledWith("llm-pi-ai", undefined, undefined); + }); + it("calls the host's method with the service as its receiver", async () => { // The real registry is a class whose `discoverModels` reads its own // discoveries through `this`, so a wrapper that calls it detached fails @@ -330,7 +664,8 @@ describe("decorateModelDiscovery", () => { const llm = new DiscoveryHost(); const stop = decorateModelDiscovery( { llm } as unknown as CordisContext, - resolveConfig() + resolveConfig(), + SERVED ); await expect( @@ -339,12 +674,44 @@ describe("decorateModelDiscovery", () => { stop?.(); }); + it("forwards the caller's receiver, as the describe graft does", async () => { + // The gateway calls the method through a view of the service, so the + // receiver is the caller's and not the object the wrapper was set on. + class DiscoveryHost { + readonly discoveries = new Map([ + ["llm-pi-ai", async () => [{ id: "installed-model" }]], + ]); + async discoverModels(ns: string, _request?: unknown): Promise { + return this.discoveries.get(ns)?.(); + } + } + const llm = new DiscoveryHost(); + const stop = decorateModelDiscovery( + { llm } as unknown as CordisContext, + resolveConfig(), + SERVED + ); + const view = { + discoveries: new Map([["llm-pi-ai", async () => [{ id: "view-model" }]]]), + }; + + const installed: unknown = Reflect.get(llm, "discoverModels"); + if (typeof installed !== "function") { + throw new TypeError("discovery decorator was not installed"); + } + await expect( + Reflect.apply(installed, view, ["llm-pi-ai", { provider: "other" }]) + ).resolves.toEqual([{ id: "view-model" }]); + stop?.(); + }); + it("leaves a wrapper installed on top of it in place on dispose", () => { const original = async () => [{ id: "adapter-model" }]; const llm: Record = { discoverModels: original }; const stop = decorateModelDiscovery( { llm } as unknown as CordisContext, - resolveConfig() + resolveConfig(), + SERVED ); const foreign = async () => []; llm.discoverModels = foreign; @@ -353,3 +720,35 @@ describe("decorateModelDiscovery", () => { expect(llm.discoverModels).toBe(foreign); }); }); + +describe("complementDiscoveredModels", () => { + it("a catalog model the registry retired since the shim, or whose SDK nothing serves, is not offered; pi's retired row is dropped too", async () => { + const retired = servableIds(OPENCODE_ZEN_CATALOG)[0] ?? ""; + expect(retired.length).toBeGreaterThan(0); + await refreshCatalog( + vi.fn().mockResolvedValue( + Response.json({ + opencode: { + models: { + [retired]: { status: "deprecated" }, + "cohere-x": { + name: "Cohere X", + provider: { npm: "@ai-sdk/cohere" }, + }, + }, + }, + }) + ) as unknown as typeof fetch, + true + ); + // The refresh landed both facts the complement has to honour. + expect(isDeprecatedModelOn("zen", retired)).toBe(true); + expect(findModelSpecOn("zen", "cohere-x")?.provider_npm).toBe( + "@ai-sdk/cohere" + ); + + expect( + complementDiscoveredModels([{ id: retired }], "zen").map((row) => row.id) + ).toEqual(servableIds(OPENCODE_ZEN_CATALOG).filter((id) => id !== retired)); + }); +}); diff --git a/test/reload.test.ts b/test/reload.test.ts index bd9da911..3d82533b 100644 --- a/test/reload.test.ts +++ b/test/reload.test.ts @@ -45,13 +45,13 @@ import { } from "./support/host.ts"; import { apply, + complementDiscoveredModels, Config, type CordisContext, decorateModelDiscovery, inject, inspectPatch, installDescribeGraft, - mergeDiscoveredModels, name, type ResolvedPluginConfig, resolveConfig, @@ -234,13 +234,19 @@ const installGraft = }; }; +/** A live claim on pi-ai's default row, as the discovery wrapper reads it. */ +const SERVED = { + claiming: () => true, + piAiNamespace: () => "llm-pi-ai", +}; + const SITES: readonly Site[] = [ { - decorated: mergeDiscoveredModels([{ id: "minimax-m3" }], "go"), + decorated: complementDiscoveredModels([{ id: "minimax-m3" }], "go"), host: (shape) => shaped(shape, { discoverModels: async () => [{ id: "minimax-m3" }] }), install: (llm, config = resolveConfig()) => - decorateModelDiscovery(asContext(llm), config), + decorateModelDiscovery(asContext(llm), config, SERVED), keys: ["discoverModels"], name: "discovery", probe: (llm) => diff --git a/test/route-claim.test.ts b/test/route-claim.test.ts index 617c126c..b85f315f 100644 --- a/test/route-claim.test.ts +++ b/test/route-claim.test.ts @@ -181,6 +181,7 @@ describe("route-claim · standing down", () => { await expect(claim.mounted).resolves.toBeUndefined(); expect(claim.committedEntries()).toEqual({}); expect(claim.piAiNamespace()).toBe("llm-pi-ai"); + expect(claim.claiming()).toBe(false); expect(warn).toHaveBeenCalledWith( expect.stringContaining("offers no on, inject, effect") ); @@ -193,13 +194,16 @@ describe("route-claim · standing down", () => { }); const pi = piFiber(); const host = standIn({ fibers: () => [pi] }); - await installRouteClaim(host.ctx, config).mounted; + const claim = installRouteClaim(host.ctx, config); + await claim.mounted; await flush(); expect(host.llm.registerAdapter).not.toHaveBeenCalled(); expect(pi.restart).toHaveBeenCalledTimes(1); expect(host.warn).toHaveBeenCalledWith( expect.stringContaining("unavailable; the plane routes stay pi-ai's") ); + // Stood down: pi-ai takes the planes and their saves again. + expect(claim.claiming()).toBe(false); // Inert from here: the listener passes through. const next = vi.fn(() => "next"); expect( @@ -687,6 +691,7 @@ describe("route-claim · reconcile", () => { const claim = installRouteClaim(host.ctx, config); await claim.mounted; await flush(); + expect(claim.claiming()).toBe(true); expect(host.info).toHaveBeenCalledWith( expect.stringContaining("still held elsewhere") ); @@ -800,13 +805,16 @@ describe("route-claim · teardown", () => { useTestModules(); const pi = piFiber(); const host = standIn({ fibers: () => [pi] }); - await installRouteClaim(host.ctx, config).mounted; + const claim = installRouteClaim(host.ctx, config); + await claim.mounted; await flush(); expect(host.llm.registerAdapter).toHaveBeenCalledTimes(1); + expect(claim.claiming()).toBe(true); const [teardown] = host.cleanups; expect(teardown?.()).toBeUndefined(); expect(host.handle).toHaveBeenCalledTimes(1); expect(pi.restart).not.toHaveBeenCalled(); + expect(claim.claiming()).toBe(false); }); it("unsubscribes from the catalog on a reload too, so a reloaded activation leaves nothing behind", async () => { diff --git a/test/support/src.ts b/test/support/src.ts index 2b3f0479..e66cba45 100644 --- a/test/support/src.ts +++ b/test/support/src.ts @@ -211,7 +211,9 @@ export { // models-discovery.ts export { + complementDiscoveredModels, decorateModelDiscovery, + type DiscoveryClaim, mergeDiscoveredModels, resolveDiscoveryProvider, } from "../../src/models-discovery.ts";