fix(models): provider-scoped allow-list + real capabilities for relay-fronted models - #28
Merged
Merged
Conversation
…r collision
Two providers can expose the same bare model id — e.g. a relay like LiteLLM
behind an OpenAI-compatible gateway re-listing `deepseek-v4-flash`, which then
appears under both DeepSeek and Qwen. The activation/entitlement allow-list was
keyed by bare model id (tenant_quotas.allowed_models / users.allowed_models,
the UI activation Set, and every runtime match site), so removing the model
from one provider silently removed it from the other.
Key the allow-list by `${provider}:${model}` instead. Provider ids never
contain ':', so the first colon is an unambiguous split even for model ids that
contain one (e.g. bedrock `...v2:0`).
- entitlements: add makeModelEntitlementKey + composite/legacy-bare matchers
(modelEntitlementAllowed, modelAllowedForAnyProvider); isModelAllowed /
assertModelAllowed take an optional providerId.
- model-router: pass selection.provider into assertModelAllowed.
- provider-failover: filter the entitled list with the composite matcher via
entry.provider.
- application-routes: buildProviderModels allowed calc + append loop,
validateKnownModelGrant now understand composite keys.
- UI: activation state keyed by modelActivationKey(provider, model); toggle,
save, add-manual, counts and render all provider-scoped.
No DB/DDL migration: the column stays a JSON string[]; format is extended and
reads stay backward-compatible with legacy bare-id lists (a bare entry matches
any provider until the operator next saves, which rewrites in composite form).
Tests: pure matcher unit tests (entitlement-key.test.ts) + a two-provider
same-id UI regression test proving removal no longer cross-deletes. Backend
DB-integration tests could not run in this environment (local Node 26 is
outside the repo's <26 engines range and better-sqlite3 fails to build against
it) — wiring for the DB-resolution path is unverified at runtime here.
Co-authored-by: Mozi <MoziAI-co@users.noreply.github.com>
… shipped registry resolveRuntimeModel returned a blanket conservative profile (context 32k, max output 4k, no tools, no vision) for any model id not in the built-in catalog. Relays like LiteLLM / OpenRouter / corporate gateways front real models (claude-sonnet-4, gpt-4o, deepseek-*) under custom ids, so a relay-fronted 200k-context vision model was silently capped to 32k with vision routing off — the degradation the earlier "no tools" diagnosis actually pointed at. MOZI already downloads LiteLLM's own price/context registry (model-registry-enrichment) but only consulted it in the API display layer, not in the live resolver that feeds the brain's context window / max-output sizing (handler) and vision routing (model-router). Wire getCachedModelMetadata into resolveRuntimeModel's unknown-model branch so every consumer that resolves through it gets the real context window, tool/vision support and pricing. Conservative profile still applies when the registry has no entry (a genuinely private id) or the cache is cold. No schema, UI, or migration. Test: relay-model-enrichment.test.ts seeds a temp registry cache and asserts a relay-fronted id resolves to its real caps, with conservative fallback intact. Co-authored-by: Mozi <MoziAI-co@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two related fixes for running MOZI behind a relay (LiteLLM / OpenRouter / corporate gateway).
1. Cross-provider allow-list collision (data bug)
A relay re-lists another provider's model id — e.g.
deepseek-v4-flashunder both DeepSeek and Qwen. The allow-list was keyed by bare model id (tenant_quotas.allowed_models,users.allowed_models, the UI activationSet, and every runtime match site), so removing the model from one provider silently removed it from the other. Reported first-hand.Fix: key the allow-list by
`${provider}:${model}`(provider ids never contain:, so the first colon splits unambiguously even for bedrock-style...v2:0ids).entitlements:makeModelEntitlementKey+ composite/legacy-bare matchers;isModelAllowed/assertModelAllowedtakeproviderId.model-routerpassesselection.provider;provider-failoverfilters viaentry.provider.application-routes:buildProviderModels+validateKnownModelGrantunderstand composite keys.modelActivationKey(provider, model)throughout.No DB/DDL migration — the column stays a JSON
string[]; format is extended and reads stay backward-compatible with legacy bare-id lists (a bare entry matches any provider until the operator next saves, which rewrites in composite form).2. Relay-fronted models lost their real capabilities
resolveRuntimeModelreturned a blanket conservative profile (context 32k, max output 4k, no tools, no vision) for any id not in the built-in catalog — so a relay-fronted 200k-context vision model was silently capped and had vision routing disabled.MOZI already downloads LiteLLM's own price/context registry (
model-registry-enrichment) but only used it in the API display layer, not in the live resolver that feeds the brain's context sizing (handler) and vision routing (model-router).Fix: wire
getCachedModelMetadataintoresolveRuntimeModel's unknown-model branch, so every consumer that resolves through it gets the model's real context window, tool/vision support and pricing. Conservative profile still applies when the registry has no entry (a private id) or the cache is cold. No schema, UI, or migration.Tests
entitlement-key.test.ts— composite matcher isolation + legacy compat + colon-in-id. ✅SettingsView.test.tsx— two-provider same-id regression: removing DeepSeek's copy keeps Qwen's; saved list is["dashscope:deepseek-v4-flash"]. ✅ 24/24relay-model-enrichment.test.ts— relay-fronted id resolves to real caps from a seeded registry; conservative fallback intact. ✅Not verified here
Backend DB-integration tests could not run locally: this machine's Node is v26, outside the repo's
<26engines range, andbetter-sqlite3fails to build against Node 26's V8. The DB-resolution path is wiring-unverified at runtime on this machine — confirm in CI / a supported Node.