Skip to content

fix(models): provider-scoped allow-list + real capabilities for relay-fronted models - #28

Merged
spytensor merged 2 commits into
mainfrom
feat/relay-model-capabilities
Aug 5, 2026
Merged

fix(models): provider-scoped allow-list + real capabilities for relay-fronted models#28
spytensor merged 2 commits into
mainfrom
feat/relay-model-capabilities

Conversation

@spytensor

@spytensor spytensor commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Two related fixes for running MOZI behind a relay (LiteLLM / OpenRouter / corporate gateway).

1. Cross-provider allow-list collision (data bug)

A relay re-lists another provider's model id — e.g. deepseek-v4-flash under both DeepSeek and Qwen. The allow-list was keyed by bare model id (tenant_quotas.allowed_models, users.allowed_models, the UI activation Set, and every runtime match site), so removing the model from one provider silently removed it from the other. Reported first-hand.

Fix: key the allow-list by `${provider}:${model}` (provider ids never contain :, so the first colon splits unambiguously even for bedrock-style ...v2:0 ids).

  • entitlements: makeModelEntitlementKey + composite/legacy-bare matchers; isModelAllowed/assertModelAllowed take providerId.
  • model-router passes selection.provider; provider-failover filters via entry.provider.
  • application-routes: buildProviderModels + validateKnownModelGrant understand composite keys.
  • UI: activation state keyed by modelActivationKey(provider, model) throughout.

No DB/DDL migration — the column stays a JSON string[]; format is extended and reads stay backward-compatible with legacy bare-id lists (a bare entry matches any provider until the operator next saves, which rewrites in composite form).

2. Relay-fronted models lost their real capabilities

resolveRuntimeModel returned a blanket conservative profile (context 32k, max output 4k, no tools, no vision) for any id not in the built-in catalog — so a relay-fronted 200k-context vision model was silently capped and had vision routing disabled.

MOZI already downloads LiteLLM's own price/context registry (model-registry-enrichment) but only used it in the API display layer, not in the live resolver that feeds the brain's context sizing (handler) and vision routing (model-router).

Fix: wire getCachedModelMetadata into resolveRuntimeModel's unknown-model branch, so every consumer that resolves through it gets the model's real context window, tool/vision support and pricing. Conservative profile still applies when the registry has no entry (a private id) or the cache is cold. No schema, UI, or migration.

Tests

  • entitlement-key.test.ts — composite matcher isolation + legacy compat + colon-in-id. ✅
  • SettingsView.test.tsx — two-provider same-id regression: removing DeepSeek's copy keeps Qwen's; saved list is ["dashscope:deepseek-v4-flash"]. ✅ 24/24
  • relay-model-enrichment.test.ts — relay-fronted id resolves to real caps from a seeded registry; conservative fallback intact. ✅
  • Typecheck clean on backend + UI; touched provider/model tests green (54).

Not verified here

Backend DB-integration tests could not run locally: this machine's Node is v26, outside the repo's <26 engines range, and better-sqlite3 fails to build against Node 26's V8. The DB-resolution path is wiring-unverified at runtime on this machine — confirm in CI / a supported Node.

spytensor and others added 2 commits August 5, 2026 22:45
…r collision

Two providers can expose the same bare model id — e.g. a relay like LiteLLM
behind an OpenAI-compatible gateway re-listing `deepseek-v4-flash`, which then
appears under both DeepSeek and Qwen. The activation/entitlement allow-list was
keyed by bare model id (tenant_quotas.allowed_models / users.allowed_models,
the UI activation Set, and every runtime match site), so removing the model
from one provider silently removed it from the other.

Key the allow-list by `${provider}:${model}` instead. Provider ids never
contain ':', so the first colon is an unambiguous split even for model ids that
contain one (e.g. bedrock `...v2:0`).

- entitlements: add makeModelEntitlementKey + composite/legacy-bare matchers
  (modelEntitlementAllowed, modelAllowedForAnyProvider); isModelAllowed /
  assertModelAllowed take an optional providerId.
- model-router: pass selection.provider into assertModelAllowed.
- provider-failover: filter the entitled list with the composite matcher via
  entry.provider.
- application-routes: buildProviderModels allowed calc + append loop,
  validateKnownModelGrant now understand composite keys.
- UI: activation state keyed by modelActivationKey(provider, model); toggle,
  save, add-manual, counts and render all provider-scoped.

No DB/DDL migration: the column stays a JSON string[]; format is extended and
reads stay backward-compatible with legacy bare-id lists (a bare entry matches
any provider until the operator next saves, which rewrites in composite form).

Tests: pure matcher unit tests (entitlement-key.test.ts) + a two-provider
same-id UI regression test proving removal no longer cross-deletes. Backend
DB-integration tests could not run in this environment (local Node 26 is
outside the repo's <26 engines range and better-sqlite3 fails to build against
it) — wiring for the DB-resolution path is unverified at runtime here.

Co-authored-by: Mozi <MoziAI-co@users.noreply.github.com>
… shipped registry

resolveRuntimeModel returned a blanket conservative profile (context 32k, max
output 4k, no tools, no vision) for any model id not in the built-in catalog.
Relays like LiteLLM / OpenRouter / corporate gateways front real models
(claude-sonnet-4, gpt-4o, deepseek-*) under custom ids, so a relay-fronted
200k-context vision model was silently capped to 32k with vision routing off —
the degradation the earlier "no tools" diagnosis actually pointed at.

MOZI already downloads LiteLLM's own price/context registry
(model-registry-enrichment) but only consulted it in the API display layer, not
in the live resolver that feeds the brain's context window / max-output sizing
(handler) and vision routing (model-router). Wire getCachedModelMetadata into
resolveRuntimeModel's unknown-model branch so every consumer that resolves
through it gets the real context window, tool/vision support and pricing.
Conservative profile still applies when the registry has no entry (a genuinely
private id) or the cache is cold. No schema, UI, or migration.

Test: relay-model-enrichment.test.ts seeds a temp registry cache and asserts a
relay-fronted id resolves to its real caps, with conservative fallback intact.

Co-authored-by: Mozi <MoziAI-co@users.noreply.github.com>
@spytensor spytensor changed the title fix(models): scope model allow-list by provider (stop cross-provider collision) fix(models): provider-scoped allow-list + real capabilities for relay-fronted models Aug 5, 2026
@spytensor
spytensor merged commit 7911d88 into main Aug 5, 2026
6 of 7 checks passed
@spytensor
spytensor deleted the feat/relay-model-capabilities branch August 5, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant