Repository navigation
fix(embeddings): require explicit opt-in and compatible models for local-to-API fallback #52
Description
Finding
When local FastEmbed initialization fails, an explicitly requested local provider becomes an API provider, retaining the local-only model name. There is no explicit fallback opt-in or usable-model validation.
Revision: 41ec093a745f1c0277893dc8f758b96b216c970f.
Evidence and scope
The existing runtime has cached sparse BM25 assets but no approved cached BAAI/bge-small-en-v1.5 dense assets. An offline TextEmbedding(..., local_files_only=True) check fails because those model assets are absent. Missing assets alone are an environment prerequisite, not this application bug.
Separately, a deterministic initialization fault probe patched fastembed.TextEmbedding to raise ValueError("injected missing approved local model asset"), then called:
init_embeddings(provider="local", dense_model="BAAI/bge-small-en-v1.5")Observed effective configuration:
requested provider: local
effective provider: api
effective model: BAAI/bge-small-en-v1.5
No gateway embedding request or source data was sent by this fault probe.
Relevant code and expected outcome
init_embeddings() catches local-model initialization errors, sets EMBEDDING_PROVIDER="api", and initializes an API client without changing/validating the model.
- Respect an explicitly configured local-only mode: fail visibly when its required assets/runtime are unavailable.
- If API fallback is a supported feature, require explicit opt-in, an approved compatible API model, and visible configured/effective provider state.
- Do not treat initialization of an API client as proof that the retained local model is available at the gateway.
- Add offline/missing-assets initialization tests that assert no unexpected outbound embedding request and clear failure status.
Activity
spelech commentedon Oct 27, 2026
Triage & Implementation Brief
1. Scope & Effort
app/services/embeddings.py)2. Root Cause & Context
In
app/services/embeddings.py::init_embeddings(), when local FastEmbed initialization fails (e.g. missing cached model assets), the exception handler silently setsEMBEDDING_PROVIDER = "api"while retaining the local model name (BAAI/bge-small-en-v1.5), causing unexpected outbound requests to fail.3. Implementation Plan
EMBEDDING_PROVIDER="local"and local assets are missing, raise a clearRuntimeErrordescribing the missing assets.ALLOW_EMBEDDING_API_FALLBACK=true).text-embedding-3-small) rather than passing the local model name to the API gateway.4. Acceptance Criteria
google-labs-jules commentedon Oct 27, 2026
Jules is on it. When finished, you will see another comment and be able to review a PR.
google-labs-jules commentedon Oct 27, 2026
Ready for a review! A PR has been created.