Summary
With a google-vertex-ai provider, non-Anthropic models (e.g. Gemini) are unreachable regardless of how the model ID is spelled. Three behaviors interact to make every spelling fail:
- The router always overwrites the request body's
model field with the configured route's model, so the sandbox cannot pick a model itself (obj.insert("model", route.model) in crates/openshell-router/src/backend.rs). The body therefore always carries the model ID exactly as configured.
- Non-Anthropic Vertex models are routed to Vertex's OpenAI-compatible endpoint (
https://{region}-aiplatform.googleapis.com/v1beta1/projects/{project}/locations/{region}/endpoints/openapi/chat/completions). That endpoint requires the body's model to carry a publisher prefix, e.g. google/gemini-2.5-flash. A bare model name is rejected with HTTP 400 "Malformed publisher model".
validate_vertex_model_id (crates/openshell-server/src/inference.rs) rejects / in model IDs (path-traversal guard), so a publisher-prefixed model ID cannot be configured in the first place.
The result is a deadlock: gemini-2.5-flash passes validation but gets a 400 from Vertex; google/gemini-2.5-flash would be accepted by Vertex but is rejected by the CLI/config validation.
Notably, infer_vertex_publisher() already infers google as the publisher for gemini-* model names, but its result is never applied to the model field in the OpenAI-compatible request body.
Anthropic models are unaffected because they use the native :rawPredict path, where the model name is placed in the URL path rather than the body.
Reproduction
openshell provider create --name v --type google-vertex-ai # any valid credential
openshell inference set --provider v --model gemini-2.5-flash
# → HTTP 400 from Vertex: "Malformed publisher model"
openshell inference set --provider v --model google/gemini-2.5-flash
# → rejected by CLI validation (slash not allowed in model ID)
Versions: openshell 0.0.72 (also confirmed the same structure in current main sources). Observed 2026-07-19.
Suggested fix
When building the request body for the non-Anthropic (OpenAI-compatible) Vertex route, prepend the publisher prefix to the model name using the result of infer_vertex_publisher() (or the VERTEX_AI_PUBLISHER config value when set), e.g. gemini-2.5-flash → google/gemini-2.5-flash. This keeps the existing model ID validation intact while producing the format the endpoint requires.
Workaround (verified)
Create an openai-type provider and point OPENAI_BASE_URL at the Vertex OpenAI-compatible endpoint (the /v1 variant, e.g. https://{region}-aiplatform.googleapis.com/v1/projects/{project}/locations/{region}/endpoints/openapi). With that provider type, google/gemini-2.5-flash is accepted as the model name and requests go through.
Related: #2063, #2039
Summary
With a
google-vertex-aiprovider, non-Anthropic models (e.g. Gemini) are unreachable regardless of how the model ID is spelled. Three behaviors interact to make every spelling fail:modelfield with the configured route's model, so the sandbox cannot pick a model itself (obj.insert("model", route.model)incrates/openshell-router/src/backend.rs). The body therefore always carries the model ID exactly as configured.https://{region}-aiplatform.googleapis.com/v1beta1/projects/{project}/locations/{region}/endpoints/openapi/chat/completions). That endpoint requires the body'smodelto carry a publisher prefix, e.g.google/gemini-2.5-flash. A bare model name is rejected with HTTP 400"Malformed publisher model".validate_vertex_model_id(crates/openshell-server/src/inference.rs) rejects/in model IDs (path-traversal guard), so a publisher-prefixed model ID cannot be configured in the first place.The result is a deadlock:
gemini-2.5-flashpasses validation but gets a 400 from Vertex;google/gemini-2.5-flashwould be accepted by Vertex but is rejected by the CLI/config validation.Notably,
infer_vertex_publisher()already infersgoogleas the publisher forgemini-*model names, but its result is never applied to themodelfield in the OpenAI-compatible request body.Anthropic models are unaffected because they use the native
:rawPredictpath, where the model name is placed in the URL path rather than the body.Reproduction
Versions: openshell 0.0.72 (also confirmed the same structure in current
mainsources). Observed 2026-07-19.Suggested fix
When building the request body for the non-Anthropic (OpenAI-compatible) Vertex route, prepend the publisher prefix to the model name using the result of
infer_vertex_publisher()(or theVERTEX_AI_PUBLISHERconfig value when set), e.g.gemini-2.5-flash→google/gemini-2.5-flash. This keeps the existing model ID validation intact while producing the format the endpoint requires.Workaround (verified)
Create an
openai-type provider and pointOPENAI_BASE_URLat the Vertex OpenAI-compatible endpoint (the/v1variant, e.g.https://{region}-aiplatform.googleapis.com/v1/projects/{project}/locations/{region}/endpoints/openapi). With that provider type,google/gemini-2.5-flashis accepted as the model name and requests go through.Related: #2063, #2039