Skip to content

google-vertex-ai provider type cannot route to Gemini models — publisher-prefix requirement conflicts with model ID validation #2351

Description

@todraco

Summary

With a google-vertex-ai provider, non-Anthropic models (e.g. Gemini) are unreachable regardless of how the model ID is spelled. Three behaviors interact to make every spelling fail:

  1. The router always overwrites the request body's model field with the configured route's model, so the sandbox cannot pick a model itself (obj.insert("model", route.model) in crates/openshell-router/src/backend.rs). The body therefore always carries the model ID exactly as configured.
  2. Non-Anthropic Vertex models are routed to Vertex's OpenAI-compatible endpoint (https://{region}-aiplatform.googleapis.com/v1beta1/projects/{project}/locations/{region}/endpoints/openapi/chat/completions). That endpoint requires the body's model to carry a publisher prefix, e.g. google/gemini-2.5-flash. A bare model name is rejected with HTTP 400 "Malformed publisher model".
  3. validate_vertex_model_id (crates/openshell-server/src/inference.rs) rejects / in model IDs (path-traversal guard), so a publisher-prefixed model ID cannot be configured in the first place.

The result is a deadlock: gemini-2.5-flash passes validation but gets a 400 from Vertex; google/gemini-2.5-flash would be accepted by Vertex but is rejected by the CLI/config validation.

Notably, infer_vertex_publisher() already infers google as the publisher for gemini-* model names, but its result is never applied to the model field in the OpenAI-compatible request body.

Anthropic models are unaffected because they use the native :rawPredict path, where the model name is placed in the URL path rather than the body.

Reproduction

openshell provider create --name v --type google-vertex-ai   # any valid credential
openshell inference set --provider v --model gemini-2.5-flash
# → HTTP 400 from Vertex: "Malformed publisher model"

openshell inference set --provider v --model google/gemini-2.5-flash
# → rejected by CLI validation (slash not allowed in model ID)

Versions: openshell 0.0.72 (also confirmed the same structure in current main sources). Observed 2026-07-19.

Suggested fix

When building the request body for the non-Anthropic (OpenAI-compatible) Vertex route, prepend the publisher prefix to the model name using the result of infer_vertex_publisher() (or the VERTEX_AI_PUBLISHER config value when set), e.g. gemini-2.5-flashgoogle/gemini-2.5-flash. This keeps the existing model ID validation intact while producing the format the endpoint requires.

Workaround (verified)

Create an openai-type provider and point OPENAI_BASE_URL at the Vertex OpenAI-compatible endpoint (the /v1 variant, e.g. https://{region}-aiplatform.googleapis.com/v1/projects/{project}/locations/{region}/endpoints/openapi). With that provider type, google/gemini-2.5-flash is accepted as the model name and requests go through.

Related: #2063, #2039

Metadata

Metadata

Assignees

No one assigned

    Labels

    state:triage-neededOpened without agent diagnostics and needs triage

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions