Skip to content

feat: add native Gemini passthrough to LiteLLM sidecar#29

Open
KillerQueen-Z wants to merge 1 commit into
mainfrom
fix/gemini-native-proxy
Open

feat: add native Gemini passthrough to LiteLLM sidecar#29
KillerQueen-Z wants to merge 1 commit into
mainfrom
fix/gemini-native-proxy

Conversation

@KillerQueen-Z

Copy link
Copy Markdown
Collaborator

What changed

  • add native Gemini generateContent and streamGenerateContent routes to the BlockRun LiteLLM sidecar
  • preserve Gemini request/response JSON, native error envelopes, and SSE frames while the sidecar handles Base or Solana x402 payment
  • derive streaming from the Gemini URL method instead of an OpenAI-style JSON stream field
  • strip client Google credentials (x-goog-api-key, Authorization, and query parameters) before forwarding
  • record the native Gemini model and /v1beta/models/... path in local usage telemetry
  • document raw REST and official google-genai client setup
  • bump the package version to 0.8.0

Why

The Solana gateway already supports Gemini's native protocol, but blockrun-litellm only exposed OpenAI-shaped chat and native Anthropic HTTP routes. Customers using the sidecar therefore could not point a Gemini-native client at it without translating the request themselves.

Customer impact

After upgrading blockrun-litellm, customers can point the Google Gen AI SDK custom base URL at the sidecar root and keep their existing native Gemini request/response shapes. The in-process litellm.completion(...) provider remains OpenAI-compatible and is unchanged.

Validation

  • full suite: 299 passed, 2 skipped
  • raw local sidecar -> Solana production E2E:
    • non-streaming native JSON returned HTTP 200 and native-ok
    • streaming native SSE returned HTTP 200 and stream-ok
  • official google-genai 2.13.0 -> local sidecar -> Solana production E2E:
    • non-streaming returned sdk-native-ok
    • streaming returned sdk-stream-ok
  • verified the local Solana wallet signed and settled all paid requests
  • git diff --check and Python bytecode compilation pass

@KillerQueen-Z
KillerQueen-Z marked this pull request as ready for review July 22, 2026 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant