feat: add native Gemini passthrough to LiteLLM sidecar

CHANGELOG.md

@@ -1,5 +1,46 @@
 # Changelog
 
+## 0.8.0 — 2026-07-22
+
+### Added
+
+- **Gemini's native `v1beta` protocol now works through the local sidecar.**
+  `POST /v1beta/models/{model}:generateContent` and
+  `POST /v1beta/models/{model}:streamGenerateContent` are forwarded verbatim
+  to BlockRun while the sidecar handles Base or Solana x402 payment locally.
+  This lets a customer keep Gemini-native request/response bodies and SSE
+  frames instead of translating through OpenAI Chat Completions.
+
+- Streaming is selected from the Gemini URL method, matching Google's wire
+  protocol (there is no OpenAI-style `"stream": true` field to inspect).
+  Upstream errors keep their native HTTP status and Gemini error envelope.
+
+- Client Google credentials are never forwarded. Google SDKs may require a
+  dummy API key and attach it as `x-goog-api-key` or `?key=...`; the sidecar
+  removes both because BlockRun authenticates with the local x402 wallet and
+  the gateway owns the upstream Google credential.
+
+- The model segment is sanitized before forwarding: `models/`/`google/`
+  prefixes are stripped and the path is rebuilt from the clean id, so a crafted
+  model name cannot move the signed upstream target outside `/v1beta/models/`.
+  Unsupported methods and invalid model ids are rejected with a `400` — and, per
+  the 0.7.6 every-exit-logs guarantee, still write an audit row.
+
+### Availability
+
+- Native Gemini passthrough is **enterprise/allowlist-only on the Base gateway**
+  (non-allowlisted payers receive `403 PERMISSION_DENIED` and are not charged);
+  it is generally available on Solana (`sol.blockrun.ai`). Retail callers on
+  Base should use `/v1/chat/completions` with `model="google/gemini-3.1-pro"`.
+  Only `generateContent`/`streamGenerateContent` are proxied.
+
+### Compatibility
+
+- Existing OpenAI, Anthropic, image, video, and audio routes are unchanged.
+- Native Gemini is available in sidecar/proxy mode. The in-process LiteLLM
+  custom provider remains OpenAI-shaped because it returns LiteLLM response
+  objects rather than exposing an HTTP protocol surface.
+
 ## 0.7.6 — 2026-07-16
 
 ### Fixed

README.md

@@ -361,6 +361,8 @@ curl http://localhost:4001/v1/chat/completions \
 | Method | Path | Notes |
 |---|---|---|
 | `POST` | `/v1/chat/completions` | OpenAI Chat Completions. `stream=True` returns `text/event-stream`; otherwise JSON. |
+| `POST` | `/v1beta/models/{model}:generateContent` | Native Gemini JSON request and response, with automatic x402 payment. |
+| `POST` | `/v1beta/models/{model}:streamGenerateContent` | Native Gemini SSE, with automatic x402 payment. |
 | `POST` | `/v1/responses` | OpenAI Responses API, bridged onto Chat Completions (`input`→`messages`, `output`/`response.*` SSE out). Text-in/text-out; for advanced tool/state flows use `/v1/chat/completions`. |
 | `POST` | `/v1/images/generations` | OpenAI Image Generations. Accepts `prompt`, `model`, `size`, `n`, and `quality` (Solana only — see below). |
 | `POST` | `/v1/images/edits` | OpenAI-compatible image editing. Accepts JSON data URIs or multipart `image`/`image[]`; supports multiple source images, `mask`, and `quality` (Solana only). `/v1/images/image2image` is an alias. |
@@ -375,6 +377,75 @@ curl http://localhost:4001/v1/chat/completions \
 | `GET`  | `/healthz` | Liveness probe (no upstream call) |
 | `GET`  | `/docs` | Auto-generated Swagger UI |
 
+### 2e. Native Gemini protocol
+
+Native Gemini calls use the sidecar root (`http://localhost:4001`), not the
+OpenAI `/v1` base. The sidecar preserves Gemini request/response JSON and SSE
+frames and adds the x402 payment signature using the configured wallet.
+
+```bash
+# Non-streaming
+curl http://localhost:4001/v1beta/models/gemini-2.5-flash:generateContent \
+  -H "Content-Type: application/json" \
+  -d '{"contents":[{"role":"user","parts":[{"text":"Reply with native-ok"}]}]}'
+
+# Streaming
+curl -N 'http://localhost:4001/v1beta/models/gemini-2.5-flash:streamGenerateContent?alt=sse' \
+  -H "Content-Type: application/json" \
+  -d '{"contents":[{"role":"user","parts":[{"text":"Reply with stream-ok"}]}]}'
+```
+
+The official Google Gen AI Python SDK can use the same native protocol. Keep
+the SDK itself; point its custom base URL at the sidecar and use any non-empty
+placeholder API key (the sidecar removes it before forwarding):
+
+```python
+from google import genai
+from google.genai import types
+
+client = genai.Client(
+    api_key="unused",
+    http_options=types.HttpOptions(base_url="http://localhost:4001"),
+)
+
+response = client.models.generate_content(
+    model="gemini-2.5-flash-lite",
+    contents="Reply with native-ok",
+)
+print(response.text)
+
+for chunk in client.models.generate_content_stream(
+    model="gemini-2.5-flash-lite",
+    contents="Reply with stream-ok",
+):
+    print(chunk.text or "", end="")
+```
+
+For Solana, start the sidecar with
+`BLOCKRUN_API_URL=https://sol.blockrun.ai/api` and a `SOLANA_WALLET_KEY` (or
+the wallet already stored at `~/.blockrun/.solana-session`). A client-supplied
+Google API key is ignored; the local wallet is the authentication and payment
+mechanism.
+
+**Availability.** Native Gemini passthrough on the **Base** gateway (the
+default, `https://blockrun.ai/api`) is limited to enterprise/allowlisted
+wallets — other payers get `403 PERMISSION_DENIED` and are **not** charged. On
+**Solana** (`https://sol.blockrun.ai/api`) it is generally available. If your
+wallet is not on the Base allowlist, use Solana for native Gemini, or call the
+OpenAI-format `/v1/chat/completions` route with `model="google/gemini-3.1-pro"`,
+which works on both chains. Only `generateContent` and `streamGenerateContent`
+are proxied; `countTokens`, model list/get, and `embedContent` are not served by
+the gateway. Streaming always returns SSE regardless of `alt=` (the client query
+string is dropped and the gateway sets `alt=sse` itself), so raw REST clients
+must parse SSE, not the chunked-JSON-array form.
+
+If you set `BLOCKRUN_PROXY_TOKEN` to guard the sidecar, the Google SDK cannot
+send it (it only sends `x-goog-api-key`), so pass it explicitly as a Bearer
+header: `types.HttpOptions(base_url=..., headers={"Authorization": "Bearer <token>"})`.
+
+This is a sidecar HTTP feature. `litellm.completion(...)` remains the
+OpenAI-compatible interface and continues to use `/v1/chat/completions`.
+
 #### Image request limits
 
 `n` is bounded to **1–10** locally. The Base `image2image` gateway schema has no
@@ -403,7 +474,7 @@ otherwise degrades silently to text-to-video and still bills you.
 Reference-to-video (`reference_videos`/`reference_audios`) is **not** supported:
 both gateways currently gate it off and would return 503.
 
-### 2e. Image generation
+### 2f. Image generation
 
 ```python
 from openai import OpenAI

blockrun_litellm/__init__.py

@@ -26,4 +26,4 @@
 from blockrun_litellm.provider import BlockRunLLM, register
 
 __all__ = ["BlockRunLLM", "register", "enable_local_logging"]
-__version__ = "0.7.6"
+__version__ = "0.8.0"

blockrun_litellm/proxy.py

@@ -11,6 +11,8 @@
 - ``POST /v1/messages``              — native Anthropic Messages (Claude Code,
   Anthropic SDK); verbatim x402-signed passthrough — tools/thinking preserved
 - ``POST /v1/messages/count_tokens`` — Anthropic token counting (passthrough)
+- ``POST /v1beta/models/{model}:generateContent`` — native Gemini JSON
+- ``POST /v1beta/models/{model}:streamGenerateContent`` — native Gemini SSE
 - ``POST /v1/images/generations``    — OpenAI Image Generations (DALL-E compatible)
 - ``POST /v1/videos/generations``    — video generation (xai/grok-imagine-video,
   Seedance); async submit+poll, settles only on completion
@@ -48,6 +50,7 @@
 import logging
 import math
 import os
+import re
 import threading
 import time
 import uuid
@@ -386,6 +389,21 @@ def _openai_fwd_headers(request: Request) -> Dict[str, str]:
     return {"content-type": request.headers.get("content-type", "application/json")}
 
 
+def _gemini_fwd_headers(request: Request) -> Dict[str, str]:
+    """Headers for native Gemini requests.
+
+    Google SDKs require an ``api_key`` even when pointed at a custom base URL
+    and may send it as ``x-goog-api-key`` or ``?key=...``.  The sidecar uses
+    the local x402 wallet instead, so neither credential is forwarded.  Keep
+    only the content type; the BlockRun gateway supplies its own Google key.
+
+    Same shape as ``_openai_fwd_headers`` today; delegate so a future change to
+    the forwarded-header policy can't silently apply to one protocol and not
+    the other.
+    """
+    return _openai_fwd_headers(request)
+
+
 def _open_upstream_stream(
     client: httpx.Client, target: str, raw: bytes, headers: Dict[str, str]
 ) -> httpx.Response:
@@ -511,35 +529,55 @@ def _body_model(raw: bytes) -> Optional[str]:
 
 
 async def _forward_passthrough(
-    request: Request, path: str, headers: Dict[str, str], *, allow_stream: bool
+    request: Request,
+    path: str,
+    headers: Dict[str, str],
+    *,
+    allow_stream: bool,
+    stream_override: Optional[bool] = None,
+    model_override: Optional[str] = None,
+    forward_query: bool = True,
 ) -> Response:
     """Verbatim x402-signed passthrough to a BlockRun native endpoint.
 
-    Shared by ``/v1/messages``, ``/v1/messages/count_tokens`` and
-    ``/v1/chat/completions``. The body is forwarded byte-for-byte — no
-    translation, no SDK typed parsing — so OpenAI clients (incl. Codex with
-    wire_api=chat) and Claude Code stay off the SDK's ``chat_completion_stream``
-    path, the source of the streamed-tool-call ``'dict' object has no attribute
-    'delta'`` crash. The upstream call is gated by ``_get_semaphore()`` like
-    every other paid route. For streaming we open the upstream first to learn its
-    real status, then either return a real error ``Response`` (preserving the
-    4xx/5xx the client must see) or stream the body.
+    Shared by ``/v1/messages``, ``/v1/messages/count_tokens``,
+    ``/v1/chat/completions`` and the native Gemini ``/v1beta/models/...`` route.
+    The body is forwarded byte-for-byte — no translation, no SDK typed parsing —
+    so OpenAI clients (incl. Codex with wire_api=chat) and Claude Code stay off
+    the SDK's ``chat_completion_stream`` path, the source of the
+    streamed-tool-call ``'dict' object has no attribute 'delta'`` crash. The
+    upstream call is gated by ``_get_semaphore()`` like every other paid route.
+    For streaming we open the upstream first to learn its real status, then
+    either return a real error ``Response`` (preserving the 4xx/5xx the client
+    must see) or stream the body.
+
+    Keyword-only overrides let a caller adapt the shared path to a protocol whose
+    conventions differ from the OpenAI body:
+
+    * ``stream_override`` — when not ``None``, decides streaming directly instead
+      of sniffing a ``"stream"`` field in the body (Gemini encodes streaming in
+      the URL method, not the body). ``allow_stream`` is then irrelevant.
+    * ``model_override`` — the model string used for the audit log, when it can't
+      be read from the body (e.g. Gemini carries it in the URL).
+    * ``forward_query`` — ``False`` drops the client query string before
+      forwarding (Gemini strips ``?key=``/``?alt=sse``; the gateway re-derives
+      both server-side).
     """
     api_url = _resolve_api_url()
     raw = await request.body()
     client = _messages_client(api_url)
-    qs = request.url.query
+    qs = request.url.query if forward_query else ""
     target = f"{api_url}{path}" + (f"?{qs}" if qs else "")
 
     _t0 = time.monotonic()
-    model = _body_model(raw)
+    model = model_override or _body_model(raw)
     req_id = request.headers.get("x-request-id")
 
     def _latency_ms() -> float:
         return (time.monotonic() - _t0) * 1000.0
 
-    wants_stream = False
-    if allow_stream:
+    wants_stream = bool(stream_override)
+    if allow_stream and stream_override is None:
         try:
             wants_stream = bool(_json.loads(raw or b"{}").get("stream"))
         except Exception:
@@ -665,6 +703,86 @@ def _post():
     )
 
 
+# ---------------------------------------------------------------------------
+# Native Gemini /v1beta passthrough (Google GenAI SDK, raw REST clients, ...)
+# ---------------------------------------------------------------------------
+# Gemini selects streaming in the URL method, not with a JSON ``stream`` field.
+# The request and response bodies stay byte-for-byte native; only x402 payment
+# signing is added by the shared transport.  A Google SDK's dummy ``key`` query
+# and x-goog-api-key header are deliberately dropped before the gateway hop.
+
+_GEMINI_METHODS = {"generateContent", "streamGenerateContent"}
+
+# A native Gemini model id, once the id-only ``models/``/``google/`` prefixes are
+# stripped: letters, digits, dot, dash, underscore. Deliberately no slash and no
+# dot-segment — those are how a crafted model name (``../../v1/chat/completions``)
+# escapes the ``/v1beta/models/`` prefix after httpx normalizes the URL, so we
+# reject them locally before the wallet ever signs a payment.
+_GEMINI_MODEL_RE = re.compile(r"[A-Za-z0-9._-]+")
+
+
+def _gemini_error(status: int, message: str, code: str = "INVALID_ARGUMENT") -> JSONResponse:
+    return JSONResponse(
+        status_code=status,
+        content={"error": {"code": status, "message": message, "status": code}},
+    )
+
+
+def _log_gemini_reject(path: str, req_id: Optional[str]) -> None:
+    """Log a local, pre-payment 400 the way the media routes do.
+
+    The 0.7.6 invariant is that every exit logs. This reject never reaches the
+    gateway, so it's genuinely free (no ``settlement_status`` flag), but it still
+    gets a row — a call that leaves no trace is invisible to reconciliation.
+    """
+    _logger.log_proxy_call(
+        model=None,
+        path=path,
+        stream=False,
+        http_status=400,
+        cost_usd=None,
+        settlement=None,
+        latency_ms=0.0,
+        request_id=req_id,
+    )
+
+
+@app.post("/v1beta/models/{model_and_method:path}", dependencies=[Depends(_require_token)])
+async def gemini_native(request: Request, model_and_method: str) -> Any:
+    req_id = request.headers.get("x-request-id")
+    log_path = f"/v1beta/models/{model_and_method}"
+    raw_model, separator, method = model_and_method.rpartition(":")
+    if not separator or method not in _GEMINI_METHODS:
+        _log_gemini_reject(log_path, req_id)
+        return _gemini_error(
+            400,
+            "This endpoint only supports generateContent and streamGenerateContent.",
+        )
+
+    # Strip the id-only prefixes the gateway also accepts, then require a clean
+    # model token. The forwarded path is rebuilt from the sanitized parts — never
+    # the raw segment — so no slash or ``..`` in the model can move the signed
+    # upstream target outside ``/v1beta/models/``.
+    model = raw_model.strip().removeprefix("models/").removeprefix("google/")
+    if not _GEMINI_MODEL_RE.fullmatch(model):
+        _log_gemini_reject(log_path, req_id)
+        return _gemini_error(400, f"Invalid Gemini model id: {raw_model.strip()!r}.")
+
+    return await _forward_passthrough(
+        request,
+        f"/v1beta/models/{model}:{method}",
+        _gemini_fwd_headers(request),
+        allow_stream=True,
+        stream_override=method == "streamGenerateContent",
+        model_override=f"google/{model}",
+        # Google SDKs append ?key=... and ?alt=sse.  The BlockRun gateway derives
+        # streaming from the URL method and re-adds ?alt=sse itself on the real
+        # Google call, so it ignores the client query entirely; dropping it here
+        # loses nothing and keeps a stray real Google key out of gateway logs.
+        forward_query=False,
+    )
+
+
 @app.post("/v1/messages", dependencies=[Depends(_require_token)])
 async def anthropic_messages(request: Request) -> Any:
     return await _forward_passthrough(

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "blockrun-litellm"
-version = "0.7.6"
+version = "0.8.0"
 description = "LiteLLM adapter for BlockRun — call x402-paid AI models via LiteLLM (custom provider or local OpenAI-compatible proxy)"
 readme = "README.md"
 license = "MIT"