Skip to content

fix(agents): a model base URL may end in /v1 for every built-in agent and the direct judge - #816

Merged
earakely-scale merged 5 commits into
mainfrom
edgararakelyan/fd3307-v1-base-url
Nov 19, 2026
Merged

earakely-scale merged 5 commits into
mainfrom
edgararakelyan/fd3307-v1-base-url

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

What changes

A model base URL may now end in /v1 or not, and every built-in agent and the direct judge call the same paths either way. Before this, a /v1 base made Anthropic-style clients call …/v1/v1/messages, which Scale's proxy refuses with a 403.

Agents. LITELLM_BASE_URL still reaches agents exactly as configured. A client that appends its own versioned path now strips one trailing /v1 first:

Agent What it builds from the base /v1 base before After
Claude Code ANTHROPIC_BASE_URL, then the CLI adds /v1/messages /v1/v1/messages /v1/messages
Codex LiteLLM's /openai/v1 route /v1/openai/v1/responses /openai/v1/responses
Gemini LiteLLM's /gemini/... and /v1/chat/completions /v1/gemini/..., /v1/v1/chat/completions /gemini/..., /v1/chat/completions
browser-use (Claude) Anthropic's client adds /v1/messages /v1/v1/messages /v1/messages

Each strip is a copy of core's api_root, since agent images can't import core, and tst/unit/agents/test_api_root_copies.py checks every copy against core's over a table of URLs. Letta Code, Hermes and OpenCode already add /v1 unless it's there, and pi, Pydantic AI and OpenHands use the URL as given with OpenAI-style clients. Those agents are unchanged; three docstrings that said the base "has no /v1 suffix" are corrected. src/agent_env/agents/AGENTS.md states the contract.

Core.

  • config.model.api_root(url) drops a trailing / and one trailing /v1, by the same rule same_endpoint compares URLs with, so the two can't disagree.

  • The unit-test verifier gives its tests ANTHROPIC_BASE_URL without the /v1; LITELLM_BASE_URL stays as configured.

  • The direct judge routes a bare model name by what litellm makes of it:

    • a name litellm maps to Anthropic (claude-*) keeps that route, with the base's /v1 dropped;
    • an OpenAI name gets the URL as configured;
    • any other bare name goes through the configured endpoint as openai/<name>. That covers a name litellm can't place (qwen3:8b, Qwen/Qwen3-8B), which used to fail with "LLM Provider NOT provided". It also covers a name litellm would send to another provider's own API, such as gemini-* (Vertex, with the host's gcloud project) or command-* (Cohere), which used to reach paths a LiteLLM proxy doesn't serve;
    • a name with a provider prefix (deepseek/deepseek-chat, hosted_vllm/…) keeps the route it names, as before.

    A custom_llm_provider in [model] params decides the provider, as it does for litellm, so the route follows it. What the judge records keeps the name as given. litellm's "Provider List" banner for an unknown name is kept off stdout.

Configs that already worked call the same paths as before. The new judge test checks the four rows that worked on main and fails on main for the six that didn't.

Tests

  • Unit tests:
    • api_root cases;
    • the judge against a loopback server through the real litellm, over 10 model-name and base combinations;
    • the route helper;
    • the Claude Code, Codex and Gemini bridges with /v1 bases;
    • the unit-test verifier's ANTHROPIC_BASE_URL.
  • Changed expectations: two existing verifier tests expected ANTHROPIC_BASE_URL to keep a configured /v1, which was the bug; they now expect it without.

E2E against the real proxy

Each run went through a logging pass-through in front of Scale's LiteLLM proxy, so every model call's path was recorded. The two configs were identical except for the base: http://…:8767 and http://…:8767/v1. Every row passed its task:

Run Bare base /v1 base
Claude Code (claude-haiku-4-5) /v1/messages /v1/messages
Codex (gpt-4.1-mini) /openai/v1/responses /openai/v1/responses
Gemini, AI Studio (gemini-2.5-flash) /gemini/v1beta/…:streamGenerateContent same
Gemini, Vertex (vertex_ai/global/gemini-3.1-pro-preview) /v1/chat/completions /v1/chat/completions
Pydantic AI (gpt-4.1-mini) /chat/completions /v1/chat/completions (URL as configured)
Direct judge, claude-haiku-4-5 /v1/messages /v1/messages
Direct judge, gpt-4.1-mini /chat/completions /v1/chat/completions (URL as configured)
Direct judge, gemini-flash-latest (served by the proxy, unknown to litellm) /chat/completions /v1/chat/completions
Direct judge, gemini-2.5-flash (litellm would send it to Vertex) /chat/completions /v1/chat/completions

The same runs on main all failed:

  • Claude Code with the /v1 base called /v1/v1/messages and got a 403.
  • Codex with the /v1 base called /v1/openai/v1/responses and got a 403.
  • The Claude judge with the /v1 base called /v1/v1/messages and got a 403.
  • The gemini-flash-latest judge failed with "LLM Provider NOT provided" even with the bare base.
  • The gemini-2.5-flash judge failed after 3 attempts with the bare base, without reaching the endpoint: litellm tried Vertex's own API.

In this first round, browser-use couldn't run its task on the test VM, because Chromium didn't build there: the host's Docker refuses to pull the browser base image by digest ("cannot overwrite digest"), unrelated to this PR. So its client was checked inside its built image instead. agent.model() was run with each base against a recording endpoint:

  • Claude went to /v1/messages with a bare, /v1 or /v1/ base, and to /prefix/v1/messages with /prefix/v1.
  • GPT used each URL as configured.

Final e2e on the head (5ca8104)

Everything above was rerun on the PR's head, in a fresh state folder, with the bare and the /v1 base: 24 runs, all passed. browser-use ran its real task on the x86 test host, where Chromium builds.

Run Bare base /v1 base
Claude Code /v1/messages /v1/messages
Codex /openai/v1/responses /openai/v1/responses
Gemini, AI Studio and Vertex /gemini/…, /v1/chat/completions same
Pydantic AI /chat/completions /v1/chat/completions
browser-use, Claude /v1/messages /v1/messages
browser-use, GPT /chat/completions /v1/chat/completions
Direct judge: claude-haiku-4-5 /v1/messages /v1/messages
Direct judge: gpt-4.1-mini, gemini-flash-latest, gemini-2.5-flash /chat/completions /v1/chat/completions
Unit-test verifier: a test that calls Anthropic's Messages API at $ANTHROPIC_BASE_URL /v1/messages, score 1 /v1/messages, score 1

After the copies were made to parse as core's does (a7eb1db), the four agents were run again on the x86 test host: Claude Code, Codex, Gemini (both routes) and browser-use (Claude and GPT), with both bases. All 12 passed, on the same paths as above.

Chaos

Six more base spellings, against an endpoint that records each path and answers 400:

Spelling Claude Code Codex Gemini (Studio / Vertex) Pydantic AI
…/ /v1/messages /openai/v1/responses /gemini/… / /v1/chat/completions /chat/completions
…/v1/ /v1/messages /openai/v1/responses /gemini/… / /v1/chat/completions /v1/chat/completions
…/prefix/v1 /prefix/v1/messages /prefix/openai/v1/responses /prefix/gemini/… / /prefix/v1/chat/completions /prefix/v1/chat/completions
…/prefix /prefix/v1/messages /prefix/openai/v1/responses /prefix/gemini/… / /prefix/v1/chat/completions /prefix/chat/completions
…/v1/v1 /v1/v1/messages /v1/openai/v1/responses /v1/gemini/… / /v1/v1/chat/completions /v1/v1/chat/completions
"…/v1 " (trailing space) /v1/messages /openai/v1/responses /gemini/… / /v1/chat/completions /v1%20/chat/completions
  • …/v1/v1 loses exactly one /v1, as same_endpoint does.
  • The trailing space reaches agents unchanged: core passes [model] base_url through with its whitespace. The agents that strip /v1 trim it, but an agent that uses the URL as given doesn't. That predates this PR and is separate from /v1, so it's left out here.

Docs

FD-6263 already lists P3's items: LITELLM_BASE_URL and [model] base_url may end in /v1 or not, and a direct judge given a bare name litellm doesn't know calls it through the endpoint as openai/<name>.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no blocking issues were found.

Summary

This PR lets clients that add their own API paths accept model URLs with or without a trailing /v1. It also lets the direct judge call model names that LiteLLM does not recognize through the configured endpoint.

  • Built-in agents build their model routes from a clean endpoint root.
  • Core and unit-test containers use one cleaned root for Anthropic clients.
  • The direct judge routes model names through the configured endpoint.

No new actionable issues found. The previous explicit-provider finding is fixed.

Reviews (8) · Last reviewed commit: "Merge origin/main into edgararakelyan/fd..." · Reviewed by Greptile

… and the direct judge

LITELLM_BASE_URL reaches agents as configured. A client that appends its own versioned path now
strips one trailing /v1 first: Claude Code's ANTHROPIC_BASE_URL, Codex's /openai/v1 route,
Gemini's /gemini and /v1/chat/completions forwards, browser-use's Anthropic client, and the
unit-test verifier's ANTHROPIC_BASE_URL, through core's api_root, which shares same_endpoint's
rule. A base ending in /v1 no longer calls .../v1/v1/messages.

The direct judge routes by what litellm makes of the model name: an Anthropic name gets the base
without its /v1, an OpenAI one the URL as configured, and a name litellm can't place goes through
the endpoint as openai/<name> instead of failing with "LLM Provider NOT provided". What it
records keeps the name as given. Configs that already worked call the same paths as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale marked this pull request as ready for review November 19, 2026 17:15
@earakely-scale
earakely-scale requested a review from a team as a code owner November 19, 2026 17:15
earakely-scale and others added 2 commits November 19, 2026 09:19
…r in its params

[model] params can name litellm's provider outright; the judge now decides whether to drop the
base's /v1 by that provider, as litellm does, so a Claude name sent through the OpenAI route keeps
its /v1/chat/completions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e to another provider's API through the endpoint

A bare name litellm places with a provider other than Anthropic or OpenAI, such as gemini-* with
Vertex or command-* with Cohere, made the judge call that provider's own API at the endpoint's
base, which a LiteLLM proxy doesn't serve. Through an endpoint it now goes as openai/<name>, as a
name litellm can't place does. A name with a provider prefix, or a custom_llm_provider in the
params, keeps the route it names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
earakely-scale and others added 2 commits November 19, 2026 11:21
…e's does, and a test keeps them equal

Claude Code, Codex, Gemini and browser-use each carry a copy of core's api_root, since their
images can't import core. The copies now take the same urlsplit path as core's, so they agree on
every base, a query string included, and a unit test runs each copy, read from its source,
against core's over a table of URLs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
claude_bridge.py: main moved the egress proxy and clock code into agent_egress; keep that move and
this branch's _api_root, and the urlsplit import it needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale merged commit c7431b6 into main Nov 19, 2026
36 checks passed
@earakely-scale
earakely-scale deleted the edgararakelyan/fd3307-v1-base-url branch November 19, 2026 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant