feat(agent): route Claude/GPT model families to their native gateway wire

[atishpatel]

Summary

Databricks v2 chooses the gateway wire format — OpenAI Responses, Anthropic Messages, or MLflow chat — purely from substrings in the endpoint name. There is no family field on the endpoint to key off, so the substring set is the routing contract. The matcher only recognised gpt-5/gpt5 and claude, which makes correct billing depend on every Claude endpoint happening to be named with the literal string "claude".

Why this matters

Getting a Claude model onto the Anthropic Messages route is exactly what lets buzz attach the cache_control breakpoint (the fix in #3463). If a Claude endpoint's catalog name omits "claude" — an alias, a bare opus-5, a goose-opus-5 — it silently falls through to the MLflow (OpenAI-wire) path, where Anthropic prompt caching is structurally impossible. The result is the same failure #3463 fixed: 0% cache reads, the full ~10x read discount lost, and no error — a naming convention quietly holding up a billing-correctness invariant.

What changed

databricks_v2_route_for_model (crates/buzz-agent/src/llm.rs) now matches broader, case-insensitive marker sets:

• Claude → Anthropic Messages: claude, opus, sonnet, haiku, mythos, fable — the Claude family names and release code names, so a Claude endpoint reaches the cache-capable route regardless of how it's named.
• GPT → OpenAI Responses: the gpt family (now gpt on its own, not just gpt-5) plus the GPT-5 launch code names sol, luna, terra.

OpenAI markers are evaluated first, preserving the prior gpt-5-first precedence for any name that could carry both. Names matching neither set still fall through to the MLflow chat route.

Testing

• cargo fmt, cargo clippy -p buzz-agent --all-targets -- -D warnings — clean.
• cargo test -p buzz-agent — all green (299 lib + integration suites, 0 failures). The databricks_v2_routes_by_model_family test was expanded to cover each new marker, the GPT-5 code names, case-insensitivity, and the unchanged MLflow fallback (including gemini).

Relationship to #3463

#3463 taught the Anthropic path to request caching; this makes sure Claude models actually land on that path. Follow-up still open: surfacing cache_creation_input_tokens end-to-end so a persistent reads == 0 && writes == 0 reveals a disabled cache regardless of which wire a model takes — happy to do that next.

🤖 Generated with Claude Code

[tlongwell-block]

Blocking correctness issue: the new classifier uses unrestricted substring matching for very short markers (sol, terra, opus, etc.). Because endpoint names are arbitrary aliases, unrelated names can now change wire protocols: for example, consolidated-llama contains sol and routes to OpenAI Responses, while octopus-model contains opus and routes to Anthropic Messages instead of the documented MLflow fallback. That can turn a cache optimization into a request-shape/parse failure.

Please make these family/code-name matches token/boundary-aware (while retaining arbitrary prefixes such as goose-opus-5) and add negative collision cases. The current positive matrix and custom-tool-model case don't exercise this regression.

[tlongwell-block]

Requesting changes on one routing-safety issue:

contains makes the newly added short/common markers match unrelated custom endpoint names. For example, consolidated-llama matches sol, terraform-coder matches terra, and corpus-reranker matches opus; each is then sent an OpenAI Responses or Anthropic Messages envelope instead of the existing MLflow chat fallback. V2 discovery intentionally retains unrecognised custom names, so this is a reachable regression rather than a theoretical input shape.

Please match model-name segments (or another boundary-aware form) instead of arbitrary substrings, while preserving the intentional gpt5 spelling. Negative tests for collision names would lock this down. The positive goose-opus-5 fix itself is correct.

[tlongwell-block]

Concurring with both prior reviews — changes requested — and adding one collision case that is strictly worse than the ones already flagged:

claude-console-5 regresses. The name contains the literal claude, which the old matcher routed correctly to Anthropic Messages. But console contains sol, and OpenAI markers are checked first, so the new matcher sends it to OpenAI Responses. The PR doesn't just make unrecognised names misroute — it can flip a name the previous code classified correctly. Verified against the exact marker sets + precedence at fe548d349:

claude-console-5:   new=OpenAiResponses  old=AnthropicMessages   <-- regression
consolidated-llama: new=OpenAiResponses  old=Mlflow
terraform-coder:    new=OpenAiResponses  old=Mlflow
octopus-model:      new=AnthropicMessages old=Mlflow
databricks-solstice: new=OpenAiResponses old=Mlflow

Suggested shape: split the name into segments on -/_/. and match the short code names (opus, sonnet, haiku, sol, luna, terra, mythos, fable) as whole segments only, keeping claude/gpt as segment prefixes (so gpt5, claude3 still hit). Segment matching alone fixes claude-console-5 — console is not the segment sol — without needing precedence surgery. Please add negative collision tests including the claude-name-with-embedded-marker case.

The motivating fix (goose-opus-5 → Anthropic Messages) is right and worth landing once the matcher is boundary-aware.

[atishpatel]

🤖 Good catch — fixed in f992d34.

The classifier no longer uses contains. It now splits the endpoint name into alphanumeric segments (breaking on any non-alphanumeric delimiter) and matches whole segments:

• Code/family names (sol, luna, terra, claude, opus, sonnet, haiku, mythos, fable) must equal a segment, so consolidated-llama, terraform-coder, corpus-reranker, and octopus-model all stay on the MLflow fallback.
• The gpt family matches a segment beginning with gpt, which preserves the intentional dashless gpt5 spelling and the gpt of a split gpt-5 without matching mid-word.
• Prefixed real names like goose-opus-5 still route correctly.

Added the four collision names plus gpt5 as explicit cases in databricks_v2_routes_by_model_family. Full cargo test -p buzz-agent green.

[tlongwell-block]

Re-tested the revised head f992d3428129b6e9fe2766f978bd4fb9e73ba703; my prior collision blocker is resolved.

In addition to the committed route table, I temporarily added request-capture integration probes through the real buzz-agent Databricks v2 prompt path and verified:

• goose-opus-5 posts to /ai-gateway/anthropic/v1/messages and carries the system cache_control: {type: ephemeral} marker.
• claude-console-5 remains on Anthropic Messages.
• consolidated-llama, terraform-coder, corpus-reranker, and octopus-model all remain on MLflow chat and do not receive a Responses envelope.

Those probes passed and were reverted; the worktree is clean. Full package fmt, clippy, and tests also pass at that exact SHA. Approving.

[tlongwell-block]

One collision class remains in the fix: segments.iter().any(|seg| seg.starts_with("gpt")) still classifies every arbitrary segment beginning with those three letters as OpenAI. For example, gptosis-reranker (or gptproxy-custom) now routes to Responses instead of MLflow. This has the same request-shape failure mode as the fixed sol/opus collisions, and the comment's claim that it avoids matching “mid-word” is not a sufficient family boundary when arbitrary aliases are retained.

Please constrain the compact GPT spelling to gpt itself or gpt followed by a digit (preserving gpt, gpt5, gpt4o, and split gpt-5), and add a negative gpt... word collision. Please also lock Eva's old-route regression (claude-console-5 → Anthropic) into the matrix; the new segmentation does fix it, but that precedence regression should stay covered.

[tlongwell-block]

Re-review at f992d3428129b6e9fe2766f978bd4fb9e73ba703 — the octopus problem is dead. Approving.

Verified the segment matcher against every collision case from round 1, including the regression I flagged:

name	route
claude-console-5	AnthropicMessages	✅ regression fixed (console no longer trips sol)
consolidated-llama	Mlflow	✅
terraform-coder	Mlflow	✅
octopus-model	Mlflow	✅
corpus-reranker	Mlflow	✅
goose-opus-5	AnthropicMessages	✅ motivating fix intact
gpt5	OpenAiResponses	✅ dashless spelling preserved via segment-prefix

Local at exact SHA (clean worktree, HEAD confirmed in-shell): cargo fmt --all -- --check ✅, cargo clippy -p buzz-agent --all-targets -- -D warnings ✅, full cargo test -p buzz-agent ✅ (299 unit + all integration/doc suites).

Two non-blocking notes:

1. Test gap (nit): the collision-guard block covers the four Mlflow negatives but not claude-console-5 — the one case where the old code was right and substring matching flipped it. The mechanism guarantees it, but it's the sharpest regression sentinel this table could carry. Worth adding whenever this file is next touched.
2. Asymmetry (deliberate-looking, just naming it): gpt is matched as a segment prefix (so gpt5 works) but claude only as a whole segment, so a hypothetical dashless claude5 falls to Mlflow. That miss degrades to lost cache discount, not a broken wire — the safe direction — so fine as-is.

Unrelated CI: the Windows failure is buzz-dev-mcp shell::tests::basic_echo (exit 124 timeout) — different crate, untouched by this PR; runner flake. Locally I also saw fake_llm::cancelled_turn_with_usage_emits_notification_before_response flake once — it flakes at the same rate on base c405ad1d4, pre-existing.