feat: custom OpenAI-compatible LLM endpoints across Telegram, web and agents by david-hummingbot · Pull Request #175 · hummingbot/condor

david-hummingbot · 2026-07-27T13:38:35Z

Makes custom OpenAI-compatible endpoints (Venice AI, Together, Fireworks, a local vLLM/LM Studio) work as a first-class model source: configure once, use from Telegram chat, the web dashboard, and agent consults/loops.

Why

The initial implementation only worked in Telegram chat. Endpoints were stored in raw user_data, so the web dashboard couldn't see them, and handlers/agents/session.py was the only caller in the codebase that resolved an endpoint's URL and key. That meant a custom: agent key died with No base URL configured on every other path — which is how a user running Venice ended up watching a trading-agent consult cascade through claude-code → ollama → gemini and fail, never trying the endpoint they had configured.

What changed

Storage

Endpoints move into condor/preferences.py as a named list synced to config.yml, so Telegram and web share one view. Legacy user_data["custom_llm"] migrates on first read.

Agent keys become custom@<endpoint>:<model-id>. The @ segment keeps the model id recoverable with a single partition even when it contains colons and slashes (custom@together:meta-llama/Llama-3.3-70B:free).

Bugs fixed

Bug	Detail
`/cancel` did nothing	The unified text handler is `~filters.COMMAND` and no `CommandHandler` existed, so the documented escape hatch matched nothing — while an invalid URL re-armed the prompt, trapping the user in a loop
Stale input state	Armed typing flags survived navigation and ate the user's next message
Endpoints unreachable off the chat path	`consult`/`delegate`/`engine` built `PydanticAIClient` with no `base_url`/`api_key`
Agents guessed their model	New agents inherited nothing, so the coordinator invented `agent_key` from the `agent_builder` skill's examples
Healthcheck skipped custom	It gated on the raw prefix, which is `custom@venice`, so failures surfaced deep in a run instead of falling back
`tool_filter_mode` forced `full`	`custom:` says nothing about capability — a 4B model on a local vLLM got the full toolset
Ollama/LM Studio hidden on web	The picker dropped any key ending in `:`, which is not the same as "is a sentinel"

UX

Telegram — saved-endpoint list, add/manage/forget, error keyboards with Retry plus a context-appropriate secondary action, a "No API key needed" button, model search, current model floated to page 1, and content-hashed pick tokens instead of list indices (indices silently resolve to the wrong model if the list is refetched between render and tap).

Web — Settings → LLM Endpoints for add/validate/forget, and a two-level chat model picker that lazy-fetches each endpoint's models. API keys are never serialized to the browser, only a has_key flag.

Hardening

Candidate base URLs probed concurrently at 8s instead of sequentially at 15s each
Embeddings/rerankers/TTS/image models filtered out of /models — picking one used to succeed and then fail opaquely at prompt time
Hostname shape-checked (urlparse happily accepts "not a url at all")
Link-local/metadata addresses always refused; loopback and RFC1918 stay allowed by default since local model servers are the common case, and can be refused via CUSTOM_LLM_BLOCK_PRIVATE_URLS on shared deployments
Labels truncate through the middle so the distinguishing tail of ...-Instruct-Turbo-Free survives

Review notes

object: "model" is uninformative. OpenAI stamps it on embeddings, Whisper and DALL·E alike, so it's treated as "not declared" and falls through to id heuristics. Worth a look in _is_chat_model.
set_active_llm() mirrors agent_llm into config.yml because the MCP subprocess can't read the PTB pickle. All nine write sites route through it, and _reclaim_default_agent backfills on read so existing users don't have to re-pick.
Agent inheritance is creation-time, i.e. a snapshot — switching providers later won't repoint existing agents. Resolution-time (agent_key: auto) is the better end state and is being discussed separately; it's deliberately not in this PR.
consult has no override chain (agent.agent_key only) while engine resolves config > strategy > agent. Pre-existing asymmetry, left alone here.
agents/ is committed and the shipped agents hardcode claude-acp:sonnet / claude-code:sonnet. Not touched in this PR, but it's the same failure mode for any user without those CLIs.

Testing

tests/test_custom_provider.py: 14 → 34 cases (URL normalization and SSRF guards, model filtering, key composition, endpoint storage and migration, picker helpers, endpoint resolution)
Full suite: 231 passed
Frontend tsc --noEmit clean; no new lint findings
Manually drove the full Telegram flow and the web REST surface end-to-end against a mock provider

tests/trading_agent/test_risk_drawdown.py fails to collect on main (imports a missing condor.trading_agent) — pre-existing and unrelated, so it was excluded rather than touched.

🤖 Generated with Claude Code

… agents Reworks the custom-endpoint support so an endpoint configured once is usable everywhere, and so the surfaces that build a model can actually resolve it. Storage - Endpoints move out of raw user_data into condor/preferences.py as a named list synced to config.yml, giving Telegram and the web dashboard one shared view. Legacy user_data["custom_llm"] migrates on first read. - Agent keys become "custom@<endpoint>:<model-id>". The "@" segment keeps the model id recoverable with a single partition even when it contains colons and slashes (e.g. meta-llama/Llama-3.3-70B:free). Fixes - /cancel matched no handler at all: the unified text handler is registered with ~filters.COMMAND and no CommandHandler existed, so the documented escape hatch silently did nothing while an invalid URL re-armed the prompt, trapping the user. Adds the command plus inline Cancel buttons. - Armed text-input flags survived navigation, so a stale flow ate the user's next message. They are now cleared in clear_all_input_states. - consult/delegate/engine constructed PydanticAIClient with no base_url or api_key, so a custom agent_key died with "No base URL configured" — session.py was the only caller that resolved endpoints. Adds a shared resolve_custom_endpoint() used by all of them. - New Agents inherited nothing, so the coordinator invented an agent_key from the agent_builder skill's examples and pinned agents to backends the user never configured. agent_key now defaults to the creator's active model, and get_user_context() reports it so the coordinator stops guessing. - healthcheck_local_backend skipped custom endpoints (it gated on the raw prefix, which is "custom@venice"), so they failed deep in a run instead of falling back. It now preflights them with the Bearer key. - _infer_tool_filter_mode forced "full" for every custom: model; "custom:" says nothing about capability, so it falls through to the size heuristics. - The web model picker dropped any key ending in ":", which is not the same as "is a sentinel" — it was silently hiding Ollama and LM Studio. Sentinels are now marked server-side and filtered from /chat/options. UX - Telegram: saved-endpoint list, add/manage/forget, error keyboards with Retry and a context-appropriate secondary action, a "No API key needed" button, model search, current model floated to page 1, and content-hashed pick tokens instead of list indices. - Web: Settings -> LLM Endpoints for add/validate/forget, and a two-level chat model picker that lazy-fetches each endpoint's models. API keys are never serialized to the browser, only a has_key flag. Hardening - Base-url candidates are probed concurrently at 8s instead of sequentially at 15s each. - /models entries that are embeddings, rerankers, TTS or image models are filtered out; picking one used to succeed here and fail opaquely at prompt time. Note object="model" is uninformative (OpenAI stamps it on everything) and falls through to id heuristics. - Hostnames are shape-checked, since urlparse accepts "not a url at all". - Link-local/metadata addresses are always refused; loopback and RFC1918 stay allowed by default (local model servers are the common case) and can be refused via CUSTOM_LLM_BLOCK_PRIVATE_URLS on shared deployments. - Model labels truncate through the middle, so the distinguishing tail of ids like ...-Instruct-Turbo-Free survives. Tests: 14 -> 34 cases in tests/test_custom_provider.py; 231 pass overall. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Resolves three conflicts, all in favour of main's newer work with this branch's custom-endpoint support folded in rather than either side winning. agent_builder/SKILL.md — main added get_available_models and a real model recommendation heuristic, which is strictly better than this branch's "omit agent_key and inherit". Took main's version wholesale and added custom endpoints to the tool's reported sources. Also corrected main's stale custom-endpoint bullet: "openai:<model> + model_base_url" is superseded by "custom@<endpoint>:<model-id>", which resolves URL and key from the user's saved endpoints at run time. api.ts / ChatPanel.tsx — both sides added a two-level model picker (main for OpenRouter and local servers, this branch for custom endpoints). Took main's AgentDropdown, which is the richer implementation and shared by both call sites, and generalized its single `showModels` boolean into a `submenu` discriminated union so OpenRouter and each saved endpoint are peer drill-downs. Also reworks how sentinels reach the web client. This branch had stripped them from /chat/options, which would have broken main's OpenRouter submenu (it needs the "openrouter:" entry present to render the drill-down). Instead every entry now carries an explicit `picker` flag. That fixes the underlying problem both versions had: the client was inferring "is a picker" from `key.endsWith(":")`, which also matches "ollama:" and "lmstudio:" — real, startable keys — so those were being silently hidden from the dashboard. get_available_models now reports custom_endpoints, re-validating each one so a dead endpoint surfaces as reachable:false instead of being recommended and failing on the first consult. Full suite: 271 passed (the whole suite runs again — main repaired the test_risk_drawdown import that previously broke collection). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The merge changed how sentinels reach the web client: they are now sent with an explicit `picker` flag instead of being filtered out server-side, so main's OpenRouter submenu still renders. Nothing guarded that contract, and the failure mode is silent — inferring "is a picker" from the key ending in ":" also matches "ollama:"/"lmstudio:", which are startable keys, and quietly drops them from the dashboard model picker. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Prompts like "Enter model manually" arm a flag that makes the next plain message mean something other than "talk to the agent". The callback router never cleared those flags, and Telegram keyboards from earlier messages stay tappable forever — so arming manual entry and then picking a different LLM (or a model from the picker) changed the LLM but left _openrouter_typing_slug set. The next real task was parsed as a model slug ("No tool-calling OpenRouter model matches 'create trading agent'") and the failure path re-armed itself, blocking agent chat until the user knew to cancel. Clear every armed mode in agent_callback_handler before dispatch: a tap is a fresh intent, and the handlers that want an armed mode set their own flag afterwards. Only the armed flags go — pending values like _openrouter_typed_slug survive, so confirm/retry steps still work. This also closes the symmetric cross-flow holes (the custom-endpoint flow only disarmed its own keys) and covers agent_compact_custom. Also give the slug prompt and its "no model matches" retry a Cancel / Back-to-list keyboard, mirroring the custom-endpoint flow, so a mistyped slug can't trap the user in a loop with /cancel as the only exit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

fengtality · 2026-07-29T15:01:39Z

@carlito-hummingbot try opencode:go with this PR

…m for opencode-go model Switches agent_key from 'opencode-go:deepseek-v4-flash' to 'custom@opencode:deepseek-v4-flash' so the agent runs on opencode-go via PR hummingbot#175's named custom OpenAI-compatible endpoint (no PR hummingbot#170 needed). Documents endpoint setup: Settings -> LLM Endpoints, or CUSTOM_LLM_BASE_URL/CUSTOM_LLM_API_KEY in .env for headless condor-bot.service deploys.

carlito-hummingbot · 2026-07-30T03:30:21Z

@carlito-hummingbot try opencode:go with this PR

This PR confirmed and tested to work with opencode-go. I'm closing PR #170.

david-hummingbot and others added 4 commits July 27, 2026 21:37

fengtality requested a review from cardosofede July 29, 2026 13:55

cardosofede approved these changes Jul 30, 2026

View reviewed changes

cardosofede merged commit fd1b064 into main Jul 30, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: custom OpenAI-compatible LLM endpoints across Telegram, web and agents - #175

feat: custom OpenAI-compatible LLM endpoints across Telegram, web and agents#175
cardosofede merged 4 commits into
mainfrom
feat/custom-openai-compatible-endpoints

david-hummingbot commented Jul 27, 2026

Uh oh!

fengtality commented Jul 29, 2026

Uh oh!

carlito-hummingbot commented Jul 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

david-hummingbot commented Jul 27, 2026

Why

What changed

Storage

Bugs fixed

UX

Hardening

Review notes

Testing

Uh oh!

fengtality commented Jul 29, 2026

Uh oh!

carlito-hummingbot commented Jul 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants