feat: support openai-compatible endpoints and model discovery for agents - #251
Draft
vigneshrajsb wants to merge 5 commits into
Draft
vigneshrajsb wants to merge 5 commits into
vigneshrajsb wants to merge 5 commits into
Conversation
Lets an administrator point the openai provider at any OpenAI-compatible endpoint, such as a self-hosted LLM gateway, by setting `baseUrl` on the provider entry in the agent runtime config. - When `baseUrl` is set, models are created against Chat Completions rather than the Responses API, since that is the interface gateways implement. - `baseUrl` applies to every key used for the provider, including keys users save themselves, and saved keys are validated against it. - The runtime config validator rejects `baseUrl` on other providers, non-http(s) URLs, and URLs with embedded credentials. - `baseUrl` is declared in the runtime config JSON schema and OpenAPI spec.
Adds `discoverModels` to the openai provider. When set together with
`baseUrl`, the model list comes from `GET {baseUrl}/models` using the shared
provider key, so adding a model on the endpoint makes it selectable without a
config change.
- Configured `models` entries act as per-model overrides for display name,
default, token limit and pricing; `enabled: false` hides a model, and entries
the endpoint no longer lists are dropped.
- Results are cached per endpoint and key for five minutes. A failed refresh
keeps serving the last known list; if discovery has never succeeded, the
configured models are used.
- The validator requires `baseUrl` for discovery and no longer requires an
enabled model on a discovering provider.
The first model is the default when no override pins one, and gateways return /models in varying order, so the default changed between refreshes. Discovered ids are now sorted.
- Reject a baseUrl with a query string or fragment. - Validate saved keys against a custom endpoint only on a 2xx response, with a timeout. - Treat an empty model list as a failed refresh. - Share the discovery cache for a baseUrl with and without a trailing slash, and sort ids by code unit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
baseUrlto theopenaiagent provider. Lifecycle sends the requests for this provider to that OpenAI-compatible endpoint, for example a self-hosted LLM gateway.baseUrlis set, Lifecycle uses the Chat Completions API. It does not use the Responses API.discoverModelsflag. When it is set, the model list comes fromGET {baseUrl}/modelswith the shared provider key. Configured models become overrides for name, default, and limits.baseUrl.baseUrlanddiscoverModelsto the runtime config schema and to the OpenAPI spec.Verifying Changes
pnpm test,pnpm lint, andtscpass.Notes
baseUrlis not set.