Skip to content

feat(#3996): resolve image output capability from models.dev - #4019

Open
aheritier wants to merge 2 commits into
gemini-request-diagnosticsfrom
gemini-output-capability
Open

feat(#3996): resolve image output capability from models.dev#4019
aheritier wants to merge 2 commits into
gemini-request-diagnosticsfrom
gemini-output-capability

Conversation

@aheritier

@aheritier aheritier commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

What and why

Add latest-only output_capabilities.image configuration/schema support. Explicit true or false wins. Omitted, empty, or unset metadata consults exact provider-qualified models.dev output metadata; unknown models, unavailable providers, and lookup failures remain disabled. No capability is inferred from a model-name prefix. Numbered configuration packages remain frozen.

Part of #3996. Review this PR against its immediate parent, #4017, rather than the aggregate stack against main.

Commit inventory

Head: 51559ff510bf01ea053e3923c329c80ba724dca5; parent SHA: 2c5fd913464ef376471cc098b8e9481367f16b6a.

Validation

Build, test compilation, owning-package tests and the named fixture passed at this PR head.

Exact deterministic fixture command:

go test -v -count=1 ./pkg/modelinfo ./pkg/config/latest -run 'TestResolveOutputImage|Test.*OutputCapabilities'

Matched top-level tests: pkg/config/latest: 7; pkg/modelinfo: 1.

Deterministic scope: the named local fixture exercises this PR boundary with disposable configuration/stores and fake or loopback providers as applicable. Every listed package ran nonzero matching top-level tests.

Deferred/live scope: No new live catalogue/provider run was executed; deterministic resolution covers explicit and omitted metadata. The final stack head passed build, lint, full tests, an uncached full suite, focused race tests and documentation checks in disposable environments. Remote CI is tracked by the checks below; no new paid-provider or active-database validation was run.

@aheritier aheritier added area/config For configuration parsing, YAML, environment variables area/docs Documentation changes area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) area/testing Test infrastructure, CI/CD, test runners, evaluation kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Aug 22, 2026
@aheritier
aheritier force-pushed the gemini-output-capability branch 2 times, most recently from ebd9306 to 33c6981 Compare August 24, 2026 15:12
@aheritier
aheritier force-pushed the gemini-output-capability branch from 33c6981 to 5bb315a Compare August 25, 2026 20:24
@aheritier
aheritier force-pushed the gemini-output-capability branch 2 times, most recently from 35efc65 to 5bb315a Compare September 1, 2026 16:19
@aheritier
aheritier force-pushed the gemini-output-capability branch from 5bb315a to d1bcf0a Compare September 1, 2026 16:29
aheritier

This comment was marked as resolved.

@aheritier
aheritier force-pushed the gemini-output-capability branch from a1f86ba to 0862c0a Compare September 8, 2026 06:17
@aheritier
aheritier force-pushed the gemini-output-capability branch from 0862c0a to bded66b Compare September 8, 2026 07:57
@aheritier
aheritier force-pushed the gemini-output-capability branch from bded66b to d5a25cb Compare September 8, 2026 17:31
Add ModelConfig.OutputCapabilities (output_capabilities.image in YAML), a latest-only tri-state override for a model's generative output capability. Explicit false has highest precedence, explicit true follows, and an omitted flag — including output_capabilities: {} — is left for models.dev resolution in the following commit. No capability is ever inferred from the model name.

Reject output_capabilities on first_available selectors because one block cannot unambiguously describe every candidate. Update the schema, model docs, and example, and cover true, false, omitted, and empty-block YAML states, shorthand behavior, cloning, schema strictness, and selector validation.
Resolve image-output capability with one explicit precedence chain: false disables it, true enables it, and an omitted image flag — whether output_capabilities is absent or {} — enables it only when the exact known models.dev record lists image in Modalities.Output. Unknown models, unavailable catalogue data, and lookup failures stay off.

Keep lookup case-insensitive and bounded by the existing models.dev timeout. Capability is never inferred from provider or model-name patterns.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config For configuration parsing, YAML, environment variables area/docs Documentation changes area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) area/testing Test infrastructure, CI/CD, test runners, evaluation kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant