feat: audit hardening — streaming/retry fixes, canonical mapping, Anthropic adaptive thinking, observability, documents, RunTools, CountTokens

[jkyberneees]

Summary

A full audit of the SDK followed by six milestones of fixes and features. Every change was test-first: a failing reproduction test (RED), then the fix (GREEN), then lint, the race suite and a weakened-test scan, then an independent review per stage. Each milestone's commits are kept separate so the history stays reviewable. Squash-merge as usual.

Behaviour changes callers may notice are listed under ⚠️ below.

M1: Streaming and retry core (bug fixes)

• The idle watchdog now measures only provider silence. A slow onDelta callback could previously trigger a false ErrIdleTimeout after partial output.
• OpenAI-compatible and Gemini mid-stream {"error":…} chunks now surface the provider's message. Before, they could end as an empty success. Non-object error keys are ignored.
• A bare [DONE] with nothing emitted is now a premature-close error, per invariant 3.
• The SSE parser strips a leading UTF-8 BOM, which silently dropped the first event, and accepts bare \r line endings.
• A huge Retry-After value no longer overflows into a negative duration.
• Oversized response bodies are terminal on chat, TTS, STT and models. Before, they were retried and billed up to 8 times.
• When a backoff or Retry-After wait outlasts the remaining budget, the call returns the last provider error immediately. Before, it slept until the deadline and returned a bare DeadlineExceeded.
• An https→http redirect to the same host is refused, so the API key is no longer resent in plaintext.

M2: Canonical mapping (bug fixes)

• OpenAI chat: stop together with tool calls now finishes tool_calls, in both buffered and streaming modes.
• Streamed parallel tool calls that arrive without an index are no longer merged into one corrupted call (invariant 8).
• An unknown Responses incomplete reason maps to "" (invariant 1). The precedence for tool-call turns is now documented in README and AGENTS.md.
• A streamed tool call with no arguments is {} on every accumulator, as it already was in buffered mode.
• Gemini IMAGE_SAFETY, IMAGE_PROHIBITED_CONTENT and IMAGE_RECITATION map to content_filter.

M3: Request validity (bug fixes)

• image/jpg is now accepted, as the README already promised.
• Empty or whitespace-only text blocks are never sent to Anthropic or Gemini. An empty Anthropic assistant turn is dropped and the adjacent user turns are merged.
• Gemini tools use parametersJsonSchema, so full JSON Schema works.
• The Gemini TTS URL is escaped and its models/ prefix stripped.
• STT formats text, srt and vtt return the raw body. Before, the transcription was billed and then failed.

M4: Anthropic current-model surface (feature)

• Thinking presets now send thinking:{type:"adaptive"} plus output_config.effort. budget_tokens is sent only when ThinkingBudget > 0, which remains the escape hatch for models older than 4.6. Background: budget_tokens returns a 400 on Opus 4.7+, Sonnet 5+, Fable and Haiku 5.5.
• JSON ResponseFormat now uses native output_config.format. It can be combined with thinking, tools and ToolChoice. Background: forced tool_choice returns a 400 on Opus 5.5, Sonnet 5.5 and Fable 5.1.
• A new AnthropicToolJSON registry quirk keeps the old forced-tool emulation, unchanged, for gateways that lack output_config.

M5: Models and embeddings (bug fixes)

• OpenAI-compatible embeddings request encoding_format:"base64". Large batches previously exceeded the 50MB body cap. Float arrays are still accepted as a fallback.
• A bare-array /v1/models response, as Together returns, is accepted.
• Anthropic has_more without a last_id is now an error instead of a silently truncated list.
• The models cache is deep-copied, and an empty list is cached.
• ListModels honours WithRetryPolicy, the request-timeout budget and Retry-After.

M6a: Response metadata and observability (feature)

• ChatResult.Refusal and DeltaRefusal on OpenAI chat, Responses and Anthropic. Refusal turns finish content_filter.
• ResponseMeta on chat, embed, speak and transcribe results: provider response ID, served model, request ID, and rate limits with a Reported flag. All fields are provider-reported or zero, never guessed.
• Per-request Headers, validated before any I/O. Reserved auth and framing headers are rejected, as are case-insensitive duplicates and control characters.
• WithAttemptObserver(AttemptEvent) fires on every HTTP attempt, on every entry point. A panicking observer is recovered, and keys are masked.

M6b: Inputs and helpers (feature)

• DocumentPart (PDF and plain text) and ImageURLPart / ImageURLPartMIME on every format, with boundary validation. MIME types are normalized to lower case.
• RunTools, a tool-loop helper. It replays thinking blocks and signatures through AssistantMessage() (invariant 7) and supports parallel handlers, MaxTurns (ErrMaxToolTurns) and an OnTurn hook.
• CountTokens for Anthropic and Gemini. Other formats return ErrUnsupported, never an estimate (invariant 5).
• TopK; a key-by-key merge of Gemini Extra["generationConfig"]; and EmbedRequest.TaskType for Gemini.
• Golden wire tests confirm that requests using none of the new fields produce byte-identical bodies on every format.
• Close-out: RunTools checks the context before each handler. On cancellation it returns the last completed result plus the context error, which is now documented. Real tests also cover the reachable lines that had lost coverage.

⚠️ Behaviour changes

• Anthropic thinking: presets now send adaptive thinking. On Claude models older than 4.6, set ThinkingBudget to keep budget_tokens.
• Anthropic JSON mode: native output_config.format. Anthropic-format gateways without it need the AnthropicToolJSON quirk.
• OpenAI-compatible embeddings: requests send encoding_format:"base64". The official OpenAI SDK sends this by default.
• Streaming: a bare [DONE] with no output, or a provider error chunk, is now an error instead of an empty success.
• Content: a message whose content would end up empty is rejected with ConfigError.
• Pre-existing tests whose expectations changed: each change was reviewed and accepted in its stage's review. They all encoded the old (buggy or replaced) contract.

Test plan

• go test -count=1 -timeout 240s -race ./...: green on the final head
• gofmt, go vet ./..., go vet -tags e2e ./..., golangci-lint run: clean
• RED→GREEN replay for every fix and feature slice, with a revert check
• Independent review per stage. The final review of every milestone passed.
• Live e2e (go test -tags e2e -run TestE2E -timeout 15m -v .): not run, because it needs provider keys and spends money. Recommended before merge, especially for Anthropic adaptive thinking and native JSON, OpenAI base64 embeddings, and Gemini parametersJsonSchema.

Known residuals (minor, non-blocking)

• Coverage is 98.5% (baseline 99.0%). The remaining gap is defensive marshal and unmarshal branches.
• Streaming Anthropic JSON mode with empty input: Content is {} but no matching delta is emitted.
• Gemini still sends a single-space text part for an empty assistant turn. Gemini accepts it.
• Batched Embed results carry Meta from the last batch only.
• On a mid-pagination failure, a paginated ListModels retry restarts from page 1.

🤖 Generated with Claude Code