Skip to content

fix(sdk): shared error-message vectors, codes and hint length as TypeScript reads them, sdk 1.31.2, python and go 0.10.2 - #730

Merged
kroqdotdev merged 3 commits into
mainfrom
t3/sdk-error-conformance-vectors
Nov 16, 2026
Merged

kroqdotdev merged 3 commits into
mainfrom
t3/sdk-error-conformance-vectors

Conversation

@kroqdotdev

@kroqdotdev kroqdotdev commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

What

The new packages/sdk/conformance/errors.json keeps the TypeScript, Python and Go SDKs from drifting apart on how they turn an API error answer into a message and a code. It follows the existing vector pattern. src/__tests__/conformance.test.ts writes the file, checks the TypeScript client against it, and fails when the file is stale. The Python and Go suites read the file from that directory and go through each SDK's public client.

Each of the 68 vectors has a status and a raw body. The expected values are:

  • message: the API's text before any hint the SDK adds (TypeScript's apiMessage).
  • code: the error code, or null.
  • hinted: whether a 401 or 404 gets the SDK's recovery hint. Only presence is compared; each SDK keeps its own hint wording.

The cases cover every rule from #729:

  • A bare code becomes the status in words, with the code in brackets when the code says more than the status.
  • The unknown-status fallback (418, 410).
  • message wins over error. A blank, U+FEFF-only or non-string message gives way to error, and a message that is only a code is treated as a code.
  • Codes with digits (e404, v2_bad). Values that are not codes: a bare 404, not-found, Not_found, non-ASCII, padded, a trailing newline, a sentence that equals the status words.
  • 1000- and 1200-character codes. The 1000-character JSON cut and the 200-character plain-body cut, neither of which splits a surrogate pair. A lone surrogate.
  • JSON strings, arrays, null and objects without text. Unterminated JSON, HTML and an empty body.
  • code fields, 5xx and 429.
  • The 29/30-character hint threshold for 401 and 404.

Behaviour changes (from round 1 review)

  • TypeScript SDK 1.31.2. code comes from the body's error when it is a code, wherever it sits in the object, else from a code field. Before, it was read only when error was the first key. The leading-key regex is kept only for a body that does not parse as JSON. The API always sends error first, so today's answers do not change.
  • Python 0.10.2.
    • _error_code, behind RateLimitError.code and InvalidCursorError, returns only values that are codes. A 429 with {"error":"Too many requests"} no longer has .code == "Too many requests".
    • The hint threshold counts UTF-16 units, as TypeScript does. It uses surrogatepass, so a lone surrogate counts as one unit instead of raising.
  • Go 0.10.2.
    • The unexported code, behind errors.Is(err, ErrInvalidCursor) and the busy-search check, is set only for a code.
    • The hint threshold counts UTF-16 units, as TypeScript does, instead of bytes.
  • Web. The hosted MCP server uses the workspace SDK, so this has a web changelog file and release: patch. MCP output does not change for any answer the API sends today, so MCP has no version bump or changelog line.

Where the ports still differ, and why

Each port's test lists these by vector name. A listed difference that stops differing fails the test, so the lists stay true.

  • code field. Python and Go read the code only from error, never from a code field. This affects {"error":"Polar did not answer","code":"outcome_unknown"} and {"error":"Search busy","code":"search_busy"}.
  • Not compared. A 429's message is each SDK's own rate-limit text, and Python's InvalidCursorError has its own message.
  • Go trims before the hint. Go trims a message before adding its hint, so its test checks that the hinted message starts with the trimmed text.

Other changes

  • Python's _plain_error_text and Go's plainErrorText are the existing code moved into a function the tests can call.
  • The hand-written tables that the vectors now cover are reduced, in all three SDKs, to what only they test: hint wording, Go's trim and the stream path.
  • conformance/README.md lists the new file.
  • Versions: packages/sdk/package.json, SDK_VERSION, PYTHON_SDK_VERSION, GO_SDK_VERSION, pyproject.toml, __init__.py, uv.lock and doc.go, with one changelog entry on each track.

Verified

  • pnpm --filter @webhooks-cc/sdk test (1009 passed) and tsc --noEmit
  • @webhooks-cc/mcp tests (263 passed)
  • Web lib/changelog.test.ts and scripts (98 passed)
  • uv run pytest (995 passed), ruff check, ruff format --check and mypy
  • go vet ./... and WHK_CONFORMANCE_REQUIRED=1 go test -race ./...
  • prettier on the changed files

🤖 Generated with Claude Code

packages/sdk/conformance/errors.json holds 66 cases of status and raw
body with the API's text before any hint, the error code as the
TypeScript SDK reads it, and whether a 401 or 404 gets a hint. The
TypeScript conformance test writes it and checks the client against it;
the Python and Go suites read it from that directory. Each port lists
the vectors where it differs on purpose (how it counts the 30-character
hint threshold, where it reads a code from), and a listed difference that
stops differing fails too. The hand-written tables the vectors now cover
are reduced to each SDK's own hint wording.

No behaviour change: Python's _plain_error_text and Go's plainErrorText
are the existing code moved into a function the tests can call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kroqdotdev kroqdotdev added the release: none Ships nothing to the web app, receiver or MX host: no version bump label Nov 16, 2026
@coderabbitai

coderabbitai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e2e00557-db32-4d30-8531-4d3f25702772
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions

Copy link
Copy Markdown

⚠️ Deprecation Warning: The deny-licenses option is deprecated for possible removal in the next major release. For more information, see issue 997.

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

…sdk 1.31.2, python and go 0.10.2

Review round 1 on #730.

- TypeScript reads `error` as the code wherever it sits in the JSON
  object, then a `code` field; the leading-key regex is kept only for a
  body that does not parse.
- Python's `_error_code` (behind RateLimitError.code and
  InvalidCursorError) and Go's unexported code take `error` only when it
  is a code, never a sentence.
- Python and Go count the 30-character hint threshold in UTF-16 units, as
  TypeScript does (Python with surrogatepass, so a lone surrogate counts
  as one unit).
- The Go conformance test goes through the public client; null-code
  vectors now assert that no code was read; the divergence lists keep
  only the `code` field, which Python and Go do not read.
- New vectors: a 429 sentence that equals the status words, and a lone
  surrogate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kroqdotdev kroqdotdev added release: patch Ships a fix or small feature: the next web release bumps the patch version and removed release: none Ships nothing to the web app, receiver or MX host: no version bump labels Nov 16, 2026
@kroqdotdev kroqdotdev changed the title test(sdk): shared conformance vectors for API error messages fix(sdk): shared error-message vectors, codes and hint length as TypeScript reads them, sdk 1.31.2, python and go 0.10.2 Nov 16, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kroqdotdev kroqdotdev added release: patch Ships a fix or small feature: the next web release bumps the patch version and removed release: patch Ships a fix or small feature: the next web release bumps the patch version labels Nov 16, 2026
@kroqdotdev
kroqdotdev merged commit 5e3c3da into main Nov 16, 2026
37 of 39 checks passed
@kroqdotdev
kroqdotdev deleted the t3/sdk-error-conformance-vectors branch November 16, 2026 02:03
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 16, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

release: patch Ships a fix or small feature: the next web release bumps the patch version

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant