The Python lane: opendpp-sdk on PyPI, and one shared spec normalizer for every generated client - #20
Merged
Merged
Conversation
… spec normalizer The five generation-input transforms (webhooks collision, @context polymorphism, additionalProperties -> extends HashMap, 3.1 multi-type containers, Accept-union content negotiation) move to scripts/normalize-spec.mjs — authored and unit-tested upstream in opendpp-node (scripts/lib/spec-codegen-normalize.mjs) and synced here like CHANGELOG.md, so every openapi-generator lane (Java today, Python next) reads ONE tested rewrite instead of a private per-lane one. prepareSpec becomes an Exec task invoking it; the generated tree is byte-identical (proven by wipe + regen + zero git diff against the same 1.13.0 contract).
…h the shared normalizer The third SDK lane, at parity with the existing two and generated with the Java lane's exact toolchain (openapi-generator 7.12.0, pydantic v2 + urllib3, sync): - python/opendpp_sdk/** committed + drift-checked (scripts/regenerate.sh is the one regeneration entrypoint: shared normalizer -> wipe -> generate). Two config choices are LOAD-BEARING, each documented where it lives: disallowAdditionalPropertiesIfNotPresent=false (the generator default turns ABSENT additionalProperties into pydantic extra="forbid") and --name-mappings @id/@type/@context=at_* (the JSON-LD documents carry both @id and id; the generator strips the @ and one attribute silently shadows the other). - opendpp_sdk/ergonomics.py — the only hand-written module: TypeScript-lane parity (create_opendpp_client + the four typed resolve_*_as helpers for the content-negotiated public resolvers). - scripts/check_version_lock.py — the TS lane's version lock, plus one python-specific check (the generated __version__ must match pyproject.toml). - tests: offline smoke + negotiation (mocked at the ApiClient.call_api boundary) and the opt-in OPENDPP_LIVE_TEST=1 payload-truth suite, which parses real hosted-node responses into the generated models. That live suite is what caught the two generator defects fixed in this change. - workflows: ci.yml python job (regen-drift, version lock, build, test), publish-sdk-py.yml (sdk-py-* tag, environment: release, keyless PyPI trusted publishing), drift-check COPIES + release.yml ARTIFACTS rows. The normalizer gains transform 6 (authored upstream, synced here): boolean single-value constraints are stripped from the generation input — the generator stringifies the allowed value into its validators, so python rejected every real ok:true / success:false payload and Java wrapped plain booleans in one-value SuccessEnum/OkEnum classes with a magic sentinel. The Java models regenerate accordingly (same wire format, simpler surface); handwritten code referenced none of the removed enum types and the full gradle build + tests pass.
…eleted inline step The section still described the Groovy pre-generation step this branch removed; it now names ../scripts/normalize-spec.mjs, the node-on-PATH requirement, and the boolean single-value transform the live tests forced.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The third SDK lane —
opendpp-sdkon PyPI (pydantic v2 + urllib3, sync, Python ≥ 3.9) — generated with the Java lane's exact toolchain (openapi-generator 7.12.0), plus the refactor that makes a third lane sane: the Java build's private GroovyprepareSpecbecomes ONE shared, upstream-tested normalizer (scripts/normalize-spec.mjs, authored in the node repo and synced here likeCHANGELOG.md).Why the normalizer moved
Python hits the same OpenAPI-3.1 constructs Java did and fails differently —
additionalProperties: falsebecomes pydanticextra=\"forbid\", i.e. runtime rejection of real payloads the moment the server adds a field. A second private rewriter would have made three lanes with two-and-counting normalizations. Now both openapi-generator lanes read one tested rewrite; the retrofit is proven inert (wipe + regen at contract 1.13.0 → zero diff overjava/src/main/java).What the live payload-truth suite caught
The new
python/tests/test_live.py(opt-in,OPENDPP_LIVE_TEST=1) parses real hosted-node responses into the generated models — and found two generator defects on its first run:if value not in set(['true'])) — every realok: true/success: falsepayload failed to parse. Fixed as normalizer transform 6 (strip booleanconst/enumfrom the generation input); the Java models simplify too (the one-valueSuccessEnum/OkEnumwrappers with theBoolean.valueOf(\"11184809\")sentinel become plainBoolean— same wire format).@id/idattribute shadowing — the generator strips the@, emitting two class attributes with one name; the second silently shadows the first. Fixed with--name-mappings @id=at_id,@type=at_type,@context=at_context.With both fixes: 14/14 tests pass including all live tests against the production node, and the generated models capture unknown response fields losslessly in
additional_propertiesinstead of raising.Lane furniture
Same shape as the siblings: committed generated sources +
scripts/regenerate.shdrift guard,scripts/check_version_lock.py(major.minor locked to the vendored contract; the generated__version__must matchpyproject.toml), hand-writtenopendpp_sdk/ergonomics.pyat TypeScript-lane parity (create_opendpp_client+ the four typedresolve_*_ashelpers),publish-sdk-py.yml(tagsdk-py-*,environment: release, keyless PyPI trusted publishing — the pending-publisher flow needs no bootstrap token), theci.ymlpython job,drift-checkCOPIES andrelease.ymlARTIFACTS rows.Wheel + sdist build clean (
twine checkpasses); the wheel ships the package,ergonomics, andpy.typed— no generated docs/tests.Sequencing
This PR vendors contract 1.13.0 (byte-identical across all three lanes). The upstream contract-hygiene release (1.14.0 — named unions,
constdiscriminants, tolerant responses) lands in the node repo; itssdk-mirror-syncrun regenerates all three lanes here post-merge. Publishing stays a deliberate tag act (sdk-py-<semver>), gated on the PyPI trusted publisher being configured forpublish-sdk-py.yml.