feat(ctl): download marketplace collections per-schema into a named subdirectory#1057
Conversation
…ubdirectory `marketplace get --collection` previously fetched a ZIP and dumped every schema flat into the output directory with the version baked into each filename (e.g. `infrahub-dcim-1.0.0.yml`). Re-downloading accumulated stale versions side by side, and the naming was inconsistent with single-schema downloads (`dcim.yml`). Collections are now resolved via the collection metadata endpoint, which lists each member schema and its latest published version. Each member is downloaded individually through the existing `_download_schema` path, so naming, versioning, and error handling match single-schema downloads. Members land in `<output_dir>/<collection name>/<schema name>.yml`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deploying infrahub-sdk-python with
|
| Latest commit: |
5ec7a95
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://f3696379.infrahub-sdk-python.pages.dev |
| Branch Preview URL: | https://unleashed-nutmeg.infrahub-sdk-python.pages.dev |
Codecov Report❌ Patch coverage is
@@ Coverage Diff @@
## stable #1057 +/- ##
==========================================
+ Coverage 81.71% 81.73% +0.01%
==========================================
Files 135 135
Lines 11625 11626 +1
Branches 1759 1757 -2
==========================================
+ Hits 9499 9502 +3
+ Misses 1575 1574 -1
+ Partials 551 550 -1
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
2 issues found across 3 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
…e collisions Address review findings on the per-schema collection download: - The summary now counts schemas actually downloaded instead of every member entry, so skipped members (missing namespace/name) no longer inflate the count; skipped members also emit a warning instead of disappearing silently. - Members sharing a schema name across namespaces are disambiguated into <collection>/<namespace>/<name>.yml instead of silently overwriting each other. - The stdout `---` separator is tied to emitted documents rather than the raw member index. - Collection metadata items with unexpected shapes are filtered out instead of raising AttributeError. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lancamat1
left a comment
There was a problem hiding this comment.
LGTM ✅
Clean change — collection members now download per-schema into <output_dir>/<collection>/<schema>.yml, dropping the version suffix so re-downloads overwrite instead of accumulating stale files. Nice reuse of the existing _download_schema path so versioning, 404 handling, and stdout streaming stay consistent. The needs_separator logic for multi-doc stdout is correct (leading-only ---, guarded against doubling), and malformed/identity-less members degrade gracefully. Lint clean, 26 marketplace unit tests pass.
No blockers. A few non-critical follow-ups worth keeping in mind (not gating this):
namespace/namecome from the server and are used as path components — worth a guard against..//at some point (single-schema path has the same pre-existing exposure).- On a member fetch failure mid-run, the "run without --version" hint is misleading since the version came from collection metadata, not the user.
- Partial failure leaves earlier members on disk + exits non-zero; fine as-is, but a changelog note would help.
🤖 Generated with Claude Code
Why
infrahubctl marketplace get <collection> --collectiondumped every schema flat into the output directory with the version baked into each filename:Two problems:
1.1.0leaves the old…-1.0.0.ymlsitting next to the new file — globbingschemas/*.ymlthen loads conflicting versions.<name>.ymlwith no version.What changed
/api/v1/collections/{ns}/{name}), which lists each member schema and its latest published version — instead of downloading a ZIP archive._download_schemapath, so naming, versioning, and error handling are identical to single-schema downloads.<output_dir>/<collection name>/<schema name>.yml:Downloaded schema infrahub/dcim v1.0.0 -> …, consistent with single-schema downloads.namespace/nameis skipped rather than aborting the whole download.Things worth knowing
latest_version(not a frozen pin), so the old ZIP wasn't a frozen snapshot either — downloading each member at itslatest_versionreproduces exactly what the ZIP contained.How to review
infrahub_sdk/ctl/marketplace.py—_collection_urlnow points at the metadata endpoint;_download_collectionlists members and delegates to_download_schema;_download_schemagained aneeds_separatorflag for multi-doc stdout streaming.tests/unit/ctl/test_marketplace_app.py— collection tests rewritten to mock the metadata endpoint + per-member version downloads.Tests: 25 marketplace unit tests pass;
format,lint-code(ruff/ty/mypy) clean; generated docs unchanged.🤖 Generated with Claude Code
Summary by cubic
infrahubctl marketplace get --collectionnow downloads each member schema individually and saves them in<output_dir>/<collection>/<schema>.yml(using<collection>/<namespace>/<name>.ymlonly when names collide). This removes versioned filenames, prevents stale files, and matches single-schema naming.New Features
_download_schemafor consistent handling and logs.<collection>/<namespace>/<name>.yml.--stdoutmode, insert---between members when needed; per-file messages showDownloaded schema <ns>/<name> v<semver>.namespace/namewith a warning and ignore malformed items; the final count reflects schemas actually downloaded.Migration
<output_dir>/*.ymlto<output_dir>/<collection>/*.yml; when collisions exist, also handle<output_dir>/<collection>/*/*.yml. Re-downloads now overwrite cleanly.Written for commit 5ec7a95. Summary will update on new commits.