pcgv3: manifest v2 for non co-located seg and meshes#543
Open
akhileshh wants to merge 8 commits into
Open
Conversation
MeshMeta wraps a ChunkedGraph and exposes mesh dir locations and per-dataset params from custom_data["mesh"] (the single source of truth), shared branch-agnostically across the pcgv2/pcgv3 lines. Resolves absolute initial/dynamic dirs so meshes can live in a different bucket than the watershed, and flags graphs old clients can't reach. Co-Authored-By: Claude <noreply@anthropic.com>
Branch-agnostic manifest/v2.py: parse the client's advertised manifest version from the Accept header, and assemble the v2 format that groups fragments by absolute mesh bucket path so mesh location travels in the manifest itself. Co-Authored-By: Claude <noreply@anthropic.com>
Replace custom_data["mesh"] reads (mesh dir, max_layer, initial_ts, dynamic path) with MeshMeta, making the meshing manifest byte-identical to the main line so v2 changes cherry-pick clean both ways. Co-Authored-By: Claude <noreply@anthropic.com>
Serve the v2 grouped-by-bucket manifest to clients that advertise manifest_version=2, else the v1 flat list (byte-identical). Refuse with 406 when a migrated graph is requested without v2, so old clients fail loudly instead of getting unreachable paths. Co-Authored-By: Claude <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## pcgv3 #543 +/- ##
=======================================
Coverage 67.66% 67.66%
=======================================
Files 89 89
Lines 8624 8626 +2
Branches 1149 1149
=======================================
+ Hits 5835 5837 +2
Misses 2600 2600
Partials 189 189
🚀 New features to boost your workflow:
|
Anchor the verify=1 shard reader at initial_path (via reader_anchor) so migrated datasets read initial meshes from the real bucket, and append Accept-Encoding to Vary so manifests keep advertising Accept. Co-Authored-By: Claude <noreply@anthropic.com>
_remeshing wrote unsharded meshes to the dataset_info-derived path, not the dynamic_path the v2 manifest advertises, so clients 404 on freshly remeshed meshes. Source mesh paths from MeshMeta. Co-Authored-By: Claude <noreply@anthropic.com>
akhileshh
force-pushed
the
meshing-v2-pcgv3
branch
from
July 13, 2026 20:31
d471fa3 to
fda70e6
Compare
v2 emits each fragment identical to v1 (seg id prepended when asked) and honors return_seg_ids, differing only by grouping fragments under their bucket key, so the client reuses its v1 handling unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
akhileshh
force-pushed
the
meshing-v2-pcgv3
branch
from
July 13, 2026 20:45
fda70e6 to
d126612
Compare
Emit initial/dynamic absolute bucket locations in mesh_metadata on both the served /info endpoint and get_json_info, so external clients and PCG's own manifest-bypass reads resolve meshes from the real buckets. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.