Add --use-work-zarr option for intermediate OME-Zarr rechunking before segmentation/vessel processing#181
Draft
Copilot wants to merge 3 commits into
Draft
Conversation
Copilot
AI
changed the title
Add --use-work-zarr option for intermediate OME-Zarr rechunking
Add --use-work-zarr option for intermediate OME-Zarr rechunking before segmentation/vessel processing
Jun 23, 2026
Copilot created this pull request from a session on behalf of
akhanf
June 23, 2026 16:41
View session
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When input SPIM data has suboptimal chunking or lives on slow network/cloud storage, bias-field correction and vessel segmentation rules suffer from inefficient random reads. This adds an optional pre-processing step that rechunks the full multi-channel, multi-scale zarr to a fast local disk before those rules run.
New option
--use-work-zarr: requires--work_dirto point to a fast local disk with sufficient capacity. RaisesValueErrorat workflow init if--work_diris unset.Changes
snakebids.yml--use-work-zarrflag with help text noting the--work_dirrequirementSnakefile--work_diris set when--use-work-zarris activework_zarr_path(always.ome.zarrdirectory, never.ozx, rooted atwork_dir)spim_path_for_seg_vessels— routes downstream rules to either the rechunked work zarr or the original SPIM pathspim_json_overridesfor rechunked paths so JSON sidecar channel/orientation overrides remain effectiverules/rechunk.smkrules/rechunk.smk(new)rechunk_spim_to_workrule: one output per subject, contains all channels and all scale levels; output isdirectory(work_zarr_path)scripts/rechunk_spim.py(new).zattrsmetadata verbatim.ome.zarrdirectories,.ome.zarr.zip, and.ozxinputs; raises a clear error for.imsfinallyblockrules/segmentation.smk,rules/vessels.smkgaussian_biasfield,n4_biasfield, andrun_vesselfminputspimnow referencesspim_path_for_seg_vesselsinstead ofinputs["spim"].pathdirectly