Repository navigation
NanoVDB: add GridHandle::adopt and metaData, deprecate HandleFactory - #2371
Conversation
harrism
left a comment
There was a problem hiding this comment.
Thanks for this. Public adopt/metaData() is the right API, both for fvdb (item 2 on #2232) and for a future capturable build. Requesting changes for two correctness issues that public adopt makes reachable. Inline comments carry the details, smaller items included.
Blocking
- Heap overrun in
mergeGrids/splitGrids. Both size or copy from the metadata but advance by each header'smGridSize. Before this PR the two always agreed, because metadata came from parsing the headers.adoptlets them differ; the test's own padded-buffer case does exactly that. copyTonow rejects handles the constructors accept.adoptrequires every grid offset to be a multiple ofNANOVDB_DATA_ALIGNMENT, butparseHostGridChainand the device parse don't check alignment.copyTogoes throughadopt, so it throws for unpadded grids (e.g. the Point grids #2329 fixes, or older files), and only after it has allocated, copied and synchronized.
Also
splitGridsstill default-constructs its output handles, so the pool path for non-default-constructible buffers can't be reached there.- The new host-accessible single-space support for merge/split does a host
memcpywithout ordering after the sources' retained streams. copyTovalidates only after the copy.- Two comments now overstate what has been validated:
parseHostGridChain("safe to adopt without re-validation") andcopyTo("validated when that handle was constructed from raw bytes"). - Test nits, inline.
Ordering with other PRs
|
Thanks @harrism. 6d4e56c addresses the review, and each inline thread has a reply.
Ordering: I'm assuming #2329 and #2370 land before this PR. I'll then rebase and switch #2370's Not changed here: host Tests: |
harrism
left a comment
There was a problem hiding this comment.
Thanks. Everything from my review is resolved, and I built and ran the suites at 6d4e56c (host 157, CUDA 66, buffer 50, memory-resource 19, all passing on one GPU).
- Overrun: fixed.
mergeGrids/splitGridswalk by the metadata and throw on a header/metadata mismatch before copying. - Alignment vs
copyTo:detail::LayoutCopyis a better fix than the one I suggested. Publicadoptkeeps the check, while handle-to-handle copies carry whatever layout the constructors accepted. You're right that landing #2329 first wouldn't have covered existing files. - Stream ordering: agreed.
PinnedResourceis synchronous andcopyTosynchronizes a host-readable destination, so the only gap is a user-supplied stream-ordered host resource, and a@notein the CUDA-free header is the right place for it. gridType: rejectingEndwhile keepingUnknownto match the parser makes sense.
One non-blocking note for the description or the pending-changes entry: the deprecated HandleFactory::make now forwards to adopt, so existing callers (fvdb's GridStorage and batched-builder layout wrapper) gain adopt's checks, including alignment, which HandleFactory never applied. That's fine if fvdb's layouts are always aligned, but it's a behaviour change worth stating.
|
Thanks @harrism. d5672ca adds the |
d5672ca to
9c25e3c
Compare
GridHandle::adopt(buffer, meta) constructs a handle over bytes whose layout the caller already knows: a copy of another handle's grids, or grids the caller laid out itself. No grid header is read, so a device buffer is wrapped without the parse kernel, readback and synchronization the public constructor pays, and buffers over non-default-constructible resources need no default construction. adopt checks only the layout arithmetic: grids contiguous from offset zero, each starting at a multiple of NANOVDB_DATA_ALIGNMENT, each at least a header, all within the buffer, and an empty layout only over an empty buffer. GridHandle::metaData() exposes the layout to pass back in. cuda::copyTo and GridHandle::copy now use adopt. cuda::detail::HandleFactory, which downstream code (fvdb) has been reaching into for the same purpose, becomes a deprecated forwarder to adopt and metaData(). mergeGrids and splitGrids also accept host-accessible single-space handles such as GridHandle<cuda::Buffer<std::byte, PinnedResource>>, allocating through the pool's resource or a default-constructed one. New tests: GridHandleAdoptRejectsBadMeta covers each layout rejection and a byte-for-byte adopt round trip; mergeSplitPinnedGrids checks pinned merge and split against the HostBuffer result byte for byte. Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
mergeGrids and splitGrids walk grids by the handle's metadata and throw when a grid header's size disagrees with it. Before, they sized from the metadata but advanced by mGridSize, so an adopted layout smaller than its headers overran the output buffer. splitGrids also builds its output with push_back, so a buffer over a non-default-constructible resource can use the pool path. cuda::copyTo reuses the source's layout through detail::LayoutCopy instead of adopt. The constructors accept grids at unaligned offsets (e.g. Point grids written before AcademySoftwareFoundation#2329), and copyTo no longer rejects them after the copy. The device GridHandle::copy() does the same. Public adopt keeps its alignment check and now also rejects GridType::End. Comments that overstated how much an adopted layout was validated are corrected, and merge/split document that they do not synchronize stream-ordered buffers. Tests: mergeSplitRejectMismatchedMetaData (heap corruption before this fix), CopyToKeepsUnalignedGrids, an End-type adopt case, and a ResourceRef pinned merge/split with a pool. Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
…::adopt The single-space merge and split from AcademySoftwareFoundation#2370 read layouts through GridHandle::metaData() and build their results with GridHandle::adopt instead of the deprecated HandleFactory. mergeGridHandles rejects a grid that would start at an unaligned offset before it allocates, since adopt would otherwise reject the merged layout only after the copies. Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
c5e839a to
27e27e5
Compare
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
The merge resolution dropped the test's closing brace, so the next test was parsed inside its body. Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
…e-adopt Signed-off-by: Jonathan Swartz <jonathan@jswartz.info> # Conflicts: # nanovdb/nanovdb/cuda/GridHandle.cuh
…le::adopt The empty-source path that came in from master called the deprecated HandleFactory, which fails builds with warnings as errors. Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Addresses item 2 of the 2026-09-17 comment on #2232, which asks for a public way to adopt validated metadata so downstream code can stop calling
cuda::detail::HandleFactory.Summary
GridHandle::adopt(buffer, meta)andGridHandle::metaData()construct a handle over bytes whose layout the caller already knows: a copy of another handle's grids, or grids the caller laid out itself.ResourceRef, work without one.adoptchecks only the layout arithmetic. Grids must be contiguous from offset zero, each must start at a multiple ofNANOVDB_DATA_ALIGNMENT, each must be at least a header, and all must fit inside the buffer. An empty layout is accepted only over an empty buffer.cuda::copyToand the deviceGridHandle::copy()reuse the source handle's layout without re-checking it, so they accept any layout the constructors accept, including unaligned grids in older files.cuda::detail::HandleFactoryis deprecated and forwards toadopt/metaData(). Downstream code has used it for this purpose. fvdb-core calls it fromGridStorageand its batched-builder layout wrapper (Finish adopting NanoVDB's injectable CUDA memory resources (openvdb #2232): move grid storage off the dual-space TorchDeviceBuffer openvdb/fvdb-core#770). Those callers can switch to the public API, and the forwarder can be removed in a later release. Behavior change:HandleFactory::makeapplied no checks before, and now appliesadopt's layout checks, including alignment, so a layoutadoptrejects now throws.mergeGrids/splitGridsaccept host-accessible single-space handles, such asGridHandle<cuda::Buffer<std::byte, PinnedResource>>. They allocate through the pool's resource, or through a default-constructed resource when no pool is given.This complements #2370, which adds single-space
cuda::mergeGridHandles/splitGridHandles. #2370 currently callsdetail::HandleFactory, so whichever PR lands second should move those calls toadopt/metaData().Test plan
TestBuffer.GridHandleAdoptRejectsBadMetacovers each layout rejection (no grids over a non-empty buffer, smaller than a header, past the end, nonzero start, gap, overlap, misaligned offset) and anadoptround trip of a byte-for-byte copy.TestNanoVDBCUDA.mergeSplitPinnedGridschecks pinned merge and split, with and without a pool, byte for byte against theHostBufferresult.HandleFactory::make/metagets the deprecation warning and a working handle.nanovdb_test_nanovdb156,nanovdb_test_cuda66,nanovdb_test_cuda_buffer49,nanovdb_test_cuda_memory_resource19 and the util suites all pass locally (CUDA 13.2, sm_120,NANOVDB_CUDA_WERRORon).