Skip to content

NanoVDB: add GridHandle::adopt and metaData, deprecate HandleFactory - #2371

Merged
swahtz merged 8 commits into
AcademySoftwareFoundation:masterfrom
swahtz:nanovdb-gridhandle-adopt
Oct 22, 2026
Merged

swahtz merged 8 commits into
AcademySoftwareFoundation:masterfrom
swahtz:nanovdb-gridhandle-adopt

Conversation

@swahtz

@swahtz swahtz commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Addresses item 2 of the 2026-09-17 comment on #2232, which asks for a public way to adopt validated metadata so downstream code can stop calling cuda::detail::HandleFactory.

Summary

  • GridHandle::adopt(buffer, meta) and GridHandle::metaData() construct a handle over bytes whose layout the caller already knows: a copy of another handle's grids, or grids the caller laid out itself.
    • No grid header is read, so a device buffer is wrapped without the parse kernel, readback and sync that the public constructor costs.
    • Buffers over resources that can't be default-constructed, such as a ResourceRef, work without one.
    • adopt checks only the layout arithmetic. Grids must be contiguous from offset zero, each must start at a multiple of NANOVDB_DATA_ALIGNMENT, each must be at least a header, and all must fit inside the buffer. An empty layout is accepted only over an empty buffer.
    • cuda::copyTo and the device GridHandle::copy() reuse the source handle's layout without re-checking it, so they accept any layout the constructors accept, including unaligned grids in older files.
  • cuda::detail::HandleFactory is deprecated and forwards to adopt/metaData(). Downstream code has used it for this purpose. fvdb-core calls it from GridStorage and its batched-builder layout wrapper (Finish adopting NanoVDB's injectable CUDA memory resources (openvdb #2232): move grid storage off the dual-space TorchDeviceBuffer openvdb/fvdb-core#770). Those callers can switch to the public API, and the forwarder can be removed in a later release. Behavior change: HandleFactory::make applied no checks before, and now applies adopt's layout checks, including alignment, so a layout adopt rejects now throws.
  • mergeGrids/splitGrids accept host-accessible single-space handles, such as GridHandle<cuda::Buffer<std::byte, PinnedResource>>. They allocate through the pool's resource, or through a default-constructed resource when no pool is given.

This complements #2370, which adds single-space cuda::mergeGridHandles/splitGridHandles. #2370 currently calls detail::HandleFactory, so whichever PR lands second should move those calls to adopt/metaData().

Test plan

  • New TestBuffer.GridHandleAdoptRejectsBadMeta covers each layout rejection (no grids over a non-empty buffer, smaller than a header, past the end, nonzero start, gap, overlap, misaligned offset) and an adopt round trip of a byte-for-byte copy.
  • New TestNanoVDBCUDA.mergeSplitPinnedGrids checks pinned merge and split, with and without a pool, byte for byte against the HostBuffer result.
  • A standalone caller of HandleFactory::make/meta gets the deprecation warning and a working handle.
  • nanovdb_test_nanovdb 156, nanovdb_test_cuda 66, nanovdb_test_cuda_buffer 49, nanovdb_test_cuda_memory_resource 19 and the util suites all pass locally (CUDA 13.2, sm_120, NANOVDB_CUDA_WERROR on).
  • CI

@harrism harrism left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. Public adopt/metaData() is the right API, both for fvdb (item 2 on #2232) and for a future capturable build. Requesting changes for two correctness issues that public adopt makes reachable. Inline comments carry the details, smaller items included.

Blocking

  1. Heap overrun in mergeGrids/splitGrids. Both size or copy from the metadata but advance by each header's mGridSize. Before this PR the two always agreed, because metadata came from parsing the headers. adopt lets them differ; the test's own padded-buffer case does exactly that.
  2. copyTo now rejects handles the constructors accept. adopt requires every grid offset to be a multiple of NANOVDB_DATA_ALIGNMENT, but parseHostGridChain and the device parse don't check alignment. copyTo goes through adopt, so it throws for unpadded grids (e.g. the Point grids #2329 fixes, or older files), and only after it has allocated, copied and synchronized.

Also

  • splitGrids still default-constructs its output handles, so the pool path for non-default-constructible buffers can't be reached there.
  • The new host-accessible single-space support for merge/split does a host memcpy without ordering after the sources' retained streams.
  • copyTo validates only after the copy.
  • Two comments now overstate what has been validated: parseHostGridChain ("safe to adopt without re-validation") and copyTo ("validated when that handle was constructed from raw bytes").
  • Test nits, inline.

Ordering with other PRs

  • #2370 calls HandleFactory. Whichever PR lands second switches those calls to adopt/metaData(). #2370's single-space merge/split already walk by metadata only, so they don't have issue 1.
  • #2329 should land before this PR because of issue 2.

Comment thread nanovdb/nanovdb/GridHandle.h Outdated
Comment thread nanovdb/nanovdb/GridHandle.h Outdated
Comment thread nanovdb/nanovdb/GridHandle.h
Comment thread nanovdb/nanovdb/GridHandle.h
Comment thread nanovdb/nanovdb/GridHandle.h
Comment thread nanovdb/nanovdb/cuda/HandleStorage.h Outdated
Comment thread nanovdb/nanovdb/unittest/TestNanoVDB.cu
Comment thread nanovdb/nanovdb/unittest/TestBuffer.cu Outdated
@swahtz
swahtz marked this pull request as ready for review October 20, 2026 00:27
@swahtz

swahtz commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @harrism. 6d4e56c addresses the review, and each inline thread has a reply.

  • Merge/split overrun: both now walk by the metadata and throw when a header's mGridSize disagrees with it. splitGrids uses push_back, so the pool path works for non-default-constructible buffers.
  • copyTo alignment: copyTo and the device copy() reuse the source layout through detail::LayoutCopy, so they accept whatever the constructors accept. Public adopt keeps the alignment check and now also rejects GridType::End.
  • Comments: corrected where they overstated the validation. Merge/split document that they don't synchronize stream-ordered buffers.

Ordering: I'm assuming #2329 and #2370 land before this PR. I'll then rebase and switch #2370's HandleFactory calls to adopt/metaData(). That switch is required, because the deprecation warning would otherwise fail the build under NANOVDB_CUDA_WERROR.

Not changed here: host GridHandle::copy(pool) still can't target a non-default-constructible buffer, because its empty-source path default-constructs the result. That predates this PR.

Tests: nanovdb_test_nanovdb 157, nanovdb_test_cuda 66, nanovdb_test_cuda_buffer 50 and nanovdb_test_cuda_memory_resource 19 pass locally. mergeSplitRejectMismatchedMetaData aborts with heap corruption against the previous commit, and CopyToKeepsUnalignedGrids fails against it.

@harrism harrism left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Everything from my review is resolved, and I built and ran the suites at 6d4e56c (host 157, CUDA 66, buffer 50, memory-resource 19, all passing on one GPU).

  • Overrun: fixed. mergeGrids/splitGrids walk by the metadata and throw on a header/metadata mismatch before copying.
  • Alignment vs copyTo: detail::LayoutCopy is a better fix than the one I suggested. Public adopt keeps the check, while handle-to-handle copies carry whatever layout the constructors accepted. You're right that landing #2329 first wouldn't have covered existing files.
  • Stream ordering: agreed. PinnedResource is synchronous and copyTo synchronizes a host-readable destination, so the only gap is a user-supplied stream-ordered host resource, and a @note in the CUDA-free header is the right place for it.
  • gridType: rejecting End while keeping Unknown to match the parser makes sense.

One non-blocking note for the description or the pending-changes entry: the deprecated HandleFactory::make now forwards to adopt, so existing callers (fvdb's GridStorage and batched-builder layout wrapper) gain adopt's checks, including alignment, which HandleFactory never applied. That's fine if fvdb's layouts are always aligned, but it's a behaviour change worth stating.

@swahtz

swahtz commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @harrism. d5672ca adds the HandleFactory::make behavior change to the pending-changes entry. It previously applied no checks, and now applies adopt's layout checks, including alignment. I've also noted it in the description, and corrected the description's stale line about copyTo using adopt.

GridHandle::adopt(buffer, meta) constructs a handle over bytes whose
layout the caller already knows: a copy of another handle's grids, or
grids the caller laid out itself. No grid header is read, so a device
buffer is wrapped without the parse kernel, readback and synchronization
the public constructor pays, and buffers over non-default-constructible
resources need no default construction. adopt checks only the layout
arithmetic: grids contiguous from offset zero, each starting at a
multiple of NANOVDB_DATA_ALIGNMENT, each at least a header, all within
the buffer, and an empty layout only over an empty buffer.
GridHandle::metaData() exposes the layout to pass back in. cuda::copyTo
and GridHandle::copy now use adopt.

cuda::detail::HandleFactory, which downstream code (fvdb) has been
reaching into for the same purpose, becomes a deprecated forwarder to
adopt and metaData().

mergeGrids and splitGrids also accept host-accessible single-space
handles such as GridHandle<cuda::Buffer<std::byte, PinnedResource>>,
allocating through the pool's resource or a default-constructed one.

New tests: GridHandleAdoptRejectsBadMeta covers each layout rejection
and a byte-for-byte adopt round trip; mergeSplitPinnedGrids checks
pinned merge and split against the HostBuffer result byte for byte.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
mergeGrids and splitGrids walk grids by the handle's metadata and throw
when a grid header's size disagrees with it. Before, they sized from the
metadata but advanced by mGridSize, so an adopted layout smaller than its
headers overran the output buffer. splitGrids also builds its output with
push_back, so a buffer over a non-default-constructible resource can use
the pool path.

cuda::copyTo reuses the source's layout through detail::LayoutCopy
instead of adopt. The constructors accept grids at unaligned offsets
(e.g. Point grids written before AcademySoftwareFoundation#2329), and copyTo no longer rejects
them after the copy. The device GridHandle::copy() does the same. Public
adopt keeps its alignment check and now also rejects GridType::End.

Comments that overstated how much an adopted layout was validated are
corrected, and merge/split document that they do not synchronize
stream-ordered buffers.

Tests: mergeSplitRejectMismatchedMetaData (heap corruption before this
fix), CopyToKeepsUnalignedGrids, an End-type adopt case, and a
ResourceRef pinned merge/split with a pool.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
…::adopt

The single-space merge and split from AcademySoftwareFoundation#2370 read layouts through
GridHandle::metaData() and build their results with GridHandle::adopt
instead of the deprecated HandleFactory. mergeGridHandles rejects a grid
that would start at an unaligned offset before it allocates, since adopt
would otherwise reject the merged layout only after the copies.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
@swahtz
swahtz force-pushed the nanovdb-gridhandle-adopt branch from c5e839a to 27e27e5 Compare October 21, 2026 01:06
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
@swahtz swahtz added the vdb13.2 label Oct 21, 2026
The merge resolution dropped the test's closing brace, so the next test
was parsed inside its body.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
…e-adopt

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

# Conflicts:
#	nanovdb/nanovdb/cuda/GridHandle.cuh
…le::adopt

The empty-source path that came in from master called the deprecated
HandleFactory, which fails builds with warnings as errors.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
@swahtz swahtz added the nanovdb label Oct 21, 2026
Comment thread nanovdb/nanovdb/GridHandle.h

@apradhana apradhana left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@swahtz
swahtz merged commit 90ed1ec into AcademySoftwareFoundation:master Oct 22, 2026
16 checks passed
@swahtz
swahtz deleted the nanovdb-gridhandle-adopt branch October 22, 2026 00:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants