cuda.core: Add copy_batch to cuda.core.utils

[juenglin]

Summary

Adds copy_batch to cuda.core.utils, wrapping cuMemcpyBatchAsync so a set of
transfers is issued in a single driver call. This is the P0 item of #1333; the P1
items (prefetch / discard / discard-prefetch) landed in #1775. Closes #1333.
Batched-only, mirroring the *_batch functions from #1775: a lone Buffer is
rejected in favour of Buffer.copy_to / Buffer.copy_from.

Public API

from cuda.core import Device, Host
from cuda.core.utils import CopyOptions, MemcpySrcAccessOrder, copy_batch
# One option applies to every copy...
copy_batch(stream, srcs, dsts, options=CopyOptions(src_access_order=MemcpySrcAccessOrder.STREAM))
# ...or one per copy, paired by index.
copy_batch(stream, srcs, dsts, options=[opt_a, opt_b, opt_c])
# Location hints, for managed-memory operands.
copy_batch(stream, srcs, dsts, options=CopyOptions(
    src_access_order=MemcpySrcAccessOrder.STREAM,
    src_location_hint=Device(0),
    dst_location_hint=Host(),
))

New exports from cuda.core.utils: copy_batch, CopyOptions,
MemcpySrcAccessOrder, MemcpyOverlapMode. Nothing is added to top-level
cuda.core.
CopyOptions is a frozen dataclass mapping onto CUmemcpyAttributes:
src_access_order, src_location_hint, dst_location_hint, overlap_mode.
options=None selects stream-ordered defaults.

Behaviour

• Source and destination sizes must match, that is src.size must equal dst.size.
  Partial-buffer copies are out of scope.
• A scalar CopyOptions broadcasts to every copy; a sequence pairs by index and
  must match len(srcs), otherwise ValueError. Non-default CopyOptions require
• Options are run-length encoded into the driver's attrs / attrsIdxs pair, so
  a broadcast reaches the driver as numAttrs == 1 instead of being repeated per
  copy.
• On a CUDA 12 build of cuda.bindings or when using an older driver, copy_batch is emulated by a sequence of cuMemcpyAsync calls, consistent with prefetch_batch.
• Not capturable into a graph. Use GraphNode.memcpy for graph copies.
• overlap_mode="prefer_overlap_with_compute" is a Tegra-only hint. On a
  non-integrated GPU the driver silently ignores it, so a UserWarning is emitted
  and the copy proceeds with default behaviour.

Refactor to existing files

_coerce_batch_buffers was duplicated between the new _copy_ops.pyx and _managed_memory_ops.pyx, so it now lives once as Buffer_coerce_batch in _buffer.pyx / _buffer.pxd, parameterized by the per-buffer API to suggest. That accounts for the changes to _buffer.* and _managed_memory_ops.pyx (no behaviour changes for the #1775 functions).

Relation to the #1775 batched-API contract

Follows the contract on pairing, scalar broadcast, ValueError on length mismatch, and options as a frozen dataclass. Three deliberate deviations, which match what #1775 shipped
rather than what that comment described:

• No unified copy() accepting one-or-many. Add managed-memory advise, prefetch, and discard-prefetch free functions #1775 shipped batch-only free
  functions with instance methods for N=1; this mirrors that split.
• Parallel srcs / dsts sequences rather than a sequence of pairs, matching
  prefetch_batch(stream, buffers, locations).
• A single stream argument: cuMemcpyBatchAsync takes one CUstream, so there
  is no per-copy stream array to mirror.

Tests

tests/memory/test_copy_batch.py covers data movement — H2D, D2H, D2D, mixed
sizes, agreement with sequential Buffer.copy_to, stream ordering, and the
graph-capture rejection. tests/memory/test_copy_batch_options.py covers the
options surface, the attribute run-length encoding, and every validation path.
Shared fixtures live in tests/memory/conftest.py. The new
examples/batched_memcpy.py is exercised by the existing example-test glob.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[juenglin]

/ok to test

[copy-pr-bot]

/ok to test

@juenglin, there was an error processing your request: E1

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/1/

[juenglin]

/ok to test fd58e8f