fix(core): count blocks, not clusters, in the cooperative-launch check - #2578
Open
LeSingh1 wants to merge 1 commit into
Open
fix(core): count blocks, not clusters, in the cooperative-launch check#2578LeSingh1 wants to merge 1 commit into
LeSingh1 wants to merge 1 commit into
Conversation
`_check_cooperative_launch` compares the requested grid against a residency
limit expressed in thread blocks:
max_grid_size = (
kernel.occupancy.max_active_blocks_per_multiprocessor(...) * num_sm
)
if prod(config.grid) > max_grid_size:
but `config.grid` counts CLUSTERS, not blocks, whenever `cluster` is set.
LaunchConfig says so in its own docstring ("When cluster is specified, the
grid parameter represents the number of clusters (not blocks)"), and
`_to_native_launch_config` implements exactly that, multiplying grid by
cluster before filling in gridDimX/Y/Z.
So with e.g. cluster=(2, 2, 1) the guard under-counts by 4x: the driver is
asked for prod(grid) * prod(cluster) blocks while the check only looks at
prod(grid). A cooperative launch that genuinely over-subscribes the device
sails past the guard and fails (or deadlocks) inside the driver instead of
getting the clean ValueError this function exists to raise. The message was
misleading too -- it printed cluster counts as "grid size" and compared them
against a block limit.
Route the comparison through a small `_cooperative_block_count(config)`
helper that applies the same cluster multiplication as
`_to_native_launch_config`, and spell the block count out in the error.
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
_check_cooperative_launch(cuda_core/cuda/core/_launcher.pyx:76-83) compares the requested grid against a residency limit expressed in thread blocks:But
config.gridcounts clusters, not blocks, wheneverclusteris set.LaunchConfigsays so itself (_launch_config.pyx:33-37,42-44):and
_to_native_launch_configimplements exactly that (_launch_config.pyx:146-150):So for
LaunchConfig(grid=g, cluster=c, block=b, is_cooperative=True)the driver is asked forprod(g) * prod(c)blocks while the guard only inspectsprod(g)— withcluster=(2, 2, 1)it under-counts by 4x. A cooperative launch that genuinely over-subscribes the device passes the check and then deadlocks or fails inside the driver, instead of getting the clean up-frontValueErrorthis function exists to raise. The message compounds it: it prints cluster counts labelled "grid size" and compares them against a block limit.Fix
Route the comparison through
_cooperative_block_count(config), which applies the same cluster multiplication_to_native_launch_configdoes, and name the block count in the error message. No behaviour change whencluster is None.Tests
test_cooperative_block_count_counts_blocks_not_clustersasserts_cooperative_block_countreturns 6 forgrid=(2, 3, 1)with no cluster and 24 for the same grid withcluster=(2, 2, 1), and thatconfig.gridis still stored in cluster units.Deviceis mocked incuda.core._launch_configexactly as the two neighbouring tests (test_launch_config_cooperative_unsupported,test_to_native_launch_config_cooperative) already do, so it runs on any GPU rather than needing Hopper+ and cooperative-launch support.What I ran
Environment: macOS, no CUDA driver and no CUDA toolkit, so
cuda.corecannot be built or imported here.cuda_core/tests/test_launcher.py— they need a builtcuda.coreand a GPU.python -m py_compile,ruff check,ruff format --checkoncuda_core/tests/test_launcher.py.ruff checkreports the same single pre-existingI001on the file's top import block as it does onmain; no new findings."exceeds the limit"occurs exactly once in the tree, at the raise site itself.config.clusteris always eitherNoneor a 3-tuple (cast_to_3_tupleinLaunchConfig.__init__), soprod(config.cluster)is always well defined on the branch that uses it.Overlap note: #2066 also touches
_launcher.pyx(adding a separate cluster-support check) and removes theDevice()calls fromLaunchConfig.__init__that the neighbouring tests — and this new one — monkeypatch. It does not change the block/cluster accounting. If #2066 lands first this needs a trivial rebase; happy to do that.