[DO NOT MERGE] flaky error experiment

[NaderAlAwar]

Test to try to figure out why CI is being flaky

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[NaderAlAwar]

/ok to test 42b0693

[NaderAlAwar]

/ok to test dba8569

[NaderAlAwar]

/ok to test 83f58ac

[NaderAlAwar]

/ok to test 83f58ac

[NaderAlAwar]

/ok to test fa5bf1b

[NaderAlAwar]

This CI job seems to show the issue with a decent traceback https://github.com/NVIDIA/cccl/actions/runs/33000505843/job/98289634645?pr=11002

[NaderAlAwar]

/ok to test a61001e

[NaderAlAwar]

/ok to test b476ab1

[github-actions]

🥳 CI Workflow Results

🟩 Finished in 38m 20s: Pass: 100%/8 | Total: 4h 29m | Max: 36m 40s

See results here.

AI failure analysis

1. Temporary matrix override intentionally fails workflow verification · 1 job

Explanation: The PR leaves `workflows.override` populated with the temporary `nvjitlink_repro` matrix. The verification step deliberately rejects any non-empty override, so the workflow cannot pass until this diagnostic configuration is reset.

Evidence:

Verify and summarize workflow results, step 3

2026-08-26T22:54:00.3519848Z ##[notice]Workflow matrix was overridden. Failing workflow.
2026-08-26T22:54:00.3520603Z Override matrix:
2026-08-26T22:54:00.3529883Z [{"jobs":["test"],"project":"nvjitlink_repro","ctk":["12.0","12.9","13.0","13.3"],"cxx":"msvc2022","gpu":"l4"}]

Root cause: `ci/matrix.yaml` intentionally contains a temporary diagnostic entry under `workflows.override`. The workflow verifier treats that state as an actionable configuration failure, independently of other failed jobs. Sources: ci/matrix.yaml:71.

Suggested next steps: Remove the `nvjitlink_repro` entry while retaining the empty `override:` key, then rerun the repository's focused matrix-generation or workflow-verification check.

Copy this prompt into a coding agent

Repository: https://github.com/NVIDIA/cccl
Workflow run: https://github.com/NVIDIA/cccl/actions/runs/33018806008
Failure group: Temporary matrix override intentionally fails workflow verification
Affected jobs:
- Verify and summarize workflow results: https://github.com/NVIDIA/cccl/actions/runs/33018806008/job/98351734163

Verify that workflow verification fails because `workflows.override` is populated in `ci/matrix.yaml`. Reproduce narrowly using the repository's matrix-generation or workflow-verification command, remove the temporary `nvjitlink_repro` override while preserving the empty `override:` key, and rerun focused validation. Keep the newly added reproducer project and job definitions unless they are separately intended for removal; the required fix is specifically to clear the temporary override.

Jobs:

• Verify and summarize workflow results

2. cuGraph temporaries select RMM's deleted device_scalar constructor · 1 job

Explanation: cuGraph compilation fails at repeated `rmm::device_scalar<size_t>` initializations that pass temporary values such as `size_t{0}`. The checked-out RMM header deletes the matching rvalue overload, preventing cuGraph and the RAPIDS compatibility job from building.

Evidence:

Build RAPIDS (optional) / rmm ucxx raft cuvs cugraph wholegraph, step 6

2026-08-26T22:24:48.2274873Z /home/coder/cugraph/cpp/include/cugraph/edge_partition_device_view.cuh(560): error: function "rmm::_RMM_26_10::device_scalar<T>::device_scalar(const rmm::_RMM_26_10::device_scalar<T>::value_type &&, rmm::_RMM_26_10::cuda_stream_view, cuda::mr::__4::__version_bump_ver4_::any_resource<cuda::mr::__4::device_accessible>) [with T=size_t]" (declared at line 122 of /home/coder/rmm/cpp/include/rmm/device_scalar.hpp) cannot be referenced -- it is a deleted function
2026-08-26T22:24:48.2277557Z       rmm::device_scalar<size_t> count(size_t{0}, stream);
2026-08-26T22:24:48.2278095Z                                        ^

Root cause: The RAPIDS job checks out both RMM and cuGraph from their current `main` branches, but the saved versions are API-incompatible: RMM rejects temporary-backed scalar construction while cuGraph still uses it. The logs do not record the cloned repositories' commit SHAs, so the exact upstream revision introducing the mismatch cannot be identified from the collected evidence.

Suggested next steps: Update the cuGraph call sites to pass an lvalue with sufficient lifetime or use RMM's supported initialization API; otherwise temporarily pin compatible RMM and cuGraph revisions. Reproduce with `RAPIDS_LIBS='rmm ucxx raft cuvs cugraph wholegraph' .devcontainer/launch.sh -d -c 13.3 -H rapids-conda -- ./ci/rapids/rapids-entrypoint.sh /bin/bash -li -c 'uninstall-all -j -qqq && clean-all -j && build-all -j0 -v || exec /bin/bash -li'`.

Copy this prompt into a coding agent

Repository: https://github.com/NVIDIA/cccl
Workflow run: https://github.com/NVIDIA/cccl/actions/runs/33018806008
Failure group: cuGraph temporaries select RMM's deleted device_scalar constructor
Affected jobs:
- Build RAPIDS (optional) / rmm ucxx raft cuvs cugraph wholegraph: https://github.com/NVIDIA/cccl/actions/runs/33018806008/job/98343901008

Reproduce the RAPIDS compatibility failure with `RAPIDS_LIBS='rmm ucxx raft cuvs cugraph wholegraph' .devcontainer/launch.sh -d -c 13.3 -H rapids-conda -- ./ci/rapids/rapids-entrypoint.sh /bin/bash -li -c 'uninstall-all -j -qqq && clean-all -j && build-all -j0 -v || exec /bin/bash -li'`. Inspect RMM's `cpp/include/rmm/device_scalar.hpp` deleted rvalue constructor and each failing cuGraph expression such as `rmm::device_scalar<size_t> count(size_t{0}, stream)`. Implement the smallest upstream-compatible correction by keeping the initial value in an lvalue with sufficient lifetime or using RMM's supported initialization API, apply it consistently to the failing cuGraph call sites, and run the focused cuGraph targets before rerunning the RAPIDS job. If this repository cannot modify cuGraph, confirm the mismatch against the checked-out upstream commits and temporarily pin a known-compatible RMM/cuGraph pair.

Jobs:

• Build RAPIDS (optional) / rmm ucxx raft cuvs cugraph wholegraph