Skip to content

Fix the GPU build with NVHPC 26.9: let -cudalib=nvtx3 pick the NVTX wrapper - #2107

Closed
ruru25254649 wants to merge 3 commits into
MFlowCode:masterfrom
ruru25254649:fix-nvhpc-nvtx-link
Closed

ruru25254649 wants to merge 3 commits into
MFlowCode:masterfrom
ruru25254649:fix-nvhpc-nvtx-link

Conversation

@ruru25254649

Copy link
Copy Markdown
Contributor

Summary

When the CUDA toolkit is 12.9 or newer, so CUDA::nvToolsExt does not exist, MFCTargets.cmake links NVHPC's NVTX wrapper twice:

target_link_libraries(${a_target} PRIVATE nvhpcwrapnvtx)
target_link_options(${a_target} PRIVATE "-cudalib=nvtx3")

NVHPC 26.9 ships the library as libnvhpcnvtx and no longer ships libnvhpcwrapnvtx. A --gpu acc or --gpu mp build with 26.9 therefore stops at its first link:

/usr/bin/ld: cannot find -lnvhpcwrapnvtx: No such file or directory
gmake[3]: *** [CMakeFiles/syscheck.dir/build.make:100: syscheck] Error 2
Error: Failed to build the syscheck target.

The CI lanes go up to NVHPC 26.3, so none of them hit this.

The library does not need to be named at all. The driver flag -cudalib=nvtx3 already adds the wrapper under whichever name the installed release uses:

nvfortran nvfortran -cudalib=nvtx3 -dryrun link line
25.9, 26.3 ... -lnvhpcwrapnvtx ...
26.9 ... -lnvhpcnvtx ...

This PR removes the explicit nvhpcwrapnvtx and keeps -cudalib=nvtx3. The CUDA ≤ 12.8 branch (CUDA::nvToolsExt) is unchanged.

The second commit adds 'cmake/**' to the cmakelist filter in .github/file-filter.yml:

Verification

On 2× H100 NVL (master 3ef5e1e):

master this PR
NVHPC 26.9, --gpu acc link fails (above) all targets build
NVHPC 25.9, --gpu acc builds builds
NVTX ranges in nsys (./mfc.sh run ... --nsys, 2D_riemann_test, 50 steps) — present on 25.9 and 26.9: TIMESTEP ×50, COMPUTE-RHS ×150, RHS-RIEMANN-SOLVER ×300, RHS-RECONSTRUCTION ×300, ...
Full --gpu acc test suite, NVHPC 26.9 cannot build 675 passed, 47 failed. All 47 are hypoelasticity tests (see below)

./mfc.sh precheck passes.

The 47 hypoelasticity failures on NVHPC 26.9 are unrelated to this PR:

  • Unpatched master, built on 26.9 with a compat symlink so it links, fails the same way. For example, 34F3999B gives abs: 1.56E-02, rel: 1.56E-05 on both.
  • A CPU-only 26.9 build, which has no NVTX at all, fails 41 of the first 51 hypoelasticity tests.
  • --reldebug (-O1) and --debug (-O0 -Mbounds -C) builds pass, with no bounds errors. Dropping -Mfprelaxed does not help.

They pass on NVHPC 25.9, and we will report them separately.

Re-checked on master 961cde5, RTX 4060 Ti (sm_89):

master this PR
NVHPC 26.9, --gpu acc ld: cannot find -lnvhpcwrapnvtx pre/sim/post build
./mfc.sh test --gpu acc -% 3, NVHPC 26.9 — 19 passed, 2 failed. Both failures are hypoelasticity tests, the known 26.9 issue above
nsys NVTX summary, 64² 2D case, 20 steps — TIMESTEP ×20, COMPUTE-RHS ×60, RHS-RIEMANN-SOLVER ×120, RHS-RECONSTRUCTION ×120
NVHPC 26.3 (newest CI lane), --gpu acc, clean build dir builds syscheck, pre_process build; ldd shows libnvhpcwrapnvtx.so, as before

Side effects

  • CUDA ≤ 12.8 (CUDA::nvToolsExt): unchanged.
  • NVHPC ≤ 26.3: -cudalib=nvtx3 links the same libnvhpcwrapnvtx that was named explicitly before (checked with -dryrun on 25.9 and 26.3, and with ldd on a 26.3 build), so the link line is equivalent.
  • The file-filter.yml line makes CI run the build matrix for any cmake/ change, which is more CI time for such PRs.

Contribution Policy

All contributions are expected to demonstrate:

  • A clear understanding of the codebase
  • Alignment with product direction
  • Thoughtful reasoning behind changes
  • Evidence of real-world usage or hands-on experience with the problem

If these expectations are not met, we would prefer to implement the changes ourselves rather than spend time reviewing low-effort submissions.

Acknowledgement

  • I confirm this PR meets the above expectations and reflects my own understanding and real-world context.

This PR is from Your Team Name participating in the SC26 Student Cluster Competition.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WPhkY8p1EHhGY7jScizJmC

ruru25254649 and others added 2 commits October 25, 2026 04:01
For CUDA >= 12.9 the targets linked nvhpcwrapnvtx by name in addition
to -cudalib=nvtx3. NVHPC 26.9 renamed that library to libnvhpcnvtx, so
the build stops at its first link (syscheck):

    /usr/bin/ld: cannot find -lnvhpcwrapnvtx

-cudalib=nvtx3 already adds the wrapper under the right name for the
installed release (-lnvhpcwrapnvtx on 26.3, -lnvhpcnvtx on 26.9, per
nvfortran -dryrun), so the explicit name is dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WPhkY8p1EHhGY7jScizJmC
file-filter.yml lists CMakeLists.txt and toolchain/cmake/** but not the
cmake/ directory added in MFlowCode#1552, so a PR that only changes cmake/*.cmake
skips every build. Same line as in MFlowCode#2019.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WPhkY8p1EHhGY7jScizJmC
@ruru25254649
ruru25254649 requested a review from a team as a code owner October 25, 2026 07:05
@ruru25254649
ruru25254649 requested review from sbryngelson and removed request for a team October 25, 2026 07:05
@github-actions

Copy link
Copy Markdown

Claude Code Review

Head SHA: aaaf38a

Files changed:

  • 2
  • .github/file-filter.yml
  • cmake/MFCTargets.cmake

Findings:

  • medium cmake/MFCTargets.cmake: The CUDA >= 12.9 branch no longer links nvhpcwrapnvtx. It now relies on -cudalib=nvtx3 to pull in NVHPC's NVTX wrapper library. The new comment says this, but the diff doesn't show it holds for NVHPC releases before 26.9 (where the library is libnvhpcwrapnvtx). If -cudalib=nvtx3 does not link the wrapper there, profiling builds with CUDA 12.9+ and older NVHPC will fail with undefined NVTX symbols. Confidence is moderate, since I can't check the NVHPC behavior from the diff. Please confirm on an NVHPC < 26.9 build with CUDA 12.9+ and --gpu profiling. If it fails, keep the old link conditional on the library existing, for example if (TARGET nvhpcwrapnvtx) or find_library.

@ruru25254649
ruru25254649 marked this pull request as draft October 25, 2026 07:18
Checked with nvfortran -cudalib=nvtx3 -dryrun on 25.9, 26.3 and 26.9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WPhkY8p1EHhGY7jScizJmC
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant