Skip to content

Claim the bundle's vendor libraries, and drop the rocSPARSE version probe - #1063

Merged
simeonschaub merged 2 commits into
sds/rocm_libsfrom
lr/rs-probe
Sep 3, 2026
Merged

Claim the bundle's vendor libraries, and drop the rocSPARSE version probe#1063
simeonschaub merged 2 commits into
sds/rocm_libsfrom
lr/rs-probe

Conversation

@luraess

@luraess luraess commented Sep 3, 2026

Copy link
Copy Markdown
Member

EDIT: rolled-back most of #1001

In artifact mode on a host with another ROCm on LD_LIBRARY_PATH (here a CSCS uenv), versioninfo() reported rocSPARSE as err and the core test asserting the probe returns a version failed. The cause is not rocSPARSE.

PRELOAD_LIBRARIES claimed the runtime closure but no vendor library, and AMDGPU claims those lazily on the first ccall, so whichever is touched first wins. TheRock's libraries carry DT_RUNPATH, which LD_LIBRARY_PATH outranks, so touching rocSPARSE first pulled its dependencies from the environment. Measured in artifact mode after rocSPARSE.version():

rocsparse -> <artifact>/lib/librocsparse.so
rocblas   -> /user-environment/env/default/lib/librocblas.so.5
roctx     -> /user-environment/env/default/lib/libroctx64.so.4
foreign_libraries() = String[]

FOREIGN_LIBRARY_NAMES listed only runtime libraries, so versioninfo() printed "Foreign ROCm libraries loaded: none" while two rocBLAS versions were live in the process.

The isolated probe from #1051 hit the same thing: a bare child has claimed nothing at all, so it resolved the bundle's librocsparse against uenv HIP 7.2.3 and rocsparse_create_handle segfaulted, which is what surfaced as err.

Changes

  • PRELOAD_LIBRARIES also claims libamdhip64, libroctx64, libhipblaslt and librocblas — the vendor libraries that other vendor libraries pull in (librocsparse needs librocblas and libroctx64; librocblas needs libhipblaslt). Order is load-bearing: HSA must precede HIP, or HIP's libhsa-runtime64.so.1 binds outside the bundle and the dlopen fails with undefined symbol: hsa_amd_queue_create, version ROCR_1.
  • FOREIGN_LIBRARY_NAMES gains the vendor libraries, so a mixed process is reported instead of passing as clean.
  • The out-of-process version probe is removed (_version_subprocess, _rocsparse_version_isolated, the cache global and its __init__ reset). With the libraries claimed correctly there is nothing left for it to protect against, and it was a lot of machinery for one cell of a table.
  • versioninfo queries rocSPARSE in-process like every other library. The shared _ver helper now guards all of them, so a failing query shows err instead of taking down the whole report — previously none were guarded.

Result

On beverin (2x MI300A, artifact ROCm under uenv), before -> after:

before after
versioninfo rocSPARSE err + misleading warning 5.0.0
librocblas / libroctx64 / libhipblaslt uenv bundle
foreign_libraries() String[] (wrong) String[] (correct)
preloaded libraries 41 45

Trade-off

A SIGSEGV inside rocsparse_create_handle (#920) again takes down the session rather than degrading to err. Two things make that acceptable: the preload fix removes the mixed-library cause that produced the crash here, and versioninfo already initialises HIP and enumerates devices in-process, so a genuinely broken install fails before the rocSPARSE row either way. try/catch still covers every non-fatal failure.

@simeonschaub

Copy link
Copy Markdown
Member

I'm really not a big fan of introducing even more complexity just for the librocsparse version check. With #1028, we might want to revisit whether the check is truly needed anymore, since now the user would have to explicitly request AMDGPU.jl to use a broken system ROCm install for this to matter. I personally think it's not too unexpected that Julia would crash in such a situation, so I'd propose reverting #1001 altogether as part of #1028

@luraess

luraess commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

Yeah, that's the other alternative, to remove all this machinery with the one caveat is that we may get inaccurate version for rocSPARSE if we ever encounter using it from system. I can do it here.

@luraess luraess changed the title Probe rocSPARSE against the provider's libraries, not the environment's Claim the bundle's vendor libraries, and drop the rocSPARSE version probe Sep 3, 2026
@luraess

luraess commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

This is now mostly rolling back #1001 but adding some guards.

@simeonschaub
simeonschaub merged commit f0d54f9 into sds/rocm_libs Sep 3, 2026
1 check was pending
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants