[AMDGPU][SYCL] Run global offset + attributor before AOT codegen#22675
Open
zjin-lcf wants to merge 1 commit into
Open
[AMDGPU][SYCL] Run global offset + attributor before AOT codegen#22675zjin-lcf wants to merge 1 commit into
zjin-lcf wants to merge 1 commit into
Conversation
DPC++ HIP nd_range kernel launches fail on gfx1200 with hipErrorIllegalState because the emitted kernels carry the conservative hidden AMDGPU kernel arguments (hostcall buffer, default queue, completion action, heap). HIP tolerates these on gfx942 but rejects them on gfx1200. For AMDGPU, SYCL AOT emits the device image with `clang -cc1 -flto=full -S`: the LTO prelink optimization pipeline immediately followed by codegen, with no post-link step. AMDGPUAttributor was gated to non -prelink phases, so it never ran before codegen in this flow, leaving every kernel with the default hidden arguments. In addition, the SYCL GlobalOffsetPass only ran in the legacy codegen pipeline (too late), so even when the attributor did run it saw an unresolved `__spirv_BuiltInGlobalOffset` callee and could not prove the hidden arguments away. Run GlobalOffsetPass (self-gated to SYCL device modules) followed by AMDGPUAttributor at OptimizerLast in all non-O0 phases, including LTO prelink, and also resolve global offset before the post-link attributor. This drops the kernarg segment from 288/304 to 32/44 bytes and removes the hidden arguments, matching the verified workaround (--lto-newpm-passes=globaloffset,lto). Validated end-to-end on gfx942. See intel#22606. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes DPC++ HIP
nd_rangekernel launch failures on gfx1200 (hipErrorIllegalState/ error 401) reported in #22606.For AMDGPU, SYCL AOT emits the device image with
clang -cc1 -flto=full -S: the LTO prelink optimization pipeline immediately followed by codegen, with no post-link step.AMDGPUAttributorwas gated to non-prelink phases, so in this flow it never ran before codegen and every kernel kept the default conservative hidden kernel arguments (hostcall buffer, default queue, completion action, heap). Additionally, the SYCLGlobalOffsetPassonly ran in the legacy codegen pipeline (too late), so even when the attributor did run it saw an unresolved__spirv_BuiltInGlobalOffsetcallee and could not prove the hidden arguments away.HIP tolerates these spurious hidden arguments on gfx942 but rejects them at launch on gfx1200.
Fix
In
AMDGPUTargetMachine::registerPassBuilderCallbacks:GlobalOffsetPass(self-gated to SYCL device modules) followed byAMDGPUAttributorPassatOptimizerLastfor all non-O0 phases, including LTO prelink.This drops the kernel argument segment from 288/304 bytes to 32/44 bytes and removes the hidden arguments — the same result as the verified workaround
-Xoffload-linker=amdgcn-amd-amdhsa --lto-newpm-passes=globaloffset,lto, but without any user-facing flags.Test plan
hidden_hostcall_buffer/hidden_heap_v1/hidden_default_queue/hidden_completion_action.nd_rangerepro prints42.opt -passes='lto-pre-link<O3>'+llcon the real device module that the attributor now runs and cleans the kernels.Refs #22606