Skip to content

[AMDGPU][SYCL] Run global offset + attributor before AOT codegen#22675

Open
zjin-lcf wants to merge 1 commit into
intel:syclfrom
zjin-lcf:fix-amdgpu-globaloffset-lto-22606
Open

[AMDGPU][SYCL] Run global offset + attributor before AOT codegen#22675
zjin-lcf wants to merge 1 commit into
intel:syclfrom
zjin-lcf:fix-amdgpu-globaloffset-lto-22606

Conversation

@zjin-lcf

Copy link
Copy Markdown
Contributor

Summary

Fixes DPC++ HIP nd_range kernel launch failures on gfx1200 (hipErrorIllegalState / error 401) reported in #22606.

For AMDGPU, SYCL AOT emits the device image with clang -cc1 -flto=full -S: the LTO prelink optimization pipeline immediately followed by codegen, with no post-link step. AMDGPUAttributor was gated to non-prelink phases, so in this flow it never ran before codegen and every kernel kept the default conservative hidden kernel arguments (hostcall buffer, default queue, completion action, heap). Additionally, the SYCL GlobalOffsetPass only ran in the legacy codegen pipeline (too late), so even when the attributor did run it saw an unresolved __spirv_BuiltInGlobalOffset callee and could not prove the hidden arguments away.

HIP tolerates these spurious hidden arguments on gfx942 but rejects them at launch on gfx1200.

Fix

In AMDGPUTargetMachine::registerPassBuilderCallbacks:

  • Run GlobalOffsetPass (self-gated to SYCL device modules) followed by AMDGPUAttributorPass at OptimizerLast for all non-O0 phases, including LTO prelink.
  • Also resolve global offset before the post-link attributor, for flows that defer codegen to the LTO link.

This drops the kernel argument segment from 288/304 bytes to 32/44 bytes and removes the hidden arguments — the same result as the verified workaround -Xoffload-linker=amdgcn-amd-amdhsa --lto-newpm-passes=globaloffset,lto, but without any user-facing flags.

Test plan

  • Reproduced the bug end-to-end on gfx942 (MI300A): kernarg segment 288/304 with hidden_hostcall_buffer/hidden_heap_v1/hidden_default_queue/hidden_completion_action.
  • With the fix, the same AOT build produces kernarg 32/44 with no hidden arguments, and the nd_range repro prints 42.
  • Confirmed via opt -passes='lto-pre-link<O3>' + llc on the real device module that the attributor now runs and cleans the kernels.
  • CI: some AMDGPU pipeline lit tests may need output updates, since the attributor now also runs in the LTO prelink phase for AMDGPU.
  • Ideally verify on gfx1200 hardware (not available to me).

Refs #22606

DPC++ HIP nd_range kernel launches fail on gfx1200 with
hipErrorIllegalState because the emitted kernels carry the conservative
hidden AMDGPU kernel arguments (hostcall buffer, default queue,
completion action, heap). HIP tolerates these on gfx942 but rejects them
on gfx1200.

For AMDGPU, SYCL AOT emits the device image with `clang -cc1 -flto=full
-S`: the LTO prelink optimization pipeline immediately followed by
codegen, with no post-link step. AMDGPUAttributor was gated to non
-prelink phases, so it never ran before codegen in this flow, leaving
every kernel with the default hidden arguments. In addition, the SYCL
GlobalOffsetPass only ran in the legacy codegen pipeline (too late), so
even when the attributor did run it saw an unresolved
`__spirv_BuiltInGlobalOffset` callee and could not prove the hidden
arguments away.

Run GlobalOffsetPass (self-gated to SYCL device modules) followed by
AMDGPUAttributor at OptimizerLast in all non-O0 phases, including LTO
prelink, and also resolve global offset before the post-link attributor.
This drops the kernarg segment from 288/304 to 32/44 bytes and removes
the hidden arguments, matching the verified workaround
(--lto-newpm-passes=globaloffset,lto). Validated end-to-end on gfx942.

See intel#22606.

Co-authored-by: Cursor <cursoragent@cursor.com>
@zjin-lcf
zjin-lcf requested a review from a team as a code owner July 19, 2026 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant