Skip to content

Support sycl backend for Intel GPU - #18

Open
khanhnd61-vr wants to merge 23 commits into
mainfrom
sycl
Open

Support sycl backend for Intel GPU#18
khanhnd61-vr wants to merge 23 commits into
mainfrom
sycl

Conversation

@khanhnd61-vr

@khanhnd61-vr khanhnd61-vr commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Adds an Intel GPU backend through oneAPI SYCL, and collapses the per-arch
backend selection that made adding one painful.

src/backend.h is the new shared ladder. Every arch used to open-code the same
accelerator-then-CPU sequence, so a new backend meant editing a dozen files;
they now call vla::backend_init and the ladder lives in one place. Net -225
lines across src/models/. Exactly one accelerator is compiled in, picked by
the CMake flag (GGML_CUDA / GGML_SYCL / GGML_METAL), and configure now
fails if two are on rather than handing ggml both backends. BitVLA keeps its own
CPU init on purpose: it pins its ggml graph to the CPU and offloads through its
own ternary CUDA kernels.

VLA_DEVICE selects the ordinal on both multi-device backends. The SYCL path
range-checks it, because ggml_backend_sycl_init guards the index with an
assert that a Release build compiles out.

Results

Intel Arc A380 (6 GB, Xe-HPG) against the CPU backend on an AMD Ryzen 5 5500,
8 threads. Best of 5-10 iterations after 3 warmups, fixed noise.

Model input CPU Arc A380 speedup
SmolVLA 512 1,920 ms 630 ms 3.0x
Evo-1 448 7,695 ms 1,176 ms 6.5x
VLA-Adapter 224 2,994 ms 517 ms 5.8x

Checked against the CPU backend on every model: max absolute deviation 2.9e-3 on
actions peaking at 0.99, RMS 2.4e-4. That is BF16/F32 kernel rounding, not a
numerical regression.

Two SYCL issues worked around

GGML_SYCL_ENABLE_VMM=0 is defaulted at backend init. ggml-sycl's VMM pool
hands out pointers oneDNN cannot wrap in a dnnl::memory, which aborts the GEMM
for any non-F32 src0 -- that is most checkpoints. It is only a default;
setting it explicitly wins. Disabling oneDNN also clears the crash but costs
~8%.

ggml-sycl has no bf16 -> f32 copy, so VLA-Adapter needs
VLA_ADAPTER_F32_WEIGHTS=1. SmolVLA and Evo-1 run on their stock BF16 defaults.

Setup, the compute-runtime prerequisites and the memory ceiling are in
docs/backend/sycl.md.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants