Support sycl backend for Intel GPU - #18
Open
khanhnd61-vr wants to merge 23 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an Intel GPU backend through oneAPI SYCL, and collapses the per-arch
backend selection that made adding one painful.
src/backend.his the new shared ladder. Every arch used to open-code the sameaccelerator-then-CPU sequence, so a new backend meant editing a dozen files;
they now call
vla::backend_initand the ladder lives in one place. Net -225lines across
src/models/. Exactly one accelerator is compiled in, picked bythe CMake flag (
GGML_CUDA/GGML_SYCL/GGML_METAL), and configure nowfails if two are on rather than handing ggml both backends. BitVLA keeps its own
CPU init on purpose: it pins its ggml graph to the CPU and offloads through its
own ternary CUDA kernels.
VLA_DEVICEselects the ordinal on both multi-device backends. The SYCL pathrange-checks it, because
ggml_backend_sycl_initguards the index with anassertthat a Release build compiles out.Results
Intel Arc A380 (6 GB, Xe-HPG) against the CPU backend on an AMD Ryzen 5 5500,
8 threads. Best of 5-10 iterations after 3 warmups, fixed noise.
Checked against the CPU backend on every model: max absolute deviation 2.9e-3 on
actions peaking at 0.99, RMS 2.4e-4. That is BF16/F32 kernel rounding, not a
numerical regression.
Two SYCL issues worked around
GGML_SYCL_ENABLE_VMM=0is defaulted at backend init. ggml-sycl's VMM poolhands out pointers oneDNN cannot wrap in a
dnnl::memory, which aborts the GEMMfor any non-F32
src0-- that is most checkpoints. It is only a default;setting it explicitly wins. Disabling oneDNN also clears the crash but costs
~8%.
ggml-sycl has no
bf16 -> f32copy, so VLA-Adapter needsVLA_ADAPTER_F32_WEIGHTS=1. SmolVLA and Evo-1 run on their stock BF16 defaults.Setup, the compute-runtime prerequisites and the memory ceiling are in
docs/backend/sycl.md.