[metascroy]

Stack

Off-graph KV cache for the native ggml engine, through serving Muse Glimmer on it. Review bottom-up; each PR builds and passes its tests on its own.

• [Native] Launch the Muse Glimmer server on the native backend #24957
• [Native] Serve Muse Glimmer on the native ggml engine #24956
• [Native] Add a batching executor for native-backend packages #25055
• [Native] Select the logits rows an off-graph export returns #25009
• [Native] Gather the rows of x[:, rows] in the ggml engine #25008
• [Native] Serve many sequences from one ggml engine KV cache #25007
• [Native] Separate the ggml engine's K/V storage from its sequence cache #25006
• [Native] Publish the context length with off-graph packages #24955
• [Native] Share the off-graph cache geometry with other callers #24954
• [Native] Export Muse Glimmer through the native backend #24951
• [Native] Replay sample sets without decoding their tokens #24950 (this PR)
• [Native] Map Muse Glimmer GGUF tensors onto its HF model #24949
• [Native] Run GGUF Q5_K weights in the ggml engine #24948
• [Native] Load the first layers of a GGUF with --layers #24947
• [Native] Run the HF LLM export on newer transformers #24946
• [Native] Fuse broadcast RoPE frequencies and lower tanh in ggml #24945
• [Native] Support sliding-window attention with the off-graph KV cache #24944
• [Native] Carry named metadata in native packages #24943
• [Native] Add chunked prefill and interactive chat to the LLM runner #24639
• [Native] Run off-graph KV cache packages in the LLM runner #24638
• [Native] Serve kvcache::update_and_attend in the ggml engine #24637
• [Native] Add load-time options to engine contexts #24636
• [Native] Export HF LLMs with an off-graph KV cache #24635
• Move off-graph HF attention to extension/llm/cache #24634

Summary

--verify_samples streamed each generated token through --tokenizer, which only decodes Llama 3 vocabularies and threw on any other model's token IDs, though replay compares logits and never needs text. generate() now takes the tokenizer as optional and streams text only when given one; sample replay passes none.

Test plan

• Muse Glimmer packages, whose token IDs exceed Llama 3's vocabulary, replay their golden samples.
• Existing packages still verify, and normal generation still streams text.

---

Authored with Claude Code.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24950

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 137 New Failures, 7 Unrelated Failures, 10 Unclassified Failures

As of commit b5c3fb1 with merge base c7f7931 ():

NEW FAILURES - The following jobs have failed:

• Build documentation / build (buck2) / Build doc (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Build Presets / apple (ios-simulator) / build (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• Build Presets / apple (ios) / build (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• Build Presets / apple (llm) / build (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• Build Presets / apple (macos) / build (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• Build Presets / apple (profiling) / build (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• Build Presets / apple (pybind) / build (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• Build Presets / linux (linux, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc11-aarch64) / build (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Build Presets / linux (linux, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Build Presets / linux (llm, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc11-aarch64) / build (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Build Presets / linux (llm, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Build Presets / linux (pybind, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc11-aarch64) / build (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Build Presets / linux (pybind, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clang12) / build (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Build Presets / windows (pybind) / build (gh)
• Build Presets / windows (windows) / build (gh)
• Build Presets / zephyr (zephyr) / build (gh)
• Cadence Build & Test / cpu-build / build (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Lint / lintrunner (gh)
  >>> Lint for backends/native/examples/hf_llm/runtime/LlmRunner.cpp:
• pull / nxp-mcuxpresso-test / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-arduino-library / run / arduino-library (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-arm-backend-no-driver (test_pytest_models_tosa) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-arm-backend-no-driver (test_pytest_ops_no_target) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-arm-backend-no-driver (test_pytest_ops_tosa) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-arm-backend-no-driver (test_run_tosa) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-arm-backend-public-api-backward-compatibility / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-arm-backend-vkml (test_ootb_tests_vgf) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-arm-backend-vkml (test_pytest_ops_vkml) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-arm-backend-vkml (test_runtime_vgf) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-arm-cortex-m-size-test (zephyr-preset) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-binary-size-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-binary-size-linux-gcc / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-build-wasm-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-custom-ops-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-llama_runner_eager-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-llama-runner-linux (bf16, custom, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-c... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, mt-l-arm64g4-16-62, executorch-ubuntu-... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, mt-l-x86iavx512-8-64, executorch-ubunt... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-g... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-llama-runner-linux-android / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-lora-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-lora-multimethod-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-mcu-cortex-m-backend / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (add_mul, portable, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (add_mul, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (add, portable, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (add, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (emformer_join, portable, mt-l-x86iavx512-16-128) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (emformer_join, xnnpack-quantization-delegation, mt-l-x86iavx512-16-128) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (emformer_transcribe, portable, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (ic3, portable, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (ic3, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (ic4, portable, mt-l-x86iavx512-16-128) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, mt-l-x86iavx512-16-128) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (linear, portable, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (linear, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (llama3_2_vision_encoder, portable, mt-l-x86iavx512-16-128) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (mobilebert, portable, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (mobilebert, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (mv2, portable, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (mv2, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (phi_4_mini, portable, mt-l-x86iavx512-16-128) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (resnet18, portable, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (resnet18, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (resnet50, portable, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (resnet50, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux (w2l, portable, mt-l-x86iavx512-16-128) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, buck2, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, cmake, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc11-... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux-basic (mv3, portable, cmake, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, mt-l-x86iavx512-8-64, execu... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, mt-l-arm64g4-16-62, executo... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, mt-l-x86iavx512-8-64, execu... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux-basic (vit, portable, buck2, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux-basic (vit, portable, cmake, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc11-... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux-basic (vit, portable, cmake, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, mt-l-x86iavx512-8-64, execu... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, mt-l-arm64g4-16-62, executo... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, mt-l-x86iavx512-8-64, execu... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-moshi-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-multimodal-linux (gemma3-4b) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-openvino-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-parakeet-xnnpack-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-phi-3-mini-runner-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-pybindings-no-aten / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-qnn-buck-build-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-qnn-delegate-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-qnn-direct-build-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-qnn-models-linux (dl3) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-qnn-models-linux (mv2) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-qnn-models-linux (mv3) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-qnn-passes-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-qnn-python-imports-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-qnn-testsuite-linux / test-backend-linux (qnn, models) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-qnn-testsuite-linux / test-backend-linux (qnn, operators) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-qnn-wheel-packages-linux (3.11) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-qnn-wheel-packages-linux (3.12) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-qnn-wheel-packages-linux (3.13) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-quantized-aot-lib-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-selective-build-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-setup-linux-gcc / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-sqnr-static-llm-qnn-linux (smollm2_135m) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-static-llama-qnn-linux (stories_110m) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-static-llama-qnn-linux (stories_260k_bc) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-voxtral-realtime-xnnpack-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-vulkan-models-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / test-vulkan-operators-linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / unittest / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest-buck / linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / unittest-buck / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / unittest-editable / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest-nxp-neutron / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / unittest-wasm-bindings (--enable-etdump) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test ARM Backend / test-arm / test-backend-linux (arm_tosa_fp, models) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test ARM Backend / test-arm / test-backend-linux (arm_tosa_fp, operators) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test ARM Backend / test-arm-vgf / test-backend-linux (arm_vgf_fp, models) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test ARM Backend / test-arm-vgf / test-backend-linux (arm_vgf_fp, operators) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test CPU Backend / test-cpu / test-backend-linux (cpu_fp32, models) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test CPU Backend / test-cpu / test-backend-linux (cpu_fp32, operators) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test QNN Backend / test-qnn / test-backend-linux (qnn, models) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test QNN Backend / test-qnn / test-backend-linux (qnn, operators) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test QNN Windows MSVC build / build-qnn-windows-x64 / windows-job (gh)
• Test Vulkan Backend / test-vulkan / test-backend-linux (vulkan, models) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test Vulkan Backend / test-vulkan / test-backend-linux (vulkan, operators) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test WebGPU Backend / test-webgpu / test-backend-linux (webgpu, models) / linux-job (gh)
  curl: (22) The requested URL returned error:
• Test WebGPU Backend / test-webgpu / test-backend-linux (webgpu, operators) / linux-job (gh)
  curl: (22) The requested URL returned error:
• Test XNNPACK Backend / test-xnnpack / test-backend-linux (xnnpack, models) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test XNNPACK Backend / test-xnnpack / test-backend-linux (xnnpack, operators) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

• Build Aarch64 Linux Native Wheels / executorch-native / build-wheel-py3_11-cpu-aarch64 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ModuleNotFoundError: No module named 'setuptools'
• Build Aarch64 Linux Native Wheels / executorch-native / upload / upload-wheel-py3_11-cpu-aarch64 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  Unable to download artifact(s): Artifact not found
• Build Linux Native Wheels / executorch-native / build-manywheel-py3_11-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ModuleNotFoundError: No module named 'setuptools'
• Build Linux Native Wheels / executorch-native / upload / upload-manywheel-py3_11-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  Unable to download artifact(s): Artifact not found
• periodic / test-models-linux (buck2, mv3, portable, mt-l-x86iavx512-8-64, 90) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• periodic / test-models-linux (buck2, mv3, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64, 90) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• periodic / test-models-linux (cmake, mv3, portable, mt-l-x86iavx512-8-64, 90) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• periodic / test-models-linux (cmake, mv3, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64, 90) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• periodic / test-models-linux (cmake, vit, portable, mt-l-x86iavx512-8-64, 90) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• periodic / test-models-linux (cmake, vit, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64, 90) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Cadence Build & Test / hifi-build / hifi4 (gh) (detected as infra flaky with no log or failing log classifier)
• Cadence Build & Test / hifi1s-build / hifi1s (gh) (detected as infra flaky with no log or failing log classifier)
• Cadence Build & Test / hifi5s-build / hifi5s (gh) (detected as infra flaky with no log or failing log classifier)
• Cadence Build & Test / vision-build / vision (gh) (detected as infra flaky with no log or failing log classifier)
• pull / unittest / windows / windows-job (gh) (detected as infra flaky with no log or failing log classifier)
• pull / unittest-editable / windows / windows-job (gh) (detected as infra flaky with no log or failing log classifier)
• Test Native Engines / native-ggml (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.