Skip to content

Reduce compiler wheels by 35-48% - #135

Merged
jtuyls merged 1 commit into
mainfrom
users/jtuyls/reduce-compiler-wheel-size
Oct 19, 2026
Merged

jtuyls merged 1 commit into
mainfrom
users/jtuyls/reduce-compiler-wheel-size

Conversation

@jtuyls

@jtuyls jtuyls commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator

Reduce the packaged compiler payload without changing its public tool interfaces.

  • Compress compiler wheels with DEFLATE level 9.
  • Replace byte-identical Windows Clang, LLD, and readelf multicall copies with deterministic native launchers that preserve each driver flavor.
  • Replace the large Linux LLD and readelf copies with flavor-preserving shell launchers while retaining the inexpensive Clang aliases.
  • Remove the byte-identical Linux libLLVM-21.so copy after validation and strip unneeded ELF symbols.
  • Exercise every forwarded mode in the existing wheel verification jobs and retain native dependency, installed-wheel, E2E, and JIT checks.

Measured compiler-core wheel sizes:

  • Windows: 632,790,558 -> 330,049,707 bytes (47.8% smaller).
  • Linux: 469,932,885 -> 303,792,202 bytes (35.4% smaller).

@jtuyls
jtuyls requested a balanced review from Copilot October 19, 2026 09:00
@jtuyls
jtuyls force-pushed the users/jtuyls/reduce-compiler-wheel-size branch from f892730 to 1beb936 Compare October 19, 2026 09:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Launchers regress direct Windows Clang frontend calls and Linux alias invocation through external symlinks.

2 open issues
What changed in this PR

Reduces compiler wheel size through stronger compression and payload deduplication while aiming to preserve existing tool interfaces.

Changes:

  • Uses DEFLATE level 9 and replaces duplicate tool binaries with launchers.
  • Removes redundant Linux LLVM payload and strips unneeded symbols.
  • Expands alias verification and documents packaging behavior.
File Description
tests/​wheels/​test_fastflow_sdk_compiler_wheels.py Tests alias forwarding, validation, and stripping.
docs/​wheel-releases.md Documents size-reduction and verification behavior.
build_tools/​native/​compiler_tool_alias_launcher.cpp Adds native Windows alias forwarding.
build_tools/​fastflow_sdk_compiler_wheels.py Implements compression, payload reduction, and alias audits.

Review effort: Balanced

Comment thread build_tools/fastflow_sdk_compiler_wheels.py Outdated
Comment thread build_tools/native/compiler_tool_alias_launcher.cpp Outdated
Reduce the packaged compiler payload without changing its public tool interfaces.

- Compress compiler wheels with DEFLATE level 9.
- Replace byte-identical Windows Clang, LLD, and readelf multicall copies with deterministic native launchers that preserve each driver flavor.
- Replace the large Linux LLD and readelf copies with flavor-preserving shell launchers while retaining the inexpensive Clang aliases.
- Remove the byte-identical Linux libLLVM-21.so copy after validation and strip unneeded ELF symbols.
- Exercise every forwarded mode in the existing wheel verification jobs and retain native dependency, installed-wheel, E2E, and JIT checks.

Measured compiler-core wheel sizes:
- Windows: 632,790,558 -> 330,061,997 bytes (47.8% smaller).
- Linux: 469,932,885 -> 303,792,202 bytes (35.4% smaller).
@jtuyls
jtuyls force-pushed the users/jtuyls/reduce-compiler-wheel-size branch from 1beb936 to a1a018c Compare October 19, 2026 09:19
@jtuyls
jtuyls merged commit 267d834 into main Oct 19, 2026
20 checks passed
@jtuyls
jtuyls deleted the users/jtuyls/reduce-compiler-wheel-size branch October 19, 2026 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants