Skip to content

compat.zlib-ng 2.3.3: zlib's API with NEON and SSE2 paths chosen at compile time - #727

Merged
Sunrisepeak merged 1 commit into
mainfrom
compat-zlib-ng-2.3.3
Nov 7, 2026
Merged

Sunrisepeak merged 1 commit into
mainfrom
compat-zlib-ng-2.3.3

Conversation

@Sunrisepeak

Copy link
Copy Markdown
Member

compat.zlib-ng 2.3.3

zlib-ng in its zlib-compatible form: zlib's API (zlib.h, compress2, uncompress, inflate, gz files) and zlib's streams, with the inner loops of inflate, adler32 and the deflate match search written for the processor's vector unit.

Why

mcppls-clangd's ok image reads its index shards through zlib. On darwin-arm64 the image's warm start against a native clangd linked to macOS's NEON libz measured, per pair, the following processor-time ratios (limit 1.15):

image built with pair 1 pair 2 pair 3 pair 4 run
compat.zlib 1.3.2 1.332 1.236 1.214 1.252 42369457883
no zlib 0.973 1.050 1.107 1.088 42386660489

The two sets don't overlap, and the gap is far wider than the 0.03 spread. That makes the decompression the cause (mcpp-community/mcpp design 2026-11-08, W5).

Shape

  • Compile-time paths. This is CMake's WITH_RUNTIME_CPU_DETECTION=OFF build: each function binds at compile time to the best path the architecture's baseline guarantees. That is NEON on aarch64 (inflate_fast, adler32, chunkmemset, slide_hash, compare256, longest_match) and SSE2 on x86_64. An ok image needs this form because openkal names no processor feature. Paths beyond the baseline (Armv8 CRC32, SSSE3, AVX2, PCLMULQDQ) need a program-wide flag or run-time detection and are not built. The CRC32 instructions matter only to gzip streams; a zlib stream, which LLVM writes, is checked with adler32.
  • Gating. The architecture sources gate themselves (#ifdef ARM_NEON, #ifdef X86_SSE2), and a generated configuration header sets the gate from the compiler's own architecture macro. arch/x86/slide_hash_sse2.c has no gate and is compiled through a generated forwarder.
  • Generated headers. install() derives zlib.h, zconf.h, zlib_name_mangling.h and gzread_mangle.h from the tarball's templates. zconf.h's HAVE_UNISTD_H line is written as #if !defined(_WIN32), so the library and every consumer compute z_off_t from one text.
  • CN mirror. mcpp-res/zlib-ng 2.3.3 is published and byte-identical to GLOBAL (sha256 f9c65aa9…07d1).

Tests

tests/examples/zlib-ng checks:

  • a stream that madler's zlib 1.3.1 wrote (as a native engine linked to its system's libz writes a shard) decompresses to its text;
  • adler32 and crc32 give their published values;
  • a MiB of mixed bytes round-trips at levels 1, 6 and 9;
  • gz files are written, read, seeked and read with gzgetc;
  • the library and the consumer agree on z_off_t.

The member is added to the openkal measurement list.

Local runs:

  • x86_64-linux-gnu (GCC 16): pass.
  • x86_64-openkal: pass.
  • aarch64-openkal: built; inflate calls inflate_fast_neon.
  • x86_64: inflate calls inflate_fast_sse2.

…ompile time

The ok image of mcppls-clangd reads its index shards through zlib; against
macOS's own libz its warm start took 1.214-1.332 times the processor time with
compat.zlib and 0.973-1.107 with no zlib (mcpp design 2026-11-08, W5).

Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com>
@Sunrisepeak
Sunrisepeak merged commit 648b727 into main Nov 7, 2026
18 checks passed
@Sunrisepeak
Sunrisepeak deleted the compat-zlib-ng-2.3.3 branch November 7, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant