runai-model-streamer-s3: Add version 0.16.1 - #2355
Merged
Merged
Conversation
Build the S3 backend plugin (libstreamers3.so) of Run:ai's Model Streamer for riscv64. It statically links the AWS SDK for C++ (s3-crt), OpenSSL, libcurl and zlib; those are built natively in the manylinux image at the versions upstream's devcontainer pins, cached between runs.
luhenry
added a commit
that referenced
this pull request
Sep 26, 2026
Contributor
|
The manylinux image's gcc does not default to PIE the way upstream's Ubuntu devcontainer gcc does, so the static libz.a has absolute R_RISCV_HI20 relocations that cannot be linked into libstreamers3.so. Key the dependency cache on the workflow file so a recipe change rebuilds it.
…ests The 0.16.1 tag's files/tests/test_files.py calls _filter_ignore() with allow_pattern=/ignore_pattern= keywords that neither helper has ever accepted (both take (paths, patterns)), so three tests raise TypeError on any architecture, including against upstream's own x86_64 PyPI wheel. Upstream's CI never runs the S3 package's tests (py/Makefile's test target only covers runai_model_streamer); dsx-ai-factory/model-streamer@c6a6543 fixed the tests on master after the release.
…does python -m pytest prepends the cwd (tests/) to sys.path, where upstream's tests/safetensors/ helper package shadows the installed safetensors, so collecting s3/test_s3.py failed on `from safetensors.torch import safe_open`. tests/Makefile's s3 target runs the pytest script, whose prepend-mode rootdir insertion adds only the repository root (gotcha 218).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
runai-model-streamer-s30.16.1Compiles the Model Streamer's S3 backend (
libstreamers3.so, C++ on the AWS SDK for C++ s3-crt) with Bazel. Upstream publishes no riscv64 wheel.Mirrors upstream's
Makefileand.devcontainer/Dockerfile, same shape asbuild-runai-model-streamer.yml.Differs from upstream
libcurl.amatches the link lineTesting
License: Wheel bundles OpenSSL (Apache-2.0), libcurl (curl), zlib (Zlib) and the AWS SDK/CRT libraries (Apache-2.0); upstream ships no licence text for them, so the build adds it.
Patches
0001/0002- To upstream. Same riscv64 toolchain patches as runai-model-streamer. riscv64-only.0003-third_party-add-riscv64-to-the-AWS-SDK-library-select.patch- To upstream.@awsselect has no riscv64 branch. riscv64-only.0004-s3-ship-the-licences-of-the-statically-linked-dependencies.patch- To upstream. Explicitlicense_filesdrops the bundled licences. All arches.