From b57db32cad4d39b4ae48a1905c78469db1c6b2c3 Mon Sep 17 00:00:00 2001 From: Rick Staa Date: Tue, 18 Aug 2026 12:40:46 +0200 Subject: [PATCH 1/9] docs: state the apache-2.0 payload and example-only status Orchestrators pull this image, so the terms it ships under and the support it does not carry both need to be explicit: the server inside is daydream's StreamDiffusion, Apache-2.0, pinned and unmodified, with its LICENSE at /src/LICENSE and no NOTICE to propagate. Model weights are fetched at runtime under their own terms. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/README.md b/README.md index 387acb0..ca6f54d 100644 --- a/README.md +++ b/README.md @@ -119,6 +119,12 @@ uvx pre-commit run --all-files CI runs the same hooks, checks the compose file parses, and builds the image. +## License and attribution + +This repo is an **example**, not a supported Livepeer artifact. The wrapper here (Dockerfile, [client.py](client.py), the compose files, [runners.json](runners.json)) is MIT, and CI publishing to `ghcr.io/livepeer/` is packaging convenience so an operator can pull it, not a product commitment. + +What runs inside the image is daydream's [StreamDiffusion](https://github.com/daydreamlive/StreamDiffusion), itself a fork of [cumulo-autumn/StreamDiffusion](https://github.com/cumulo-autumn/StreamDiffusion). Both are **Apache-2.0**, and this repo builds the fork pinned at `94b9b96` and **unmodified**, so redistribution is permitted and there are no changes to state under section 4(b). The fork ships no `NOTICE` file; its `LICENSE` travels in the image at `/src/LICENSE`. Model weights are downloaded from Hugging Face on first run under their own licenses and are not redistributed here. + ## Building your own Start from [**template-livepeer-runner**](https://github.com/livepeer/template-livepeer-runner), then list yours in [**runner-app-examples**](https://github.com/livepeer/runner-app-examples#external-examples). That repo also has a minimal example of each transport, mode, registration, and pricing option; the [live runner docs](https://github.com/livepeer/go-livepeer/blob/master/doc/live-runner.md) are the reference. From 5ede421bf53e7ba4f8535b2a05d332cc48ac9463 Mon Sep 17 00:00:00 2001 From: Rick Staa Date: Tue, 18 Aug 2026 13:53:32 +0200 Subject: [PATCH 2/9] refactor: build on python-slim so the image drops the CUDA devel base Nothing in this image compiles against CUDA. torch carries its runtime in the cu128 wheels, and streamdiffusion.tools.install-tensorrt is pure pip: it installs the tensorrt and nvidia-cudnn-cu12 wheels and reads torch.version.cuda, never nvcc. Upstream's devel base is one commit from Dec 2023, when xformers and stable-fast still built native extensions, and this image installs neither. Without a system CUDA tree the loader needs the wheels' lib dirs, so register them with ldconfig and assert the stack loads at build time rather than on the first stream. Co-Authored-By: Claude Opus 5 (1M context) --- Dockerfile | 64 ++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 50 insertions(+), 14 deletions(-) diff --git a/Dockerfile b/Dockerfile index 7c03ca2..6c74774 100644 --- a/Dockerfile +++ b/Dockerfile @@ -5,22 +5,32 @@ # the whole point of the static-registration path — an app needs no SDK, and no # awareness of Livepeer at all, to run on the network. # -# NOTE: the fork's own Dockerfile clones the UPSTREAM cumulo-autumn repo, not the -# fork, so it would not run this rewritten server. We clone the fork at a pinned -# commit instead. -FROM nvidia/cuda:12.8.1-cudnn-devel-ubuntu22.04 +# NOTE: the fork's demo/realtime-img2img/Dockerfile clones the UPSTREAM +# cumulo-autumn repo, not the fork, so it would not run this rewritten server. +# We clone the fork at a pinned commit instead. +# +# Base is plain python-slim, NOT nvidia/cuda:*-devel, because nothing here +# compiles against CUDA: torch carries its own runtime in the cu128 wheels, and +# streamdiffusion.tools.install-tensorrt is pure pip — it installs the tensorrt +# and nvidia-cudnn-cu12 wheels and reads torch.version.cuda, never nvcc. +# Upstream's devel base is a single commit from Dec 2023, back when xformers and +# stable-fast still built native extensions; this image installs neither. The +# driver arrives through the NVIDIA container runtime (compose `gpus: all`), +# which is how the other Livepeer runner examples get a GPU too. +FROM python:3.11-slim + +LABEL org.opencontainers.image.title="streamdiffusion-livepeer-runner" +LABEL org.opencontainers.image.description="daydream's StreamDiffusion realtime-img2img server, packaged unmodified to run as a Livepeer live runner" +LABEL org.opencontainers.image.source="https://github.com/livepeer/streamdiffusion-livepeer-runner" +LABEL org.opencontainers.image.licenses="Apache-2.0" ENV DEBIAN_FRONTEND=noninteractive PYTHONUNBUFFERED=1 ENV HF_HUB_ENABLE_HF_TRANSFER=1 +# git: pip resolves streamdiffusion from the fork, and demo/ is cloned below. +# libgl1/libglib2.0-0 are opencv's, not CUDA's, so slim needs them either way. RUN apt-get update && apt-get install -y --no-install-recommends \ - software-properties-common ca-certificates curl git \ - && add-apt-repository ppa:deadsnakes/ppa \ - && apt-get update && apt-get install -y --no-install-recommends \ - python3.11 python3.11-venv python3.11-dev \ - && ln -sf /usr/bin/python3.11 /usr/local/bin/python \ - && curl -sS https://bootstrap.pypa.io/get-pip.py | python \ - && python -m pip --version \ + git libgl1 libglib2.0-0 \ && apt-get clean && rm -rf /var/lib/apt/lists/* RUN python -m pip install --no-cache-dir \ @@ -28,12 +38,38 @@ RUN python -m pip install --no-cache-dir \ --index-url https://download.pytorch.org/whl/cu128 RUN python -m pip install --no-cache-dir \ "streamdiffusion[tensorrt,controlnet,ipadapter] @ git+https://github.com/daydreamlive/StreamDiffusion.git@94b9b96cb8a17d401ffbce516393d6482326ce62" -RUN apt-get update && apt-get install -y --no-install-recommends \ - libgl1 libglib2.0-0 \ - && apt-get clean && rm -rf /var/lib/apt/lists/* +# Pulls the tensorrt + cuDNN wheels, polygraphy and onnx-graphsurgeon. Pure pip: +# no compiler and no CUDA headers are involved. RUN python -m streamdiffusion.tools.install-tensorrt +# With no system CUDA tree, the dynamic loader has to find cuDNN, cuBLAS and +# TensorRT inside the wheels. Register whatever lib dirs the wheels actually +# installed rather than hardcoding paths that move on every version bump. +RUN python - <<'PY' +import glob, site +dirs = sorted(d for r in site.getsitepackages() + for pat in ("nvidia/*/lib", "tensorrt_libs", "tensorrt/lib") + for d in glob.glob(r + "/" + pat)) +open("/etc/ld.so.conf.d/nvidia-wheels.conf", "w").write(chr(10).join(dirs) + chr(10)) +print(*dirs, sep=chr(10)) +PY +RUN ldconfig + +# Fail the build rather than the first stream if the CUDA stack cannot load. +# install-tensorrt pins nvidia-cudnn-cu12, which can disagree with the cuDNN +# torch wants, and on this base there is no system copy to fall back on. +RUN python - <<'PY' +import ctypes, glob, site, sys +import torch, tensorrt +print("torch", torch.__version__, "/ cuda", torch.version.cuda, "/ tensorrt", tensorrt.__version__) +libs = [p for r in site.getsitepackages() for p in glob.glob(r + "/nvidia/cudnn/lib/libcudnn.so.*")] +if not libs: + sys.exit("cuDNN wheel not found") +ctypes.CDLL(sorted(libs)[0]) +print("cudnn loadable:", sorted(libs)[0]) +PY + # The pip package doesn't ship the demo/ dir, so clone the fork (pinned) for the server. RUN git clone https://github.com/daydreamlive/StreamDiffusion.git /src \ && cd /src && git checkout 94b9b96cb8a17d401ffbce516393d6482326ce62 From b4e8af9162bae2bba500b8304455d462e138c62b Mon Sep 17 00:00:00 2001 From: Rick Staa Date: Tue, 18 Aug 2026 16:20:18 +0200 Subject: [PATCH 3/9] fix: install a transient toolchain so insightface can build on slim insightface arrives with the ipadapter extra and is sdist-only, so it compiles a Cython/C++ extension at install time. That compiler, not any CUDA tooling, is what the devel base was actually supplying. Install it for the one step that needs it and purge it in the same layer, so the app keeps every extra it had and the image still ships no toolchain. Measured: 24.9 GB to 14.6 GB. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/build.yml | 7 ++++--- Dockerfile | 11 +++++++++-- README.md | 2 +- 3 files changed, 14 insertions(+), 6 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 205c7dd..b502883 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -3,9 +3,9 @@ name: build # Builds the app image so an orchestrator can pull and run it; amd64 only, since # Livepeer GPU work is NVIDIA/amd64. # -# This image is ~25 GB (CUDA devel, torch, TensorRT, ONNX Runtime), so it does not +# This image is ~15 GB (torch, TensorRT, ONNX Runtime), so it does not # fit a GitHub-hosted runner as shipped: the job reclaims disk first, skips the -# layer cache (a 25 GB image would blow the 10 GB Actions cache anyway), and does +# layer cache (an image this size would blow the 10 GB Actions cache anyway), and does # not run on pull requests, where a build this size buys little for what it costs. # # GHCR needs no setup (ghcr.io//, public if the repo is). Docker Hub @@ -32,7 +32,8 @@ jobs: steps: - uses: actions/checkout@v7 - # A stock runner has roughly 25 GB free, which this image alone exceeds. + # A stock runner has roughly 25 GB free, which this image plus its build + # layers still crowds. # These are preinstalled toolchains the build never touches; dropping them # frees about 25 GB more. - name: Reclaim disk diff --git a/Dockerfile b/Dockerfile index 6c74774..7c517df 100644 --- a/Dockerfile +++ b/Dockerfile @@ -36,8 +36,15 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ RUN python -m pip install --no-cache-dir \ torch==2.7.1+cu128 torchvision==0.22.1+cu128 torchaudio==2.7.1+cu128 \ --index-url https://download.pytorch.org/whl/cu128 -RUN python -m pip install --no-cache-dir \ - "streamdiffusion[tensorrt,controlnet,ipadapter] @ git+https://github.com/daydreamlive/StreamDiffusion.git@94b9b96cb8a17d401ffbce516393d6482326ce62" +# insightface (via the ipadapter extra) is sdist-only on PyPI, so it compiles a +# Cython/C++ extension here. That toolchain is the one thing the devel base was +# really providing; it is plain g++, nothing CUDA, so install it just for this +# step and purge it in the same layer rather than shipping it. +RUN apt-get update && apt-get install -y --no-install-recommends build-essential \ + && python -m pip install --no-cache-dir \ + "streamdiffusion[tensorrt,controlnet,ipadapter] @ git+https://github.com/daydreamlive/StreamDiffusion.git@94b9b96cb8a17d401ffbce516393d6482326ce62" \ + && apt-get purge -y --auto-remove build-essential \ + && apt-get clean && rm -rf /var/lib/apt/lists/* # Pulls the tensorrt + cuDNN wheels, polygraphy and onnx-graphsurgeon. Pure pip: # no compiler and no CUDA headers are involved. diff --git a/README.md b/README.md index ca6f54d..65830f9 100644 --- a/README.md +++ b/README.md @@ -106,7 +106,7 @@ The price is unchanged by the overlay: static runners advertise it from `runners CI publishes the image to `ghcr.io/livepeer/streamdiffusion-livepeer-runner` on `main` and `v*` tags. An operator then runs it with a `runners.json` like this repo's, pointed at wherever they run the container, and needs no credentials for a public package. -The image is **~25 GB** (CUDA devel, torch, TensorRT, ONNX Runtime), which is more than a GitHub-hosted runner has free out of the box, so [build.yml](.github/workflows/build.yml) reclaims disk before building and skips the build on pull requests. Building locally is `docker compose build`. +The image is **~15 GB** (torch, TensorRT, ONNX Runtime), which is still close enough to what a GitHub-hosted runner has free that [build.yml](.github/workflows/build.yml) reclaims disk before building and skips the build on pull requests. Building locally is `docker compose build`. To publish to Docker Hub as well, set the repository variable `DOCKERHUB_NAMESPACE` and the secrets `DOCKERHUB_USERNAME` and `DOCKERHUB_TOKEN`. GHCR keeps working either way. From 127fa626e7606d80873001e1283817c95f84581a Mon Sep 17 00:00:00 2001 From: Rick Staa Date: Wed, 19 Aug 2026 10:29:33 +0200 Subject: [PATCH 4/9] fix: create the engine dir at start so a cold run can compile engines The volume mounts over /models at start, so /models/engines cannot be created at build time. Without it the engines symlink dangles, and Path.mkdir(exist_ok=True) re-raises on a dangling symlink, so the first stream on a clean checkout dies with FileExistsError: 'engines'. Predates the slim base: any cold start without ./models/engines already on the host hits it. Co-Authored-By: Claude Opus 5 (1M context) --- Dockerfile | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index 7c517df..37de69a 100644 --- a/Dockerfile +++ b/Dockerfile @@ -98,4 +98,9 @@ RUN rm -rf engines && ln -s /models/engines engines EXPOSE 7860 # --api-only: skip the built Node frontend (a Livepeer client drives the API directly). -CMD ["python", "main.py", "--host=0.0.0.0", "--port=7860", "--acceleration=tensorrt", "--api-only"] +# +# /models/engines is created here, not at build time: the volume mounts over +# /models at start and would mask a build-time mkdir. The symlink above would +# then be dangling, and Path.mkdir(exist_ok=True) re-raises on a dangling +# symlink, so the first stream dies with FileExistsError: 'engines'. +CMD ["sh", "-c", "mkdir -p /models/engines && exec python main.py --host=0.0.0.0 --port=7860 --acceleration=tensorrt --api-only"] From 8db675963daf1107315ff303a2d67df4b789e451 Mon Sep 17 00:00:00 2001 From: Rick Staa Date: Wed, 19 Aug 2026 10:29:34 +0200 Subject: [PATCH 5/9] fix: link the unversioned SONAMEs the nvidia wheels leave out The wheels ship libcudart.so.12 but not the plain libcudart.so, which is a dev symlink the CUDA devel base used to provide. Parts of the TensorRT path dlopen the bare name, and ldconfig does not help because its cache is keyed on SONAMEs. Recreate the symlinks the -dev packages gave us and assert the bare-name dlopen at build time. Verified end to end: engines compile and the pipeline returns frames. Co-Authored-By: Claude Opus 5 (1M context) --- Dockerfile | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index 37de69a..fec4f65 100644 --- a/Dockerfile +++ b/Dockerfile @@ -54,11 +54,20 @@ RUN python -m streamdiffusion.tools.install-tensorrt # TensorRT inside the wheels. Register whatever lib dirs the wheels actually # installed rather than hardcoding paths that move on every version bump. RUN python - <<'PY' -import glob, site +import glob, os, site dirs = sorted(d for r in site.getsitepackages() for pat in ("nvidia/*/lib", "tensorrt_libs", "tensorrt/lib") for d in glob.glob(r + "/" + pat)) open("/etc/ld.so.conf.d/nvidia-wheels.conf", "w").write(chr(10).join(dirs) + chr(10)) +# The wheels ship only versioned SONAMEs (libcudart.so.12). The devel base used to +# supply the unversioned dev symlinks, and parts of the TensorRT path dlopen the +# plain name -- "libcudart.so" -- so recreate what the -dev packages gave us. +for d in dirs: + for so in sorted(glob.glob(d + "/*.so.*"), key=len): + base = so.split(".so.")[0] + ".so" + if not os.path.exists(base): + os.symlink(os.path.basename(so), base) + print("linked", base) print(*dirs, sep=chr(10)) PY RUN ldconfig @@ -75,6 +84,9 @@ if not libs: sys.exit("cuDNN wheel not found") ctypes.CDLL(sorted(libs)[0]) print("cudnn loadable:", sorted(libs)[0]) +for soname in ("libcudart.so", "libnvinfer.so"): + ctypes.CDLL(soname) + print("dlopen ok:", soname) PY # The pip package doesn't ship the demo/ dir, so clone the fork (pinned) for the server. From c0ec8542c53fcc23aff348c095d615288507b7be Mon Sep 17 00:00:00 2001 From: Rick Staa Date: Wed, 19 Aug 2026 11:10:36 +0200 Subject: [PATCH 6/9] docs: trim the dockerfile comments to the file's own density The comments added with the slim base ran four to eight lines where the surrounding ones run one to three. Keep the reasoning that is not recoverable from the code and drop the rest; the detail lives in the commit messages and the PR. Co-Authored-By: Claude Opus 5 (1M context) --- Dockerfile | 42 ++++++++++++++---------------------------- 1 file changed, 14 insertions(+), 28 deletions(-) diff --git a/Dockerfile b/Dockerfile index fec4f65..231350a 100644 --- a/Dockerfile +++ b/Dockerfile @@ -9,14 +9,9 @@ # cumulo-autumn repo, not the fork, so it would not run this rewritten server. # We clone the fork at a pinned commit instead. # -# Base is plain python-slim, NOT nvidia/cuda:*-devel, because nothing here -# compiles against CUDA: torch carries its own runtime in the cu128 wheels, and -# streamdiffusion.tools.install-tensorrt is pure pip — it installs the tensorrt -# and nvidia-cudnn-cu12 wheels and reads torch.version.cuda, never nvcc. -# Upstream's devel base is a single commit from Dec 2023, back when xformers and -# stable-fast still built native extensions; this image installs neither. The -# driver arrives through the NVIDIA container runtime (compose `gpus: all`), -# which is how the other Livepeer runner examples get a GPU too. +# Base is python-slim, not nvidia/cuda:*-devel: nothing here compiles against +# CUDA (torch's cu128 wheels carry the runtime, install-tensorrt is pure pip), +# and the driver arrives through the container runtime. FROM python:3.11-slim LABEL org.opencontainers.image.title="streamdiffusion-livepeer-runner" @@ -36,32 +31,27 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ RUN python -m pip install --no-cache-dir \ torch==2.7.1+cu128 torchvision==0.22.1+cu128 torchaudio==2.7.1+cu128 \ --index-url https://download.pytorch.org/whl/cu128 -# insightface (via the ipadapter extra) is sdist-only on PyPI, so it compiles a -# Cython/C++ extension here. That toolchain is the one thing the devel base was -# really providing; it is plain g++, nothing CUDA, so install it just for this -# step and purge it in the same layer rather than shipping it. +# insightface (ipadapter extra) is sdist-only, so it compiles here. Plain g++, +# nothing CUDA: install for this step only, purge in the same layer. RUN apt-get update && apt-get install -y --no-install-recommends build-essential \ && python -m pip install --no-cache-dir \ "streamdiffusion[tensorrt,controlnet,ipadapter] @ git+https://github.com/daydreamlive/StreamDiffusion.git@94b9b96cb8a17d401ffbce516393d6482326ce62" \ && apt-get purge -y --auto-remove build-essential \ && apt-get clean && rm -rf /var/lib/apt/lists/* -# Pulls the tensorrt + cuDNN wheels, polygraphy and onnx-graphsurgeon. Pure pip: -# no compiler and no CUDA headers are involved. +# Pure pip: the tensorrt + cuDNN wheels, polygraphy, onnx-graphsurgeon. RUN python -m streamdiffusion.tools.install-tensorrt -# With no system CUDA tree, the dynamic loader has to find cuDNN, cuBLAS and -# TensorRT inside the wheels. Register whatever lib dirs the wheels actually -# installed rather than hardcoding paths that move on every version bump. +# No system CUDA tree, so point the loader at the wheels' lib dirs. Globbed +# rather than hardcoded: the paths move on every version bump. RUN python - <<'PY' import glob, os, site dirs = sorted(d for r in site.getsitepackages() for pat in ("nvidia/*/lib", "tensorrt_libs", "tensorrt/lib") for d in glob.glob(r + "/" + pat)) open("/etc/ld.so.conf.d/nvidia-wheels.conf", "w").write(chr(10).join(dirs) + chr(10)) -# The wheels ship only versioned SONAMEs (libcudart.so.12). The devel base used to -# supply the unversioned dev symlinks, and parts of the TensorRT path dlopen the -# plain name -- "libcudart.so" -- so recreate what the -dev packages gave us. +# Wheels ship only versioned SONAMEs, but parts of the TensorRT path dlopen the +# plain "libcudart.so", so recreate the symlinks the -dev packages gave us. for d in dirs: for so in sorted(glob.glob(d + "/*.so.*"), key=len): base = so.split(".so.")[0] + ".so" @@ -72,9 +62,8 @@ print(*dirs, sep=chr(10)) PY RUN ldconfig -# Fail the build rather than the first stream if the CUDA stack cannot load. -# install-tensorrt pins nvidia-cudnn-cu12, which can disagree with the cuDNN -# torch wants, and on this base there is no system copy to fall back on. +# Fail the build, not the first stream: install-tensorrt pins a cuDNN that can +# disagree with torch's, and there is no system copy to fall back on. RUN python - <<'PY' import ctypes, glob, site, sys import torch, tensorrt @@ -110,9 +99,6 @@ RUN rm -rf engines && ln -s /models/engines engines EXPOSE 7860 # --api-only: skip the built Node frontend (a Livepeer client drives the API directly). -# -# /models/engines is created here, not at build time: the volume mounts over -# /models at start and would mask a build-time mkdir. The symlink above would -# then be dangling, and Path.mkdir(exist_ok=True) re-raises on a dangling -# symlink, so the first stream dies with FileExistsError: 'engines'. +# /models/engines is made at start because the volume masks a build-time mkdir, +# and mkdir(exist_ok=True) re-raises on the symlink while it dangles. CMD ["sh", "-c", "mkdir -p /models/engines && exec python main.py --host=0.0.0.0 --port=7860 --acceleration=tensorrt --api-only"] From 10e2967acae08508ce1e7ef23bbd92a3d224c05b Mon Sep 17 00:00:00 2001 From: Rick Staa Date: Wed, 19 Aug 2026 11:11:16 +0200 Subject: [PATCH 7/9] docs: frame the repo as a live runner example, not a pipeline Say what the example demonstrates and point at the live runner docs, rather than only stating what it is not. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 65830f9..d41435e 100644 --- a/README.md +++ b/README.md @@ -121,7 +121,7 @@ CI runs the same hooks, checks the compose file parses, and builds the image. ## License and attribution -This repo is an **example**, not a supported Livepeer artifact. The wrapper here (Dockerfile, [client.py](client.py), the compose files, [runners.json](runners.json)) is MIT, and CI publishing to `ghcr.io/livepeer/` is packaging convenience so an operator can pull it, not a product commitment. +This repo is an **example** of how to run StreamDiffusion on the [live runner](https://github.com/livepeer/go-livepeer/blob/master/doc/live-runner.md), not a production-ready pipeline. The wrapper here (Dockerfile, [client.py](client.py), the compose files, [runners.json](runners.json)) is MIT, and CI publishing to `ghcr.io/livepeer/` is packaging convenience so an operator can pull it, not a product commitment. What runs inside the image is daydream's [StreamDiffusion](https://github.com/daydreamlive/StreamDiffusion), itself a fork of [cumulo-autumn/StreamDiffusion](https://github.com/cumulo-autumn/StreamDiffusion). Both are **Apache-2.0**, and this repo builds the fork pinned at `94b9b96` and **unmodified**, so redistribution is permitted and there are no changes to state under section 4(b). The fork ships no `NOTICE` file; its `LICENSE` travels in the image at `/src/LICENSE`. Model weights are downloaded from Hugging Face on first run under their own licenses and are not redistributed here. From 88b0959e9c934718deb1b9de7adfa4a22445d1c7 Mon Sep 17 00:00:00 2001 From: Rick Staa Date: Wed, 19 Aug 2026 11:59:59 +0200 Subject: [PATCH 8/9] refactor: replace the dev-symlink shim with LD_LIBRARY_PATH polygraphy globs libcudart.so* across LD_LIBRARY_PATH, /usr/local/cuda/ lib64, /usr/lib and /lib, and only falls back to dlopening the bare "libcudart.so" when that search finds nothing. The devel base satisfied the search with /usr/local/cuda/lib64; slim satisfies it by naming the wheel dirs. Recreating 33 dev symlinks and an ldconfig entry was treating the fallback as if it were the primary path. The assert now constructs polygraphy's Cuda(), which is the consumer that has to work, and catches a python bump moving the hardcoded paths. Verified with a cold engine compile: 3 engines built, frames returned. Co-Authored-By: Claude Opus 5 (1M context) --- Dockerfile | 38 ++++++++------------------------------ 1 file changed, 8 insertions(+), 30 deletions(-) diff --git a/Dockerfile b/Dockerfile index 231350a..e949fce 100644 --- a/Dockerfile +++ b/Dockerfile @@ -42,40 +42,18 @@ RUN apt-get update && apt-get install -y --no-install-recommends build-essential # Pure pip: the tensorrt + cuDNN wheels, polygraphy, onnx-graphsurgeon. RUN python -m streamdiffusion.tools.install-tensorrt -# No system CUDA tree, so point the loader at the wheels' lib dirs. Globbed -# rather than hardcoded: the paths move on every version bump. -RUN python - <<'PY' -import glob, os, site -dirs = sorted(d for r in site.getsitepackages() - for pat in ("nvidia/*/lib", "tensorrt_libs", "tensorrt/lib") - for d in glob.glob(r + "/" + pat)) -open("/etc/ld.so.conf.d/nvidia-wheels.conf", "w").write(chr(10).join(dirs) + chr(10)) -# Wheels ship only versioned SONAMEs, but parts of the TensorRT path dlopen the -# plain "libcudart.so", so recreate the symlinks the -dev packages gave us. -for d in dirs: - for so in sorted(glob.glob(d + "/*.so.*"), key=len): - base = so.split(".so.")[0] + ".so" - if not os.path.exists(base): - os.symlink(os.path.basename(so), base) - print("linked", base) -print(*dirs, sep=chr(10)) -PY -RUN ldconfig +# polygraphy globs libcudart.so* across LD_LIBRARY_PATH, /usr/local/cuda/lib64, +# /usr/lib and /lib. None of those exist here, so point it at the wheels. +ENV LD_LIBRARY_PATH=/usr/local/lib/python3.11/site-packages/nvidia/cuda_runtime/lib:/usr/local/lib/python3.11/site-packages/nvidia/cudnn/lib:/usr/local/lib/python3.11/site-packages/nvidia/cublas/lib:/usr/local/lib/python3.11/site-packages/tensorrt_libs -# Fail the build, not the first stream: install-tensorrt pins a cuDNN that can -# disagree with torch's, and there is no system copy to fall back on. +# Fail the build, not the first stream. Also catches a python bump moving the +# paths hardcoded above. RUN python - <<'PY' -import ctypes, glob, site, sys import torch, tensorrt +from polygraphy.cuda.cuda import Cuda print("torch", torch.__version__, "/ cuda", torch.version.cuda, "/ tensorrt", tensorrt.__version__) -libs = [p for r in site.getsitepackages() for p in glob.glob(r + "/nvidia/cudnn/lib/libcudnn.so.*")] -if not libs: - sys.exit("cuDNN wheel not found") -ctypes.CDLL(sorted(libs)[0]) -print("cudnn loadable:", sorted(libs)[0]) -for soname in ("libcudart.so", "libnvinfer.so"): - ctypes.CDLL(soname) - print("dlopen ok:", soname) +Cuda() +print("polygraphy found the cuda runtime") PY # The pip package doesn't ship the demo/ dir, so clone the fork (pinned) for the server. From 3482a00bcf9c8fc8ade8652794ba7dfddf33a4f7 Mon Sep 17 00:00:00 2001 From: Rick Staa Date: Wed, 19 Aug 2026 12:12:06 +0200 Subject: [PATCH 9/9] refactor: fold the cuda check into the install-tensorrt step Same guarantee in two lines instead of seven, and one layer fewer. The ENV moves above the install, which nothing before it reads. Co-Authored-By: Claude Opus 5 (1M context) --- Dockerfile | 19 +++++-------------- 1 file changed, 5 insertions(+), 14 deletions(-) diff --git a/Dockerfile b/Dockerfile index e949fce..f5f66d7 100644 --- a/Dockerfile +++ b/Dockerfile @@ -39,22 +39,13 @@ RUN apt-get update && apt-get install -y --no-install-recommends build-essential && apt-get purge -y --auto-remove build-essential \ && apt-get clean && rm -rf /var/lib/apt/lists/* -# Pure pip: the tensorrt + cuDNN wheels, polygraphy, onnx-graphsurgeon. -RUN python -m streamdiffusion.tools.install-tensorrt - -# polygraphy globs libcudart.so* across LD_LIBRARY_PATH, /usr/local/cuda/lib64, -# /usr/lib and /lib. None of those exist here, so point it at the wheels. ENV LD_LIBRARY_PATH=/usr/local/lib/python3.11/site-packages/nvidia/cuda_runtime/lib:/usr/local/lib/python3.11/site-packages/nvidia/cudnn/lib:/usr/local/lib/python3.11/site-packages/nvidia/cublas/lib:/usr/local/lib/python3.11/site-packages/tensorrt_libs -# Fail the build, not the first stream. Also catches a python bump moving the -# paths hardcoded above. -RUN python - <<'PY' -import torch, tensorrt -from polygraphy.cuda.cuda import Cuda -print("torch", torch.__version__, "/ cuda", torch.version.cuda, "/ tensorrt", tensorrt.__version__) -Cuda() -print("polygraphy found the cuda runtime") -PY +# Pure pip: the tensorrt + cuDNN wheels, polygraphy, onnx-graphsurgeon. polygraphy +# globs libcudart.so* over LD_LIBRARY_PATH, which on slim is the only place it +# will find one; the check fails the build rather than the first stream. +RUN python -m streamdiffusion.tools.install-tensorrt \ + && python -c "import torch,tensorrt; from polygraphy.cuda.cuda import Cuda; Cuda(); print('ok',torch.__version__,tensorrt.__version__)" # The pip package doesn't ship the demo/ dir, so clone the fork (pinned) for the server. RUN git clone https://github.com/daydreamlive/StreamDiffusion.git /src \