From 6dc95620cefe96467670c086efdb0b9085b5f8da Mon Sep 17 00:00:00 2001 From: firejune Date: Thu, 29 Oct 2026 10:44:15 +0900 Subject: [PATCH] docs: route A measured on a GPU box, route C's real limits, keypoints on a Mac CPU, and route B's download list MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Getting Started #1 items 2 and 3, for the person setting up the machine. - Route A (own NVIDIA GPU): run as the guide writes it, through `rig-parts comfy paint` and `comfy seethrough`, on an RTX 5070 Ti box with everything already installed — painting 44.1 s on the GPU, the full See-through run 225.3 s and 29 layers. Which checkpoint and ControlNet upload the guide means is now pinned by file hash, and the checkpoint downloads without a login. Install time and peak VRAM on an empty card are marked not measured. - Route C (no local GPU): the See-through Space's actual limits — 120 s of GPU per run, Hugging Face's 2 / 5 / 40 minutes a day by tier — replace "1–2 runs a day"; the reader is told to tick "Split arms, eyes and ears" and match the resolution. Its crop-or-pad wording contradicts itself across its own pages; recorded, and the advice holds either way. - Keypoints on CPU: the `uv run` recipe on an Apple M4 — 44 s with an empty package cache, 19 s the first time the models load, about 2 s after; outputs byte-identical across runs. A body-18 diagram drawn from the format, not from art. - Route B (Mac, no NVIDIA GPU): the section and the exact download list (about 23 GB of models plus nightly PyTorch), marked not measured yet — running it is the owner's call. Every figure carries machine, date and versions in docs/REFERENCE.md. Link check 122/0; the repo's tests green. Refs #1 Co-Authored-By: Roa <342598975+fj-roa[bot]@users.noreply.github.com> --- README.md | 127 ++++++++++++++++------ docs/KEYPOINTS.md | 9 ++ docs/REFERENCE.md | 216 ++++++++++++++++++++++++++++++-------- docs/keypoints-body18.svg | 62 +++++++++++ 4 files changed, 339 insertions(+), 75 deletions(-) create mode 100644 docs/keypoints-body18.svg diff --git a/README.md b/README.md index 98b369d..eaa4c7c 100644 --- a/README.md +++ b/README.md @@ -128,8 +128,9 @@ small specks are normal; a large red area means a part is missing. uv run --with rtmlib --with onnxruntime python ../getting-started/scripts/keypoints.py inputs/painting.png --out keypoints.json ``` -The first run downloads two pose models (about 300 MB) and some Python packages. You -should see `18 observed, 0 occluded, 0 missing -> keypoints.json`. +The first run downloads two pose models (330 MB) and some Python packages (580 MB), about a +minute in all on a fast connection; after that it takes about 2 seconds. You should see +`18 observed, 0 occluded, 0 missing -> keypoints.json`. No Python? [docs/KEYPOINTS.md](docs/KEYPOINTS.md) has a hand-placed set for this very painting; save it as `keypoints.json`. Its left ankle is already where step 7's first line @@ -457,8 +458,8 @@ come [after](#the-joints). | | Route A: NVIDIA GPU | Route B: Apple Silicon Mac | Route C: no GPU | | --- | --- | --- | --- | | **You'll need** | a card with 12–16 GB of memory, ~23 GB of disk, Python | an M1–M4 Mac, ~23 GB of disk, Python | a browser and a free Hugging Face account | -| **Per character** | seconds for the painting, a few minutes per See-through run | we don't know yet | two web runs, 2–3 minutes each, at 1–2 runs a day | -| **Tried by** | the project itself | nobody yet | written from See-through's own pages | +| **Per character** | 44 seconds for the painting and under 4 minutes per See-through run, on a 16 GB card | we don't know yet | two web runs of up to 2 minutes each, on a free tier of 5 GPU minutes a day | +| **Tried by** | the project, 2026-10-29, on an RTX 5070 Ti | nobody yet | written from the demo's own pages, read 2026-10-29 | ### Route A: your own NVIDIA GPU @@ -469,9 +470,14 @@ git. Each requirement's exact wording and source is in [docs/REFERENCE.md](docs/REFERENCE.md#requirements-and-their-sources). You install ComfyUI, one set of custom nodes and four model files; then two `rig-parts` -commands drive ComfyUI for you. (Any other way of running See-through works too.) The -project's own runs this way took 18–20 seconds per painting and about 3 minutes per -See-through run, on a card that record doesn't name — an order of magnitude, not a promise. +commands drive ComfyUI for you. (Any other way of running See-through works too.) We ran +both commands on 2026-10-29 against the project's own box — Windows, an NVIDIA GeForce RTX +5070 Ti with 16 GB, ComfyUI 0.29.2, the See-through nodes at 0.5.0 — with the models already +on disk: **44 seconds** for the painting and **3 minutes 45 seconds** for one See-through run +at the default settings. One run each, on one card; your card will differ. The versions, +the exact lines and what the card's memory did are in +[docs/REFERENCE.md](docs/REFERENCE.md#route-a-measured-2026-10-29). What we have not +measured: installing ComfyUI from nothing, and the model downloads themselves. **1. Install ComfyUI**, following its own [manual install](https://github.com/Comfy-Org/ComfyUI#manual-install-windows-linux): @@ -512,6 +518,12 @@ in [docs/REFERENCE.md](docs/REFERENCE.md#the-model-files-exactly). | `4x-AnimeSharp.pth` | `models/upscale_models/` | [Kim2091/AnimeSharp](https://huggingface.co/Kim2091/AnimeSharp) (non-commercial licence; only enlarges the painting) | | See-through's two models | none | the wrapper downloads them from [Hugging Face](https://huggingface.co/layerdifforg/seethroughv0.0.2_layerdiff3d) on its first run | +The first three are 6.94 GB, 2.50 GB and 0.07 GB; See-through's two are 10.16 GB and +3.28 GB — about 23 GB in all. Those are the exact files on the project's box: on +2026-10-29 each one's SHA-256 matched its download page, so the ControlNet row above is +xinsir's upload and not another file of the same name. Civitai sent us the checkpoint +without asking us to sign in. + **4. Make the painting.** Do [step 1 of Try it first](#1-install-the-packages) and work in that `my-rig` folder. Copy the example `config.json` and `pose_a.json` from [docs/PAINTING-RECIPE.md](docs/PAINTING-RECIPE.md) — a checkpoint, a prompt in parts, and a @@ -521,8 +533,13 @@ stick figure that holds the arms away from the body — then: bunx rig-parts comfy paint --config config.json --out inputs --host http://127.0.0.1:8188 ``` -You should get `inputs/painting_74101.png` at 1664 x 2432, with the full prompts in a -`_meta.json` beside it. `--seeds 4` makes four to choose from. Rename the one you keep to +You should see `box: every node, input and model of the painting graph is on its +/object_info`, then `seed 74101: GPU job started`, and after a wait a line ending +`-> inputs/painting_74101.png (1664x2432)`. That wait was **44 seconds** on the box above. +You get `inputs/painting_74101.png` at 1664 x 2432 — with this recipe, a full-figure +character on white with her arms held out from her sides — the full prompts in a +`_meta.json` beside it, and the stick figure the ControlNet was given as +`control_pose-….png`. `--seeds 4` makes four to choose from. Rename the one you keep to `inputs/painting.png`. **5. Run See-through, twice.** Follow [Try it first](#try-it-first--no-gpu-about-3-minutes) @@ -536,8 +553,12 @@ bunx rig-parts comfy seethrough --image work/st_input_full.png --out inputs/laye bunx rig-parts comfy seethrough --image work/st_input_head.png --out inputs/layers/head --host http://127.0.0.1:8188 ``` -Each `--out` folder gets `layers.json`, `meta.json`, a `parts/` folder of PNGs and -`previews/`. The defaults are resolution 1024, 30 steps, seed 42. +Each run prints `queue empty`, `GPU job started`, then `GPU job finished in … s` and a line +like `29 layer(s) on a 1024x1024 canvas, read back green by the layer reader`, naming the +layers. The full run took **225 seconds** on the box above (3 minutes 45 seconds; the +head run was not timed). Each `--out` folder gets `layers.json`, `meta.json`, a `parts/` +folder of PNGs and `previews/`; `meta.json` records the run's own time and what the card's +free memory did. The defaults are resolution 1024, 30 steps, seed 42. ### Route B: an Apple Silicon Mac @@ -577,10 +598,32 @@ should name its address, `http://127.0.0.1:8188` by default. assume an NVIDIA card. (ComfyUI's [desktop app for macOS](https://docs.comfy.org/installation/desktop/macos) exists too; this guide doesn't walk through it.) +**What this route downloads** — *not measured yet*: these are the names and the sizes +their publishers state, read on 2026-10-29. Nobody has timed them or run the result on a +Mac for this guide. + +| What | Where from | Size, as stated | +| --- | --- | --- | +| ComfyUI, the repository | `git clone` of [Comfy-Org/ComfyUI](https://github.com/Comfy-Org/ComfyUI) (its latest release was v0.43.0 on 2026-10-27) | not stated; not measured | +| PyTorch for the Mac's GPU: `torch`, `torchvision`, `torchaudio` nightly wheels, `macosx_14_0_arm64` | the `nightly/cpu` index in step 2 (2.16.0.dev, 0.30.0.dev, 2.11.0.dev on 2026-10-29) | the nightly server does not answer size requests; the current stable `torch` wheel for the same Mac is 138,556,890 bytes on PyPI | +| ComfyUI's other Python packages (`requirements.txt`, 33 entries) | PyPI | not stated; not measured | +| The See-through nodes, [jtydhr88/ComfyUI-See-through](https://github.com/jtydhr88/ComfyUI-See-through) 0.5.0, and its 7 packages (`diffusers`, `peft`, `accelerate`, `opencv-python`, `scikit-learn`, `bitsandbytes`, `matplotlib`) | GitHub, PyPI | not stated; not measured. Whether `bitsandbytes` installs on macOS is not measured | +| `novaAnimeXL_ilV80.safetensors` | [Civitai](https://civitai.com/models/376130) | 6,938,418,620 bytes (6.94 GB) | +| `controlnet-openpose-sdxl-1.0.safetensors` | [xinsir](https://huggingface.co/xinsir/controlnet-openpose-sdxl-1.0), `diffusion_pytorch_model.safetensors` | 2,502,139,104 bytes (2.50 GB) | +| `4x-AnimeSharp.pth` | [Kim2091/AnimeSharp](https://huggingface.co/Kim2091/AnimeSharp) | 67,010,245 bytes (0.07 GB) | +| `layerdifforg/seethroughv0.0.2_layerdiff3d` | Hugging Face, fetched by the nodes on first use | 10,159,649,987 bytes (10.16 GB) | +| `layerdifforg/seethroughv0.0.1_marigold` | Hugging Face, fetched by the nodes on first use | 3,276,274,643 bytes (3.28 GB) | + +The five model files come to 22.95 GB. The exact file sizes, hashes and licences are in +[docs/REFERENCE.md](docs/REFERENCE.md#the-model-files-exactly). + ### Route C: no GPU at all -**You'll need:** a web browser, a free Hugging Face account, and patience: the demo allows -about 1–2 runs a day, and a character needs two. +**You'll need:** a web browser, a free Hugging Face account, and patience: the demo runs +on Hugging Face's shared GPUs, which give a free account 5 minutes of GPU a day (2 minutes +signed out), each run is budgeted at up to 2 minutes, and a character needs two runs. So +expect one character per day or two. (Read from the demo's and Hugging Face's own pages on +2026-10-29; we have not run it ourselves.) **The painting.** Any image generator will do, as long as the result is one character, front-facing, full body, taller than wide, on a plain background, arms away from the body. @@ -588,24 +631,40 @@ Save it as `inputs/painting.png` in the `my-rig` folder of [step 1 of Try it first](#1-install-the-packages). **The layers.** See-through's authors run two free web demos — a -[Hugging Face Space](https://huggingface.co/spaces/24yearsold/see-through-demo) (1–2 runs -a day once you register, 2–3 minutes each) and a +[Hugging Face Space](https://huggingface.co/spaces/24yearsold/see-through-demo) and a [ModelScope demo](https://modelscope.cn/studios/ljsabc/See-Through) for users in mainland China. Each gives you a layered `.psd`, which `rig-parts` reads as it is. +The Space's page has an image box, a **Resolution** slider from 768 to 1280 (768 is what it +recommends for the free tier: higher "may timeout or exhaust your daily quota"), a +**Seed** slider, a **Split arms, eyes and ears** box, and a **Download layered PSD** link +that appears when a run is done. + 1. Do steps 1 and 3 of [Try it first](#try-it-first--no-gpu-about-3-minutes) with your painting (skip step 2). Step 3 writes `work/st_input_full.png`, your painting centred on - a white square. **Upload that square, not the painting** — the Space centre-crops - anything that isn't square, which would cut off the head and the feet. -2. Download the PSD as `inputs/full.psd`. Read it with `bunx rig-parts layers inputs/full.psd`. -3. Continue to step 4 for the head box; upload `work/st_input_head.png` and save the result - as `inputs/head.psd`. -4. Everywhere the steps write `inputs/layers/full` and `inputs/layers/head`, write + a white square. **Upload that square, not the painting.** The Space's page says a + non-square image is centre-cropped and its upload box says it is padded (both read + 2026-10-29); the square makes the question moot, and a crop would cut off the head and + the feet. +2. Tick **Split arms, eyes and ears**: Route A's ComfyUI nodes run with that split on, and + the example's layers are split that way (`handwear-r`, `handwear-l`, …), so your PSD + then matches what the rest of this guide reads. Set + **Resolution** to 1024 to match the `seethrough.resolution` in your `config.json` — or + leave it at 768 and write 768 there. Submit and wait. +3. Download the PSD as `inputs/full.psd`. Read it with `bunx rig-parts layers inputs/full.psd`. +4. Continue to step 4 for the head box; upload `work/st_input_head.png` with the same + settings and save the result as `inputs/head.psd`. +5. Everywhere the steps write `inputs/layers/full` and `inputs/layers/head`, write `inputs/full.psd` and `inputs/head.psd`. +The ModelScope demo's own page, read the same day, states no limit and no login; See-through's +README says it is "completely free now, and supports slightly higher resolution than the +HuggingFace demo". + A PSD carries no per-layer depth, so one optional feature — turning the head along with the figure — isn't available on this route; the rest works the same. We haven't run the PSD -path end to end ourselves yet. +path end to end ourselves yet, and the layer names a PSD comes back with have not been +checked against what `rig-parts` reads from Route A's `layers.json`. ### The joints @@ -613,13 +672,16 @@ path end to end ourselves yet. guesses the torso from the clothes and draws no legs. **On the CPU**: [`scripts/keypoints.py`](scripts/keypoints.py) finds the 18 joints with a -pose model (RTMPose), as in [step 6](#6-find-the-joints): about 300 MB of models the first -time, then a few seconds per painting. On the example painting its joints were within 4–52 -pixels of a first hand-placed set, and the rig built green on them unchanged. +pose model (RTMPose), as in [step 6](#6-find-the-joints). No GPU, no account. Measured on +2026-10-29 on an Apple M4 Mac with empty caches: the Python packages the first time, 584 MB +in 44 seconds; the two model files the first time, 330 MB in 19 seconds; after that **about +2 seconds per painting**, of which the model's own work is 0.6–1.0 s. The same joints came +out every run, byte for byte. On the example painting they were within 4–52 pixels of a +first hand-placed set, and the rig built green on them unchanged. **By hand**: read the 18 positions off the painting in an image editor and write them into -a small JSON file. The format, a diagram of where each joint goes, and a complete example -are in [docs/KEYPOINTS.md](docs/KEYPOINTS.md). +a small JSON file. The format, a picture of where each of the 18 joints goes, and a +complete example are in [docs/KEYPOINTS.md](docs/KEYPOINTS.md). ## Troubleshooting @@ -650,7 +712,9 @@ are in [docs/KEYPOINTS.md](docs/KEYPOINTS.md). didn't load, or a model file is missing or misnamed. Restart ComfyUI, read its `[ComfyUI-See-through]` lines, and check the file names against the table in Route A. - *Out of GPU memory during See-through* — the wrapper's README says, in order: a lower - `resolution`, then group offload (`--offload` here, about 2–3x slower). `--nf4` turns on + `resolution`, then group offload (`--offload` here, about 2–3x slower; See-through's own + README puts it at about 1.5x, read 2026-10-29). `--offload` needs `diffusers` 0.37 or + later, which the project's box does not have, so it was not measured. `--nf4` turns on its 4-bit mode, which needs the `bitsandbytes` package. - *The command waits and prints nothing new* — `rig-parts` waits for ComfyUI's queue to be empty before it sends a job, so it never cuts in front of someone else's. `--wait` @@ -665,8 +729,11 @@ are in [docs/KEYPOINTS.md](docs/KEYPOINTS.md). **The web demo (Route C)** -- *The Space says you're out of quota, or asks you to log in* — the daily limit. Sign in to - Hugging Face, or come back tomorrow. +- *The Space says you're out of quota, or asks you to log in* — the daily limit: 2 minutes + of GPU a day signed out, 5 with a free account, and it resets 24 hours after your first + run. Sign in to Hugging Face, or come back tomorrow. +- *The run times out on the Space* — lower **Resolution** to 768; each run has at most 2 + minutes of GPU. Then write `768` into `seethrough.resolution` in `config.json`. - *`rig-parts layers` refuses the PSD* — it names the reason (not 8-bit RGB, a hidden layer, a layer at less than full opacity, a layer with no pixels). Re-export without changing the layers, or edit in an image editor that keeps the layer names. diff --git a/docs/KEYPOINTS.md b/docs/KEYPOINTS.md index 10fae4c..756032e 100644 --- a/docs/KEYPOINTS.md +++ b/docs/KEYPOINTS.md @@ -12,6 +12,15 @@ each joint, with x to the right and y down from the top-left corner. `r_` and `l **character's** right and left: on a figure facing you, `r_shoulder` is on the left of the picture. +![The 18 joints on a standing figure: nose, eyes and ears on the head; neck between the shoulders; shoulder, elbow and wrist down each arm; hip, knee and ankle down each leg](keypoints-body18.svg) + +The picture is drawn from the stick figure in [PAINTING-RECIPE.md](PAINTING-RECIPE.md), +the same 18 names on an 832 x 1216 canvas, not from any painting. Each dot goes where the +joint turns: the shoulder at the top of the arm where it meets the body, the elbow and the +knee at the bend, the wrist and the ankle at the end of the limb, the hips at the top of +each thigh, the eyes at their centres, the ears where they meet the head. The same points, +as text: + ``` r_eye l_eye r_ear nose l_ear diff --git a/docs/REFERENCE.md b/docs/REFERENCE.md index 02f99a4..16ab1d1 100644 --- a/docs/REFERENCE.md +++ b/docs/REFERENCE.md @@ -137,7 +137,10 @@ yours differs in a detail from what the README says you should see, the versions the ones that produced that text. The third-party READMEs quoted below were read on 2026-10-13: ComfyUI at `f9c6185`, -See-through at `df019de`, the ComfyUI-See-through wrapper at `98d754b` (its 0.5.0). +See-through at `df019de`, the ComfyUI-See-through wrapper at `98d754b` (its 0.5.0). They +were read again on 2026-10-29: See-through's head was still `df019de` and the wrapper's +`98d754b`; ComfyUI's head was `0df1646a`, its latest release v0.43.0 (2026-10-27). Where a +sentence changed between the two readings, the section that quotes it says so. ### What was run @@ -192,9 +195,12 @@ output of the second run was byte-identical to the first (`skeleton.json`, `skeleton.model.json`, `skeleton.png`, `keypoints.json`, `wave.pose.json`, `wave.motion.json`, `config.json`). -Of the GPU routes, only the joint script was run. Route A's times come from the project's -own record (below). Route B was not run (owner decision, 2026-10-13: written from the -official docs, one visible note). Route C was not run. +Of the GPU routes, the joint script was run, and on 2026-10-29 Route A's two commands were +run once each against the project's own GPU box, with the models already on it +([*Route A, measured 2026-10-29*](#route-a-measured-2026-10-29)); the ComfyUI install and +the downloads were not. Route B was not run (owner decision, 2026-10-13: written from the +official docs, one visible note; its download list was read from the publishers on +2026-10-29). Route C was not run; its two demos' pages were read on 2026-10-29. Of *Use what you made*, the Spine editor import was run, from the command line, on a copy of the smoke run's build — and refused; the whole run is under @@ -790,11 +796,34 @@ animation at a time); the page's breeze is read there only through the player. ### The joint script, `scripts/keypoints.py` -- **Downloads, first run**: two model zips, 94,223,081 and 212,739,469 bytes (sizes from the - download server by HEAD request), unpacked in `~/.cache/rtmlib` to 101,400,344 + - 228,708,578 bytes (330 MB). uv installs 224 MB of Python packages and, if it needs one, a - Python (72 MB here). The download time is not measured yet: an earlier job on this Mac had - already cached both models. +**Measured 2026-10-29**, on the Apple M4 Mac of *Tested with* (macOS 26.6.1, 32 GB; uv +0.12.17, which chose Python 3.12.14; rtmlib 0.0.16, onnxruntime 1.31.1, opencv 5.0.0, +numpy 2.5.4), with the README's step-6 command as written, pointed at the example painting +of *Try it first* (rig-c/exam `01436ee`, sha256 `1064fc12…`), and every cache empty: uv's +(`UV_CACHE_DIR`) and rtmlib's (`XDG_CACHE_HOME`), both in a scratch directory. Each line is +`/usr/bin/time -p` wall clock of the whole `uv run …` command. + +| Run | What was cold | Wall | The script's own lines | +| --- | --- | --- | --- | +| packages only (`python -c "import rtmlib"`) | uv's cache | 44.21 s | — ; uv's cache afterwards 584 MB, 9 packages, Python 3.12.14 | +| first `keypoints.py` | the two models | 18.92 s | `models loaded in 16.4 s, one image in 0.6 s` | +| second, third, fourth | nothing | 2.36, 1.94, 1.73 s | `models loaded in 0.2–0.3 s, one image in 0.6–1.0 s` | + +Every run printed `18 observed, 0 occluded, 0 missing`, and all four wrote the same bytes +(sha256 `93367973…`); the left ankle is at `[855.7, 2154.1]`, as the README's step 7 says. +The model downloads were 89.9 and 203 MB zipped (the server's own progress figures), +unpacked to 101,400,344 + 228,708,578 bytes = 330,108,922 bytes (319 MiB) in +`rtmlib/hub/checkpoints`. The same cold-model run on a different painting of the same size +took 19.12 s (`models loaded in 17.5 s, one image in 0.4 s`). So the first run of step 6 +on an empty machine is about a minute (44 + 19 s) plus however long `uv` takes to fetch a +Python if it has none; after that a painting takes about two seconds. + +- **Downloads, first run** (2026-10-13 reading): two model zips, 94,223,081 and 212,739,469 + bytes (sizes from the download server by HEAD request), unpacked in `~/.cache/rtmlib` to + 101,400,344 + 228,708,578 bytes (330 MB). uv installs 224 MB of Python packages and, if + it needs one, a Python (72 MB here). ~~The download time is not measured yet~~ — measured + above on 2026-10-29. (uv's cache held 584 MB in that run against 224 MB installed + here: the cache keeps the wheels as well as the unpacked packages.) - **Time, warm**: on the example painting, 3.6 s wall, of which models loaded in 0.6 s and one image in 1.6 s. On a second painting: models 0.3–0.6 s, inference 1.4–2.4 s, 2.4–2.5 s wall. The first run in a new uv environment took 43.5 s wall (5.5 s user), measured twice, @@ -827,12 +856,56 @@ why. Both build green. The README's map leaves the legs out for this reason — not move them, and a map written for one bone list misses on the other. To give each leg its own part, the parts guide's `assemble.cuts` cuts the layer in two. -### Route A, from the project's own record +### Route A, measured 2026-10-29 + +One painting and one See-through run, through the README's own two commands, against the +project's GPU box. The box was not installed for this guide: ComfyUI, the nodes and the +models were already on it, so the install and the downloads are still not measured, and +the figures are one run each on one card. + +| | | +| --- | --- | +| Box | Windows (ComfyUI reports `win32`), NVIDIA GeForce RTX 5070 Ti, 16 GB (ComfyUI reports 17,094,475,776 bytes; `nvidia-smi` 16,303 MiB), driver 610.88 | +| ComfyUI | 0.29.2, its Windows standalone build, Python 3.12.10, PyTorch `2.8.0.dev20250613+cu128`, started with `--fast fp16_accumulation --disable-pinned-memory`; `diffusers` 0.33.1 | +| See-through nodes | ComfyUI-See-through 0.5.0 at `98d754b` (read from its `pyproject.toml` and `.git/HEAD` on the box); 7 `SeeThrough_*` node classes on `/object_info`, which is the wrapper's "Loaded 7 nodes" | +| `@rig-c/parts` | 1.73.1, installed with `bun add @rig-c/parts` into an empty folder; Bun 1.4.4 | +| Client | the Apple M4 Mac of *Tested with*, on the same LAN | +| Queue | empty on both of the box's instances before each job, read by the adapter (`queue empty`) and by hand | + +The painting, `bunx rig-parts comfy paint --config config.json --out inputs --host … --wait 300` +with [PAINTING-RECIPE.md](PAINTING-RECIPE.md)'s `config.json` and `pose_a.json` as written: + +| | | +| --- | --- | +| Adapter's lines | `box: every node, input and model of the painting graph is on its /object_info`; `control: pose file … at 832x1216 -> inputs/control_pose-102906e39d33.png`; `seed 74101: GPU job finished in 44.1 s -> inputs/painting_74101.png (1664x2432)` | +| GPU job | **44.1 s** (`elapsed_s` in `painting_74101_meta.json`) | +| Whole command | 50.67 s wall, 0.38 s user | +| Output | `painting_74101.png`, 1664 x 2432, 1,380,261 bytes: one full-figure character, front view, arms out from the body, white background — the recipe's own prompt | -The parts guide, [AUTHORING §8](https://github.com/rig-c/parts/blob/main/docs/AUTHORING.md#8-what-one-character-costs), -records 18 to 20 seconds per painting and 171 to 199 seconds per See-through run through the -same ComfyUI wrapper. That page does not name the GPU. The ComfyUI install time, the -PyTorch size, the model download times and the peak GPU memory are not measured yet. +The See-through full run, `bunx rig-parts comfy seethrough --image work/st_input_full.png +--out inputs/layers/full --host … --wait 300`, on the 2432 x 2432 square `rig-parts inputs` +made from that painting, at the defaults (resolution 1024, 30 steps, seed 42, no offload, no +LaMa, no nf4): + +| | | +| --- | --- | +| Adapter's lines | `queue empty`; `GPU job finished in 225.3 s`; `29 layer(s) on a 1024x1024 canvas, read back green by the layer reader` | +| GPU job | **225.3 s** (`elapsed_s` in `meta.json`) | +| Whole command | 234.44 s wall, 1.29 s user | +| Output | `layers.json`, `meta.json`, `parts/` (29 PNG, 1024 x 1024), `previews/` (2 PNG); 9.6 MB | +| Card memory | the adapter's `meta.json`: `vram_free_before_gb` 7.16, `vram_free_low_gb` 5.0 on the card. Polled from the Mac every 2 s (111 samples): free memory fell to 5,755,945,498 bytes at its lowest, so **10.56 GiB of the card's 15.92 GiB was in use at the peak** — but the box keeps other models resident (free was 7.16 GB before the job began), so the run's own footprint is somewhere between the 2.2 GB drop and the 10.56 GiB total, and is not separated here | +| The head run | not run (one See-through job was the measurement's scope) | + +For comparison, the parts guide, [AUTHORING §8](https://github.com/rig-c/parts/blob/main/docs/AUTHORING.md#8-what-one-character-costs), +records 18 to 20 seconds per painting and 171 to 199 seconds per See-through run through +the same wrapper on a card it does not name. Today's painting took more than twice as long +on the 5070 Ti (44.1 s against 18.3–20.2; the same recipe, so the difference is the box, +its `fp16_accumulation` flag, a cold checkpoint load, or the card — not separated), and +the See-through run 13 to 32 percent longer (225.3 s against 171–199). + +Still not measured: the ComfyUI install time and disk, the PyTorch download, the model +download times, `--offload` (the wrapper's README says it needs `diffusers>=0.37.0`, and +the box has 0.33.1), `--nf4`, and the run's own peak memory on an otherwise empty card. The wrapper's README describes its group offload (`--offload`) as cutting peak VRAM a great deal at "2–3x slower". The parts guide says the painting comes back at twice the @@ -880,14 +953,26 @@ Where a project states no minimum, this says so rather than filling one in. - **PyTorch**: ComfyUI says 2.7 "is minimally supported", and a CUDA 13.0 build is required on 20 series and above. - **git**, to clone ComfyUI and the wrapper (no minimum stated). -- **Accounts**: a Civitai download page for the checkpoint; whether it asks you to sign in - is not measured yet. The See-through models download from Hugging Face without a stated - login ([wrapper README][wrapper]). +- **Accounts**: a Civitai download page for the checkpoint. Measured 2026-10-29, signed + out: a HEAD request to `https://civitai.com/api/download/models/1854228` answered `307` + straight to the file host, which answered `200`, `content-length: 6938418620`, + `content-disposition: attachment; filename="novaAnimeXL_ilV80.safetensors"` — no login + page. (A HEAD, not the 6.9 GB download itself.) The See-through models download from + Hugging Face without a stated login ([wrapper README][wrapper]). - **Starting ComfyUI for another machine**: `python main.py --listen` is ComfyUI's flag; it listens on every interface. - **The wrapper's startup line**: its code prints `[ComfyUI-See-through] Loaded 7 nodes` - (seven at 0.5.0; read in the code, not run). Its README says the nodes appear under a - **SeeThrough** category. + (seven at 0.5.0; read in the code). On 2026-10-29 the project's box, running 0.5.0, + listed exactly seven `SeeThrough_*` classes on `/object_info`: `LoadLayerDiffModel`, + `GenerateLayers`, `LoadDepthModel`, `GenerateDepth`, `PostProcess`, `PartsToLayers`, + `SavePSD`. Its README says the nodes appear under a **SeeThrough** category. +- **Offload and the 4-bit mode, re-read 2026-10-29**: the wrapper's README still says + group offload is "2–3x slower" and its own table reads 138 s against 385 s at 1280 + (2.8x), with "Requires `diffusers>=0.37.0`"; See-through's README now says "Group + offload trades speed for VRAM savings (roughly 1.5x slower)", "~10 GB at 1280 + resolution" for 12 GB cards, "~8 GB peak VRAM at 1280" for the NF4 pipeline, and lists a + third option, a "block swap pipeline", also "~8 GB peak VRAM at 1280 resolution with bf16 + precision". The two projects disagree on the slowdown; neither figure was measured here. ### Route B: Apple Silicon Mac @@ -913,18 +998,61 @@ Where a project states no minimum, this says so rather than filling one in. needs, while its `pyproject.toml` does not list it among its dependencies. Whether `bitsandbytes` installs on macOS is not measured yet. - **Accounts**: as Route A. +- **What it downloads**: the README's table under Route B, read on 2026-10-29. Sizes for + the model files are the Hugging Face and Civitai APIs' (*The model files, exactly*). The + PyTorch nightly index (`download.pytorch.org/whl/nightly/cpu`) listed + `torch-2.16.0.dev20261028-cp313-cp313-macosx_14_0_arm64.whl`, + `torchvision-0.30.0.dev20261028-py3-none-macosx_14_0_arm64.whl` and + `torchaudio-2.11.0.dev20261028-py3-none-macosx_14_0_arm64.whl` that day; a HEAD and a + ranged GET on each answered `403`, so their sizes are not stated here. The stable + `torch` 2.15.0 wheel for the same platform is 138,556,890 bytes on PyPI (a ranged GET, + `content-range: bytes 0-0/138556890`). ComfyUI's `requirements.txt` at `0df1646a` has 33 + entries, pinned only for its own `comfyui-*` packages; the wrapper's has 7 + (`diffusers>=0.25.0`, `peft>=0.18.0`, `accelerate>=0.20.0`, `opencv-python>=4.0.0`, + `scikit-learn>=1.0.0`, `bitsandbytes>=0.49.2`, `matplotlib`). None of the Python + package sizes, the clone sizes, the download times, the run times or the memory peak + are measured. ### Route C: no GPU - **Needs**: any system with a web browser. -- **Hugging Face**: "if you register with HuggingFace, you should be able to run 1-2 PSD +- **Hugging Face, from See-through's README** (read 2026-10-13 and again 2026-10-29, the + same words): "if you register with HuggingFace, you should be able to run 1-2 PSD extractions per day (approximately 2-3 mins each, at 1280 resolution)" ([See-through README][st-readme], *Online Demo*). Two runs per character, so with that limit one character can take two days. -- **ModelScope**: "completely free now" (same section), offered for users in mainland - China; whether it needs an account is not stated there. -- **The Space centre-crops**: its own page says "non-square images are center-cropped", - which is why the README says to upload `work/st_input_full.png` rather than the painting. +- **The Space itself, read 2026-10-29** (`24yearsold/see-through-demo` at + `049a1ab0`, last modified 2026-10-05; Gradio 6.12.0; hardware `zero-a10g` per the Hub's + API, "Running on Zero" on the page). Its `app.py` budgets each run with + `@spaces.GPU(duration=120)`; its slider is `minimum=768, maximum=1280, value=768, + step=64`, with the note "768 recommended for ZeroGPU free tier. Higher resolutions may + timeout or use up your daily quota quickly."; the seed slider runs 0–9999, default 42; + the checkbox "Split arms, eyes and ears" is off by default; the output is + `seethrough_output.psd`, "Download layered PSD". No steps control. No login check in + the code: the quota is Hugging Face's. +- **Hugging Face's ZeroGPU quota, read 2026-10-29** ([Spaces ZeroGPU][zerogpu], *Usage + Tiers*): "GPU usage is subject to **daily** quotas, per account tier" — Unauthenticated + 2 minutes, Free account 5 minutes, PRO 40 minutes; "Included daily quota resets exactly + 24 hours after your first GPU usage." The default duration per call is 60 s, which this + Space raises to 120 s. So a free account has room for two runs a day if each stays under + 2.5 minutes of GPU; the README's "one or two" follows from that and from See-through's + own "1-2", not from a run. +- **ModelScope, read 2026-10-29**: the studio page is a JavaScript shell whose only text + is its title; its README (the studio's `README.md` through ModelScope's API) is in Chinese, + names the two models `ljsabc/seethroughv0.0.2_layerdiff3d` and + `ljsabc/seethroughv0.0.1_marigold`, licence Apache 2.0, and states no limit and no login. + See-through's README: "completely free now, and supports slightly higher resolution than + the HuggingFace demo", offered for users in mainland China. +- **Crop or pad**: the Space's README says "non-square images are center-cropped"; its + `app.py` labels the image box "Upload image (non-square images will be padded)"; the + wrapper's README says "image is center-padded to square". All three read 2026-10-29, + and which one the Space does was not measured (it would take an upload). The README says + to upload `work/st_input_full.png`, already square, so none of them applies. +- **The split box**: the ComfyUI template `rig-parts` sends sets the wrapper's + `tblr_split` to `true` ("Split symmetric parts (eyes, ears, handwear) into left/right", + the node's own tooltip on the box), and the measured Route A run came back with + `handwear-r`, `handwear-l`, `eyebrow-l`, `eyebrow-r`, `ears-l`, `ears-r` and the eyes + split. That the Space's checkbox is the same switch is read from its label, not measured. - **What a PSD does not carry**: a per-layer depth figure, which the wrapper's `layers.json` has. The parts README says a PSD's stacking order is read as the draw order; one optional proposer feature (turning the head along with `propose @@ -938,25 +1066,26 @@ Where a project states no minimum, this says so rather than filling one in. [wrapper]: https://github.com/jtydhr88/ComfyUI-See-through [core]: https://github.com/rig-c/core#install [authoring]: https://github.com/rig-c/parts/blob/main/docs/AUTHORING.md +[zerogpu]: https://huggingface.co/docs/hub/spaces-zerogpu ## The model files, exactly -Sizes as Hugging Face's and Civitai's APIs reported them on 2026-10-13, read without -downloading. +Sizes as Hugging Face's and Civitai's APIs reported them on 2026-10-13 and again on +2026-10-29 (unchanged), read without downloading. The last column is what the project's +GPU box holds, read on 2026-10-29 with `Get-FileHash -Algorithm SHA256`. -| File | Source | Licence | Size | -| --- | --- | --- | --- | -| `novaAnimeXL_ilV80.safetensors` | [Nova Anime XL "IL v8.0"](https://civitai.com/models/376130) by Crody, Civitai model-version 1854228 | the model's own, on its page | 6,775,799 KB (6.94 GB) | -| `controlnet-openpose-sdxl-1.0.safetensors` | [xinsir/controlnet-openpose-sdxl-1.0](https://huggingface.co/xinsir/controlnet-openpose-sdxl-1.0), `diffusion_pytorch_model.safetensors` | Apache-2.0 | 2,502,139,104 B (2.50 GB) | -| the same name, another public upload | thibaud `OpenPoseXL2.safetensors` | its own | 5,004,167,829 B | -| `4x-AnimeSharp.pth` | [Kim2091/AnimeSharp](https://huggingface.co/Kim2091/AnimeSharp) | CC BY-NC-SA 4.0 | 67,010,245 B (0.07 GB) | -| `layerdifforg/seethroughv0.0.2_layerdiff3d` | [Hugging Face](https://huggingface.co/layerdifforg/seethroughv0.0.2_layerdiff3d) | See-through's | 10.16 GB (sum of repo files) | -| `layerdifforg/seethroughv0.0.1_marigold` | Hugging Face | See-through's | 3.28 GB (sum of repo files) | - -Two different public ControlNets share the file name rig-parts expects. Which upload the -project's own GPU box holds is not recorded, so the README's choice of the xinsir upload is -not measured yet. The upscaler's licence is non-commercial; it is used only to enlarge the -painting. +| File | Source | Licence | Size | On the box | +| --- | --- | --- | --- | --- | +| `novaAnimeXL_ilV80.safetensors` | [Nova Anime XL "IL v8.0"](https://civitai.com/models/376130) by Crody, Civitai model-version 1854228 | the model's own, on its page | 6,775,799 KB = 6,938,418,620 B (6.94 GB) | 6,938,418,620 B, sha256 `4c497bb7…` = Civitai's | +| `controlnet-openpose-sdxl-1.0.safetensors` | [xinsir/controlnet-openpose-sdxl-1.0](https://huggingface.co/xinsir/controlnet-openpose-sdxl-1.0), `diffusion_pytorch_model.safetensors` | Apache-2.0 | 2,502,139,104 B (2.50 GB) | 2,502,139,104 B, sha256 `b8524e55…` = xinsir's | +| the same name, another public upload | thibaud `OpenPoseXL2.safetensors` | its own | 5,004,167,829 B | not this one | +| `4x-AnimeSharp.pth` | [Kim2091/AnimeSharp](https://huggingface.co/Kim2091/AnimeSharp) | CC BY-NC-SA 4.0 | 67,010,245 B (0.07 GB) | 67,010,245 B, sha256 `e7a7de2d…` = Hugging Face's | +| `layerdifforg/seethroughv0.0.2_layerdiff3d` | [Hugging Face](https://huggingface.co/layerdifforg/seethroughv0.0.2_layerdiff3d), repo at `4477e6ce` (2026-09-28) | See-through's | 10,159,649,987 B (10.16 GB, sum of repo files; the UNet alone 8.14 GB) | present, under `models/SeeThrough/`; not hashed | +| `layerdifforg/seethroughv0.0.1_marigold` | Hugging Face, repo at `4f4ffc46` (2026-09-28) | See-through's | 3,276,274,643 B (3.28 GB, sum of repo files) | present; not hashed | + +Two different public ControlNets share the file name rig-parts expects. The box holds +xinsir's, byte for byte, so the README's choice is the measured one. The upscaler's licence +is non-commercial; it is used only to enlarge the painting. ## The inputs, field by field @@ -1148,12 +1277,9 @@ changes is the packages' decision. | Claim | What it needs | | --- | --- | -| Route A end to end: ComfyUI install time and disk, PyTorch size, model download times, `comfy paint` and `comfy seethrough` times and peak VRAM, on a named NVIDIA GPU | a run from a clean ComfyUI install on a named card | -| Which ControlNet upload to recommend | reading which upload the project's box holds (sha256 against xinsir's and thibaud's) | -| Whether the Civitai checkpoint download asks for a login | one download attempt, signed out | -| Route B, all of it | an Apple Silicon Mac, ~23 GB of models, and the two code-read risks checked (`--offload` → `'cuda'`; `bitsandbytes` on macOS) | -| Route C: the Space run, its real quota, the PSD it returns going through *Try it first*; ModelScope | a Hugging Face account and two runs (likely two days at the stated 1–2/day), then the steps with `--full x.psd --head y.psd` | -| The joint model download time by itself | a separate download-only timer; the 2026-10-25 cold detector stage above includes both model downloads and inference | +| Route A from nothing: ComfyUI install time and disk, PyTorch size, model download times; the See-through head run; `--offload` and `--nf4`; the run's own peak memory on an empty card | a run from a clean ComfyUI install on a named card (the two commands themselves, on a box with everything installed, were measured 2026-10-29 above) | +| Route B, all of it | an Apple Silicon Mac, ~23 GB of models, and the two code-read risks checked (`--offload` → `'cuda'`; `bitsandbytes` on macOS); the download list and sizes were read 2026-10-29 | +| Route C: the Space run, its real quota, whether it crops or pads, the PSD it returns going through *Try it first*; ModelScope | a Hugging Face account and two runs (likely two days at the stated quota), then the steps with `--full x.psd --head y.psd` | | *Try it first* on Windows; the page in Safari and Firefox | runs on those systems and browsers (Linux/Chromium was run on 2026-10-25 above) | | `git clone https://github.com/rig-c/getting-started` as written | the guide published (the smoke copied the checkout instead) | | ComfyUI Desktop on macOS (address, custom nodes folder) | reading its docs further and one install | diff --git a/docs/keypoints-body18.svg b/docs/keypoints-body18.svg new file mode 100644 index 0000000..f3ee4b1 --- /dev/null +++ b/docs/keypoints-body18.svg @@ -0,0 +1,62 @@ + + + + +the painting, 832 x 1216 px here; x right, y down from the top-left corner + + + + + + + + + + + + + + + + + + + +nose + +r_eye + +l_eye + +r_ear + +l_ear + +neck + +r_shoulder + +l_shoulder + +r_elbow + +l_elbow + +r_wrist + +l_wrist + +r_hip + +l_hip + +r_knee + +l_knee + +r_ankle + +l_ankle +the character faces you: r_ is the character's right, on the left of the picture + +