Hi, I generate long videos with the MiniMax-H3 Ref2VA model entirely through sd-cli -M vid_gen --ref-video <frames_dir> --ref-video-audio <wav> on Vulkan (AMD RX 6950 XT): the tail of chunk N (last frames + audio, extracted with ffmpeg) becomes the reference of chunk N+1 — the CLI equivalent of hradec's ComfyUI "HR Endless Sampler" node. It works well, thanks for the great H3 support!
Two things would make chaining cheaper/seamless:
1. Reference resolution control
As far as I can read in the source, ref-video frames are resized to a nominal size based on 768 px (aspect kept, capped 768x1344, aligned 32; the source size is kept only if smaller). There is no way to deliberately encode the reference below that nominal size.
ComfyUI's HR Endless Sampler has video_continuation_res (decode -> resize -> re-encode the continuation block) exactly to shrink the reference latent: less ref-attention VRAM, bigger chunks, faster VAE encode (on CPU backends this is significant — on my 16 GB VRAM / 32 GB RAM machine the CPU VAE encode of a 12-frame reference takes ~14.5 min).
Would you consider a --ref-video-res WxH (or a scale factor) that clamps the reference to a user-chosen size? (Going up from a small source is already possible implicitly; the ask is going down on purpose.)
2. Latent tail carry-over (harder, long-term)
Chaining currently re-encodes the reference each chunk (decode -> resize -> encode), so a tiny generation loss sits at every seam and can accumulate over very long chains. ComfyUI-H3-Motion-Context slices the previous chunk's tail in latent space (bit-exact, zero drift).
I understand this may not fit the CLI pipeline (the ref would have to skip the VAE encode stage entirely), but I'd like to put it on the radar — even a "reuse previous output latents as reference" hook would remove seams completely.
Happy to A/B test patches on this machine (Vulkan, 16 GB VRAM) — the single-chunk recipe is validated and documented on our side.
Environment: stable-diffusion.cpp master-841 win vulkan-x64, Windows 11, AMD RX 6950 XT 16 GB (RDNA2), model minimax_h3_ref2va_pruned-Q4_K_M.gguf.
Hi, I generate long videos with the MiniMax-H3 Ref2VA model entirely through
sd-cli -M vid_gen --ref-video <frames_dir> --ref-video-audio <wav>on Vulkan (AMD RX 6950 XT): the tail of chunk N (last frames + audio, extracted with ffmpeg) becomes the reference of chunk N+1 — the CLI equivalent of hradec's ComfyUI "HR Endless Sampler" node. It works well, thanks for the great H3 support!Two things would make chaining cheaper/seamless:
1. Reference resolution control
As far as I can read in the source, ref-video frames are resized to a nominal size based on 768 px (aspect kept, capped 768x1344, aligned 32; the source size is kept only if smaller). There is no way to deliberately encode the reference below that nominal size.
ComfyUI's HR Endless Sampler has
video_continuation_res(decode -> resize -> re-encode the continuation block) exactly to shrink the reference latent: less ref-attention VRAM, bigger chunks, faster VAE encode (on CPU backends this is significant — on my 16 GB VRAM / 32 GB RAM machine the CPU VAE encode of a 12-frame reference takes ~14.5 min).Would you consider a
--ref-video-res WxH(or a scale factor) that clamps the reference to a user-chosen size? (Going up from a small source is already possible implicitly; the ask is going down on purpose.)2. Latent tail carry-over (harder, long-term)
Chaining currently re-encodes the reference each chunk (decode -> resize -> encode), so a tiny generation loss sits at every seam and can accumulate over very long chains. ComfyUI-H3-Motion-Context slices the previous chunk's tail in latent space (bit-exact, zero drift).
I understand this may not fit the CLI pipeline (the ref would have to skip the VAE encode stage entirely), but I'd like to put it on the radar — even a "reuse previous output latents as reference" hook would remove seams completely.
Happy to A/B test patches on this machine (Vulkan, 16 GB VRAM) — the single-chunk recipe is validated and documented on our side.
Environment: stable-diffusion.cpp master-841 win vulkan-x64, Windows 11, AMD RX 6950 XT 16 GB (RDNA2), model
minimax_h3_ref2va_pruned-Q4_K_M.gguf.