Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -320,7 +320,8 @@
"tutorials/video/wan/fun-control",
"tutorials/video/wan/fun-camera",
"tutorials/video/wan/fun-inp",
"tutorials/video/wan/wan-flf"
"tutorials/video/wan/wan-flf",
"tutorials/video/wan/wan-infinitetalk"
]
}
]
Expand Down
123 changes: 123 additions & 0 deletions tutorials/video/wan/wan-infinitetalk.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
---
title: "ComfyUI Wan2.1 InfiniteTalk Workflow Example"
description: "InfiniteTalk is an audio-driven full-body video dubbing model built on Wan2.1, enabling character lip-sync and body motion synchronization with any input audio."
sidebarTitle: "InfiniteTalk"
---

import UpdateReminder from '/snippets/tutorials/update-reminder.mdx'

**Wan2.1 InfiniteTalk** is an open-source audio-driven video generation model built on Wan2.1, developed by [MeiGen-AI](https://github.com/MeiGen-AI/InfiniteTalk). It enables you to generate full-body talking videos from a single reference image and an audio input. The character's mouth movements and body motions are automatically synchronized to match the provided audio.

**Key Features**:
- **Audio-Driven Lip Sync**: Generate natural mouth movements that match the input audio
- **Full-Body Motion**: Preserves identity, background, and camera movement while adding synchronized body motion
- **Dual Mode**: Supports both single-character and multi-character scenarios
- **ComfyUI Native**: Built-in `WanInfiniteTalkToVideo` node, no custom nodes required

**Related Links**:
- [InfiniteTalk GitHub Repository](https://github.com/MeiGen-AI/InfiniteTalk)
- [Wan2.1 Code Repository](https://github.com/Wan-Video/Wan2.1)
- [Wan2.1 Model Repository](https://huggingface.co/Wan-AI)

<Card title="Learn about Subgraph" icon="book-open" href="/interface/features/subgraph">
This workflow uses Subgraph nodes for modular processing. Check out the Subgraph documentation to learn how to customize and extend the workflow.
</Card>

## InfiniteTalk image-to-video workflow

<CardGroup cols={2}>
<Card title="Run in Comfy Cloud" icon="cloud" href="https://cloud.comfy.org/?template=video_wan2_1_infinitetalk&utm_source=docs&utm_medium=referral&utm_campaign=wan2-1-infinitetalk">
Open in Comfy Cloud
</Card>
<Card title="Download Workflow" icon="download" href="https://github.com/Comfy-Org/workflow_templates/blob/main/templates/video_wan2_1_infinitetalk.json">
Download JSON or search "InfiniteTalk" in Template Library
</Card>
</CardGroup>

![Wan2.1 InfiniteTalk Workflow Preview](https://raw.githubusercontent.com/Comfy-Org/workflow_templates/main/templates/video_wan2_1_infinitetalk-1.webp)

<UpdateReminder />

## Model Installation

The following models need to be downloaded and placed in the correct directories:

**diffusion_models**

- [Wan2_1-I2V-14B-480p_fp8_e4m3fn_scaled_KJ.safetensors](https://huggingface.co/Kijai/WanVideo_comfy_fp8_scaled/resolve/main/I2V/Wan2_1-I2V-14B-480p_fp8_e4m3fn_scaled_KJ.safetensors)

**text_encoders**

- [umt5_xxl_fp8_e4m3fn_scaled.safetensors](https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/resolve/main/split_files/text_encoders/umt5_xxl_fp8_e4m3fn_scaled.safetensors)

**model_patches**

- [wan2.1_infiniteTalk_single_fp16.safetensors](https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/resolve/main/split_files/model_patches/wan2.1_infiniteTalk_single_fp16.safetensors): For single-character scenarios
- [wan2.1_infiniteTalk_multi_fp16.safetensors](https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/resolve/main/split_files/model_patches/wan2.1_infiniteTalk_multi_fp16.safetensors): For multi-character scenarios

**audio_encoders**

- [wav2vec2-chinese-base_fp16.safetensors](https://huggingface.co/Kijai/wav2vec2_safetensors/resolve/main/wav2vec2-chinese-base_fp16.safetensors)

**vae**

- [Wan2_1_VAE_bf16.safetensors](https://huggingface.co/Kijai/WanVideo_comfy/resolve/main/Wan2_1_VAE_bf16.safetensors)

**loras**

- [lightx2v_I2V_14B_480p_cfg_step_distill_rank64_bf16.safetensors](https://huggingface.co/Kijai/WanVideo_comfy/resolve/main/Lightx2v/lightx2v_I2V_14B_480p_cfg_step_distill_rank64_bf16.safetensors)

### Model Storage Location

```
📂 ComfyUI/
├── 📂 models/
│ ├── 📂 diffusion_models/
│ │ └── Wan2_1-I2V-14B-480p_fp8_e4m3fn_scaled_KJ.safetensors
│ ├── 📂 text_encoders/
│ │ └── umt5_xxl_fp8_e4m3fn_scaled.safetensors
│ ├── 📂 model_patches/
│ │ ├── wan2.1_infiniteTalk_single_fp16.safetensors
│ │ └── wan2.1_infiniteTalk_multi_fp16.safetensors
│ ├── 📂 audio_encoders/
│ │ └── wav2vec2-chinese-base_fp16.safetensors
│ ├── 📂 vae/
│ │ └── Wan2_1_VAE_bf16.safetensors
│ └── 📂 loras/
│ └── lightx2v_I2V_14B_480p_cfg_step_distill_rank64_bf16.safetensors
```

### Sample Input Files

Download these sample input files and drag them into the corresponding nodes:

<CardGroup cols={2}>
<Card title="Download Sample Image" icon="image" href="https://raw.githubusercontent.com/Comfy-Org/workflow_templates/main/input/two_character_talking.png">
Character reference image
</Card>
<Card title="Download Speaker 1 Audio" icon="music" href="https://raw.githubusercontent.com/Comfy-Org/workflow_templates/main/input/audio_speaker1_woman.mp3">
Audio track for character 1
</Card>
<Card title="Download Speaker 2 Audio" icon="music" href="https://raw.githubusercontent.com/Comfy-Org/workflow_templates/main/input/audio_speaker2_man.mp3">
Audio track for character 2
</Card>
</CardGroup>

## Workflow Steps

1. **Load the input image**: Drag the character reference image to the `Load Image` node. For multi-character scenarios, use the Mask Editor to draw masks for each character.
2. **Load audio tracks**: Connect audio files to the `Load Audio` nodes (one per character).
3. **Load diffusion model**: Ensure the `Load Diffusion Model` node is using `Wan2_1-I2V-14B-480p_fp8_e4m3fn_scaled_KJ.safetensors`.
4. **Load model patches**: Load the appropriate InfiniteTalk patch (`single` or `multi` variant) via the `ModelPatchLoader` node.
5. **Configure InfiniteTalk**: Adjust the `WanInfiniteTalkToVideo` node parameters (video length, motion amount, etc.).
6. **Generate**: Run the workflow. The model will produce a full-body talking video synchronized with the input audio.

### Extending Video Length

Each **Video Extend** group extends the video by approximately 3.24 seconds (81 frames at 25 fps). If your audio is longer, you can:

1. Box-select the "Video Extend" group
2. Press `Ctrl-C` (copy), then `Ctrl-Shift-V` (paste with connections)
3. Connect the `Batch Images` node's `IMAGE` output to the new `WanInfiniteTalkToVideo` node's `previous_frames`
4. Connect the `Batch Images` node's `IMAGE` output to the new `Batch Images` node's `images`
5. Adjust the connection between the previous group's `WanInfiniteTalkToVideo` output and the new group's `VAEDecode` input
Loading