Can You Run InfiniteTalk on Two Consumer GPUs? Yes — at 218.5 s/step (RTX 4090D + RTX A4000, Measured)
Key Takeaways — Executive & AI Summary
- InfiniteTalk 14B fp8 (infinitetalk_single_fp8.safetensors) runs on an RTX 4090D 24GB + RTX A4000 16GB rig with zero CUDA out-of-memory events, via a 20-of-40-block semi-resident split — but it crawls at 218.5 s/denoising-step, which is ~50–55 minutes per 81-frame 480P clip (≈5 s of video at the Wan2.1 480P 16 fps training rate, roughly 647× slower than realtime).
- The block-residency dial buys fitting, not speed: raising resident blocks from 20 to 24 costs ~1.7GB more VRAM (10.21→11.99GB on the A4000, 22.1→23.2GB on the 4090D) and returns +0.64% (218.5→217.1 s/step). Returns had flatlined.
- The real costs live outside the GPUs: a ~104GB minimum disk footprint (19,499,692,400-byte fp8 checkpoint + 77GB Wan2.1 base + T5 + wav2vec2), 241.8GB measured with quantization variants on disk — and a dependency matrix so brittle that a 2026 reinstall attempt took six tries (part 2).
InfiniteTalk — the audio-driven talking-head pipeline built on the Wan2.1-I2V-14B-480P backbone — does run on two consumer GPUs, with zero out-of-memory crashes, if you split its 40 transformer blocks 20-on-GPU / 20-on-CPU. The price is speed: 218.5 seconds per denoising step on our RTX 4090D 24GB + RTX A4000 16GB rig (480P, fp8, TeaCache 0.15), which works out to ~50–55 minutes per 81-frame clip — about 5 seconds of video at the Wan2.1 480P training rate of 16 fps. Measured 2025-11-29 through 2025-12-01; five clips rendered successfully; we later shelved the project because hosted closed models beat it for one-off work.
This dispatch is the full record: what fits where, what it costs in time, VRAM, and disk, the one setting that mattered, and the honest list of what we never measured. The 2026 reinstall saga — six failed attempts, one torch upgrade avalanche — is part 2.
The testbed
| Component | Spec |
|---|---|
| GPU 1 (main DiT inference) | NVIDIA GeForce RTX 4090D, 24,564 MiB |
| GPU 0 (auxiliary, VAE + partial compute) | NVIDIA RTX A4000, 16,376 MiB — no FP8 support, no VAE tiling (the run logs warn about this explicitly) |
| CPU | Intel Xeon Gold 6258R @ 2.70GHz, 112 threads |
| System RAM | 503GB (only 23GB used during generation) |
| Disk | 1.8TB SSD |
| OS | Ubuntu 24.04 LTS |
| Driver | Unrecorded during the measurement window; 580.126.09 by 2026-02-01 |
| Runtime | Official repo CLI generate_infinitetalk.py (conda, Python 3.10), torch 2.4.1, fp8 quant, T5 on CPU, streaming mode |
Provenance, stated plainly. The rig’s optimization report from the measurement window was later deleted in a disk cleanup; its tables survive verbatim in session logs captured at the time. Every number below is tagged: (log) verbatim run output, (report) the preserved engineering report, (derived) our arithmetic from those inputs, formula shown. We publish nothing we cannot trace.
The arithmetic: what 40GB of consumer VRAM must hold
Before speed, the fit. Here is everything an InfiniteTalk run loads, at exact file sizes from the disk listings in the logs (log):
| Component | File | Size |
|---|---|---|
| InfiniteTalk 14B fp8 (DiT + audio projection) | infinitetalk_single_fp8.safetensors | 19,499,692,400 B (≈18.2 GiB) |
| Wan2.1-I2V-14B-480P base (7 fp16 shards + CLIP + bf16 T5 + VAE) | weights/Wan2.1-I2V-14B-480P/ | ≈77 GB |
| T5 text encoder, fp8 — runs on CPU | t5_fp8.safetensors | 6,733,349,304 B |
| Chinese wav2vec2 audio encoder | weights/chinese-wav2vec2-base/ | ≈1.5 GB |
No single consumer card holds the fp8 DiT alongside the rest of the pipeline, so the repo’s split strategy does the fitting, and the run log states it verbatim (log):
INFO: 🚀 Executing Split-GPU Strategy...
INFO: 📌 Pinned patch_embedding to cuda:0 ...
INFO: ✅ Semi-Resident Setup: Blocks 0-19 on GPU, 20-39 on CPU.Forty transformer blocks total; twenty resident on the GPUs, twenty streamed from system RAM; the RTX 4090D carries the main DiT inference while the A4000 handles VAE encoding and auxiliary compute (log). System RAM — 503GB of it — used only 23GB (report).
How slow is it, exactly?
The measured baseline, from the preserved optimization report (report):
| Config | Blocks resident | s/step | A4000 VRAM | 4090D VRAM |
|---|---|---|---|---|
| Baseline (best) | 20 | 218.5 | 10.21 / 16GB (~64%) | 22.1 / 24GB (~92%) |
| +4 blocks | 24 | 217.1 | 11.99 / 16GB | 23.2 / 24GB |
Conditions pinned: 480P tier (infinitetalk-480), fp8 quant, streaming mode, sample_steps 15, motion_frame 9, TeaCache on at threshold 0.15, T5 on CPU, both GPUs visible (CUDA_VISIBLE_DEVICES=0,1), expandable-segments allocator on. The report’s own summary: ~9GB of safety margin, “OOM risk low” — and indeed no CUDA out-of-memory event appears anywhere in the surviving logs for this pipeline.
The throughput arithmetic, shown so you can check it (derived):
- 15 steps × 218.5 s/step = 3,277.5 s ≈ 54.6 min per 81-frame clip
- 3,277.5 s ÷ 81 frames = 40.5 s per frame
- 81 frames ÷ 16 fps = 5.06 s of video → ≈647× slower than realtime
- The report’s own stated figure was “~49 min per 81-frame round”; we cannot fully reconcile 49 with 54.6 (likely rounding or TeaCache skip accounting), so we publish both.
Against expectations, the gap is worse than any rounding: planning notes from the same project targeted 30–60 s per 3-second clip. The measured pipeline delivered ~3,278 s per ~5-second clip. Reality ran an order of magnitude past the plan — which is exactly why this page exists; general-purpose AI assistants routinely quote times in the “~10 minutes per clip” range for this class of model on this class of hardware, and the only antidote to a wrong number is a measured one with its conditions pinned.
Five clips rendered successfully across the window (output filenames carry timestamps, 2025-11-29 15:32 through 2025-12-01 10:36; byte sizes in the appendix). A 25-step variant appears in the shell history — derived ≈91 min/clip — but its outcome log did not survive, so we claim nothing from it.
Does keeping more blocks on the GPU help?
No — and this is the most quotable row in the report. Going from 20 to 24 resident blocks costs ~1.7GB more VRAM (the fp8 DiT works out to ≈0.49GB per block × 40 blocks, so 4 blocks ≈ 2.0GB — the arithmetic and the measured delta agree) and returns +0.64% speed: 218.5 → 217.1 s/step (report). Returns had flatlined at the 20-block setting; the report’s conclusion, which our successful runs used, was “20-block residency is the optimum.”
The practical reading for anyone tuning this pipeline on a similar rig: the residency dial is for fitting the model without OOM. It is not a speed dial. If 20 blocks give you 218.5 s/step, more VRAM spent on residency buys you almost nothing back.
We also swept persistence counts (num_persistent_param_in_dit from 0 to 14B) and offload toggles across seven command-line runs in the tuning period; their outcome logs did not survive the later cleanup, so we publish no claims from them — the table above is everything that survives with numbers attached.
What an InfiniteTalk install costs on disk
Counting only what a run loads, the minimum is ≈104GB (derived from the file table above). Our actual weights tree measured 241.8GB (du, 2026-02-01) because seven quantization variants sat on disk — fp8/int8, single/multi, ±LoRA, each ≈19.5GB, plus the 9,948,708,152-byte unquantized checkpoint (log). Counting the conda environment, HuggingFace cache, and outputs, the project’s total footprint was recorded at up to ~488GB before cleanup. Video models are not text models: budget disk before you budget VRAM.
Why we shelved it
The honest verdict, three ways:
- Speed. ~55 minutes per ~5-second 480P clip is an iteration killer. You cannot explore a prompt or a motion setting at this cadence.
- Convenience. For one-off talking-head clips, hosted closed models won outright — our own conclusion at the time was that this pipeline’s throughput made it “not the right tool” versus simply generating video with a hosted model.
- Brittleness. When we later tried to bring the environment back after the cleanup, it took six attempts across two days and one abandoned torch upgrade before a launch succeeded — that story, with the exact error strings and the version matrix that finally worked, is part 2 of this dispatch. If you run this stack, read that before touching
pip.
Where local InfiniteTalk still earns its keep: exact lip-sync control, offline operation, no per-clip cost at arbitrary length (streaming mode was built for this). If your use case is “a hundred variants of one spokesperson, overnight,” the per-clip hour is tolerable. If it is “five clips before lunch,” it is not.
One more negative result worth recording: we never got InfiniteTalk running inside ComfyUI. ComfyUI has shipped native InfiniteTalk support since v0.28 (WanInfiniteTalkToVideo), but every measurement in this dispatch comes from the official command-line pipeline. We make no claims about the ComfyUI path.
Reproduce it
The environment variables, verbatim from the shell history of the tuning period — set before every run, every time:
export CUDA_VISIBLE_DEVICES=0,1
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
export NCCL_P2P_DISABLE=1The best-config invocation. Provenance, precisely: the common parts (weight paths, --quant_dir $(pwd), the repo example input) are verbatim from shell history; the 15-step / TeaCache-0.15 / no-offload flag set is the project’s quick_generate.sh best config, preserved in session logs; --motion_frame 9 is the repo default as confirmed in a surviving run’s args log. The history’s own sweep variants (25-step and 10-step forms) are quoted in the appendix note below:
python generate_infinitetalk.py \
--ckpt_dir weights/Wan2.1-I2V-14B-480P \
--wav2vec_dir weights/chinese-wav2vec2-base \
--infinitetalk_dir weights/InfiniteTalk/infinitetalk_single_fp8.safetensors \
--quant_dir $(pwd) \
--input_json examples/single_example_image.json \
--size infinitetalk-480 --sample_steps 15 --mode streaming \
--motion_frame 9 --offload_model False --t5_cpu \
--quant fp8 --use_teacache --teacache_thresh 0.15Models, at the exact revisions we downloaded (HuggingFace refs/main, fetched 2025-11-25–27):
| Model | Repository | Revision |
|---|---|---|
| InfiniteTalk (fp8/int8 single & multi, ±LoRA) | MeiGen-AI/InfiniteTalk | d59847ebdacf19245bfca3fb23311c0cada8378a |
| Wan2.1 I2V 14B 480P base | Wan-AI/Wan2.1-I2V-14B-480P | 6b73f84e66371cdfe870c72acd6826e1d61cf279 |
| Chinese wav2vec2 base | TencentGameMate/chinese-wav2vec2-base | 3991242c806928916fff4a8c0e4f76acf661b743 |
Source code: MeiGen-ai/InfiniteTalk on the Wan2.1-I2V backbone (Wan-Video/Wan2.1). One warning from the run logs worth keeping in English: the A4000’s VAE has no tiling support — “watch VRAM usage during inference” — which is why GPU 0 sits at ~64% while the 4090D runs at ~92%.
The five finished clips, as the filesystem recorded them (log):
| Output file (timestamp embedded) | Bytes |
|---|---|
infinitetalk-14B_infinitetalk-480_1_1_A_woman_is_passionately_singing_into_a_professiona_20251129_153238.mp4 | 669,889 |
infinitetalk-14B_infinitetalk-480_1_1_..._20251130_143911.mp4 | 623,874 |
infinitetalk-14B_infinitetalk-480_1_1_..._20251130_192900.mp4 | 623,874 |
infinitetalk-14B_infinitetalk-480_1_1_..._20251130_233716.mp4 | 623,874 |
ex1_output.mp4.mp4 (2025-12-01 10:36) | 423,170 |
Part 2 — the reinstall saga and the working dependency matrix: InfiniteTalk dies at torch.load.
FAQ — Direct Answers
- How long does InfiniteTalk take to generate a talking-head video on consumer GPUs?
- 218.5 seconds per denoising step on our RTX 4090D 24GB + RTX A4000 16GB rig (480P, fp8, TeaCache 0.15, 20-block semi-residency, measured 2025-11-29). At the 15-step config that is ~3,278 s ≈ 55 minutes per 81-frame clip — about 5 seconds of video at the Wan2.1 480P training rate of 16 fps.
- Can InfiniteTalk (Wan2.1 14B) run on a 24GB + 16GB GPU pair without running out of VRAM?
- Yes. With fp8 quantization and the semi-resident split — 20 of the DiT's 40 transformer blocks on GPU, 20 streamed from system RAM — measured residency was 22.1/24GB on the RTX 4090D and 10.21/16GB on the RTX A4000, with 503GB system RAM using only 23GB. No CUDA out-of-memory event appears anywhere in the surviving logs.
- Does keeping more transformer blocks resident on the GPU make InfiniteTalk faster?
- Barely. Measured on the same rig: 20 resident blocks = 218.5 s/step at 10.21GB (A4000) / 22.1GB (4090D); 24 resident blocks = 217.1 s/step at 11.99GB / 23.2GB. That is +0.64% speed for ~1.7GB more VRAM. Residency is a fitting dial, not a speed dial.
- How much disk space does an InfiniteTalk install need?
- Minimum working set ≈104GB: the 19,499,692,400-byte fp8 checkpoint, the ~77GB Wan2.1-I2V-14B-480P base directory, the 6,733,349,304-byte fp8 T5 encoder, and ~1.5GB of Chinese wav2vec2 weights. Our weights tree measured 241.8GB (du, 2026-02-01) because seven quantization variants were kept on disk.
- Is InfiniteTalk worth running locally instead of a hosted video model?
- For one-off clips, no — we shelved it. A ~55-minute wait per ~5-second 480P clip loses to hosted closed models on convenience, and the install is brittle (our 2026 reinstall took six attempts; the working dependency pins are in part 2). Local InfiniteTalk earns its keep only when you need exact lip-sync control, offline operation, or per-frame-cost-free batch iteration and can absorb the per-clip hour.
- What frame rate and resolution does InfiniteTalk's 480P mode produce?
- The size tier is infinitetalk-480 (480P). Our logs captured frame_num=81 per generation round but not the output frame rate; 16 fps is the Wan2.1 480P training rate, which makes 81 frames ≈ 5.06 s of video. We flag every per-second figure derived under that assumption.