Add Wan2.2-I2V-A14B image-to-video support to the video backend
Wan-AI/Wan2.2-I2V-A14B-Diffusers is the image-to-video sibling of the already supported T2V-A14B: the same dual-expert WanTransformer3DModel pair (boundary_ratio 0.9 in the pipeline config) behind WanImageToVideoPipeline, conditioning through the VAE latent (no CLIP-vision image encoder in this 2.2 variant). - New wan2.2-i2v-a14b family: image_conditioned flag, card recipe defaults (40 steps, CFG 3.5, 81 frames at 16 fps), the T2V memory table (57.2 GB both experts bf16), fp32-pinned VAE, and a wan2.2-i2v generation-defaults key ahead of the generic wan 50/5.0 entry. - Source-image plumbing: /video/generate takes init_image (base64/data URL); begin_generate 400s synchronously when an image-to-video family has no image or a text-only family is given one, and generate() decodes and resizes the image to the snapped output size before threading it as the pipeline's image kwarg. status() reports image_input so the UI can gate the control. - Trust the official -Diffusers repo for pipeline loads and transfer the measured wan quant recipes: fp8 keeps condition_embedder in bf16 (zero padding-row scale), mxfp8 and nvfp4 stay denied, the UMT5 auto TE quant resolves dense, and the balanced FBCache pin carries over. All tables share the T2V DiT profile. - Tests: family detection/aliases/defaults, the dual-DiT image pipeline load, the image gates on both begin_generate and generate, init_image route pass-through, and the quant exclude/deny/auto entries. GPU-verified on a B200: bf16 resident load (offload none, 72.8 GB peak) animating a conditioning image at 832x480/33f in 28.7 s with first-frame MAE 5.7 vs the source, and an int8 load with both experts quantized (43.1 GB peak, clean output).
This commit is contained in:
parent
63cbf7878e
commit
cffee73135
11 changed files with 281 additions and 2 deletions
|
|
@ -89,6 +89,8 @@ _FBCACHE_QUALITY_THRESHOLDS: dict[str, tuple[float, float]] = {
|
|||
# 0.08 is the least-drift cache-on point, not a compliant one.
|
||||
_FAMILY_FBCACHE_THRESHOLDS: dict[tuple[str, str], tuple[float, float]] = {
|
||||
("wan2.2-t2v-a14b", CQ_BALANCED): (DEFAULT_FBCACHE_THRESHOLD, DEFAULT_FBCACHE_THRESHOLD),
|
||||
# I2V-A14B shares the T2V DiT pair; inherit the balanced pin (same drift profile).
|
||||
("wan2.2-i2v-a14b", CQ_BALANCED): (DEFAULT_FBCACHE_THRESHOLD, DEFAULT_FBCACHE_THRESHOLD),
|
||||
}
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue