ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/studio/backend/core/inference
History
Daniel Han cffee73135 Add Wan2.2-I2V-A14B image-to-video support to the video backend 
Wan-AI/Wan2.2-I2V-A14B-Diffusers is the image-to-video sibling of the already
supported T2V-A14B: the same dual-expert WanTransformer3DModel pair (boundary_ratio
0.9 in the pipeline config) behind WanImageToVideoPipeline, conditioning through the
VAE latent (no CLIP-vision image encoder in this 2.2 variant).

- New wan2.2-i2v-a14b family: image_conditioned flag, card recipe defaults (40 steps,
  CFG 3.5, 81 frames at 16 fps), the T2V memory table (57.2 GB both experts bf16),
  fp32-pinned VAE, and a wan2.2-i2v generation-defaults key ahead of the generic wan
  50/5.0 entry.
- Source-image plumbing: /video/generate takes init_image (base64/data URL);
  begin_generate 400s synchronously when an image-to-video family has no image or a
  text-only family is given one, and generate() decodes and resizes the image to the
  snapped output size before threading it as the pipeline's image kwarg. status()
  reports image_input so the UI can gate the control.
- Trust the official -Diffusers repo for pipeline loads and transfer the measured wan
  quant recipes: fp8 keeps condition_embedder in bf16 (zero padding-row scale), mxfp8
  and nvfp4 stay denied, the UMT5 auto TE quant resolves dense, and the balanced
  FBCache pin carries over. All tables share the T2V DiT profile.
- Tests: family detection/aliases/defaults, the dual-DiT image pipeline load, the
  image gates on both begin_generate and generate, init_image route pass-through, and
  the quant exclude/deny/auto entries.

GPU-verified on a B200: bf16 resident load (offload none, 72.8 GB peak) animating a
conditioning image at 832x480/33f in 28.7 s with first-frame MAE 5.7 vs the source,
and an int8 load with both experts quantized (43.1 GB peak, clean output).
2026-07-17 11:18:01 +00:00
..
__init__.py Studio: stop chat generation on the assistant-turn-end token (fixes Qwen3.5 loop) (#6804) 2026-07-06 10:07:56 -07:00
_html_to_md.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
_vulkan_probe.py Studio: add Vulkan llama.cpp support (#5819) 2026-07-09 03:39:48 -07:00
anthropic_compat.py Studio: stream reasoning tokens in the tool-loop generator (fixes DeepSeek thinking not streaming with a pill on) (#6947) 2026-07-07 19:50:40 -03:00
api_monitor.py Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
audio_codecs.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
chat_eos.py Studio: stop chat generation on the assistant-turn-end token (fixes Qwen3.5 loop) (#6804) 2026-07-06 10:07:56 -07:00
chat_template_helpers.py Studio: render thinking blocks for safetensors inference with prefilled <think> templates (#6816) 2026-07-08 08:14:03 -07:00
chat_templates.py Studio: bundle Gemma 4 chat templates (E2B/E4B + larger) and auto-apply to unsloth/gemma-4-*-GGUF (#6245) 2026-06-12 05:49:39 -07:00
defaults.py Add DeepSeek-V4-Flash-GGUF to Studio with none/high/max reasoning (#6908) 2026-07-07 06:13:43 -07:00
diffusion.py Add the Lumina Image 2.0 family to the image catalog 2026-07-17 11:15:42 +00:00
diffusion_arch_patches.py Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
diffusion_attention.py Tighten fault-path comments added by the video/diffusion hardening pass 2026-07-13 17:33:01 +00:00
diffusion_auto_policy.py Add the Lumina Image 2.0 family to the image catalog 2026-07-17 11:15:42 +00:00
diffusion_cache.py Add Wan2.2-I2V-A14B image-to-video support to the video backend 2026-07-17 11:18:01 +00:00
diffusion_cfg_parallel.py Tighten fault-path comments added by the video/diffusion hardening pass 2026-07-13 17:33:01 +00:00
diffusion_compile_cache.py Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
diffusion_controlnet.py Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
diffusion_device.py Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
diffusion_eager_patches.py Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
diffusion_engine_router.py Tighten comments and docstrings added by the image-generation fixes 2026-07-13 05:29:09 +00:00
diffusion_families.py Add the Lumina Image 2.0 family to the image catalog 2026-07-17 11:15:42 +00:00
diffusion_gguf_compile.py Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
diffusion_ideogram4.py Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
diffusion_inference_info.py Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
diffusion_krea2.py Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
diffusion_lora.py Support LoRA adapters on torchao int8/fp8 quantized image pipelines 2026-07-17 09:37:39 +00:00
diffusion_memory.py Harden the diffusion memory plan against transient free-VRAM undercounts 2026-07-17 08:46:24 +00:00
diffusion_patch_backend.py Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
diffusion_precision.py Add Wan2.2-I2V-A14B image-to-video support to the video backend 2026-07-17 11:18:01 +00:00
diffusion_prequant.py Resolve pre-quantized checkpoints per base variant 2026-07-17 11:15:29 +00:00
diffusion_speed.py Reflow verbose diffusion comments to fewer lines 2026-07-13 03:30:48 +00:00
diffusion_transformer_quant.py Add Wan2.2-I2V-A14B image-to-video support to the video backend 2026-07-17 11:18:01 +00:00
diffusion_vae_quant.py Harden video diffusion cache, CFG-parallel replica, and layerwise-fp8 rollback 2026-07-13 01:30:22 +00:00
external_provider.py Studio: harden background consumer loops and streaming paths against silent UI freezes (#6653) 2026-06-26 03:31:33 -07:00
gpu_arbiter.py Studio: close arbiter load-registration race and surface native progress + local pipeline folders 2026-07-13 06:31:00 +00:00
image_gallery.py Tighten comments and docstrings added by the image-generation fixes 2026-07-13 05:29:09 +00:00
inference.py Studio: render thinking blocks for safetensors inference with prefilled <think> templates (#6816) 2026-07-08 08:14:03 -07:00
key_exchange.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
llama_admission.py Studio: queue local GGUF OpenAI-compatible requests before llama-server (#7047) 2026-07-10 17:05:48 -03:00
llama_cpp.py Studio: queue local GGUF OpenAI-compatible requests before llama-server (#7047) 2026-07-10 17:05:48 -03:00
llama_http.py fix(studio/llama_cpp): disable trust_env on the loopback health probe (#6750) (#6752) 2026-06-30 19:09:26 +02:00
llama_keepwarm.py Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
llama_server_args.py studio: return a clean model id from the OpenAI API instead of the local .gguf path (#6518) 2026-06-26 16:07:53 -03:00
llama_stats.py Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
local_model_resolver.py Studio: opt-in OpenAI /v1 model auto-switch and idle keep-warm (#6392) 2026-07-01 06:42:23 -07:00
mcp_client.py Studio: enable stdio MCP servers on a loopback bind (#6295) 2026-06-15 03:02:32 +01:00
mcp_config_import.py studio: show MCP "Import config" on the add-server form (#6030) 2026-06-11 16:17:22 +01:00
message_content.py fix(studio): handle multimodal list content in inference text paths (#4383) (#6480) 2026-06-23 01:26:11 -07:00
mlx_inference.py Studio: render thinking blocks for safetensors inference with prefilled <think> templates (#6816) 2026-07-08 08:14:03 -07:00
model_ids.py studio: list the full local model catalog from /v1/models (#6519) 2026-06-26 20:42:06 -03:00
orchestrator.py feat(cli): support MLX distributed inference (#6845) 2026-07-08 03:25:39 -07:00
passthrough_healing.py Studio: parse Mistral [TOOL_CALLS] and rehearsal tool-call shapes (#5704) 2026-07-06 18:52:13 -07:00
presence_penalty.py Studio: apply presence_penalty on the safetensors and MLX inference paths (#6923) 2026-07-06 22:24:47 -07:00
pricing.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
providers.py Studio: Add custom provider option to Connections (#6112) 2026-06-12 13:09:35 +02:00
runtime_context.py Expose runtime context length for hub models (#6154) 2026-06-11 22:13:53 +03:00
safetensors_agentic.py Studio chat: tool-call nudging on by default (API stays opt-in) (#6883) 2026-07-06 19:41:19 -07:00
sd_cpp_args.py Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
sd_cpp_backend.py Support LoRA adapters on torchao int8/fp8 quantized image pipelines 2026-07-17 09:37:39 +00:00
sd_cpp_engine.py Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
sd_cpp_server.py Tighten comments across the remaining image stack files 2026-07-12 11:40:05 +00:00
tensor_fallback.py studio: deterministic VRAM auto-fit for GGUF (MTP reserve, compute buffer, total-based budget) (#6312) 2026-06-17 03:10:22 -07:00
tool_call_parser.py Studio chat: tool-call nudging on by default (API stays opt-in) (#6883) 2026-07-06 19:41:19 -07:00
tool_loop_controller.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
tools.py Whole-document context for RAG chat attachments (#6693) 2026-06-30 15:55:23 +02:00
video.py Add Wan2.2-I2V-A14B image-to-video support to the video backend 2026-07-17 11:18:01 +00:00
video_families.py Add Wan2.2-I2V-A14B image-to-video support to the video backend 2026-07-17 11:18:01 +00:00
video_gallery.py Tighten comments and docstrings added by the image-generation fixes 2026-07-13 05:29:09 +00:00
video_ltx2.py Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
worker.py feat(cli): support MLX distributed inference (#6845) 2026-07-08 03:25:39 -07:00