ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/studio/backend/core/inference
History
Daniel Han f77fa80007 Add HunyuanVideo-1.5 family and the video quality gate 
HunyuanVideo-1.5 (the 8.3B DiT with Qwen2.5-VL + ByT5 text encoders) loads
through the hunyuanvideo-community Diffusers repacks; tencent's own repo is
the original non-diffusers layout and cannot load as a pipeline, so only the
community 480p/720p t2v repos are trusted. Two pipeline quirks, both verified
against pipeline_hunyuan_video1_5.py in diffusers 0.39, shape the wiring:

- __call__ takes no guidance kwarg: CFG lives on the pipeline's guider
  component (ClassifierFreeGuidance, shipped at scale 6.0). The family gains
  guidance_via_guider and generate() writes the requested scale onto
  pipe.guider instead of passing cfg_kwarg, which the pipeline would reject.
- __call__ has no callback_on_step_end: progress and cancellation fall back
  to a scheduler.step wrapper (one call per denoise step), installed for the
  duration of the call and always restored. Cancellation unwinds the loop by
  raising through the wrapper and surfaces the same cancelled sentinel the
  callback path uses.

The VAE compresses 16x spatial / 4x temporal, so sizes snap to /16 and frame
counts to 4k+1. The transformer declares _repeated_blocks and CacheMixin, so
the regional compile profile and the step cache both apply unchanged.

scripts/video_quality.py is the video accuracy gate, the analogue of
scripts/diffusion_quality.py with the same pure-numpy PSNR/SSIM math so image
and video budgets compare: fixed prompt/seed/shape, one short clip per
candidate against a reference clip, per-frame SSIM/PSNR over sampled frames,
a temporal-consistency deviation (motion-energy series error, catching
flicker SSIM alone misses), black-frame/NaN collapse checks, an audio RMS
silence trip-wire for LTX-2, and wall time + peak VRAM per candidate.
Verdicts map the standing budget: ssim >= 0.75 passes, >= 0.50 warns,
anything lower or any collapse fails. --selftest runs the metric path on
synthetic clips with no GPU or model.
2026-07-04 14:22:10 +00:00
..
__init__.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
_html_to_md.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
anthropic_compat.py Tool-call healing (default on) and opt-in nudging for the client-tool passthrough (#6801) 2026-07-03 08:22:42 -07:00
api_monitor.py Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
audio_codecs.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
chat_template_helpers.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
chat_templates.py Studio: bundle Gemma 4 chat templates (E2B/E4B + larger) and auto-apply to unsloth/gemma-4-*-GGUF (#6245) 2026-06-12 05:49:39 -07:00
defaults.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
diffusion.py Add Ideogram 4 family, structured HunyuanImage exclusion, curated Krea 2 LoRAs 2026-07-04 12:46:24 +00:00
diffusion_arch_patches.py Studio diffusion: image workflows (safetensors, image-conditioned, editing) + Images UI redesign (#6769) 2026-07-03 13:45:14 -03:00
diffusion_attention.py Auto-install optional attention kernels and toggle the step cache per generation 2026-07-04 07:33:07 +00:00
diffusion_auto_policy.py Add Ideogram 4 family, structured HunyuanImage exclusion, curated Krea 2 LoRAs 2026-07-04 12:46:24 +00:00
diffusion_cache.py Auto-install optional attention kernels and toggle the step cache per generation 2026-07-04 07:33:07 +00:00
diffusion_compile_cache.py Studio diffusion: image workflows (safetensors, image-conditioned, editing) + Images UI redesign (#6769) 2026-07-03 13:45:14 -03:00
diffusion_controlnet.py Studio diffusion: ControlNet for the Images workflow (diffusers) (#6773) 2026-07-03 15:57:28 -03:00
diffusion_device.py Studio diffusion (Phase 16): route no-GPU loads to the native sd.cpp engine (#6724) 2026-07-01 15:43:56 -03:00
diffusion_eager_patches.py Add grad norm chart, clearer completion state, Windows caption keys, GGUF compute copy 2026-07-04 04:31:04 +00:00
diffusion_engine_router.py Studio diffusion: image workflows (safetensors, image-conditioned, editing) + Images UI redesign (#6769) 2026-07-03 13:45:14 -03:00
diffusion_families.py Add Ideogram 4 family, structured HunyuanImage exclusion, curated Krea 2 LoRAs 2026-07-04 12:46:24 +00:00
diffusion_gguf_compile.py Studio diffusion: image workflows (safetensors, image-conditioned, editing) + Images UI redesign (#6769) 2026-07-03 13:45:14 -03:00
diffusion_inference_info.py Advertise per-family footprints and surface Auto badges for resolved controls 2026-07-04 07:47:01 +00:00
diffusion_krea2.py Fail clearly when a local Krea 2 dir lacks model_index.json 2026-07-04 04:31:09 +00:00
diffusion_lora.py Add Ideogram 4 family, structured HunyuanImage exclusion, curated Krea 2 LoRAs 2026-07-04 12:46:24 +00:00
diffusion_memory.py Video inference engine: LTX-2 family registry, VideoBackend, MP4 gallery 2026-07-04 13:08:43 +00:00
diffusion_patch_backend.py Studio diffusion: image workflows (safetensors, image-conditioned, editing) + Images UI redesign (#6769) 2026-07-03 13:45:14 -03:00
diffusion_precision.py Studio diffusion (Phase 4): native stable-diffusion.cpp engine for CPU/Mac (#6679) 2026-07-01 15:03:53 -03:00
diffusion_prequant.py Studio diffusion: fix FP8 transformer quant producing noise (per-row scaling) (#6772) 2026-07-03 13:51:26 -03:00
diffusion_speed.py [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-07-04 07:47:51 +00:00
diffusion_transformer_quant.py Merge diffusion-train-tab-2: qwen dense-quant family deny (black frames, measured) 2026-07-04 08:52:32 +00:00
external_provider.py Studio: harden background consumer loops and streaming paths against silent UI freezes (#6653) 2026-06-26 03:31:33 -07:00
gpu_arbiter.py Video inference engine: LTX-2 family registry, VideoBackend, MP4 gallery 2026-07-04 13:08:43 +00:00
image_gallery.py Studio diffusion (Phase 1): cross-platform device policy, fp16 guard, lock split, validate-before-evict (#6670) 2026-06-30 16:33:47 -03:00
inference.py fix(studio): handle multimodal list content in inference text paths (#4383) (#6480) 2026-06-23 01:26:11 -07:00
key_exchange.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
llama_cpp.py Studio: reserve CUDA context and mmproj/MTP soft overhead in the GGUF fit budget (#6718) 2026-07-03 13:07:30 -03:00
llama_http.py fix(studio/llama_cpp): disable trust_env on the loopback health probe (#6750) (#6752) 2026-06-30 19:09:26 +02:00
llama_keepwarm.py Studio: opt-in OpenAI /v1 model auto-switch and idle keep-warm (#6392) 2026-07-01 06:42:23 -07:00
llama_server_args.py studio: return a clean model id from the OpenAI API instead of the local .gguf path (#6518) 2026-06-26 16:07:53 -03:00
llama_stats.py Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
local_model_resolver.py Studio: opt-in OpenAI /v1 model auto-switch and idle keep-warm (#6392) 2026-07-01 06:42:23 -07:00
mcp_client.py Studio: enable stdio MCP servers on a loopback bind (#6295) 2026-06-15 03:02:32 +01:00
mcp_config_import.py studio: show MCP "Import config" on the add-server form (#6030) 2026-06-11 16:17:22 +01:00
message_content.py fix(studio): handle multimodal list content in inference text paths (#4383) (#6480) 2026-06-23 01:26:11 -07:00
mlx_inference.py Expose runtime context length for hub models (#6154) 2026-06-11 22:13:53 +03:00
model_ids.py studio: list the full local model catalog from /v1/models (#6519) 2026-06-26 20:42:06 -03:00
orchestrator.py Studio: harden background consumer loops and streaming paths against silent UI freezes (#6653) 2026-06-26 03:31:33 -07:00
passthrough_healing.py Tool-call healing (default on) and opt-in nudging for the client-tool passthrough (#6801) 2026-07-03 08:22:42 -07:00
pricing.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
providers.py Studio: Add custom provider option to Connections (#6112) 2026-06-12 13:09:35 +02:00
runtime_context.py Expose runtime context length for hub models (#6154) 2026-06-11 22:13:53 +03:00
safetensors_agentic.py Studio: show tool-call progress for large GGUF tool arguments (#6484) 2026-06-22 05:50:10 -07:00
sd_cpp_args.py Studio diffusion: LoRA adapters for the Images workflow (#6771) 2026-07-03 15:42:56 -03:00
sd_cpp_backend.py Pin the sd.cpp CPU backend to physical cores 2026-07-04 07:40:34 +00:00
sd_cpp_engine.py Studio diffusion: fix FP8 transformer quant producing noise (per-row scaling) (#6772) 2026-07-03 13:51:26 -03:00
sd_cpp_server.py Studio diffusion: LoRA adapters for the Images workflow (#6771) 2026-07-03 15:42:56 -03:00
tensor_fallback.py studio: deterministic VRAM auto-fit for GGUF (MTP reserve, compute buffer, total-based budget) (#6312) 2026-06-17 03:10:22 -07:00
tool_call_parser.py Fix Gemma 4 GGUF OpenAI API streams (#6476) 2026-06-23 06:13:56 -07:00
tool_loop_controller.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
tools.py Whole-document context for RAG chat attachments (#6693) 2026-06-30 15:55:23 +02:00
video.py Add HunyuanVideo-1.5 family and the video quality gate 2026-07-04 14:22:10 +00:00
video_families.py Add HunyuanVideo-1.5 family and the video quality gate 2026-07-04 14:22:10 +00:00
video_gallery.py Video inference engine: LTX-2 family registry, VideoBackend, MP4 gallery 2026-07-04 13:08:43 +00:00
video_ltx2.py LTX-2.3 checkpoint support: full pipeline assembly 2026-07-04 13:50:44 +00:00
worker.py Generalize transformers tier selection by probing AutoConfig (#6550) 2026-06-22 08:20:06 -07:00