ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/studio/backend/core/inference
History
Daniel Han 5c249ebada video: use the quantized FBCache threshold when the transformer is quantized 
apply_step_cache on the video load path omitted quant_active, so a quantized video
transformer (an engaged dense transformer_quant, or a GGUF checkpoint) that also
enabled First-Block-Cache without an explicit threshold used the dense bf16 threshold
(0.08) instead of the higher quantized threshold (0.12) the cache helper documents as
needed for quantized transformers to trigger. The advertised quant plus FBCache path
therefore cached far less than intended. Thread quant_active through exactly as the
image path (diffusion.py) does: an engaged transformer_quant or a GGUF transformer both
count as quant-active here.
2026-07-06 09:31:10 +00:00
..
__init__.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
_html_to_md.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
anthropic_compat.py Tool-call healing (default on) and opt-in nudging for the client-tool passthrough (#6801) 2026-07-03 08:22:42 -07:00
api_monitor.py Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
audio_codecs.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
chat_template_helpers.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
chat_templates.py Studio: bundle Gemma 4 chat templates (E2B/E4B + larger) and auto-apply to unsloth/gemma-4-*-GGUF (#6245) 2026-06-12 05:49:39 -07:00
defaults.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
diffusion.py Studio: reject transformer-only loads for Ideogram 4 and size local fp8 mirrors 2026-07-05 12:08:45 +00:00
diffusion_arch_patches.py Studio diffusion: image workflows (safetensors, image-conditioned, editing) + Images UI redesign (#6769) 2026-07-03 13:45:14 -03:00
diffusion_attention.py Invalidate import caches after a wheel install so the new kernel is importable 2026-07-05 11:48:34 +00:00
diffusion_auto_policy.py Add Ideogram 4 family, structured HunyuanImage exclusion, curated Krea 2 LoRAs 2026-07-04 12:46:24 +00:00
diffusion_cache.py Merge branch 'diffusion-fp16-accum' into diffusion-auto-install 2026-07-05 02:17:17 +00:00
diffusion_compile_cache.py Studio diffusion: image workflows (safetensors, image-conditioned, editing) + Images UI redesign (#6769) 2026-07-03 13:45:14 -03:00
diffusion_controlnet.py Studio diffusion: ControlNet for the Images workflow (diffusers) (#6773) 2026-07-03 15:57:28 -03:00
diffusion_device.py Studio diffusion (Phase 16): route no-GPU loads to the native sd.cpp engine (#6724) 2026-07-01 15:43:56 -03:00
diffusion_eager_patches.py Add grad norm chart, clearer completion state, Windows caption keys, GGUF compute copy 2026-07-04 04:31:04 +00:00
diffusion_engine_router.py Studio diffusion: image workflows (safetensors, image-conditioned, editing) + Images UI redesign (#6769) 2026-07-03 13:45:14 -03:00
diffusion_families.py Studio: reject transformer-only loads for Ideogram 4 and size local fp8 mirrors 2026-07-05 12:08:45 +00:00
diffusion_gguf_compile.py Studio diffusion: image workflows (safetensors, image-conditioned, editing) + Images UI redesign (#6769) 2026-07-03 13:45:14 -03:00
diffusion_ideogram4.py Studio: reject transformer-only loads for Ideogram 4 and size local fp8 mirrors 2026-07-05 12:08:45 +00:00
diffusion_inference_info.py Advertise per-family footprints and surface Auto badges for resolved controls 2026-07-04 07:47:01 +00:00
diffusion_krea2.py Fail clearly when a local Krea 2 dir lacks model_index.json 2026-07-04 04:31:09 +00:00
diffusion_lora.py Add Ideogram 4 family, structured HunyuanImage exclusion, curated Krea 2 LoRAs 2026-07-04 12:46:24 +00:00
diffusion_memory.py Merge branch 'diffusion-more-families' into video-inference 2026-07-05 05:36:02 +00:00
diffusion_patch_backend.py Studio diffusion: image workflows (safetensors, image-conditioned, editing) + Images UI redesign (#6769) 2026-07-03 13:45:14 -03:00
diffusion_precision.py Studio diffusion (Phase 4): native stable-diffusion.cpp engine for CPU/Mac (#6679) 2026-07-01 15:03:53 -03:00
diffusion_prequant.py Studio diffusion: fix FP8 transformer quant producing noise (per-row scaling) (#6772) 2026-07-03 13:51:26 -03:00
diffusion_speed.py Pass normalized speed mode to fp16 accumulation gate 2026-07-05 00:06:38 +00:00
diffusion_transformer_quant.py Merge diffusion-train-tab-2: qwen dense-quant family deny (black frames, measured) 2026-07-04 08:52:32 +00:00
external_provider.py Studio: harden background consumer loops and streaming paths against silent UI freezes (#6653) 2026-06-26 03:31:33 -07:00
gpu_arbiter.py Video inference engine: LTX-2 family registry, VideoBackend, MP4 gallery 2026-07-04 13:08:43 +00:00
image_gallery.py Studio diffusion (Phase 1): cross-platform device policy, fp16 guard, lock split, validate-before-evict (#6670) 2026-06-30 16:33:47 -03:00
inference.py fix(studio): handle multimodal list content in inference text paths (#4383) (#6480) 2026-06-23 01:26:11 -07:00
key_exchange.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
llama_cpp.py Studio: reserve CUDA context and mmproj/MTP soft overhead in the GGUF fit budget (#6718) 2026-07-03 13:07:30 -03:00
llama_http.py fix(studio/llama_cpp): disable trust_env on the loopback health probe (#6750) (#6752) 2026-06-30 19:09:26 +02:00
llama_keepwarm.py Studio: opt-in OpenAI /v1 model auto-switch and idle keep-warm (#6392) 2026-07-01 06:42:23 -07:00
llama_server_args.py studio: return a clean model id from the OpenAI API instead of the local .gguf path (#6518) 2026-06-26 16:07:53 -03:00
llama_stats.py Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
local_model_resolver.py Studio: opt-in OpenAI /v1 model auto-switch and idle keep-warm (#6392) 2026-07-01 06:42:23 -07:00
mcp_client.py Studio: enable stdio MCP servers on a loopback bind (#6295) 2026-06-15 03:02:32 +01:00
mcp_config_import.py studio: show MCP "Import config" on the add-server form (#6030) 2026-06-11 16:17:22 +01:00
message_content.py fix(studio): handle multimodal list content in inference text paths (#4383) (#6480) 2026-06-23 01:26:11 -07:00
mlx_inference.py Expose runtime context length for hub models (#6154) 2026-06-11 22:13:53 +03:00
model_ids.py studio: list the full local model catalog from /v1/models (#6519) 2026-06-26 20:42:06 -03:00
orchestrator.py Studio: harden background consumer loops and streaming paths against silent UI freezes (#6653) 2026-06-26 03:31:33 -07:00
passthrough_healing.py Tool-call healing (default on) and opt-in nudging for the client-tool passthrough (#6801) 2026-07-03 08:22:42 -07:00
pricing.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
providers.py Studio: Add custom provider option to Connections (#6112) 2026-06-12 13:09:35 +02:00
runtime_context.py Expose runtime context length for hub models (#6154) 2026-06-11 22:13:53 +03:00
safetensors_agentic.py Studio: show tool-call progress for large GGUF tool arguments (#6484) 2026-06-22 05:50:10 -07:00
sd_cpp_args.py Reject sd-cli batch runs and clear stale output targets before a run 2026-07-05 01:49:19 +00:00
sd_cpp_backend.py Merge branch 'diffusion-fp16-accum' into diffusion-auto-install 2026-07-05 04:56:24 +00:00
sd_cpp_engine.py Reject sd-cli batch runs and clear stale output targets before a run 2026-07-05 01:49:19 +00:00
sd_cpp_server.py Studio diffusion: LoRA adapters for the Images workflow (#6771) 2026-07-03 15:42:56 -03:00
tensor_fallback.py studio: deterministic VRAM auto-fit for GGUF (MTP reserve, compute buffer, total-based budget) (#6312) 2026-06-17 03:10:22 -07:00
tool_call_parser.py Fix Gemma 4 GGUF OpenAI API streams (#6476) 2026-06-23 06:13:56 -07:00
tool_loop_controller.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
tools.py Whole-document context for RAG chat attachments (#6693) 2026-06-30 15:55:23 +02:00
video.py video: use the quantized FBCache threshold when the transformer is quantized 2026-07-06 09:31:10 +00:00
video_families.py Refuse single-file loads for the dual-expert Wan A14B and write through the second-DiT view 2026-07-05 00:13:50 +00:00
video_gallery.py Video inference engine: LTX-2 family registry, VideoBackend, MP4 gallery 2026-07-04 13:08:43 +00:00
video_ltx2.py [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-07-04 14:45:58 +00:00
worker.py Generalize transformers tier selection by probing AutoConfig (#6550) 2026-06-22 08:20:06 -07:00