ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/studio/backend/tests
History
Daniel Han c3604d01f7
 Studio: enable MTP for sub-3B Gemma separate-drafter GGUFs (#6191) 
* Studio: enable MTP for sub-3B Gemma separate-drafter GGUFs

The sub-3B auto-drop to ngram-mod was tuned for an embedded draft head
(Qwen), whose per-token cost regresses below 3B. Gemma ships the head as a
separate root mtp-*.gguf drafter, a tiny standalone model that is cheap
enough to win below 3B: B200 Q4_K_XL bench, draft-mtp n=2 vs spec-off,
gemma-4-E2B (2B) = 1.21x (accept ~0.65) while ngram-mod is 1.00x.

Exempt a separate drafter from the sub-3B gate everywhere the threshold is
applied: the resolver (_mtp_too_small), the auto-fit VRAM reserve, the
drafter auto-download decision, and the reload-skip mirror via a
has_separate_drafter flag on _auto_mode_drops_mtp. Embedded sub-3B heads
(Qwen) still drop to ngram-mod. A drafter the binary cannot build (older
prebuilt, or a CUDA kernel limit) still aborts the spawn and the load
retries once without speculative decoding.

Adds the full Qwen3.5 + Gemma-4 (regular and QAT) auto/off/forced resolver
matrix, plus explicit sub-3B exemption tests.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Always compare the separate drafter in the reload-skip mirror

The sub-3B wrapper around the drafter compare could skip it when the drafter
was deleted out from under a running sub-3B server (detected None, stored set),
leaving a stale launch. The resolved-path compare is cheap and already handles
every case, so drop the _auto_mode_drops_mtp guard (and its now-unused imports)
and always compare when the mode can use a drafter and the user does not own
--spec-type. Addresses review feedback on #6191.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-11 05:27:46 -07:00
..
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
conftest.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_amd_apu_unified_memory.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
test_anthropic_cache_ttl.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_citations.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_citations_edge.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_anthropic_code_execution.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_compaction.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_fast_mode_and_refusal.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_anthropic_fast_mode_edge.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_anthropic_messages.py Studio: improve OpenAI- and Anthropic-compatible API spec compliance (#6010) 2026-06-09 17:13:25 +02:00
test_anthropic_thinking_translation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_tool_versions.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_web_fetch.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_audio_token_detection.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_browse_folders_route.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_cache_case_resolution.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_cached_gguf_routes.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_chat_history_routes.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
test_chat_history_storage.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_cleanup_cancelled_checkpoints.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_cpu_threads.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_data_recipe_github_progress.py Studio: add github_repo seed reader and GitHub Support Bot recipe (#5169) 2026-04-24 12:02:03 -07:00
test_data_recipe_seed.py fix(seed): disable remote code execution in seed inspect dataset loads (#4275) 2026-03-13 19:37:43 +04:00
test_dataset_upload_limits.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
test_desktop_auth.py Studio: in-app Update llama.cpp button to install the latest prebuilt (#6097) 2026-06-10 10:04:26 -07:00
test_detect_mmproj_file.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_export_log_cursor.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_external_provider_proxy_env.py Studio: ignore unsupported env proxy during Studio startup (#6102) 2026-06-11 05:13:27 -07:00
test_external_provider_usage_chunk.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_frontend_resolution.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_gemini_provider.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_gguf_completion_usage.py Studio: improve OpenAI- and Anthropic-compatible API spec compliance (#6010) 2026-06-09 17:13:25 +02:00
test_gguf_metadata.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_gguf_reload_inheritance.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_gguf_route_cursor_reset.py Improve local chat tool call flow (#5962) 2026-06-09 07:28:44 -07:00
test_gguf_routing.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_gpu_selection.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_gpu_selection_sandbox.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_host_defaults.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_index_bootstrap_origin.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_index_bootstrap_origin_extra.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_inference_model_validation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_inference_orchestrator_crash_message.py Studio: fix Gemma-4-12B-it not loading (#6054) 2026-06-10 08:39:07 -07:00
test_install_resolve_prebuilt.py Studio: offer the in-app llama.cpp update for source-build (markerless) installs (#6188) 2026-06-11 02:45:12 -07:00
test_kv_cache_estimation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_lemonade_llamacpp_rocm_bins_mock.py Source llama.cpp prebuilts from unslothai/llama.cpp (CUDA, ROCm, macOS) (#5963) 2026-06-10 08:49:57 -07:00
test_llama_cpp_cache_aware_disk_check.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_context_fit.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_freshness.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_load_progress.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_load_progress_live.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_load_progress_matrix.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_max_context_threshold.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_mmproj_fallback.py Studio: fall back to text-only when llama.cpp is too old for a model's vision projector (#6075) 2026-06-10 08:58:39 -07:00
test_llama_cpp_mtp_detection.py Studio: enable MTP for sub-3B Gemma separate-drafter GGUFs (#6191) 2026-06-11 05:27:46 -07:00
test_llama_cpp_no_context_shift.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_start_failure_classification.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_llama_cpp_tool_loop.py Improve local chat tool call flow (#5962) 2026-06-09 07:28:44 -07:00
test_llama_cpp_update.py Studio: offer the in-app llama.cpp update for source-build (markerless) installs (#6188) 2026-06-11 02:45:12 -07:00
test_llama_cpp_wait_for_health.py Windows/WSL installer: fix winget msstore cert failure, amd-smi DiskPart prompt, and enable AMD GPU (Strix Halo gfx1151) (#5940) 2026-06-10 04:24:49 -07:00
test_llama_cpp_wait_for_vram_settle.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_windows_nvidia_path.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_route.py Studio: offer the in-app llama.cpp update for source-build (markerless) installs (#6188) 2026-06-11 02:45:12 -07:00
test_llama_server_args.py Studio: support separate-file MTP GGUF drafters (Gemma 4) (#6125) 2026-06-10 08:45:12 -07:00
test_llm_assist_startup_opt_in.py Studio: make Helper LLM startup pre-cache opt in (#6113) 2026-06-09 15:28:34 +02:00
test_log_filter_no_truncation.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_login_rate_limit.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_mcp_servers.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_mcp_stdio_improvements.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_mcp_stdio_pr5863.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_middleware.py feat(studio): Hub + Download Manager (#5916) 2026-06-09 04:11:24 -07:00
test_mlx_inference_backend.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_mlx_training_worker_config.py Fix Studio MLX VLM resized image layout (#6019) 2026-06-10 07:55:37 -07:00
test_models_get_model_config_case_resolution.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
test_mtp_drafter_companion.py Studio: support separate-file MTP GGUF drafters (Gemma 4) (#6125) 2026-06-10 08:45:12 -07:00
test_multimodal_document.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_native_context_length.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_offline_gguf_cache_fallback.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_offline_inference_parent.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_openai_citation_markers.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_openai_citation_markers_edge.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_openai_code_execution.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_openai_compaction.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_openai_container_crud.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_openai_image_generation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_openai_responses_translation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_openai_tool_passthrough.py Studio: forward preserve_thinking + reasoning_effort on the OpenAI passthrough (#6171) 2026-06-10 22:55:43 -07:00
test_openai_tool_result_fallbacks.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_pricing.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_pricing_edge.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_providers_api.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_pytorch_mirror.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_rag_captioning.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_chunking.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_embed_llama_server.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_embeddings.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_ingestion.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_preview.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_retrieval.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_store.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_recommended_folders_permission.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_responses_api.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_responses_tool_passthrough.py Studio: forward preserve_thinking + reasoning_effort on the OpenAI passthrough (#6171) 2026-06-10 22:55:43 -07:00
test_rocm_oom_guard.py Windows/WSL installer: fix winget msstore cert failure, amd-smi DiskPart prompt, and enable AMD GPU (Strix Halo gfx1151) (#5940) 2026-06-10 04:24:49 -07:00
test_safetensors_capability_advertise.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_safetensors_tool_loop.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_sandbox_tools.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_server_disk_logging.py Windows/WSL installer: fix winget msstore cert failure, amd-smi DiskPart prompt, and enable AMD GPU (Strix Halo gfx1151) (#5940) 2026-06-10 04:24:49 -07:00
test_studio_api.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_studio_train_validation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_tool_call_parser_strict.py Improve local chat tool call flow (#5962) 2026-06-09 07:28:44 -07:00
test_tool_loop_controller.py Improve local chat tool call flow (#5962) 2026-06-09 07:28:44 -07:00
test_tool_policy_gates.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_tool_policy_state.py unsloth run: add --enable-tools/--disable-tools server-side tool policy (#5277) 2026-05-05 12:45:15 +04:00
test_tool_xml_strip.py Improve local chat tool call flow (#5962) 2026-06-09 07:28:44 -07:00
test_trained_model_scan.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
test_training_history_update.py Studio: Dark theme refactor, right sidebar redesign, and chat UI polish (#5150) 2026-05-07 14:33:31 +04:00
test_training_raw_support.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
test_training_worker_flash_attn.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_transformers_version.py Studio: fix Gemma-4-12B-it not loading (#6054) 2026-06-10 08:39:07 -07:00
test_utils.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_vision_cache.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_vram_estimation.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_windows_gpu_detection_mock.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00