ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/studio/backend/tests
History
oobabooga 72e67ae5a6
 Studio: Add Tensor-Parallel llama.cpp support (#6040) 
* Studio: Add Tensor-Parallel llama.cpp support

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: harden Tensor-Parallel fallback and GPU selection

* Studio: reconcile split-mode extras and harden tensor-split planning

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: reconcile split-mode extras in backend duplicate-load guard

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: preserve inherited non-tensor split modes on reload

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: honor cancellation in tensor fallback, preserve tensor mode on rollback, and don't raise an explicit small context

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: reconcile split-mode in reload check and strip it on tensor downgrade

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Strip --tensor-split alongside --split-mode so inherited ratios don't override the tensor planner

An inherited or stale --tensor-split in llama_extra_args was appended after
Studio's computed --tensor-split and won last in llama.cpp, re-introducing the
asymmetric-GPU OOM tensor mode is meant to prevent. Group -ts/--tensor-split
into the split-mode shadow set so it is stripped on inherit and on the layer
fallback; parse_split_mode_override still keys on the mode value only.

* Drop quantized KV for the tensor attempt and report native max context

Tensor mode aborts on a quantized KV cache, so a user with q8_0/q4_1 etc. who
enabled Tensor Parallelism silently fell back to layer split. Clear the cache
type (and strip inherited/explicit --cache-type) for the tensor attempt only;
the layer fallback re-runs with tensor off and keeps the user's choice.

Also report max_available_ctx from the native context, not an explicit small
-c, so the context slider no longer warns too early in tensor mode.

* Reconcile inherited split-mode extras in the already-loaded check

When a same-model load omitted llama_extra_args, the tensor comparison resolved
the raw (None) request and treated an inherited --split-mode tensor server as a
mismatch, forcing a needless reload. Compare using the stored extras stripped
the same way the reload strips them.

* Pass tensor_parallel through compare-mode loads

The generalized compare path loaded each GGUF without tensor_parallel, so
compare ran layer split even with the toggle on and left the settings sheet
stale. Send the toggle and hydrate the loaded state from the response, matching
the main chat and recipe load paths.

* Add --tensor-parallel flag to unsloth studio run

The headless one-liner could only reach tensor mode by passing --split-mode
tensor as a raw llama.cpp extra. Add a first-class --tensor-parallel/
--no-tensor-parallel option that sets the tensor_parallel field on the
/api/inference/load payload, forwarded through the studio-venv re-exec like the
other polarity flags. Matches the web UI toggle and the API field.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
2026-06-12 04:00:52 -07:00
..
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
conftest.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_amd_apu_unified_memory.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
test_anthropic_cache_ttl.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_citations.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_citations_edge.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_anthropic_code_execution.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_compaction.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_fast_mode_and_refusal.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_anthropic_fast_mode_edge.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_anthropic_messages.py Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls (#5869) 2026-06-12 10:55:26 +02:00
test_anthropic_thinking_translation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_tool_versions.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_web_fetch.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_apple_gpu_sensors.py Studio: show Apple GPU temperature and power in the GPU monitor (macOS) (#6187) 2026-06-12 01:50:45 -07:00
test_audio_token_detection.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_browse_folders_route.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_cache_case_resolution.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_cached_gguf_routes.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_chat_history_routes.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
test_chat_history_storage.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_cleanup_cancelled_checkpoints.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_cloudflare_tunnel.py Studio: reword the Cloudflare line when the public probe fails (#6217) 2026-06-11 20:07:51 -07:00
test_context_overflow_truncation.py Studio: report the real llama-server context window and add an opt-in overflow policy for OpenAI-compatible serving (#6164) 2026-06-11 07:49:55 -07:00
test_cpu_threads.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_data_recipe_github_progress.py Studio: add github_repo seed reader and GitHub Support Bot recipe (#5169) 2026-04-24 12:02:03 -07:00
test_data_recipe_seed.py fix(seed): disable remote code execution in seed inspect dataset loads (#4275) 2026-03-13 19:37:43 +04:00
test_datacenter_gpu_tuning.py Studio: tune llama.cpp env for data-center GPUs (#6098) 2026-06-12 02:39:01 -07:00
test_dataset_upload_limits.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
test_desktop_auth.py Studio: in-app Update llama.cpp button to install the latest prebuilt (#6097) 2026-06-10 10:04:26 -07:00
test_detect_mmproj_file.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_export_absolute_paths.py fix: respect absolute export paths to prevent cross-drive copy failures (WinError 112) (#6088) 2026-06-12 12:52:57 +02:00
test_export_log_cursor.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_external_provider_proxy_env.py Studio: ignore unsupported env proxy during Studio startup (#6102) 2026-06-11 05:13:27 -07:00
test_external_provider_usage_chunk.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_frontend_resolution.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_gemini_provider.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_gguf_completion_usage.py Studio: improve OpenAI- and Anthropic-compatible API spec compliance (#6010) 2026-06-09 17:13:25 +02:00
test_gguf_metadata.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_gguf_reload_inheritance.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_gguf_route_cursor_reset.py Improve local chat tool call flow (#5962) 2026-06-09 07:28:44 -07:00
test_gguf_routing.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_gpu_selection.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_gpu_selection_sandbox.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_host_defaults.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_index_bootstrap_origin.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_index_bootstrap_origin_extra.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_inference_model_validation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_inference_orchestrator_crash_message.py Studio: fix Gemma-4-12B-it not loading (#6054) 2026-06-10 08:39:07 -07:00
test_install_resolve_prebuilt.py Studio: offer the in-app llama.cpp update for source-build (markerless) installs (#6188) 2026-06-11 02:45:12 -07:00
test_kv_cache_estimation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_lemonade_llamacpp_rocm_bins_mock.py fix: deduplicate lemonade ROCm prebuilt selection log (#6021) 2026-06-11 20:14:22 -07:00
test_llama_cpp_cache_aware_disk_check.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_context_fit.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_freshness.py Studio: drop the on-disk freshness cache after a llama.cpp update (#6234) 2026-06-12 02:43:55 -07:00
test_llama_cpp_load_progress.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_load_progress_live.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_load_progress_matrix.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_max_context_threshold.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_mmproj_fallback.py Studio: fall back to text-only when llama.cpp is too old for a model's vision projector (#6075) 2026-06-10 08:58:39 -07:00
test_llama_cpp_mtp_detection.py Studio: surface the llama.cpp update affordance when MTP is disabled (#6192) 2026-06-11 06:10:17 -07:00
test_llama_cpp_no_context_shift.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_props_readback.py Studio: report the real llama-server context window and add an opt-in overflow policy for OpenAI-compatible serving (#6164) 2026-06-11 07:49:55 -07:00
test_llama_cpp_start_failure_classification.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_llama_cpp_tool_loop.py Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls (#5869) 2026-06-12 10:55:26 +02:00
test_llama_cpp_update.py Studio: fix llama.cpp update banner offering a downgrade / sticking on mix releases (#6219) 2026-06-11 23:23:14 -03:00
test_llama_cpp_wait_for_health.py Windows/WSL installer: fix winget msstore cert failure, amd-smi DiskPart prompt, and enable AMD GPU (Strix Halo gfx1151) (#5940) 2026-06-10 04:24:49 -07:00
test_llama_cpp_wait_for_vram_settle.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_windows_nvidia_path.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_route.py Studio: offer the in-app llama.cpp update for source-build (markerless) installs (#6188) 2026-06-11 02:45:12 -07:00
test_llama_server_args.py Studio: Add Tensor-Parallel llama.cpp support (#6040) 2026-06-12 04:00:52 -07:00
test_llm_assist_startup_opt_in.py Studio: make Helper LLM startup pre-cache opt in (#6113) 2026-06-09 15:28:34 +02:00
test_log_filter_no_truncation.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_login_rate_limit.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_mcp_config_import.py studio: show MCP "Import config" on the add-server form (#6030) 2026-06-11 16:17:22 +01:00
test_mcp_servers.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_mcp_stdio_improvements.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_mcp_stdio_pr5863.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_middleware.py feat(studio): Hub + Download Manager (#5916) 2026-06-09 04:11:24 -07:00
test_mlx_inference_backend.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_mlx_training_worker_config.py Fix Studio MLX VLM resized image layout (#6019) 2026-06-10 07:55:37 -07:00
test_models_get_model_config_case_resolution.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
test_mtp_drafter_companion.py Studio: support separate-file MTP GGUF drafters (Gemma 4) (#6125) 2026-06-10 08:45:12 -07:00
test_multimodal_document.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_native_context_length.py Expose runtime context length for hub models (#6154) 2026-06-11 22:13:53 +03:00
test_offline_gguf_cache_fallback.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_offline_inference_parent.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_openai_citation_markers.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_openai_citation_markers_edge.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_openai_code_execution.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_openai_compaction.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_openai_container_crud.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_openai_image_generation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_openai_responses_translation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_openai_tool_passthrough.py Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls (#5869) 2026-06-12 10:55:26 +02:00
test_openai_tool_result_fallbacks.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_pricing.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_pricing_edge.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_providers_api.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_pytorch_mirror.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_rag_captioning.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_chunking.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_embed_llama_server.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_embeddings.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_ingestion.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_preview.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_retrieval.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_store.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_recommended_folders_permission.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_responses_api.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_responses_tool_passthrough.py fix(studio): handle empty Responses tool output (#6167) 2026-06-11 17:35:50 +02:00
test_rocm_oom_guard.py Windows/WSL installer: fix winget msstore cert failure, amd-smi DiskPart prompt, and enable AMD GPU (Strix Halo gfx1151) (#5940) 2026-06-10 04:24:49 -07:00
test_safetensors_capability_advertise.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_safetensors_tool_loop.py Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls (#5869) 2026-06-12 10:55:26 +02:00
test_sandbox_tools.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_server_disk_logging.py Windows/WSL installer: fix winget msstore cert failure, amd-smi DiskPart prompt, and enable AMD GPU (Strix Halo gfx1151) (#5940) 2026-06-10 04:24:49 -07:00
test_studio_api.py Studio: Add Tensor-Parallel llama.cpp support (#6040) 2026-06-12 04:00:52 -07:00
test_studio_train_validation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_tensor_parallel.py Studio: Add Tensor-Parallel llama.cpp support (#6040) 2026-06-12 04:00:52 -07:00
test_tool_approvals.py Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls (#5869) 2026-06-12 10:55:26 +02:00
test_tool_call_parser_strict.py Improve local chat tool call flow (#5962) 2026-06-09 07:28:44 -07:00
test_tool_confirm_loop.py Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls (#5869) 2026-06-12 10:55:26 +02:00
test_tool_confirm_stream.py Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls (#5869) 2026-06-12 10:55:26 +02:00
test_tool_loop_controller.py Improve local chat tool call flow (#5962) 2026-06-09 07:28:44 -07:00
test_tool_message_empty_content.py Studio: report the real llama-server context window and add an opt-in overflow policy for OpenAI-compatible serving (#6164) 2026-06-11 07:49:55 -07:00
test_tool_policy_gates.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_tool_policy_state.py unsloth run: add --enable-tools/--disable-tools server-side tool policy (#5277) 2026-05-05 12:45:15 +04:00
test_tool_xml_strip.py Improve local chat tool call flow (#5962) 2026-06-09 07:28:44 -07:00
test_trained_model_scan.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
test_training_history_update.py Studio: Dark theme refactor, right sidebar redesign, and chat UI polish (#5150) 2026-05-07 14:33:31 +04:00
test_training_nan_loss_handling.py fix(studio): surface live step with null loss through the SSE progress stream (#6206) 2026-06-11 07:50:13 -07:00
test_training_progress_stream_nan.py fix(studio): surface live step with null loss through the SSE progress stream (#6206) 2026-06-11 07:50:13 -07:00
test_training_raw_support.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
test_training_worker_flash_attn.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_transformers_version.py Studio: fix Gemma-4-12B-it not loading (#6054) 2026-06-10 08:39:07 -07:00
test_utils.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_vision_cache.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_vram_estimation.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_windows_gpu_detection_mock.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00