ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/studio/backend/tests
History
Daniel Han 4beb0a3a5f
 Studio: force-terminate a stuck training stop after a grace period (#7099) 
* Studio: force-terminate a stuck training stop after a grace period

A Stop-with-save only signals the worker and waits for it to save and exit;
force_terminate() was reachable only from the /reset cancel path. On Windows +
ROCm the worker saves the adapter fine but then wedges in post-save GPU/HIP
teardown and never exits, so the run stays in "Stopping..." forever, is_training
stays true, and /reset returns 409.

Add a stop watchdog: when a stop is requested, a daemon escalates to
force_terminate() a short grace after the worker's "complete" (save done), or
after an absolute cap covering a hang during save. After escalation the parent
state is finalized (is_training=False, "Training stopped.") even if the OS never
reaps the wedged worker, so the UI leaves "Stopping..." and a new run can start.
No behavior change on a clean quick exit. Grace and timeout are configurable via
UNSLOTH_STUDIO_TRAINING_STOP_GRACE_S (15) and
UNSLOTH_STUDIO_TRAINING_STOP_TIMEOUT_S (120).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: harden the training stop watchdog per review

Address review feedback so a stop can never corrupt a checkpoint or leave the
run stuck:

- Never force-kill an in-progress save. The absolute cap is now a last-resort
  backstop: raise the save default to 600s and only kill past that long window;
  a not-yet-complete save is not treated as a hang. Cancels have nothing to save,
  so they keep a shorter 120s cap via UNSLOTH_STUDIO_TRAINING_CANCEL_TIMEOUT_S.
  The save vs cancel path is now explicit and the backstop logs a clear warning.
- Always finalize even if force_terminate raises on a wedged child (try/finally),
  so the watchdog never dies leaving the run in "Stopping...".
- Preserve output_dir when the watchdog finalizes so a saved checkpoint is still
  recorded in run history.
- Track the watched process per watchdog: a new run always gets its own watcher,
  and a stale watchdog on an old proc no longer suppresses it.
- Terminate only the captured proc; force_terminate revalidates under the lock
  that it is still the current worker, so it can never kill a fresh run.
- Name the watchdog thread for debuggability.

* studio: tighten training-stop watchdog comments

Comment-only pass: collapse the watchdog docstrings and inline notes to fewer
lines while keeping the rationale. No behavior change.

* Studio: make the stop watchdog safe against concurrent runs and the pump

Target-scope the escalation finalize so a stale watchdog can never clobber a
run that replaced its worker: capture the watched proc and job id, and no-op
the finalize (handle, progress, and DB) when a new run has already taken over.
Honor a later cancel by tightening an in-flight save watchdog to the shorter
cancel cap. Serialize the DB helpers on the lock so the watchdog and pump can
no longer double-create, double-finalize, or corrupt the metric buffer when a
force-terminate hands off to a still-finalizing pump.

Add regression tests: finalize no-ops when superseded, finalize runs for its
own worker, a later cancel tightens the cap, finalize is single-winner under
concurrency, finalize honors expected_job_id, and concurrent flushes claim
each metric exactly once.

* Studio: close the remaining stop-watchdog vs start/pump races

Guard the escalation finalize by the watched job id in addition to the proc:
start_training sets current_job_id before it installs the new _proc, so a stale
watchdog entering during that startup window still sees the old dead handle and
was not caught by the proc-only guard. Capture the job id when the watchdog
starts and require it to still match before touching state.

Snapshot the run id and final progress under the finalize lock and thread them
through the flush and finish_run calls, so a new run that starts between the
finalize claim and the DB writes cannot be flushed or marked stopped under the
old run's finalizer.

Publish _db_run_created only after create_run commits, gated by a dedicated
in-progress flag, so a concurrent finalize can no longer run finish_run against
a not-yet-inserted row and leave the run stuck as running.

Add regression tests for the startup-window job-id guard, run-id pinned flush,
snapshot-based finalize across a new run, and create-not-published-before-insert.

* Studio: finalize a force-stopped run by its captured id

If a new run starts in the gap after the watchdog clears _proc and marks the
backend idle, current_job_id changes, so the previous expected_job_id guard made
the finalize skip and left the stopped run recorded as running. Capture the run
id, metrics, and final progress under the lock (where current_job_id is still the
watched run) and finalize by that captured id via _finish_stopped_run: finish_run
is an idempotent UPDATE and insert_metrics_batch upserts, so a concurrent pump
finalize of the same run is harmless and a newly started run is never touched.

Add a test that the watched run is finalized by id with its buffered metrics, and
update the escalation tests to assert finalize goes through _finish_stopped_run.

* Studio: keep force-stop finalization retryable and unclaimed until the row exists

Only claim _run_finalized in the escalation when the DB row already exists; if an
early create failed and the pump is retrying it, claiming would make the pump's
later finalize no-op and strand the row as running, so leave the finalize to that
create-then-finalize path.

On a DB error in _finish_stopped_run (e.g. a transient SQLite lock), unclaim the
finalize and requeue the drained metrics when the run is still current, so the
pump or a later retry can still record the run stopped instead of leaving history
with an active run and lost metrics. A superseded run's state is never touched.

Add tests: no claim before the row exists, requeue+unclaim on a DB error, and a
superseded run left untouched on error.

* Studio: tighten stop-watchdog comments

Reduce the wording of the docstrings and inline comments added by this PR without
dropping any of the concurrency invariants (dual proc/job-id supersession guard,
finalize-by-captured-id, publish-after-commit, snapshot-under-lock, unclaim and
requeue on error). Comments and docstrings only; no code change.

* Studio: record the stopped run's DB state before dropping _proc

A wedged worker still reports alive, so the pump never reaches its own finalize
and bails on its _proc-is-None guard once the escalation drops the handle. So the
watchdog is the sole finalizer: record the terminal DB state (create the row if a
start-time create failed, then finish by captured id) BEFORE dropping _proc. While
the handle is held is_training_active() stays true, so no new run can start and
current_job_id stays the watched run for the write; _proc is dropped last, guarded
on target_proc so a run that did replace the worker keeps its handle.

_finish_stopped_run retries a transient DB error a few times (the pump can no
longer retry once _proc is gone) and unclaims on final failure only when the run
is still current. Add tests for create-then-finalize, retry-then-unclaim, and not
dropping a new run's handle.

* Studio: job-guard the DB create flags against a racing new run

_ensure_db_run_created publishes backend-wide _db_run_created and
_db_create_in_progress flags. When the watchdog creates a missing row for an
escalated stop, the killed worker lets a new /start proceed mid-create, so the
stale create could publish those flags against the new current_job_id, making the
new run skip inserting its own row (metric/finalize then target a missing run).
Publish the flags only when the captured job id is still current; the row is still
created by id, and the new run owns/creates its own. Also reset
_db_create_in_progress in start_training so a stale claim can't block a new run.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-15 02:46:55 -07:00
..
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
conftest.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_amd_apu_unified_memory.py Studio: graceful recovery ladder when llama-server hard-crashes at startup (#6291) 2026-06-18 09:07:25 -07:00
test_anthropic_cache_ttl.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_citations.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_citations_edge.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_anthropic_code_execution.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_compaction.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_fast_mode_and_refusal.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_anthropic_fast_mode_edge.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_anthropic_messages.py Studio /v1/messages: accept thinking and unknown content blocks (#7017) 2026-07-09 12:20:02 +02:00
test_anthropic_thinking_translation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_tool_versions.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_anthropic_web_fetch.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_api_key_expiry.py Studio: add --secure Cloudflare-only mode and revamp API usage examples (#6300) 2026-06-15 04:18:15 -07:00
test_api_monitor.py Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
test_api_perf_serialization.py fix(studio/llama_cpp): disable trust_env on the loopback health probe (#6750) (#6752) 2026-06-30 19:09:26 +02:00
test_apple_gpu_sensors.py Studio: show Apple GPU temperature and power in the GPU monitor (macOS) (#6187) 2026-06-12 01:50:45 -07:00
test_audio_token_detection.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_bootstrap_timeout.py Studio: auto-shut-down an exposed first-run instance if the admin password is never changed (#6651) 2026-06-26 01:27:27 -07:00
test_browse_denylist.py Studio: expose Windows drive roots in the folder browser (#7082) 2026-07-15 00:24:11 -07:00
test_browse_folders_route.py Studio: expose Windows drive roots in the folder browser (#7082) 2026-07-15 00:24:11 -07:00
test_bypass_permissions.py Studio: persistent stdio MCP sessions so server state survives across tool calls (#7080) 2026-07-14 02:28:43 -07:00
test_cache_case_resolution.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_cached_gguf_routes.py studio: allow updating HF models through UI (#5388) 2026-07-01 01:54:57 +03:00
test_capability_detection.py Harden model fetching (#6391) 2026-06-18 05:39:52 -07:00
test_chat_eos_template_refresh.py Studio: stop chat generation on the assistant-turn-end token (fixes Qwen3.5 loop) (#6804) 2026-07-06 10:07:56 -07:00
test_chat_history_routes.py Studio chat: tool-call nudging on by default (API stays opt-in) (#6883) 2026-07-06 19:41:19 -07:00
test_chat_history_storage.py Sort chat recents by last activity (#6844) 2026-07-07 17:54:32 +01:00
test_chat_load_during_training.py Studio: free chat model VRAM at training start only when the GPU is tight (#6243) 2026-06-18 09:04:01 -07:00
test_chat_only_reason.py Studio macOS: faster startup, MLX self-heal, drop obsolete prebuilt pins (#6494) 2026-06-22 02:20:08 -07:00
test_chat_template_tool_arguments.py Studio: coerce tool_call arguments to dict before chat templating (fixes MLX tool follow-up error) (#6807) 2026-07-06 10:12:22 -07:00
test_chat_turn_end_eos.py Studio: stop chat generation on the assistant-turn-end token (fixes Qwen3.5 loop) (#6804) 2026-07-06 10:07:56 -07:00
test_checkpoints_scan.py (feat) Add project names to studio training runs (#6512) 2026-06-29 16:06:36 +02:00
test_cleanup_cancelled_checkpoints.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_cloudflare_tunnel.py studio: explicit Cloudflare tunnel notice and public-exposure warning at startup (#6515) 2026-06-30 17:47:48 +02:00
test_coding_agents.py feat: detect installed coding agent CLIs in Studio settings (#6909) 2026-07-08 05:26:50 -07:00
test_completion_masking.py Studio: auto-detect completion masking markers, stop silent full-sequence training (#7054) 2026-07-11 05:13:45 -07:00
test_compute_buffer.py Studio: account for DeepSeek-V4 compute buffer in context auto-fit (#6940) 2026-07-07 07:20:31 -07:00
test_consent_gate.py Studio: persistent per-user trust_remote_code approval cache (#6551) 2026-06-22 05:12:49 -07:00
test_context_overflow_truncation.py Studio: report the real llama-server context window and add an opt-in overflow policy for OpenAI-compatible serving (#6164) 2026-06-11 07:49:55 -07:00
test_cpu_threads.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_data_recipe_github_progress.py Studio: add github_repo seed reader and GitHub Support Bot recipe (#5169) 2026-04-24 12:02:03 -07:00
test_data_recipe_pump_resilience.py Studio: harden background consumer loops and streaming paths against silent UI freezes (#6653) 2026-06-26 03:31:33 -07:00
test_data_recipe_seed.py Fix per-block ID collisions and add block cleanup for unstructured uploads (#6944) 2026-07-08 20:03:03 -07:00
test_datacenter_gpu_tuning.py Studio: tune llama.cpp env for data-center GPUs (#6098) 2026-06-12 02:39:01 -07:00
test_dataset_upload_limits.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
test_deepseek_v4_thinking_effort.py Add DeepSeek-V4-Flash-GGUF to Studio with none/high/max reasoning (#6908) 2026-07-07 06:13:43 -07:00
test_default_output_dir_name.py Studio: fix training output dir escaping outputs root for models on another drive (#6293) 2026-06-13 04:06:17 -07:00
test_desktop_auth.py Studio macOS: faster startup, MLX self-heal, drop obsolete prebuilt pins (#6494) 2026-06-22 02:20:08 -07:00
test_detect_mmproj_file.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_embedding_model_security_gate.py Run the malware gate on the RAG embedding model before it loads (#6887) 2026-07-07 04:30:21 -07:00
test_embedding_model_settings.py Studio: customizable RAG embedding model with HF search, settings tab reorganization (#6800) 2026-07-02 05:26:33 -07:00
test_exec_utf8.py Use UTF-8 for Python code-execution subprocess I/O (#6489 class) (#6548) 2026-06-22 09:06:03 -07:00
test_export_absolute_paths.py Studio: scale export GGUF size estimates from the real model size (#6418) 2026-06-18 05:44:17 -07:00
test_export_capability.py Studio: multi-select export formats, portable FP8/INT8, GGUF LoRA, and source parity (#6767) 2026-07-03 08:25:10 -07:00
test_export_imatrix_compressed.py Studio: multi-select export formats, portable FP8/INT8, GGUF LoRA, and source parity (#6767) 2026-07-03 08:25:10 -07:00
test_export_log_cursor.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_export_size_estimate.py Studio: scale export GGUF size estimates from the real model size (#6418) 2026-06-18 05:44:17 -07:00
test_external_provider_proxy_env.py Studio: ignore unsupported env proxy during Studio startup (#6102) 2026-06-11 05:13:27 -07:00
test_external_provider_usage_chunk.py Studio: Add custom provider option to Connections (#6112) 2026-06-12 13:09:35 +02:00
test_file_security.py Harden model fetching (#6391) 2026-06-18 05:39:52 -07:00
test_frontend_resolution.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_gemini_provider.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_gemma4_chat_template_override.py Studio: bundle Gemma 4 chat templates (E2B/E4B + larger) and auto-apply to unsloth/gemma-4-*-GGUF (#6245) 2026-06-12 05:49:39 -07:00
test_gemma_tool_parse_edge_cases.py Studio: parse Mistral [TOOL_CALLS] and rehearsal tool-call shapes (#5704) 2026-07-06 18:52:13 -07:00
test_gguf_completion_usage.py Studio: improve OpenAI- and Anthropic-compatible API spec compliance (#6010) 2026-06-09 17:13:25 +02:00
test_gguf_metadata.py Studio: add 'Load on selection' toggle to configure load options before loading (#6348) 2026-06-17 16:24:10 +01:00
test_gguf_reload_inheritance.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_gguf_route_cursor_reset.py Studio: extend llama.cpp first-token timeout (#5841) 2026-06-12 18:41:38 +02:00
test_gguf_routing.py Fix GGUF variant file selection (#6342) 2026-06-16 12:42:58 +02:00
test_gguf_tool_non_streaming.py Studio: honor stream=false on the GGUF agentic tool path (#6570) (#6618) 2026-06-24 15:37:08 +01:00
test_gguf_xet_fallback_integration.py Studio: Xet-primary model downloads with automatic HTTP fallback on stall (#6372) 2026-06-16 06:17:54 -07:00
test_gpu_selection.py feat: Implementation of the Portuguese (Brazil) language and VRAM/RAM monitor (#6509) 2026-07-02 18:06:39 +02:00
test_gpu_selection_sandbox.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_hf_xet_fallback.py Studio: don't pin transformers before the training worker activates the 5.x sidecar (#6968) 2026-07-08 05:33:16 -07:00
test_host_defaults.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_identity.py CLI: stop unsloth connect from leaking Studio credentials to unverified servers (#6479) 2026-06-21 21:28:38 -07:00
test_index_bootstrap_origin.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_index_bootstrap_origin_extra.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_inference_default_models_non_blocking.py Speed up Studio startup path (#6899) 2026-07-07 18:08:07 -07:00
test_inference_dispatcher_resilience.py Studio: harden OpenAI-compatible GGUF streaming (#6950) 2026-07-09 12:09:08 -03:00
test_inference_model_validation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_inference_orchestrator_crash_message.py Studio: fix Gemma-4-12B-it not loading (#6054) 2026-06-10 08:39:07 -07:00
test_install_resolve_prebuilt.py Studio: detect Windows Intel GPUs via the registry before WMI (#7064) 2026-07-10 17:59:04 -03:00
test_kv_cache_estimation.py studio: deterministic VRAM auto-fit for GGUF (MTP reserve, compute buffer, total-based budget) (#6312) 2026-06-17 03:10:22 -07:00
test_lifespan_shutdown.py Studio: make lifespan shutdown resilient to a dead default executor (#6307) 2026-06-15 22:51:46 -07:00
test_linux_external_media_paths.py Studio: expose Windows drive roots in the folder browser (#7082) 2026-07-15 00:24:11 -07:00
test_llama_admission.py Studio: queue local GGUF OpenAI-compatible requests before llama-server (#7047) 2026-07-10 17:05:48 -03:00
test_llama_cpp_cache_aware_disk_check.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_context_fit.py Studio: cap GGUF context to unified memory on Apple Silicon (#6622) 2026-06-24 04:39:33 -07:00
test_llama_cpp_effective_parallel_slots.py Studio: queue local GGUF OpenAI-compatible requests before llama-server (#7047) 2026-07-10 17:05:48 -03:00
test_llama_cpp_freshness.py Studio: source CPU llama.cpp prebuilts from unslothai/llama.cpp (#6311) 2026-07-08 05:34:59 -07:00
test_llama_cpp_load_progress.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_load_progress_live.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_load_progress_matrix.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_max_context_threshold.py studio: deterministic VRAM auto-fit for GGUF (MTP reserve, compute buffer, total-based budget) (#6312) 2026-06-17 03:10:22 -07:00
test_llama_cpp_mmproj_fallback.py Studio: graceful recovery ladder when llama-server hard-crashes at startup (#6291) 2026-06-18 09:07:25 -07:00
test_llama_cpp_mtp_detection.py Studio: fix Gemma 4 separate-drafter MTP detection and fallback (#6459) 2026-06-20 04:43:37 -07:00
test_llama_cpp_no_context_shift.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_cpp_props_readback.py fix(studio/llama_cpp): disable trust_env on the loopback health probe (#6750) (#6752) 2026-06-30 19:09:26 +02:00
test_llama_cpp_start_failure_classification.py report a complete load once llama-server is healthy (#6790) 2026-07-03 14:13:54 +01:00
test_llama_cpp_stream_cancel.py Studio: make Stop interrupt a llama.cpp generation stalled mid-stream (#7117) 2026-07-14 05:11:56 -07:00
test_llama_cpp_tool_loop.py Studio: stream reasoning tokens in the tool-loop generator (fixes DeepSeek thinking not streaming with a pill on) (#6947) 2026-07-07 19:50:40 -03:00
test_llama_cpp_update.py Studio: pin llama.cpp update apply to the release the banner offered (#7112) 2026-07-14 06:45:16 -07:00
test_llama_cpp_vulkan_probe.py Studio: add Vulkan llama.cpp support (#5819) 2026-07-09 03:39:48 -07:00
test_llama_cpp_wait_for_health.py report a complete load once llama-server is healthy (#6790) 2026-07-03 14:13:54 +01:00
test_llama_cpp_wait_for_vram_settle.py Studio: cross-session backstop to reap a leftover llama-server on startup (#6431) 2026-06-18 05:52:18 -07:00
test_llama_cpp_windows_nvidia_path.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_llama_route.py Studio: fix llama.cpp update toast tag and reload hint (#6493) 2026-06-21 05:40:49 -07:00
test_llama_route_timeouts.py Studio: harden OpenAI-compatible GGUF streaming (#6950) 2026-07-09 12:09:08 -03:00
test_llama_server_args.py studio: deterministic VRAM auto-fit for GGUF (MTP reserve, compute buffer, total-based budget) (#6312) 2026-06-17 03:10:22 -07:00
test_llama_stats.py Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
test_llm_assist_startup_opt_in.py Studio: make Helper LLM startup pre-cache opt in (#6113) 2026-06-09 15:28:34 +02:00
test_load_progress_ready_fraction.py report a complete load once llama-server is healthy (#6790) 2026-07-03 14:13:54 +01:00
test_local_llama_cpp_link.py [Studio] Add --with-llama-cpp-dir installer flag to reuse a local llama.cpp (#6472) 2026-07-02 22:11:20 +01:00
test_local_model_format.py Studio: redesign Select model dropdown to match Hub design (#6364) 2026-06-22 04:33:04 -07:00
test_log_filter_no_truncation.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_logging_middleware.py Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
test_login_rate_limit.py Studio: harden background consumer loops and streaming paths against silent UI freezes (#6653) 2026-06-26 03:31:33 -07:00
test_mcp_config_import.py studio: show MCP "Import config" on the add-server form (#6030) 2026-06-11 16:17:22 +01:00
test_mcp_flatten_result.py Studio: render image content returned by MCP tools (#7081) 2026-07-14 00:30:21 -07:00
test_mcp_servers.py Studio: persistent stdio MCP sessions so server state survives across tool calls (#7080) 2026-07-14 02:28:43 -07:00
test_mcp_stdio_improvements.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_mcp_stdio_pr5863.py Studio: render image content returned by MCP tools (#7081) 2026-07-14 00:30:21 -07:00
test_mcp_stdio_sessions.py Studio: persistent stdio MCP sessions so server state survives across tool calls (#7080) 2026-07-14 02:28:43 -07:00
test_message_content.py fix(studio): handle multimodal list content in inference text paths (#4383) (#6480) 2026-06-23 01:26:11 -07:00
test_middleware.py test: add regression guards for SecurityHeadersMiddleware pure-ASGI (#6424) 2026-06-17 22:53:43 -07:00
test_mlx_inference_backend.py feat(cli): support MLX distributed inference (#6845) 2026-07-08 03:25:39 -07:00
test_mlx_repair.py Studio: exclude mlx-lm 0.31.3 (broke gemma4/qwen3_5 QK-norm load on Apple Silicon) (#6803) 2026-07-06 19:40:06 -07:00
test_mlx_training_worker_config.py Add has_blackwell_gpu to the mlx worker test's wheel_utils stub (#6980) 2026-07-08 07:22:54 -07:00
test_mmproj_vram_accounting.py Studio: account for mmproj VRAM in GGUF fit budget (#5825) (#5849) 2026-06-12 15:04:08 +01:00
test_model_defaults_none_guard.py Studio macOS: faster startup, MLX self-heal, drop obsolete prebuilt pins (#6494) 2026-06-22 02:20:08 -07:00
test_model_ids.py studio: list the full local model catalog from /v1/models (#6519) 2026-06-26 20:42:06 -03:00
test_model_update_robustness.py Studio: fix the permanent GGUF "update available" on no-symlink caches (#7113) 2026-07-14 06:45:46 -07:00
test_models_get_model_config_case_resolution.py Harden model fetching (#6391) 2026-06-18 05:39:52 -07:00
test_mtp_drafter_companion.py Studio: resolve the repo-root MTP drafter after the MTP/ GGUF rename (#7031) 2026-07-09 06:46:00 -07:00
test_mtp_mla_target_ctx.py Studio: gate the MTP target-KV reserve to MTP spec mode, not just MLA (#6449) 2026-06-19 05:51:35 -07:00
test_mtp_vram_budget.py Studio: simplify the inference backend (#6490) 2026-06-21 20:01:09 -03:00
test_multimodal_document.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_namespace_shadow_guard_pr6269.py Studio: self-heal unsloth namespace shadows; clearer failed-load messages (#6532) 2026-06-21 22:43:31 -07:00
test_native_context_length.py Expose runtime context length for hub models (#6154) 2026-06-11 22:13:53 +03:00
test_native_template_trust_remote_code.py studio: tool calling for DeepSeek (R1/V3/V3.1), GLM 4.x, Kimi K2 on safetensors + MLX (#5624) 2026-07-06 15:40:46 -07:00
test_nudge_tool_calls_wiring.py Studio: keep the nudge wiring test collectable without the unsloth stack (#6924) 2026-07-06 21:57:56 -07:00
test_offline_gguf_cache_fallback.py Fix case-variant model matching and GGUF cache reuse in unsloth start (#6900) 2026-07-08 02:32:06 -07:00
test_offline_inference_parent.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_openai_auto_switch.py Studio: hint at Model auto-switch in the OpenAI "No model loaded" 400 (#7006) 2026-07-10 17:48:27 -03:00
test_openai_catalog.py Studio: opt-in OpenAI /v1 model auto-switch and idle keep-warm (#6392) 2026-07-01 06:42:23 -07:00
test_openai_citation_markers.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_openai_citation_markers_edge.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_openai_code_execution.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_openai_compaction.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_openai_container_crud.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_openai_image_generation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_openai_models_path_leak.py studio: return a clean model id from the OpenAI API instead of the local .gguf path (#6518) 2026-06-26 16:07:53 -03:00
test_openai_responses_translation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_openai_tool_passthrough.py Studio: queue local GGUF OpenAI-compatible requests before llama-server (#7047) 2026-07-10 17:05:48 -03:00
test_openai_tool_result_fallbacks.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_orchestrator_unload_cancel.py Studio: serialize the compare-mode dispatcher lifecycle to fix a start race (#6922) 2026-07-06 22:09:41 -07:00
test_passthrough_healing.py Studio: harden OpenAI-compatible GGUF streaming (#6950) 2026-07-09 12:09:08 -03:00
test_personalization_settings.py Unsloth: appearance palettes, customization options, and control restyle (#7077) 2026-07-14 05:12:08 -07:00
test_pr5624_regressions.py studio: tool calling for DeepSeek (R1/V3/V3.1), GLM 4.x, Kimi K2 on safetensors + MLX (#5624) 2026-07-06 15:40:46 -07:00
test_presence_penalty.py Studio: apply presence_penalty on the safetensors and MLX inference paths (#6923) 2026-07-06 22:24:47 -07:00
test_preview.py Studio: shareable per-checkpoint preview links (#6486) 2026-06-24 06:31:53 -07:00
test_preview_followups.py Studio: require signed capability tokens for /p preview links (#6666) 2026-06-25 21:40:48 -07:00
test_preview_routes.py Studio: require signed capability tokens for /p preview links (#6666) 2026-06-25 21:40:48 -07:00
test_preview_sharing_settings.py Studio: require signed capability tokens for /p preview links (#6666) 2026-06-25 21:40:48 -07:00
test_preview_token.py Studio: require signed capability tokens for /p preview links (#6666) 2026-06-25 21:40:48 -07:00
test_pricing.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_pricing_edge.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_process_lifetime.py Reap Studio child processes when the parent dies abnormally (#6425) 2026-06-18 05:51:22 -07:00
test_providers_api.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_pytorch_mirror.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_rag_captioning.py Studio RAG: disable trust_env on loopback llama-server httpx clients (#6775) 2026-07-01 03:58:08 -07:00
test_rag_chunking.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_embed_llama_server.py Studio: customizable RAG embedding model with HF search, settings tab reorganization (#6800) 2026-07-02 05:26:33 -07:00
test_rag_embeddings.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_ingestion.py Whole-document context for RAG chat attachments (#6693) 2026-06-30 15:55:23 +02:00
test_rag_job_events_queue_lifecycle.py Studio: harden background consumer loops and streaming paths against silent UI freezes (#6653) 2026-06-26 03:31:33 -07:00
test_rag_loopback_trust_env.py Studio RAG: disable trust_env on loopback llama-server httpx clients (#6775) 2026-07-01 03:58:08 -07:00
test_rag_ocr_fallback.py Whole-document context for RAG chat attachments (#6693) 2026-06-30 15:55:23 +02:00
test_rag_parsing.py Studio RAG: fix RTL/Indic PDF corruption and dropped DOCX tables (#6780) 2026-07-02 11:38:48 +01:00
test_rag_preview.py Whole-document context for RAG chat attachments (#6693) 2026-06-30 15:55:23 +02:00
test_rag_reconcile_orphaned.py Studio: harden background consumer loops and streaming paths against silent UI freezes (#6653) 2026-06-26 03:31:33 -07:00
test_rag_retrieval.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_store.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
test_rag_whole_document.py Whole-document context for RAG chat attachments (#6693) 2026-06-30 15:55:23 +02:00
test_recommended_folders_has_model.py Studio: redesign Select model dropdown to match Hub design (#6364) 2026-06-22 04:33:04 -07:00
test_recommended_folders_permission.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_resolve_quant_gguf.py Studio: redesign Select model dropdown to match Hub design (#6364) 2026-06-22 04:33:04 -07:00
test_response_template_markers.py Studio: fix the manual response-template markers that never match their rendered templates (#7062) 2026-07-11 21:29:19 -07:00
test_responses_api.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_responses_tool_passthrough.py Studio: parse Mistral [TOOL_CALLS] and rehearsal tool-call shapes (#5704) 2026-07-06 18:52:13 -07:00
test_rocm_oom_guard.py Windows/WSL installer: fix winget msstore cert failure, amd-smi DiskPart prompt, and enable AMD GPU (Strix Halo gfx1151) (#5940) 2026-06-10 04:24:49 -07:00
test_s3_dataset.py feat(studio): implement S3 dataset loading (completes #5951) (#6222) 2026-06-12 14:52:04 +02:00
test_safetensors_capability_advertise.py Add DeepSeek-V4-Flash-GGUF to Studio with none/high/max reasoning (#6908) 2026-07-07 06:13:43 -07:00
test_safetensors_reasoning_stream.py Studio: parse Mistral [TOOL_CALLS] and rehearsal tool-call shapes (#5704) 2026-07-06 18:52:13 -07:00
test_safetensors_tool_loop.py Studio: persistent stdio MCP sessions so server state survives across tool calls (#7080) 2026-07-14 02:28:43 -07:00
test_safetensors_toolcall_wiring.py Studio: persistent stdio MCP sessions so server state survives across tool calls (#7080) 2026-07-14 02:28:43 -07:00
test_sandbox_tools.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_secure_tools_execute.py Keep server-side tools enabled under --secure (#6403) 2026-06-18 05:52:40 -07:00
test_secure_tunnel_gate.py studio: explicit Cloudflare tunnel notice and public-exposure warning at startup (#6515) 2026-06-30 17:47:48 +02:00
test_security_gate_consistency.py Run the malware gate on the RAG embedding model before it loads (#6887) 2026-07-07 04:30:21 -07:00
test_server_disk_logging.py Windows/WSL installer: fix winget msstore cert failure, amd-smi DiskPart prompt, and enable AMD GPU (Strix Halo gfx1151) (#5940) 2026-06-10 04:24:49 -07:00
test_setup_cache_env_hf_home.py Studio: honor custom HF_HOME for model download and load (#6510) 2026-06-22 08:21:59 -07:00
test_sf_client_tools_passthrough.py Studio: client-tool passthrough healing for safetensors and MLX (#6870) 2026-07-06 19:48:36 -07:00
test_slot_offload_fit.py Studio: reserve CUDA context and mmproj/MTP soft overhead in the GGUF fit budget (#6718) 2026-07-03 13:07:30 -03:00
test_sse_streaming_headers.py Studio: simplify the inference backend (#6490) 2026-06-21 20:01:09 -03:00
test_ssm_runtime.py Fix test isolation: restore sys.modules after the pre-import gate test (#6578) 2026-06-22 08:45:34 -07:00
test_startup_banner_loopback.py Fix Windows Studio UTF-8 startup handling (#6614) 2026-07-01 13:47:33 +01:00
test_startup_llama_probe_non_blocking.py Studio macOS: faster startup, MLX self-heal, drop obsolete prebuilt pins (#6494) 2026-06-22 02:20:08 -07:00
test_studio_api.py Studio: Add Tensor-Parallel llama.cpp support (#6040) 2026-06-12 04:00:52 -07:00
test_studio_train_validation.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
test_tensor_parallel.py Studio: reserve CUDA context and mmproj/MTP soft overhead in the GGUF fit budget (#6718) 2026-07-03 13:07:30 -03:00
test_think_prefill_reemit.py Studio: render thinking blocks for safetensors inference with prefilled <think> templates (#6816) 2026-07-08 08:14:03 -07:00
test_thinking_parameter.py feat: add Anthropic-compatible thinking parameter (#5856) 2026-06-15 10:35:33 +01:00
test_tool_approvals.py Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls (#5869) 2026-06-12 10:55:26 +02:00
test_tool_call_parser_strict.py Studio: parse Mistral [TOOL_CALLS] and rehearsal tool-call shapes (#5704) 2026-07-06 18:52:13 -07:00
test_tool_confirm_loop.py Studio: persistent stdio MCP sessions so server state survives across tool calls (#7080) 2026-07-14 02:28:43 -07:00
test_tool_confirm_stream.py Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls (#5869) 2026-06-12 10:55:26 +02:00
test_tool_loop_controller.py Rename chat artifacts copy to canvas (#6298) 2026-06-15 13:38:39 +02:00
test_tool_message_empty_content.py Studio: report the real llama-server context window and add an opt-in overflow policy for OpenAI-compatible serving (#6164) 2026-06-11 07:49:55 -07:00
test_tool_policy_gates.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_tool_policy_state.py unsloth run: add --enable-tools/--disable-tools server-side tool policy (#5277) 2026-05-05 12:45:15 +04:00
test_tool_strip_guard.py Studio: Gemma tool-call streaming follow-ups + nested-XML escape fix (#6476) (#6611) 2026-07-06 10:39:37 -07:00
test_tool_xml_strip.py Studio: parse Mistral [TOOL_CALLS] and rehearsal tool-call shapes (#5704) 2026-07-06 18:52:13 -07:00
test_torchao_select.py Studio: fix flash-attn and torchao install on Blackwell (sm_100+) GPUs (Closes #6961) (#6970) 2026-07-08 06:38:10 -07:00
test_torchao_stub_worker_parity.py Studio: install torchao Windows ROCm stub in the inference worker (#7000) 2026-07-13 19:20:25 -03:00
test_tp_vision_regression.py Studio: restore tensor parallelism for vision/mmproj GGUFs (#6659) 2026-06-27 01:52:18 -07:00
test_trained_model_scan.py Harden model fetching (#6391) 2026-06-18 05:39:52 -07:00
test_training_before_spawn.py Studio: free chat model VRAM at training start only when the GPU is tight (#6243) 2026-06-18 09:04:01 -07:00
test_training_history_update.py Studio: require signed capability tokens for /p preview links (#6666) 2026-06-25 21:40:48 -07:00
test_training_nan_loss_handling.py fix(studio): surface live step with null loss through the SSE progress stream (#6206) 2026-06-11 07:50:13 -07:00
test_training_preflight.py Add MLX backend support for CLI unsloth train (#6709) 2026-07-08 03:25:26 -07:00
test_training_progress_prep_timeout.py Studio: harden background consumer loops and streaming paths against silent UI freezes (#6653) 2026-06-26 03:31:33 -07:00
test_training_progress_stream_nan.py Studio: harden background consumer loops and streaming paths against silent UI freezes (#6653) 2026-06-26 03:31:33 -07:00
test_training_pump_resilience.py Studio: keep the training event pump alive so progress can't silently freeze (#6643) 2026-06-25 05:19:32 -07:00
test_training_raw_support.py MLX Training updates (#5656) 2026-06-14 04:58:50 -07:00
test_training_resume.py feat(studio): implement S3 dataset loading (completes #5951) (#6222) 2026-06-12 14:52:04 +02:00
test_training_runs.py (feat) Add project names to studio training runs (#6512) 2026-06-29 16:06:36 +02:00
test_training_stop_watchdog.py Studio: force-terminate a stuck training stop after a grace period (#7099) 2026-07-15 02:46:55 -07:00
test_training_streaming.py (feat) Add project names to studio training runs (#6512) 2026-06-29 16:06:36 +02:00
test_training_vram_coexistence.py Studio: free chat model VRAM at training start only when the GPU is tight (#6243) 2026-06-18 09:04:01 -07:00
test_training_worker_flash_attn.py Studio: fix flash-attn and torchao install on Blackwell (sm_100+) GPUs (Closes #6961) (#6970) 2026-07-08 06:38:10 -07:00
test_training_worker_import_discipline.py Studio: don't pin transformers before the training worker activates the 5.x sidecar (#6968) 2026-07-08 05:33:16 -07:00
test_training_xet_fallback.py Studio: Xet-primary model downloads with automatic HTTP fallback on stall (#6372) 2026-06-16 06:17:54 -07:00
test_transformers_dtype.py Studio: Fix torch_dtype deprecation warning on startup and ASR load (#6999) 2026-07-13 17:34:25 -03:00
test_transformers_version.py Fix offline checkpoint load/export: "tokenizer is weirdly not loaded" (#6554) 2026-06-25 23:16:53 -07:00
test_trc_approval_cache.py Studio: persistent per-user trust_remote_code approval cache (#6551) 2026-06-22 05:12:49 -07:00
test_utils.py Studio macOS: faster startup, MLX self-heal, drop obsolete prebuilt pins (#6494) 2026-06-22 02:20:08 -07:00
test_validate_gguf_runtime_message.py Studio: show an actionable message when the GGUF runtime is missing (#6327) 2026-06-18 06:00:00 -07:00
test_validate_model_error.py Harden model fetching (#6391) 2026-06-18 05:39:52 -07:00
test_vision_cache.py Fix offline checkpoint load/export: "tokenizer is weirdly not loaded" (#6554) 2026-06-25 23:16:53 -07:00
test_vram_estimation.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
test_windows_external_drive_paths.py Studio: expose Windows drive roots in the folder browser (#7082) 2026-07-15 00:24:11 -07:00
test_windows_gpu_detection_mock.py studio: deterministic VRAM auto-fit for GGUF (MTP reserve, compute buffer, total-based budget) (#6312) 2026-06-17 03:10:22 -07:00
test_worker_activates_correct_transformers.py Studio: don't pin transformers before the training worker activates the 5.x sidecar (#6968) 2026-07-08 05:33:16 -07:00
test_yaml_trust_remote_code_removed.py Harden trust_remote_code consent: scan GGUF-only auto_map and drop pre-set TRC defaults (#6478) 2026-06-22 02:10:35 -07:00