* Studio: honor custom HF_HOME for model download and load
_setup_cache_env always derived HF_HUB_CACHE and HF_XET_CACHE from
XDG_CACHE_HOME / ~/.cache, ignoring a user-set HF_HOME. Because it sets
HF_HUB_CACHE explicitly and that variable takes precedence over HF_HOME
in huggingface_hub, the hub cache was pinned to the standard location: a
model already present under a custom HF_HOME was detected but then
re-downloaded from scratch on load.
Seed HF_HUB_CACHE and HF_XET_CACHE from HF_HOME when the user set it
(HF's own default is $HF_HOME/hub and $HF_HOME/xet), and honor the legacy
HUGGINGFACE_HUB_CACHE alias. The hub download workers call
snapshot_download without a cache_dir for both the Xet and HTTP-fallback
paths, so they follow HF_HUB_CACHE; fixing it here unifies detection and
both transports on one root. Explicit HF_HUB_CACHE / HF_XET_CACHE stay
untouched. Adds tests for the custom-HF_HOME, default, explicit-override,
and legacy-alias cases. Fixes#5182.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: do not crash startup when a custom HF_HOME is not writable
Seeding HF_HUB_CACHE/HF_XET_CACHE from HF_HOME means _setup_cache_env now
mkdir's under a user-controlled path. A non-writable or not-yet-mounted
HF_HOME (typo, offline drive) would raise and crash startup, where the old
code silently fell back. Make the mkdir best-effort; the env var is still
set, so HF reports a clear error at download time. Adds a regression test.
* Studio: strip blank HF_HOME and isolate cache-env tests
Address review: a whitespace-only HF_HOME no longer derives " /hub";
strip it and fall back to the default (matches studio_root). Tests set
UNSLOTH_STUDIO_HOME to a tmp dir so _setup_cache_env's UV/VLLM mkdirs do
not touch the real ~/.unsloth/studio. Adds a whitespace regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: derive training output dir from model basename for local-drive models
A LoRA/QLoRA run started from a model loaded by absolute path (common when
models live on a non-system drive, e.g. G:\modelsAI\...\gemma-4-12B-it) seeded
the default output dir with that full path. resolve_output_dir then raised
"path escapes root ... is not under the studio outputs folder", so training
could not start from a model stored off the system drive.
Add default_run_dir_name(): Hugging Face repo ids keep their namespace
(org/model becomes org_model), while local paths collapse to their final
component so an absolute source path can no longer leak into the output dir.
Use it at the three worker derivation sites and add a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: cap run dir name length and drop redundant resolve
Apply PR review feedback: length-cap the auto-generated output dir component so an unusually long model name stays under the filesystem name limit, and drop the redundant double resolve_output_dir at the embedding site so all three derivation sites match.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: allow absolute save_directory in export paths to prevent cross-drive copy failures
The GGUF export pipeline (and all other export flows) forced every
save_directory through resolve_export_dir(), which always resolved
the path under exports_root() — typically ~/.unsloth/studio/exports/
on the system drive (C: on Windows).
When a user selected an output directory on a different drive (E:):
1. The absolute path was rejected at the Pydantic validator level.
2. Even if it got through, resolve_export_dir would re-resolve it
under C:\Users\.unsloth\studio\exports\.
3. After GGUF conversion completed on E:, the relocation step would
try to move/copy the finished files to C:, causing:
- WinError 17 (cross-drive move failure when shutil.move falls
through to a cross-filesystem copy)
- WinError 112 (disk full on C:)
Fix both layers:
- _validate_save_directory: accept absolute paths (they represent an
explicit user choice of output location).
- resolve_export_dir, resolve_output_dir, resolve_tensorboard_dir:
return absolute paths as-is instead of forcing them under the
default root. Keep the existing safety checks (null bytes, '..'
segments) and fall through to resolve_under_root for relative paths.
Fixes: https://github.com/unslothai/unsloth/issues/6082
* refactor: centralize user path validation into _resolve_user_path helper
Addresses code review feedback: the null-byte, '..', and absolute-path
checks were duplicated across resolve_output_dir, resolve_export_dir,
and resolve_tensorboard_dir. Extract a single _resolve_user_path helper
that all three delegate to.
No behavioral change — pure consolidation.
* fix: address code review — contain destructive cleanup and scope absolute paths
Address all review feedback from gemini-code-assist:
1. P1: destructive subdirectory cleanup (export_gguf)
The flattening loop in export_gguf previously rmtree'd every
subdirectory under abs_save_dir. When targeting an existing user
directory on a different drive (#6082), this could nuke unrelated
subdirectories. Now snapshot existing subdirectories before the
export and only clean up dirs created during this run.
2. P2: keep scan/read endpoints contained
Only resolve_export_dir accepts absolute paths (export is a write
path where user picks location). Reverted resolve_output_dir and
resolve_tensorboard_dir to use resolve_under_root directly — these
are used by scan/read/training endpoints that must stay contained
under their respective roots.
3. Centralization feedback
Removed the _resolve_user_path helper since it's no longer needed
with the narrowed scope. resolve_export_dir has the absolute path
logic inline with a clear docstring.
* fix: skip pre-existing subdirs in GGUF flatten loop and clean stale export intermediates
Two issues caught in code review (chatgpt-codex-connector):
1. The flattening loop moved ALL .gguf files from ALL subdirectories
into abs_save_dir, including pre-existing unrelated user subdirs.
Now skip pre-existing subdirs entirely unless they are known
export-owned intermediates (model/, model_gguf/).
2. After a failed export, known export-owned subdirectories (model/,
model_gguf/) were snapshotted as pre-existing on retry and never
cleaned up. These are now always cleaned up regardless, since they
are known intermediates created by the export pipeline.
* fix: separate write vs read export paths, guard same-dir rmtree
Three issues caught in code review (chatgpt-codex-connector):
1. P1: scan endpoint containment
resolve_export_dir was changed to accept absolute paths, but it's
also used by scan/read endpoints (routes/models.py) that must stay
contained under exports_root(). Split into:
- resolve_export_dir: contained, used by scans
- resolve_export_write_dir: accepts absolute paths, used by export
backend only
2. P1: same-directory rmtree
When a non-PEFT checkpoint's gguf_dir resolves to the same path as
abs_save_dir (user selected the checkpoint's gguf output as their
export directory), shutil.rmtree(gguf_dir) would delete the user's
chosen output directory. Now skip relocation when both paths resolve
to the same location.
3. P1: pre-existing subdir flatten loop
Reverted _EXPORT_OWNED_SUBDIRS logic — 'model/' and 'model_gguf/'
are common directory names in shared model folders and don't prove
export ownership. Now only clean up subdirs that didn't exist before
the export started.
* fix: remove dead _EXPORT_OWNED_SUBDIRS and fix _export_details for absolute paths
Two fixes from review comments:
1. Remove unused _EXPORT_OWNED_SUBDIRS declaration (leftover from
previous iteration that was intentionally removed).
2. _export_details now returns the full absolute path when the export
target is outside exports_root(), instead of truncating to basename.
Users who export to E:\ can now see the full destination path in
the success dialog.
* fix: use unique tmp dir for GGUF intermediates to avoid overwriting user dirs
When exporting to an absolute destination that already contains a
model/ subdirectory (e.g. a shared models folder), the hard-coded
model_save_path would overwrite files in that unrelated directory.
Use _tmp_model_<uuid> as the intermediate path instead, so user
directories are never touched. The tmp dir is created as a new subdir
of abs_save_dir and cleaned up by the flatten loop after GGUF files
are relocated.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GGUF local export paths for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address GGUF export follow-ups for PR #6088
* Clean GGUF temp dirs on export failure for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust export path tests for PR #6088
* Fix/adjust export path review findings for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust home export path handling for PR #6088
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: auto Cloudflare tunnel for 0.0.0.0 launches
Binding Studio to 0.0.0.0 for remote access often leaves the raw
http://<ip>:<port> URL unreachable (https-vs-http, blocked high ports,
closed cloud security groups). On a wildcard bind, auto-start a free
cloudflared quick tunnel and show its https://*.trycloudflare.com URL in
the startup banner:
Secure link access via Cloudflare: https://<random>.trycloudflare.com
- new studio/backend/cloudflare_tunnel.py: find or download+cache the
cloudflared binary (per-OS/arch GitHub release, safe .tgz extract),
start the tunnel, parse the URL, tear it down. Stdlib only; best-effort
and non-fatal throughout (a missing binary or offline box never blocks
or slows startup).
- run_server starts the tunnel for 0.0.0.0 only (skips loopback, api-only
and Colab), prints the line in the banner, and _graceful_shutdown stops
the child so it never orphans.
- --cloudflare/--no-cloudflare flag (default on) on `unsloth studio` and
`unsloth studio run`, forwarded through the re-exec into run_server.
- tests for the helper, the CLI flag forwarding, and the run.py defaults.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio cloudflare: send a User-Agent on the cloudflared download
GitHub's CDN can 403 the default Python-urllib User-Agent on release asset
downloads. Set an explicit UA and pin it with a test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio cloudflare: address review (opt-out for subcommands, tunnel teardown)
- reject --no-cloudflare placed before a subcommand (it would not reach the
subcommand), mirroring the --parallel guard
- register the tunnel before waiting for its URL so a shutdown during the wait
stops cloudflared instead of orphaning it
- tear the server + children down if `unsloth studio run` startup aborts
(health timeout, model-load error, Ctrl+C) before the wait loop
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Adds a self-contained RAG stack to Studio: knowledge bases with chunked indexing, hybrid (dense + lexical) retrieval, and an automatic first-pass context inject into chat. Embeddings run through a local llama-server GGUF backend (default unsloth/bge-small-en-v1.5-GGUF) with a sentence-transformers fallback. The chat tool loop gains a search_knowledge_base tool, a per-turn re-search cap, and source citation, layered on top of the shared ToolLoopController.
Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison.
Trim and tighten code comments and docstrings across studio/ Python. Comment-only: every changed file verified code-identical to main via AST/token comparison.
Raise ruff line-length to 100 and extend the local pre-commit format pipeline (def-signature magic-comma normalization, short multi-line assert collapse, kwarg '=' spacing, blank-line-after-short-import removal, adjacent string-literal / f-string+plain merge, redundant-pass pruning). Every transform re-checks the file AST and is dropped if it would differ; the whole-repo reformat is verified AST-identical per file and idempotent.
* feat: align project sidebar UX with ChatGPT
* feat: align project sidebar UX with ChatGPT
* feat(chat): load stored project list
* feat(chat): add project sidebar workflows
* fix: stabilize project page navigation
* fix: projects chat loading
* fix: show project chat thread
* style: sidebar project spacing and hover clipping
* style: add expandable project chat history and move-to-project submenu
* feat: polish project sidebar
* feat: persist project sandbox paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: only create sandbox project workspace dir
* feat: add optional project workspace deletion from delete dialog
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: stabilize chat projects CI failures
* fix: polish project chat navigation
* Studio: manage chat history with projects
Group chats into projects with a dedicated projects page and route.
Sidebar shows recents with per-row actions and a vertical more-vertical
menu, and the sidebar scrollbar stays hidden so rows never shift on
hover. Includes chat settings and composer refinements.
* Studio: projects sidebar and breadcrumb polish
Sidebar:
- Remove the Compare nav item.
- Widen the sidebar to match the projects layout.
- Replace the scroll-gated bottom fade with a static fade pinned above
the profile box, so it no longer attaches to Recents or lags the
collapse and expand animation.
Topbar breadcrumb (chat-page):
- On a project landing show "Projects" linking to the projects list.
- Inside a project chat show the project name and chat title, with the
project name linking back to that specific project page.
- Drop the divider between the model selector and the breadcrumb.
* Studio: make project workspace delete test cross-platform
test_chat_project_delete_files_removes_workspace rooted the project under
pytest tmp_path, which resolves to /private/tmp on macOS. The workspace
delete guard refuses paths under the system denylist by design, so the
test passed on Linux CI but failed on macOS.
Add a workspace_projects_home fixture that keeps tmp_path on Linux and
Windows (CI unchanged) and falls back to a home subdir only when the temp
root is on the platform denylist. Derive the workspace path from the
created project so it tracks the projects home.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: satisfy import-hoist check for new path re-exports
documents_root and project_workspaces_root are re-exported from
utils.paths but only referenced as __all__ string literals, which the
import-hoist safety net does not count as a use. It flagged the two newly
added re-exports as unused imports and failed Source lint.
Name-load both via a module-level _REEXPORTED tuple so the check sees
them used. No behaviour change; consumers still import them from
utils.paths.
* fix: avoid projects empty-state flash
* fix: batch chat search indexing
* Studio: polish chat sidebar, run settings, and search
- Use the native OS scrollbar for the chat sidebar, Run settings panel, and chat search list instead of a custom scrollbar
- Highlight the active run in the sidebar and keep chat search available during training
- Stop the training log view from replaying when navigating back to a run
- Rename the chat settings panel to Run settings and align its toggle icon and position
- Tighten heading and sidebar letter spacing and lighten the Train and Recents labels
- Match the search dialog corner style across light and dark and drop the stray border
- Make the MCP Servers section header plain text instead of a link
- Remove a stray .orig backup file
* studio/frontend: restore Compare entry point in the sidebar
The chat-projects sidebar redesign dropped the Compare nav item and moved
it to thread-sidebar.tsx, which is not imported or rendered anywhere. That
left no way for a user to start a new model comparison (enterCompare only
fired from the guided tour and the training handoff), and broke the
Compare/Recipes/Export UI smoke test that clicks [data-tour="chat-compare"].
Re-add the Compare NavItem to the New Chat / Search group, carrying
data-tour="chat-compare" and the same new-comparison navigation as before.
* studio/frontend: use Unsloth green for the fallback profile avatar
Switch the initials-avatar background from blue to #14b789 so the sidebar
and edit-profile avatar match the Unsloth brand colour.
* studio/frontend: turn project breadcrumb into a project switcher dropdown
* studio/frontend: stop project card kebab clicks from opening the project
* studio/frontend: hide project switcher outside projects
* studio/frontend: stabilize project switcher loading
* style: project switcher alignment
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
* studio: contain export and dataset paths under their configured roots
resolve_under_root and resolve_dataset_path previously returned absolute
paths unchanged, so an authenticated client could supply
save_directory="/tmp/escape" (or any other absolute path) and have the
exporter drop adapter files anywhere the server user could write. This
turned up during a recent audit pass where an authenticated POST to
/api/export/export/lora with save_directory="/tmp/lora_escape_test"
returned 200 and wrote adapter_model.safetensors, adapter_config.json,
and tokenizer files under /tmp.
The fix is two-layered:
storage_roots.py adds an _assert_contained(resolved, root) helper that
runs after path resolution and rejects any result whose realpath does
not sit under realpath(root). resolve_under_root now rejects '..'
segments and null bytes outright, and only accepts absolute inputs when
they are already inside the configured root (internal call sites that
re-resolve a stored absolute path stay idempotent;
worker.py:resolve_output_dir(output_dir) etc. continue to work).
resolve_dataset_path picks up the same containment rule, scoped to the
three dataset roots.
models/export.py adds field_validator("save_directory", mode="before")
to ExportCommonOptions and ExportGGUFRequest so bad input fails fast at
422 with a clear message rather than a 500 deep inside the resolver.
The validator rejects empty/whitespace, null bytes, control chars,
strings longer than 255 chars, absolute paths, and '..' segments.
routes/export.py:_export_details now returns os.path.relpath(output_path,
exports_root()) so the Export Complete dialog and /api/models/loras no
longer leak the absolute install prefix to the UI; the basename is
used as a last-resort fallback.
Verified end to end:
- POST /api/export/export/lora {"save_directory":"/tmp/foo"} -> 422
"save_directory must be a name or relative path under the export
root; absolute paths are rejected". /tmp/foo is not created.
- "../../etc/escape" -> 422 "may not contain '..' segments".
- save_directory="my_subdir" -> still accepted (400 only because the
test had no checkpoint loaded yet, not because of validation).
- Internal idempotent re-resolve via resolve_export_dir(absolute path
that is already under exports_root) returns the same path unchanged.
* studio/sandbox: harden bash + python tool execution
The sandboxed Bash and Python tool channels in Chat ran with a thin
preexec hook (PR_SET_NO_NEW_PRIVS + RLIMIT_FSIZE only). Bash had a
small word blocklist; Python had an AST safety pass aimed at
signal-tampering and shell-escape primitives. An audit pass showed
several gaps that a tool-calling model could trigger inadvertently:
- bash curl/wget/nc reached AWS IMDSv2 and returned live STS
credentials for the instance role.
- python "import socket; s.connect((169.254.169.254, 80))"
reached the same endpoint regardless of the bash blocklist.
- "cat /etc/passwd" was blocked at the bash side (because "passwd"
is in the blocklist), but "open('/etc/passwd').read()" in Python
happily returned its contents.
- "chr(115)+chr(117)+chr(100)+chr(111)" style dynamic-arg
construction slipped through the AST shell-escape check.
- The supervisor used proc.kill() on timeout, which only signals
the immediate pid; bash-backgrounded children survived. A fork
bomb could spawn for the full 300s timeout window.
- Session work directories under ~/studio_sandbox/<id>/ were
created with default umask (0o755), so any other UID on the host
could enumerate them.
- session_id sanitisation used a one-shot str.replace("..",""),
which is non-iterative and a small footgun.
This commit takes a conservative middle path: the sandbox still
runs as the Studio UID with no namespace tricks where the kernel
disallows them, but every chokepoint is tightened.
_sandbox_preexec now:
- calls os.setsid() so children share a process group; the
supervisor uses os.killpg(SIGKILL) on timeout/cancel so
backgrounded children die with the parent (new _kill_process_tree
helper, wired into _cancel_watcher and both _bash_exec /
_python_exec timeout branches).
- calls os.umask(0o077) so files the child writes default to 0o600.
- applies PR_SET_PDEATHSIG=SIGKILL so an orphaned child dies if
Studio exits.
- best-effort unshare(CLONE_NEWNET) for a private network namespace
(failure is logged and swallowed; defense-in-depth is still in
place via the bash blocklist and the AST checker below).
- sets RLIMIT_NPROC=10000 (tunable via UNSLOTH_STUDIO_SANDBOX_NPROC),
RLIMIT_AS=8GB, RLIMIT_CPU=300, RLIMIT_NOFILE=1024. The 10k NPROC
figure is chosen to sit well above the ~500 LWPs a healthy Studio
+ llama-server combination already uses while still capping a
runaway fork bomb. NPROC counts LWPs per real UID, so a lower
figure (e.g. 256) starves legitimate bash forks
("bash: fork: retry: Resource temporarily unavailable").
_get_workdir:
- rejects session_id that doesn't match [A-Za-z0-9_-]{1,64};
non-matching values bucket into a shared "_invalid" dir.
- chmod 0o700 on both the workdir and on ~/studio_sandbox/ so
other UIDs cannot read another session's contents.
_BLOCKED_COMMANDS_COMMON gains: doas, pkexec, halt, poweroff, curl,
wget, nc, ncat, netcat, socat, ssh, scp, sftp, rsync, eval, source.
The intent is to keep general bash usage working (echo, ls, pipes,
loops, for, head, etc.) while denying the obvious egress and
escalation paths.
The AST checker (_check_signal_escape_patterns) is split into the
existing shell/signal/loop checks plus a new narrow IO denylist:
- Always flag non-literal args to anything in _SHELL_EXEC_FUNCS,
not just _STRING_SHELL_FUNCS. Closes the dynamic-arg bypass.
- Reject calls to socket.create_connection, socket.socket().connect,
urllib.request.urlopen, http.client.HTTP*Connection, requests.*,
httpx.* whose literal host argument is in a cloud-metadata
denylist (169.254.169.254 + 169.254.* + 100.64.*, plus the
GCP/Alibaba/ECS metadata hostnames and IPv6 link-local). Public
hosts (example.com, huggingface.co, ...) still work. Dynamic
hosts cannot be statically blocked; mitigated by the bash
blocklist + the netns where the kernel allows it.
- Reject literal open("/etc/passwd"), /etc/shadow, /etc/sudoers,
/etc/ssh/*, and /proc/<pid>/environ. Other files
(/etc/os-release, /etc/hostname, /tmp/*, user dirs) still work.
The _check_code_safety summariser is updated to include the new
network_calls and sensitive_file_reads buckets in its error string.
Regression-checked: echo, sleep, ls /tmp, for loops, piped helpers
(echo a | tr a A), urllib.request.urlopen("http://example.com"),
socket.getaddrinfo("example.com",80), open("/etc/os-release"),
open("/tmp/...","w") all still succeed. curl, wget, nc, ssh, rm,
socket.create_connection(("169.254.169.254",80)),
open("/etc/passwd"), open("/proc/self/environ") all correctly
blocked.
* studio: rate-limit login, rotate refresh tokens, add logout, security headers, gate bootstrap injection
A pass over the auth surface found a cluster of related issues that this
commit closes together.
Login (routes/auth.py):
- Add an in-memory per-IP login rate limiter. Five failed POSTs to
/api/auth/login inside a 60s window produce 429 with Retry-After.
A successful login clears the bucket. Previously 30 wrong passwords
in under one second was accepted as 30x 401, which combined with
the (now fixed) admin-username leak from /api/auth/status made
brute-force trivial against a small password.
Logout (routes/auth.py):
- New POST /api/auth/logout returns 204 and calls
storage.revoke_user_refresh_tokens(subject) so the refresh token
is no longer valid. Previously POST /api/auth/logout returned 405
and there was no way to invalidate refresh tokens short of
changing the password. Frontend session.ts already calls
clearAuthTokens() to drop localStorage; the new endpoint lets the
client also tell the server to revoke server-side state.
Refresh-token rotation (routes/auth.py + auth/storage.py):
- New storage.consume_refresh_token(token) atomically validates +
deletes a refresh token, returning (username, is_desktop). The
/api/auth/refresh handler now mints both a new access AND a new
refresh token; the supplied token becomes invalid. Replaying a
consumed refresh returns 401 "Invalid or expired refresh token".
The previous refresh_access_token helper is left in place for
callers that intentionally want the non-rotating shape; nothing
in the route layer uses it now.
/api/auth/status no longer leaks default_username (models/auth.py +
routes/auth.py):
- AuthStatusResponse.default_username becomes Optional[str] with a
None default; the handler always returns None. The frontend already
hardcodes HIDDEN_LOGIN_USERNAME = "unsloth" (auth-form.tsx:82), so
no UI change is required.
window.__UNSLOTH_BOOTSTRAP__ no longer auto-injects (main.py):
- _inject_bootstrap is now opt-in via the
UNSLOTH_STUDIO_INJECT_BOOTSTRAP env var. The previous default
(inject whenever requires_password_change is true) embedded the
plaintext bootstrap password into the first-boot HTML for any
caller that hit /, /change-password, or any unknown SPA path.
Browser extensions and any XSS payload on the page could read it
trivially. With the new gate the bootstrap password lives only in
the auth/.bootstrap_password file (mode 0o600) where it has always
been; users typing it into a current-password field is the right
UX. routes/auth.py:change_password also clears
app.state.bootstrap_password defensively.
Security headers + server fingerprint (main.py + run.py):
- New SecurityHeadersMiddleware adds Content-Security-Policy,
X-Frame-Options: DENY, X-Content-Type-Options: nosniff,
Referrer-Policy: no-referrer,
Permissions-Policy: camera=(), microphone=(), geolocation=(),
interest-cohort=(), and stamps server: unsloth-studio so the
generic uvicorn banner no longer fingerprints the stack. The
uvicorn.Config gains server_header=False so it stops emitting its
own Server header.
/api/health minimisation (main.py):
- Unauthenticated GET /api/health returns just
{"status":"healthy","timestamp":...} so load-balancer liveness
probes keep working without leaking version, device_type,
chat_only, desktop_protocol_version, or studio_root_id to
arbitrary callers. A request that presents a valid Bearer token
still gets the full diagnostic payload so internal launchers and
sibling-Studio detection (which compares studio_root_id) keep
working.
Verification:
- 30 wrong-password POSTs to /api/auth/login -> first 5 = 401, 6th
through 30th = 429.
- POST /api/auth/logout with a fresh token -> 204. The matching
refresh token then fails 401.
- Login -> R1; /api/auth/refresh with R1 -> new access + R2 (R2 !=
R1); /api/auth/refresh with R1 again -> 401; /api/auth/refresh
with R2 -> still succeeds once and rotates again.
- curl /api/auth/status -> default_username: null.
- curl http://127.0.0.1/ does not contain __UNSLOTH_BOOTSTRAP__.
- curl -I / shows CSP, X-Frame-Options: DENY,
X-Content-Type-Options: nosniff, Referrer-Policy: no-referrer,
Permissions-Policy, and server: unsloth-studio.
- curl /api/health unauthenticated -> {status, timestamp} only.
curl with Authorization: Bearer <valid> -> full payload.
- Existing /api/system, /api/models/list, /api/train/status,
/api/inference/status, /api/auth/api-keys, login flow, SPA root
all still return 200 after the changes (regression smoke).
* studio: add SecurityHeadersMiddleware, MaxBodyMiddleware, /recipes redirect, gate _inject_bootstrap, minimise /api/health
This commit lands the main.py-side changes that share a single
middleware-registration spot. They are kept together because every
change here is either (a) a top-level middleware definition that has
to be added next to LoggingMiddleware, or (b) a route handler at the
same file-level.
SecurityHeadersMiddleware (Content-Security-Policy, X-Frame-Options:
DENY, X-Content-Type-Options: nosniff, Referrer-Policy: no-referrer,
Permissions-Policy, server: unsloth-studio). The previous responses
emitted no CSP, no XFO, no Referrer-Policy and were stamped
server: uvicorn.
MaxBodyMiddleware rejects POST/PUT/PATCH on the inference / dataset /
data-recipe / train / export prefixes when Content-Length exceeds
UNSLOTH_STUDIO_MAX_BODY_MB (default 100). The audit hit this by
attaching a 50 MB plain-text file to a chat message and watching
Studio base64-encode it into the JSON body; uvicorn has no enforced
cap so the only previous guard was the per-file 50 MB ceiling that
data-recipe upload routes already enforce. The new middleware extends
that ceiling to the OpenAI-compat path that the Chat attachments
flow through. Verified: a 200 MB JSON POST to /v1/chat/completions
returns HTTP 413 "Request body too large (209,715,264 bytes; max
104,857,600)". A small valid request continues to reach the handler.
_inject_bootstrap is gated behind UNSLOTH_STUDIO_INJECT_BOOTSTRAP.
The previous default was to inline window.__UNSLOTH_BOOTSTRAP__ =
{username, password} into the first-boot HTML whenever
requires_password_change was true, which exposed the plaintext
bootstrap password to any browser extension, page script, or LAN
caller on -H 0.0.0.0. The bootstrap password remains in the on-disk
.bootstrap_password file (mode 0o600) where it has always lived;
users typing it into a current-password field is the right UX.
/api/health unauthenticated returns {"status":"healthy","timestamp":
...} only; the previous payload (version, device_type, chat_only,
desktop_protocol_version, supports_desktop_auth, studio_root_id,
native_path_leases_supported) is preserved for callers that present
a valid Bearer token, so internal launchers and sibling-Studio
detection (which compares studio_root_id) keep working.
/recipes -> /data-recipes 308 redirect. The Data Recipes page lives
at /data-recipes; users typing /recipes hit the SPA catch-all and
saw "Not Found". The redirect also preserves any tail path, so
/recipes/<rest> -> /data-recipes/<rest>.
Verified end to end with curl: CSP / XFO / X-Content-Type-Options /
Referrer-Policy / Permissions-Policy all present on /, server header
is now unsloth-studio (uvicorn's own banner is suppressed via
server_header=False in run.py from the auth-batch commit). Followed
the /recipes redirect lands on the SPA HTML.
* studio: bound TrainingStartRequest hyperparameters at the schema level
POST /api/train/start accepted any value for learning_rate, batch_size,
max_steps, max_seq_length, warmup_steps, warmup_ratio, num_epochs,
save_steps, weight_decay, gradient_accumulation_steps, lora_r,
lora_alpha and lora_dropout, including -1, 0, 1e9, and non-numeric
strings like 'abc' or 'two' (which silently coerce to 0 in the
trainer). Probing showed the API returning 200 to learning_rate=-1
and batch_size=0; only max_steps had any partial clamping.
This commit adds field_validator on every numeric hyperparameter.
Bounds are chosen wide enough to span realistic single-host
configurations (B200 with 180 GB of memory comfortably fits the
upper end) while rejecting the values that always produce broken
training:
- learning_rate: parses str/float, requires 0 < lr < 1.0. Non-numeric
input raises with "learning_rate must be parseable as float (got
'abc')" instead of silently coercing to 0.
- batch_size: [1, 1024].
- gradient_accumulation_steps: [1, 4096].
- num_epochs: [1, 1000].
- max_steps: [1, 1_000_000].
- max_seq_length: [1, 131072].
- warmup_steps: [0, max_steps].
- warmup_ratio: [0.0, 1.0].
- save_steps: [0, 1_000_000].
- weight_decay: [0, 10] (typical 0..0.1).
- lora_r: [1, 512].
- lora_alpha: [1, 1024].
- lora_dropout: [0.0, 1.0).
Each validator names the offending field in its ValueError message
so the 422 response body identifies which input is bad. The
learning_rate validator returns its result as str (the schema field
type is str("2e-4") for backwards compatibility) so existing call
sites that float() the value continue to work.
Verified:
- learning_rate=-1 -> 422 "learning_rate must be > 0 (got -1.0);
typical range is 1e-6 .. 1e-3".
- learning_rate='abc' -> 422 "must be parseable as float".
- batch_size=-1 / 0 / 999999 -> 422 "batch_size must be in [1, 1024]".
- batch_size='two' -> 422 (pydantic int parser).
- max_steps=0 / -5 -> 422 "must be a positive int".
- max_seq_length=200000 -> 422 "must be in [1, 131072]".
- warmup_ratio=2.5 -> 422 "must be in [0.0, 1.0]".
- lora_dropout=1.5 -> 422 "must be in [0.0, 1.0)".
- Valid request with learning_rate='2e-4', batch_size=1, max_steps=5
passes validation and the training run starts as normal.
* studio: redact image-decode errors, clean checkpoint dirs on cancel, tolerate Stop-button + tool-result message shapes
Three small fixes that fall under "do not let the audit findings
become user-visible papercuts".
routes/inference.py - image-decode error redaction (the audit hit
this with a 0-byte / malformed / wrong-extension image upload). The
three image-normalise sites previously raised HTTPException(400,
detail=f"Failed to process image: {e}"). When PIL raised
UnidentifiedImageError(io.BytesIO(raw)) the message string included
"<_io.BytesIO object at 0x7e40a5d7bf60>", leaking both the Python
class name (confirming the PIL/io stack) and a heap address (mildly
useful for ASLR-bypass chaining if another memory-corruption bug is
ever found). Each site now catches UnidentifiedImageError and
returns the generic "Unsupported or corrupt image format"; the
fall-through generic except returns "Failed to process image". No
exception-repr is interpolated into a response body anywhere along
these paths.
core/training/training.py - checkpoint cleanup on cancel. When a
user clicks Cancel Training, the trainer flips _cancel_requested=True
and the supervisor force-terminates the subprocess. The trainer
writes checkpoint-<step> directories under output_dir every
save_steps; previously these survived the cancel and accumulated on
disk (the audit recorded ~67 MB stuck after a 200-step cancel with
save_steps=20). New helper _cleanup_cancelled_checkpoints(output_dir)
globs checkpoint-<int> entries and removes them. It is gated by a
realpath containment check against outputs_root() so it cannot
accidentally rmtree anything outside the configured outputs root.
force_terminate() invokes the helper after the subprocess join when
_cancel_requested is true. Stop-and-Save runs are unaffected because
that path keeps _cancel_requested=False.
models/inference.py - chat message shape tolerance. Two related
frontend interactions used to crash the request validator:
- After the Stop button truncates a generation, the frontend
retained {role:"assistant", content:""} in the conversation
history and replayed it on the next send. ChatMessage previously
required role="assistant" to have non-empty content or tool_calls,
so the next message returned 422 and the thread was permanently
broken. The validator now normalises empty assistant content to
None so the request round-trips and the trailing empty turn can
be ignored downstream.
- The frontend's second-round tool POST drops the streamed
tool_call_id, hitting the strict-spec check "role=tool requires
tool_call_id". The validator now synthesises an opaque id
(call_<8 hex>) when missing, so the request reaches the handler
and the model's final summarising response gets generated. The
proper fix lives in the frontend (carry the streamed id through
the second POST) and will follow.
Verified end to end with curl: HTTP 400 (model not loaded) on both
the empty-assistant history shape and the tool-result-without-id
shape, instead of HTTP 422 from the schema validator.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: tighten code comments from security-hardening pass
Trim verbose docstrings and inline finding references added in the
previous commits in this branch. Functionality unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: await get_current_subject in /api/health and make refresh-token consumption atomic
The /api/health auth probe called get_current_subject(creds) without
awaiting it. The coroutine object is truthy, so any caller presenting a
Bearer header (valid or not) received the full diagnostic payload
including version, device_type, studio_root_id, etc. Await the coroutine
and treat HTTPException as 'fall back to the minimal liveness payload'.
consume_refresh_token did SELECT then DELETE WHERE id under default
autocommit isolation. Two concurrent POST /api/auth/refresh requests
could both win the SELECT before either DELETE ran, defeating
single-use refresh-token rotation. Replace with a single
DELETE ... WHERE token_hash = ? AND expires_at >= ? RETURNING ...
statement so the validate-and-delete lands as one atomic op under
SQLite's write lock (3.45.1 supports RETURNING; min was 3.35).
* studio: enforce body cap on chunked uploads and drop unsafe-inline from script-src
MaxBodyMiddleware previously only inspected the declared Content-Length
header; clients omitting it or sending Transfer-Encoding: chunked
bypassed the cap and could still drive an OOM via the downstream
JSON / file readers on /v1/chat/completions, /api/inference, /api/data-recipe,
/api/datasets, /api/train, /api/export. Rewrite as a raw ASGI middleware
that drains and counts http.request frames, replies 413 once the running
total exceeds UNSLOTH_STUDIO_MAX_BODY_MB before invoking the FastAPI
handler, and replays the buffered body to downstream so route code that
calls request.json() / await request.body() works unchanged.
CSP previously included 'unsafe-inline' on script-src, which defeats the
main XSS protection. The frontend bundle does not need inline scripts;
the only inline <script> the backend ever emits is _inject_bootstrap,
which is opt-in via UNSLOTH_STUDIO_INJECT_BOOTSTRAP. Drop 'unsafe-inline'
from script-src by default; when _inject_bootstrap fires, generate a
per-response nonce, embed it on the inlined <script>, and have
SecurityHeadersMiddleware splice 'nonce-XXX' into the CSP for that one
response (the internal x-internal-script-nonce header is popped before
the response leaves the server). 'unsafe-inline' stays on style-src for
Vite-injected styles.
* studio: drop empty assistant sentinel before passthrough
ChatMessage._validate_role_shape normalises role="assistant", content=""
(the post-Stop sentinel emitted by the frontend) to content=None so the
in-process path can drop it via _extract_content_parts. The passthrough
path then ran m.model_dump(exclude_none=True), which strips the now-None
content key entirely, sending {"role":"assistant"} to llama-server / the
OpenAI-compat backend. That fails upstream and leaves the user without a
recoverable Stop->resume.
Add _drop_empty_assistant_sentinels and call it at both passthrough
message origins: _openai_messages_for_passthrough (covers
/v1/chat/completions and the Responses API which routes through it) and
the anthropic_messages_to_openai output before
_anthropic_passthrough_*. Assistant messages that carry only tool_calls
(no content) are preserved.
* studio/tests: cover audit-fix surfaces and rebase pre-existing tests
Adds and updates pytest coverage for the four bot-flagged audit fixes
landed earlier in this branch and rebases two pre-existing tests that
were broken by the relaxed-validator and /api/health auth-gate changes.
studio/backend/tests/test_middleware.py (new)
MaxBodyMiddleware: small protected, large declared, unprotected
passthrough, chunked-upload-over-cap rejection (the regression for
the original Content-Length-only gap), and chunked-under-cap replay.
SecurityHeadersMiddleware: script-src no longer carries
'unsafe-inline', style-src still does, default headers
(XFO/XCTO/Referrer-Policy/Permissions-Policy/server), and the
internal x-internal-script-nonce header is consumed by the
middleware and converted to 'nonce-XXX' in the CSP.
/api/health: no auth -> minimal, invalid Bearer -> minimal
(the await regression), valid Bearer -> full diagnostic payload.
studio/backend/tests/test_desktop_auth.py
consume_refresh_token: second-call returns None, expired returns
None, and a 64-thread concurrent pile-up against the same hash
produces exactly one successful consumer (regression for the
SELECT-then-DELETE race).
test_health_response_reports_desktop_capability_fields: rebase
against the new health_check(request) signature by going through
TestClient with a real bearer instead of asyncio.run-ing the
handler directly.
studio/backend/tests/test_openai_tool_passthrough.py
Pin the new ChatMessage tolerance: assistant without content or
tool_calls is tolerated (normalises content -> None), empty-string
and empty-list assistant content normalise to None, and a missing
/ empty tool_call_id on role='tool' is synthesised as call_<hex>
rather than raising. Tests for _drop_empty_assistant_sentinels
cover the three drop shapes (empty string, empty list, missing
content key), preservation of assistant text and tool_calls-only
messages, and end-to-end through
_openai_messages_for_passthrough.
studio/backend/main.py
SecurityHeadersMiddleware.dispatch used response.headers.pop(...)
for the nonce-header handoff; Starlette's MutableHeaders has no
pop. Read-then-del so the internal handoff header is still
stripped before the response leaves the server.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/tests: rebase three more pre-existing CI tests against this branch
CI on PR #5375 was red on three tests that were tuned for behaviour
predating this branch. Updates each so the assertions match what the
audit fixes intentionally changed; no production code touched.
studio/backend/tests/test_trained_model_scan.py
test_scan_trained_models_includes_lora_and_full_finetune_outputs
passed an absolute tmp_path through scan_trained_models, which now
runs resolve_output_dir / _assert_contained against outputs_root().
Repoint outputs_root() at tmp_path via monkeypatch so the fixture
dirs land under the configured root and the realpath containment
check passes.
tests/test_studio_install_workspace_guard.py
test_health_endpoint_exposes_studio_root_id_not_raw_path read
the first 1500 bytes after @app.get("/api/health") and asserted on
the studio_root_id literal. The handler grew (unauth short-circuit
+ await dependency gate) and the literal slid past the byte window.
Replace the fixed window with a slice up to the next top-level
@app.* decorator so the test surveys the whole handler regardless
of size.
tests/studio/studio_api_smoke.py
The "login burst (5x wrong pw) -> 401 each" assertion was tagged
"When/if we add one, this assertion updates in the same PR." We
added the per-IP rate-limit in routes/auth.py
(_LOGIN_MAX_FAILS=5/60s) but missed the assertion update. Rewrite
the burst probe to observe the new invariant: at least one 401,
eventual transition to 429, and Retry-After present on the 429.
Adds a small _login_with_headers helper since the existing login()
helper drops response headers.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* ci(studio-ui): set UNSLOTH_STUDIO_INJECT_BOOTSTRAP=1 for Playwright Studios
The Chat UI Playwright test drives the first-boot change-password
form, which (per playwright_chat_ui.py step "1. Change-password
through the UI") pre-seeds the hidden current_password field from
window.__UNSLOTH_BOOTSTRAP__. That global is only emitted when the
backend's _inject_bootstrap path fires, which since the security
pass on this branch is gated behind UNSLOTH_STUDIO_INJECT_BOOTSTRAP
and defaults to off. Without the global, the React form's
current_password validator never satisfies, the submit button stays
disabled, and the composer.wait_for() probe times out on
/change-password.
Re-enable injection only for the CI Studios that drive the chat UI
across linux/mac/windows. Production deployments are unaffected: the
env var has to be explicitly opted into, and the on-disk
auth/.bootstrap_password remains the source of truth for human users
typing the password in by hand.
Covers all eight Studio launch sites: the primary chat-ui boot and
the "extra UI tests" boot for each of the three OSes, plus the
pipeTransport JSON-crash retry relaunches in the macOS workflow that
re-spawn Studio mid-job.
A follow-up frontend PR will add a visible current_password input so
the form satisfies its own validator without needing the bootstrap
auto-fill at all; once that lands this CI knob can come back out.
* studio/sandbox: drop unshare(CLONE_NEWNET); add trusted-host allowlist; block sandbox file uploads; raise CPU rlimit default to 600 s
CLONE_NEWNET inside _sandbox_preexec silently killed every outbound
HTTP request from sandboxed Python whenever the kernel allowed
unprivileged user namespaces. requests.get('https://huggingface.co'),
urllib.request.urlopen('https://en.wikipedia.org/wiki/...'),
socket.connect(('arxiv.org', 443)) all failed despite the AST visitor
intending to allow them. The bash blocklist (curl / wget / nc / ssh /
scp / sftp / rsync / socat / eval / source) plus the AST-level
metadata-host denylist still carry the network policy after this
change; CLONE_NEWNET was redundant with both.
Add _TRUSTED_PUBLIC_HOST_LITERALS + _TRUSTED_PUBLIC_HOST_SUFFIXES
(~100 informational hosts: Wikipedia language subdomains, Wikimedia,
Wikidata, Google search, Bing, DuckDuckGo, HuggingFace, GitHub,
raw.githubusercontent.com, arXiv, StackOverflow / Stack Exchange,
MDN, docs.python.org, PyTorch / TensorFlow / NumPy / pandas docs,
pypi / files.pythonhosted.org / npmjs / crates.io, ReadTheDocs,
arXiv, Britannica, BBC / Reuters / Nature / Science, NASA / CDC /
NIH / WHO open data, api.weather.gov). The visitor now blocks
literal hosts that are neither metadata nor trusted with a short
LLM-readable string so the model can retry with an allowed source
instead of choking on a multi-line error.
Block upload-shape calls regardless of host: requests.post / put /
patch / delete / request with files= or data=open(...) /
data=bytes_literal; httpx equivalents; urllib.request.urlopen /
Request with data=...; HuggingFace upload_file / upload_folder /
upload_large_folder / create_commit (module-level FQ paths AND
method-name match on any receiver). Message: "Blocked: file upload
disallowed in sandbox".
Bump UNSLOTH_STUDIO_SANDBOX_CPU_S default 300 -> 600 s so long
agentic chains that span multiple tool calls don't get SIGXCPU'd
mid-stride. Env-var override path is unchanged.
Host normalisation now strips trailing dot, userinfo @, and explicit
port before allowlist / denylist comparison so trailing-DNS-dot,
userinfo-smuggling, and explicit-:443 URLs are decided correctly.
* studio: raise default request-body cap from 100 MB to 500 MB
UNSLOTH_STUDIO_MAX_BODY_MB default goes 100 -> 500 to comfortably
cover vision + audio + multi-recipe-batch JSON payloads. The
MaxBodyMiddleware stream-counting logic from this branch's earlier
06ec088 already handles chunked bodies up to the new cap; env-var
override path is unchanged for callers that want a tighter limit.
* studio/auth: restore /api/auth/status.default_username to 'unsloth'
This branch's earlier b39e9a4 changed default_username to None on the
public /api/auth/status endpoint so the username field didn't leak to
unauthenticated callers. In practice this regressed third-party
clients (and the in-tree React login form's pre-fill UX) without
adding meaningful security: the bootstrap password is the actual
secret, and the username 'unsloth' is the documented default.
Pin default_username to storage.DEFAULT_ADMIN_USERNAME ('unsloth')
and tighten the response model so the field is required rather than
Optional. Anyone who needs anonymisation can still reach for an
allow-list deployment with auth disabled.
* studio/training: raise max_seq_length / batch_size / lora_r / lora_alpha caps
This branch's 7102815 introduced field validators with conservative
caps. The follow-up loosens them so long-context experiments and
high-rank LoRA exploration aren't gated at the schema layer:
_MAX_BATCH_SIZE 1024 -> 4096
_MAX_SEQ_LENGTH 131_072 -> 2_000_000 (2M tokens)
lora_r cap 512 -> 16_384 (_MAX_LORA_R)
lora_alpha cap 1024 -> 32_768 (_MAX_LORA_ALPHA)
_MAX_GRAD_ACCUM / _MAX_STEPS / _MAX_EPOCHS / lora_dropout /
warmup_ratio / weight_decay are unchanged. Hardware (VRAM, host
RAM, kernel launch latency) is now the binding constraint at the
new caps, which is the correct ordering -- the validator stays a
sanity check on -1 / 0 / 'abc' style garbage, not a usability gate.
* studio/tests: cover sandbox allowlist + upload block + raised training caps
studio/backend/tests/test_sandbox_tools.py (new):
TestMetadataHostDenylist -- short "Blocked: cloud-metadata host"
message on AWS IMDS, GCP metadata,
Alibaba ECS, AWS IPv6 IMDS, 169.254/16.
TestTrustedHostAllowlist -- Wikipedia (any language subdomain),
Google, DuckDuckGo, HF, raw GitHub,
arXiv, StackOverflow / family,
MDN, docs.python.org, pypi, BBC,
api.weather.gov, NumPy / PyTorch docs.
TestUntrustedHostBlock -- example.com / random unlisted host
rejected with the short "Blocked: host
not in sandbox allowlist; use an
allowed informational source" message.
Dynamic URLs (computed var) still pass
-- documented limit of static analysis.
TestHostNormalization -- trailing dot, explicit :443, uppercase,
userinfo-@-smuggle all decided
correctly without false-block /
false-pass.
TestUploadDenylist -- requests / httpx / urllib.urlopen with
files= / data=open / data=bytes,
HfApi().upload_file / upload_folder /
create_commit, module-level
huggingface_hub.upload_folder. POST
json= to trusted host still passes.
TestSandboxCpuRlimitDefault -- pin UNSLOTH_STUDIO_SANDBOX_CPU_S=600
default and confirm CLONE_NEWNET
source line is gone.
TestMaxBodyDefault -- pin UNSLOTH_STUDIO_MAX_BODY_MB=500
default.
studio/backend/tests/test_studio_train_validation.py (new):
Pin at-cap-accepts / over-cap-rejects boundaries for
max_seq_length=2_000_000, batch_size=4_096, lora_r=16_384,
lora_alpha=32_768 so a future regression that tightens them back
without explicit user opt-in is caught.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: tighten code comments across the security-hardening pass
* studio: always inject bootstrap credentials on first boot
The UNSLOTH_STUDIO_INJECT_BOOTSTRAP gate added an extra
terminal-to-browser copy-paste on every fresh install. In practice
the LAN credential leak it guarded against is narrow: the password
is one-time, the user rotates it on the very next click, the
default Studio bind is 127.0.0.1, and -H 0.0.0.0 already exposes
the entire API surface. Drop the gate so the inject fires whenever
a bootstrap password is still pending. The CSP nonce wiring stays
in place; the inline script remains the only inline script the
backend ever emits.
The three Playwright UI smoke workflows lose their
UNSLOTH_STUDIO_INJECT_BOOTSTRAP=1 lines along with the explanatory
comment blocks since the inject now happens by default.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* install: support STUDIO_HOME / UNSLOTH_STUDIO_HOME for custom install paths
Currently install.sh and install.ps1 hardcode all install paths off
$HOME / $env:USERPROFILE with no env-var fallback. This blocks
workspace-isolated installs (CI sandboxes, per-PR test environments,
multi-tenant boxes) unless the entire HOME / USERPROFILE is faked,
which also relocates ~/.gitconfig, ~/.ssh, and other unrelated state.
Add an opt-in env-var override that does only what is needed.
Resolution priority (highest first):
1. HOME / USERPROFILE explicitly redirected vs the password-database
default. Detected via getent (Linux), dscl (macOS), or
[Environment]::GetFolderPath (Windows). Best-effort: when the
detection mechanism is unavailable the check is skipped and we
fall through to step 2.
2. UNSLOTH_STUDIO_HOME, if set.
3. STUDIO_HOME, if set (alias for convenience; the variable name
already matches the internal var install.sh sets).
4. Default: legacy $HOME/.unsloth/studio (or
$USERPROFILE\.unsloth\studio on Windows). Identical to today's
behavior when no env var is set.
When an env var override fires:
* DATA_DIR is nested inside ($STUDIO_HOME/share, or $StudioHome\share
on Windows) so the runtime launcher and shortcuts find studio.conf
in the same place install-time wrote it.
* The unsloth CLI shim lands at $STUDIO_HOME/bin/unsloth (Unix) or
$StudioHome\bin\unsloth.exe (Windows). On Windows the shim already
lives under $StudioHome; the change only redirects DATA_DIR and
skips the persistent registry PATH update.
* Persistent shell PATH modifications are skipped (no .bashrc /
.zshrc / .profile append on Unix; no Add-ToUserPath on Windows).
Caller is expected to invoke via absolute path or add the bin dir
to PATH explicitly. Avoids polluting the user's profile with a
workspace-scoped path that may be deleted.
The Unix launcher script is the only piece that must read DATA_DIR
at runtime (it sources studio.conf from there). The hardcoded
DATA_DIR inside the LAUNCHER_EOF heredoc is replaced with an
@@DATA_DIR@@ placeholder substituted via sed at install time, using
the same approach the script already uses for other install-time
substitutions.
Default path behavior is unchanged: when no env var is set and HOME
is not redirected, install.sh / install.ps1 produce exactly the same
file layout as today.
Test scenarios verified locally on install.sh:
* Default (no env vars) -> $HOME/.unsloth/studio (legacy)
* HOME=/tmp/x -> /tmp/x/.unsloth/studio
* UNSLOTH_STUDIO_HOME=/tmp/y -> /tmp/y as STUDIO_HOME root
* STUDIO_HOME=/tmp/z (alias) -> /tmp/z as STUDIO_HOME root
* HOME redirect + env var (HOME wins) -> install follows HOME
* Unwritable override -> exits with clear ERROR message
* install: priority change -- env vars now win over HOME redirect
Flip the resolution order so explicit env vars take precedence over
HOME / USERPROFILE redirection.
New priority (highest first):
1. UNSLOTH_STUDIO_HOME, if set.
2. STUDIO_HOME, if set.
3. HOME / USERPROFILE explicitly redirected.
4. Default.
Rationale: the env vars are explicit single-purpose signals (the user
typed UNSLOTH_STUDIO_HOME=... specifically to redirect Studio). HOME
redirection is broader and incidental -- the user may have redirected
HOME for unrelated reasons (workspace tools, container builds) without
wanting Studio to follow it. When both are set, the more specific
signal should win.
When only HOME is redirected (no env var), behavior is unchanged from
the previous commit: install follows $HOME.
* install: address review feedback (sed escape, downstream propagation, edge cases)
Fixes from gemini-code-assist + chatgpt-codex-connector + reviewer.py
20-parallel run on the open PR.
install.sh:
* Escape sed replacement metacharacters before substituting @@DATA_DIR@@.
Two-stage escape: ' -> '\'' for safe single-quote shell embedding,
then \, &, | for sed replacement string + chosen delimiter. Heredoc
switched to single-quoted DATA_DIR='@@DATA_DIR@@' so we only need
single-quote escaping at runtime. Verified end-to-end with paths
containing & and | (the sed delimiter).
* Pass UNSLOTH_STUDIO_HOME into both setup.sh invocations
(--local and PyPI paths) so the downstream install resolves the
same Studio root install.sh picked.
* macOS .app stub: replace hardcoded
exec "$HOME/.local/share/unsloth/launch-studio.sh" with
exec "$_css_data_dir/launch-studio.sh" so the .app launches the
resolved launcher even in env-override mode.
* Use mkdir -p -- and cd -- when validating the env override so
paths starting with - cannot be misread as flags.
install.ps1:
* Drop .Guid from [guid]::NewGuid().Guid: the property does not
exist; the probe filename was always identical and not unique.
Default ToString() on System.Guid produces the canonical UUID
string we want.
* Guard LOCALAPPDATA before Join-Path to avoid aborting the
installer in service / CI contexts where LOCALAPPDATA is unset
(Join-Path under $ErrorActionPreference='Stop' would otherwise
throw). Computed once into $defaultDataDir; both 'profile' and
'default' branches reuse it.
* Set $env:UNSLOTH_STUDIO_HOME for the duration of the
'unsloth studio setup' subprocess so studio/setup.ps1 and
unsloth_cli see the same install root install.ps1 picked.
Restored in a finally block.
studio/setup.sh:
* Honor UNSLOTH_STUDIO_HOME / STUDIO_HOME (alias) when resolving
STUDIO_HOME, VENV_DIR, VENV_T5_*_DIR. Falls back to the legacy
$HOME/.unsloth/studio when no override is set.
studio/setup.ps1:
* Same change in PowerShell: honor $env:UNSLOTH_STUDIO_HOME /
$env:STUDIO_HOME for $StudioHome / $VenvDir resolution.
unsloth_cli/commands/studio.py:
* Replace the module-level constant
STUDIO_HOME = Path.home() / ".unsloth" / "studio"
with a resolver that honors UNSLOTH_STUDIO_HOME / STUDIO_HOME
before falling through to the legacy default. Same precedence
the installers use.
Verified locally: 6 install.sh scenarios still produce correct paths
(default, HOME redirect, env var, alias, both, bad override). New
sed-escape unit tests pass for paths containing & and |. Python
resolver matches priority: UNSLOTH_STUDIO_HOME > STUDIO_HOME > default.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh: portable sed (no -i.bak) per gemini review feedback
GNU sed -i.bak vs BSD/macOS sed -i.bak vs BusyBox sed have subtly
different semantics. Use the POSIX-portable redirect-then-mv pattern
instead. Functionally identical, runs everywhere.
* studio: persist UNSLOTH_STUDIO_HOME so fresh shells find custom installs
Without this, a custom-root install (UNSLOTH_STUDIO_HOME=/work/studio
bash install.sh --local) only worked in the same shell that ran the
installer. Closing the terminal and reopening lost the env var, the
PATH was deliberately not persisted, and the Python CLI fell back to
~/.unsloth/studio. Result: 'Studio not set up' or quietly operating on
a stale legacy install.
Three persistence layers, all backwards-compatible (default installs
emit zero changes):
1. Unix studio.conf
install.sh now writes 'export UNSLOTH_STUDIO_HOME=...' next to
UNSLOTH_EXE in studio.conf when in env-override mode. The launcher
sources studio.conf at startup so the exec'd binary gets the var.
Default installs do not write this line; studio.conf stays
byte-identical to before.
2. Windows launch-studio.ps1
install.ps1 prepends '$env:UNSLOTH_STUDIO_HOME = ...' to the
generated launcher when in env-override mode. Default installs
produce the same launcher content as before.
3. Python sys.prefix inference
storage_roots.studio_root() and unsloth_cli/commands/studio.py
now infer the install root from sys.prefix when no env var is
set (Path(sys.prefix).parent for unsloth_studio venvs). Catches
direct invocations of <STUDIO_HOME>/bin/unsloth that bypass the
launcher entirely.
unsloth_cli/commands/studio.py also re-exports the resolved
UNSLOTH_STUDIO_HOME via os.environ.setdefault so child processes
(setup script, backend run.py) inherit it.
Backend storage roots (storage_roots.studio_root, cache_root) now
respect the env var via the shared resolver. run.py PID file,
transformers_version.py T5 venvs, and model_config.py vision-check
venv all switch to studio_root() so custom installs are
self-contained.
studio/setup.ps1: T5 sidecar venvs now resolve under $StudioHome
(was $env:USERPROFILE\.unsloth\studio\.venv_t5_*).
studio/setup.sh + studio/setup.ps1: llama.cpp build dir nests under
$STUDIO_HOME / $StudioHome when env-override is active, otherwise
keeps the legacy ~/.unsloth/llama.cpp.
Verified locally:
* studio.conf write block: env-override mode emits the export line;
default mode does not (byte-identical to today).
* PowerShell heredoc interpolation: correct output for both modes.
* studio_root() resolver: default, UNSLOTH_STUDIO_HOME, STUDIO_HOME
alias, and sys.prefix-based inference all return correct paths.
* cache_root() now derives from studio_root().
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: tilde expansion + macOS .app stub safe-quoting
Two fixes from running a 25-scenario simulation sweep against install.sh
across path edge cases (spaces, apostrophes, ampersands, pipes,
backslashes, dollar signs, Unicode, trailing slash, relative paths).
1. UNSLOTH_STUDIO_HOME=~/foo was landing as literal '~/foo' (env vars
are not subject to tilde expansion). Added a POSIX-portable case
block in install.sh, install.ps1, studio/setup.sh, studio/setup.ps1
that expands a leading ~ or ~/ to $HOME / $env:USERPROFILE.
The prefix-removal pattern is single-quoted ('${var#'~/'}') so the
shell does not tilde-expand the pattern back to $HOME/ before
matching -- a subtle dash/bash gotcha.
2. macOS .app stub used an unquoted heredoc ('<< STUB_EOF'), so any
$VAR / backtick / etc in the path would expand at .app launch time.
Switched to single-quoted heredoc ('<< 'STUB_EOF'') with a
placeholder + sed substitution + single-quoted shell embedding,
matching the @@DATA_DIR@@ pattern already used for launch-studio.sh.
Verified: 25/25 simulation scenarios pass on Linux dash + bash,
including paths with $VAR, &, |, \\, ', spaces, and Unicode. End-to-end
install in env-mode + fresh-shell launcher invocation confirmed: studio
binds to /api/health from a clean env, and sys.prefix-based inference
correctly returns the workspace root.
* install: stop accidentally treating default installs as env-override
Reviewer.py 20-runs cycle 1 found a unanimous P1 regression: a default
'unsloth studio update' relocates llama.cpp from ~/.unsloth/llama.cpp
to ~/.unsloth/studio/llama.cpp, because the CLI was re-exporting
UNSLOTH_STUDIO_HOME unconditionally and install.sh / install.ps1 were
passing it into setup.{sh,ps1} unconditionally. The setup scripts
treated the var's mere presence as "env-override mode" and relocated
the llama.cpp build dir away from the legacy path, breaking the
runtime backend's _find_llama_server_binary lookup on default installs.
Fixes:
* unsloth_cli/commands/studio.py: _resolve_studio_home now returns
(path, is_custom). Re-export only when is_custom -- a real env
override or a sys.prefix inference that resolves to a non-legacy
path. Default installs leave UNSLOTH_STUDIO_HOME unset.
* install.sh: gate UNSLOTH_STUDIO_HOME on $_STUDIO_HOME_REDIRECT == env
before calling setup.sh. Use 'env $VARS bash setup.sh' so the var
is set only for the subprocess, never leaked.
* install.ps1: gate $env:UNSLOTH_STUDIO_HOME on $StudioRedirectMode
-eq 'env' before invoking 'unsloth studio setup'. Restore prior
value in finally block (unset if it wasn't set).
* studio/setup.sh + setup.ps1: decide llama.cpp install root from
the resolved $STUDIO_HOME (not from env-var presence). If the
resolved path equals the legacy default ($HOME/.unsloth/studio),
fall back to ~/.unsloth/llama.cpp. This makes setup robust against
a stale UNSLOTH_STUDIO_HOME inherited from a parent process that
happens to point at the legacy default.
* studio/backend/core/inference/llama_cpp.py:
- _find_llama_server_binary() now searches studio_root() / llama.cpp
AND the legacy ~/.unsloth/llama.cpp (de-duped). Custom-root
installs become discoverable; default installs unaffected.
- kill_orphaned_servers ownership allowlist also includes
studio_root() / llama.cpp so custom-root processes are cleanable.
Verified locally:
* 25/25 sim scenarios still pass (path edge cases unchanged).
* setup.sh unit test: default-mode lands UNSLOTH_HOME at $HOME/.unsloth;
env-mode lands at $STUDIO_HOME.
* Python CLI unit test: default-mode returns is_custom=False and does
NOT setdefault UNSLOTH_STUDIO_HOME; env-mode sets is_custom=True.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: || exit 1 on STUDIO_HOME subshell (dash set -e gap)
Gemini review feedback: in dash, set -e does not trigger on subshell
failures inside variable assignments. If 'cd -- "$_override" && pwd'
fails, STUDIO_HOME stays empty and DATA_DIR collapses to /share. Add
explicit '|| exit 1' on both install.sh:187 and setup.sh:413.
* install.sh: argv-safe setup invocation for paths with spaces
Cycle 2 reviewer.py 20-runs found a unanimous P1: passing the env-var
through 'env $_STUDIO_ENV_FOR_SETUP' word-splits on whitespace, so a
custom root like '/tmp/Unsloth Studio' becomes 'UNSLOTH_STUDIO_HOME=
/tmp/Unsloth' followed by env trying to exec 'Studio'.
Replaced with a tiny helper that prepends the env-var directly to the
argv (no string-form intermediary), so spaces are preserved as a
single argument. Default-mode invocation skips the env-var entirely.
Verified: 'UNSLOTH_STUDIO_HOME=/tmp/test space/studio' now reaches
setup.sh as a single value.
* studio: tighten sys.prefix inference + Tauri env handling + llama.cpp env
Cycle 3 reviewer.py findings (3 P1s converging):
* sys.prefix inference too broad: a developer venv named 'unsloth_studio'
was being treated as a custom Studio root. Narrow with an installer-
sentinel check (presence of share/studio.conf or bin/unsloth shim
inside the parent dir) in both unsloth_cli/commands/studio.py and
studio/backend/utils/paths/storage_roots.py.
* Tauri studio/src-tauri/src/process.rs::find_unsloth_binary() hardcoded
~/.unsloth/studio. Honor UNSLOTH_STUDIO_HOME / STUDIO_HOME (in that
priority order) before falling back to legacy.
* unsloth-zoo's GGUF export binds LLAMA_CPP_DEFAULT_DIR at import time
from UNSLOTH_LLAMA_CPP_PATH. For env-override installs, persist
UNSLOTH_LLAMA_CPP_PATH alongside UNSLOTH_STUDIO_HOME in studio.conf
(Unix), in the generated PowerShell launcher (Windows), and via
os.environ.setdefault in the Python CLI when running on a custom
root, so GGUF export uses the custom-root llama.cpp build instead
of the legacy ~/.unsloth/llama.cpp.
Default behaviour unchanged: no env vars are written to studio.conf
in default mode, no LLAMA_CPP_PATH is set, and the dev-venv inference
falls through to legacy when no installer sentinels are present.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: desktop_auth env-aware + legacy-root llama.cpp consistency
- desktop_auth.rs: honor UNSLOTH_STUDIO_HOME / STUDIO_HOME for the
.desktop_secret path so Tauri desktop login works against custom-root
installs instead of always reading ~/.unsloth/studio/auth/.
- install.sh / install.ps1 / unsloth_cli/commands/studio.py: when an env
override resolves to the legacy default ($HOME/.unsloth/studio), set
UNSLOTH_LLAMA_CPP_PATH to ~/.unsloth/llama.cpp (matching setup.sh /
setup.ps1's legacy-equality branch). Previously the persisted value
pointed at $STUDIO_HOME/llama.cpp, which was a non-existent location
and broke unsloth-zoo's import-time GGUF binding for that edge case.
* studio: tauri studio_root helper + marker-file persistence + ~ expansion
Address cycle-5 reviewer findings:
- Add studio/src-tauri/src/studio_root.rs: shared resolver with
UNSLOTH_STUDIO_HOME / STUDIO_HOME (priority order), tilde expansion
(~, ~/..., ~\...), installer-written marker fallback, then
~/.unsloth/studio. 5 unit tests cover the expansion paths.
- Tauri lookups now go through the shared resolver:
- process.rs::find_unsloth_binary
- desktop_auth.rs::desktop_secret_path
- main.rs::setup_logging (tauri.log under custom root)
- commands.rs::open_logs_dir (opens custom root dir)
- install.rs work_dir uses parent of resolved root (avoids creating
a stray ~/.unsloth on a custom-root install)
- install.sh / install.ps1 (env-mode only): write
~/.unsloth/studio-home marker so the desktop app launched from
Finder/Start Menu (no shell env inheritance) still resolves the
custom root.
- install.sh / install.ps1 non-interactive completion: when
StudioRedirectMode=env, print the absolute custom-root shim path
since the persistent rc/registry PATH update is intentionally
skipped in env-override mode.
- unsloth_cli/commands/studio.py: replace setdefault() with
truthy-check so a blank UNSLOTH_STUDIO_HOME / UNSLOTH_LLAMA_CPP_PATH
in the parent env doesn't suppress the inferred custom root.
40/40 cargo test --bins pass.
* studio: validate marker file + write in --tauri mode + propagate to subprocess
Cycle-6 reviewer follow-ups:
- studio_root.rs marker resolver now validates the persisted path before
using it. A stale ~/.unsloth/studio-home pointing at a deleted/moved
workspace is ignored (resolution falls back to the legacy default
rather than hijacking it). Validation accepts share/studio.conf
sentinel or bin/unsloth shim. Trailing newline strip uses
trim_end_matches(['\n','\r']) so paths whose content legitimately has
leading/trailing spaces survive.
- install.sh / install.ps1: marker write moved out of the launcher
generation path so it runs before the Tauri-mode early exit. Both
shell-launcher and Tauri-installed env-mode roots now persist the
marker. Removed the duplicate marker write that was previously inside
install.ps1's $studioHomeExport block.
- studio/src-tauri/src/install.rs: pass UNSLOTH_STUDIO_HOME to the
installer subprocess (when not already in scope) so app-initiated
repair / update flows reach the same root the running app uses.
cargo test --bins -- --test-threads=1: 44/44 pass (4 new tests for
marker validation: sentinel accepted, bin shim accepted, empty dir
rejected, missing path rejected).
* studio: fix Tauri legacy-fallback regression + stale marker cleanup
Cycle-7 reviewer follow-ups (regression I introduced in cycle 6):
- studio_root.rs: add StudioRootSource enum + resolve_studio_root_with_source().
Lets callers distinguish a real custom override (Env / Marker) from the
legacy fallback (Default).
- studio/src-tauri/src/install.rs: only forward UNSLOTH_STUDIO_HOME to the
installer subprocess when the resolution source is Env or Marker. The
Default fallback must NOT be passed -- install.sh / install.ps1 treat
any non-empty UNSLOTH_STUDIO_HOME as env-override mode and would
relocate DATA_DIR to $STUDIO_HOME/share and _LOCAL_BIN to $STUDIO_HOME/bin
(regressing default Tauri repair / update flows from the legacy
~/.local/share/unsloth and ~/.local/bin).
- install.sh / install.ps1: clear stale marker on default / HOME-redirect
installs. A user who first installed with UNSLOTH_STUDIO_HOME=/work/studio
then later reinstalls without env vars no longer has the desktop app
hijacked by ~/.unsloth/studio-home pointing at the old custom root.
- install.sh / install.ps1: when env mode wins over a redirected
HOME / USERPROFILE, write the marker into the OS-reported real profile
home (getent / dscl on Unix; [Environment]::GetFolderPath on Windows)
so a later desktop launch from the user's normal session still finds
it. Falls back to the current HOME / USERPROFILE.
cargo test --bins -- --test-threads=1: 45/45 pass (1 new for the source
enum invariants).
* install: scrub stale marker from real-home on HOME-redirect cleanup
Cycle-8 reviewer follow-up: the previous cleanup branch only removed
\$HOME/.unsloth/studio-home, leaving a stale marker in the real
password-database home after a prior env-mode install. A later default
install with redirected HOME / USERPROFILE would still see the desktop
app resolving the old custom root.
- install.sh: compute the real password-database home (via getent /
dscl) unconditionally, and scrub markers from BOTH \$HOME and the
real-home in the default / HOME-redirect cleanup branch.
- install.ps1: build a profile-candidate list (current USERPROFILE
+ OS-reported real profile) and remove markers from EVERY candidate
in the default / profile-redirect cleanup branch.
bash -n + cleanup smoke verified.
* revert: drop Tauri env-var support + marker file mechanism
Keep this PR scoped to shell installer + Python backend env-var support.
Tauri desktop integration with custom Studio roots is deferred to a
separate, focused PR.
Reverts to pre-PR state:
- studio/src-tauri/src/process.rs (find_unsloth_binary)
- studio/src-tauri/src/desktop_auth.rs (auth_secret_path)
- studio/src-tauri/src/main.rs (setup_logging tauri.log path)
- studio/src-tauri/src/commands.rs (open_logs_dir)
- studio/src-tauri/src/install.rs (work_dir + subprocess env)
- studio/src-tauri/src/studio_root.rs DELETED
Removes from install.sh / install.ps1:
- ~/.unsloth/studio-home marker write/read/cleanup
- HOME-redirect-aware marker location logic
What this PR keeps (the original scope):
- install.sh / install.ps1: UNSLOTH_STUDIO_HOME / STUDIO_HOME env-var
resolver with HOME-redirect detection, tilde expansion, legacy
fallback. Default installs are byte-identical to pre-PR.
- studio/setup.sh / studio/setup.ps1: legacy-equality llama.cpp path.
- studio.conf / launcher persists UNSLOTH_STUDIO_HOME +
UNSLOTH_LLAMA_CPP_PATH for fresh shells (env-mode only).
- unsloth_cli/commands/studio.py: env > sys.prefix sentinel > legacy
resolver, conditional re-export.
- studio/backend/utils/paths/storage_roots.py: same resolver.
- Backend modules use storage_roots (run.py, model_config.py,
transformers_version.py, llama_cpp.py).
cargo test --bins -- --test-threads=1: 34/34 pass (pre-PR baseline).
bash -n install.sh: clean.
* install: cycle-10 fixes (default launcher, --tauri guard, env-mode shortcuts, win PATH)
- install.sh launcher: default and HOME-redirect installs keep the
legacy DATA_DIR=\"\$HOME/.local/share/unsloth\" runtime form so a
later shell with a different \$HOME still resolves DATA_DIR. Only
env-mode bakes the resolved absolute path. Restores byte-identical
default behavior.
- install.sh / install.ps1: fail fast when --tauri is combined with
UNSLOTH_STUDIO_HOME / STUDIO_HOME. The desktop app still resolves
the legacy ~/.unsloth/studio root, so a custom-root --tauri install
would yield a desktop app that cannot find its binary or auth
secret. Print the right alternative.
- install.sh / install.ps1: skip persistent desktop / Start-Menu
shortcuts in env-override mode. Workspace-scoped installs would
otherwise leave launchers pointing at a path the user may delete.
Default and HOME/profile-redirect installs keep the shortcut.
- install.ps1: re-prepend env-override \$ShimDir AFTER
Refresh-SessionPath. Refresh rebuilds PATH as Machine > User >
current \$env:Path, so a previously-installed legacy User PATH
entry would otherwise win precedence over the current-session
env-override shim.
bash -n install.sh, pwsh parser install.ps1 + setup.ps1: clean.
cargo test --bins -- --test-threads=1: 34/34 (Tauri unchanged).
* install: cycle-11 fixes (env-mode launcher writes, --tauri legacy passthrough, run.py llama path)
- install.sh / install.ps1: env-mode no longer skips the entire
create_studio_shortcuts / New-StudioShortcuts function. Move the
early-return INSIDE those functions, just before the persistent
desktop / Start-Menu shortcut creation. The runtime launcher
(launch-studio.sh / launch-studio.ps1), studio.conf with
UNSLOTH_STUDIO_HOME / UNSLOTH_LLAMA_CPP_PATH exports, and the icon
ARE always written so env-mode shims can resolve via fresh shells.
- install.sh / install.ps1: --tauri guard passes through when the
override resolves to the legacy default ($HOME/.unsloth/studio /
%USERPROFILE%\.unsloth\studio). The desktop app already uses that
path, so explicit-equality is a supported edge case (matches the
llama.cpp legacy-equality branch).
- studio/backend/run.py: when launched directly (bypassing the
unsloth CLI), set UNSLOTH_STUDIO_HOME and UNSLOTH_LLAMA_CPP_PATH
before the rest of import chain runs so unsloth-zoo's import-time
LLAMA_CPP_DEFAULT_DIR binding picks up the custom-root build. Only
set when STUDIO_ROOT is a real custom override; legacy default
installs leave them unset.
bash -n install.sh, pwsh parser install.ps1: clean.
python ast parse studio/backend/run.py: clean.
cargo test --bins -- --test-threads=1: 34/34 pass (Tauri unchanged).
* install: cycle-12 fixes (--tauri trailing slash + main.py uvicorn env)
- install.sh / install.ps1 --tauri legacy passthrough: strip trailing
separators before comparing the override to the legacy default.
Previously UNSLOTH_STUDIO_HOME=\"\$HOME/.unsloth/studio/\" (with
trailing slash) was rejected even though it resolves to the
supported legacy root.
- studio/backend/main.py: when launched directly via
\`uvicorn main:app\` from a custom-root venv (bypassing both
unsloth_cli and run.py), export UNSLOTH_STUDIO_HOME and
UNSLOTH_LLAMA_CPP_PATH before any unsloth-zoo import so its
import-time LLAMA_CPP_DEFAULT_DIR binding picks up the custom-root
build. Only sets when STUDIO_ROOT is a real custom override.
bash -n install.sh, pwsh parser install.ps1, python ast main.py: clean.
Smoke probe: UNSLOTH_STUDIO_HOME=\$HOME/.unsloth/studio/ install.sh --tauri
no longer exits with the unsupported-custom-root error.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.ps1: skip CWD-relative venv migration in env-override mode
The legacy ~/unsloth_studio venv migration path on Windows reads
%USERPROFILE%\unsloth_studio\Scripts\python.exe (a fixed home-relative
path). Under env-override mode this would Move-Item the user's
pre-existing default-install venv into $StudioHome\unsloth_studio,
breaking the default install and contaminating the workspace root.
Gate the migration on $StudioRedirectMode -ne 'env' so workspace-scoped
installs leave the user's default-install venv untouched.
No Linux equivalent: install.sh migrates from \$STUDIO_HOME/.venv which
is already env-mode-aware (points at the workspace root, not \$HOME).
* install: cycle-14 fixes (Tauri env scrub + setup.ps1 missing-root error)
Tauri does not honor UNSLOTH_STUDIO_HOME / STUDIO_HOME / UNSLOTH_LLAMA_CPP_PATH
yet -- the desktop app's Rust paths use the legacy ~/.unsloth/studio root.
If the user's shell has these env vars set, spawned Python subprocesses would
diverge from the Rust paths (custom-root Python <-> legacy-root Rust).
Scrub the three env vars at all Tauri subprocess spawn sites:
- process.rs: backend launch
- desktop_auth.rs: provision-desktop-auth subprocess
- install.rs: install.sh / install.ps1 invoked from the desktop app
(also prevents the --tauri guard from rejecting an inherited override).
setup.ps1: when UNSLOTH_STUDIO_HOME points at a non-existent directory,
'Resolve-Path -LiteralPath' threw a confusing PSObject error under
$ErrorActionPreference = "Stop". Test-Path the override first and emit a
friendly "run install.ps1 to create the install root" message instead.
* install: cycle-15 fixes (preserve UNSLOTH_LLAMA_CPP_PATH + add update.rs scrub)
UNSLOTH_LLAMA_CPP_PATH is a pre-existing custom-llama.cpp-directory override
the Python backend (studio/backend/core/inference/llama_cpp.py) and unsloth-zoo
intentionally support. It is unrelated to the Studio install root. Cycle 14
over-scrubbed it from the Tauri spawn sites, regressing desktop GGUF/llama.cpp
workflows for users who set it in their shell.
- process.rs / desktop_auth.rs / install.rs: stop scrubbing
UNSLOTH_LLAMA_CPP_PATH; only scrub UNSLOTH_STUDIO_HOME and STUDIO_HOME.
- update.rs: missed Tauri spawn site -- add the same UNSLOTH_STUDIO_HOME /
STUDIO_HOME scrub so 'unsloth studio update' from the desktop app updates
the legacy-root install Tauri actually manages.
Verified: cargo test --bins -- --test-threads=1 -> 34/34 pass.
* install.sh: document apostrophe-escape derivation inline
The shell quoting at install.sh:642 / 659 / 679 / 680 / 823 has been
flagged as broken across multiple review cycles, but every end-to-end
verification (DATA_DIR=\"a b's&c|d\$e\" -> generated launcher -> source ->
recovered exact input) passes. The proposed "8 backslash" fix would
double the escape and actually break what currently works.
Strengthen the inline comments to spell out the derivation:
- shell pattern \"s/'/'\\\\''/g\" passes \"s/'/'\\''/g\" to sed (\\\\ -> \\)
- sed replacement '\\'' yields close-quote / escaped-quote / open-quote
- stage 2 (\\, &, |) only needed where the value is then sed-replaced
into a launcher template via s|@@DATA_DIR@@|VALUE|g
studio.conf is written via printf, not sed, so it only needs stage 1.
No behavior change, only inline doc to head off future false positives.
* install/setup .ps1: use -LiteralPath for $StudioHome-derived paths
Pre-PR, $StudioHome was hardcoded to %USERPROFILE%\.unsloth\studio --
no wildcard characters possible. The PR introduces UNSLOTH_STUDIO_HOME /
STUDIO_HOME, so $StudioHome (and every path derived from it: $VenvDir,
$VenvPyExe, $UnslothExe, $UnslothHome, $LlamaCppDir, $VenvT5_*, etc.)
can now contain bracket characters that PowerShell would interpret as
wildcards.
Reproducer (from cycle 17 review 20):
pwsh> Test-Path 'studio[abc]/Scripts/python.exe'
False
pwsh> Test-Path -LiteralPath 'studio[abc]/Scripts/python.exe'
True
Switch the relevant Test-Path / Remove-Item / New-Item / Move-Item calls
in install.ps1 and studio/setup.ps1 to -LiteralPath. Sites where the
path is fixed (the shim under %LOCALAPPDATA%\Microsoft\WindowsApps,
$RepoRoot from -PSCommandPath) keep the wildcard-aware form.
* install/setup .ps1: fix New-Item -LiteralPath regression from cycle 17
Cycle 17 added -LiteralPath to all $StudioHome-derived path operations,
but New-Item has no -LiteralPath parameter (verified pwsh 7.6 syntax:
"New-Item [-Path] <string[]> [-ItemType <string>] ..."). Every directory-
creation site would throw "A parameter cannot be found that matches
parameter name 'LiteralPath'" at runtime, blocking T5 sidecar setup,
llama.cpp parent creation, and StudioHome creation.
Likewise, "Split-Path -LiteralPath $X -Parent" cannot mix LiteralPath
with -Parent (separate parameter sets). The default LiteralPath mode
already returns the parent.
Switch to [System.IO.Directory]::CreateDirectory($X), which natively
takes a literal path, and drop the trailing -Parent on Split-Path.
Verified end-to-end on a bracketed path "/tmp/...[abc]":
- CreateDirectory: created
- Test-Path -LiteralPath: detects
- nested CreateDirectory(Split-Path -LiteralPath ...): works
* install/setup .ps1: extend -LiteralPath sweep to remaining \$StudioHome paths
Cycle 17/18 missed several wildcard-aware operations on user-controlled
\$StudioHome-derived paths. Reviewers identified remaining sites:
install.ps1:
- \$UnslothExePath (Test-Path / Resolve-Path) at the shortcut creator
- \$VenvDir (Get-ChildItem) at the no-torch-runtime resolver
- \$ShimDir (New-Item Directory -- replaced with .NET CreateDirectory)
- \$ShimExe (Test-Path / Remove-Item / re-prepend guards) -- the shim
lives at \$StudioHome\\bin\\unsloth.exe in env-override mode, so it
inherits bracket sensitivity from \$StudioHome.
- \$UnslothExe (Copy-Item fallback) when HardLink fails.
studio/setup.ps1:
- \$LlamaServerBin (Test-Path) at the prebuilt-bundle / source-build
validation gates (3 sites). \$LlamaServerBin lives under \$BuildDir
under \$LlamaCppDir under \$UnslothHome under \$StudioHome.
New-Item HardLink keeps -Path because creating a non-existent target
with brackets succeeds (verified via direct pwsh smoke test).
* install: cycle-20 fixes (more setup.ps1 -LiteralPath + shell-quote launch hints)
setup.ps1: extend -LiteralPath sweep to remaining \$BuildDir-derived paths
that the cycle-19 commit missed:
- \$CmakeCacheFile (Test-Path + Select-String -Path)
- \$buildTmp (10 Test-Path / Remove-Item sites in source-build cleanup)
- \$QuantizeBin (Test-Path)
- \$altBin (Test-Path)
These all live under \$BuildDir -> \$LlamaCppDir -> \$UnslothHome ->
\$StudioHome, which is now user-controlled via UNSLOTH_STUDIO_HOME.
Bracket characters in the override would silently skip rebuild
detection or leave stale build artifacts.
install.sh: shell-quote the launch-instruction substep lines for env-
override mode. UNSLOTH_STUDIO_HOME values containing spaces or
apostrophes (e.g. "/tmp/O'Brien Studio") would print copy-paste-
unsafe commands -- the install succeeded but the printed launch
instructions split at the space. Now wraps with the canonical
'\\''-style escape so the printed lines parse with bash -n.
Verified end-to-end:
- printed shim line: '/tmp/O'\''Brien Studio/bin/unsloth' studio ...
- bash -n on the printed line passes.
* install.ps1: -LiteralPath for macOS-stub-launcher \$appDir-derived paths
The shortcut/launcher generator at install.ps1:418-693 writes the
stub launcher, .vbs, and icon under \$appDir = \$StudioDataDir, which in
env-override mode is \$StudioHome\share. Cycle 17/19/20 missed the
following wildcard-aware ops on these paths:
- Test-Path \$appDir (with New-Item Directory swap to .NET CreateDirectory)
- Set-Content -Path \$launcherVbs (for the WSH .vbs stub)
- Test-Path / Copy-Item \$bundledIcon (bundled icon copy)
- Test-Path / Remove-Item \$iconPath (icon header validation)
In env-override mode \$StudioHome can contain bracket characters;
without -LiteralPath the .vbs write fails outright and the icon
validation can either skip a present icon or fail to delete a
malformed one. (The COM shortcut creation downstream returns early
in env-override mode, so its path values don't need this treatment.)
* install: don't override pre-existing UNSLOTH_LLAMA_CPP_PATH in launchers
Cycle 14/15 established UNSLOTH_LLAMA_CPP_PATH as a pre-existing
custom-llama.cpp-directory override the Python backend and unsloth-zoo
intentionally support, independent of the Studio install root.
The launchers (studio.conf sourced by Unix launch-studio.sh, and the
PowerShell launch-studio.ps1) were unconditionally re-exporting it,
which silently overrides a user's pre-existing value when they invoke
the launcher from a shell where UNSLOTH_LLAMA_CPP_PATH is already set.
Make the assignment conditional in both launchers:
install.sh studio.conf:
if [ -z "\${UNSLOTH_LLAMA_CPP_PATH:-}" ]; then
export UNSLOTH_LLAMA_CPP_PATH='...'
fi
install.ps1 launch-studio.ps1:
if (-not \$env:UNSLOTH_LLAMA_CPP_PATH) {
\$env:UNSLOTH_LLAMA_CPP_PATH = '...'
}
UNSLOTH_STUDIO_HOME stays unconditional: the launcher is bound to a
specific install, so its STUDIO_HOME must always match that install.
* install.sh: harden --tauri legacy resolver against CDPATH and symlinks
Reviewer cycle 23 (inst 19) noted that the bare \`cd -- ... && pwd\` form
in the --tauri legacy comparison can echo a CDPATH-prefixed path when the
user has CDPATH set in their environment, contaminating the resolved
absolute path used in the legacy-equality check.
Switch to \`CDPATH= cd -P -- ... && pwd -P\` so:
- CDPATH= clears the cd-prefix-echo behavior
- -P / pwd -P resolves any symlinks to a canonical path
No behavior change for users without CDPATH set; correctness fix for
users who have it set in their shell.
* install + llama_cpp backend: cycle-24 hardening
Three real findings from cycle 24 reviewers:
1. install.sh:231 + studio/setup.sh:413 -- main \$STUDIO_HOME
resolvers used the same bare \`cd -- ... && pwd\` form that cycle 23
only fixed for the --tauri guard. Switch both to:
\$(CDPATH= cd -P -- "\$override" && pwd -P)
so relative custom-root values don't get CDPATH-prefixed or have
the cd-on-CDPATH stdout newline contaminate the captured value.
2. install.sh --tauri legacy root used logical \$HOME/.unsloth/studio
while the override side was canonicalized via pwd -P. A symlinked
\$HOME (e.g. /home/alice -> /u/alice) made the comparison fail even
when both sides pointed at the same directory. Canonicalize the
legacy side too when the dir exists.
3. studio/backend/core/inference/llama_cpp.py:_find_llama_server_binary
searched \$STUDIO_HOME/llama.cpp first then ~/.unsloth/llama.cpp
in default-mode installs. setup.sh / setup.ps1 only install llama.cpp
under \$STUDIO_HOME/llama.cpp in env-override mode; in default mode
it always lives at ~/.unsloth/llama.cpp. The post-PR search would
pick up a stale partial install at ~/.unsloth/studio/llama.cpp over
the real legacy binary.
Mirror setup's legacy-equality check: when studio_root() resolves
equal to ~/.unsloth/studio, search ONLY the legacy ~/.unsloth/llama.cpp.
Otherwise (env-override custom root), search custom first, legacy
fallback.
* install + setup: canonicalize legacy-equality comparison sites
Cycle 24 made \$STUDIO_HOME canonical via 'CDPATH= cd -P -- ... && pwd -P',
but the legacy-equality comparison sites still used the bare logical
"\$HOME/.unsloth/studio" string. With a symlinked \$HOME (e.g.
/home/alice -> /u/alice), the comparison fails even when both sides
point at the same dir, and llama.cpp ends up under a custom-root path
the Python backend's legacy comparison cannot find.
Reviewer cycle 25 inst 2 reproduced this with HOME=/tmp/link -> /tmp/real
and UNSLOTH_STUDIO_HOME=\$HOME/.unsloth/studio: setup.sh resolves
UNSLOTH_HOME to /tmp/real/.unsloth/studio while the backend search
resolves both physically equal and looks at /tmp/link/.unsloth/llama.cpp.
Canonicalize the legacy side at all four sites:
- install.sh:695 (create_studio_shortcuts llama.cpp path)
- studio/setup.sh:577 (UNSLOTH_HOME selection)
- install.ps1:462 (launcher UNSLOTH_LLAMA_CPP_PATH path)
- studio/setup.ps1:1829 (UnslothHome selection)
Apply CDPATH= cd -P -- ... && pwd -P (Unix) or Resolve-Path -LiteralPath
(Windows) when the legacy dir exists. unsloth_cli/commands/studio.py
already does this via Path.resolve().
* llama_cpp: gate _kill_orphaned_servers studio-root allowlist on env-override
Cycle 24 fixed _find_llama_server_binary to only search
\$STUDIO_HOME/llama.cpp when STUDIO_HOME is a real env override (not
the legacy default), but the symmetric _kill_orphaned_servers
allowlist still appended _sr() / "llama.cpp" unconditionally.
In default mode _sr() resolves to ~/.unsloth/studio, so
~/.unsloth/studio/llama.cpp would be treated as a Studio-owned install
root for the orphan-kill scan even though the default installer does
not own that path. A llama-server process running there from a
different tool or a stale partial install would be killed.
Apply the same legacy-equality check used in _find_llama_server_binary
and the install/setup scripts: only add _sr()/"llama.cpp" to the
allowlist when STUDIO_HOME != legacy default.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* setup.sh + setup.ps1: canonicalize both sides of legacy-equality check
Proactive audit pass found one real asymmetry the cycle-by-cycle
review process had not yet flagged:
- install.sh:704 / install.ps1:469 are gated on env-mode and only
run when STUDIO_HOME has already been canonicalized (cycle 24).
Symmetric.
- studio/setup.sh:577 / studio/setup.ps1:1829 run UNCONDITIONALLY,
including in default mode. In default mode STUDIO_HOME is set to
the bare logical \$HOME/.unsloth/studio (setup.sh:416) or
Join-Path \$env:USERPROFILE ".unsloth\\studio" (setup.ps1:1480).
Cycle 25 canonicalized only the legacy side, creating an
asymmetry under symlinked \$HOME / junctioned %USERPROFILE%.
Result of the asymmetry: a default-mode install on a host with
\$HOME=/tmp/link -> /tmp/real treats the legacy default as a custom
root, putting llama.cpp at \$STUDIO_HOME/llama.cpp instead of
~/.unsloth/llama.cpp -- and the Python backend's _find_llama_server_binary
(which uses .resolve() on both sides) then can't find the install.
Fix: canonicalize STUDIO_HOME on the fly at the comparison site, in
both setup.sh and setup.ps1. Symmetric with the now-canonicalized
legacy side from cycle 25, regardless of which mode set STUDIO_HOME.
The other two comparison sites (install.sh:704, install.ps1:469) are
already symmetric because they only run when STUDIO_HOME comes from
the env-override resolution path that already does pwd -P / Resolve-Path.
unsloth_cli/commands/studio.py + studio/backend/run.py + main.py +
llama_cpp.py already use .resolve() on both sides -- symmetric.
* install.ps1: env-override resolution uses .NET API for literal paths
Gemini code-review (review 4177641398, commit 2ea2c91) caught two
remaining New-Item -Path sites in the env-override resolution block
that the cycle 18 sweep missed:
- Line 123: New-Item -ItemType Directory -Path \$envOverride
- Line 132: New-Item -ItemType File -Path \$probe (writability test)
Both use -Path which interprets square brackets as wildcards. For a
user with UNSLOTH_STUDIO_HOME=C:\\workspaces\\studio[abc], both calls
would fail before the install starts. New-Item also has no
-LiteralPath in PowerShell 5.1.
Replace both with the .NET API:
- [System.IO.Directory]::CreateDirectory(\$envOverride)
- [System.IO.File]::WriteAllText(\$probe, "") -- closes the file
handle before the Remove-Item below.
End-to-end verified with /tmp/test-envoverride-[abc]-* path:
CreateDirectory + WriteAllText + Test-Path -LiteralPath all work.
* comments: condense multiline blocks added by this PR
Across the 27-cycle review process, comments accumulated as multiline
blocks explaining each fix's history (cycle numbers, prior bugs,
reviewer rationale). Compress every block to 1-2 lines that capture
just the WHY, dropping cycle references and history that belongs in
the PR description / commit log instead.
Net: 268 deletions / 124 insertions (-144 lines) of comments only.
Behavior unchanged. Verified: bash -n, pwsh parser, python ast.parse,
cargo check all pass.
* install.ps1: use 'return' over 'exit 1' for Install-UnslothStudio bail-outs
Per Gemini review #4177659001: when users run install.ps1 via
'irm ... | iex', 'exit 1' inside the function terminates the entire
PowerShell process and closes the user's terminal. 'return' bails out
of the function while keeping the shell open, matching existing error
sites at lines 34, 50, 57.
Three sites fixed: --tauri+env-override guard, env-override mkdir/access
failure, and write-probe failure. The 'exit' calls at lines 591/611
are inside a generated launcher here-string (a separate top-level .ps1
that runs as its own process), so they correctly stay as 'exit'.
* install.{sh,ps1}: address Gemini review #4177680451
Three medium fixes:
1. install.sh redirection detection: canonicalize both sides of the
$HOME vs passwd-DB comparison via 'CDPATH= cd -P -- ... && pwd -P'
so a trailing slash on $HOME (or symlink-vs-realpath mismatch with
getent/dscl output) doesn't misfire the redirection branch.
2. install.sh shim symlink: 'ln -sf' into an existing directory creates
the link INSIDE it ($_LOCAL_BIN/unsloth/unsloth instead of the
intended file). Pre-strip a real (non-symlink) directory at
$_LOCAL_BIN/unsloth before linking.
3. install.ps1 ShimExe: add -Recurse to Remove-Item so the launcher
refresh recovers if $ShimExe somehow exists as a directory rather
than a file (would otherwise drop into the catch and skip the
shim update).
* install.ps1: use 'throw' over 'return' for fatal validation failures
Cycle 28 reviewer.py (12/8 RC/APPROVE) caught a regression introduced
by the previous Gemini-review fix (#4177659001 -> commit 393e676b).
'return' inside Install-UnslothStudio kept iex'd terminals alive but
made 'pwsh -File install.ps1' exit with code 0 on fatal validation
failures (--tauri+custom-root rejected, STUDIO_HOME unwritable, etc.),
so CI / wrapper scripts treated failed installs as successful.
'throw' satisfies both constraints:
- pwsh -File install.ps1: exits with code 1 (CI sees failure)
- irm | iex: shows error to user, does NOT close the host terminal
Three sites: --tauri+env-override guard, mkdir/access failure,
write-probe failure. Verified throw -> exit code 1 under pwsh -File.
* install.ps1 launcher: single-quote child -Command path
Cycle 28 P2 finding: the generated launch-studio.ps1 builds the child
PowerShell -Command string with the executable path inside double
quotes, so a custom Studio root containing PowerShell metacharacters
(\$, backtick) re-expands in the child shell. Example:
D:\work\\\$job\studio -> child reparses \$job and runs the wrong path.
Fix: single-quote the path inside the child command and double any
apostrophes (PowerShell's literal-quote-escape form) so paths like
"O'Brien Studio & x|y" or "C:\work\\\$bad\studio" survive verbatim.
* install: harden custom Studio root handling
- install.sh shim refresh: refuse to recursively delete a real directory
at $_LOCAL_BIN/unsloth before creating the symlink. The previous rm -rf
could destroy unrelated user data living at that path.
- install.ps1 shim refresh: drop -Recurse from Remove-Item on $ShimExe and
refuse early when the shim path is a directory; mirrors the install.sh
guard so a directory at $StudioHome\bin\unsloth.exe is not blown away.
- install.ps1 PATH wiring: remove the redundant first $ShimDir prepend in
env-override mode; the post-Refresh-SessionPath prepend is the one that
takes effect, and the duplicate left $ShimDir in $env:Path twice.
- install.ps1 manual launch instructions: single-quote the printed shim
and Activate.ps1 paths so '$' / backtick metacharacters in custom roots
do not reparse when the user copies and pastes the command.
- studio/setup.sh: validate writability of UNSLOTH_STUDIO_HOME with the
same [ -w ] check install.sh already has, so a read-only override fails
with a clear message instead of an obscure uv pip permission error.
- Drop the STUDIO_HOME alias everywhere (storage_roots.py, studio.py,
install.sh, studio/setup.sh, install.ps1, studio/setup.ps1). The name
is too generic and an ambient STUDIO_HOME from unrelated tooling could
silently redirect the install. Only UNSLOTH_STUDIO_HOME is honored.
- unsloth_cli/commands/studio.py: defer UNSLOTH_STUDIO_HOME / UNSLOTH_LLAMA_CPP_PATH
re-export from import time into a helper invoked by the studio app
callback. Importing the module no longer mutates os.environ as a side
effect, so test runners and CLI introspection stop leaking those vars
into unrelated subprocesses.
- studio/backend/core/inference/llama_cpp.py: replace set-mutation inside
list comprehension with an explicit dedup loop for readability.
* install: harden custom Studio root edge cases
- install.ps1 shim refresh: move the directory-collision preflight outside
the lock-handling try/catch. The previous throw inside the try block was
swallowed by the surrounding catch and downgraded to a "Continuing with
the existing launcher" warning, leaving the install in a broken state
with no usable shim on disk.
- storage_roots.py / unsloth_cli/commands/studio.py: tighten the bin-shim
sentinel from .exists() to .is_file(). A directory at the candidate
bin/unsloth (or bin/unsloth.exe) path would otherwise false-positive
the venv inference and pick the wrong Studio root.
- storage_roots.py / unsloth_cli/commands/studio.py: wrap the env-var
override Path(...).expanduser().resolve() in try/except (OSError, ValueError),
matching the defensive pattern already used in studio/backend/main.py
and studio/backend/run.py. An invalid override (unresolvable network
drive, bad characters) now falls back to the un-resolved path instead
of crashing at import time.
* install: fail fast on missing custom root, allow brackets in shim path
- install.ps1 shim hardlink: switch the New-Item -ItemType HardLink call
from -Path to -LiteralPath so a custom Studio root containing bracket
characters does not fail under PowerShell's wildcard-aware -Path
parameter. Matches the -LiteralPath usage on every other Test-Path /
Remove-Item / Copy-Item call against the same shim path.
- studio/setup.sh override branch: replace the silent mkdir -p of the
override directory with an existence check that exits 1 with a clear
message. setup.sh runs against an existing install (via 'unsloth
studio update'), so a typo in UNSLOTH_STUDIO_HOME must not materialize
an empty workspace dir. Brings the Unix flow in line with setup.ps1,
which already errors on a missing override root.
* llama_cpp: scope orphan-server kill to the active install root
_kill_orphaned_servers used to unconditionally include the legacy
~/.unsloth/llama.cpp tree in install_roots, even when the running
Studio is in env-override mode and operates out of a custom root.
On a single OS user running both a default-install Studio and a
custom-root Studio concurrently, the custom Studio would kill the
default Studio's llama-server during startup orphan cleanup.
Hoist _is_custom_root out of the import try/catch so the legacy-
append decision sees it (default to False on ImportError so default
mode behaviour is unchanged), and gate the legacy ~/.unsloth/llama.cpp
append on `not _is_custom_root`.
* install: harden custom-root .venv migration and shim hardlink
- install.sh / install.ps1 OLD-layout .venv migration: gate on
default-mode only. Without the guard, pointing UNSLOTH_STUDIO_HOME at a
workspace that already has .venv (e.g. an unrelated Python project)
caused the torch validation to fail and the installer to recursively
remove the user's project venv. Mirrors the existing env-mode skip on
the CWD-relative venv migration immediately below.
- install.ps1 shim hardlink: revert to New-Item -ItemType HardLink -Path.
-LiteralPath is not accepted on the HardLink ItemType in any PowerShell
version, so the previous form always threw and silently fell back to
Copy-Item, breaking hardlink-update propagation. Bracket characters in
$ShimExe are still defended by the directory-collision preflight added
earlier.
- storage_roots.py / unsloth_cli/commands/studio.py: strip whitespace
from the UNSLOTH_STUDIO_HOME env var before the truthy check so a
blank " " override does not become a real path with trailing spaces
(which would silently break every downstream Studio path operation).
* Studio paths: tolerate stat / resolve failures during root inference
- storage_roots._infer_studio_home_from_venv: wrap the share/studio.conf
and bin/shim is_file() sentinel checks in try/except OSError. A
PermissionError on a restricted candidate dir would otherwise propagate
out of studio_root() and crash module import in run.py / main.py /
transformers_version.py / model_config.py at server startup.
- llama_cpp._kill_orphaned_servers: broaden the studio_root() guard from
ImportError-only to (ImportError, OSError, ValueError) so transient
resolve / sentinel failures do not crash the orphan-killer at server
startup. Matches _find_llama_server_binary's existing pattern.
- llama_cpp._find_llama_server_binary: nest the inner resolve() in its
own try/except and fall back to unresolved-path comparison instead of
dropping the custom search root entirely. A transient resolve() error
on the legacy path no longer loses the custom-root llama.cpp lookup.
* Add Studio install-root resilience tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: isolate custom-root installs from default-install state
- llama.cpp discovery in env-override mode no longer falls back to the
legacy ~/.unsloth/llama.cpp tree. The orphan-cleanup path already
excludes that root in custom mode; aligning discovery prevents a
custom-root Studio from launching a sibling install's binary it then
refuses to manage. Users who want a shared build set
UNSLOTH_LLAMA_CPP_PATH explicitly.
- Generated POSIX launcher (install.sh heredoc) namespaces LOCK_DIR with
a hash of DATA_DIR and persists the launched port to
$DATA_DIR/studio.port; in env-override mode the fast-path attaches only
to a port we ourselves wrote, never to a sibling Studio that happens
to be healthy on 8888..8908.
- Generated Windows launcher (install.ps1 heredoc) bakes a per-install
$portFile and SHA-256-suffixed mutex name, mirroring the POSIX side;
Find-HealthyStudioPort uses the port file in env-override mode.
- studio/setup.sh and studio/setup.ps1 require an .unsloth-studio-owned
marker before deleting $STUDIO_HOME/.venv_t5*, $STUDIO_HOME/llama.cpp,
and the sidecar T5 venvs in env-override mode. The marker is dropped
after fresh creation so subsequent runs of 'unsloth studio update'
proceed cleanly. Mirrors the existing .venv guard in install.sh.
- Wrap bare Path.resolve() calls on the legacy STUDIO_HOME constant in
studio/backend/main.py, studio/backend/run.py, and
unsloth_cli/commands/studio.py in the same try/except (OSError,
ValueError) used adjacently, so a restricted parent or recursive
symlink on $HOME does not crash module import / CLI startup.
* Studio: guard env-mode workspace against destructive cleanup
- install.sh and install.ps1 unconditionally rm -rf / Remove-Item the
new-layout $STUDIO_HOME/unsloth_studio when it has a python; in
env-override mode that path is a user-chosen workspace, mirroring
the .venv migration concern the .venv branch already guards. Refuse
to remove an existing $STUDIO_HOME/unsloth_studio that lacks Studio
sentinels (share/studio.conf or bin/unsloth).
- studio/setup.ps1 only checked Test-Path -PathType Container on the
custom root; setup.sh and install.ps1 both also write-probe via
WriteAllText / Remove-Item. Add the matching probe so 'unsloth
studio update' against an ACL-restricted root fails fast with a
clear message instead of erroring later while creating sidecar
venvs.
* Add Studio install/setup workspace-isolation tests
* Studio: tighten installer rationale comments
- install.sh: collapse a 5-line restatement into 3 lines, naming
env-mode behavior up front and the byte-identical pre-override
fallback after.
- install.ps1: correct misleading hardlink comment that claimed the
directory-collision preflight guards against wildcard expansion;
bracket characters in $ShimExe still glob-expand here, with the
Copy-Item -LiteralPath fallback handling them.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Split: keep only 2 file(s)
* Studio: harden env-mode workspace guards across installers and update path
Tightens the UNSLOTH_STUDIO_HOME custom-root protections so destructive
installer paths cannot displace unrelated user data when the override
points at a workspace.
install.sh / install.ps1: env-mode sentinel that gates rm -rf $VENV_DIR /
Remove-Item $VenvDir now requires share/studio.conf or the bin/unsloth(.exe)
shim to be a real file or symlink. Previously a directory at bin/unsloth or
bin\unsloth.exe satisfied the check (-e and bare Test-Path accept any path
type), so a workspace with unrelated content under unsloth_studio plus a
sibling directory at bin/unsloth could be wiped.
studio/setup.ps1: stale-venv rebuild branch now mirrors install.ps1's
env-mode guard before Remove-Item -LiteralPath $VenvDir -Recurse -Force.
Without this, "unsloth studio update" pointed at a custom workspace whose
unsloth_studio venv fails torch validation deletes the venv even when the
root carries no Studio sentinels.
studio/setup.sh / studio/setup.ps1: prebuilt llama.cpp install path now
calls _assert_studio_owned_or_absent / Assert-StudioOwnedOrAbsent before
invoking install_llama_prebuilt.py, and writes the .unsloth-studio-owned
marker on success. install_llama_prebuilt.py uses os.replace() to move
any existing install_dir aside before staging, so an unrelated
$STUDIO_HOME/llama.cpp could otherwise be displaced before the existing
source-build ownership guard ever ran.
* Studio: gate ownership guards on canonical custom-root and add venv marker
Tightens UNSLOTH_STUDIO_HOME ownership semantics so they fire only for a
genuinely custom root, never for an explicit override that resolves to the
legacy default. Adds an in-VENV marker that lets a partial install be
repaired and provides a strong primary sentinel for the deletion guard.
studio/setup.sh + studio/setup.ps1: hoist the canonical $STUDIO_HOME vs
legacy-default comparison so it sits next to the marker definition, derive
_STUDIO_HOME_IS_CUSTOM / $StudioHomeIsCustom once, and gate the
_assert_studio_owned_or_absent / Assert-StudioOwnedOrAbsent helpers and the
prebuilt llama.cpp marker writes on that flag instead of raw env-var
presence. UNSLOTH_STUDIO_HOME=$HOME/.unsloth/studio (legacy override) no
longer trips the guard for pre-PR T5 sidecar venvs or llama.cpp dirs that
predate the .unsloth-studio-owned marker. The duplicate canonical block
inside the llama.cpp section is removed; the new flag is reused.
studio/setup.ps1: Assert-StudioOwnedOrAbsent's marker check now requires
-PathType Leaf so a directory at .unsloth-studio-owned cannot satisfy it.
The in-place git-sync branch in the source-build path now calls
Mark-StudioOwned after a successful sync so a later prebuilt-update path
does not fail Assert-StudioOwnedOrAbsent on the same root.
install.sh + install.ps1: write $VENV_DIR/.unsloth-studio-owned right after
uv venv succeeds and accept it as the primary sentinel in the env-mode
deletion guard. This recovers from a partial install that was previously
unrepairable, and is a stronger sentinel than sibling shim files (the
marker is inside the venv that is about to be wiped, so an unrelated
workspace cannot accidentally satisfy it).
install.sh: drop the standalone -L test on $STUDIO_HOME/bin/unsloth in the
deletion guard. -L returns true for any symlink including symlinks to
directories and broken symlinks; -f already accepts the legitimate
file-targeted symlink shape created by ln -s at install.sh:1864.
* Studio: close residual workspace-isolation gaps for custom roots
Four follow-on hardenings that close the remaining cross-root leaks the
custom-root install plumbing still left open.
studio/setup.ps1 in-place git-sync: when the source-build path finds an
existing $LlamaCppDir/.git, it ran git remote set-url, checkout -B, and
clean -fdx in place before any ownership check. The previous fix marked
the tree as Studio-owned AFTER the sync but did not guard the BEFORE
case, so an unrelated workspace .git could be silently rewritten on the
first source-build under a custom UNSLOTH_STUDIO_HOME. Add the same
Assert-StudioOwnedOrAbsent guard already used by the prebuilt path and
the temp-dir swap path (gated on $StudioHomeIsCustom for parity).
Launcher port-file workspace isolation: the env-mode launchers' fast
path attached to any backend listening on the cached port that returned
a healthy /api/health, even when that backend belonged to a different
install root. studio/backend/main.py /api/health now returns the
resolved studio_root; install.sh _check_health and install.ps1
Test-StudioHealth verify it against UNSLOTH_STUDIO_HOME when set, so a
stale studio.port pointing at a sibling Studio is rejected instead of
opening the wrong UI.
studio/src-tauri preflight + commands: the Tauri desktop app stays on
the legacy root by design. process.rs / install.rs / desktop_auth.rs /
update.rs already strip UNSLOTH_STUDIO_HOME and STUDIO_HOME from their
CLI subprocesses, but preflight.rs run_cli_probe / probe_cli_capability
and commands.rs check_install_status did not, so a desktop launch from
a shell carrying those env vars produced status reflecting a different
root than the desktop manages. Mirror the existing scrub.
install.sh shim install: the previous `rm -f -- $_shim_path; ln -s ...`
pair leaves a window with no shim if interrupted. Use ln -sfn for an
atomic replace; the -n flag prevents descent into a symlink-to-directory
target (the existing directory guard above already rejects a real dir).
* Studio: replace launcher root verify with hex digest baked at install time
The previous launcher identity check returned the absolute resolved Studio
install root from /api/health and matched it against $UNSLOTH_STUDIO_HOME
in the launcher. Three problems that this commit closes:
- POSIX launcher used a raw bash `case` against the JSON-encoded value, so
paths containing characters that JSON escapes (e.g. /tmp/back\slash,
/tmp/O"Brien) caused the launcher to reject its own healthy backend.
- /api/health is unauthenticated and Studio supports `-H 0.0.0.0`, so any
reachable client could read the absolute install path (username, home
dir, workspace name, CI checkout path).
- The verification was gated on $UNSLOTH_STUDIO_HOME being set at runtime,
so a default-mode launcher would attach to a sibling env-mode Studio
listening on the same port instead of starting its own.
The fix replaces the raw path with a SHA-256 hex digest computed at install
time and baked into the generated launcher (mirroring how @@DATA_DIR@@ is
substituted today):
studio/backend/main.py: /api/health now returns `studio_root_id =
sha256(str(_studio_root()))` instead of the raw `studio_root` path.
install.sh: computes `_css_studio_root_id` once from $STUDIO_HOME using
python3, bakes `_EXPECTED_STUDIO_ROOT_ID='@@STUDIO_ROOT_ID@@'` into the
launcher heredoc, and adds `s|@@STUDIO_ROOT_ID@@|...|g` to the existing
sed pipeline for ALL modes (env / home / default). _check_health verifies
the baked id substring-matches the JSON response. Hex-only so no shell or
sed escape corner cases.
install.ps1: same shape on Windows. SHA256 the $StudioHome bytes, lower
hex, bake `$_ExpectedStudioRootId = '...'` into the launcher heredoc.
Test-StudioHealth now compares `$resp.studio_root_id -eq
$_ExpectedStudioRootId` unconditionally (no special-case for env-mode).
Default-mode launchers also bake their expected id, so two coexisting
Studio installs on the same machine can no longer cross-attach.
* Studio: harden launcher root-id and split install-time mode from runtime env
- install.sh launcher: compute studio_root_id with the venv Python (uv-managed
systems may not have system python3) and canonicalize STUDIO_HOME with
cd -P/pwd -P so default and home-redirect modes match the backend's
Path(sys.prefix).resolve() canonicalization. Fail fast instead of silently
baking an empty discriminator.
- install.sh launcher heredoc: gate PORT_FILE / namespaced LOCK_DIR on a baked
install-time mode flag (@@INSTALLED_IS_ENV_MODE@@) instead of the runtime
UNSLOTH_STUDIO_HOME variable so a sourced custom-root studio.conf cannot flip
a default-mode launcher into env-mode behavior with stale state.
- studio/backend/main.py: cache the studio_root_id digest at module load so
/api/health does not recompute hashlib + filesystem probes on every poll.
- studio/backend/core/inference/llama_cpp.py: widen the studio_root() probe
except clause from ImportError to (ImportError, OSError, ValueError) so it
matches the sibling _kill_orphaned_servers handler and tolerates Path.resolve
failures from broken symlinks or odd codecs.
* Studio: align launcher root-id digest with backend canonicalization
- studio/backend/main.py: hash the already-resolved _STUDIO_ROOT_RESOLVED
instead of recomputing str(_studio_root()); the default fallback in
storage_roots returns Path.home()/.unsloth/studio without .resolve(), so
on systems where $HOME is a symlink (NFS / AFS / Docker) the cached
digest now matches install.sh's cd -P/pwd -P canonicalization and the
launcher no longer rejects its own healthy backend.
- install.ps1: canonicalize $StudioHome via Resolve-Path before the SHA256
compute (env-mode already resolves at line 121, only default and profile
branches were raw); a junctioned USERPROFILE now produces the same digest
the backend computes via Path.resolve() for the same install.
- install.sh launcher template: substitute the non-user-controlled
@@STUDIO_ROOT_ID@@ and @@INSTALLED_IS_ENV_MODE@@ placeholders before the
user-controlled @@DATA_DIR@@ pass so a $DATA_DIR that contains the
literal placeholder text cannot be mutated by the second sed.
* Studio: tighten installer rationale comments
* Studio install: extend workspace-guard test coverage
Add behavioral coverage for env-mode workspace guards across install.sh,
install.ps1, studio/setup.sh, studio/setup.ps1, the launcher root-id
discriminator, and the backend's /api/health response. Also refresh the
custom-mode llama.cpp resilience assertion so it matches the implementation
that intentionally excludes the legacy tree from search_roots.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor STUDIO_HOME alias, fix workspace-guard test harness, harden rollback
The PR title and description promise STUDIO_HOME as a priority-2 alias
to UNSLOTH_STUDIO_HOME, but the implementation only read the longer name
in all six resolution sites. Wire the alias through install.sh,
install.ps1, studio/setup.sh, studio/setup.ps1, the Python storage_roots
resolver, and the unsloth_cli studio resolver. UNSLOTH_STUDIO_HOME wins
when both are set (more specific signal beats the generic alias).
Whitespace-only values are now treated as unset to match the Python
resolvers' .strip() semantics, preventing install/runtime layout drift
where the installer would create a literal " " directory while the
backend fell through to the legacy default.
Error messages and the substep status line report the env-var name the
user actually set ("UNSLOTH_STUDIO_HOME=..." vs "STUDIO_HOME=...") so
diagnostics stay accurate under either spelling.
Test harness fix: tests/test_studio_install_workspace_guard.py extracted
the install.sh venv-replacement block, but after the merge that block
delegates to _start_studio_venv_replacement (defined further up in
install.sh, not in the extracted snippet). Five sentinel-positive tests
echoed RESULT=ok but never moved $VENV_DIR. Add a single
_INSTALL_GUARD_STUBS constant that stands in a minimal mv-based stub
plus a no-op substep, and route every inline test script through a new
_build_install_guard_script() helper. All 50 tests now pass (was 45/50).
Rollback hardening: Start-StudioVenvRollback / Restore-StudioVenvRollback
/ Complete-StudioVenvRollback in install.ps1 used plain Test-Path,
Move-Item, Remove-Item against paths derived from $StudioHome. With a
custom UNSLOTH_STUDIO_HOME containing brackets (the very motivation for
the broader -LiteralPath sweep this PR set out to do), rollback would
silently misbehave under wildcard interpretation, turning a recoverable
install error into a destroyed env. Same fix for the --local Tauri
overlay block (Test-Path / Copy-Item / Get-FileHash on $VenvDir-derived
paths).
* Replace studio_root_id path-hash with per-install opaque id
The previous design computed studio_root_id as sha256 of the resolved
$STUDIO_HOME path, both at install time (baked into the launcher) and
at backend startup (returned via /api/health). This worked but had
three weaknesses:
1. Information disclosure on -H 0.0.0.0: anyone reaching /api/health
could confirm a guessed install path (username, workspace name,
etc.) by replaying the same hash.
2. Canonicalization brittleness: launcher (cd -P/pwd -P) and backend
(Path.resolve()) had to produce identical strings, which required
careful symlink/junction handling on every site (cycles 17-27 of
the PR review history were entirely about closing this drift).
3. Stale-launcher attach: an uninstall + reinstall at the same path
produced the same hash, so a launcher from the previous install
would silently attach to the new (incompatible) backend.
Replace the path-hash with a per-install opaque id:
- install.sh and install.ps1 generate 32 bytes from the platform CSPRNG
(/dev/urandom on POSIX with a python3 secrets fallback;
RandomNumberGenerator.Create().GetBytes on Windows) and persist it to
$STUDIO_HOME/share/studio_install_id with mode 0600. Atomic
temp-file-rename so a crash mid-install can't leave a half-written id.
The check 'if [ ! -s "$_css_id_file" ]' / Test-Path makes generation
idempotent across re-runs (so re-running install.sh doesn't invalidate
previously-baked launchers in the same install root).
- studio/backend/main.py replaces hashlib.sha256 with
_read_studio_install_id(), which reads $STUDIO_HOME/share/studio_install_id
once at module load. Validates the content against ^[0-9a-f]{64}$ so
malformed/truncated/uppercase/wrong-length content returns "" and
triggers the launcher's existing "no baked id, accept any healthy
Unsloth backend" fallback path.
- /api/health field name (studio_root_id) and wire format (64 hex chars)
preserved for compatibility with launchers already shipped via earlier
PR iterations.
Tests:
- Drop test_install_sh_root_id_matches_backend_resolved_under_symlinked_home
and test_install_ps1_canonicalizes_studio_home_before_root_id_hash --
the entire reason these existed (cd -P/Resolve-Path/Path.resolve()
digest agreement under symlinks/junctions) is moot when the id comes
from a file rather than from the path.
- Drop test_main_py_studio_root_id_hashes_resolved_root_not_unresolved
(no more hashing).
- Rewrite test_main_py_studio_root_id_caches_at_module_load to assert
the file-read pattern; add test_main_py_read_studio_install_id_validates_hex_and_handles_missing
to pin the exact rejection rules (empty / non-hex / wrong case /
wrong length all -> "").
- Rewrite test_install_sh_create_shortcuts_uses_venv_python_first as
test_install_sh_create_shortcuts_seeds_id_from_csprng_with_python_fallback
with a behavioral subprocess check that re-invocation is idempotent.
- Rename test_check_health_handles_path_with_backslash_via_hash to
test_check_health_handles_arbitrary_id_token (the JSON-escape concern
it pinned is preserved -- ids are hex-only by construction -- but the
test no longer derives the id from a path).
- Add test_install_sh_install_id_survives_symlinked_studio_home as a
regression test pinning that the new design has zero canonicalization
drift across symlinked parents.
- Update test_install_sh_bakes_studio_root_id_into_launcher and
test_install_ps1_bakes_studio_root_id_into_launcher to assert the
CSPRNG seed and the file location.
49/49 tests pass. Behavioral verification: install.sh-style generation
is idempotent across runs, three parallel installs at different roots
get distinct ids, reinstall at the same path produces a new id (so
stale launchers correctly fail to attach to the new backend), and
symlinked-\$HOME no longer causes launcher/backend disagreement.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <unslothai@gmail.com>
* Studio: add folder browser modal for Custom Folders
The Custom Folders row in the model picker currently only accepts a
typed path. On a remote-served Studio (Colab, shared workstation) that
means the user has to guess or paste the exact server-side absolute
path. A native browser folder picker can't solve this: HTML
`<input type="file" webkitdirectory>` hides the absolute path for
security, and the File System Access API (Chrome/Edge only) returns
handles rather than strings, neither of which the server can act on.
This PR adds a small in-app directory browser that lists paths on the
server and hands the chosen string back to the existing
`POST /api/models/scan-folders` flow.
## Backend
* New endpoint `GET /api/models/browse-folders`:
* `path` query param (expands `~`, accepts relative or absolute; empty
defaults to the user's home directory).
* `show_hidden` boolean to include dotfiles/dotdirs.
* Returns `{current, parent, entries[], suggestions[]}`. `parent` is
null at the filesystem root.
* Immediate subdirectories only (no recursion); files are never
returned.
* `entries[].has_models` is a cheap hint: the directory looks like it
holds models if it is named `models--*` (HF hub cache layout) or
one of the first 64 children is a .gguf/.safetensors/config.json/
adapter_config.json or another `models--*` subfolder.
* Sort order: model-bearing dirs, then plain, then hidden; case-
insensitive alphabetical within each bucket.
* Suggestions auto-populate from HOME, the HF cache root, and any
already-registered scan folders, deduplicated.
* Error surface: 404 for missing path, 400 for non-directory, 403 on
permission errors. Auth-required like the other models routes.
* New Pydantic schemas `BrowseEntry` and `BrowseFoldersResponse` in
`studio/backend/models/models.py`.
## Frontend
* New `FolderBrowser` component
(`studio/frontend/src/components/assistant-ui/model-selector/folder-browser.tsx`)
using the existing `Dialog` primitive. Features:
* Clickable breadcrumb with a `..` row for parent navigation.
* Quick-pick chips for the server-provided suggestions.
* `Show hidden` checkbox.
* In-flight fetch cancellation via AbortController so rapid
navigation doesn't flash stale results.
* Badges model-bearing directories inline.
* `chat-api.ts` gains `browseFolders(path?, showHidden?)` and matching
types.
* `pickers.tsx` adds a folder-magnifier icon next to the existing `Add`
button. Opening the browser seeds it with whatever the user has
already typed; confirming fills the text input, leaving the existing
validation and save flow unchanged.
## What it does NOT change
* The existing text-input flow still works; the browser is additive.
* No new permissions or escalation; the endpoint reads only directories
the server process is already allowed to read.
* No model scanning or filesystem mutation happens from the browser
itself -- it just returns basenames for render.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: cap folder-browser entries and expose truncated flag
Pointing the folder browser at a huge directory (``/usr/lib``,
``/proc``, or a synthetic tree with thousands of subfolders) previously
walked the whole listing and stat-probed every child via
``_looks_like_model_dir``. That is both a DoS shape for the server
process and a large-payload surprise for the client.
Introduce a hard cap of 2000 subdirectory entries and a
``truncated: bool`` field on the response. The frontend renders a small
hint below the list when it fires, prompting the user to narrow the
path. Below-cap directories are unchanged.
Verified end-to-end against the live backend with a synthetic tree of
2050 directories: response lands at 2000 entries, ``truncated=true``,
listing finishes in sub-second time (versus tens of seconds if we were
stat-storming).
* Studio: suggest LM Studio / Ollama dirs + 2-level model probe
Three improvements to the folder-browser, driven by actually dropping
an LM Studio-style install (publisher/model/weights.gguf) into the
sandbox and walking the UX:
## 1. Quick-pick chips for other local-LLM tools
`well_known_model_dirs()` (new) returns paths commonly used by
adjacent tools. Only paths that exist are returned so the UI never
shows dead chips.
* LM Studio current + legacy roots + user-configured
`downloadsFolder` from its `settings.json` (reuses the existing
`lmstudio_model_dirs()` helper).
* Ollama: `$OLLAMA_MODELS` env override, then `~/.ollama/models`,
`/usr/share/ollama/.ollama/models`, and `/var/lib/ollama/.ollama/models`
(the systemd-service install path surfaced in the upstream "where is
everything?" issue).
* Generic user-choice locations: `~/models`, `~/Models`.
Dedup is stable across all sources.
## 2. Two-level model-bearing probe
LM Studio and Ollama both use `root/publisher/model/weights.gguf`.
The previous `has_models` heuristic only probed one level, so the
publisher dir (whose immediate children are model dirs, not weight
files) was always marked as non-model-bearing. Pulled the direct-
signal logic into `_has_direct_model_signal` and added a grandchild
probe so the classic layout is now recognised.
Still O(PROBE^2) worst-case, still returns immediately for
`models--*` names (HF cache layout) and for any direct weight file.
## 3. model_files_here hint on response body
A leaf model dir (just GGUFs, no subdirs) previously rendered as
`(empty directory)` in the modal, confusing users into thinking the
folder wasn't scannable. Added a `model_files_here` count on the
response (capped at 200) and a small hint row in the modal: `N model
files in this folder. Click "Use this folder" to scan it.`
## Verification
Simulated an LM Studio install by downloading the real 84 MB
`unsloth/SmolLM2-135M-Instruct-Q2_K.gguf` into
`~/.lmstudio/models/unsloth/SmolLM2-135M-Instruct-GGUF/`. Confirmed
end-to-end:
* Home listing suggests `~/.lmstudio/models` as a chip.
* Browsing `~/.lmstudio/models` flags `unsloth` (publisher) as
`has_models=true` via the 2-level probe.
* Browsing the publisher flags `SmolLM2-135M-Instruct-GGUF` (model
dir) as `has_models=true`.
* Browsing the model dir returns empty entries but
`model_files_here=1`, and the frontend renders a hint telling the
user it is a valid target.
* Studio: one-click scan-folder add + prominent remove + plain search icon
Three small Custom Folders UX fixes after real-use walkthrough:
* **One-click add from the folder browser**. Confirming `Use this
folder` now submits the path directly to
`POST /api/models/scan-folders` instead of just populating the text
input. `handleAddFolder` takes an optional explicit path so the
submit lands in the same tick as `setFolderInput`, avoiding a
state-flush race. The typed-path + `Add` button flow is unchanged.
* **Prominent remove X on scan folders**. The per-folder delete
button was `text-muted-foreground/40` and hidden entirely on
desktop until hovered (`md:opacity-0 md:group-hover:opacity-100`).
Dropped the hover-only cloak, bumped color to `text-foreground/70`,
added a red hover/focus background, and sized the icon up from
`size-2.5` to `size-3`. Always visible on every viewport.
* **Plain search icon for the Browse button**. `FolderSearchIcon`
replaced with `Search01Icon` so it reads as a simple "find a
folder" action alongside the existing `Add01Icon`.
* Studio: align Custom Folders + and X buttons on the same right edge
The Custom Folders header used `px-2.5` with a `p-0.5` icon button,
while each folder row used `px-3` with a `p-1` button. That put the
X icon 4px further from the right edge than the +. Normalised both
rows to `px-2.5` with `p-1` so the two icons share a column.
* Studio: empty-state button opens the folder browser directly
The first-run empty state for Custom Folders was a text link reading
"+ Add a folder to scan for local models" whose click toggled the
text input. That's the wrong default: a user hitting the empty state
usually doesn't know what absolute path to type, which is exactly
what the folder browser is for.
* Reword to "Browse for a models folder" with a search-icon
affordance so the label matches what the click does.
* Click opens the folder browser modal directly. The typed-path +
Add button flow is still available via the + icon in the
section header, so users who know their path keep that option.
* Slightly bump the muted foreground opacity (70 -> hover:foreground)
so the button reads as a primary empty-state action rather than a
throwaway hint.
* Studio: Custom Folders header gets a dedicated search + add button pair
The Custom Folders section header had a single toggle button that
flipped between + and X. That put the folder-browser entry point
behind the separate empty-state link. Cleaner layout: two buttons in
the header, search first, then add.
* Search icon (left) opens the folder browser modal directly.
* Plus icon (right) toggles the text-path input (unchanged).
* The first-run empty-state link is removed -- the two header icons
cover both flows on every state.
Both buttons share the same padding / icon size so they line up with
each other and with the per-folder remove X.
* Studio: sandbox folder browser + bound caps + UX recoveries
PR review fixes for the Custom Folders folder browser. Closes the
high-severity CodeQL path-traversal alert and addresses the codex /
gemini P2 findings.
Backend (studio/backend/routes/models.py):
* New _build_browse_allowlist + _is_path_inside_allowlist sandbox.
browse_folders now refuses any target that doesn't resolve under
HOME, HF cache, Studio dirs, registered scan folders, or the
well-known third-party model dirs. realpath() is used so symlink
traversal cannot escape the sandbox. Also gates the parent crumb
so the up-row hides instead of 403'ing.
* _BROWSE_ENTRY_CAP now bounds *visited* iterdir entries, not
*appended* entries. Dirs full of files (or hidden subdirs when
show_hidden is False) used to defeat the cap.
* _count_model_files gets the same visited-count fix.
* PermissionError no longer swallowed silently inside the
enumeration / counter loops -- now logged at debug.
Frontend (folder-browser.tsx, pickers.tsx, chat-api.ts):
* splitBreadcrumb stops mangling literal backslashes inside POSIX
filenames; only Windows-style absolute paths trigger separator
normalization. The Windows drive crumb value is now C:/ (drive
root) instead of C: (drive-relative CWD-on-C).
* browseFolders accepts and forwards an AbortSignal so cancelled
navigations actually cancel the in-flight backend enumeration.
* On initial-path fetch error, FolderBrowser now falls back to HOME
instead of leaving the modal as an empty dead end.
* When the auto-add path (one-click "Use this folder") fails, the
failure now surfaces via toast in addition to the inline
paragraph (which is hidden when the typed-input panel is closed).
* Studio: rebuild browse target from trusted root for CodeQL clean dataflow
CodeQL's py/path-injection rule kept flagging the post-validation
filesystem operations because the sandbox check lived inside a
helper function (_is_path_inside_allowlist) and CodeQL only does
intra-procedural taint tracking by default. The user-derived
``target`` was still flowing into ``target.exists`` /
``target.is_dir`` / ``target.iterdir``.
The fix: after resolving the user-supplied ``candidate_path``,
locate the matching trusted root from the allowlist and rebuild
``target`` by appending each individually-validated segment to
that trusted root. Each segment is rejected if it isn't a single
safe path component (no separators, no ``..``, no empty/dot).
The downstream filesystem ops now operate on a Path constructed
entirely from ``allowed_roots`` (trusted) plus those validated
segments, so CodeQL's dataflow no longer sees a tainted source.
Behavior is unchanged for all valid inputs -- only the
construction of ``target`` is restructured. Live + unit tests
all pass (58 selected, 7 deselected for Playwright env).
* Studio: walk browse paths from trusted roots for CodeQL
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@h100-8-cheapest.us-east5-a.c.unsloth.internal>
* fix(studio): reuse HF cached repo casing to prevent duplicate downloads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move cache case resolution tests to separate PR
Tests for resolve_cached_repo_id_case and get_model_config case resolution
belong in their own PR to keep this change focused on the runtime fix.
* fix(studio): debug-log HF_HUB_CACHE fallback in path_utils
* Fix stale memoization in resolve_cached_repo_id_case
- Check exact-case path before memo to ensure a newly-appeared exact
match always wins over a previously memoized variant
- Validate memoized entries still exist on disk before returning them
to prevent stale results when cache dirs are deleted/recreated
* Minor cleanups for cache case resolution
- Use .is_dir() instead of .exists() for exact-case cache check
(cache entries are always directories)
- Remove redundant fallback in _detect_audio_from_tokenizer since
get_cache_path already handles case resolution and returns None
when the model is not cached
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
normalize_path() unconditionally converted Windows paths like
C:\Users\... to WSL format /mnt/c/Users/..., which breaks path
resolution on native Windows. This caused LM Studio GGUF models
to fail detection (detect_gguf_model returned None for the invalid
path), falling through to the Unsloth import path which requires
a GPU.
Now only performs the /mnt/ mapping when actually running under WSL.
On native Windows, drive letters are preserved and backslashes are
normalized to forward slashes.
* fix: default HF cache to standard platform path instead of legacy Unsloth cache
* feat: show LM Studio and local models in chat Fine-tuned tab
* feat: show LM Studio models in Hub models tab
* fix: fetch local models after auth refresh completes
* Revert "fix: fetch local models after auth refresh completes"
This reverts commit cfd61f0ac7.
* fix: increase llama-server health check timeout to 600s for large models
* feat: expandable GGUF variant picker for LM Studio local models
* fix: show GGUF variant label for locally loaded LM Studio models
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: show publisher name in LM Studio model labels
* fix: set model_id for loose GGUF files in LM Studio publisher dirs
* fix: show publisher prefix in Fine-tuned tab LM Studio models
* fix: only use model_id for lmstudio source models
* fix: only show LM Studio models in Hub tab on Mac/chat-only mode
* fix: respect XDG_CACHE_HOME, handle Windows paths in isLocalPath, refresh LM Studio on remount
- _setup_cache_env now reads XDG_CACHE_HOME (falls back to ~/.cache)
instead of hard-coding ~/.cache/huggingface. This follows the standard
HF cache resolution chain and respects distro/container overrides.
- isLocalPath in GgufVariantExpander uses a regex that covers Windows
drive letters (C:\, D:/), UNC paths (\\server\share), relative paths
(./, ../), and tilde (~/) -- not just startsWith("/").
- HubModelPicker.useEffect now calls listLocalModels() before the
alreadyCached early-return gate so LM Studio models are always
refreshed on remount. Also seeds useState from _lmStudioCache for
instant display on re-open.
* fix: add comment explaining isLocalPath regex for Windows/cross-platform paths
* fix: prioritize unsloth publisher in LM Studio model list
* fix: scope unsloth-first sort to LM Studio models on all platforms
* fix: add missing _lmStudioCache module-level declaration
* fix: prioritize unsloth publisher before timestamp sort in LM Studio group
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* feat: multi-source model discovery (HF default, legacy cache, LM Studio)
* Fix multi-source model discovery bugs
- Fix lmstudio_model_dirs: add ~/.lmstudio/models as default path,
remove dead sys.platform branch, add dedup via seen set
- Fix _setup_cache_env: preserve legacy HF cache env vars when the
legacy hub directory exists and is non-empty
- Fix _scan_lmstudio_dir: use absolute path for id field so
is_local_path() returns True
- Remove LM Studio dirs from allowed_roots (scanned unconditionally)
- Replace bare except passes with logger.warning in legacy cache blocks
- Fix delete_cached_model to search both default and legacy HF caches
- Make lmstudio_dirs non-optional in TS interface (matches Python schema)
- Exclude lmstudio source from trainable model filter
- Remove unused import sys
* Scan HF default cache alongside legacy and active caches
When _setup_cache_env overrides HF_HUB_CACHE to the legacy Unsloth
path, the standard HF default cache (~/.cache/huggingface/hub) was
never scanned, hiding models downloaded before Unsloth Studio was
installed.
Add hf_default_cache_dir() and _all_hf_cache_scans() helper that
deduplicates and scans all three HF cache locations (active, legacy,
default). Used in list_local_models, list_cached_gguf,
list_cached_models, and delete_cached_model.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat(db): add SQLite storage layer for training history
* feat(api): add training history endpoints and response models
* feat(training): integrate DB persistence into training event loop
* feat(ui): add training history views and card grid
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): address review issues in training history persistence
- Strip hf_token/wandb_token from config before SQLite storage
- Add UUID suffix to job_id for collision resistance
- Use isfinite() for 0.0 metric handling throughout
- Respect _should_stop in error event finalization
- Run schema DDL once per process, not per connection
- Close connection on schema init failure
- Guard cleanup_orphaned_runs at startup
- Cap _metric_buffer at 500 entries
- Make FLUSH_THRESHOLD a class constant
- Map 'running' to 'training' phase in historical view
- Derive LR/GradNorm from history arrays in historical view
- Fix nested button with div[role=button] in history cards
- Guard String(value) against null/undefined in config popover
- Clear selectedHistoryRunId on auto tab switch
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): address round-2 review findings across training backend and frontend
Backend (training.py):
- Move state mutation after proc.start() so a failed spawn does not wedge
the backend with is_training=True
- Create DB run row eagerly after proc.start() so runs appear in history
during model loading, not after first metric event
- Rewrite _flush_metrics_to_db() with snapshot-before-insert pattern to
preserve metrics arriving during the write and retain buffer on failure
- Guard eval_loss with float() coercion and math.isfinite(), matching the
existing grad_norm guard
- Increase pump thread join timeout from 3s to 8s to cover SQLite's
default 5s lock timeout
Frontend (studio-page.tsx):
- Fix history navigation: check isTrainingRunning instead of
showTrainingView in onSelectRun so completed runs are not misrouted
- Replace activeTab state + auto-switch useEffect with derived tab to
eliminate react-hooks/set-state-in-effect lint violation
Frontend (historical-training-view.tsx):
- Add explicit "running" branch to message ternary so running runs no
longer fall through to "Training errored"
- Derive loading from detail/error state and move cleanup to effect
return to eliminate react-hooks/set-state-in-effect lint violation
Frontend (progress-section.tsx):
- Derive stopRequested from isTrainingRunning && stopRequestedLocal to
eliminate react-hooks/set-state-in-effect lint violation and remove
unused useEffect import
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): resolve 3 remaining bugs from round-2 review
1. Stuck on Current Run tab [12/20]: Only force "current-run" tab when
isTrainingRunning is true, not when stale completed-run data exists.
After training ends, users can freely navigate to Configure.
2. Incomplete metric sanitization [7/20]: Apply float() coercion and
isfinite() guards to loss and learning_rate, matching the existing
pattern used by grad_norm and eval_loss. Prevents TypeError from
string values and NaN leaks into history arrays.
3. Stop button state leak across runs [10/20]: Add key={runtime.jobId}
to ProgressSection so React remounts it when a new run starts,
resetting stopRequestedLocal state.
* fix(studio): deduplicate loss/lr sanitization in training event handler
Reuse _safe_loss/_safe_lr from the progress update block instead of
re-sanitizing the same raw event values for metric history.
* fix(studio): restore loss > 0 guard to prevent eval steps injecting 0.0 into metric histories
Round-2/3 fixes relaxed the history append guard from `loss > 0` to
`loss is not None`, which let eval-only log events (where loss defaults
to 0.0) append fake zeros into loss_history and lr_history. Restore the
`loss > 0` check to match the worker's own has_train_loss gate. The
float() coercion and isfinite() sanitization from round-3 remain intact.
* fix(studio): resolve training history bugs — nullable loss/lr, tab nav, sparkline
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(recipe-studio): prevent fitView from zooming to wrong location on recipe load
* feat: add pymupdf/python-docx deps and unstructured uploads storage root
* feat: add POST /seed/upload-unstructured-file endpoint
* feat: add multi-file chunking with source_file column
* feat: update frontend types and API layer for multi-file upload
* feat: round-robin preview rows across source files
Ensures every uploaded file is represented in the preview table
by cycling through sources instead of just taking the first N rows.
* fix: disable OCR, fix auto-load timing, fix persistence on reload
- Disable pymupdf4llm OCR with write_images=False, show_progress=False
- Replace onAllUploaded callback with useEffect that detects uploading→done
transition (avoids stale closure reading empty file IDs)
- Fix importer to preserve file IDs from saved recipes instead of clearing
(clearing only happens at share time via sanitizeSeedForShare)
* fix: harden unstructured upload with input validation and state fixes
Validate block_id/file_id with alphanumeric regex to prevent path
traversal, use exact stem match for file deletion, add error handling
for metadata writes and empty files, fix React stale closures and
object mutations in upload loop, and correct validation logic for
unstructured seed resolved_paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: address PR review - legacy path import, share sanitizer, sync effect
Promote legacy source.path into resolved_paths for old unstructured
recipes, clear source.paths in share sanitizer to prevent leaking local
filesystem paths, and gate file sync effect to dialog open transition
so users can actually delete all uploaded files.
* fix: CSV column fix (BOM + whitespace + unnamed index re-save) for #4470
* fix: harden unstructured upload flow and polish dialog UX
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Set HF_HOME, HF_HUB_CACHE, HF_XET_CACHE, UV_CACHE_DIR, and
VLLM_CACHE_ROOT to a unified location under ~/.unsloth/studio/cache/
on startup. This keeps all model downloads, datasets, and caches
in one place instead of scattered across ~/.cache/huggingface,
~/.cache/uv, etc.
Layout:
~/.unsloth/studio/cache/
huggingface/ (HF_HOME)
hub/ (HF_HUB_CACHE -- model/dataset downloads)
xet/ (HF_XET_CACHE -- xet blob store)
uv/ (UV_CACHE_DIR -- uv package cache)
vllm/ (VLLM_CACHE_ROOT -- vllm compiled kernels)
Only sets variables that are not already in the environment, so
user overrides (e.g. HF_HOME=/data/models) are respected.
Cross-platform: uses Path.home() which resolves correctly on
Linux (~), macOS (~), and Windows (C:\Users\<user>).
Michael Han