6,358 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
alkinun
|
97475be347 |
fix(studio): support hostname-based enterprise proxies (#7416)
* fix(studio): support hostname-based enterprise proxies * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio): strip userinfo from proxy fetch targets --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> |
||
|
Leo Borcherding
|
3ea6d14c39 |
AMD: CI coverage for recent fixes, plus three wrong gfx ids (#7431)
* ROCm/AMD CI coverage: arch-table parity, native-Linux lib prepend, RDNA4 grouped_mm, discovery-based shell suite
Three merged ROCm fixes shipped without tests, and the CI wiring that
would have run them was gated on files the fixes do not touch.
Tests added (113):
tests/studio/install/test_rocm_arch_table_parity.py (27)
diffs the four duplicated gfx -> AMD pip-index tables across
install.sh, install.ps1, studio/setup.ps1 and install_python_stack.py,
plus the GPU-name -> arch tables and the torch 2.11 pin allowlist.
tests/studio/install/test_rocm_native_linux_lib_dirs.py (26)
covers #7233: system-ROCm lib dirs prepended ahead of bundled
libggml-hip, the /dev/kfd + not-WSL + libhsa gate, the opt-out env
var, root resolution order, and source parity between the two copies.
studio/backend/tests/test_grouped_mm_rdna4_fallback.py (46)
covers #7292: registration on the CUDA dispatch key, grouped and
ungrouped numerics, bias/dtype promotion, and the Linux HIP<7.13 +
RDNA4 name gate, executed from the shipped source rather than a copy.
tests/studio/test_ci_shell_suite_coverage.py (14)
fails if either shell runner goes back to a hardcoded list or skips
a file without a recorded reason.
CI wiring:
studio-backend-ci.yml: add install.sh / install.ps1 to the path filter
(the suites it runs assert against those two files, so install-only
changes -- the shape most AMD/ROCm routing fixes take -- skipped it),
and replace the 13-file hardcoded shell list with directory
discovery. That list had fallen seven files behind, including
test_strixhalo_wsl_reroute.sh, the only shell coverage of the ROCm
WSL reroute, which had never run on a PR.
tests/run_all.sh: same discovery loop so local and CI agree.
* Test review fixes: assert on outcomes, not on the code under test
Self-review of the previous commit found four tests that passed for the
wrong reason.
1. The arch-table parity test pinned expected gfx ids copied out of the
shipped tables, which enshrined three upstream inaccuracies as
correct: RX 9070 (non-XT) is gfx1201 not gfx1200, RX 7800 XT is
gfx1101 not gfx1100, and PRO V710 is gfx1101 not gfx1102 per AMD's
ROCm compatibility matrix. The expectation is now the AMD pip index
leaf -- the thing the tables exist to produce, and what a wrong
answer costs the user. The three known drifts are listed explicitly
with a test asserting they stay cosmetic, i.e. that the wrong and
right ids still map to the same wheel index. That test turns red the
day one of them starts routing users to the wrong wheel.
2. The RDNA4 device-name test extracted the regex from worker.py and
then matched with it, so it could not fail. Widening the pattern --
the dangerous edit, since it forces the slow Python mm fallback onto
RDNA3 users -- would have been silently accepted. It now reads the
live pattern and checks it against fixed cases, plus asserts the
name match stays guarded by `not _lin_arch` and that the name is
lowercased before matching.
3. The CI-coverage test matched a verbatim line of studio-backend-ci.yml,
so reindenting the step would fail the build while a real regression
to a hardcoded list could slip past a reformat. It now parses the
YAML, finds the step by name, and asserts on the glob plus the
absence of individual filenames. The path-filter test likewise reads
the parsed trigger instead of scanning raw text.
4. A set comprehension in the parity helper had a ternary whose branches
were identical.
Mutation-tested: widening the RDNA4 regex, desyncing one copy of the
name table, dropping install.sh from the path filter, and re-skipping
the ROCm WSL shell suite each fail at least two tests. Verified on
Linux (WSL Ubuntu 24.04) with CI's torch pin: 86 + 48 pass.
* Fix three wrong gfx ids in the GPU-name arch tables
The name -> gfx tables disagreed with AMD's ROCm compatibility matrix on
three entries. Corrected against the "Radeon GPU" list at
rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html:
RX 9070, RX 9070 GRE gfx1200 -> gfx1201 (Navi 48, same die as the XT)
RX 7800 XT, RX 7700 XT gfx1100 -> gfx1101 (Navi 32, not Navi 31)
PRO W7700 gfx1100 -> gfx1101
PRO V710 gfx1102 -> gfx1101 (Navi 32, not Navi 33)
No wheel changes for anyone: gfx1200/gfx1201 both resolve to gfx120X-all
and gfx1100/gfx1101/gfx1102 all resolve to gfx110X-all, in all four copies
of the index-family map. That collapse is why the errors survived being
copied into six places -- the leaf-level tests could not see them.
It was not purely cosmetic, though. install.sh's second copy feeds
"Tip: set UNSLOTH_ROCM_GFX_ARCH=<arch>", so a 7800 XT user following the
printed advice exported gfx1100 and made a wrong id authoritative for
every later run. It would also have become a real misroute the moment AMD
split a family across index leaves, as they already do for gfx1151/gfx1150.
Fixed in all six places, which is two more than the table's own "kept in
sync with" comments claim exist:
install.sh _infer_amd_gfx_arch_from_gpu_name
install.sh case "$_gpu_disp_mkt" (banner + env tip; undocumented)
studio/setup.sh
install.ps1
studio/setup.ps1
studio/install_python_stack.py
Ordering is preserved: the gfx1102 arm still precedes gfx1101 in the shell
copies so "RX 7700S" cannot fall onto the "RX 7700" glob, and the
PowerShell copies keep the (?!S) lookahead.
Test changes:
- test_rocm_arch_table_parity.py gains _AMD_DOCUMENTED_ARCH, exact gfx
ids transcribed from AMD rather than from the tables. Agreement between
six copies proves nothing when all six were transcribed from the same
mistake, so the ground truth has to come from outside. Verified it
catches the bug: against the pre-fix tables it fails 6 tests.
- The parity check now covers all six copies. It had four; the two
install.sh copies were being treated as one, and
_WIN_GPU_NAME_ARCH_TABLE was not checked at all.
- test_rocm_support.py's TestGfxArchNameFallback pinned two of the wrong
ids as expected values; updated, and extended with a 9060 XT and a
7900 XTX case so each RDNA3/4 die is represented.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard against unregistered copies of the GPU-name arch table
Counting the copies by hand is what let them drift: the in-code "kept in
sync with" comments claimed four, the arch-id fix found six, and scanning
the tree turns up a seventh.
TestNoUnregisteredArchTable rediscovers the copies from the source tree
instead of trusting a hand-maintained list. A table line is one that names
a card and gives its arch; real tables score 9-17 such lines and the only
other hits in the repo are two single-line prose comments, so the
three-line threshold is not load-bearing. A companion test asserts the
scan still finds the known copies, so the heuristic cannot go blind and
pass by finding nothing.
The seventh copy is tests/_zoo_rocm_spoof.py, the fixture other ROCm tests
build their fake AMD host from. It states the mapping backwards (gfx ->
the name torch should report), which makes it an independent witness: it
had gfx1101 -> RX 7800 XT and gfx1201 -> RX 9070 XT right while all six
installer copies were wrong, and nothing compared the two. Now they are
round-tripped against each other.
RX 6700 XT is pinned as a known divergence rather than normalised. AMD's
compatibility matrix documents no consumer RX 6000 card and no gfx1031 at
all, the installer arm is commented "gfx103X family", and gfx1031 appears
only as an index-family key, never as a value a name table emits. With no
external source to correct against, changing shipped behaviour would be
guesswork. A test fails if the divergence ever disappears, so the
exemption cannot go stale.
Also adds the reverse of the AMD-matrix check: a documented card that
matches no arm anywhere is a silent CPU fallback rather than a wrong id.
This cannot detect hardware nobody transcribed, which would need a live
fetch of AMD's matrix and a non-hermetic suite; the docstring says so
rather than implying coverage that is not there.
Verified on Linux: 478 passed, plus all five new guards mutation-tested
to confirm each fails when its invariant is broken.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Docstring said six copies; the list under it now has seven
* tests: run discovered shell tests with bash, not sh
tests/run_all.sh discovered tests/sh/ instead of listing files, but still
invoked each one with sh. Every file there declares a bash shebang, and on
Debian/Ubuntu /bin/sh is dash: test_apt_distro_prompt.sh,
test_studio_home_node_dir.sh and test_with_llama_cpp_dir_link_behavior.sh
fail on bashisms under dash and pass under bash. The old hand-written list
happened to name only dash-clean files, so switching to discovery is what
surfaced it. Backend CI already used bash, so this was a local-only break.
Guarded by a new test asserting both runners invoke tests/sh/ with bash.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Krackan Point (Radeon 860M/840M) routed to the gfx1150 wheel index
The GPU-name tables map 860M/840M and the Ryzen AI 7 350 / AI 5 340 CPU
strings to gfx1150, but Krackan Point is gfx1152. AMD's own lemonade table
(src/cpp/server/system_info.cpp) maps both Krackan iGPUs to gfx1152.
Unlike the three ids already fixed here, this one is not wheel-neutral:
repo.amd.com publishes gfx1150 and gfx1152 as separate index leaves with
separately built torch wheels, so these laptops were installing wheels
built for a different LLVM target. gfx1152 was absent from the codebase
entirely, so it needed the index-family maps, the torch 2.11 floor lists
(same _grouped_mm bug as gfx1150/1151), the Strix reroute set and the
Windows arch allowlist as well as the seven name tables.
The parity test added in this PR did not catch it because its AMD-matrix
expectations stopped at 890M/880M. Added the APU rows, so the case that
actually changes a wheel is now covered: reverting the tables fails 9
tests naming 860M, 840M and Krackan.
gfx1153 (Ryzen AI 5 430 era) is left alone; AMD publishes no gfx1153
wheel family, so there is nothing to route it to.
Verified: bash -n on both shell installers, PowerShell AST parse on both
.ps1 files, python ast.parse on all touched modules, install suite 1334
passed with no new failures against main, shell suite 20 files.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add gfx1152 to unified-memory classifiers, make parity allowlist set-based
Krackan Point (gfx1152, Radeon 860M/840M) is the third RDNA 3.5 APU and
shares one GPU/system-RAM pool exactly like Strix Point (gfx1150) and
Strix Halo (gfx1151), but only the installers knew about it. The two
runtime classifiers still had two-element arch sets, so a Krackan laptop
got the 0.90 discrete headroom factor on a shared pool and ran llama.cpp
without GGML_CUDA_ENABLE_UNIFIED_MEMORY.
- worker.py _rocm_classify_unified_memory: add gfx1152 to the arch set,
and 860m/840m to the device-name fallback. The NVIDIA GeForce 840M
cannot collide there: the function is only reached under _hw.IS_ROCM.
- llama_cpp.py _amd_apu_wants_unified_memory: add gfx1152 to the arch set.
- Tests for both, including the :sramecc-:xnack- suffix form.
TestGfx211AllowlistParity compared four hardcoded allowlist strings, so
adding gfx1152 to all four installers correctly turned three assertions
red without any installer actually disagreeing with another. Each test
now extracts the set its installer holds and compares it to one EXPECTED
constant. Order and spacing are free, membership is not, and the next
leaf is a one-line edit instead of four.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
|
||
|
Daniel Han
|
85f6231a2f |
tests: anchor the gguf ordering assertion on the branch that owns the marker (#7443)
_load_model_impl contains more than one `if config.is_gguf:`, so source.index() returned the earlier one, which belongs to a different check than the branch the assertion is reasoning about. The inheritance call sits at line 4543, the earlier branch at 4508 and the branch holding the load marker at 4567, so the comparison read 186995 < 185014 and failed on main. The branch is now located from the load marker itself, which is the landmark the rest of the test already relies on, so the assertion compares the inheritance call against the branch that actually guards it. The slice used by the following assertions is anchored the same way, which also tightens them: they previously searched from the earlier branch to end of file. The invariant is unchanged and still has teeth: moving the inheritance call after the branch makes the assertion fail. Co-authored-by: danielhanchen <unslothai@gmail.com> |
||
|
Daniel Han
|
c4b777263d |
fix(studio/colab): fix OutStream startup crash and tidy the notebook cards (#7404)
* fix(studio/colab): survive ipykernel OutStream close() during startup Unsloth Studio crashed at server startup on Colab with: Unsloth Studio failed to start: 'OutStream' object has no attribute 'watch_fd_thread' Root cause: - Colab's ipykernel OutStream is created with watchfd=False, so it never gains a watch_fd_thread. The OutStream.close() in the affected ipykernel versions joins that thread unconditionally and raises AttributeError (ipython/ipykernel#867). - _setup_server_disk_logging() replaces sys.stdout/sys.stderr with a tee. That changes the console object identity, so Colab's absl logging handler (which captured the original OutStream and whose close() deliberately skips sys.stdout/sys.stderr) no longer treats it as the live console. - run_server builds uvicorn.Config(...), whose configure_logging runs logging.config.dictConfig -> logging.shutdown, closing every existing handler. The absl handler then calls close() on the orphaned OutStream and the AttributeError propagates out of uvicorn.Config and aborts startup. Fix: - Before installing the tee, harden the displaced console streams' close() so only the ipykernel#867 AttributeError is swallowed; a healthy close() runs unchanged and any other error still propagates. The buggy close() raises before it nulls pub_thread, so the stream stays fully usable. - Give _TeeStream its own close() that flushes the log copy and forwards close() to the wrapped console stream best-effort, so a handler that captured the tee cannot crash startup either. Add regression tests reproducing the exact path (an absl-style handler closing a watchfd=False OutStream stand-in during logging.shutdown) and asserting the tee/console path survives and keeps logging. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Show the Colab login password in the shareable link card * Tighten Colab card comments for PR #7404 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make the Colab tunnel URL clickable and emphasise the password * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Narrow the console close() hardening to the watch_fd_thread AttributeError * Put the Colab password on its own line so selection excludes the label * Keep the Colab password as plain selectable text --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
2d026a1184 |
Studio: reset quantized KV cache to f16 when the flash-attn-off crash-recovery fallback fires (#7390)
* Studio: reset quantized KV cache to f16 when flash-attn-off fallback fires Studio force-enables --flash-attn on for GGUF launches. On a hard startup or first-decode crash it retries via _with_flash_attn_off, which flipped FA off but left --cache-type-k/-v untouched. A quantized KV cache (q8_0, q4_0, q4_1, q5_0, q5_1, iq4_nl) requires flash attention in llama.cpp, so the retry itself aborted at init with 'V cache quantization requires flash_attn' instead of recovering. Reset any quantized --cache-type-k/-v to f16 in the FA-off fallback path so the retry can actually launch. Non-quantized types (f16, bf16, f32) run fine without flash attention and are left unchanged. Handles long and short flag forms and both space and equals syntax, rewriting in place to preserve list length. Adds pytest coverage. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: FA-off fallback resets only the quantized V cache and drops env-only V cache Only the V cache requires flash attention in llama.cpp; a quantized K cache runs fine without it. Restrict the FA-off crash-recovery reset to the V axis (main and draft) so a memory-constrained config keeps its quantized K cache instead of risking an OOM on the recovery. Also drop an inherited quantized V cache set purely through the environment (LLAMA_ARG_CACHE_TYPE_V / LLAMA_ARG_SPEC_DRAFT_CACHE_TYPE_V) at the FA-off retry sites, which the argv rewrite cannot reach, so the child falls back to the f16 default rather than aborting. * Studio: normalize underscore V-cache aliases in the FA-off fallback llama.cpp rewrites '_' to '-' for any '--' long option before matching, so a pass-through --cache_type_v q8_0 enables a quantized V cache just like --cache-type-v. The FA-off crash-recovery reset only matched the hyphenated spelling, so the underscore alias slipped through and the retry still aborted with "V cache quantization requires flash_attn". Canonicalize the flag name the same way before matching (short flags and the type value are untouched). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
oobabooga
|
b9d92c41b3 |
Studio: prevent long reasoning from jumping the chat on completion (#7388) | ||
|
Souravrajvi0
|
dffea2af27 |
fix(studio): honor run settings on initial model load (#7346) (#7351)
* fix(studio): honor run settings on initial model load When loading a model from the gear-icon run-settings page, Context Length and KV Cache Dtype were ignored if the user clicked Load before blurring the context field, or before React flushed staged config into the store. - Add NumericValueInput.commit() to flush a focused draft on Load - Pass effectiveLoadConfig from model-config-page to onRun - Prefer selection.config in performLoad for all load knobs - Preserve meta.forceReload from the config-page reload path Fixes #7346 * fix(studio): flush NumericValueInput draft when Load blurs first Clicking Load blurs the context field before handleRun runs, so commit() returned the stale value prop. Keep draft in a ref and parse it even when the input is no longer focused. * fix(studio): preserve Auto context when Load is clicked without edits NumericValueInput.commit() now returns null unless the user actually changed the field, so GGUF Load/Save no longer pins the displayed native context into customContextLength when Auto was left untouched. * fix(studio): clear NumericValueInput dirty state after blur commit After a normal blur commit, reset dirtyRef so a later Load cannot replay a stale draftRef when the user changed context via Reset or the slider. * test(studio): pin NumericValueInput Auto/dirty contracts for #7346 Lock Codex P1/P2: commit returns null unless dirty, blur clears dirtyRef, and handleRun only promotes a non-null committed context. * fix(studio): keep same-click context draft after blur (#7346) Blur can commit and clear dirtyRef before Load's onClick; stash that committed value for one imperative commit() so typed context is not lost. * chore: refresh PR head for #7351 * fix(studio): handle context commit edge cases * chore: refresh PR head * test(studio): guard invalid context drafts * style(studio): format context draft guard * test(studio): exercise same-click model config loads * fix(studio): drop stale blur pin when the typed context equals the shown value NumericValueInput cached every blur commit in lastBlurCommittedRef, even when the draft equalled the current value and no onChange was dispatched. Because the displayed value never changed, the useEffect([value]) clear never fired, so a later Reset or external edit that leaves the shown value unchanged could not drop the cache and the next commit() replayed it into an override that Reset had removed. Only cache the blur result when it actually dispatched onChange (final !== value); when final === value the parent is already current and there is nothing to bridge. Add a Playwright regression that re-types the shown context and asserts no override is stored. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: commit every same-click numeric draft before staging the load config The run-settings Load/Reload button flushed only the GGUF Context Length draft imperatively before building the load config. Max Seq Length (non-GGUF), GPU Layers and MoE Layers on CPU (GGUF) are the same NumericValueInput and stage their typed value only on blur, so editing one and clicking Load in the same gesture staged the load from a still-stale parent config and dropped the value the user just typed. Wire an imperative commit handle through those inputs too and fold every committed draft into the effective config, recomputing the non-GGUF load-time max sequence length from the committed draft. * fix(studio): recompute fixed-layer context pin and drop stale blur cache on every render Two run-settings edge cases on the model-config page: 1) pinFixedLayerContext was computed from the render-time config, before a same-click GPU Layers draft is committed in handleRun. Typing a positive fixed-layer value on an auto-fit GGUF and clicking Reload therefore built the runtime config with customContextLength: null, so a later fresh load sent the native context with fixed layers (the OOM the pin exists to avoid). Recompute the pin from the committed effectiveConfig. 2) NumericValueInput cleared its blur bridge only on a value change. A real edit (final !== value) that Reset then reverts to the same shown number nets value back unchanged, so the effect never re-ran and the stale pin survived into the next Load/Save, replaying the override Reset removed. The bridge is only valid across the single synchronous same-click gesture that set it, so clear it on every settled render instead. Add source-contract regressions for both. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
95f42bccee |
tests: restore the inheritance-before-guard ordering assertion (#7251)
The gguf order fix that landed on main dropped the only assertion
covering the prerequisite that llama_extra_args inheritance runs before
the GGUF branch: the inherited value (a carried --no-mmproj) shapes the
hub guard's require_mmproj, so a future reorder could reject a load
over an mmproj download the inherited arguments would disable. The
comment also misattributed the inheritance site to
_guard_chat_load_against_training.
The assertion is restored anchored on the call form
"= _resolve_inherited_extra_args(", which pins the endpoint's call site
(the bare name would match the function definition, which always
precedes the endpoint, making the check vacuous), and the comment now
names the real inheritance site. 32 tests pass.
|
||
|
Leo Borcherding
|
478d30f361 |
Unsloth Studio (desktop): fix canvas preview, download file button, toast placement, and model-load typing lag (#7391)
* Studio desktop: fix loading-toast overlap and typing lag on model load - Toaster: on desktop, offset toasts below the ~34px custom window titlebar (top 46 when isTauri) so they no longer cover the min/max/close controls. Web is unchanged (top 12). - Model load: the 2s load poll wrote loadProgress state every tick, which re-renders the whole chat page during "Starting model" (cheap in Chrome, janky in the desktop WebView2 -> laggy typing). That state is only read by the dismissed-toast inline status, so gate all four poll branches to write it only when the inline view is live; while the toast is up it updates via Sonner alone. * Studio desktop: fix HTML canvas preview, download, and panel offset - CSP: add frame-src for localhost/127.0.0.1 so the desktop webview can frame the backend-served artifact preview. default-src 'self' (no frame-src) blocked it -> "127.0.0.1 refused to connect"; web is same-origin so it already worked. - Download: route the canvas Download button through the native save dialog (downloadFile) instead of a blob-anchor click, which the Tauri WebView2 silently drops. - Nudge the artifact panel down 8px so its top edge/shadow isn't tucked under the window top bar. * Studio desktop: add HTML filter for native canvas save dialog Canvas Download saves .html via save_native_file, but save_filter() had no html/htm case, so the native dialog fell back to the JSON/CSV/etc filter and could block saving/browsing the .html export. Add an HTML filter and include html/htm in the catch-all. Addresses Codex review on #7391. * Studio desktop: unblock canvas preview in dev shell + clear header fade - Preview: the app CSP frame-src fix wasn't enough in the tauri dev shell. The preview endpoint sets its own frame-ancestors response header, which only allowed 'self' tauri://localhost http://tauri.localhost -- so the Vite dev origin (http://localhost:5173) was blocked and the frame stayed "refused to connect". Extend the allowlist with http://localhost:* and http://127.0.0.1:* (the endpoint only renders postMessage'd HTML in a no-same-origin sandbox, so it exposes no server resource). - Shadow: the artifact panel toolbar sat under the full-width chat-header-fade; lower the panel top (mt 80->90px) so the controls clear the fade. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Leo Borcherding
|
938e786eb9 |
Recipe Studio: full-height canvas and in-app maximize control (#7394)
* studio recipes: full-height canvas and in-app maximize control - Recipe editor fills its container (drop the outer padding and the fixed 75vh height); the canvas reaches the window edges - Viewport controls: the fit button now reads as center (it always fit/centered); add an expand-to-full-view button that collapses the sidebar and maximizes the canvas in-app, toggling back to restore * recipe studio: exit full view when leaving the editor tab Addresses review: the Exit full view control lives inside the editor canvas, which unmounts on the Easy/Runs tabs. Clear maximized (and restore the sidebar) when activeView leaves "editor" so those views aren't left stuck under the fixed full-view overlay. * recipe studio: keep full view below titlebar and off the sidebar state |
||
|
oobabooga
|
91a89806d7 |
Studio: prevent empty responses after model thinking (#7418)
* Fix reasoning-only Qwen3.6 completions in Studio * Address reasoning-only review findings * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Michael Han
|
0e3e4f3180 |
Studio: scale menu, toast, chat and composer icons with the UI font size (#7400)
* Studio: scale menu, toast, chat and composer icons with the UI font size Glyphs that sit beside scaled labels now follow the preference: the shared --icon-size token (nav, settings tabs, chat action bars, code block actions), classed svgs inside dropdown, select, context, menubar, popover and command surfaces, toasts, the chat thread and both composers, and the composer pill glyph slot. Sonner toast text is unpinned from its injected 13px. Hit targets, paddings and surface geometry stay fixed and every value is identity at the default size. * Studio: icons scale at half the UI font size rate; cover review gaps Icons now follow the preference at half the rate of the text, matching the logo lockup: base + (setting - 16) / 2. The menu specific rules that outranked the scoped block (app-user-menu, unsloth-plus-menu, unsloth-tick) carry the scale too, which also restores the plus menu's intended 1.15rem glyph base at the default size. From review: closed select triggers join the scoped surfaces so their chevron tracks the label, sonner action button labels scale at full text rate alongside the title and description, and the unused built-in sonner loader gets a defensive size override. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: icons match the text scale below the default, half rate above Piecewise icon scaling: below the 16px default icons follow the UI font size at the full text rate, above it they move at half the rate so glyphs stay slightly smaller than the text. Written as min(full, half) since the smaller branch is correct on each side. Applies to the shared --icon-size token, the scoped menu, toast, chat and composer overrides, and the menu rules that outrank them. * Studio: cap icons at their default size above the 16px setting Below the default icons still match the text scale; above it they now keep their default size instead of growing at half rate, so enlarged text dominates and glyphs read slightly smaller than the text. The curve is min(full rate, base). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: icons above the default scale at half rate, not capped A 16px glyph at setting 20 renders 18px, as if the setting were 18: above the default icons move at half the rate of the text, below it they match the text scale. The curve is min(full rate, half rate). * Studio: standard icons render at the UI font size itself One shared --ui-icon-size token replaces the per-base curves for every glyph with a 16px or larger base: icons match the UI font size below the default and grow at half the change above it, so setting 12 gives 12px icons, 16 gives 16px and 20 gives 18px, slightly smaller than the enlarged text. Sub 16px glyphs keep their proportions through the same curve as a factor. This also slims the previous 18px to 21px icon bases down to the font size at the default setting. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: icon scale review fixes for ticks, comboboxes and art glyphs From review: thinking ticks keep their own size inside plus menus (the important menu rule now excludes them), combobox popups and triggers join the scoped surfaces, 24px size-6 art glyphs such as attachment tile icons go back to proportional scaling instead of the uniform token, branch picker 36px chevrons scale proportionally beside their counter, and buttons that default un-classed icons to size-4 get the shared token (xs buttons keep their pinned small icons). Sonner cancel labels already scale: sonner renders cancel with data-button set, so the existing override reaches it. * Studio: keep the toast close glyph compact The button icon fallback matched Sonner's close button, whose unclassed 12px X then rendered at the shared icon size inside its fixed control. Exclude data-close-button from the fallback. * Studio: use text-ui-11 for the new chat settings sheet caption The raw px guard caught a text-[11px] added on main; raw px text ignores the UI font size preference. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Long Yixing
|
1bd080208c |
Fix Studio model picker toolbar overflow (#7403)
* fix(studio): contain model picker toolbar * fix(studio): preserve model picker tab icons |
||
|
Michael Han
|
275c046c09 |
studio: use Hugeicons AI Security glyph for Run automatically (#7409)
Swap the lucide CircleOff icon on the Run automatically permission mode for the Hugeicons AI Security 03 glyph, matching the app's existing Hugeicons usage. A small lucide-compatible wrapper lets it drop into the option list. Icon-only change, no behavior change. Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> |
||
|
Lionel Arce
|
a1907fd4fe |
feat(studio): add DoRA support to studio (#7315)
* feat(studio): add DoRA support to studio * fix: added use_dora fast encoder LoraConfig and gated use_dora on AdapterMethod * fix(studio) serverside normalization for use_dora=true - add note documenting use_dora is silently dropped on diffusion * fix: dora button disabled on mac, add preflight guard on GGUF lora export, mismatch now correctly falls through to existing error instead of silently no-opping * Studio: add dora to the WizardState LoRA variant union for consistency * Reject --use_dora on the MLX (Apple Silicon) CLI path --------- Co-authored-by: danielhanchen <unslothai@gmail.com> |
||
|
Souravrajvi0
|
434fac6ffc |
feat(studio): presets include load settings (#7347) (#7352)
* feat(studio): save load settings in chat presets Presets previously stored only sampling params (temperature, top_p, etc.). Extend them with an optional loadConfig blob that captures context length, KV cache dtype, speculative decoding, and GPU layer knobs from the current runtime when saving. - Apply loadConfig when switching presets or hydrating on startup - Show a short summary under the preset controls - Prompt to reload when a model is already loaded Fixes #7347 * fix(studio): persist preset loadConfig and capture GGUF context Add ChatPresetLoadConfig to the chat settings API schema so presets with load settings no longer 400 on save. Capture effective GGUF context from ggufContextLength when customContextLength is cleared after auto-mode load. * fix(studio): address Codex review on preset load settings Coalesce default maxSeqLength/speculative/gpu knobs when capturing presets, no-op apply for legacy presets without loadConfig, preserve GPU pin on apply, and stop replaying stale loadConfig during settings hydration. * Remove unused getOrderedPresets import --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Souravrajvi0
|
d17567af3e |
fix(studio/colab): restore blank Colab iframe embed (#7344) (#7349)
* fix(studio/colab): restore iframe embed via serve_kernel_port_as_iframe Colab's output sanitizer often strips custom <iframe> tags from IPython.display.HTML without raising, leaving a blank cell even though display() succeeded. The kernel-port helper is the supported embedding path and registers the proxy correctly. - Prefer serve_kernel_port_as_iframe; keep raw HTML iframe as fallback - Always show the clickable link card via show_link() so the proxy URL is visible even when iframe embedding fails - Add regression tests for embed ordering and URL truncation Fixes #7344 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio/colab): harden iframe embed fallbacks per Codex review Guard show_link so a display failure cannot skip embedding, and only use serve_kernel_port_as_iframe when get_colab_url returned a real Colab proxy URL so localhost/colabtools environments still get the HTML iframe path. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio/colab): stop opening Colab proxy URLs in a new tab (#7349) Colab *.prod.colab.dev proxy hosts are session-scoped and return HTTP 404 when opened as a top-level tab or from another device. Replace the clickable Open button for those URLs with an in-notebook ready card, keep serve_kernel_port_as_iframe for the UI, and point users at start(cloudflare=True) for a real shareable / new-window link. * fix(studio/colab): use kernel iframe on real Colab when eval_js fails (#7349) Gate serve_kernel_port_as_iframe on COLAB_RELEASE_TAG + google.colab import instead of a successful proxyPort URL. When eval_js fails and get_colab_url falls back to localhost, real Colab notebooks still embed via the kernel helper (port-only). colabtools without COLAB_RELEASE_TAG keeps the HTML iframe path. Thanks @mfielding92 for the runtime diagnosis. * Mock top-level google package in Colab embed tests * test(studio/colab): mock top-level google package in Colab tests Patching only sys.modules["google.colab"] fails when no google namespace is installed: import google.colab resolves the parent first and returns False in _is_colab_runtime(). Add a shared helper that mocks both google and google.colab for deterministic tests across environments. * Tighten comments in Colab embed helpers and tests * fix(studio/colab): default Cloudflare on Colab with durable login credentials Colab proxy iframes often load an empty document even when the kernel helper appends the frame, leaving users unable to reach Studio to change the bootstrap password and blocking start(cloudflare=True). On real Colab runtime: - Default cloudflare to True (pass cloudflare=False to opt out) - Finalize the random admin password and print credentials in the notebook - Persist credentials across cell re-runs after interrupt - Show Cloudflare link before login credentials; skip blank proxy iframe when ready - Reuse main._IS_COLAB for runtime detection (not COLAB_RELEASE_TAG alone) - Only trust serve_kernel_port_as_iframe on real Colab; colabtools falls back to HTML - Keep embedding when the link card display fails Addresses Codex review feedback on #7349 and @mfielding92's catch-22 report. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio/colab): skip credential finalize when cloudflare=False Only call _finalize_colab_admin_password() when opening a Cloudflare tunnel. start(cloudflare=False) should not clear the bootstrap-password gate or show a login card that references a missing tunnel link. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio/colab): drop stale cached Colab credentials after password change On a Colab rerun the finalize path redisplayed the cached first-run password whenever the bootstrap gate was already cleared. If the admin changed the password through the app, that cached copy no longer authenticates, so the notebook printed dead credentials. Validate the cached password against the current stored hash before redisplaying and drop the cache when it no longer matches. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Souravrajvi0
|
330586de7c |
feat(studio): expose full KV cache dtype list in model config UI (#7348)
Fixes #7244 The Studio per-model config dropdown only surfaced bf16, q8_0, q5_1, and q4_1 even though llama.cpp already accepts q4_0, q5_0, iq4_nl, and f32. Add the missing options to KV_CACHE_DTYPES and align API field descriptions with the backend _valid_cache_types set. Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Souravrajvi0
|
0e800d213a |
fix(studio): stop false MTP/vision capability reports (#7332)
* fix(studio): stop false MTP/vision capability reports (#7302) MTP probing only inspected the first physical --spec-type help line and treated empty/crash --help output as "lacks MTP", which false-warned on otherwise capable builds. Parse the full --spec-type help block, fail open when the probe is inconclusive, and stop blaming bare mmproj crashes on a projector-format mismatch when the text-only retry also fails. Fixes #7302 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio): tighten MTP probe semantics per Codex review (#7302) Treat nonempty --help without --spec-type as definitive no-MTP, keep only empty/crash probes inconclusive, skip binary_no_mtp UI hint on inconclusive loads, and stop reporting supports_mtp=True in /status for unknown probes. * Treat failed llama-server --help probes as inconclusive (#7302) Gate definitive no-MTP results on a zero exit code so crash diagnostics with nonempty stderr do not re-enable the false lacks-MTP warning path. * Add returncode to probe test mock so probe_ok gating passes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fail open in /status when the MTP probe is inconclusive * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Report missing llama-server as lacking MTP in /status * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten comments in MTP/mmproj probe changes --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Daniel Han
|
418ae14388 |
Fix ROCm wheel-index unit test: extract the gfx-arch probe helpers get_torch_index_url now calls (#7399)
* Fix ROCm wheel-index test: extract the gfx-arch probe helpers get_torch_index_url now calls get_torch_index_url gained a gfx-arch probe on the ROCm path (Strix reroute work) and now calls _ensure_rocm_probe_env, _probe_amd_gfx_arch, _infer_linux_amd_gfx_arch and friends. The unit test in tests/sh/test_get_torch_index_url.sh sources a curated subset of install.sh functions, and that list was never updated, so those helpers were undefined in the harness. On the ROCm path the gfx probe hit an undefined function, the branch silently fell through to the CPU wheel index, and every ROCm assertion failed (9 failures: all ROCm versions resolved to /whl/cpu). Extract the six missing helpers so the ROCm branch runs end to end. All 49 assertions pass. Adds a comment noting these must stay in sync with install.sh. * Keep the ROCm wheel-index test hermetic: redirect the /opt/rocm prefix Extracting _ensure_rocm_probe_env pulled its absolute-path host probe into the harness: it appends /opt/rocm/bin to PATH and runs the real host rocminfo, and version detection reads /opt/rocm/.info/version. On a host with ROCm installed that leaks the host GPU into the minimal-PATH test, so the no-GPU and CUDA-visible-device assertions could select a host ROCm wheel index instead of their expected CPU result, making the test host-dependent. Redirect the whole /opt/rocm prefix to an empty temp dir in the same sed pass that stubs /usr/bin/nvidia-smi, so the probes stay hermetic. All 49 assertions pass and the generated harness contains no real /opt/rocm path. --------- Co-authored-by: danielhanchen <unslothai@gmail.com> |
||
|
Daniel Han
|
6e91d1dff8 |
Studio: scan HF cache snapshot loads by their repo id (#7398)
* Studio: scan HF cache snapshot loads by their repo id Inactive Hugging Face caches (legacy, default, and previously selected download locations) are loaded by their resolved snapshot path so they keep using the selected cache instead of re-downloading. That path is a local filesystem path, so evaluate_file_security exempted it with "local path; no Hub scan" and skipped Hugging Face's pickle/malware scan. Active caches load by repo id and are still scanned, so the same model could dodge the gate simply by being in an inactive cache. An HF cache snapshot keeps the canonical models--org--repo/snapshots/<rev> layout, so recover the repo id from that path and scan it instead of exempting it. Non-cache local paths (models directory, custom folders) still skip the scan, and a remote ref is still scanned by repo id. Adds a regression test that a flagged pickle in an inactive-cache snapshot path blocks the load. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: scan the exact cached commit for inactive HF caches An HF cache snapshot path encodes the commit, not just the repo id (models--org--repo/snapshots/<rev>). Recover the revision alongside the repo id and pass it to model_info and the shard-index lookup so the scan covers the exact files that will be deserialized, rather than the repo's default branch. Without this, a pickle in an older cached commit that was later removed from the branch would scan clean and still load. Extends the regression test to assert the recovered revision is forwarded to the Hub scan. --------- Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Michael Han
|
140b3fbe05 |
Studio: register text-ui tokens with tailwind-merge so cn() keeps them (#7396)
* Studio: register text-ui tokens with tailwind-merge so cn keeps them Stock tailwind-merge classifies text-ui-* as a text color, so cn() dropped the size class whenever a color utility followed it in the same call. The element then fell back to the unscaled 16px root font, which made hub tabs and capability pills look oversized at small UI font sizes. Extend the merge config so text-ui-* and leading-ui-* resolve as font-size and line-height groups, and cover the failure in the contract and Playwright regression tests. * Studio: rename the Models page to Model hub Page heading, sidebar navigation label in all locales, and the chat download toasts that point at the tab. |
||
|
Lei Zhenyuan
|
47fa4ca6c1 |
Add Intel XPU support to Unsloth Studio (#4724)
--------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> |
||
|
Daniel Han
|
63d8da34d3 |
Studio: use text-ui-* tokens instead of raw px in the voice tab
Replaces the raw text-[9px] and text-[10px] classes with the text-ui-9 and text-ui-10 scale tokens so the voice tab labels honor the --ui-font-scale typography setting like the rest of the UI. |
||
|
Daniel Han
|
707b74fac3 |
Studio UI test: recover from voice-picker renderer crash, scoped to macOS runners
Downgrades a headless-Chromium renderer crash in the voice model-picker step to a warning plus page recovery on macos-14, where CheckMediaAccessPermission can kill the tab. Linux and Windows strict smoke jobs keep hard crash coverage and any live-page failure stays a hard fail. |
||
|
Daniel Han
|
a7761e1740 |
Studio: refine GGUF per-GPU selection (gpu_ids) (#7239)
--------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> |
||
|
Daniel Han
|
629cc50f1a |
Unsloth run/start: per-model recommended sampling and override flags (#7335)
Seed each request with the model's recommended sampling (matching the Chat UI), add per-field override flags, ignore oversized overrides, warn when sampling pins cannot apply to a reused server, and apply pins to the completions endpoint. |
||
|
oobabooga
|
3875479803 |
Complete local subagent delegation for Codex, Claude plan mode, and Pi (#7329)
Add session-scoped MCP bridges so Codex, Claude plan mode, and Pi subagents run on the loaded local model, with cloud credentials and Codex state isolated per session and a process-wide Pi agent cap. |
||
|
Daniel Han
|
a0f58c1128 |
Unsloth start: keep Claude subagents on the local model (#7333)
Add CLAUDE_CODE_SUBAGENT_MODEL=inherit to the session-only claude settings overlay so built-in subagents stay on the loaded local model. |
||
|
Daniel Han
|
6f60bf4f82 |
Studio whisper: pair slim bundles on the ggml commit, not the full llama tag (#7381)
The slim whisper bundle is ggml-less and links the ggml runtime out of the installed llama.cpp prebuilt, so each whisper release pins a paired llama tag. The gate required an exact tag match, but llama fork tags are b<upstream_build>-mix-<ggml_commit> and the build number tracks upstream llama and fork PRs that live outside ggml. When llama republishes a newer build with the same ggml commit (a frequent event), the installed llama advances past the whisper pin and curated dictation goes unavailable until whisper is republished, even though the ggml runtime is ABI-identical. Key the pairing gate on the ggml commit after -mix- instead of the full tag, in all three comparison sites (slim_pairing_for_artifact, _slim_release_incompatibility, resolve_selection). requires_ggml_sonames stays the real per-file ABI gate, and a genuine ggml skew still fails closed. Tags without a -mix- marker fall back to exact matching. |
||
|
Daniel Han
|
c2114d64dd |
Studio: fail closed on index-referenced nested pickle shards in the offline embedding gate (#7366)
* Studio: fail closed on index-referenced nested pickle shards in the offline embedding gate
The offline embedding security gate (HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE)
only scanned the direct files of each SentenceTransformer load root and never
parsed local weight indexes, so a cached snapshot whose pytorch_model.bin.index.json
maps a weight to a nested shard (e.g. shards/pytorch_model-00001-of-00001.bin) was
treated as inert and allowed. The loader then follows the index into the subdir and
unpickles the shard. The online gate already blocks index-referenced subdir pickles,
so the offline path was strictly weaker.
Parse each local weight index in a load root and follow weight_map into nested dirs,
flagging any referenced pickle-extension shard. Paths resolve lexically (normpath),
never Path.resolve(), since HF cache snapshot files symlink into blobs/ and resolving
would leave the snapshot dir and false-block every sharded model offline. An absolute
path, a .. traversal that escapes the snapshot, or an unreadable/invalid index fails
closed. The existing safetensors-sibling suppression is kept.
* Studio: classify offline indexed shards by torch.load path, not pickle extension
load_state_dict picks safetensors vs torch.load per shard by the shard's own
suffix, so two offline-gate gaps remained:
- A model.safetensors.index.json whose weight_map points at a .bin shard was
suppressed by has_base_safetensors (the index file itself matches the base
safetensors regex), yet Transformers still torch.loads that shard. Only the
pytorch index is superseded by a base safetensors now; a safetensors index is
the chosen archive, so its non-safetensors targets are always flagged.
- A pytorch index can map weights to arbitrary names (shards/payload,
weights.data); the loader torch.loads any target not ending in .safetensors.
Flag indexed shards by that rule instead of a pickle-extension allowlist.
Restrict the scan to the two torch-family indexes (tf/flax load via non-pickle
loaders). Add regression tests for both cases.
* Studio: match offline weight-index filenames case-insensitively
The index-name check compared the on-disk filename exactly, while the
surrounding weight and safetensors matches use case-insensitive rules. On a
case-insensitive volume (Windows or macOS) from_pretrained opens an oddly-cased
cache file such as PYTORCH_MODEL.BIN.INDEX.JSON when it requests the canonical
lowercase name, so the exact-case check skipped it and a nested pickle shard it
referenced was allowed through. Lower-case the index name before matching, as
the rest of the gate does, and add a regression test.
* Studio: match load_state_dict format/selection exactly in the offline index scan
Two edge cases in the offline weight-index scan:
- load_state_dict decides safetensors vs torch.load with a case-sensitive
endswith(".safetensors"), so a shard named payload.SAFETENSORS still
deserializes via torch.load. Classify indexed shard suffixes case-sensitively
to match, instead of lower-casing (which treated such a shard as inert).
- A complete direct model.safetensors is selected before either sharded index,
so a stale model.safetensors.index.json referencing a .bin shard never loads.
Skip both indexes when a direct model.safetensors is present, so an otherwise
loadable model is not over-blocked.
Add regression tests for both.
* Studio: read the offline weight index as UTF-8
Path.read_text() uses the locale default, which is cp1252 on Windows, so a
UTF-8 weight index with non-ASCII bytes raised UnicodeDecodeError and the gate
blocked an otherwise loadable model. JSON is UTF-8 by spec (and how the loader
reads it), so pin the encoding.
* Studio: resolve safetensors alternatives via the loader's own filename lookup
The offline gate decided a safetensors alternative existed by case-folding the
directory listing. On a case-sensitive filesystem that let an uppercase decoy
such as MODEL.SAFETENSORS suppress the pickle scan, yet from_pretrained asks for
the canonical lowercase model.safetensors, does not find the decoy, and selects
the pickle (a direct pytorch_model.bin or the pytorch index) and deserializes it.
Probe each alternative with (root / name).is_file() instead, mirroring the
loader: is_file() honors the platform's case rules, so a decoy suppresses only
where the loader would truly open it. Suppression must never fail open; detection
stays case-insensitive (fail closed). Add regression tests for the direct and
indexed pickle decoys (skipped on case-insensitive volumes, where no bypass
exists).
* Studio: resolve indexes and shards exactly as from_pretrained does
Two more loader-fidelity gaps in the offline index scan:
- Shard lookup normalized backslashes to forward slashes. On POSIX a backslash
is a literal filename character, so an index naming dir\payload.bin matches a
real pickle of that exact name that Transformers joins and deserializes, while
the normalized dir/payload.bin missed it. Join the raw weight_map value with
os.path.join so the probe mirrors the loader on each platform.
- Index detection case-folded the directory listing, so on a case-sensitive
filesystem an uppercase PYTORCH_MODEL.BIN.INDEX.JSON artifact the loader never
opens was treated as live and its shard blocked. Probe the canonical name with
the loader's own is_file lookup instead, so an index counts only where
from_pretrained would actually load it.
Update the uppercase-index tests to assert the correct per-filesystem behavior
and add a POSIX backslash-shard regression test.
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
|
||
|
Souravrajvi0
|
0807d03ed0 |
fix(install): show detected distro in sudo apt Accept prompt (#7324)
* fix(install): show detected distro in sudo apt Accept prompt Make the package-install elevation prompt name the detected distro and state that packages come from official apt repos, so users know we are not installing a tarball outside their package manager (#6207). * fix(install): avoid case/;; inside $() for bash 3.2 macOS CI uses bash 3.2, which misparses case arms inside command substitution and fails install.sh at the apt distro helper. Use a plain subshell so the Accept? prompt still works everywhere. |
||
|
Souravrajvi0
|
f5a0c2226b |
fix(studio): resolve bare git on Windows sandbox PATH (#7323)
* fix(studio): resolve bare git on Windows sandbox PATH Sandboxed terminal tools rebuilt PATH as venv + System32 only, so user-installed Git under Program Files never resolved by bare name. Append absolute host PATH dirs after the curated prefix and inherit PATHEXT on Windows (#7317). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio): restrict sandbox PATH inheritance to Windows Git dirs (#7323) Only append Git-for-Windows install directories from the host PATH on Windows, instead of every absolute entry. This fixes bare `git` resolution (#7317) without letting user-writable dirs (venv, node_modules/.bin) shadow auto-safe terminal commands. * Pin sandbox PATHEXT to block cwd script hijacks (#7317) Use a fixed .EXE;.COM list instead of inheriting the host PATHEXT so cmd cannot resolve auto-approved bare names from workdir .BAT/.CMD stubs. * Resolve sandbox git dir via shutil.which and disable cwd exe lookup * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep non-exe git launchers resolvable under restricted PATHEXT * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restrict inherited sandbox git dir to system install roots * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Drop SystemRoot trust and canonicalize short paths for sandbox git * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Resolve Program Files via known-folder API and append canonical git dir * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trust native Program Files on 32-bit Windows and stub program roots in tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scan PATH for a trusted git and derive native Program Files root * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Drop ProgramFiles env from the trusted-root fallback * Fail closed when trusted Program Files root cannot be resolved --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Souravrajvi0
|
09b6bf6c39 |
fix(studio): opt-in source-build GPU smoke validation (#7322)
* fix(studio): opt-in source-build GPU smoke validation (#5854) Gap 1 (empty CUDA arch -> CPU) already landed in #6481. Wire gap 2: after a GPU source build, optionally run the same staged llama-server smoke test as the prebuilt path, then CPU-fallback on failure. Gated by UNSLOTH_LLAMA_STAGED_VALIDATION (default off) to avoid Blackwell JIT stalls. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(install): normalize staged validation env in setup.sh (#7322) Strip and lowercase UNSLOTH_LLAMA_STAGED_VALIDATION before the shell gate so values like True and surrounding whitespace match the Python staged_validation_enabled() helper. * Rebuild visual server after staged-validation CPU fallback (#5854) Mirror the primary source-build path by best-effort building llama-diffusion-gemma-visual-server after smoke-failure CPU fallback. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Souravrajvi0
|
8c975fcbaf |
fix: pin torchcodec for torch 2.10 and warn on ABI mismatch (#7299)
* fix: pin torchcodec for torch 2.10 and warn on ABI mismatch Add unsloth[audio] extra with torchcodec>=0.10.0,<0.11.0 and emit a clear warning when installed torchcodec minors disagree with torch (unslothai/unsloth#7225). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(packaging): address Codex review on torchcodec/torch 2.10 compat (#7299) - Postpone annotations so import_fixes loads on Python 3.9 - Align TORCH_TORCHCODEC matrix with upstream (2.9: 0.8/0.9, 2.8: 0.6/0.7) - Fix mismatch hint upper bound (<0.11.0) and gate audio-torch210 suggestion - Split audio extra per torch minor; gate torch210 pin behind python>=3.10 - Bundle audio-torch210 only in *-torch2100 install extras * fix(security): refresh openai CRITICAL scan baseline hashes (#7299) openai package code drift reopened five CRITICAL findings in the extras pip-scan-packages shard (C2 loop body hashes + IMDS/network evidence). Update the reviewed allowlist evidence/hashes so CI gates on new findings only, not benign SDK churn. * chore: retrigger CI after baseline refresh (#7299) * chore: touch scan baseline comment to retrigger security audit (#7299) * Guard torchcodec version parsing so bad version strings cannot break import * Bundle audio pin into intel-gpu-torch210 and guard the mismatch warning * Tighten comments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Souravrajvi0
|
b448fb5de0 |
fix(studio): persist connection model selections for remote clients (#7298)
* fix(studio): persist connection model selections server-side Remote Studio clients could see saved connections but not their enabled model lists because models lived only in browser localStorage. Store models and available_models in llm_providers and sync them through the providers API so alternate clients inherit the same catalog state. Fixes #7281 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Hydrate external connections on chat startup (#7281) Extract provider sync logic into sync-external-providers.ts and call it from chat-page on mount so persisted model selections appear in the Connected picker without opening Settings → Connections first. * fix(studio): backfill connection models and preserve local options (#7298) Address Codex P2 on remote connection persistence: - Backfill localStorage model selections to /api/providers when backend rows still have empty models_json (legacy upgrades) - Carry promptCacheTtl and openaiContainerTtlMinutes through startup sync - Await hydratePersistedSettings before syncing on ChatPage mount Contract tests: 7 passed; npm run typecheck passed. * Tighten comments * Tighten comments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Souravrajvi0
|
e2ccf4d376 |
fix(studio): show chat sidebar menu on touch devices (#7297)
* fix(studio): show chat sidebar menu on touch devices Recents/Pinned chat row actions were hidden until hover, so iPad users could not open the kebab menu to delete chats. Reveal actions on coarse pointers using the same pattern as hub model rows. Fixes #7276 * Fix coarse-pointer sidebar row action visibility (#7276) Move the touch-device override into index.css after .sidebar-row-action so it wins the cascade. Arbitrary Tailwind media utilities on the element had equal specificity and were overridden by the base rule. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope coarse-pointer sidebar actions to chat rows (#7276) Only chat kebabs/unpin buttons that reserve touch padding get sidebar-touch-reveal, so project/run/nav rows stay hover-revealed. * Tighten comments * Reserve full kebab hit area on coarse-pointer unpinned rows --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Nilay
|
447f16f49a |
Studio: fix composer reset after failed send (#7377)
* fix composer reset * Studio: clear composer draft on send * Studio: cancel the pending draft save when clearing on send |
||
|
Lee Jackson
|
a26692612d |
Normalize PWD for POSIX agent launches (#7110)
Keep the child process environment consistent with the cwd used to launch native POSIX coding agents. Some Node-based agents use PWD during project-root discovery, so inheriting a stale PWD can make them edit files in a parent or unrelated directory even when the wrapper process cwd is correct. Only apply this normalization for native POSIX launches. WSL-launched Windows shims stay on the existing WSLENV bridge path so path translation behavior is unchanged. Add regression coverage that launches an agent with a deliberately stale inherited PWD and asserts the child environment is normalized to os.getcwd(). Co-authored-by: Leo Borcherding <borchborchmail@gmail.com> |
||
|
Daniel Han
|
6e868860bd |
Bump install.sh / install.ps1 pin to unsloth>=2026.7.5 (#7365) | ||
|
Daniel Han
|
734cec9e7a |
Studio STT: only load safetensors weights for custom dictation models (RCE fix) (#7364)
* Studio STT: only load safetensors weights for custom dictation models The STT sidecar accepts arbitrary Hugging Face owner/model repos for custom dictation models and, when safetensors were absent, downloaded and loaded pytorch_model.bin through WhisperForConditionalGeneration .from_pretrained. PyTorch checkpoints are pickles that execute code during deserialization, and this path does not run the malware gate the normal model loader applies, so an authenticated client on an exposed Studio instance could load a crafted Whisper-looking repo and run code in the backend. Restrict custom STT repos to safetensors: the snapshot selector no longer falls back to pytorch_model.bin(.index.json), the cached-snapshot completeness check ignores pickle weights, and the load forces use_safetensors so a stray cached pickle still cannot execute. The five curated Whisper defaults already ship safetensors only, so this changes nothing for the built-in models. * STT: reject safetensors indexes that reference non-safetensors shards A safetensors index (model.safetensors.index.json) is attacker-supplied JSON and can name pytorch_model-*.bin shards in its weight_map. Transformers dispatches shard loading per file by extension, so those .bin shards still load through torch.load (pickle) even with use_safetensors set. Require every weight_map value to end in .safetensors in both the snapshot selector and the completeness check so no pickle shard is downloaded or reused. |
||
|
Daniel Han
|
beddfc963e |
Studio: viewport-gated highlight for the executed Python script (follow-up to #7240) (#7363)
v0.1.501-beta
* Studio: keep the executed Python script visible in chat, with download + viewport-gated highlight Always show the executed Python script under the tool card (not only inside the collapsible run/output section, which is unmounted from history), with Copy and a client-side .py Download button, so the script stays visible on reopen (#7165). The script is rendered eagerly, but shiki syntax-highlighting only runs once the block scrolls near the viewport (IntersectionObserver, 200px margin); until then a plain monospace placeholder shows the same source with matching padding, so there is no layout jump. This bounds highlighting to the cards actually on screen instead of tokenizing every script up front. Measured shiki cost is ~8 ms per typical 2 KB script, so eager highlighting of a long agentic transcript (20-50+ Python calls) would add ~170-420 ms of main-thread work on load; viewport-gating keeps it to the few visible cards (~15-35 ms) regardless of transcript length. Falls back to immediate highlight when IntersectionObserver is unavailable (SSR / tests). * Use a div for the pre-highlight placeholder so container [&_pre]:!p-0 doesn't strip its p-3 The placeholder shares the highlighted block's p-3 padding to avoid a layout jump, but as a <pre> it was caught by the container's [&_pre]:!p-0 !important rule and rendered with no padding, so the script shifted by p-3 when shiki swapped in. A plain div keeps the padding. * Match placeholder wrapping to the highlighted pre (whitespace-pre, not pre-wrap) The placeholder wrapped long lines while the highlighted Streamdown <pre> keeps them on one line and scrolls in the container's overflow-auto, so a script with a long line changed height when shiki swapped in. Use whitespace-pre so the placeholder scrolls the same way and the height stays stable. --------- Co-authored-by: danielhanchen <unslothai@gmail.com> |
||
|
Daniel Han
|
d49d23ffab | Update pyproject.toml | ||
|
Daniel Han
|
02ee08afe3 | Update _utils.py | ||
|
Long Yixing
|
e143e1ce33 |
feat(studio): Mac-aware training controls for MLX (optimizers, LoftQ, packing) (#7358)
* feat(studio): offer MLX-supported optimizers on Apple Silicon The training form's optimizer dropdown only listed CUDA/bitsandbytes optimizers (adamw_8bit, paged variants, torch fused). On Apple Silicon the MLX trainer supports a different set (adamw, adam, lion, muon, sgd, adafactor) and remaps every bitsandbytes/torch name to plain AdamW, so the dropdown misrepresented what actually runs. Offer the MLX optimizer list when the device is a Mac, and derive the displayed value so the control is never blank: the shared CUDA default and the other bitsandbytes/torch options render as AdamW (exactly how the MLX backend normalizes them), while any other value is shown as-is so an unrecognized or non-canonical imported optimizer is never mislabeled. Non-Mac behavior is unchanged. The run-summary optimizer label now resolves from both lists. * feat(studio): show an MLX-appropriate optimizer tooltip on Apple Silicon The optimizer tooltip described "8-bit variants" and recommended "Fused" for vision models, neither of which is offered when training runs on MLX. On Apple Silicon, show a tooltip that matches the MLX optimizer set and notes that Lion typically needs a lower learning rate than AdamW. Copy-only: no change to the selected optimizer or the learning rate, and the non-Mac tooltip is unchanged. The new string is added to the English locale; other locales fall back to English until translated, matching how new keys are handled elsewhere. * fix(studio): label Mac CUDA-alias optimizers as AdamW in the run summary On Apple Silicon the run-configuration summary looked up the stored optimizer name directly, so a run that kept a CUDA/bitsandbytes default such as adamw_8bit was labeled "AdamW 8-bit" even though the picker shows "AdamW" and the MLX backend runs plain AdamW. Mirror the training form's derivation so those aliases are labeled AdamW in the summary too. Display-only: no change to the stored or submitted optimizer, and non-Mac summaries are unchanged. * feat(studio): disable LoftQ and sequence packing on Apple Silicon Neither LoftQ nor sequence packing is supported on MLX — the backend rejects LoftQ and the trainer silently forces packing off — yet the training form still offered both on Apple Silicon. Disable the LoftQ LoRA-init option (greyed and unclickable, with an inline "Not supported on Apple Silicon" note) and the "Enable packing" checkbox (greyed, with a tooltip explaining why), matching how the unsupported "Enable streaming" control is presented. Clearing effects reset a stale loftq/packing value to its default on Mac so the disabled controls never submit it. Non-Mac behavior is unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
5aedfd0b46 |
Studio: always show the executed Python script in chat with a download option (#7240)
The script the python tool runs was rendered inside a collapsible that closes when the run ends or the thread is reopened, so the code disappeared from the transcript and there was no way to save it. Render the script outside the collapsible so it stays visible, and add a Download button that saves it as script.py. Other tools and normal chat are unaffected. |
||
|
Daniel Han
|
b0d6131567 |
Security audit: refresh scan baselines for current dependency set (#7362)
The pip scan-packages extras shard has been red on main because openai
2.47.0 changed the code inside five previously baselined findings, so
their evidence hashes no longer matched the allowlist. The hf-stack
shard was about to go red the same way: unsloth-zoo 2026.7.5 changed
two baselined test files.
All seven reopened findings were re-verified against the exact resolved
archives before re-baselining:
- openai/_base_client.py: while True in SyncPage.iter_pages, the
pagination iterator.
- openai/auth/_workload.py: Azure IMDS and GCP metadata token
providers for the documented workload identity federation feature.
- openai/resources/{beta/responses,realtime,responses}: while True in
websocket __aiter__ event loops; the loop bodies gained reconnect
handling in 2.47.0, which is what shifted the hashes.
- unsloth-zoo tests/test_vision_collator_audio.py: asserts that an
inline /tmp/a.wav path is passed through by the audio collator.
- unsloth-zoo tests/test_gemma4_forced_float32_ple_dtype.py:
compile()/exec() of the project's own generated Gemma4 PLE cast
helper source in tests.
No existing entries were removed. All three shards now exit 0 locally
against the same requirement sets CI uses.
|
||
|
Daniel Han
|
4fedb51b73 |
unsloth start/run: tool-call flags, positional model, and grouped help (#7328)
* unsloth start/run: tool-call flags, positional model, grouped help Expose the existing tool-call controls as first-class CLI flags on both unsloth run and unsloth start, add positional model detection with a GGUF quant default, and group --help into rich panels. Flags (unsloth run): --enable-tool-call-healing/--disable-tool-call-healing (default on), --enable-tool-call-nudging/--disable-tool-call-nudging (default on). Resolved before any re-exec and written to the existing env controls (UNSLOTH_DISABLE_TOOL_CALL_HEALING, UNSLOTH_TOOL_CALL_NUDGE) so the in-venv server reads them at import; an omitted flag respects a value the parent already set. Flags (unsloth start): --enable-tools/--disable-tools (default off, passthrough), plus the same healing/nudging flags (default on). start conveys them to the auto-started run via the child env and the tools flag, so it stays correct even if run re-execs into an older Studio venv. Positional model: a leading org/name(:variant) token routes to --model when --model is absent, without stealing an option value or an agent passthrough arg. A bare GGUF repo with no variant defaults to UD-Q4_K_XL for the unsloth namespace and Q4_K_M elsewhere, applied only on the fresh auto-serve path so attaching to a loaded model never reloads. Help is grouped into rich panels (Model / Server / Session for start; Model / Server and network / Tool calls / Advanced for run) so --help reads cleanly. Adds unit coverage for the helpers, the start command-and-env forwarding, the positional/quant defaulting, and the run env resolution. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Positional model: reuse _is_hub_model_id so local dirs and paths are not stolen Route a bare org/name positional to --model only when it resolves as a hub id (via the existing _is_hub_model_id, which rejects local paths and existing dirs), so an OpenCode project dir like owner/repo is left for the agent. Apply the same guard to the auto-serve GGUF quant default so a local -GGUF path is not forced to a quant it may not contain. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * unsloth start: typer floor, drop redundant GGUF quant default, respect inherited tool-call env - Require typer>=0.12.0. The rich_help_panel options added here crash at import on typer<0.6, and the dependency was previously unbounded. - Stop forcing a default GGUF quant for a bare org/name-GGUF on auto-serve. The server's own quant preference already picks UD-Q4_K_XL for Unsloth uploads and Q4_K_M otherwise, and falls back when that exact quant is missing, so forcing a fixed variant broke external repos that only publish Q5_K_M/Q8_0. - Make the healing/nudging start flags tri-state so an omitted flag keeps an operator's inherited UNSLOTH_DISABLE_TOOL_CALL_HEALING / UNSLOTH_TOOL_CALL_NUDGE instead of overwriting it with the start defaults. * Fix start passthrough and inherited tool settings --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> |
||
|
Michael Han
|
d5cf96d628 |
Studio: add local speech-to-text dictation engine (#7095)
* Studio: add Voice settings tab (dictation, dictionary, read aloud) New Voice tab in Settings, placed just before About: - Dictation: microphone picker, browser STT engine, recognition language, and an inline mic test with a live transcript - Dictation dictionary: entries rewrite matching speech to their exact spelling and casing, applied in both dictation paths - Recent dictations: last 20 final transcripts with copy and clear, so text can be recovered if it lands in the wrong place - Read aloud: optional button on assistant responses with two engines, curated system voices (novelty and legacy voices filtered, quality ranked, capped at 20) or the TTS audio model loaded in Unsloth via /audio/generate (e.g. Orpheus), plus speed, pitch, volume and preview Settings persist in localStorage (unsloth_voice_settings) and are read at call time so changes apply without reloading the runtime. Adds en keys plus the tab label for ja, zh-CN and pt-BR. * Studio: drop the single option STT engine select, rename TTS option The STT engine dropdown only had one entry, so it added noise without giving a real choice. The engine row can come back once local STT models land. Also renames the TTS engine option Unsloth TTS model to Load TTS model to make the action clearer. * Studio: harden Voice settings against edge cases found in simulation Simulated the feature across Chromium, Firefox and WebKit plus node level unit runs and backend contract checks. Fixes from the findings: - Dictionary rewrite used a replacement string, so entries containing dollar patterns corrupted transcripts (A$$AP became A$AP, $& injected the match). Switched to the callback form of String.replace - Persisted voice settings now validate types on hydration: non string micDeviceId, dictationLanguage and ttsVoiceURI, and non boolean ttsEnabled fall back to defaults instead of flowing into the UI - Dictionary entries are trimmed, capped at 120 chars and re-sanitized on hydration - The Test dictation panel now falls back to the default microphone when the saved device is unplugged, matching the composer adapter Test coverage: 46 unit assertions (dictionary regex edge cases across unicode, word boundaries and injection, voice curation for simulated macOS, Windows and Linux voice inventories, corrupt storage merge), 13 backend contract checks against /audio/generate on an isolated instance, and 60 browser assertions across the three engines covering rendering, degradation without SpeechRecognition, curation in a real DOM, dictionary persistence with unicode and dollar entries, the no-model preview error path and corrupt localStorage recovery. * Studio: address Voice settings review feedback Verified each review comment before acting. Confirmed and fixed: - Editing a dictionary entry was broken in two ways: the store trimmed on every keystroke so spaces could not be typed, and clearing the field deleted the entry and unmounted the input mid edit. Updates now keep the raw value and a blur commit trims or removes the entry - The unplugged mic fallback checked instanceof DOMException, but a cross browser probe showed Firefox and WebKit throw OverconstrainedError objects that are not DOMExceptions, so the fallback never fired there. Matching on the error name now - When the browser ended a dictation test on its own (silence timeout), the mic stream stayed open. All recognition end paths now stop the tracks and save the transcript through a single finalize path - The studio TTS audio element now releases its WAV data URL as soon as playback ends, fails or is cancelled - Allow microphone now reports insecure contexts (no mediaDevices) accurately instead of claiming access was blocked - Voice tab copy moved into i18n keys per src/i18n/AGENTS.md, so locale overlays can translate it; en is the baseline and parity passes - unsloth_voice_settings added to the Reset all local preferences key list so voice preferences obey the reset - Non default microphones note that the system default is used when the browser speech engine cannot bind a specific device, since browsers without the start(track) overload ignore the argument silently Re-ran the full simulation set after the changes: 46 unit assertions, 13 backend contract checks and 60 browser assertions across Chromium, Firefox and WebKit all pass, plus a dedicated browser probe for the dictionary editing behavior. * Studio: use the chat mic icon in Voice settings for consistency The Voice tab and its buttons used the hugeicons Mic02 glyph while the chat composer uses a custom filled mic. Extract that composer icon into a shared lib/mic-icon component, drop the duplicate inline copies in thread.tsx and shared-composer.tsx, and use it for the Voice tab icon and the tab's mic buttons so the microphone looks the same everywhere. * Studio: address second round of Voice settings review feedback Verified each new comment against the current code first. One item was already fixed in the previous round (recording transcripts when the browser ends a dictation test on its own). Confirmed and fixed: - The microphone row showed a picker with generic names when browsers enumerate unlabeled devices before permission, leaving no way to grant access from the row. It now branches on whether labels are visible and shows Allow microphone otherwise - Compare chat dictation ignored the selected microphone. It now opens the chosen device with the same fallback rules as the main adapter, passes the track to recognition where supported and releases the stream when recognition ends - Closing the Voice tab cancelled the shared speechSynthesis even when read aloud was playing a chat message. Cleanup now only cancels when the tab owns an active preview - Double clicking Start test could race two recognizers and leak the first stream. A starting flag set before the getUserMedia await makes start reentrancy safe - Turning off the read aloud setting mid playback removed the only stop control. The stop button now renders whenever a message is speaking - When an engine lacks the start(track) overload, both dictation paths now release the selected device stream before retrying with the default microphone instead of holding it open - Read aloud support no longer requires Web Speech synthesis: the Unsloth TTS engine only needs audio playback, so it stays available in WebViews without speechSynthesis, with a clear error if the system engine is chosen there Not addressed here: cancelling in flight backend TTS generation on stop. The route runs generation in a worker thread without a cancellation path, which is shared pre existing behavior with audio chat generation and belongs in a backend change. All suites re-run green: 46 unit, 13 backend contract and 60 browser matrix assertions across Chromium, Firefox and WebKit, plus probes for the unlabeled device branch and the double click race. * Studio: drop empty and duplicate voiceURIs so the Voice tab never renders a crashing Select item * Studio: guard dictation mic lifecycle in Voice test and Compare composer Release a microphone opened after the component unmounts, and stop Compare dictation on a permission or security failure instead of silently recording from the default device, matching the main chat adapter. * Studio: fix dictation and read-aloud lifecycle edge cases in Voice settings - Join final dictation chunks with a space so recorded transcripts do not merge words - Ignore a stale recognizer onend so a quick stop then restart is not torn down - Use previewingRef so a double click on TTS preview does not orphan the first request - Keep the read-aloud stop control visible when a new run starts while a message is spoken - Stop the dictionary remove button from deleting an adjacent entry on a blur then click race * Studio: trim redundant Voice settings comments * Studio: fix Voice preview and Compare dictation edge cases - Only cancel the shared speechSynthesis for a system-voice preview, so stopping a Studio preview no longer stops an unrelated chat read-aloud - Release the Studio preview audio and its WAV data URL on normal completion - Iterate every finalized result in Compare dictation so batched phrases are kept - Cap persisted recent dictations to the last 20 on hydration * Studio: use clipboard fallback for recents and release failed preview audio - Copy recent dictations via the copyToClipboard helper so the execCommand fallback works in Safari and insecure http LAN contexts - Release the Studio preview audio when play() rejects, not just on ended/error * Studio: add local speech-to-text dictation engine Add an offline dictation engine that transcribes with a local faster-whisper model, alongside the existing browser (Web Speech) engine. The browser engine streams audio to Apple or Google speech services and needs internet; the new engine runs on the server, works offline, and drives any chat model without evicting it (it loads in the backend process, separate from the model subprocess). It also gives Firefox dictation, which has no Web Speech support. Backend: a lazily-loaded, kept-warm faster-whisper sidecar and three routes under /api/inference/audio (stt/status, stt/load, transcribe). faster-whisper is torch-free, so this does not disturb the existing model stack. Frontend: a Dictation engine setting (browser or local model), a curated model picker with sizes, and MediaRecorder capture posted to the transcribe route. The model warms automatically when the engine is selected, with live status. * Studio: stream local STT transcription as you speak Local dictation showed nothing until you stopped, because the whole clip was transcribed once on stop. Now the growing recording is re-transcribed on a fast pass every second and emitted as live interim text, with an accurate final pass on stop. Partial recordings decode fine, and the model refines earlier words as more audio arrives. Adds an interim flag to the transcribe route (beam 1, no VAD) for the fast preview pass; the final stop uses the accurate path. * Studio: make local dictation stop instant and reliable Stopping local dictation waited for a final network transcription before the session ended, so the stop button did not flip and a second click ended the session early and dropped the text. Now stop commits the live transcript immediately, releases the mic at once, and ignores a second stop while finalizing. Previews run more often so the committed text is current. * Studio: record local dictation in short clips for reliable streaming Re-transcribing a growing buffer every second got slower as it grew, flooded the backend, showed stale words, and could leave the stop button stuck waiting on a backlog. Record short independent clips instead and transcribe each once, appending the text as you speak. Work per clip is bounded, so stopping is prompt (with a hard timeout as a safety net) and long dictations stay smooth. * Studio: dictate then transcribe once on stop, ChatGPT style Local STT dictation streamed by re-transcribing the growing clip, which was quadratic and saturated the backend (multi-second lag), and stop only halted the recorder without releasing the mic, so it kept recording. Record the microphone continuously, release it the instant the user stops, and transcribe the whole clip once. Stopping is immediate and the transcript lands in about a second. Also add the tiny model for the fastest option. * Studio: surface dictation and read-aloud failures instead of failing silently - Compare dictation reports microphone and speech-recognition errors via toast, reusing the main chat adapter's describeMediaError and describeSpeechError - Read-aloud toasts genuine model or synthesis failures while ignoring cancellations * Studio: ChatGPT-style recording bar for dictation Clicking the mic now drops the composer into a dedicated recording bar with a live waveform, a discard (X) and a confirm (tick), instead of a plain stop button. The tick stops recording and transcribes the clip; the X throws the recording away and keeps whatever text was already in the composer. The model adapter taps the mic with an analyser to drive the waveform, and the router tracks the live session so the X can cancel it without transcribing. * Studio: transcribe dictation while speaking, ChatGPT layout Match ChatGPT's recording layout: the bar now renders in place of the input with the left plus button kept, the waveform in the middle, and the discard and confirm buttons together on the right. Cut the post-confirm delay by transcribing in the background as the user talks. The audio is split at natural pauses (voice-activity detection off the same analyser that drives the waveform) and each clip is transcribed as it is cut, so confirming only has to finish the short final tail. The model is also warmed when recording starts so the first run never pays a cold load. * Studio: ChatGPT waveform, hide tools while dictating, faster STT Make the recording UI read like ChatGPT: the waveform is now a dense row of round dots that rise into thin centered bars, and while dictating only the plus button shows, with the mode badge and tool toggles hidden so the bar is just the waveform and controls. Speed up transcription: decode greedily (beam_size=1), which is several times faster on CPU with negligible accuracy loss on short dictation clips, and cap background segments at 6s so the final tail after confirm stays short. * Studio: finish ChatGPT voice bar and low-latency STT * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: full-width waveform with a timer that freezes on stop Use the full-width waveform for the recording bar: brighter, bigger bars that advance on a fixed cadence (keeping peaks between advances) so they glide instead of racing by, inset from the composer edges. Keep a visible timer and the green confirm button, matching the ChatGPT reference, and freeze the timer and waveform the moment the user confirms. * Studio: fix multilingual local dictation * Studio: speed up dictation and release local STT * Studio: harden dictation finalization and STT decoding * Studio: restore Firefox dictation fallback * Studio: add dictation history manager * Studio: manage speech model downloads * Studio: remove em dash from voice model label * Studio: move dictation history into Voice * Studio: source local STT from Unsloth Whisper models Point the dictation STT sidecar and its Model Hub download entries at Unsloth's Hugging Face Whisper repos (small, large-v3-turbo, large-v3) and run them through Transformers, so Studio only ever downloads Unsloth-uploaded weights. Drop faster-whisper and the Systran/mobiuslabs repos; keep the Model Hub as the only download path via local_files_only, and keep PyAV for audio decoding. Device selection uses float16 on CUDA and float32 on MPS and CPU, since Whisper's decoder is unstable in float16 on MPS and repeats tokens. Shorten the model picker labels to name plus download size and update the STT tests for the new backend. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: smooth dictation waveform and keep pill height * Studio: align STT model dropdown width and tidy voice copy * Studio: guide to local engine when browser dictation is offline * Studio: clarify voice section and STT model copy * Studio: keep STT warm with training-aware eviction * Harden STT lifecycle and browser compatibility * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix model discovery test lint * Harden cross-browser microphone errors * Harden cross-browser microphone errors * Surface voice test recognition errors and fall back to Studio TTS - Voice test now toasts non-abort speech-recognition failures instead of ending silently, matching the main and Compare dictation paths. - Read-aloud routes to the backend model when the runtime lacks Web Speech synthesis (audio-only WebView), so it no longer errors immediately. * Fix reviewed STT lifecycle races * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix read-aloud fallback controls * Guard read-aloud stop when deleting a non-speaking message aui.message().stopSpeaking() throws unless this message is the one being read aloud, so calling it unconditionally rejected the delete handler before the message was removed. Only stop speech when this message is speaking. * Cap recent dictation transcript length before persisting Recent dictations only limited entry count, so a long transcript stored the full text in the persisted voice settings and a few could exceed the localStorage quota, throwing synchronously from the uncaught dictation cleanup path. Truncate each entry on save and on hydration, matching the dictionary cap. * Studio: keep dictation mic clickable and guide to local model Register the dictation adapter unconditionally so the mic stays enabled for any engine and starts working right after switching to the local model on an already-open thread. When the browser engine cannot run (Firefox, Brave, non-secure origins), clicking the mic shows a toast that points to the local speech-to-text model instead of leaving a disabled button. The toast stacks its action below the text with a fully rounded button. * Studio: add bottom padding below the dictation guidance toast button * Studio: increase bottom padding under the dictation toast button * Studio: add bottom padding inside the dictation toast button * Studio: add five Whisper defaults and custom model search Add private UnslothAI Tiny and Base mirrors to the curated local STT choices while keeping Small as the default. Let users search or paste a Transformers-compatible Whisper repository and validate it end to end. Keep short dictations in one clip to avoid repeated padded encoder work, then split longer recordings near Whisper's 30-second boundary. Update hidden model filters and tests, including the CPU-only CI runtime stub for PyAV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: use public Unsloth Whisper repositories Point the Tiny and Base dictation defaults to the public unsloth repositories and remove the private mirror references from model filtering and tests. * Studio: update Whisper download sizes Reflect the cleaned public Tiny and Base repositories in the curated model labels. * Studio: right-align STT model size, fix dropdown wheel scroll, refresh sizes - Show the download size on the right of each model row so long names like Whisper Large v3 Turbo no longer hide it - Update curated Whisper sizes to the safetensors weights actually downloaded: Tiny 151 MB, Base 290 MB, Small 967 MB - Drive the model list scroll from a wheel handler so the mouse wheel scrolls it inside the Settings dialog, not just the scrollbar - Add a search icon and shorten the placeholder to Search model * Studio: do not search when a dictation model is picked, shrink repo label - Treat the filled-in model text as a selection, not a query, so choosing a model no longer kicks off a Hugging Face search - Make the repository line under each model name smaller * Studio: tighten dictation model and local engine descriptions * Studio: keep model display on pick instead of the query, shrink row text - Guard the combobox input so selecting a model shows its name and does not echo the typed query back or start a search - Map the item label to the friendly display so picks fill the field - Reduce the model name and size text in each row * Studio: show only the model name in the dictation field, shrink size label - Drop the download size from the search field; the name alone is shown once a model is selected, with sizes kept in the dropdown list - Reduce the size label text in each row * Studio: clarify the dictation model description * Studio: drop Hugging Face from the dictation model description * Studio: move the dictation dictionary to its own Manage subpage - Replace the inline entry list with a Manage row, matching Dictation history, so a long dictionary no longer crowds Voice settings - Add a DictationDictionaryView subpage that holds the entry editor * Studio: match STT field font, use best voice for System default - Bump the dictation model field text to text-sm so it matches the engine dropdown next to it - Resolve the System default read-aloud voice to the top curated voice instead of the browser default, which is a robotic legacy voice on macOS * Studio: rerank read-aloud voices and drop duplicate voice entries - Rank by vendor quality, then the user's locale, then a preferred list of natural voices, so the best voice leads instead of the first alphabetically - Collapse voices that macOS reports twice under one name and language * Studio: fold dictionary and recents into the dictation section - Drop the separate Dictation dictionary and Recent dictations headings; their Manage rows now sit under Dictation, split by the row divider - Shorten the custom spellings description * Studio: add search and sort to dictation history - Filter saved dictations by text with a search field - Sort by newest, oldest, or A to Z; show a no-matches message - Keep Clear all available regardless of the current filter * Studio: settle cancelled STT loads before training and fix dictation review items Wait for a cancelled STT load to exit and release its memory before reporting it freed for training, so the loader cannot still be inside from_pretrained()/.to(device) holding VRAM when the training subprocess starts. A load that finishes before observing the cancel now gets unloaded so the memory is actually reclaimed. Clear the accelerator cache before the CPU fallback in load() so a failed CUDA/MPS load does not strand reserved VRAM once the sidecar is marked CPU-resident. Send the saved Hugging Face token when polling STT download progress so a gated or private repo resolves and shows the correct Load/Downloaded state instead of reporting missing. Mark the composer Dictate button as type="button" so clicking it does not also submit the draft when the composer already has text or attachments. * Studio: pin dictation settings per session and close STT startup races Capture the STT model and language when a dictation session starts and pass them to every queued segment and the warm-up load, so changing the model or language mid-recording no longer transcribes the same clip with the wrong model or a model that is not downloaded. Check the local runtime at the top of transcribe(), before the model cache lookup and the bounded audio decode, so a server missing PyTorch or Transformers returns 501 up front instead of decoding a long clip first. Treat the training startup window as active for STT device selection. start_training frees VRAM in before_spawn but only assigns _proc later, so a concurrent STT load could take the GPU that was just cleared. A startup flag now reports training active from the free until the process is live, forcing those loads to CPU; a finally clears it on every exit. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: stub the STT runtime check in transcribe orchestration tests transcribe() now verifies the local runtime up front, so the unit tests that exercise transcription orchestration must treat the runtime as present to keep passing where PyTorch, Transformers, and PyAV are not installed. Stub ensure_stt_available in the shared fixture and restore the real check in the availability and load-rejection tests. * Harden custom Whisper dictation models * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add whisper.cpp dictation engine with per-engine downloads and history rework Engines - New GGML STT sidecar that runs a managed whisper-server subprocess with idle unload, plus a pinned static build script (scripts/build_whisper_cpp.sh) - Dictation engine picker now offers Browser, Local transcription (whisper.cpp), and Local transcription (Transformers) - Both local engines serve the same five curated Whisper models and download them directly with byte-level progress reported by /audio/stt/status - Models auto load on selection and when their download finishes - Unload and training admission account for both engines Benchmarks (Apple Silicon, greedy, warm, same checkpoints) - whisper.cpp transcribes 2.4x to 5x faster than Transformers and loads in about 0.45s vs 0.86s for Whisper Small - whisper.cpp GGUF path is unchanged by the Transformers addition (load 0.445s -> 0.444s, short clip 0.391s -> 0.347s, long 1.197s -> 1.129s) Voice settings UI - Plain curated model select replaces the searchable combobox - Single download progress bar with transfer rate for both engines - Dictation history now stores every dictation with Show more pagination, a top Clear history action, and links back to the chat it was spoken into - Archived chats dialog gets the same pagination - Delete dialog offers deleting a dictation together with its chat Tests: 88 backend STT tests pass, including new snapshot download coverage. Frontend typecheck, lint, i18n parity, and production build pass. * Merge local engines into one option and source GGML models from unslothai Engine selection - The dictation engine dropdown is back to two choices: Browser and Local transcription. The selected model decides the backend: curated ids run GGML checkpoints through whisper.cpp, searched Hugging Face repositories run safetensors through Transformers - Model picker lists the curated models and searches Hugging Face for other Whisper repositories, validating them before selection. The trigger is a plain button so the selection never renders inside a text input - /audio/stt/status accepts a model query param so downloaded state works for custom repositories; the engine param on load, transcribe, and download routes is derived from the model everywhere Model source - Curated GGML checkpoints now download from the Unsloth-hosted unslothai/whisper-*-GGUF repositories (one repo per model) instead of ggerganov/whisper.cpp; cache lookups, progress totals, and in-flight blob tracking are per-model Fixes - Voice settings and dictation history were not persisting: the quota-safe localStorage wrapper was declared after the store that uses it, so the persist storage factory failed silently. Every settings write also threw mid-click, which kept the model picker popover from closing on selection - is_model_downloaded now verifies config, preprocessor config, and real weight files instead of trusting an offline snapshot lookup, so a partial download left by an aborted fetch shows the Download button instead of failing to load - Removed whisper.cpp mentions from user-facing text: the ready status shows Loaded instead of the runtime name, picker rows show the source repository, and runtime error messages say local transcription runtime Verified with automated browser sessions and live API checks: selection closes the picker with no page errors, persisted settings hydrate on reload, a stale partial snapshot triggers download then loads on MPS and transcribes, and curated models download from the unslothai repos. 88 backend STT tests, typecheck, lint, i18n parity, and build pass. * Skip the duplicate source line for custom models in the STT picker A custom repository's display name is its id, so search results and the appended current selection rendered the same string twice. The source line now only renders when it differs from the name; curated rows keep their name, unslothai source repository, and download size. * Verify every shard of a sharded checkpoint in the downloaded check A snapshot holding one of N shards (or a corrupt shard index) passed the downloaded check and then failed at load. When model.safetensors.index.json exists, every shard in its weight map must now be present. Found by simulation; covered by a regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename stale _starting references in the pump resilience tests The startup flag on TrainingBackend was renamed to _spawn_in_progress but two tests added alongside it still asserted on the old name, failing the Python 3.11 to 3.13 CI jobs. * Make the selected model row clearly highlighted in the STT picker The current selection was a faint background tint. It now uses the accent background with a medium weight name. Two line rows use a small corner radius; single line custom repo rows keep the pill shape. * Address review feedback on STT snapshot checks, VRAM release, and dictation UX Verify snapshot completeness in the load preflight so a partial download fails before the audio is decoded, for curated and custom repos alike. Drop the failed accelerator traceback before the CPU retry so the cache clear can actually release that memory. Keep unloading the GGUF sidecar after cancelling an in-flight Transformers load; both engines can hold memory at once. Allow Auto language with English-only .en checkpoints, matching the backend which sends no forced language. Keep the discard button usable while a transcription is pending so a slow or hung request cannot trap the composer in dictation mode. Stop linking Compare and settings test dictations to the unrelated active single chat thread. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Move the CPU retry out of the exception handler On Python 3.10 the interpreter exception state keeps its own reference to the traceback, so dropping it from the caught exception was not enough to release the failed accelerator load during the retry. Leaving the handler before clearing the cache works on every supported version. * Address review feedback on session handoff, chat pinning, and server lifetime Starting a dictation from a second entry point now cancels the session it replaces, so the old recording cannot keep the microphone open or save a transcript with no discard button pointing at it. The linked chat is pinned when recording starts, so switching threads while a transcription finalizes cannot relink the transcript to the newly opened chat. whisper-server is now bound to Studio's lifetime like the other long-lived children: PDEATHSIG on Linux, the parent job object on Windows, and pid adoption so the shutdown sweep reaps it; before this it survived a Ctrl+C exit as an orphan still holding the model. * Remove the dictation mic test from Voice settings The composer dictate button covers the same check, so the test row, its transcript panel, the unsupported fallback row, and their strings and search entry are gone. * Studio STT: gate GGUF whisper-server on training and fix dictation retry and dictionary edits GGUF (whisper.cpp) sidecar: - Launch whisper-server with --no-gpu while training is active, mirroring the Transformers sidecar's CPU device choice, so a mid-training dictation cannot reclaim the VRAM training just freed. - Report is_loading() during whisper-server startup so training VRAM admission accounts for the accelerator memory it is about to bind. - Require PyAV in is_available() so /audio/stt/status reports the engine unavailable when uploads cannot be decoded, instead of loading fine and then 501ing at transcription. - Reject a missing model before decoding audio, matching the Transformers download preflight. Voice settings: - The download Retry button now restarts the download; the sidecar error is sticky until a new start(), so re-polling alone never cleared it. Dictation dictionary: - Tabbing from an emptied entry to its remove button no longer commit-splices the row first, which shifted indices and deleted the wrong entry. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio STT: fix curated GGUF whisper filenames to match hosted repos The unslothai/whisper-*-GGUF repos host the checkpoint as whisper-<id>.bin, not ggml-<id>.bin, so every curated dictation download and cached-path lookup 404'd and the whisper.cpp engine could never load a model. Point GGML_STT_MODELS at the real filenames and guard the naming with a test. * Studio STT: validate a custom dictation repo before downloading it The Transformers STT engine accepts an arbitrary owner/model repo, but the download route handed it straight to snapshot_download, pulling a possibly large non-Whisper repository into the shared HF cache. Confirm the repo is a Whisper checkpoint first with the existing metadata-only validate_remote_model (no weights); curated ids short-circuit and the GGUF engine (curated-only) is unaffected. A non-Whisper repo now 422s before any download. * Studio STT: preempt a still-loading GGUF server for training admission A whisper-server still in its startup window binds accelerator memory but has no loaded_model yet, so training admission could miss it and launch into an OOM. Make the GGUF startup cancellable (cancel_pending_load signals an abort event and terminates the starting process without the load lock; _wait_for_server observes it and raises SttLoadCancelledError; wait_for_load_to_settle blocks on the lock until the killed server is reaped), and always fold the GGUF sidecar into the resident-STT summary so a resident Transformers model cannot mask a loading GGUF server. free_stt_model_for_training now cancels an in-flight load and waits for it to settle before training claims the memory. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio STT: fall back to Transformers when whisper-server is absent A curated dictation model (including the default small) hard-pinned the GGUF engine, but standard installs do not ship whisper-server, so every recording 501'd instead of using the Transformers engine that serves the same checkpoint -- the GGUF sidecar's own documented contract. Add _resolve_serving_stt_engine: a GGUF request for a curated id (the only ids GGUF accepts, all Transformers- servable) downgrades to Transformers when whisper-server is unavailable, applied consistently to download, load and transcribe (not unload, which targets a specific engine). The Voice tab likewise falls back to the Transformers status so the model is not shown unavailable and download is not blocked. * Studio STT: hide custom Whisper caches from the legacy model pickers The legacy /cached-models (and /cached-gguf) routes called is_hidden_model with only the owner/model id, which cannot reach the config-based Whisper check, so a downloaded custom (non-curated) Whisper checkpoint was still offered as a chat model. Pass the cached snapshot path so _path_is_whisper_model inspects the repo config and hides it, matching the discovery route. * Studio STT: hide GGUF dictation repos, lock-free status, unload fallback, split training eviction - Hide the curated GGUF dictation repos (unslothai/whisper-*-GGUF) from the chat model inventory and pickers, backend and frontend. Only their Transformers safetensors companions were hidden; the GGUF repos use a different org and a -GGUF suffix and carry a raw .bin with no whisper config.json, so they leaked into chat pickers. - Make the GGUF sidecar loaded_model/device accessors lock-free, mirroring the Transformers sidecar. transcribe() holds self._lock across the whole inference call, so /audio/stt status polls and training admission previously blocked behind an in-flight transcription. - stt_unload resolves through the serving resolver: a "gguf" pick on a host without whisper-server is served by the Transformers fallback, so unload must target that engine or the resident model is never freed. Unload also attempts every engine even if one raises, so a failure freeing one backend no longer skips the other. - free_stt_model_for_training frees the Transformers and GGUF sidecars under independent exception boundaries so a failure unloading one no longer skips the other before training claims the memory. Adds tests/test_stt_review_fixes.py covering all four. * Studio STT: resolve Auto dictation language for the model engine + snapshot process liveness - The model dictation adapter sent the raw setting (the literal "auto") to the backend, while the browser engine resolves Auto via resolveDictationLanguage. A batch of non-English voice notes came back mostly English on Auto. Add resolveModelDictationLanguage: only the literal "auto" is resolved to a concrete locale, gated so it becomes a language the model AND Whisper can honor (mirroring the backend's known-whisper-languages set); an explicit language, or a locale Whisper cannot honor, stays unchanged/auto-detect. Wire it into both adapter call sites. - GgmlSttSidecar._process_alive() read self._process twice; a concurrent unload() nulls it under the lock while loaded_model/device read lock-free, so a null between the two reads called None.poll(). Snapshot once. Adds a deterministic regression test. * studio: tighten comments and docstrings in the dictation modules * studio: harden dictation model downloads, GGML readiness, and recording paths Address review findings on the STT dictation feature: - build_whisper_cpp.sh refuses to delete a whisper.cpp tree under a custom Studio home unless it carries the Studio ownership marker, matching the setup.sh policy, and marks trees it creates - _snapshot_is_complete validates every shard of a sharded PyTorch (pytorch_model.bin.index.json) checkpoint like the safetensors path, and requires tokenizer assets (tokenizer.json or vocab.json + merges.txt) - custom-repo downloads pin the revision resolved at validation time and restrict snapshot_download to the model/tokenizer/config/preprocessor file classes Studio loads - the GGML sidecar holds its port reservation until just before spawning whisper-server and only accepts readiness from a responder that both looks like whisper.cpp's server and belongs to the still-running managed child, probing twice, so mic audio cannot be posted to a foreign local process - the recording adapter transcribes every non-empty segment; the RMS meter only shapes segment boundaries and can no longer discard quiet speech - Compare-pane dictation can cancel a pending transcription on second click, with the button relabeled while finalizing - localStorage quota recovery halves the dictation history until the save fits, so small histories shrink too - the System default TTS voice resolves to the platform default voice - new dictation UI imports go through the chat and hub feature barrels Regression tests cover the build-script gate, sharded PyTorch and tokenizer completeness, revision pinning and allow patterns, and the whisper-server readiness probe. * Fix STT download and voice picker follow-ups * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add dictation button regression coverage * Studio: prebuilt whisper.cpp via the shared llama.cpp install core, slim bundles paired to the llama prebuilt (#7294) * Studio STT: add prebuilt whisper.cpp (whisper-server) installer New install_whisper_prebuilt.py downloads a per-platform whisper-server bundle published by the unslothai/whisper.cpp prebuilt CI into the managed whisper.cpp dir (build/bin/whisper-server) so local dictation needs no compiler. Mirrors install_node_prebuilt.py / install_llama_prebuilt.py: host + backend detection, sha256 pins (whisper_prebuilt_pins.json) as the trust anchor, staging + install lock + atomic swap, traversal-safe extract, co-located shared libs (RUNPATH=$ORIGIN), an UNSLOTH_WHISPER_PREBUILT_INFO.json marker with idempotent "already matches", and exit codes 0/1/2/3. Not wired into setup yet; the pins ship empty so every asset fails closed until the first fork release is published and its digests are reviewed in. * Studio STT: install prebuilt whisper.cpp during setup and update Add a fail-open whisper.cpp block to setup.sh after the llama.cpp section so `unsloth studio update` (and a fresh install) fetch the prebuilt whisper-server into the managed whisper.cpp dir the sidecar discovers. It skips a user-set WHISPER_SERVER_PATH/UNSLOTH_WHISPER_CPP_PATH, honors UNSLOTH_SKIP_WHISPER_INSTALL, forwards the resolved ROCm gfx, and never aborts setup: a busy install keeps the existing runtime, and an unavailable prebuilt stays quiet (source build is opt-in via UNSLOTH_WHISPER_FORCE_COMPILE) since Transformers STT and browser dictation remain. Register UNSLOTH_WHISPER_PREBUILT_INFO.json as Studio-owned evidence. * Studio STT: harden whisper-server child env + WSL ROCm detection - Sidecar spawns whisper-server with a scrubbed child env that prepends the binary dir (co-located GPU libs) to the loader path, and on WSL2 ROCm loads the system HIP first (HSA_ENABLE_DXG_DETECTION=1) so a bundle's bare-metal HIP does not segfault on /dev/dxg. Secret-bearing vars are dropped from the child. - find_whisper_server_binary now requires an executable, not just a file. - Installer rocm probe passes HSA_ENABLE_DXG_DETECTION and falls back to /opt/rocm/bin/rocminfo so a WSL ROCm host is not misdetected as CPU-only; gfx parsing skips the gfx000 CPU agent and generic ISA lines. - Tests for the child env (secret scrub, lib dir, WSL HIP precedence), the executable check, and the WSL rocm detection. * Studio STT: in-app whisper.cpp prebuilt update stack + ship pins in the wheel Mirror the llama.cpp update stack for the whisper.cpp prebuilt so Studio can detect and install a newer whisper-server release from inside the app: - backend/utils/whisper_cpp_freshness.py: read UNSLOTH_WHISPER_PREBUILT_INFO.json and compare the installed release against the newest unslothai/whisper.cpp release. Whisper tags are v<upstream>-unsloth.<N>, so is_behind compares a (major, minor, patch, serial) key with a strict downgrade guard; 24h cache; fail-open. - backend/utils/whisper_cpp_update.py: run install_whisper_prebuilt.py to fetch and atomically swap the newest bundle, unloading the warm GGUF sidecar first. - backend/routes/whisper.py mounted at /api/whisper (update-status + update). - pyproject: add whisper_prebuilt_pins.json to studio package-data so the installer's trust anchor ships in the wheel (it is a data file, not a .py module, so package discovery alone does not include it; node_prebuilt_pins.json is listed for the same reason). Without this a pip-installed wheel had no pins and the prebuilt install aborted to Transformers STT. Adds test_whisper_cpp_freshness.py (version parser, is_behind matrix + downgrade guard, marker layouts, stale decision, fail-open). * Studio STT: verify whisper prebuilts via the release checksum index, like llama.cpp Re-align the whisper.cpp prebuilt installer to install_llama_prebuilt.py's trust model: instead of a committed whisper_prebuilt_pins.json, verify every download against the release's own whisper-prebuilt-sha256.json checksum index, fetched from the same GitHub release. - parse_release_checksums / fetch_release_checksums / expected_sha256_for replace the pins layer. The index is validated for schema/component and that its release_tag matches the resolved release; an asset absent from it, a release that does not publish it, or a manifest sha256 that disagrees with it all fail closed to a source build. - resolve_release_tag now resolves the newest published release at runtime (or an explicit --published-release-tag), matching llama and the freshness check; removed the pinned-default and the UNSLOTH_WHISPER_ALLOW_UNVERIFIED opt-in. - Delete studio/whisper_prebuilt_pins.json and drop its pyproject package-data entry (nothing to ship now, same as llama which has no committed pins). - Adds test_install_whisper_prebuilt_checksums.py (index parser, fail-closed on uncovered asset, tampered-manifest guard, newest-release resolution). This is a same-origin checksum (integrity, not authenticity), identical to the llama.cpp installer; pair releases with GitHub artifact attestations for provenance. * Resolve whisper prebuilt release via the download host (no GitHub API) Mirror install_llama_prebuilt.py's fast path: resolve the release tag from the releases/latest redirect and fetch the manifest + checksum index from constructed releases/download URLs, so the common install path makes zero api.github.com calls (unauthenticated api.github.com is capped at 60 req/hour per IP; the download host is not). Fall back to the GitHub API only on a 404, malformed asset, or tag mismatch. * Studio STT: coverage-aware whisper prebuilt selection via a shared core whisper's select_artifact returned the first os/arch/backend manifest match and ignored the SM-coverage fields the release manifest already carries, so a Blackwell B200 (sm_100) was served cuda12-legacy (sms 50-61) -- runnable only via forward PTX JIT. install_llama_prebuilt.py on the same host correctly picks cuda13-newer. Extract the coverage-aware selection into a shared, component-agnostic core under studio/backend/utils/prebuilt/ (selection + GPU host-capability detection), lifted from llama's linux_cuda_choice_from_release / _artifact_covers_sms / _sm_range and generalised over a normalised artifact. whisper's HostInfo now records the GPU compute caps + driver CUDA version (honoring CUDA_VISIBLE_DEVICES), and select_artifact routes CUDA/ROCm through the shared selector: every visible SM must be covered, the tightest-covering profile wins (Blackwell-aware runtime-line ordering), ROCm matches the gfx target exactly, and an uncovered GPU falls back to the CPU bundle. CPU/Metal/Vulkan keep first-match. The resolver JSON, exit codes, and "already matches" contract are unchanged. On the B200 the installer now resolves cuda13-newer, matching llama. * Studio STT: gate whisper CUDA selection on the on-disk runtime, like llama The prebuilt CUDA bundles are dynamically linked and intentionally do NOT ship libcudart/libcublas -- they load the same runtime the host already has. So the driver's advertised CUDA version is only an upper bound: a cuda13 bundle still needs cuda13 runtime libraries present on disk. Port llama's on-disk runtime scan (detected_linux_runtime_lines / detected_windows_runtime_lines) into the shared core and intersect it with the driver-compatible lines in select_cuda_attempts. A host with a cuda13 driver but only cuda12 runtime (e.g. torch-cuda12) now correctly gets a cuda12 bundle instead of an unloadable cuda13 one; a host with no CUDA runtime at all falls back to CPU. Fixes a glob bug in the port (any(Path(d).glob(p) for d in dirs) tests generator truthiness, not a match) that made every major report present; add a real filesystem test that exercises the scan. * studio: harden shared prebuilt core to full llama parity Apply the review findings on the shared coverage-aware prebuilt-consumer core so whisper.cpp selection is exactly equivalent to the llama.cpp path. hosts.py: port llama's CUDA_VISIBLE_DEVICES handling. A GPU hidden by an index/UUID selector now reports has_usable_nvidia False instead of staying usable, via supports_explicit_visible_device_matching plus the physical / explicit-match branches, and _select_visible_rows now matches rows the way llama does (index or UUID, gpu- prefix optional) and skips unmatched tokens rather than keeping all rows. Adds the Linux /proc/driver/nvidia/gpus fallback and has_physical_nvidia. Adds parse_macos_version. runtime_libs.py: the Linux on-disk scan now requires the exact libcudart / libcublas SONAME (libcudart.so.13), not a libcudart.so.13* glob, so a bare versioned file without the SONAME symlink no longer counts as loadable. Hardens the ldconfig parse against an empty left-hand side. selection.py: fix the Blackwell/torch reordering so it keys on the covering runtime lines (falls through to the torch preference when the covering lines were filtered out), matching linux_cuda_choice_from_release. Corrects the compatible_runtime_lines_for_driver docstring: the bundles do not ship the CUDA runtime, so the driver version is only an upper bound and the caller must intersect with the on-disk scan. install_whisper_prebuilt.py: enforce a macOS artifact's min_os (new HostInfo.macos_version) so a bundle that cannot load on the host OS version is dropped. Keep resolver stdout to only the JSON line by leaving logs on stderr in --resolve-prebuilt mode, and map an unexpected probe failure to prebuilt_available False instead of a traceback. Tests: new host-probe suite for the visible-device logic, exact-SONAME runtime-scan cases, macOS min_os filtering, resolver stdout-only-JSON, exit-code mapping, and the repo key. * studio: fix whisper prebuilt selection + launch parity gaps from review A parallel review surfaced integration defects where the whisper path could select or launch a bundle that cannot run on a concrete host. Each is fixed to match install_llama_prebuilt.py. macOS min_os: the manifest labels macOS requirements as macos-<version> (e.g. macos-14.0), which the version parser could not read, so the guard was a no-op and a macOS-13 host would install the macos-14 Metal bundle. Strip the platform prefix before parsing. ROCm gfx detection: _detect_rocm_gfx returned the first gfx token and ignored HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES. Since exact ROCm matching treats that token as the active GPU, a mixed APU + dGPU host (gfx1151 + gfx1100) with HIP_VISIBLE_DEVICES=1 installed the wrong archive. Route through a shared pick_rocm_gfx_target (lifted from llama) that parses per-GPU sections and honors the visibility vars (empty / -1 -> no AMD GPU). --rocm-gfx override: recording the arch without setting has_rocm left the host on its CUDA/CPU path so the ROCm bundle was never picked. --rocm-gfx now implies has_rocm and clears NVIDIA state, like llama's _apply_host_overrides. CUDA launch env: a CUDA bundle ships the ggml CUDA backend but not libcudart/libcublas, and the sidecar launch env exposed only the bundle dir, so on a host whose CUDA runtime lives only in the PyTorch wheels the selection would gate cuda usable but the server could not load it. Add the CUDA-from-PyTorch runtime dirs to the child loader path for CUDA bundles (bundle dir still first), mirroring binary_env. Also normalize a manifest artifact's supported_sms defensively (parity with llama's parser) and document that blackwell_min_toolkit_for_caps is retained for the Phase B llama Windows path. Not changed (verified parity, not defects): Linux/Windows min_os is enforced nowhere in llama (macOS only); the resolver is optimistic about the checksum index and the install path verifies. * studio: tighten prebuilt-core code comments * studio: lift shared prebuilt installer core out of the whisper installer * studio: reuse the llama.cpp prebuilt installer machinery for whisper * studio: unify llama and whisper prebuilt installers on a shared descriptor core * studio: consolidate prebuilt installer tests into the shared core suite Grow tests/studio/install/test_prebuilt_core.py from 62 to 164 tests so every component-agnostic behavior runs against both descriptors: the full seven profile CUDA release matrix (multi-GPU, on-disk runtime gating, shuffle stability, missing SM metadata, dotted SM normalization, no-driver fallback policy), the ROCm gfx family matrix, macOS min_os gating and its helper, backend resolution incl. cpu-fallback precedence and Intel-mac auto detect, checksum-index non-object and plain-lookup cases, the tar symlink/hardlink extraction guards moved from the llama suite, and the compute-cap, visible device, runtime-line and Blackwell helper value tables moved verbatim from the llama characterization suites. Delete only tests whose exact behavior the master now asserts for the same component: 40 pure-alias helper cases in test_selection_logic.py (replaced by value-identical master tables plus an alias-identity pin), 6 extraction moves and the master-absorbed zip-symlink case in the llama logic suite, 3 routing twins in test_rocm_support.py already pinned byte-for-byte in test_selection_logic.py, the 2 Blackwell helper tables in the backend resolve suite, 28 whisper logic tests and 10 whisper checksum tests re-asserted by the master whisper parameterization. Wrapper wiring pins, the llama release plan dialect, fingerprints and every llama-only behavior stay untouched. * studio: dedupe sidecar and update helpers into the backend prebuilt package * studio: chain whisper.cpp prebuilt updates onto the llama.cpp update flow * studio: consume paired slim whisper prebuilts via the llama ggml runtime * studio: serve every whisper backend from slim prebuilts * studio: drop the whisper fat per-accelerator selection chain unslothai/whisper.cpp releases are slim-only from v1.9.1-unsloth.2: one ggml-less bundle per os/arch, paired to the llama.cpp prebuilt that provides every ggml backend. Delete the whisper-side fat CUDA/ROCm/metal/vulkan selection glue; keep slim selection + pairing, link_ggml_runtime, and one legacy shape, the published fat CPU bundle of an explicitly pinned pre-slim release. Exit 2 now reads as prebuilt unavailable (whisper never source builds); setup already treats it that way. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Wire libomp runtime DLL alongside ggml in slim whisper installs llama's clang-built windows-arm64 ggml-base.dll imports libomp140.aarch64.dll, shipped in the llama bundle but not a system DLL. Without it next to whisper-server.exe the loader fails with STATUS_DLL_NOT_FOUND before main. MSVC x64 links vcomp140.dll from System32 and Linux ggml uses system libgomp.so.1, so only windows-arm64 was affected. The empty-runtime guard still requires a real ggml library; libomp alone is not a pairing. * studio: drop whisper-side fat-selection support structure Slim whisper bundles are selected per os/arch only; all accelerator capability comes from the installed llama.cpp prebuilt, whose installer already did the coverage-aware selection. Remove the machinery that only existed to pick among fat per-accelerator whisper bundles: - prebuilt_core: delete the generic CUDA/ROCm coverage selection (select_cuda_artifact, select_rocm_artifact, ArtifactView adapters, detected_cuda_runtime_lines, the exact-SONAME linux probe) that no shipped component routes through; llama keeps its own selection chain and whisper shadows select_artifact with the slim-only version. select_artifact is now a plain os/arch/backend first-match. - install_whisper_prebuilt: drop the HostInfo CUDA fields (compute_caps, driver_cuda_version, torch_runtime_line) and the torch runtime probe that populated them; nothing reachable reads them, and the resolver payload sources runtime_line from the artifact. - whisper_cpp_update: delete the standalone start_update job worker; whisper applies only run as the chained phase of the combined llama+whisper update. The status payload keeps its job field (idle). - routes/whisper: drop the progress logger that could never fire. - tests: remove tests of the deleted paths and tests duplicating the descriptor-parameterized core suite or the llama freshness suite. Contracts unchanged: resolver JSON keys, exit codes, marker fields, pairing logs, and the pinned pre-slim fat CPU escape hatch. * Address review feedback on the whisper prebuilt update and install paths - Pin the chained whisper phase to the release the freshness check offered, so the download-host latest pointer cannot reinstall an older build in a loop - Wire the whisper prebuilt install into setup.ps1 (Windows setup previously skipped it entirely) - Treat a non-executable server or missing wired ggml libraries as a broken install instead of reporting already matches - Keep whisper sidecar reloads out of the job-level reload flag and resync chat state after a partial chained update that unloaded llama - Repoint home and profile vars for the whisper-server subprocess at a managed scratch dir and drop credential-store pointers - Clear the prebuilt marker before the opt-in source build overwrite - Write the prebuilt marker with explicit utf-8 encoding * Tighten comments in the whisper prebuilt consumer * Harden the Windows whisper setup phase and the chained update edges - setup.ps1: honor WHISPER_SERVER_PATH / UNSLOTH_WHISPER_CPP_PATH / UNSLOTH_SKIP_WHISPER_INSTALL, run the custom-home ownership guard before the atomic install, and forward the release-tag pin and ROCm hints like setup.sh - sidecar: a cpu-selected install launches whisper-server with --no-gpu (slim wiring links every llama backend, so the flag is what keeps a deliberate CPU choice off the GPU) - chained update: leave whisper unpinned on macOS (the llama phase can walk back there, and a newest-tag pin could be an impossible pairing on every retry) and treat installer exit 2 as kept-existing-runtime instead of failing the combined job - job.to_tag now comes only from the llama phase, so a whisper-only round cannot report a llama update that never ran * Fix slim whisper runtime follow-ups * Address remaining whisper update reviews * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address remaining prebuilt update reviews * Fix remaining chained update reviews * Fix remaining whisper runtime review edges * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> |
||
|
oobabooga
|
dbb06ff60e |
Studio: add configurable model download location (#7274)
Adds a configurable Hugging Face model download cache location to Unsloth Studio, selectable from Settings, with per-cache download manifests, scoped deletion, and read-only inventory of previously selected caches. |