32 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Daniel Han
|
ef97f3c961 |
tests: record the gfx1152 llama.cpp bundle gap so the next one is not silent (#7491)
* tests: record the gfx1152 llama.cpp bundle gap so the next one is not silent #7431 made gfx1152 (Krackan Point, Radeon 860M/840M) a first-class arch, which fixed torch wheel selection: those laptops were pulling gfx1150 wheels built for a different LLVM target. It also changed llama.cpp prebuilt selection, because no gfx1152 bundle is published. published_rocm_choice_for_host deliberately refuses to serve a sibling-family bundle, so those hosts now fall back to a HIP source build. That is the right outcome, a wrong-ISA binary fails at the first BLAS call rather than merely installing slowly, but nothing recorded it and nothing would have caught it. TestPublishedRocmGfxSelection builds its release from a hardcoded family list, so it can only assert about arches someone already thought to add. Adds TestPublishedRocmBundleCoverage: - PUBLISHED mirrors the mapped_targets in llama-prebuilt-manifest.json. - KNOWN_GAPS lists arches _GFX_TO_AMD_INDEX_ARCH routes torch for that no bundle covers: gfx1033/1035/1036 (RDNA 2, never built) and gfx1152. - test_known_gaps_fall_back_to_source_build pins each to None. - test_every_torch_routed_arch_is_covered_or_a_known_gap compares the routed set against bundle coverage, so adding an arch for torch without a bundle has to be a deliberate KNOWN_GAPS entry. The invariant fires both ways. Simulating a new routed arch fails with "coverage drifted: ['gfx1153'] newly uncovered"; simulating a published gfx1152 bundle fails with "gfx1152 is in KNOWN_GAPS but a bundle now matches it; drop it from the set", so closing the gap cannot leave the list stale. Reads _GFX_TO_AMD_INDEX_ARCH from source instead of importing install_python_stack, which this suite does not otherwise depend on. No production code changes. Install suite 1355 passed, no new failures. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Michael Han
|
d5cf96d628 |
Studio: add local speech-to-text dictation engine (#7095)
* Studio: add Voice settings tab (dictation, dictionary, read aloud) New Voice tab in Settings, placed just before About: - Dictation: microphone picker, browser STT engine, recognition language, and an inline mic test with a live transcript - Dictation dictionary: entries rewrite matching speech to their exact spelling and casing, applied in both dictation paths - Recent dictations: last 20 final transcripts with copy and clear, so text can be recovered if it lands in the wrong place - Read aloud: optional button on assistant responses with two engines, curated system voices (novelty and legacy voices filtered, quality ranked, capped at 20) or the TTS audio model loaded in Unsloth via /audio/generate (e.g. Orpheus), plus speed, pitch, volume and preview Settings persist in localStorage (unsloth_voice_settings) and are read at call time so changes apply without reloading the runtime. Adds en keys plus the tab label for ja, zh-CN and pt-BR. * Studio: drop the single option STT engine select, rename TTS option The STT engine dropdown only had one entry, so it added noise without giving a real choice. The engine row can come back once local STT models land. Also renames the TTS engine option Unsloth TTS model to Load TTS model to make the action clearer. * Studio: harden Voice settings against edge cases found in simulation Simulated the feature across Chromium, Firefox and WebKit plus node level unit runs and backend contract checks. Fixes from the findings: - Dictionary rewrite used a replacement string, so entries containing dollar patterns corrupted transcripts (A$$AP became A$AP, $& injected the match). Switched to the callback form of String.replace - Persisted voice settings now validate types on hydration: non string micDeviceId, dictationLanguage and ttsVoiceURI, and non boolean ttsEnabled fall back to defaults instead of flowing into the UI - Dictionary entries are trimmed, capped at 120 chars and re-sanitized on hydration - The Test dictation panel now falls back to the default microphone when the saved device is unplugged, matching the composer adapter Test coverage: 46 unit assertions (dictionary regex edge cases across unicode, word boundaries and injection, voice curation for simulated macOS, Windows and Linux voice inventories, corrupt storage merge), 13 backend contract checks against /audio/generate on an isolated instance, and 60 browser assertions across the three engines covering rendering, degradation without SpeechRecognition, curation in a real DOM, dictionary persistence with unicode and dollar entries, the no-model preview error path and corrupt localStorage recovery. * Studio: address Voice settings review feedback Verified each review comment before acting. Confirmed and fixed: - Editing a dictionary entry was broken in two ways: the store trimmed on every keystroke so spaces could not be typed, and clearing the field deleted the entry and unmounted the input mid edit. Updates now keep the raw value and a blur commit trims or removes the entry - The unplugged mic fallback checked instanceof DOMException, but a cross browser probe showed Firefox and WebKit throw OverconstrainedError objects that are not DOMExceptions, so the fallback never fired there. Matching on the error name now - When the browser ended a dictation test on its own (silence timeout), the mic stream stayed open. All recognition end paths now stop the tracks and save the transcript through a single finalize path - The studio TTS audio element now releases its WAV data URL as soon as playback ends, fails or is cancelled - Allow microphone now reports insecure contexts (no mediaDevices) accurately instead of claiming access was blocked - Voice tab copy moved into i18n keys per src/i18n/AGENTS.md, so locale overlays can translate it; en is the baseline and parity passes - unsloth_voice_settings added to the Reset all local preferences key list so voice preferences obey the reset - Non default microphones note that the system default is used when the browser speech engine cannot bind a specific device, since browsers without the start(track) overload ignore the argument silently Re-ran the full simulation set after the changes: 46 unit assertions, 13 backend contract checks and 60 browser assertions across Chromium, Firefox and WebKit all pass, plus a dedicated browser probe for the dictionary editing behavior. * Studio: use the chat mic icon in Voice settings for consistency The Voice tab and its buttons used the hugeicons Mic02 glyph while the chat composer uses a custom filled mic. Extract that composer icon into a shared lib/mic-icon component, drop the duplicate inline copies in thread.tsx and shared-composer.tsx, and use it for the Voice tab icon and the tab's mic buttons so the microphone looks the same everywhere. * Studio: address second round of Voice settings review feedback Verified each new comment against the current code first. One item was already fixed in the previous round (recording transcripts when the browser ends a dictation test on its own). Confirmed and fixed: - The microphone row showed a picker with generic names when browsers enumerate unlabeled devices before permission, leaving no way to grant access from the row. It now branches on whether labels are visible and shows Allow microphone otherwise - Compare chat dictation ignored the selected microphone. It now opens the chosen device with the same fallback rules as the main adapter, passes the track to recognition where supported and releases the stream when recognition ends - Closing the Voice tab cancelled the shared speechSynthesis even when read aloud was playing a chat message. Cleanup now only cancels when the tab owns an active preview - Double clicking Start test could race two recognizers and leak the first stream. A starting flag set before the getUserMedia await makes start reentrancy safe - Turning off the read aloud setting mid playback removed the only stop control. The stop button now renders whenever a message is speaking - When an engine lacks the start(track) overload, both dictation paths now release the selected device stream before retrying with the default microphone instead of holding it open - Read aloud support no longer requires Web Speech synthesis: the Unsloth TTS engine only needs audio playback, so it stays available in WebViews without speechSynthesis, with a clear error if the system engine is chosen there Not addressed here: cancelling in flight backend TTS generation on stop. The route runs generation in a worker thread without a cancellation path, which is shared pre existing behavior with audio chat generation and belongs in a backend change. All suites re-run green: 46 unit, 13 backend contract and 60 browser matrix assertions across Chromium, Firefox and WebKit, plus probes for the unlabeled device branch and the double click race. * Studio: drop empty and duplicate voiceURIs so the Voice tab never renders a crashing Select item * Studio: guard dictation mic lifecycle in Voice test and Compare composer Release a microphone opened after the component unmounts, and stop Compare dictation on a permission or security failure instead of silently recording from the default device, matching the main chat adapter. * Studio: fix dictation and read-aloud lifecycle edge cases in Voice settings - Join final dictation chunks with a space so recorded transcripts do not merge words - Ignore a stale recognizer onend so a quick stop then restart is not torn down - Use previewingRef so a double click on TTS preview does not orphan the first request - Keep the read-aloud stop control visible when a new run starts while a message is spoken - Stop the dictionary remove button from deleting an adjacent entry on a blur then click race * Studio: trim redundant Voice settings comments * Studio: fix Voice preview and Compare dictation edge cases - Only cancel the shared speechSynthesis for a system-voice preview, so stopping a Studio preview no longer stops an unrelated chat read-aloud - Release the Studio preview audio and its WAV data URL on normal completion - Iterate every finalized result in Compare dictation so batched phrases are kept - Cap persisted recent dictations to the last 20 on hydration * Studio: use clipboard fallback for recents and release failed preview audio - Copy recent dictations via the copyToClipboard helper so the execCommand fallback works in Safari and insecure http LAN contexts - Release the Studio preview audio when play() rejects, not just on ended/error * Studio: add local speech-to-text dictation engine Add an offline dictation engine that transcribes with a local faster-whisper model, alongside the existing browser (Web Speech) engine. The browser engine streams audio to Apple or Google speech services and needs internet; the new engine runs on the server, works offline, and drives any chat model without evicting it (it loads in the backend process, separate from the model subprocess). It also gives Firefox dictation, which has no Web Speech support. Backend: a lazily-loaded, kept-warm faster-whisper sidecar and three routes under /api/inference/audio (stt/status, stt/load, transcribe). faster-whisper is torch-free, so this does not disturb the existing model stack. Frontend: a Dictation engine setting (browser or local model), a curated model picker with sizes, and MediaRecorder capture posted to the transcribe route. The model warms automatically when the engine is selected, with live status. * Studio: stream local STT transcription as you speak Local dictation showed nothing until you stopped, because the whole clip was transcribed once on stop. Now the growing recording is re-transcribed on a fast pass every second and emitted as live interim text, with an accurate final pass on stop. Partial recordings decode fine, and the model refines earlier words as more audio arrives. Adds an interim flag to the transcribe route (beam 1, no VAD) for the fast preview pass; the final stop uses the accurate path. * Studio: make local dictation stop instant and reliable Stopping local dictation waited for a final network transcription before the session ended, so the stop button did not flip and a second click ended the session early and dropped the text. Now stop commits the live transcript immediately, releases the mic at once, and ignores a second stop while finalizing. Previews run more often so the committed text is current. * Studio: record local dictation in short clips for reliable streaming Re-transcribing a growing buffer every second got slower as it grew, flooded the backend, showed stale words, and could leave the stop button stuck waiting on a backlog. Record short independent clips instead and transcribe each once, appending the text as you speak. Work per clip is bounded, so stopping is prompt (with a hard timeout as a safety net) and long dictations stay smooth. * Studio: dictate then transcribe once on stop, ChatGPT style Local STT dictation streamed by re-transcribing the growing clip, which was quadratic and saturated the backend (multi-second lag), and stop only halted the recorder without releasing the mic, so it kept recording. Record the microphone continuously, release it the instant the user stops, and transcribe the whole clip once. Stopping is immediate and the transcript lands in about a second. Also add the tiny model for the fastest option. * Studio: surface dictation and read-aloud failures instead of failing silently - Compare dictation reports microphone and speech-recognition errors via toast, reusing the main chat adapter's describeMediaError and describeSpeechError - Read-aloud toasts genuine model or synthesis failures while ignoring cancellations * Studio: ChatGPT-style recording bar for dictation Clicking the mic now drops the composer into a dedicated recording bar with a live waveform, a discard (X) and a confirm (tick), instead of a plain stop button. The tick stops recording and transcribes the clip; the X throws the recording away and keeps whatever text was already in the composer. The model adapter taps the mic with an analyser to drive the waveform, and the router tracks the live session so the X can cancel it without transcribing. * Studio: transcribe dictation while speaking, ChatGPT layout Match ChatGPT's recording layout: the bar now renders in place of the input with the left plus button kept, the waveform in the middle, and the discard and confirm buttons together on the right. Cut the post-confirm delay by transcribing in the background as the user talks. The audio is split at natural pauses (voice-activity detection off the same analyser that drives the waveform) and each clip is transcribed as it is cut, so confirming only has to finish the short final tail. The model is also warmed when recording starts so the first run never pays a cold load. * Studio: ChatGPT waveform, hide tools while dictating, faster STT Make the recording UI read like ChatGPT: the waveform is now a dense row of round dots that rise into thin centered bars, and while dictating only the plus button shows, with the mode badge and tool toggles hidden so the bar is just the waveform and controls. Speed up transcription: decode greedily (beam_size=1), which is several times faster on CPU with negligible accuracy loss on short dictation clips, and cap background segments at 6s so the final tail after confirm stays short. * Studio: finish ChatGPT voice bar and low-latency STT * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: full-width waveform with a timer that freezes on stop Use the full-width waveform for the recording bar: brighter, bigger bars that advance on a fixed cadence (keeping peaks between advances) so they glide instead of racing by, inset from the composer edges. Keep a visible timer and the green confirm button, matching the ChatGPT reference, and freeze the timer and waveform the moment the user confirms. * Studio: fix multilingual local dictation * Studio: speed up dictation and release local STT * Studio: harden dictation finalization and STT decoding * Studio: restore Firefox dictation fallback * Studio: add dictation history manager * Studio: manage speech model downloads * Studio: remove em dash from voice model label * Studio: move dictation history into Voice * Studio: source local STT from Unsloth Whisper models Point the dictation STT sidecar and its Model Hub download entries at Unsloth's Hugging Face Whisper repos (small, large-v3-turbo, large-v3) and run them through Transformers, so Studio only ever downloads Unsloth-uploaded weights. Drop faster-whisper and the Systran/mobiuslabs repos; keep the Model Hub as the only download path via local_files_only, and keep PyAV for audio decoding. Device selection uses float16 on CUDA and float32 on MPS and CPU, since Whisper's decoder is unstable in float16 on MPS and repeats tokens. Shorten the model picker labels to name plus download size and update the STT tests for the new backend. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: smooth dictation waveform and keep pill height * Studio: align STT model dropdown width and tidy voice copy * Studio: guide to local engine when browser dictation is offline * Studio: clarify voice section and STT model copy * Studio: keep STT warm with training-aware eviction * Harden STT lifecycle and browser compatibility * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix model discovery test lint * Harden cross-browser microphone errors * Harden cross-browser microphone errors * Surface voice test recognition errors and fall back to Studio TTS - Voice test now toasts non-abort speech-recognition failures instead of ending silently, matching the main and Compare dictation paths. - Read-aloud routes to the backend model when the runtime lacks Web Speech synthesis (audio-only WebView), so it no longer errors immediately. * Fix reviewed STT lifecycle races * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix read-aloud fallback controls * Guard read-aloud stop when deleting a non-speaking message aui.message().stopSpeaking() throws unless this message is the one being read aloud, so calling it unconditionally rejected the delete handler before the message was removed. Only stop speech when this message is speaking. * Cap recent dictation transcript length before persisting Recent dictations only limited entry count, so a long transcript stored the full text in the persisted voice settings and a few could exceed the localStorage quota, throwing synchronously from the uncaught dictation cleanup path. Truncate each entry on save and on hydration, matching the dictionary cap. * Studio: keep dictation mic clickable and guide to local model Register the dictation adapter unconditionally so the mic stays enabled for any engine and starts working right after switching to the local model on an already-open thread. When the browser engine cannot run (Firefox, Brave, non-secure origins), clicking the mic shows a toast that points to the local speech-to-text model instead of leaving a disabled button. The toast stacks its action below the text with a fully rounded button. * Studio: add bottom padding below the dictation guidance toast button * Studio: increase bottom padding under the dictation toast button * Studio: add bottom padding inside the dictation toast button * Studio: add five Whisper defaults and custom model search Add private UnslothAI Tiny and Base mirrors to the curated local STT choices while keeping Small as the default. Let users search or paste a Transformers-compatible Whisper repository and validate it end to end. Keep short dictations in one clip to avoid repeated padded encoder work, then split longer recordings near Whisper's 30-second boundary. Update hidden model filters and tests, including the CPU-only CI runtime stub for PyAV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: use public Unsloth Whisper repositories Point the Tiny and Base dictation defaults to the public unsloth repositories and remove the private mirror references from model filtering and tests. * Studio: update Whisper download sizes Reflect the cleaned public Tiny and Base repositories in the curated model labels. * Studio: right-align STT model size, fix dropdown wheel scroll, refresh sizes - Show the download size on the right of each model row so long names like Whisper Large v3 Turbo no longer hide it - Update curated Whisper sizes to the safetensors weights actually downloaded: Tiny 151 MB, Base 290 MB, Small 967 MB - Drive the model list scroll from a wheel handler so the mouse wheel scrolls it inside the Settings dialog, not just the scrollbar - Add a search icon and shorten the placeholder to Search model * Studio: do not search when a dictation model is picked, shrink repo label - Treat the filled-in model text as a selection, not a query, so choosing a model no longer kicks off a Hugging Face search - Make the repository line under each model name smaller * Studio: tighten dictation model and local engine descriptions * Studio: keep model display on pick instead of the query, shrink row text - Guard the combobox input so selecting a model shows its name and does not echo the typed query back or start a search - Map the item label to the friendly display so picks fill the field - Reduce the model name and size text in each row * Studio: show only the model name in the dictation field, shrink size label - Drop the download size from the search field; the name alone is shown once a model is selected, with sizes kept in the dropdown list - Reduce the size label text in each row * Studio: clarify the dictation model description * Studio: drop Hugging Face from the dictation model description * Studio: move the dictation dictionary to its own Manage subpage - Replace the inline entry list with a Manage row, matching Dictation history, so a long dictionary no longer crowds Voice settings - Add a DictationDictionaryView subpage that holds the entry editor * Studio: match STT field font, use best voice for System default - Bump the dictation model field text to text-sm so it matches the engine dropdown next to it - Resolve the System default read-aloud voice to the top curated voice instead of the browser default, which is a robotic legacy voice on macOS * Studio: rerank read-aloud voices and drop duplicate voice entries - Rank by vendor quality, then the user's locale, then a preferred list of natural voices, so the best voice leads instead of the first alphabetically - Collapse voices that macOS reports twice under one name and language * Studio: fold dictionary and recents into the dictation section - Drop the separate Dictation dictionary and Recent dictations headings; their Manage rows now sit under Dictation, split by the row divider - Shorten the custom spellings description * Studio: add search and sort to dictation history - Filter saved dictations by text with a search field - Sort by newest, oldest, or A to Z; show a no-matches message - Keep Clear all available regardless of the current filter * Studio: settle cancelled STT loads before training and fix dictation review items Wait for a cancelled STT load to exit and release its memory before reporting it freed for training, so the loader cannot still be inside from_pretrained()/.to(device) holding VRAM when the training subprocess starts. A load that finishes before observing the cancel now gets unloaded so the memory is actually reclaimed. Clear the accelerator cache before the CPU fallback in load() so a failed CUDA/MPS load does not strand reserved VRAM once the sidecar is marked CPU-resident. Send the saved Hugging Face token when polling STT download progress so a gated or private repo resolves and shows the correct Load/Downloaded state instead of reporting missing. Mark the composer Dictate button as type="button" so clicking it does not also submit the draft when the composer already has text or attachments. * Studio: pin dictation settings per session and close STT startup races Capture the STT model and language when a dictation session starts and pass them to every queued segment and the warm-up load, so changing the model or language mid-recording no longer transcribes the same clip with the wrong model or a model that is not downloaded. Check the local runtime at the top of transcribe(), before the model cache lookup and the bounded audio decode, so a server missing PyTorch or Transformers returns 501 up front instead of decoding a long clip first. Treat the training startup window as active for STT device selection. start_training frees VRAM in before_spawn but only assigns _proc later, so a concurrent STT load could take the GPU that was just cleared. A startup flag now reports training active from the free until the process is live, forcing those loads to CPU; a finally clears it on every exit. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: stub the STT runtime check in transcribe orchestration tests transcribe() now verifies the local runtime up front, so the unit tests that exercise transcription orchestration must treat the runtime as present to keep passing where PyTorch, Transformers, and PyAV are not installed. Stub ensure_stt_available in the shared fixture and restore the real check in the availability and load-rejection tests. * Harden custom Whisper dictation models * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add whisper.cpp dictation engine with per-engine downloads and history rework Engines - New GGML STT sidecar that runs a managed whisper-server subprocess with idle unload, plus a pinned static build script (scripts/build_whisper_cpp.sh) - Dictation engine picker now offers Browser, Local transcription (whisper.cpp), and Local transcription (Transformers) - Both local engines serve the same five curated Whisper models and download them directly with byte-level progress reported by /audio/stt/status - Models auto load on selection and when their download finishes - Unload and training admission account for both engines Benchmarks (Apple Silicon, greedy, warm, same checkpoints) - whisper.cpp transcribes 2.4x to 5x faster than Transformers and loads in about 0.45s vs 0.86s for Whisper Small - whisper.cpp GGUF path is unchanged by the Transformers addition (load 0.445s -> 0.444s, short clip 0.391s -> 0.347s, long 1.197s -> 1.129s) Voice settings UI - Plain curated model select replaces the searchable combobox - Single download progress bar with transfer rate for both engines - Dictation history now stores every dictation with Show more pagination, a top Clear history action, and links back to the chat it was spoken into - Archived chats dialog gets the same pagination - Delete dialog offers deleting a dictation together with its chat Tests: 88 backend STT tests pass, including new snapshot download coverage. Frontend typecheck, lint, i18n parity, and production build pass. * Merge local engines into one option and source GGML models from unslothai Engine selection - The dictation engine dropdown is back to two choices: Browser and Local transcription. The selected model decides the backend: curated ids run GGML checkpoints through whisper.cpp, searched Hugging Face repositories run safetensors through Transformers - Model picker lists the curated models and searches Hugging Face for other Whisper repositories, validating them before selection. The trigger is a plain button so the selection never renders inside a text input - /audio/stt/status accepts a model query param so downloaded state works for custom repositories; the engine param on load, transcribe, and download routes is derived from the model everywhere Model source - Curated GGML checkpoints now download from the Unsloth-hosted unslothai/whisper-*-GGUF repositories (one repo per model) instead of ggerganov/whisper.cpp; cache lookups, progress totals, and in-flight blob tracking are per-model Fixes - Voice settings and dictation history were not persisting: the quota-safe localStorage wrapper was declared after the store that uses it, so the persist storage factory failed silently. Every settings write also threw mid-click, which kept the model picker popover from closing on selection - is_model_downloaded now verifies config, preprocessor config, and real weight files instead of trusting an offline snapshot lookup, so a partial download left by an aborted fetch shows the Download button instead of failing to load - Removed whisper.cpp mentions from user-facing text: the ready status shows Loaded instead of the runtime name, picker rows show the source repository, and runtime error messages say local transcription runtime Verified with automated browser sessions and live API checks: selection closes the picker with no page errors, persisted settings hydrate on reload, a stale partial snapshot triggers download then loads on MPS and transcribes, and curated models download from the unslothai repos. 88 backend STT tests, typecheck, lint, i18n parity, and build pass. * Skip the duplicate source line for custom models in the STT picker A custom repository's display name is its id, so search results and the appended current selection rendered the same string twice. The source line now only renders when it differs from the name; curated rows keep their name, unslothai source repository, and download size. * Verify every shard of a sharded checkpoint in the downloaded check A snapshot holding one of N shards (or a corrupt shard index) passed the downloaded check and then failed at load. When model.safetensors.index.json exists, every shard in its weight map must now be present. Found by simulation; covered by a regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename stale _starting references in the pump resilience tests The startup flag on TrainingBackend was renamed to _spawn_in_progress but two tests added alongside it still asserted on the old name, failing the Python 3.11 to 3.13 CI jobs. * Make the selected model row clearly highlighted in the STT picker The current selection was a faint background tint. It now uses the accent background with a medium weight name. Two line rows use a small corner radius; single line custom repo rows keep the pill shape. * Address review feedback on STT snapshot checks, VRAM release, and dictation UX Verify snapshot completeness in the load preflight so a partial download fails before the audio is decoded, for curated and custom repos alike. Drop the failed accelerator traceback before the CPU retry so the cache clear can actually release that memory. Keep unloading the GGUF sidecar after cancelling an in-flight Transformers load; both engines can hold memory at once. Allow Auto language with English-only .en checkpoints, matching the backend which sends no forced language. Keep the discard button usable while a transcription is pending so a slow or hung request cannot trap the composer in dictation mode. Stop linking Compare and settings test dictations to the unrelated active single chat thread. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Move the CPU retry out of the exception handler On Python 3.10 the interpreter exception state keeps its own reference to the traceback, so dropping it from the caught exception was not enough to release the failed accelerator load during the retry. Leaving the handler before clearing the cache works on every supported version. * Address review feedback on session handoff, chat pinning, and server lifetime Starting a dictation from a second entry point now cancels the session it replaces, so the old recording cannot keep the microphone open or save a transcript with no discard button pointing at it. The linked chat is pinned when recording starts, so switching threads while a transcription finalizes cannot relink the transcript to the newly opened chat. whisper-server is now bound to Studio's lifetime like the other long-lived children: PDEATHSIG on Linux, the parent job object on Windows, and pid adoption so the shutdown sweep reaps it; before this it survived a Ctrl+C exit as an orphan still holding the model. * Remove the dictation mic test from Voice settings The composer dictate button covers the same check, so the test row, its transcript panel, the unsupported fallback row, and their strings and search entry are gone. * Studio STT: gate GGUF whisper-server on training and fix dictation retry and dictionary edits GGUF (whisper.cpp) sidecar: - Launch whisper-server with --no-gpu while training is active, mirroring the Transformers sidecar's CPU device choice, so a mid-training dictation cannot reclaim the VRAM training just freed. - Report is_loading() during whisper-server startup so training VRAM admission accounts for the accelerator memory it is about to bind. - Require PyAV in is_available() so /audio/stt/status reports the engine unavailable when uploads cannot be decoded, instead of loading fine and then 501ing at transcription. - Reject a missing model before decoding audio, matching the Transformers download preflight. Voice settings: - The download Retry button now restarts the download; the sidecar error is sticky until a new start(), so re-polling alone never cleared it. Dictation dictionary: - Tabbing from an emptied entry to its remove button no longer commit-splices the row first, which shifted indices and deleted the wrong entry. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio STT: fix curated GGUF whisper filenames to match hosted repos The unslothai/whisper-*-GGUF repos host the checkpoint as whisper-<id>.bin, not ggml-<id>.bin, so every curated dictation download and cached-path lookup 404'd and the whisper.cpp engine could never load a model. Point GGML_STT_MODELS at the real filenames and guard the naming with a test. * Studio STT: validate a custom dictation repo before downloading it The Transformers STT engine accepts an arbitrary owner/model repo, but the download route handed it straight to snapshot_download, pulling a possibly large non-Whisper repository into the shared HF cache. Confirm the repo is a Whisper checkpoint first with the existing metadata-only validate_remote_model (no weights); curated ids short-circuit and the GGUF engine (curated-only) is unaffected. A non-Whisper repo now 422s before any download. * Studio STT: preempt a still-loading GGUF server for training admission A whisper-server still in its startup window binds accelerator memory but has no loaded_model yet, so training admission could miss it and launch into an OOM. Make the GGUF startup cancellable (cancel_pending_load signals an abort event and terminates the starting process without the load lock; _wait_for_server observes it and raises SttLoadCancelledError; wait_for_load_to_settle blocks on the lock until the killed server is reaped), and always fold the GGUF sidecar into the resident-STT summary so a resident Transformers model cannot mask a loading GGUF server. free_stt_model_for_training now cancels an in-flight load and waits for it to settle before training claims the memory. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio STT: fall back to Transformers when whisper-server is absent A curated dictation model (including the default small) hard-pinned the GGUF engine, but standard installs do not ship whisper-server, so every recording 501'd instead of using the Transformers engine that serves the same checkpoint -- the GGUF sidecar's own documented contract. Add _resolve_serving_stt_engine: a GGUF request for a curated id (the only ids GGUF accepts, all Transformers- servable) downgrades to Transformers when whisper-server is unavailable, applied consistently to download, load and transcribe (not unload, which targets a specific engine). The Voice tab likewise falls back to the Transformers status so the model is not shown unavailable and download is not blocked. * Studio STT: hide custom Whisper caches from the legacy model pickers The legacy /cached-models (and /cached-gguf) routes called is_hidden_model with only the owner/model id, which cannot reach the config-based Whisper check, so a downloaded custom (non-curated) Whisper checkpoint was still offered as a chat model. Pass the cached snapshot path so _path_is_whisper_model inspects the repo config and hides it, matching the discovery route. * Studio STT: hide GGUF dictation repos, lock-free status, unload fallback, split training eviction - Hide the curated GGUF dictation repos (unslothai/whisper-*-GGUF) from the chat model inventory and pickers, backend and frontend. Only their Transformers safetensors companions were hidden; the GGUF repos use a different org and a -GGUF suffix and carry a raw .bin with no whisper config.json, so they leaked into chat pickers. - Make the GGUF sidecar loaded_model/device accessors lock-free, mirroring the Transformers sidecar. transcribe() holds self._lock across the whole inference call, so /audio/stt status polls and training admission previously blocked behind an in-flight transcription. - stt_unload resolves through the serving resolver: a "gguf" pick on a host without whisper-server is served by the Transformers fallback, so unload must target that engine or the resident model is never freed. Unload also attempts every engine even if one raises, so a failure freeing one backend no longer skips the other. - free_stt_model_for_training frees the Transformers and GGUF sidecars under independent exception boundaries so a failure unloading one no longer skips the other before training claims the memory. Adds tests/test_stt_review_fixes.py covering all four. * Studio STT: resolve Auto dictation language for the model engine + snapshot process liveness - The model dictation adapter sent the raw setting (the literal "auto") to the backend, while the browser engine resolves Auto via resolveDictationLanguage. A batch of non-English voice notes came back mostly English on Auto. Add resolveModelDictationLanguage: only the literal "auto" is resolved to a concrete locale, gated so it becomes a language the model AND Whisper can honor (mirroring the backend's known-whisper-languages set); an explicit language, or a locale Whisper cannot honor, stays unchanged/auto-detect. Wire it into both adapter call sites. - GgmlSttSidecar._process_alive() read self._process twice; a concurrent unload() nulls it under the lock while loaded_model/device read lock-free, so a null between the two reads called None.poll(). Snapshot once. Adds a deterministic regression test. * studio: tighten comments and docstrings in the dictation modules * studio: harden dictation model downloads, GGML readiness, and recording paths Address review findings on the STT dictation feature: - build_whisper_cpp.sh refuses to delete a whisper.cpp tree under a custom Studio home unless it carries the Studio ownership marker, matching the setup.sh policy, and marks trees it creates - _snapshot_is_complete validates every shard of a sharded PyTorch (pytorch_model.bin.index.json) checkpoint like the safetensors path, and requires tokenizer assets (tokenizer.json or vocab.json + merges.txt) - custom-repo downloads pin the revision resolved at validation time and restrict snapshot_download to the model/tokenizer/config/preprocessor file classes Studio loads - the GGML sidecar holds its port reservation until just before spawning whisper-server and only accepts readiness from a responder that both looks like whisper.cpp's server and belongs to the still-running managed child, probing twice, so mic audio cannot be posted to a foreign local process - the recording adapter transcribes every non-empty segment; the RMS meter only shapes segment boundaries and can no longer discard quiet speech - Compare-pane dictation can cancel a pending transcription on second click, with the button relabeled while finalizing - localStorage quota recovery halves the dictation history until the save fits, so small histories shrink too - the System default TTS voice resolves to the platform default voice - new dictation UI imports go through the chat and hub feature barrels Regression tests cover the build-script gate, sharded PyTorch and tokenizer completeness, revision pinning and allow patterns, and the whisper-server readiness probe. * Fix STT download and voice picker follow-ups * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add dictation button regression coverage * Studio: prebuilt whisper.cpp via the shared llama.cpp install core, slim bundles paired to the llama prebuilt (#7294) * Studio STT: add prebuilt whisper.cpp (whisper-server) installer New install_whisper_prebuilt.py downloads a per-platform whisper-server bundle published by the unslothai/whisper.cpp prebuilt CI into the managed whisper.cpp dir (build/bin/whisper-server) so local dictation needs no compiler. Mirrors install_node_prebuilt.py / install_llama_prebuilt.py: host + backend detection, sha256 pins (whisper_prebuilt_pins.json) as the trust anchor, staging + install lock + atomic swap, traversal-safe extract, co-located shared libs (RUNPATH=$ORIGIN), an UNSLOTH_WHISPER_PREBUILT_INFO.json marker with idempotent "already matches", and exit codes 0/1/2/3. Not wired into setup yet; the pins ship empty so every asset fails closed until the first fork release is published and its digests are reviewed in. * Studio STT: install prebuilt whisper.cpp during setup and update Add a fail-open whisper.cpp block to setup.sh after the llama.cpp section so `unsloth studio update` (and a fresh install) fetch the prebuilt whisper-server into the managed whisper.cpp dir the sidecar discovers. It skips a user-set WHISPER_SERVER_PATH/UNSLOTH_WHISPER_CPP_PATH, honors UNSLOTH_SKIP_WHISPER_INSTALL, forwards the resolved ROCm gfx, and never aborts setup: a busy install keeps the existing runtime, and an unavailable prebuilt stays quiet (source build is opt-in via UNSLOTH_WHISPER_FORCE_COMPILE) since Transformers STT and browser dictation remain. Register UNSLOTH_WHISPER_PREBUILT_INFO.json as Studio-owned evidence. * Studio STT: harden whisper-server child env + WSL ROCm detection - Sidecar spawns whisper-server with a scrubbed child env that prepends the binary dir (co-located GPU libs) to the loader path, and on WSL2 ROCm loads the system HIP first (HSA_ENABLE_DXG_DETECTION=1) so a bundle's bare-metal HIP does not segfault on /dev/dxg. Secret-bearing vars are dropped from the child. - find_whisper_server_binary now requires an executable, not just a file. - Installer rocm probe passes HSA_ENABLE_DXG_DETECTION and falls back to /opt/rocm/bin/rocminfo so a WSL ROCm host is not misdetected as CPU-only; gfx parsing skips the gfx000 CPU agent and generic ISA lines. - Tests for the child env (secret scrub, lib dir, WSL HIP precedence), the executable check, and the WSL rocm detection. * Studio STT: in-app whisper.cpp prebuilt update stack + ship pins in the wheel Mirror the llama.cpp update stack for the whisper.cpp prebuilt so Studio can detect and install a newer whisper-server release from inside the app: - backend/utils/whisper_cpp_freshness.py: read UNSLOTH_WHISPER_PREBUILT_INFO.json and compare the installed release against the newest unslothai/whisper.cpp release. Whisper tags are v<upstream>-unsloth.<N>, so is_behind compares a (major, minor, patch, serial) key with a strict downgrade guard; 24h cache; fail-open. - backend/utils/whisper_cpp_update.py: run install_whisper_prebuilt.py to fetch and atomically swap the newest bundle, unloading the warm GGUF sidecar first. - backend/routes/whisper.py mounted at /api/whisper (update-status + update). - pyproject: add whisper_prebuilt_pins.json to studio package-data so the installer's trust anchor ships in the wheel (it is a data file, not a .py module, so package discovery alone does not include it; node_prebuilt_pins.json is listed for the same reason). Without this a pip-installed wheel had no pins and the prebuilt install aborted to Transformers STT. Adds test_whisper_cpp_freshness.py (version parser, is_behind matrix + downgrade guard, marker layouts, stale decision, fail-open). * Studio STT: verify whisper prebuilts via the release checksum index, like llama.cpp Re-align the whisper.cpp prebuilt installer to install_llama_prebuilt.py's trust model: instead of a committed whisper_prebuilt_pins.json, verify every download against the release's own whisper-prebuilt-sha256.json checksum index, fetched from the same GitHub release. - parse_release_checksums / fetch_release_checksums / expected_sha256_for replace the pins layer. The index is validated for schema/component and that its release_tag matches the resolved release; an asset absent from it, a release that does not publish it, or a manifest sha256 that disagrees with it all fail closed to a source build. - resolve_release_tag now resolves the newest published release at runtime (or an explicit --published-release-tag), matching llama and the freshness check; removed the pinned-default and the UNSLOTH_WHISPER_ALLOW_UNVERIFIED opt-in. - Delete studio/whisper_prebuilt_pins.json and drop its pyproject package-data entry (nothing to ship now, same as llama which has no committed pins). - Adds test_install_whisper_prebuilt_checksums.py (index parser, fail-closed on uncovered asset, tampered-manifest guard, newest-release resolution). This is a same-origin checksum (integrity, not authenticity), identical to the llama.cpp installer; pair releases with GitHub artifact attestations for provenance. * Resolve whisper prebuilt release via the download host (no GitHub API) Mirror install_llama_prebuilt.py's fast path: resolve the release tag from the releases/latest redirect and fetch the manifest + checksum index from constructed releases/download URLs, so the common install path makes zero api.github.com calls (unauthenticated api.github.com is capped at 60 req/hour per IP; the download host is not). Fall back to the GitHub API only on a 404, malformed asset, or tag mismatch. * Studio STT: coverage-aware whisper prebuilt selection via a shared core whisper's select_artifact returned the first os/arch/backend manifest match and ignored the SM-coverage fields the release manifest already carries, so a Blackwell B200 (sm_100) was served cuda12-legacy (sms 50-61) -- runnable only via forward PTX JIT. install_llama_prebuilt.py on the same host correctly picks cuda13-newer. Extract the coverage-aware selection into a shared, component-agnostic core under studio/backend/utils/prebuilt/ (selection + GPU host-capability detection), lifted from llama's linux_cuda_choice_from_release / _artifact_covers_sms / _sm_range and generalised over a normalised artifact. whisper's HostInfo now records the GPU compute caps + driver CUDA version (honoring CUDA_VISIBLE_DEVICES), and select_artifact routes CUDA/ROCm through the shared selector: every visible SM must be covered, the tightest-covering profile wins (Blackwell-aware runtime-line ordering), ROCm matches the gfx target exactly, and an uncovered GPU falls back to the CPU bundle. CPU/Metal/Vulkan keep first-match. The resolver JSON, exit codes, and "already matches" contract are unchanged. On the B200 the installer now resolves cuda13-newer, matching llama. * Studio STT: gate whisper CUDA selection on the on-disk runtime, like llama The prebuilt CUDA bundles are dynamically linked and intentionally do NOT ship libcudart/libcublas -- they load the same runtime the host already has. So the driver's advertised CUDA version is only an upper bound: a cuda13 bundle still needs cuda13 runtime libraries present on disk. Port llama's on-disk runtime scan (detected_linux_runtime_lines / detected_windows_runtime_lines) into the shared core and intersect it with the driver-compatible lines in select_cuda_attempts. A host with a cuda13 driver but only cuda12 runtime (e.g. torch-cuda12) now correctly gets a cuda12 bundle instead of an unloadable cuda13 one; a host with no CUDA runtime at all falls back to CPU. Fixes a glob bug in the port (any(Path(d).glob(p) for d in dirs) tests generator truthiness, not a match) that made every major report present; add a real filesystem test that exercises the scan. * studio: harden shared prebuilt core to full llama parity Apply the review findings on the shared coverage-aware prebuilt-consumer core so whisper.cpp selection is exactly equivalent to the llama.cpp path. hosts.py: port llama's CUDA_VISIBLE_DEVICES handling. A GPU hidden by an index/UUID selector now reports has_usable_nvidia False instead of staying usable, via supports_explicit_visible_device_matching plus the physical / explicit-match branches, and _select_visible_rows now matches rows the way llama does (index or UUID, gpu- prefix optional) and skips unmatched tokens rather than keeping all rows. Adds the Linux /proc/driver/nvidia/gpus fallback and has_physical_nvidia. Adds parse_macos_version. runtime_libs.py: the Linux on-disk scan now requires the exact libcudart / libcublas SONAME (libcudart.so.13), not a libcudart.so.13* glob, so a bare versioned file without the SONAME symlink no longer counts as loadable. Hardens the ldconfig parse against an empty left-hand side. selection.py: fix the Blackwell/torch reordering so it keys on the covering runtime lines (falls through to the torch preference when the covering lines were filtered out), matching linux_cuda_choice_from_release. Corrects the compatible_runtime_lines_for_driver docstring: the bundles do not ship the CUDA runtime, so the driver version is only an upper bound and the caller must intersect with the on-disk scan. install_whisper_prebuilt.py: enforce a macOS artifact's min_os (new HostInfo.macos_version) so a bundle that cannot load on the host OS version is dropped. Keep resolver stdout to only the JSON line by leaving logs on stderr in --resolve-prebuilt mode, and map an unexpected probe failure to prebuilt_available False instead of a traceback. Tests: new host-probe suite for the visible-device logic, exact-SONAME runtime-scan cases, macOS min_os filtering, resolver stdout-only-JSON, exit-code mapping, and the repo key. * studio: fix whisper prebuilt selection + launch parity gaps from review A parallel review surfaced integration defects where the whisper path could select or launch a bundle that cannot run on a concrete host. Each is fixed to match install_llama_prebuilt.py. macOS min_os: the manifest labels macOS requirements as macos-<version> (e.g. macos-14.0), which the version parser could not read, so the guard was a no-op and a macOS-13 host would install the macos-14 Metal bundle. Strip the platform prefix before parsing. ROCm gfx detection: _detect_rocm_gfx returned the first gfx token and ignored HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES. Since exact ROCm matching treats that token as the active GPU, a mixed APU + dGPU host (gfx1151 + gfx1100) with HIP_VISIBLE_DEVICES=1 installed the wrong archive. Route through a shared pick_rocm_gfx_target (lifted from llama) that parses per-GPU sections and honors the visibility vars (empty / -1 -> no AMD GPU). --rocm-gfx override: recording the arch without setting has_rocm left the host on its CUDA/CPU path so the ROCm bundle was never picked. --rocm-gfx now implies has_rocm and clears NVIDIA state, like llama's _apply_host_overrides. CUDA launch env: a CUDA bundle ships the ggml CUDA backend but not libcudart/libcublas, and the sidecar launch env exposed only the bundle dir, so on a host whose CUDA runtime lives only in the PyTorch wheels the selection would gate cuda usable but the server could not load it. Add the CUDA-from-PyTorch runtime dirs to the child loader path for CUDA bundles (bundle dir still first), mirroring binary_env. Also normalize a manifest artifact's supported_sms defensively (parity with llama's parser) and document that blackwell_min_toolkit_for_caps is retained for the Phase B llama Windows path. Not changed (verified parity, not defects): Linux/Windows min_os is enforced nowhere in llama (macOS only); the resolver is optimistic about the checksum index and the install path verifies. * studio: tighten prebuilt-core code comments * studio: lift shared prebuilt installer core out of the whisper installer * studio: reuse the llama.cpp prebuilt installer machinery for whisper * studio: unify llama and whisper prebuilt installers on a shared descriptor core * studio: consolidate prebuilt installer tests into the shared core suite Grow tests/studio/install/test_prebuilt_core.py from 62 to 164 tests so every component-agnostic behavior runs against both descriptors: the full seven profile CUDA release matrix (multi-GPU, on-disk runtime gating, shuffle stability, missing SM metadata, dotted SM normalization, no-driver fallback policy), the ROCm gfx family matrix, macOS min_os gating and its helper, backend resolution incl. cpu-fallback precedence and Intel-mac auto detect, checksum-index non-object and plain-lookup cases, the tar symlink/hardlink extraction guards moved from the llama suite, and the compute-cap, visible device, runtime-line and Blackwell helper value tables moved verbatim from the llama characterization suites. Delete only tests whose exact behavior the master now asserts for the same component: 40 pure-alias helper cases in test_selection_logic.py (replaced by value-identical master tables plus an alias-identity pin), 6 extraction moves and the master-absorbed zip-symlink case in the llama logic suite, 3 routing twins in test_rocm_support.py already pinned byte-for-byte in test_selection_logic.py, the 2 Blackwell helper tables in the backend resolve suite, 28 whisper logic tests and 10 whisper checksum tests re-asserted by the master whisper parameterization. Wrapper wiring pins, the llama release plan dialect, fingerprints and every llama-only behavior stay untouched. * studio: dedupe sidecar and update helpers into the backend prebuilt package * studio: chain whisper.cpp prebuilt updates onto the llama.cpp update flow * studio: consume paired slim whisper prebuilts via the llama ggml runtime * studio: serve every whisper backend from slim prebuilts * studio: drop the whisper fat per-accelerator selection chain unslothai/whisper.cpp releases are slim-only from v1.9.1-unsloth.2: one ggml-less bundle per os/arch, paired to the llama.cpp prebuilt that provides every ggml backend. Delete the whisper-side fat CUDA/ROCm/metal/vulkan selection glue; keep slim selection + pairing, link_ggml_runtime, and one legacy shape, the published fat CPU bundle of an explicitly pinned pre-slim release. Exit 2 now reads as prebuilt unavailable (whisper never source builds); setup already treats it that way. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Wire libomp runtime DLL alongside ggml in slim whisper installs llama's clang-built windows-arm64 ggml-base.dll imports libomp140.aarch64.dll, shipped in the llama bundle but not a system DLL. Without it next to whisper-server.exe the loader fails with STATUS_DLL_NOT_FOUND before main. MSVC x64 links vcomp140.dll from System32 and Linux ggml uses system libgomp.so.1, so only windows-arm64 was affected. The empty-runtime guard still requires a real ggml library; libomp alone is not a pairing. * studio: drop whisper-side fat-selection support structure Slim whisper bundles are selected per os/arch only; all accelerator capability comes from the installed llama.cpp prebuilt, whose installer already did the coverage-aware selection. Remove the machinery that only existed to pick among fat per-accelerator whisper bundles: - prebuilt_core: delete the generic CUDA/ROCm coverage selection (select_cuda_artifact, select_rocm_artifact, ArtifactView adapters, detected_cuda_runtime_lines, the exact-SONAME linux probe) that no shipped component routes through; llama keeps its own selection chain and whisper shadows select_artifact with the slim-only version. select_artifact is now a plain os/arch/backend first-match. - install_whisper_prebuilt: drop the HostInfo CUDA fields (compute_caps, driver_cuda_version, torch_runtime_line) and the torch runtime probe that populated them; nothing reachable reads them, and the resolver payload sources runtime_line from the artifact. - whisper_cpp_update: delete the standalone start_update job worker; whisper applies only run as the chained phase of the combined llama+whisper update. The status payload keeps its job field (idle). - routes/whisper: drop the progress logger that could never fire. - tests: remove tests of the deleted paths and tests duplicating the descriptor-parameterized core suite or the llama freshness suite. Contracts unchanged: resolver JSON keys, exit codes, marker fields, pairing logs, and the pinned pre-slim fat CPU escape hatch. * Address review feedback on the whisper prebuilt update and install paths - Pin the chained whisper phase to the release the freshness check offered, so the download-host latest pointer cannot reinstall an older build in a loop - Wire the whisper prebuilt install into setup.ps1 (Windows setup previously skipped it entirely) - Treat a non-executable server or missing wired ggml libraries as a broken install instead of reporting already matches - Keep whisper sidecar reloads out of the job-level reload flag and resync chat state after a partial chained update that unloaded llama - Repoint home and profile vars for the whisper-server subprocess at a managed scratch dir and drop credential-store pointers - Clear the prebuilt marker before the opt-in source build overwrite - Write the prebuilt marker with explicit utf-8 encoding * Tighten comments in the whisper prebuilt consumer * Harden the Windows whisper setup phase and the chained update edges - setup.ps1: honor WHISPER_SERVER_PATH / UNSLOTH_WHISPER_CPP_PATH / UNSLOTH_SKIP_WHISPER_INSTALL, run the custom-home ownership guard before the atomic install, and forward the release-tag pin and ROCm hints like setup.sh - sidecar: a cpu-selected install launches whisper-server with --no-gpu (slim wiring links every llama backend, so the flag is what keeps a deliberate CPU choice off the GPU) - chained update: leave whisper unpinned on macOS (the llama phase can walk back there, and a newest-tag pin could be an impossible pairing on every retry) and treat installer exit 2 as kept-existing-runtime instead of failing the combined job - job.to_tag now comes only from the llama phase, so a whisper-only round cannot report a llama update that never ran * Fix slim whisper runtime follow-ups * Address remaining whisper update reviews * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address remaining prebuilt update reviews * Fix remaining chained update reviews * Fix remaining whisper runtime review edges * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> |
||
|
Michael Han
|
6d8c18cd1a |
Replace standalone Studio wording with Unsloth (#7221)
* Replace standalone Studio wording with Unsloth Replace the single word Studio with Unsloth wherever it is used as shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n locales, workflow display names, comments and docstrings. Kept unchanged: the full name Unsloth Studio, third party product names (LM Studio, Visual Studio, Mac Studio), feature names (Recipe Studio, Fine-tuning Studio and its translations), and all identifiers such as env vars, commands, paths and filenames. * Address review feedback on the Studio wording rename Use "an" before Unsloth where the rename left the article as "a". Restore the split brand where Unsloth and Studio render as two halves of the full product name: the onboarding sidebar subtitle and the IPv6 localhost warning. Scope two messages to the full name Unsloth Studio where plain Unsloth was misleading: the AMD README bullet and the CLI studio setup error. |
||
|
Daniel Han
|
2f3eae9846 |
Studio: resolve llama.cpp prebuilts via the release-assets CDN to avoid GitHub API rate limits (#7086)
* Studio: resolve llama.cpp prebuilts via the release-assets CDN to avoid GitHub API rate limits * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: resolve manifest-named prebuilt assets on the download-host fast path Add tag-pinned CDN URLs for any manifest artifact whose hash is keyed under an upstream-tag alias in the checksum asset, so the fast path resolves the same assets the API path does. Cover the resolve body directly (only download_bytes stubbed) and soften the doc's validation-equivalence wording. * Studio: pin llama.cpp fast path to the releases/latest redirect tag Derive the authoritative latest tag from GitHub's /releases/latest redirect target instead of trusting the checksum asset's self-reported release_tag, so the existing release_tag cross-check in parse_approved_release_checksums is a real check again: a stale or mis-tagged checksum asset now falls back to the API. Pin every fast-path URL to that tag. Fall back to the API on a manifest 404 as well, since an in-progress release can publish the checksum asset before the manifest, matching the sha256 404 handling. Document the releases/latest (created_at / make_latest) versus published_at ordering divergence and why it is an accepted, mitigated tradeoff. * Studio: drop the llama.cpp prebuilt-resolution doc Remove studio/docs/llama-cpp-prebuilt-resolution.md and the docstring pointer to it; the resolution rationale (the created_at/make_latest vs published_at ordering nuance) stays inline in _download_host_latest_release_tag. * Studio: tighten llama.cpp download-host fast-path comments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
fb5dc91bb4 |
Studio: remove dead direct_linux_release_plan path (#7030)
parse_direct_linux_release_bundle and direct_linux_release_plan are no longer reached by any live code path. Fork Linux installs resolve through _fork_manifest_release_plans -> _linux_published_attempts, and the upstream (ggml-org) path uses direct_upstream_release_plan. The dead parser also called _resolve_linux_bundle_profile, which no longer exists, so its CUDA branch would raise NameError if ever executed. Drop both functions and the obsolete TestDirectLinuxNvidiaCpuGate; its live equivalent TestLinuxPublishedAttemptsNvidiaCpuGate already covers the NVIDIA no-silent-CPU behaviour. |
||
|
oobabooga
|
fcb1152c76 |
Studio: source CPU llama.cpp prebuilts from unslothai/llama.cpp (#6311)
* Studio: source CPU llama.cpp prebuilts from the unslothai fork * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: reject unknown Linux CPU arches and keep ROCm-tooling hosts off the CPU prebuilt * Studio: extend the resolve-prebuilt ROCm-tooling guard to Windows * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: let ROCm-SDK-only CPU hosts take the fork CPU prebuilt * Studio: accept windows-arm64 prebuilt kind and refresh stale fork-routing comments * Studio: correct stale fork-routing comments and --resolve-prebuilt help * Refresh stale ggml-org routing comments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Daniel Han
|
b72a8c4263 |
studio: explicit Cloudflare tunnel notice and public-exposure warning at startup (#6515)
* studio: announce Cloudflare tunnel state and warn about public exposure on startup The startup banner only printed a line when a tunnel URL was up, so a plain `unsloth studio -H 0.0.0.0` launch silently created a public trycloudflare.com URL with no indication that Studio had become reachable from the internet. The only hint at the tunnel was the CLI help, shown when an invalid command was typed. Make the banner always state the tunnel state for wildcard binds: - ON: the public URL plus a warning that anyone with it can reach Studio from outside the network, and that --no-cloudflare keeps it local-only. - FAILED: requested but did not start (local network only). - OFF: --no-cloudflare was passed (local network only). Secure mode keeps its existing wording (the authenticated tunnel is intended and --no-cloudflare is not valid there). Clarify the --cloudflare help text in both the argparse and typer definitions. Default behavior is unchanged. Also surface the state on the `unsloth studio run` banner, which runs the server with silent=True and prints its own banner: it now calls _print_cloudflare_line too, so the ON/OFF/FAILED notice and public-exposure warning are no longer skipped on that path (previously it only echoed the URL when a tunnel was up). For the OFF and FAILED notices, do not claim "local network only" when the reachability probe just confirmed the raw port is reachable from the public internet: --no-cloudflare and a failed tunnel disable only the Cloudflare link, not the wildcard bind, so the message is reworded to flag the public raw port. * Fix/adjust Cloudflare banner warnings for PR #6515 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix/adjust Cloudflare banner comments for PR #6515 * Fix/adjust IPv6 Cloudflare tunnel gate for PR #6515 * Fix/adjust Cloudflare review comments for PR #6515 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix silent run Cloudflare notice * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: wasimysaid <wasimysdev@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
378e33c8a5 |
Studio macOS: faster startup, MLX self-heal, drop obsolete prebuilt pins (#6494)
* Studio: defer llama.cpp update probes and self-heal MLX on macOS Two macOS startup problems shared one root area in the FastAPI lifespan: - The llama.cpp capability + freshness probes ran inline before the server yielded, so a cold/slow/flaky network on the GitHub freshness check blocked 'Application startup complete' (~34s on CI, longer in the field). Move both probes to a daemon thread; app.state stays None until ready (status routes already re-probe at request time). Opt out with UNSLOTH_DISABLE_UPDATE_CHECK=1. - Train and Export were greyed out because mlx/mlx-lm/mlx-vlm arrive only transitively and a resolver backtrack silently drops them, so CHAT_ONLY stayed true. Add utils/mlx_repair.py: when Apple Silicon is detected without MLX, reinstall mlx/mlx-lm/mlx-vlm by name on a daemon thread and re-run hardware detection (opt out UNSLOTH_DISABLE_MLX_AUTOREPAIR=1). Surface a chat_only_reason in /api/health plus a sidebar tooltip so a greyed Train/Export explains itself instead of failing silently. * Studio: guard model defaults against a None model name load_model_defaults(None) called model_name.lower() with no guard, raising 'Error loading model defaults for None' before any model is selected. Return an empty dict for a falsy/non-str name. * Studio: drop obsolete upstream macOS + Windows Blackwell prebuilt pins Both pins worked around gaps in ggml-org upstream prebuilts, but Studio now routes every GPU host and all of macOS to the unslothai/llama.cpp fork (published_repo_for_host), which ships the needed bundles, so both pins are dead code on the default install path: - macOS b9415: macOS always routes to the fork (its own macOS bundles), and host_supports_macos_minos() is the backstop. The pin only fired under an explicit --published-repo ggml-org override. - Windows Blackwell b9360: Windows-NVIDIA routes to the fork, whose windows-x64-cuda13 bundle covers Blackwell (manifest max_sm 120, toolkit 13.3), so the pin's self-disable check makes it dormant on every default install; it could only activate under the same upstream override on a 13.0-13.2 driver. Remove the pin constants, functions, and call sites. Keep the Blackwell capability detection (_drop_blackwell_incapable_windows_cuda, _host_is_blackwell, _windows_cuda_attempt_covers_blackwell) that still drops a non-sm_120 cuda-12.4 build on a Blackwell host. After this, an explicit --published-repo ggml-org override on a Blackwell 13.0-13.2 host loses its GPU fallback and lands on CPU; the default fork path is unaffected. Update the install selection-logic and macOS-compat unit tests for the new no-pin behavior. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: walk back deeper on the macOS upstream prebuilt path After removing the b9415 macOS pin, the explicit --published-repo ggml-org upstream path still used the default 2-release fallback, so a pre-macOS-26 host behind a run of macOS-26-only builds would exhaust two too-new plans (minos is only checked post-download) and drop to a source build before reaching a loadable older release. Walk back as deep as the fork macOS path (DEFAULT_MAX_MACOS_RELEASE_FALLBACKS), turning the removed static pin into dynamic discovery. Addresses review feedback on the macOS upstream fallback. * Studio: pin transformers during MLX self-heal so it cannot break Studio mlx-lm/mlx-vlm declare transformers>=5, but the single-env install pins transformers==4.57.6. The self-heal used --upgrade with no constraint, so it could upgrade transformers in the live venv and break the rest of Studio just to make import mlx.core pass. Pin transformers to the installed version via a constraint file: the resolver either finds an mlx build compatible with it or fails (we stay chat-only), never upgrading transformers underneath Studio. Addresses review feedback on the MLX repair install. * Studio: harden MLX self-heal against an unsupported mlx-vlm Pinning transformers alone made uv backtrack mlx-vlm to 0.3.9 (below unsloth-zoo's mlx-vlm>=0.4.4), which imports but breaks VLM Train/Export -- so the self-heal could clear chat-only onto a broken stack. Mirror the main installer: set UV_OVERRIDE=overrides-darwin-arm64.txt so a current mlx-vlm coexists with the transformers pin, require the same minimum versions unsloth-zoo declares, and gate/validate on a full mlx_stack_available() check (not a bare import) so an old or partial stack stays chat-only. Addresses PR review. * Studio: filter Blackwell-incapable CUDA in resolve_upstream_asset_choice resolve_upstream_asset_choice returned the first windows-cuda choice unfiltered, so a Blackwell host could be handed an sm_120-incapable cuda-12.4 build while the sibling planners drop it. Apply _drop_blackwell_incapable_windows_cuda here too and fall through to the CPU bundle on a Blackwell host with no capable GPU asset. Addresses PR review. * Studio: re-poll health so MLX self-heal reaches an open UI The sidebar cached the initial /api/health, so a successful background MLX self-heal (chat_only flips false) did not re-enable Train/Export until a manual reload. While chat-only for the recoverable mlx_unavailable reason, re-poll /api/health and stop once Train/Export become available. Addresses PR review. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: make the disabled Train/Export tooltip reachable The greyed Train/Export items pass a tooltip explaining why (e.g. MLX missing), but a disabled <button> fires no pointer events and SidebarMenuButton only showed tooltips while collapsed, so the explanation never appeared. Wrap a disabled button in a focusable span and show its tooltip while expanded too; enabled items keep the collapsed-only behavior. Addresses PR review. * Studio: gate Train/Export on the full MLX stack, not bare mlx.core detect_hardware enabled MLX training whenever `import mlx.core` worked, but the MLX self-heal (utils/mlx_repair) treats a stack without mlx-lm/mlx-vlm at the versions unsloth-zoo requires as inadequate. That asymmetry let the UI enable Train/Export on exactly the partial/backtracked stack the self-heal is trying to repair (greyed-in-but-broken VLM export). Gate on the same mlx_stack_available() criterion so a partial stack stays chat-only (reason mlx_unavailable) and the background repair restores it. Addresses PR review. * Fix MLX repair and health auth for PR #6494 * Fix macOS upstream prebuilt fallback for PR #6494 * Fix MLX stack validation for PR #6494 * Fix MLX self-heal validation for PR #6494 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Review fixes: isolate hardware-state test, robust transformers pin - test_chat_only_reason.py: detect_hardware() assigns module globals directly, which monkeypatch does not revert; the autouse fixture now saves and restores DEVICE/CHAT_ONLY/CHAT_ONLY_REASON/IS_ROCM so a chat-only verdict here cannot leak into other backend tests (e.g. test_utils.py) on a GPU host. - mlx_repair.py: read the transformers version from importlib.metadata instead of importing transformers, so the install pin is not silently dropped when transformers has valid metadata but fails to import. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix CI: model full MLX stack in dispatch tests, keep selection test offline dispatch (macOS) job: - detect_hardware now gates MLX on the full stack (mlx_stack_available imports mlx_lm/mlx_vlm and checks dist versions), so faking only mlx.core makes the apple_silicon_mlx profile resolve to CPU. The dispatch tests assert the routing decision when the stack IS usable, so model a complete stack: test_hardware_dispatch_matrix patches utils.mlx_repair.mlx_stack_available and test_is_mlx_dispatch_gate patches hardware._has_usable_mlx_stack. The stack predicate's own internals stay covered by test_mlx_repair.py. Repo tests (CPU) job: - test_no_cuda_attempt_on_published_path_for_13_1 fell through to a live github_release_assets() upstream fetch after the Blackwell filter dropped every published attempt, which the offline security scanner blocks. Stub that fetch so the walk-back deterministically finds no usable CUDA build and raises PrebuiltFallback without network. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden MLX self-heal: prepare transformers constraint inside the try attempt_mlx_repair runs on a daemon thread, but _transformers_constraint_args was called before the try. A failure there (e.g. tempfile.mkstemp on a full disk or a bad TMPDIR) would propagate unhandled and silently kill the self-heal thread. Move the call inside the try and initialize constraint_path so any such failure is caught and leaves Studio chat-only instead of crashing the thread. --------- Co-authored-by: Daniel Han <michaelhan2050@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: wasimysaid <wasimysdev@gmail.com> |
||
|
Daniel Han
|
23a3d972bf |
Studio: reach the published source asset when a mix build's commit 404s (#6314)
* Studio: reach the published source asset when a mix build's commit 404s A llama.cpp "mix" prebuilt records a merge commit that is never pushed to the fork, so the codeload/archive URLs for that commit 404. The merged source tree is instead published as a release asset alongside the prebuilt (llama.cpp-source-commit-<sha>.tar.gz). The installer resolves that asset URL from the approved-checksums manifest, but when the manifest omits the top-level repo/release_tag the URL resolves empty, hydration falls through to the 404-ing commit archive, and the prebuilt install drops to a slow source build (or fails outright). Extract exact_source_asset_url() and resolve the asset's host and tag defensively: the artifact's own repo, then the manifest repo, then the source repo; and the manifest release tag, then the tag we actually installed the prebuilt from (the source asset is its sibling on the same release). Normal installs build the identical URL as before, so this only adds a working fallback for the degenerate manifest. Add unit coverage for the resolver, including the empty repo/release_tag regressions. * Studio: cover exact_source_asset_url through the real parser chain Add TestExactSourceAssetUrl.test_resolves_through_real_parser_chain, which runs parse_approved_release_checksums -> preferred_source_archive -> exact_source_asset_url so a regression in the parser or source-selection wiring cannot pass while only the hand-built helper unit tests stay green. |
||
|
Daniel Han
|
ce193c243d |
Keep server-side tools enabled under --secure (#6403)
* Keep server-side tools enabled under --secure and on every bind --secure binds loopback and exposes Studio only through an authenticated Cloudflare HTTPS tunnel, but it was grouped with a raw 0.0.0.0 bind and force-disabled all server-side tools (web search, Python, terminal). The process tool policy overrode the client's enable_tools request, so the model was never told the tools existed and answered in plain text. The plain 'unsloth studio' command had no way to re-enable and printed nothing. Tools now default on for every bind. The bind host and --secure no longer change the tool policy; only an explicit --enable-tools/--disable-tools forces it on or off. Both 'unsloth studio' and 'unsloth studio run' accept the flags and the startup banner states the resolved policy. - run.py: replace _apply_default_tool_policy(host, secure) with _apply_cli_tool_policy(enable_tools); add an enable_tools kwarg to run_server and --enable-tools/--disable-tools to the argparse. - _tool_policy.py: resolve_tool_policy defaults to on for every host and no longer prompts on a network bind. - studio.py: drop the secure-as-public tool gating, add the flags to the plain command, and reword the startup banner. - Update and extend the secure-flag and tool-policy tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add tool-policy notice to plain server banner and refresh run --help Follow-up to PR review: - run.py: the plain 'unsloth studio' / --secure / direct run.py path went through _emit_startup_output without any tool-policy line, so a network-reachable launch was silent about code execution now that tools default on. Thread enable_tools through _emit_startup_output / _emit_secure_startup_output and print a one-line policy notice, followed by a single stop hint. - studio.py: the 'unsloth studio run' --enable-tools/--disable-tools and --yes help still described the removed loopback-on/network-off default and the confirmation prompt; reword to match the new policy. - Add tests for the banner notice and the refreshed help text. * Update CI tool-policy resolver tests for default-on behavior tests/python/test_unsloth_run_tool_policy_resolver.py still asserted the removed network-bind policy (0.0.0.0 and LAN IP default off, explicit enable prompts and aborts on a declined prompt), so it failed the Python CI jobs. Rewrite the truth table: every bind defaults on, explicit on/off always wins, and the resolver never prompts (yes/silent/prompt kept for compatibility). * Trim comments for the tool-policy change Shorten the verbose docstrings and block comments added for --secure tool handling; keep the security-relevant intent. Verified comment-only via an AST diff (code unchanged). * Add deterministic test that server-side tools execute under --secure Drive the GGUF agentic tool loop with a fake llama-server stream and let the real execute_tool run: python counts 1..100, terminal returns a UTC datetime, and web_search runs through real _web_search with only the ddgs network boundary mocked. A policy assertion pins that the post-fix --secure path (policy None + per-request enable_tools) is what keeps these executions reachable. No model, GPU, or live network; runs in the existing backend CI. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Align _emit_startup_output banner test with the moved stop hint The tool-policy notice now prints between the access banner and the stop hint, so the stop hint is emitted once at the end instead of inline in the banner (include_stop_hint is False and print_studio_stop_hint runs once). Update the plain-localhost case to match; the mismatch and wildcard cases already asserted this wiring. --------- Co-authored-by: Michael Han <michaelhan2050@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
a6dc10dad2 |
Reduce and tighten comments and docstrings across the test suite (#6429)
* Reduce and tighten comments and docstrings in tests Shorten verbose comments and docstrings across the test suite without changing any test logic. Remove narration that restates the next line, collapse long module and test docstrings to a single line, and drop banner separators. Keep regression context (issue and PR references, run ids), skip reasons, mocking and timing rationale, license headers, lint and type directives, and commented-out code. Comments and docstrings only: an AST signature check confirms no code, assertions, or string literals changed, and the suite byte-compiles cleanly. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
77acea751c |
Studio: prefer native cuda13 over torch's cuda12 line on Blackwell Linux hosts (#6379)
The Linux installer ordered its CUDA runtime-line attempts purely by torch's reported CUDA major (preferred_runtime_line), so a Blackwell host running a cu12x torch build hoisted cuda12 ahead of an available native cuda13 bundle. This brings the Linux selector to parity with the existing Windows Blackwell preference: on an sm_120 host, prefer the highest CUDA-major line that ships a bundle covering every visible host SM, then fall back to the torch line. Selection-time only. No external pin and no source build: in-release cuda13 bundles already cover sm_120, and the per-artifact SM filter still drops any incapable bundle (cuda12-older / cuda13-older) and prevents fall-through to a non-Blackwell build. The override is gated on _host_is_blackwell and only reorders lines that are already detected and driver-compatible, so it never forces cuda13 when its runtime libraries are absent or the driver is pre-13, and non-Blackwell hosts keep the exact torch-preference behavior. The runtime-line ranking only considers well-formed "cuda<major>" lines and skips any malformed or future-format value (e.g. "cuda13.1") instead of crashing the major sort, matching how the surrounding selector already tolerates unknown lines. Adds focused selection tests covering the override, the incapable-cuda13 skip, the cuda13-unavailable fallback, the non-Blackwell no-op, the malformed-runtime_line skip, and cuda14 forward-compat. |
||
|
oobabooga
|
a3ca5d2a5a |
Studio: don't silently fall back to a CPU prebuilt on NVIDIA Linux GPU hosts (#6310) | ||
|
oobabooga
|
5300c047b6 |
Installer: drop the lemonade ROCm fallback now the fork ships identical per-gfx prebuilts (#6225)
--------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
oobabooga
|
cc1a724efc |
Source llama.cpp prebuilts from unslothai/llama.cpp (CUDA, ROCm, macOS) (#5963)
* Studio: route arm64 Linux CUDA hosts to linux-arm64-cuda prebuilts
* Studio: SM-aware selection for windows-cuda app bundles
* Studio: select published ROCm bundles by gfx target (linux + windows)
* Studio: route macOS installs to the fork's prebuilt bundles
* Studio: fix windows cuda13 driver-13.0 gate and ROCm gfx prefix overreach
* Fix Blackwell Windows pin shadowing native app-bundle (b9360 over b9457)
* Match Windows cuda12 driver floor to Linux (12.x minor-version compat)
* Fix Windows app-bundle dropped when runtime DLLs come from torch/lib
* Fold the manifest resolver into the simple-path resolver (one entry, no dormant full path)
* Remove unused UNSLOTH_LLAMA_PUBLISHED_REPO override
* Route Windows GPU hosts to the fork prebuilts in setup.ps1
* Document sm_103 path divergence and mark --simple-policy as a no-op
* Note sm_103 coverage now comes from the producer manifest
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove the now-vestigial --simple-policy flag (one resolver handles all hosts)
* Unify the fork onto the manifest path; drop the linux-x64 filename path and hardcoded coverage tables
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Strip whitespace from manifest gfx_target/mapped_targets when parsing
* Windows CUDA: sort coverage-unknown bundles last so they can't outrank targeted ones
* Share the SM-coverage sort key between the linux and windows selectors via _sm_range
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reject approved releases with an exact source archive but no source repo to clone from
* Studio: accept the fork's windows-rocm kind in the Windows reinstall check
* Studio: accept a manifest-bundle source repo in the exact-source release check
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: route Linux hosts to the fork only when a usable GPU is present
* Fix Windows AMD lemonade tag resolution for PR #5963
The fork release scan passes each scanned release's upstream tag
(b9518, ...) to the lemonade lookup, but lemonade publishes its own tag
series (b1292, ...) that never contains upstream tag numbers. On a
Windows AMD host every scanned release therefore 404s the lemonade
fetch twice, the upstream HIP zip is dropped by the approved-hash gate,
and the scan walks the whole release history until it dies on the
unauthenticated GitHub rate limit or falls to a HIP source build. The
Linux path already passes the requested tag ("latest") and works.
Thread the requested tag through resolve_release_asset_choice ->
resolve_asset_choice -> resolve_upstream_asset_choice as lemonade_tag,
used only by the lemonade lookups. Upstream asset names keep the
concrete per-release tag and all new parameters default to the old
behavior.
Verified on a gfx1151 box: before, the native Windows install scanned
b9518..b8811 and aborted on rate limit; after, it selects
llama-b1292-windows-rocm-gfx1151-x64.zip (lemonade) from fork release
b9518, passes staged validation, and the installed llama-server
enumerates ROCm0. WSL keeps selecting the matching ubuntu bundle.
Adds a regression test pinning that the Windows fork path resolves
lemonade via /releases/latest, never /releases/tags/<fork-tag>.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Plan lemonade for Linux ROCm hosts on the ggml-org direct path for PR #5963
Audit follow-up to
|
||
|
pre-commit-ci[bot]
|
1d9090ff6b |
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci |
||
|
danielhanchen
|
1d7f9b56f4 |
Installer: never plan a non-sm_120 CUDA build on Blackwell, never plan CPU on an NVIDIA host
Two hardening fixes from the fleet-validation audit. Blackwell Windows hosts drop windows-cuda attempts that cannot offload sm_120 instead of leaving them ranked behind the b9360 pin. A cuda-12.4 upstream build loads and passes the functional validator but runs the model on a slow non-native path (an RTX 5090 measured 7.1 tok/s vs 551.2 on cuda-13.3), so one failed pin download away from that is too close. The coverage check now also reads manifest SM metadata first, so published cuda12 app bundles (toolkit 12.8, sm_120 included) stay selectable and make the pin go dormant correctly. The fork-release Linux planner no longer appends the linux-cpu bundle for NVIDIA hosts whose CUDA selection produced nothing; it raises so the caller walks back to an older release with a usable CUDA line, mirroring the deliberate ROCm policy. Today's walk-back only works because partial releases ship no CPU bundle; this keeps it working if a future partial release does. |
||
|
Matt Van Horn
|
5f622f6c2f |
fix: clearer Studio setup error when GPU driver is too old for the installed CUDA toolkit (#5993)
* fix: clearer Studio setup error when GPU driver is too old for the installed CUDA toolkit * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Honor optional color arg in setup.sh substep so driver/toolkit warnings render in C_WARN * Add regression test that setup.sh _cuda_version_gt compares numerically for PR #5993 * Update Resolve-CudaToolkit test for the new driver-too-old messaging setup.ps1 now routes the too-new-toolkit case through Write-CudaDriverToolkitMismatch instead of the old 'is installed but INCOMPATIBLE' banner. Extract that helper alongside Resolve-CudaToolkit so the child pwsh can run it, and assert the new driver-too-old guidance (and the one-line source-build error) instead of the removed INCOMPATIBLE text. * Address review nits: document Windows hard-exit asymmetry and add toolkit/driver edge tests setup.ps1: note that only a forced source build reaches the hard-exit branch (the prebuilt path returned above), unlike setup.sh which degrades to CPU. test_selection_logic.py: cover the CUDA UMD Version variant, the empty nvcc version guard, and the too_old (< 12.4) short-circuit. * fix(studio): allow CUDA minor-version compat and try installed toolkits before CPU fallback The driver check now compares CUDA major versions only, per NVIDIA minor-version compatibility, and when the selected nvcc is still too new the setup iterates other installed toolkits and uses the newest driver-compatible one before falling back to a CPU llama.cpp build. Same rule mirrored in setup.ps1. * style: apply ruff kwarg-spacing format after rebase * Accept a same-major CUDA toolkit found only on PATH in the Windows fallback The major-only compatibility fix updated the side-by-side scan (Find-Nvcc -MaxVersion) and the CUDA_PATH check, but the fallback that runs when Find-Nvcc -MaxVersion returns null still recorded any plain Find-Nvcc result as an incompatible toolkit without re-checking the major. A same-major toolkit discoverable only via PATH, process CUDA_PATH, or a custom location (e.g. toolkit 13.3 with a driver supporting CUDA 13.2) was therefore rejected even though it is compatible. Re-apply the same major-only rule in the fallback: use the toolkit when its major is within the driver's, otherwise record it as too-new. Adds a regression test covering the PATH-only same-major case. * Add CUDA driver/toolkit selection edge-case tests for Studio setup * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten comments in Studio CUDA driver/toolkit setup Collapse multi-line comments, drop obvious ones, keep the load-bearing intent (the major-compat invariant, the Windows hard-exit vs setup.sh-CPU asymmetry, the PATH-only fallback rationale). Comment-only; no code change. * Clarify the Windows source-build hard-exit comment The path is reached by any committed source build (forced, or after a prebuilt-install failure), not only a forced one. Comment-only. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Drop two obvious comments in setup.ps1 CUDA detection --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Daniel Han
|
187144d4e7 |
Reduce and tighten code comments and docstrings repo-wide (#6095)
Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison. |
||
|
Daniel Han
|
3ce187da02 |
Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079)
Raise ruff line-length to 100 and extend the local pre-commit format pipeline (def-signature magic-comma normalization, short multi-line assert collapse, kwarg '=' spacing, blank-line-after-short-import removal, adjacent string-literal / f-string+plain merge, redundant-pass pruning). Every transform re-checks the file AST and is dropped if it would differ; the whole-repo reformat is verified AST-identical per file and idempotent. |
||
|
Matt Van Horn
|
5cdbfef390 |
fix: warn when localhost resolves to ::1 but Studio is bound only to 127.0.0.1 (#5994)
* fix: warn when localhost resolves to ::1 but Studio is bound only to 127.0.0.1 * studio: fix localhost/::1 warning suppression and cover _run wiring Addresses the Codex review on #5994 plus review-team findings: - Remove the `_local_port_open("::1", port)` early-return. Studio binds 127.0.0.1 only, so a successful connect to ::1:<port> means a *different* process is there -- exactly when http://localhost opens the wrong service and the user most needs the warning. Dropping the probe also removes the ~0.25s startup latency and the probe/warn race. - Extract the banner/warning block from `_run` into `_emit_startup_output` so the wiring is unit-testable, and make the mismatch vs wildcard paths an explicit if/elif (they are mutually exclusive by construction). - Hoist the `_working_local_url` confirmation out of the try block and reorder `_stdout_color_ok` before its only caller. - Tests: add `_emit_startup_output` integration coverage (banner include_stop_hint, warning emission, single stop hint), a regression test that ::1 being occupied does NOT suppress the warning, dual-stack and non-positive-port cases; drop the unreachable `None` getaddrinfo arm. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Etherll <mrmrmidessam@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
ae5d60728b |
Studio: cover B300 (sm_103) with the Linux prebuilt bundles (#5930)
* Studio: cover B300 (sm_103) with the Linux prebuilt bundles sm_103 (B300 / GB300 Blackwell Ultra) was in no bundle's supported_sms, so those hosts fell through to a slow source compile. The newer and portable bundles already ship base compute_100 PTX, which the driver JIT-compiles forward to sm_103, so list sm_103 alongside sm_100 in those bundles and let B300 install the prebuilt. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
172d9d1c8e |
Studio: source-build arm64 Linux GPU hosts, with a CPU prebuilt fallback (#5924)
* Studio: fall back to source build for arm64 Linux GPU hosts setup.sh routes a Linux host with any GPU tool to the unslothai/llama.cpp fork, which publishes only linux-x64 bundles. On an arm64 host with a GPU (GH200, GB200, DGX Spark) the resolver then selected an x64 CUDA bundle, which cannot run on aarch64. Routing those hosts to ggml-org instead would install a CPU-only arm64 build, silently losing the GPU. Guard resolve_simple_install_release_plans so an arm64 Linux host on the fork raises PrebuiltFallback before any release is fetched, letting setup.sh do a source build that actually targets the GPU. x86_64 hosts and arm64 CPU hosts (which route to ggml-org) are unaffected. Add tests covering the arm64 fork raise, the x86_64 pass-through, and the arm64 CPU ggml-org path. * Studio: install ggml-org arm64 CPU prebuilt when the arm64 GPU source build fails Per review of #5924: arm64 Linux GPU hosts have no CUDA prebuilt anywhere (the unslothai fork is x64 only, ggml-org ships no Linux CUDA build), so they source build for the GPU. If that build produces no binary, the host was left without llama.cpp. Add a --cpu-fallback flag to install_llama_prebuilt.py that drops the host GPU attributes so the CPU prebuilt for the host arch is selected (a GPU host cannot otherwise pick the CPU bundle). setup.sh calls it against ggml-org as a last resort for arm64 Linux when the source build degraded, installing the ubuntu-arm64 CPU build instead of leaving the host with no llama.cpp. Add tests: force_cpu drops GPU attrs before planning, a CPU-forced arm64 host selects the ggml-org ubuntu-arm64 bundle, and setup.sh wires the fallback. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
f695fbd0fe |
Studio: extend the pinned Blackwell GPU fallback to CUDA 13.0 drivers (#5920)
The pinned b9360 cuda-13.1 Windows fallback for Blackwell (sm_120) only fired when the driver advertised CUDA >= 13.1. The mainstream Blackwell branch ships the r580 driver that reports CUDA 13.0, so those hosts missed the pin, were gated off the in-release 13.3 build, and dropped to the CPU-only cuda-12.4 build. b9360's binary is native sm_120a SASS (no PTX, so no JIT) and its bundled runtime is cuda-13.1 cudart; both run on a 13.0 r580+ driver under CUDA minor-version compatibility. Lower _PINNED_BLACKWELL_DRIVER_FLOOR to (13, 0) so the pin covers the 13.0 branch too. The generic published-runtime gate stays conservative (an unverified in-release 13.1 build is still gated off 13.0); only the specific, hash-pinned, SASS-verified b9360 build is extended. Refs #5887. |
||
|
Daniel Han
|
dad2695fde |
Studio: pin the last pre-macOS-26 llama.cpp prebuilt instead of walking back (#5896)
* Studio: pin the last pre-macOS-26 llama.cpp prebuilt instead of walking back ggml-org moved their macOS build runner to macOS 26 (Tahoe) at b9428, so b9428 and every newer upstream prebuilt is stamped minos 26 and fails to dyld-load on macOS 14 / 15. #5883 handled this by walking back release by release at install time. Replace that with a deterministic pin: a host below macOS 26 selects b9415 directly (the last upstream build stamped below 26: arm64 minos 14, x64 minos 13.3), so it loads on macOS 13.3 / 14 / 15 / 26. Hosts on macOS 26+ and unknown-version hosts keep latest selection unchanged. Only the ggml-org upstream path is pinned; the unslothai/llama.cpp fork ships its own minos-13.3 prebuilts (#5893), so the pin is a no-op there and goes dormant once macOS routes to the fork. The Mach-O minos preflight from #5883 stays as a post-download backstop. Refs #5883, #5893. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * docs: fork ships arm64 minos 14 / x64 minos 13.3, not uniform 13.3 The per-slice fork producer pins arm64 to 14.0 and x64 to 13.3; update the pinned_macos_release_tag docstring to match. No behavior change. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <michaelhan2050@gmail.com> |
||
|
Daniel Han
|
5076837cf0 |
studio: make NVIDIA prebuilt selection track CUDA version bumps (Windows + Linux) (#5879)
Make the NVIDIA prebuilt selection version-dynamic and driver-gated on both the Windows (ggml-org) and Linux (unslothai/llama.cpp) paths, so CUDA version bumps are handled with no code change while staying safe across driver versions. - Derive candidate CUDA runtime lines from the driver major; pick the highest upstream-published minor the driver can actually run, so a sub-13.3 driver is never handed an unguaranteed 13.3 build. - Pin b9360's cuda-13.1 build (immutable, hash-verified) as a GPU fallback for a Blackwell host on a 13.1/13.2 driver that the in-release 13.3 build gates off, on both the simple and published install paths. Dormant for every other host and self-disabling once upstream ships a driver-runnable build again. - Seed the published-path ordering from the release's real published minors so a future CUDA major is selectable with no code change. Refs #5861, #5817, #5807, #5858, #5854, #5826, #5887. |
||
|
Daniel Han
|
e346193ae8 |
Studio: download paired cudart bundle on Windows CUDA installs (#5322)
* Studio: download paired cudart bundle on Windows CUDA installs
Upstream ggml-org/llama.cpp publishes Windows CUDA in two archives
that the release notes explicitly say are both required:
llama-<tag>-bin-win-cuda-X.Y-x64.zip (binaries + ggml DLLs)
cudart-llama-bin-win-cuda-X.Y-x64.zip (cudart64, cublas64, cublasLt64)
Studio's installer was downloading only the first one. The
``runtime_name`` / ``runtime_url`` fields on AssetChoice existed but
were never populated, and ``install_from_archives`` only handled
``choice.url``. With the cudart DLLs missing from
``install_dir/build/bin/Release``, the prebuilt binary's LoadLibrary
calls only resolved at runtime when the user happened to have a
version-matched system CUDA toolkit on PATH. That is the underlying
cause for the Windows reports in #5106 ("GPU detected but model
loaded entirely on RAM"): the prebuilt's CUDA backend silently fails
to load and llama-server falls back to CPU regardless of ``-ngl`` or
``--fit on``.
Wires the pairing through end to end:
* ``windows_cuda_attempts`` and ``published_windows_cuda_attempts``
look up the matching ``cudart-llama-bin-win-cuda-X.Y-x64.zip``
asset URL alongside the main archive and store it as
``runtime_url`` / ``runtime_name`` on the AssetChoice. We only
pair when the selected main archive is the binary archive
(``llama-...zip``) so the legacy cudart-only naming path is
unaffected.
* ``apply_approved_hashes`` resolves the runtime archive's hash from
the approved manifest. If the manifest does not list the runtime
archive, the pairing is dropped rather than installing without
checksum coverage. Preserves the supply-chain guarantee for
published bundles; upstream installs with no manifest are
unaffected (same risk surface as the existing main-archive
download).
* ``install_from_archives`` now downloads the runtime archive into a
separate temp dir and runs ``copy_globs`` against both source dirs.
Separate dirs avoid the "ambiguous archive layout" guard tripping
on shared filenames like LICENSE.txt, while the second
``copy_globs`` overlay drops the cudart DLLs into the same
``install_dir/build/bin/Release`` directory as the main binary.
Adds a ``runtime_sha256`` field on AssetChoice to carry the
verified hash through to the download step, alongside the existing
``runtime_name`` / ``runtime_url`` slots.
Tests: 5 new cases in tests/studio/install/test_selection_logic.py:
* upstream pairing populates runtime_url / runtime_name
* graceful degrade when cudart asset is absent in the release
* legacy cudart-only naming path does not self-pair
* apply_approved_hashes threads runtime_sha256 when the manifest
lists it
* apply_approved_hashes drops the pair when the runtime hash is
missing rather than installing without verification
130 install tests pass (125 baseline + 5 new). No regressions.
Refs #5106
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments to be more succinct
* Studio: refresh installs that pre-date the paired cudart bundle
expected_install_fingerprint did not hash the new runtime_name /
runtime_sha256 fields, and runtime_payload_health_groups for windows-
cuda only checked llama.dll / ggml-cuda.dll. The combination meant that
an install made before this PR -- the exact installs reporting #5106 --
would still match the post-PR choice: same main asset name + sha, same
llama.dll, same ggml-cuda.dll, missing cudart64_*.dll, but
existing_install_matches_choice returned True and the cudart download
path in install_from_archives never ran. Fresh installs got the fix;
existing affected installs did not.
This commit:
* Adds runtime_asset and runtime_sha256 to the fingerprint payload so
any change to (or first introduction of) the cudart pair invalidates
pre-existing installs.
* Refactors write_prebuilt_metadata to call expected_install_fingerprint
so the recorded fingerprint cannot drift from the expected one when
new keys are added.
* Extends runtime_payload_health_groups for windows-cuda to require
cudart64_*.dll and cublas64_*.dll *only when the choice carries a
paired runtime archive*. Gating on choice.runtime_name keeps the
no-pair fallback path (manifest missing cudart hash, upstream
without paired bundle) from looping on reinstall.
New tests:
* test_existing_install_matches_plan_windows_cuda_paired_requires_cudart
-- paired choice rejects installs missing cudart / cublas.
* test_existing_install_matches_plan_windows_cuda_unpaired_skips_cudart_check
-- unpaired choice still accepts legacy cudart-less installs.
* test_existing_install_fingerprint_changes_when_cudart_pair_added
-- direct fingerprint mismatch between the legacy and paired choice.
Refs #5106
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten paired Windows CUDA install gates
Three follow-ups from a 12-reviewer batch over
|
||
|
DoubleMathew
|
7d227ed708 |
Fix/windowsprebuilt (#5241)
* update prebuilt logic * Add test case --------- Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com> |
||
|
DoubleMathew
|
7ae9b7f45f |
fix windows llama.cpp compile from source issue (#4793)
* fix windows llama.cpp compile from source issue * undo local repo usage * fix llama.cpp install * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix windows * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: route resolve-source-build call through Invoke-LlamaHelper The --resolve-source-build call at the source-build resolution path was still calling install_llama_prebuilt.py directly instead of going through Invoke-LlamaHelper. On PS7+ with ErrorActionPreference=Stop, stderr from the 422 response (when tag is "master") would trigger a terminating NativeCommandError and crash setup. * fix: suppress stderr error records from Invoke-LlamaHelper ErrorActionPreference=Continue prevents termination but PowerShell still displays stderr lines as visible ErrorRecord objects. Capture all output via 2>&1 and split stdout from stderr manually so that stderr lines never appear on the console. When StderrPath is given the stderr content is written to that file for diagnostics. * fix: always rebuild llama.cpp on Windows when tag is master When the requested llama.cpp tag is "master" (a moving target), skip the "already built" early exit so the build path runs and syncs to the latest commit. Without this, existing llama-server binaries from an older build (e.g. b8635 which lacks Gemma 4 support) are reused and model loading fails. Pinned tags (e.g. b8635) still skip the rebuild when the binary already exists, since the tag is immutable. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com> |
||
|
Daniel Han
|
b20efc370a |
Add regression tests for custom llama prebuilt installer (#4772)
Expand test coverage for install_llama_prebuilt.py: - Add tests for source build plan resolution with custom repos - Add tests for branch/commit/PR ref matching and normalization - Add tests for manifest checksum validation - Add tests for Windows CUDA upstream asset name patterns - Update capsys checks to capture stderr after log() redirect |
||
|
Daniel Han
|
f84c2d03d3 |
Add installer test coverage for prebuilt llama.cpp changes (#4756)
Split out from #4741 to keep the main PR focused on installer logic. - New test_install_llama_prebuilt_logic.py: tests for resolve logic, fallback behavior, env_int, busy/lock handling - New test_validate_llama_prebuilt.py: validator tests for staged release_tag/upstream_tag handling - New test_llama_pr_force_and_source.py: tests for PR_FORCE and LLAMA_SOURCE maintainer defaults - Updated test_selection_logic.py: expanded selection/fallback coverage - Updated test_pr4562_bugfixes.py: updated bugfix tests for new logic - Updated smoke_test_llama_prebuilt.py: minor update |
||
|
DoubleMathew
|
f4d8a246bf |
Use prebuilt llama.cpp for unsloth studio setup (#4562)
* Use prebuilt llama.cpp for unsloth studio setup * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix 3 issues that cause unnecessary fallback to source build 1. Make filelock import optional -- environments without filelock (e.g. minimal installs) crashed at import time instead of gracefully skipping the lock. 2. Use already-verified converter script from the hydrated source tree instead of re-downloading from raw.githubusercontent.com with no checksum. Adds symlink with copy fallback for the legacy filename. 3. Initialize $SkipPrebuiltInstall in setup.ps1 before first use to prevent potential uninitialized variable errors. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep network fallback in ensure_converter_scripts Prefer the local verified copy from the hydrated source tree, but retain the original network download as a fallback if the file is missing. Create the legacy hyphenated filename as a symlink with a copy fallback instead of writing a second full copy. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix 4 bugs in source-build fallback and binary_env paths - setup.ps1: Replace git pull + checkout FETCH_HEAD with fetch + checkout -B to avoid detached HEAD state that breaks re-runs. Use pinned tag in both fetch and clone paths. - setup.sh: Move rm -rf after cmake/git prerequisite checks so a missing tool no longer deletes the existing install. Add --branch tag to clone. - install_llama_prebuilt.py: Add binary_path.parent to Linux LD_LIBRARY_PATH in binary_env() so bundled .so files in build/bin are found even without RPATH, matching the existing Windows PATH logic. - Add test for binary_env LD_LIBRARY_PATH on Linux. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Handle unresolved "latest" tag in source-build fallback clone When tag resolution fails and the requested tag is "latest", both setup scripts now omit --branch from git clone so the default branch is cloned instead of failing on a nonexistent "latest" branch/tag. Similarly, the PS1 fetch path fetches the default ref when the tag is "latest". * Resolve actual latest ggml-org tag instead of using literal "latest" When both Python tag resolution attempts fail and the requested tag is "latest", query the GitHub API for the actual latest release tag from ggml-org/llama.cpp (e.g. b8508) instead of passing the literal string "latest" to git clone --branch, which would fail since no such branch/tag exists. setup.sh uses curl + python json parsing; setup.ps1 uses Invoke-RestMethod. Both fall back to the raw requested tag if the API call also fails. * Try Unsloth release repo before ggml-org when resolving latest tag When falling back to the GitHub API to resolve "latest", query the Unsloth release repo (unslothai/llama.cpp) first since it has the prebuilt binaries pinned to tested tags. Only fall back to ggml-org/llama.cpp if the Unsloth repo query fails. * Add comprehensive sandbox tests for PR #4562 bug fixes 35 tests covering all fixes across platforms: - binary_env cross-platform (Linux LD_LIBRARY_PATH, Windows PATH, macOS DYLD_LIBRARY_PATH) with edge cases (dedup, ordering, existing paths) - resolve_requested_llama_tag (concrete, latest, None, empty) - setup.sh logic via subprocess: prereq check ordering (cmake/git missing preserves install), pinned tag in clone, fetch+checkout -B pattern, fetch failure warns instead of aborting - "latest" tag resolution fallback chain (Unsloth API -> ggml-org -> raw) with mock curl: success, failure, malformed JSON, empty body, empty tag_name, env overrides - Source code pattern verification for both .sh and .ps1 files All 138 tests pass in isolated uv venv. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add binary_path.parent to macOS DYLD_LIBRARY_PATH in binary_env macOS prebuilt .dylib files are overlaid into build/bin (same as Linux), but binary_env only added install_dir to DYLD_LIBRARY_PATH. Add binary_path.parent so the loader can find sibling dylibs even without embedded loader paths. Mirrors the existing fix for Linux LD_LIBRARY_PATH and the Windows PATH pattern. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Guard --branch when resolved tag is "latest"; fix broken test assertion When all API fallbacks fail and the tag stays as literal "latest", omit --branch from git clone (clones default branch instead of failing). Both setup.sh and setup.ps1 now check for "latest" before passing --branch to git clone/fetch. Also fix test_setup_ps1_clone_uses_branch_tag which used Python tuple syntax (assert "x", "y" in z) that always passes. Changed to assert "x" in z and "y" in z. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix macOS DYLD trailing colon, install_lock no-op, and debug log - binary_env macOS: use dedupe_existing_dirs instead of raw string concatenation. Eliminates trailing colon in DYLD_LIBRARY_PATH (which causes dyld to search CWD for libraries) and deduplicates when binary_path.parent == install_dir. Now consistent with the Linux and Windows branches. - install_lock: when filelock is not installed, use os.O_CREAT|O_EXCL as a fallback exclusive file lock with timeout, instead of yielding with no locking. Prevents concurrent installs from corrupting each other's staging directories. - setup.ps1: remove [DEBUG] log line that printed to every user on every Windows setup run. * Add stale-lock detection and atomic clone-then-swap install_lock fallback (no filelock): write PID to lock file and check if the holder process is still alive on contention. Dead PIDs (ProcessLookupError) and unreadable lock files trigger immediate cleanup. Live processes owned by other users (PermissionError) are correctly recognized as alive -- the lock is not removed. setup.sh/setup.ps1 source-build: clone into a temporary directory first, then swap into place only on success. If git clone fails, the existing install is preserved instead of being deleted by the premature rm -rf. * Remove redundant upstream_tag != release_tag check load_approved_release_checksums compared checksums.upstream_tag against the Unsloth release_tag, which are different namespaces (upstream ggml-org tag vs Unsloth published tag). This only worked because both happened to be "b8508" by convention. Would break if Unsloth ever uses a different release naming scheme. The existing check at parse_approved_release_checksums (line 950) already validates the release_tag field correctly. * Fix lock TOCTOU race and build-in-temp-dir swap install_lock fallback: add os.fsync(fd) after writing PID to ensure the PID is visible to racing processes before they check. Treat empty lock files (PID not yet written) as "wait and retry" instead of stale, closing the window where two processes could both see an empty file, both unlink it, and both acquire the lock. setup.sh/setup.ps1 source-build: clone AND build in a temp directory (LLAMA_CPP_DIR.build.$$). Only swap into the final LLAMA_CPP_DIR after the build succeeds. If clone or cmake or build fails, the temp dir is cleaned up and the existing working install is preserved. Previously, rm -rf ran after clone but before build, destroying the existing install even if the build later failed. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |