llama-quantize exits nonzero on --help/--version while still printing
usage, so a bare invocation fails the build even when the binary is
healthy. Grep for the usage banner instead; a loader failure prints
error while loading shared libraries and no usage text.
The build/bin hardlink mirror skipped symlinks, so the soname links
(libllama-common.so.0 and friends) never reached build/bin. Studio's
setup.sh relinks the root llama-quantize to build/bin/llama-quantize,
whose RUNPATH is $ORIGIN, so the loader failed with libllama-common.so.0
not found and GGUF export from Studio died with No working quantizer
found, then hit the interactive source-build prompt in a non-TTY export
subprocess (EOFError). Mirror same-directory soname symlinks into
build/bin and extend the bake sanity check to execute llama-quantize from
both the install root and build/bin. Dockerfile.studio now also runs the
studio-visible quantizer after install.sh so a regression fails the
image build instead of runtime exports.
unsloth_zoo.device_type.get_device_type() deliberately returns cuda when
UNSLOTH_ALLOW_CPU=1 and no accelerator exists (CPU CI, Docker Desktop
without GPU passthrough), but the DEVICE_TYPE == cuda import paths probed
torch.cuda.get_device_capability() unconditionally and raised
RuntimeError: Found no NVIDIA driver. Guard the module level probes with
torch.cuda.is_available(); bf16 stays enabled in the degrade branch since
CPU bf16 kernels exist while fp16 ones largely do not. Healthy GPU hosts
take the original branches unchanged. Found by the cross platform CPU
mode validation of the Docker images.
_detect_cuda_torch_index_url now respects the explicit family override
before probing nvidia-smi, matching install.sh get_torch_index_url and
install.ps1 Get-TorchIndexUrl. Without it, a GPU-less environment falls
back to cu126 wheels which lack sm_100/sm_120 kernels and break training
on Blackwell. ROCm repair path is intentionally unchanged.
Dockerfile.studio now fails the build if the Studio venv torch local
version tag does not match the pinned TORCH_FAMILY, so a studio ref whose
installer ignores the override can never ship a silently wrong image.
Metadata-only check so QEMU arm64 builds do not need to load torch.
SyntheticDataKit.from_pretrained waits for 'Starting vLLM API server on'
in the child's stdout before declaring the server up. vLLM 0.19 renamed
the line to 'Starting vLLM server on ...', so the regex never matched,
the 1200 s readiness timeout expired with a perfectly healthy server,
and the launcher tore it down; every downstream synthetic-data-kit step
then failed on missing files. Accept both wordings, watch stderr too
(vLLM has moved its logging between pipes across versions), and bail
out of the wait early if the child exits.
With this plus the ninja-build and flashinfer-jit-cache image fixes the
Meta synthetic data notebook goes from a 21 min timeout-and-fail to a
3 min pass inside the container.
The notebook validation matrix caught the synthetic-data notebook dying
because the vllm server SyntheticDataKit launches never came up. Two
layers to the failure:
1. flashinfer's cpp_ext JIT shells out to ninja. The pip ninja lives in
the venv bin, which subprocesses like vllm serve do not always
inherit on PATH, so the JIT failed with exit 127. Install ninja-build
so the binary is reachable from any PATH.
2. With ninja present the JIT still cannot succeed for device code: the
runtime image deliberately ships no nvcc. Bake flashinfer-jit-cache
(cu128) so ops missing from the cubin package (fmha_gen on sm_100a
was the repro) come precompiled. In-process GRPO never hit this
because unsloth-zoo blocks the FlashInfer JIT path; standalone
vllm serve gets no zoo patches.
Fail-soft on the jit-cache for arches without a wheel; the vLLM chain
itself stays fail-loud on amd64.
Notebooks fetch sample assets with !wget; without the binary the shell
prints not-found to stderr, the cell still exits zero from Jupyter's
perspective, and the next cell crashes confusingly on the missing file.
The Whisper notebook died exactly this way in the validation matrix.
PyPI has shipped aarch64 abi3 wheels for every vLLM release since 0.17,
so the arm64 skip rested on a stale premise. With torch held at 2.10.0
the resolver lands on vllm 0.19.1 (the release pinning torch==2.10.0)
on both arches; verified by cross-resolving the exact index set for
aarch64-unknown-linux-gnu.
amd64 keeps fail-loud semantics. arm64 is fail-soft because the aarch64
wheels are newer and their GPU kernels get validated on Spark hardware
via docker_confirm.sh rather than in CI; on failure the fallback
uninstalls vllm and restores the numpy/numba floor so a partial install
cannot break import unsloth (numpy 2.2.6 ships a broken numpy.testing).
The install steps form an explicit && chain instead of a set -e
subshell: POSIX shells disable errexit inside condition contexts
(verified on dash), so a (set -e; ...) condition would mask failures.
Both confirm scripts gain a 5b vLLM phase: ok on import, bad if missing
on x86_64, warn on other arches where fast_inference=True is best-effort.
IOReport energy counters can reset (sleep/wake, power gating), making a poll
delta negative. Return None for a negative total so the monitor shows -- for
that poll instead of a bogus negative wattage; it self-corrects next poll.
* Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix race in tool-call confirmation gate
* Studio: gate built-in tool calls and harden the confirmation handshake
The Allow / Always allow / Deny controls only lived in the fallback tool
card, but the built-in tools (web search, python, terminal, code
execution, image generation) render with their own components and so
never showed the buttons. Those calls paused after tool_start with no way
to approve them, hanging until the 1 hour timeout. Only MCP tools, which
use the fallback renderer, actually worked.
Render the controls for every tool card by wrapping each registered tool
component (and the fallback) in thread.tsx with a shared
ToolConfirmationControls, so the gate applies uniformly.
Also make the handshake robust:
- The gate keys on a per-call approval_id minted by the backend and
echoed in tool_start, instead of session_id alone, so a stale or
concurrent confirmation can no longer resolve the wrong call.
- The approval slot is registered before tool_start is yielded, closing
the race where a fast click or an auto "Always allow" could reach the
backend before the waiter existed.
- The frontend resolves with the same session id the request was sent
with (plus the approval_id), fixing the new-thread mismatch where the
confirmation targeted a different session than the blocked stream.
- The confirm endpoint returns {resolved}; the UI keeps the buttons and
shows a retry hint until the backend confirms a match, instead of
hiding them on a failed or mistargeted post.
- The gate runs after the disabled-tool and duplicate-call checks, so a
call that will not execute is not put up for approval. A denied call is
still excluded from duplicate detection, so re-issuing and approving it
works.
- "Always allow" is scoped per session to match the backend gate.
Add backend tests for the approval registry, the SSE no-deadlock
handshake, and the loop integration (allow, deny, disabled, duplicate,
re-issue after deny).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move "Confirm tool calls" to the Tools section
* Studio: Keep tool group open while a tool call awaits confirmation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix tool confirmation session scope for PR #5869
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix confirmation follow-ups for PR #5869
* Apply pre-commit formatting for PR #5869
* Fix confirmation cleanup for PR #5869
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden confirmation lookups for PR #5869
* Studio: make the tool-call confirmation decision immutable
resolve_tool_decision accepted a second confirmation for the same approval_id
and overwrote slot["decision"] in the window before the waiter reads it and
pops the slot, so a duplicate or out-of-order POST could flip an Allow to Deny
(and returned a misleading resolved:true). Reject once the slot's event is
already set so the first decision wins. Adds a regression test.
* Fix/adjust tool confirmations for PR #5869
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: show Apple GPU temperature and power in the GPU monitor (macOS)
The GPU monitor on Apple Silicon always showed -- for Temperature and
Power: the MLX branch of get_gpu_utilization() hardcoded None because
ioreg's AGXAccelerator PerformanceStatistics carries neither metric.
Add utils/hardware/apple.py, mirroring macmon's no-sudo approach:
- Temperature: average of the AppleSMC "Tg*" float keys via the
AppleSMCKeysEndpoint user client (ctypes/IOKit, macOS 14+).
- Power: IOReport "Energy Model" group, "GPU Energy" channels; each
poll diffs the energy counter against the previous poll's sample, so
the value is the average wattage over the polling window. The first
poll only sets the baseline and returns None.
Both readers latch to None on first failure and never raise, so
non-Mac platforms and locked-down hosts keep the previous behavior.
* Sample IOReport with the subscribed channels descriptor for PR #6187
IOReportCreateSubscription writes the channel descriptor that later samples
must use; sampling with the original requested group can return no Energy
Model entries on hosts that normalize the channel set, leaving power_draw_w
null after the baseline. Use the subscribed descriptor (matching macmon) and
fall back to the requested channels if the OS leaves it unset.
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <23090290+danielhanchen@users.noreply.github.com>
* Fix EmptyLogits gathering in nested structure and patch recursively_apply on accelerator module
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Wire EmptyLogits Accelerate patch into startup and fix find_device, pickling, tests for PR #6092
- Call patch_accelerate_recursively_apply() in _gpu_init.py so real imports
install it; previously it was only invoked by the tests
- Make both wrappers idempotent so repeated calls do not stack
- Rework find_device: skip EmptyLogits while still finding real tensors in any
order, keep returning None for tensor-free payloads (AlignDevicesHook relies
on None), fall back to PartialState().device only for sentinel-only payloads
- Give EmptyLogits stateless __reduce__ and drop the stomped pickle stubs on
EMPTY_LOGITS so debug mode gather_object works in real distributed runs
- Put test tensors on PartialState().device so the debug mode test also passes
on GPU machines, and add drift tests for startup wiring, idempotency and
find_device ordering
Verified on 2x B200: ACCELERATE_DEBUG_MODE=1 torchrun gather/broadcast/pad of
sentinel and mixed payloads all pass, training losses unchanged, full drift
suite 25/25.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Define EmptyLogits equality on the class for PR #6092
Gathered sentinel copies must compare equal in accelerate debug mode
regardless of whether the patched recursively_apply saw the sentinel first
in that process. Class body __eq__ requires restoring __hash__ explicitly.
Verified: 123 case simulation battery on accelerate 0.34.2 through latest,
2 process gloo CPU and NCCL GPU debug mode runs, drift suite 25/25.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Fix UnicodeDecodeError on Windows reading subprocess output in save path
On Windows the default text encoding is the locale code page (cp1252), not
UTF-8. The text-mode subprocess calls in save.py (text=True /
universal_newlines=True) set no explicit encoding, so they decode
llama.cpp / Ollama output with cp1252. When a child process emits a byte
undefined in cp1252 -- e.g. 0x9d, which appears inside the UTF-8 encoding
of common punctuation / box-drawing glyphs and in non-ASCII file paths --
the read raises UnicodeDecodeError and aborts GGUF export.
Add encoding="utf-8", errors="replace" to all 8 text-mode subprocess calls.
errors="replace" also avoids silent mojibake for inputs whose bytes happen
to be valid-but-wrong in cp1252.
Add tests/saving/test_save_subprocess_utf8_encoding.py:
- an AST drift detector asserting every text-mode subprocess call in
save.py pins encoding="utf-8" (runs without importing torch/unsloth_zoo)
- a behavioural test reproducing the cp1252 failure and the utf-8 fix
Relates-to: #2660
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* add FastModel config passthrough
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix fastmodel config passthrough for task configs
* fix config-driven FastModel task model selection
* fix text only fastmodel task config selection
* fix fastmodel task config inference from user configs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix fastmodel problem_type config passthrough
* fix fastlanguagemodel config passthrough: FastLlamaModel owns user config
* fix fastlanguagemodel config passthrough: forward user config to causal loads and keep checkpoint quantization_config
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
The post-download llama-quantize / llama-server smoke test JIT-compiles CUDA kernels on the first GPU forward pass and stalls every install and update by minutes on Blackwell (sm_100). Gate it behind _RUN_STAGED_PREBUILT_VALIDATION, disabled for now, keeping the smoke test and the source-build fallback it triggers fully intact so it can be restored by flipping the flag to True.
Hashless external prebuilts (e.g. lemonade) are not in the approved-sha256 manifest and rely on the functional smoke test as their only integrity gate, so they are always validated regardless of the flag; only approved bundles, already proven by the sha256 manifest, skip it.
The sha256 archive verification and the static Linux/macOS preflights are unchanged and still run for every install.
The wheel install belongs in the builder (the venv copy carries it),
but the ld.so.conf.d registration and the import check belong in the
runtime stage: the conf file does not survive the stage copy and the
import needs ffmpeg, which only the runtime stage installs.
test_sidebar_account_block_uses_leading_tight hardcoded gap-0.5 in its selector, but the sidebar account-block div moved to gap-px during UI polish (#6196), so the regex stopped matching and the test failed across every studio PR's Repo tests (CPU). Match the gap utility loosely (gap-\S+) since this guard is about the leading-* class for descender clipping, not the spacing.
The TTS/STT notebooks decode datasets Audio features through torchcodec,
which fails three different ways on a fresh image: the PyPI wheel pairs
with the cu13 torch line and dlopens libnvrtc.so.13; builds newer than
0.10 reference torch 2.11+ symbols; and the matching +cu128 build dlopens
torch and NVIDIA runtime libraries that live inside the venv where the
loader cannot see them. Bake ffmpeg, torchcodec==0.10.0 from the cu128
channel, nvidia-npp-cu12, and register the venv lib dirs via ld.so.conf.d
(not LD_LIBRARY_PATH, so the llama.cpp bundle keeps winning through its
own RUNPATH). Verified in-container: AudioDecoder imports and llama-server
still resolves its bundled libraries.
* fix(studio): adopt server-loaded model before chat auto-load
When the user starts Studio via `studio run -m`, the web UI could still
auto-load a different cached GGUF on the first message because the chat
checkpoint was empty. Sync from /api/inference/status before falling back
to autoLoadSmallestModel so CLI-loaded models are not replaced.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(studio): hydrate adopted CLI model and harden auto-load errors
Extract shared inference-status hydration for refresh() and CLI adopt
paths so the first chat turn gets reasoning/tools flags. Wrap auto-load
(including adopt) in try/catch for image-edit cleanup, and drop the
redundant adopt call in run().
Co-authored-by: Cursor <cursoragent@cursor.com>
* Guard model adoption against status failures and mid-flight selection for PR #5900
* ci: trigger pre-commit.ci after main merge
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(studio): inherit llama_extra_args and honor --no-mmproj
Reloading the same GGUF from the UI without gguf_variant no longer drops
CLI pass-through args like --no-mmproj. Skip mmproj download and launch
when --no-mmproj is present in llama_extra_args.
Co-authored-by: Cursor <cursoragent@cursor.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): tighten GGUF llama_extra_args variant inheritance guard
Reject inherited CLI args when the request changes gguf_variant or when
omitted variant resolves differently from the stored extra_args source.
Co-authored-by: Cursor <cursoragent@cursor.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Treat --no-mmproj-auto and --mmproj-auto with last-wins parsing for PR #5902
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Running the published unslothai/notebooks set inside the image surfaced
two gaps: the Ollama export notebook installs ollama in-container and
that installer needs zstd for extraction, and DeepSeek-OCR's
trust_remote_code modeling file imports matplotlib unconditionally
(plotting is also simply expected in a Jupyter image).
* fix(studio): load run.py by path for editable installs
`studio update` can leave a partial site-packages/studio/backend/ tree
(plugin build artefacts only). That shadowed tree wins over an editable
install and breaks `from studio.backend.run import ...`. Loading run.py
by file path via importlib sidesteps the conflict.
The module is cached in _RUN_MODULE so repeated calls are cheap.
If exec_module fails, the module is removed from sys.modules before
re-raising so a subsequent retry starts clean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle None __file__ when checking cached run module for PR #5909
* Harden _load_backend_auth_storage against None __file__ and resolve cache-key path (PR #5909)
* Adapt studio run/cloudflare in-venv tests to _load_run_module loader (PR #5909)
---------
Co-authored-by: Jim Dawdy <jimdawdy@Jims-MacBook-Pro.local>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
The checker does not count attribute assignment on an aliased module
import as a use and flagged _zoo_common as added-but-unused. Set the
attribute through importlib.import_module instead; importlib is already
a module-level import here. Behaviour unchanged.
A staging run of the studio image build died with ENOSPC during the
Studio venv install: the hosted runners' default free space does not
fit the base image plus buildkit state plus the Studio layer. Drop all
unused preinstalled toolchains and the runner's preloaded docker
images in both build jobs.
- Dockerfile: lift numba past vllm's 0.61.2 pin after the numpy>=2.4
re-upgrade; 0.61.2 refuses numpy 2.3+ at import time and the stack
cannot move numpy down. Verified numba 0.65 + numpy 2.4.6 + vllm
import cleanly together.
- docker-publish.yml: resolve UNSLOTH_ZOO_REF in a step that mirrors
the pushed tag only when the tag exists in unsloth-zoo (the zoo
currently cuts no tags, so blind mirroring broke every tag publish);
falls back to main.
- Dockerfile.studio: Studio venv stays on cu128 for arm64 too, matching
the base venv (cu130 wheels would lift the driver floor to 580+), and
gets the same NVRTC cu13 swap for DGX Spark / GB10 sm_121 support.
- docker_confirm.sh: do not drop to CPU mode when docker info lacks a
nvidia runtime entry; CDI installs and Docker Desktop WSL2 expose
GPUs without one. The phase 3 --gpus probe is now the authority.
- docker_confirm.ps1: GPU selector built as an args array; comma device
lists get version-aware CSV quoting (native arg passing changed in
PowerShell 7.3).
- studio_launch.sh: no fixed Jupyter default password; generate a
random one and print it when JUPYTER_PASSWORD is unset. Env snapshot
for SSH sessions now written via shlex.quote instead of sed so
values with quotes or command substitution cannot break or inject
into /etc/profile.d.
- install.ps1: honour UNSLOTH_TORCH_INDEX_FAMILY like install.sh does.
entrypoint.sh: a container started without a GPU request has no
nvidia-smi at all (the toolkit injects it), so the old check 1 reported
'CUDA runtime in this image is broken, re-pull' for the most common user
error. Fold the missing-binary case into the actionable 'No GPU visible'
message and document the CPU-only option (UNSLOTH_ALLOW_CPU=1).
run.sh / test_locally.sh: guard empty-array expansions with the
${arr[@]+...} form; bash 3.2 (macOS /bin/bash) treats "${empty[@]}"
as unbound under set -u, which broke the documented macOS CPU path.
studio_launch.sh: exclude *_TOKEN, *_API_KEY, *_PASSWORD, *_SECRET,
*_LICENSE from the env snapshot written for SSH sessions; secrets stay
in process env only, never on disk.
supervisord.conf / Dockerfile.studio: pin HOME=/root for the studio and
jupyter programs (jupyter would silently fall back to token auth if HOME
were unset), default JUPYTER_PORT and UNSLOTH_ENABLE_SSHD at the image
level so a direct supervisord invocation cannot hit a bad %(ENV_*)s
expansion, and document the root-services decision (non-root parity with
the previous production image is a tracked follow-up).
docker_confirm.ps1: mirror the bash script's GPU selector translation so
GPUS=0 / 0,1 select devices instead of silently using all GPUs.
docker-publish.yml: studio cache scope moves to mode=min; a mode=max
cache of a ~24GB image would evict everything else in the 10GB GHA
quota for no hit-rate gain.
Studio's setup.sh provisioning runs install_llama_prebuilt.py, whose
host-probing cannot succeed inside an image build, so it fell back to a
CPU-only llama.cpp source build layered over the baked CUDA bundle.
setup.sh skips that fallback when build/bin/llama-server and
build/bin/llama-quantize are executable, so hardlink the installed bundle
into build/bin: zero extra bytes, $ORIGIN rpath still resolves, and no
symlink cycle when setup.sh later relinks the root quantizer to
build/bin/llama-quantize.
Two failures from the first in-image Studio install, both rooted in
install.sh probing the build host:
1. setup.sh aborted on the pre-linked llama.cpp dir: 'already exists and
is not marked as a Studio-owned llama.cpp install'. The dir is the
image's baked prebuilt, provisioned exclusively for Studio, so write
the .unsloth-studio-owned marker next to the binaries.
2. With no GPU and no nvidia-smi in the build container, install.sh fell
back to cu126 torch wheels for the Studio venv (and would pick cpu
wheels on a CI runner without /proc/driver/nvidia), so the published
image's Studio venv would depend on which host built it and could not
train on Blackwell. get_torch_index_url now honours an explicit
UNSLOTH_TORCH_INDEX_FAMILY override naming the index leaf (cu128,
cu130, rocm7.2, cpu, ...). The resolved family flows into
UNSLOTH_TORCH_BACKEND, which install_python_stack.py already consumes,
so the whole downstream chain follows the pin. Dockerfile.studio sets
cu128 on amd64 and cu130 on arm64 (DGX Spark / Grace).
The dockerignore uses an everything-out whitelist; fetch_llama_prebuilt.py
(base bake) and supervisord.conf + studio_launch.sh (Dockerfile.studio)
need explicit entries. docker_confirm.ps1 is the Windows Docker Desktop
counterpart of docker_confirm.sh.
The first bake attempt reused studio/install_llama_prebuilt.py, but that
resolver selects a bundle for the CURRENT host: on a GPU build host
/proc/driver/nvidia leaks into docker build and the resolver goes down the
CUDA path with no readable driver runtime (chosen_asset=none, exit 2),
while on a GPU-less CI runner it would resolve a CPU bundle instead. Both
violate the image's build-host-independence rule.
fetch_llama_prebuilt.py pins by build target only: amd64 takes the
linux-x64-cuda12-portable bundle, arm64 the linux-arm64-cuda13-portable
bundle (DGX Spark / Grace), both sha256-verified against the release's
llama-prebuilt-sha256.json. convert_hf_to_gguf.py plus gguf-py/ are
hydrated from the same release's source tarball so the converter's tensor
mappings match the binaries, mirroring unsloth_zoo's
_hydrate_converter_sources layout. LLAMA_PREBUILT_TAG build-arg overrides
the pinned release.
docker_confirm.sh: one-command confirmation script for any machine
(Linux / WSL2 / macOS) following the staging confirm-script conventions:
host + docker + GPU detection with CPU-mode auto-fallback, image pulls,
in-container torch.cuda check, 5-step LoRA training smoke, baked llama.cpp
verification, full-image boot probing Studio /api/health and JupyterLab
/api, PASS/WARN/FAIL summary with RESULT line.
Base image (docker/Dockerfile):
- Install JupyterLab + notebook + ipywidgets in a separate pure-Python uv
pass so the cu128 pin set cannot move; EXPOSE 8888.
- Bake the prebuilt llama.cpp bundle into /opt/unsloth/llama.cpp at the
runtime stage using studio/install_llama_prebuilt.py from the same
UNSLOTH_REF (sha256-verified, portable CUDA bundle since the build host
has no GPU; arm64 resolves the linux-arm64-cuda13 bundle). Export
UNSLOTH_LLAMA_CPP_PATH so unsloth_zoo's save_pretrained_gguf finds it
and never reaches the interactive install prompt or a source build.
- Optional github_token BuildKit secret for the resolver's API calls on
shared CI runner IPs.
Entrypoint: UNSLOTH_ALLOW_CPU=1 degrades a missing GPU to a warning so
Docker Desktop on macOS / Windows-without-WSL2-GPU and plain CPU hosts can
run Jupyter, GGUF tooling and Studio chat; with a GPU visible the normal
pre-flight still runs.
Full image (docker/Dockerfile.studio): now mirrors the production service
set under supervisord - Studio on 8000, JupyterLab on 8888, key-only sshd
on 22 (enabled only when PUBLIC_KEY/SSH_KEY is set). Points Studio's
llama.cpp dir at the baked bundle to skip a duplicate download, accepts
any git ref via fetch+checkout (CI passes commit SHAs), and FROMs a
digest-pinned BASE_IMAGE.
Publish workflow: base image moves to the base-* tag namespace; new
build-studio/merge-studio jobs publish the full image as :latest (hub
parity with the previous production image, which shipped Studio + Jupyter
+ SSH). Studio builds FROM the exact base manifest digest published by the
same run. GPU smoke job now also boots the full image and probes Studio
/api/health and Jupyter /api.
run.sh: UNSLOTH_GPUS=none, UNSLOTH_ALLOW_CPU forwarding, UNSLOTH_PORTS
publish flags, CPU-mode and Jupyter usage examples.
* Handle rope_type 'default' on transformers 5 to stop false RoPE warning
transformers 5 reports rope_type="default" for every plain (unscaled) config
and dropped "default" from ROPE_INIT_FUNCTIONS. _compute_config_rope_inv_freq
then did ROPE_INIT_FUNCTIONS["default"], hit KeyError, returned None and logged
"Could not apply RoPE scaling 'default'; long-context generation may degrade"
on every model load. The inv_freq was still correct (the constructor recomputes
vanilla on None), but the warning is a false alarm for unscaled models.
Compute the unscaled inv_freq directly for rope_type "default"/None instead of
going through ROPE_INIT_FUNCTIONS, so plain configs return the right value with
no warning. Scaled types (llama3/linear/yarn/...) are unchanged.
Also skip test_object_style_rope_scaling_on_config_delegates_correctly when
transformers strict-validates rope_scaling (5.x): it rejects a non-dict object
on config.rope_scaling, so the object-style delegation path cannot be set up
there. The test still runs and asserts on transformers <5.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: deduplicate lemonade ROCm prebuilt selection log
resolve_lemonade_rocm_choice() is called twice per install (direct
planner + resolve_upstream_asset_choice). The API fetch is already
memoised via _fetch_lemonade_release_cached but the selection log
lines were still emitted on both calls, printing the 'trying
lemonade-sdk ROCm prebuilt' banner and hash-manifest NOTE twice.
Add _lemonade_selection_logged set keyed on (gfx_target, asset_name)
and guard the two log() calls behind a membership check so they print
exactly once per process regardless of call count.
Also extend the _clear_lemonade_release_cache test fixture to clear
the new set between tests to prevent cross-test state bleed.
Fixes#6020
* fix: write log() output to stdout to avoid PowerShell NativeCommandError
On Windows, PowerShell treats any stderr output from a native process as
an error record and prefixes it with 'python.exe :' and sets the
ErrorId to NativeCommandError. Since log() wrote to sys.stderr, every
[llama-prebuilt] status line triggered this, making normal progress
output look like errors in the installer console.
Switch log() to sys.stdout. The download progress bar (DownloadProgress)
retains its stderr/tty logic unchanged -- that path is for interactive
terminal rendering, not status logging.
* fix: remove redundant 'or ""' in lemonade log_key
host.rocm_gfx_target is already guaranteed truthy by the early
return at the top of resolve_lemonade_rocm_choice. The fallback
was dead code.
* Keep resolver stdout machine-readable, route install logs to stdout
log() sending everything to stdout breaks the resolver modes: setup.sh
json.load()s the whole stdout, so one helper log line (network retry,
release-tag scan) corrupts the parse and silently drops back to building
"latest". Default log() to stderr and flip to stdout only on the install
path, where PowerShell otherwise renders stderr as NativeCommandError
noise. Also tighten the lemonade dedup comments.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
On a 0.0.0.0 bind whose public ip:port is not reachable (cloud firewall),
the banner still printed "Secure link access via Cloudflare: <url>" right
after "is NOT reachable from the public internet", which reads as if the
tunnel might also be blocked. The Cloudflare quick-tunnel works regardless.
Thread the reachability probe result through a module-level _public_reachable
tri-state and, when the public probe definitively failed but the tunnel is
up, print "Also, the secure link access via Cloudflare works: <url>".
Reachable or undecided cases keep the existing wording.
PyPI release unsloth 2026.6.3 is now live. Bump the pinned floor in
install.sh and install.ps1 from unsloth>=2026.6.2 to unsloth>=2026.6.3
so fresh installs resolve to the new wheel.
* Studio: llama.cpp update banner redesign, About tab license info, inline system prompt editing, naming cleanup
- Redesign the llama.cpp update banner to match the chat composer surface
(borderless rounded card, composer shadow, Hellix Medium title), rename
actions to Update and add a 15 minute Remind me later snooze
- Keep the banner up until the user explicitly acts on it; drop the
outside click dismissal
- Add a Settings > General > Notifications toggle to disable the banner
for training-only setups (on by default)
- Rename the Help settings tab to About and add a License section
(Unsloth Studio AGPL-3.0, Unsloth Core Apache-2.0) linking to the
license files in this repo
- Make the run settings system prompt box an inline editable textarea;
the popup editor opens when the prompt overflows the box
- Pointer cursor on the preset dropdown chevron
- Dark mode toasts use the chat composer surface color
- Replace standalone Studio with Unsloth in user facing strings; keep
Unsloth Studio, LM Studio, Fine-tuning Studio, Recipe Studio and CLI
commands unchanged
* Studio: open the system prompt popup on box click, balance banner padding
- The system prompt box opens the Edit System Prompt dialog on click,
matching the pencil action
- Slightly more bottom padding on the llama.cpp update banner so the
spacing reads even next to the action pills
* Studio: replace unsloth studio update with the installer commands in update guidance
- The unsloth studio update command no longer works, so the About tab
update section now shows the one-line installer (curl or irm) for
PyPI and unknown installs, and git pull plus the local installer for
checkouts
- Add a short note that unsloth studio update is no longer supported
- Link the Installation, Updating and Windows install docs pages
- The package update banner now copies the platform installer command
instead of unsloth studio update
* Studio: rounder account menu, inline system prompt box with popup from the label
- Account menu corners go from 14px to 18px via a specific override,
since list menus pin border-radius globally
- llama.cpp banner bottom padding 22px
- System prompt is an inline editable textarea again; clicking the
System Prompt label opens the popup editor, and an overflowing
prompt opens it on box click
* Studio: show the standard install commands in the About update section
- Both one-line install commands (MacOS/Linux/WSL and Windows
PowerShell) are always shown, labeled like the docs, since running
them again updates an existing install
- Drop the unsloth studio update deprecation note
- Add the Mac install guide to the docs links
* Studio: clearer platform toggle and layout in the About update section
- Section heading is Update
- Platform picker is a pair of pill buttons, MacOS / Linux and Windows,
and only the selected platform's install command is shown
- Intro reads: To install or update Unsloth
- Local update heading separates checkout guidance from the standard
install command
* Studio: report GitHub branch instead of dev for source checkouts
A source checkout not on an exact release tag now shows
GitHub <branch> (e.g. GitHub main) as the Studio version in About.
Detached or unusual HEADs still fall back to dev.
* Studio: tighten the About update section copy and toggle styling
- Platform toggle buttons are borderless pills
- Shorter local update wording and restart note
- Docs links read Mac and Windows
* Studio: tighten line spacing in the sidebar account button
* Studio: fix vanishing compact MCP icon on hover, single line pill tooltips
- Compact caret pills (MCP, RAG) keep their icon on hover for inactive
pills too; the off switch hover rules hid the icon while compact mode
hid the X, leaving an empty slot
- Compact icon tooltips and single line compact tooltips render as full
pills; wrapped tooltips keep the 9px corners. TooltipContent measures
line count in a ref callback since Radix mounts portal content
without re-rendering the wrapper
- 1px gap between the name and Unsloth lines in the sidebar account
button
* Studio: Projects hover plus button, align recents with the label
- Hovering the Projects nav item reveals a plus button that opens the
New project dialog, with the same circular hover treatment as the
chat row actions
- Recent chat titles start at the same x as the Recents label
- The system prompt overflow lock only engages for a non-empty prompt
with a laid-out box, so a mis-measure cannot turn clicks into the
popup
* Clip system prompt overflow inside the rounded box
Wrap the inline system prompt textarea in a rounded overflow-hidden
surface so scrolled text and the scrollbar stay inside the box. The
focus ring moves to the wrapper via focus-within.
* Add updating progress bar to llama banner and shorten settings copy
While an update is applying, the banner action row becomes an
indeterminate progress bar that keeps animating under reduced motion,
matching the other loading indicators. Settings descriptions across
General, Profile, Appearance, Chat, Connections, API, and About are
trimmed without losing meaning.
* Address review: desktop update note, server platform detection, zh-CN keys
The About tab no longer shows terminal install commands in the desktop
app, where the bundled backend updates through the built-in updater;
it shows a short note and the docs links instead.
fetchDeviceType now sends the auth token to /api/health, which only
reports the server platform to authed callers, and caches only a
server-reported value. Copied install commands then match the host
platform rather than the browser when they differ (WSL, SSH).
zh-CN gains translations for the new notification and license keys,
the renamed About tab title, and the desktop update note.
* Real download progress for llama.cpp updates, prompt and sidebar polish
The update worker now streams the installer output and parses its
download percent lines into job progress, exposed via the update-status
API. The installer emits finer non-tty milestones when
UNSLOTH_PROGRESS_PERCENT_STEP is set; the worker requests 5 percent
steps. The banner renders a determinate bar from the reported fraction
and falls back to the sweep until the first percent arrives.
Also removes the focus ring on the inline system prompt box and
slightly shrinks the Projects hover plus icon.
