Updating Studio in the container previously meant pulling a fresh ~25GB image
(or at best the ~6GB fused Studio layer) for what is usually a small Python/UI
change. Add a baked helper so a running container can update in place:
docker exec <container> unsloth-studio-update
It updates only the Studio packages -- the backend code and the pre-built
frontend, which ships inside the unsloth wheel -- with `pip install -U
--no-deps unsloth unsloth_zoo`, then restarts just the studio service via
supervisor. The torch/CUDA stack is left untouched, so it is safe in both GPU
and CPU-only containers. This deliberately avoids `unsloth studio update`,
which re-runs the full installer and re-probes the GPU to pick torch wheels --
in a container started without --gpus that finds no GPU and can downgrade torch
to CPU/cu126.
Options: --ref <branch|tag|sha> installs from git (track main) instead of the
latest PyPI release; --with-deps also updates dependencies; --no-restart defers
the restart. After the swap the helper smoke-imports studio.backend.main and,
if a transitive dep is now missing, points the user at --with-deps.
The update lands in the container's writable layer (survives docker restart);
mount -v unsloth_studio_home:/opt/unsloth-studio to keep it across a recreate.
Notebook dependency coverage (base Dockerfile):
- Bake omegaconf, einx, librosa, decord, ftfy so the TTS/STT and vision
notebooks stop dying on a silent No module named X. Installed in the
notebook-deps layer (after the torch/vLLM resolve) with an assertion that
the resolve did not move torch 2.10.0 / numpy>=2.3 / numba>=0.65.
Image size (no functional change):
- Base: prune npp to the two libs torchcodec actually dlopens
(libnppicc + libnppc), drop link-time-only .a archives and the nvshmem
device bitcode. Headers (torch/include etc) are kept so causal-conv1d /
mamba-ssm still build at notebook time with --no-build-isolation.
- Studio: pin the Studio venv to Python 3.12 (matches base) so its
nvidia-*-cu12 wheels are byte-identical to the base venv's, then symlink
the heavy arch-independent CUDA libs (cudnn/cublas/nccl/...) into the base
venv copy. cuda_nvrtc and cuda_runtime are excluded (the arm64 nvrtc swap
mutates nvrtc in place). Also remove the build-only frontend node_modules
(runtime serves the committed dist). Studio image drops ~4.8GB.
Per-notebook transformers version, run notebooks unchanged:
- Bake coherent transformers sidecars (4.57.6 default + 5.3.0/5.5.0/5.10.2),
each transformers==X with its matched huggingface_hub/tokenizers/
safetensors installed --no-deps into its own dir. Companion versions are
resolved at build time so they satisfy each transformers' requirements.
- unsloth_nb_compat.py: pick the sidecar from the notebook's pin or the
model name and activate it (prepend to sys.path) before any ML import,
without touching the base cu128 torch/vLLM/unsloth stack.
- pip/uv shim on PATH: a notebook install cell becomes safe and idempotent
inside a kernel (keeps the baked stack, records the requested transformers
for its sidecar); passthrough to the real tool everywhere else.
- IPython startup hook for manual JupyterLab, and unsloth-run for the
headless driven path.
llama-quantize exits nonzero on --help/--version while still printing
usage, so a bare invocation fails the build even when the binary is
healthy. Grep for the usage banner instead; a loader failure prints
error while loading shared libraries and no usage text.
The build/bin hardlink mirror skipped symlinks, so the soname links
(libllama-common.so.0 and friends) never reached build/bin. Studio's
setup.sh relinks the root llama-quantize to build/bin/llama-quantize,
whose RUNPATH is $ORIGIN, so the loader failed with libllama-common.so.0
not found and GGUF export from Studio died with No working quantizer
found, then hit the interactive source-build prompt in a non-TTY export
subprocess (EOFError). Mirror same-directory soname symlinks into
build/bin and extend the bake sanity check to execute llama-quantize from
both the install root and build/bin. Dockerfile.studio now also runs the
studio-visible quantizer after install.sh so a regression fails the
image build instead of runtime exports.
_detect_cuda_torch_index_url now respects the explicit family override
before probing nvidia-smi, matching install.sh get_torch_index_url and
install.ps1 Get-TorchIndexUrl. Without it, a GPU-less environment falls
back to cu126 wheels which lack sm_100/sm_120 kernels and break training
on Blackwell. ROCm repair path is intentionally unchanged.
Dockerfile.studio now fails the build if the Studio venv torch local
version tag does not match the pinned TORCH_FAMILY, so a studio ref whose
installer ignores the override can never ship a silently wrong image.
Metadata-only check so QEMU arm64 builds do not need to load torch.
- Dockerfile: lift numba past vllm's 0.61.2 pin after the numpy>=2.4
re-upgrade; 0.61.2 refuses numpy 2.3+ at import time and the stack
cannot move numpy down. Verified numba 0.65 + numpy 2.4.6 + vllm
import cleanly together.
- docker-publish.yml: resolve UNSLOTH_ZOO_REF in a step that mirrors
the pushed tag only when the tag exists in unsloth-zoo (the zoo
currently cuts no tags, so blind mirroring broke every tag publish);
falls back to main.
- Dockerfile.studio: Studio venv stays on cu128 for arm64 too, matching
the base venv (cu130 wheels would lift the driver floor to 580+), and
gets the same NVRTC cu13 swap for DGX Spark / GB10 sm_121 support.
- docker_confirm.sh: do not drop to CPU mode when docker info lacks a
nvidia runtime entry; CDI installs and Docker Desktop WSL2 expose
GPUs without one. The phase 3 --gpus probe is now the authority.
- docker_confirm.ps1: GPU selector built as an args array; comma device
lists get version-aware CSV quoting (native arg passing changed in
PowerShell 7.3).
- studio_launch.sh: no fixed Jupyter default password; generate a
random one and print it when JUPYTER_PASSWORD is unset. Env snapshot
for SSH sessions now written via shlex.quote instead of sed so
values with quotes or command substitution cannot break or inject
into /etc/profile.d.
- install.ps1: honour UNSLOTH_TORCH_INDEX_FAMILY like install.sh does.
entrypoint.sh: a container started without a GPU request has no
nvidia-smi at all (the toolkit injects it), so the old check 1 reported
'CUDA runtime in this image is broken, re-pull' for the most common user
error. Fold the missing-binary case into the actionable 'No GPU visible'
message and document the CPU-only option (UNSLOTH_ALLOW_CPU=1).
run.sh / test_locally.sh: guard empty-array expansions with the
${arr[@]+...} form; bash 3.2 (macOS /bin/bash) treats "${empty[@]}"
as unbound under set -u, which broke the documented macOS CPU path.
studio_launch.sh: exclude *_TOKEN, *_API_KEY, *_PASSWORD, *_SECRET,
*_LICENSE from the env snapshot written for SSH sessions; secrets stay
in process env only, never on disk.
supervisord.conf / Dockerfile.studio: pin HOME=/root for the studio and
jupyter programs (jupyter would silently fall back to token auth if HOME
were unset), default JUPYTER_PORT and UNSLOTH_ENABLE_SSHD at the image
level so a direct supervisord invocation cannot hit a bad %(ENV_*)s
expansion, and document the root-services decision (non-root parity with
the previous production image is a tracked follow-up).
docker_confirm.ps1: mirror the bash script's GPU selector translation so
GPUS=0 / 0,1 select devices instead of silently using all GPUs.
docker-publish.yml: studio cache scope moves to mode=min; a mode=max
cache of a ~24GB image would evict everything else in the 10GB GHA
quota for no hit-rate gain.
Two failures from the first in-image Studio install, both rooted in
install.sh probing the build host:
1. setup.sh aborted on the pre-linked llama.cpp dir: 'already exists and
is not marked as a Studio-owned llama.cpp install'. The dir is the
image's baked prebuilt, provisioned exclusively for Studio, so write
the .unsloth-studio-owned marker next to the binaries.
2. With no GPU and no nvidia-smi in the build container, install.sh fell
back to cu126 torch wheels for the Studio venv (and would pick cpu
wheels on a CI runner without /proc/driver/nvidia), so the published
image's Studio venv would depend on which host built it and could not
train on Blackwell. get_torch_index_url now honours an explicit
UNSLOTH_TORCH_INDEX_FAMILY override naming the index leaf (cu128,
cu130, rocm7.2, cpu, ...). The resolved family flows into
UNSLOTH_TORCH_BACKEND, which install_python_stack.py already consumes,
so the whole downstream chain follows the pin. Dockerfile.studio sets
cu128 on amd64 and cu130 on arm64 (DGX Spark / Grace).
Base image (docker/Dockerfile):
- Install JupyterLab + notebook + ipywidgets in a separate pure-Python uv
pass so the cu128 pin set cannot move; EXPOSE 8888.
- Bake the prebuilt llama.cpp bundle into /opt/unsloth/llama.cpp at the
runtime stage using studio/install_llama_prebuilt.py from the same
UNSLOTH_REF (sha256-verified, portable CUDA bundle since the build host
has no GPU; arm64 resolves the linux-arm64-cuda13 bundle). Export
UNSLOTH_LLAMA_CPP_PATH so unsloth_zoo's save_pretrained_gguf finds it
and never reaches the interactive install prompt or a source build.
- Optional github_token BuildKit secret for the resolver's API calls on
shared CI runner IPs.
Entrypoint: UNSLOTH_ALLOW_CPU=1 degrades a missing GPU to a warning so
Docker Desktop on macOS / Windows-without-WSL2-GPU and plain CPU hosts can
run Jupyter, GGUF tooling and Studio chat; with a GPU visible the normal
pre-flight still runs.
Full image (docker/Dockerfile.studio): now mirrors the production service
set under supervisord - Studio on 8000, JupyterLab on 8888, key-only sshd
on 22 (enabled only when PUBLIC_KEY/SSH_KEY is set). Points Studio's
llama.cpp dir at the baked bundle to skip a duplicate download, accepts
any git ref via fetch+checkout (CI passes commit SHAs), and FROMs a
digest-pinned BASE_IMAGE.
Publish workflow: base image moves to the base-* tag namespace; new
build-studio/merge-studio jobs publish the full image as :latest (hub
parity with the previous production image, which shipped Studio + Jupyter
+ SSH). Studio builds FROM the exact base manifest digest published by the
same run. GPU smoke job now also boots the full image and probes Studio
/api/health and Jupyter /api.
run.sh: UNSLOTH_GPUS=none, UNSLOTH_ALLOW_CPU forwarding, UNSLOTH_PORTS
publish flags, CPU-mode and Jupyter usage examples.
Round-2 of the 12-persona reviewer.py pass found 17 issues. Address the
P1s + the regression-class P2s in this commit; the remaining nits are
left for a follow-up cleanup pass.
1. unsloth/_gpu_init.py: the `NVIDIA_VISIBLE_DEVICES in os.environ` check
triggered for every NVIDIA-runtime container including `--gpus all`
(NVIDIA_VISIBLE_DEVICES=all is the default). Gate strictly on a
non-special device list. Also drop the precondition that the env var
was absent: if the user already pinned TORCHINDUCTOR_COMPILE_THREADS=1
we should still plant the UNSLOTH_FORCE_SINGLE_COMPILE_WORKER sentinel
so the zoo-side patch knows to preserve the forcing.
2. unsloth/_gpu_init.py: after the post-`import unsloth_zoo` reassertion,
monkey-patch `unsloth_zoo.temporary_patches.common.determine_compile_threads`
to return 1, so any later `torch.compile` call that rebuilds the
options dict still sees the single-worker forcing even if a downstream
patch_torch_compile pops the env var again.
3. docker/Dockerfile: torchaudio==2.11.0 mismatched the torch==2.10.0
release pairing; pin to 2.10.0 so the ABI is correct and the audio
stack matches torch/cu128.
4. docker/Dockerfile: drop `12.1+PTX` from TORCH_CUDA_ARCH_LIST. The
cu128 toolkit compiler does not know about compute_121; the trailing
PTX entry forced nvcc to emit a `sm_121` gencode that breaks any
in-container source builds.
5. docker/smoke_test.py: the device-capability floor said `cap[0] < 8`,
rejecting Turing (sm_75) while the Dockerfile + entrypoint advertise
sm_75 as supported. Lower the smoke floor to sm_75 and print a hint
that bf16 is not available on Turing.
6. docker/run.sh: `-it` is unconditional; CI / non-TTY invocations died
with "the input device is not a TTY". Probe `[ -t 0 ] && [ -t 1 ]`
first. Also remove `set -x` which echoed the forwarded HF_TOKEN /
WANDB_API_KEY / UNSLOTH_LICENSE values to stdout.
7. docker/test_locally.sh: `-e HF_TOKEN="${HF_TOKEN:-}"` either pasted
the secret verbatim into the process arg list or shadowed any
in-container value with an empty string. Forward conditionally.
8. .github/workflows/docker-publish.yml: gate `latest` on default branch
AND on `unsloth_ref` not being overridden via workflow_dispatch.
Otherwise a maintainer testing a feature SHA from main could overwrite
`:latest` with non-main source.
9. docker/Dockerfile.studio: add an `UNSLOTH_STUDIO_REF` build-arg so
the Studio companion image is pinned to a known unsloth ref instead
of cloning `main` whenever it builds.
install.sh --local installs unsloth into the Studio venv as an editable
package keyed to the just-cloned source tree. We were rm-rf'ing that
tree in the same RUN; the resulting `unsloth_cli` import then failed at
container start with `ModuleNotFoundError: No module named 'unsloth_cli'`.
Clone the source directly under UNSLOTH_STUDIO_HOME/src so it persists
in the image layer, and strip only .git to save ~120MB.
The base unsloth-blackwell image ships the `unsloth` CLI but refuses to
start `unsloth studio` until the dedicated Studio venv is laid down under
UNSLOTH_STUDIO_HOME by install.sh. Build it once and commit the result as
an opt-in companion tag (`:studio`) instead of bloating the base image.
Build:
docker buildx build --build-arg BASE_TAG=test \
-f docker/Dockerfile.studio -t unsloth-blackwell:studio docker/
Run:
docker run --rm --gpus '"device=0"' -p 8888:8888 unsloth-blackwell:studio
Open http://localhost:8888. Inference (llama.cpp CPU + GPU) and training
are both available. First-boot admin password lands in container logs
and at /opt/unsloth-studio/auth/.bootstrap_password.
