Stacks a Colab-like JupyterLab and Studio experience on top of the
existing Blackwell image. Additive only: the training stack, CUDA/torch
pinning, and the Studio/JupyterLab/sshd service trio are unchanged.
JupyterLab labextension (prebuilt in a throwaway builder stage, so the
runtime image stays Node-free):
- Unsloth Dark (Monokai) theme, adaptive light/dark by system preference
- Colab-style ArrowDown/Up cell navigation
- top-bar Unsloth logo (stock Jupyter logo disabled and locked)
- #@title lines render as collapsible Heading-2 form bars
- Ctrl+A in a cell output selects only that output, not the whole
notebook (the old behaviour ran notebook:select-all and was laggy)
- right activity bar hidden by default
- overrides.json: per-cell run button without auto-advance, labeled
Restart and Run All, windowing off so collapsing an output does not
snap to the cell top, news/update prompts suppressed
Studio and login branding: Unsloth favicon, page logo, and a dark
Unsloth login page that rotates through the curated Studio sloth
stickers (fail-soft to the logo).
Notebook organization and Colab compatibility (base image):
- categorized folder view built from relative symlinks mirroring the
README sections, rebuilt each boot; real .ipynb files never moved,
and the symlink tree is invisible to the sync state machine
- AMD-* notebooks shown only on an AMD/HIP host (autodetected)
- Docker-only strip of the Colab "Run all on Colab" intro sentence
from unedited notebooks (upstream notebooks unchanged)
- hoist %%capture above a leading #@title form so the cell runs
- the per-cell transformers-sidecar log is silent unless
UNSLOTH_ENABLE_LOGGING=1
Dependency pinning and naming: the curated notebook extras are pinned to
their resolved versions for reproducible rebuilds; decord is split into
its own fail-soft install (no aarch64 wheel). The lean base image is
renamed from :base to :core.
Adds tests/validate_studio_features.py, a static self-test for the
labextension plugins, overrides keys, and branding wiring.
Mirror the public-link convenience Studio already has for its own UI, for
JupyterLab. Off by default; opt in two ways:
docker run -e UNSLOTH_JUPYTER_CLOUDFLARE=1 ... unsloth/unsloth
docker exec <container> unsloth-jupyter-tunnel --force
unsloth-jupyter-tunnel waits for JupyterLab, reuses a cached cloudflared (or
fetches the static binary for the arch, no account needed), and starts a
quick tunnel to the Jupyter port; the https://<name>.trycloudflare.com URL is
printed to docker logs. supervisord runs it as the jupyter-cloudflare program,
autostarted only when UNSLOTH_JUPYTER_CLOUDFLARE=1 (studio_launch.sh exports a
0 default so the autostart gate expands, matching the sshd pattern). JupyterLab
still enforces its password, so the tunnel is not an open door.
Verified: the helper fetches cloudflared and mints a working trycloudflare URL
that reaches JupyterLab (HTTP 200) inside a running container.
Parity with unsloth-studio-update: update the baked llama.cpp prebuilt in a
running container without pulling a new image.
docker exec <container> unsloth-llama-update # latest release
docker exec <container> unsloth-llama-update --check # report only
It reuses the build-time fetcher (fetch_llama_prebuilt.py, now baked at
/usr/local/lib/unsloth) rather than the host-probing installer behind the
in-app banner. The fetcher resolves the latest release via the GitHub
/releases/latest redirect (no API token, not rate-limited) and installs the
portable CUDA bundle that runs on CPU and every supported GPU, so it works the
same in a CPU-only or a --gpus container. The installer path, by contrast,
scans the GitHub API (rate-limited to 403 in practice) and probes the host GPU,
which falls back to a slow source build in a container started without --gpus.
The fetch lands in a sibling temp dir on the same filesystem and is swapped in
with an atomic rename; on any failure the existing install is left untouched.
The Studio ownership marker is preserved across the swap. Verified end to end
in a CPU-only container: b9596-mix-e6f2453 -> b9773-mix-1f1aaa4.
Updating Studio in the container previously meant pulling a fresh ~25GB image
(or at best the ~6GB fused Studio layer) for what is usually a small Python/UI
change. Add a baked helper so a running container can update in place:
docker exec <container> unsloth-studio-update
It updates only the Studio packages -- the backend code and the pre-built
frontend, which ships inside the unsloth wheel -- with `pip install -U
--no-deps unsloth unsloth_zoo`, then restarts just the studio service via
supervisor. The torch/CUDA stack is left untouched, so it is safe in both GPU
and CPU-only containers. This deliberately avoids `unsloth studio update`,
which re-runs the full installer and re-probes the GPU to pick torch wheels --
in a container started without --gpus that finds no GPU and can downgrade torch
to CPU/cu126.
Options: --ref <branch|tag|sha> installs from git (track main) instead of the
latest PyPI release; --with-deps also updates dependencies; --no-restart defers
the restart. After the swap the helper smoke-imports studio.backend.main and,
if a transitive dep is now missing, points the user at --with-deps.
The update lands in the container's writable layer (survives docker restart);
mount -v unsloth_studio_home:/opt/unsloth-studio to keep it across a recreate.
Notebook dependency coverage (base Dockerfile):
- Bake omegaconf, einx, librosa, decord, ftfy so the TTS/STT and vision
notebooks stop dying on a silent No module named X. Installed in the
notebook-deps layer (after the torch/vLLM resolve) with an assertion that
the resolve did not move torch 2.10.0 / numpy>=2.3 / numba>=0.65.
Image size (no functional change):
- Base: prune npp to the two libs torchcodec actually dlopens
(libnppicc + libnppc), drop link-time-only .a archives and the nvshmem
device bitcode. Headers (torch/include etc) are kept so causal-conv1d /
mamba-ssm still build at notebook time with --no-build-isolation.
- Studio: pin the Studio venv to Python 3.12 (matches base) so its
nvidia-*-cu12 wheels are byte-identical to the base venv's, then symlink
the heavy arch-independent CUDA libs (cudnn/cublas/nccl/...) into the base
venv copy. cuda_nvrtc and cuda_runtime are excluded (the arm64 nvrtc swap
mutates nvrtc in place). Also remove the build-only frontend node_modules
(runtime serves the committed dist). Studio image drops ~4.8GB.
Per-notebook transformers version, run notebooks unchanged:
- Bake coherent transformers sidecars (4.57.6 default + 5.3.0/5.5.0/5.10.2),
each transformers==X with its matched huggingface_hub/tokenizers/
safetensors installed --no-deps into its own dir. Companion versions are
resolved at build time so they satisfy each transformers' requirements.
- unsloth_nb_compat.py: pick the sidecar from the notebook's pin or the
model name and activate it (prepend to sys.path) before any ML import,
without touching the base cu128 torch/vLLM/unsloth stack.
- pip/uv shim on PATH: a notebook install cell becomes safe and idempotent
inside a kernel (keeps the baked stack, records the requested transformers
for its sidecar); passthrough to the real tool everywhere else.
- IPython startup hook for manual JupyterLab, and unsloth-run for the
headless driven path.
llama-quantize exits nonzero on --help/--version while still printing
usage, so a bare invocation fails the build even when the binary is
healthy. Grep for the usage banner instead; a loader failure prints
error while loading shared libraries and no usage text.
The build/bin hardlink mirror skipped symlinks, so the soname links
(libllama-common.so.0 and friends) never reached build/bin. Studio's
setup.sh relinks the root llama-quantize to build/bin/llama-quantize,
whose RUNPATH is $ORIGIN, so the loader failed with libllama-common.so.0
not found and GGUF export from Studio died with No working quantizer
found, then hit the interactive source-build prompt in a non-TTY export
subprocess (EOFError). Mirror same-directory soname symlinks into
build/bin and extend the bake sanity check to execute llama-quantize from
both the install root and build/bin. Dockerfile.studio now also runs the
studio-visible quantizer after install.sh so a regression fails the
image build instead of runtime exports.
_detect_cuda_torch_index_url now respects the explicit family override
before probing nvidia-smi, matching install.sh get_torch_index_url and
install.ps1 Get-TorchIndexUrl. Without it, a GPU-less environment falls
back to cu126 wheels which lack sm_100/sm_120 kernels and break training
on Blackwell. ROCm repair path is intentionally unchanged.
Dockerfile.studio now fails the build if the Studio venv torch local
version tag does not match the pinned TORCH_FAMILY, so a studio ref whose
installer ignores the override can never ship a silently wrong image.
Metadata-only check so QEMU arm64 builds do not need to load torch.
- Dockerfile: lift numba past vllm's 0.61.2 pin after the numpy>=2.4
re-upgrade; 0.61.2 refuses numpy 2.3+ at import time and the stack
cannot move numpy down. Verified numba 0.65 + numpy 2.4.6 + vllm
import cleanly together.
- docker-publish.yml: resolve UNSLOTH_ZOO_REF in a step that mirrors
the pushed tag only when the tag exists in unsloth-zoo (the zoo
currently cuts no tags, so blind mirroring broke every tag publish);
falls back to main.
- Dockerfile.studio: Studio venv stays on cu128 for arm64 too, matching
the base venv (cu130 wheels would lift the driver floor to 580+), and
gets the same NVRTC cu13 swap for DGX Spark / GB10 sm_121 support.
- docker_confirm.sh: do not drop to CPU mode when docker info lacks a
nvidia runtime entry; CDI installs and Docker Desktop WSL2 expose
GPUs without one. The phase 3 --gpus probe is now the authority.
- docker_confirm.ps1: GPU selector built as an args array; comma device
lists get version-aware CSV quoting (native arg passing changed in
PowerShell 7.3).
- studio_launch.sh: no fixed Jupyter default password; generate a
random one and print it when JUPYTER_PASSWORD is unset. Env snapshot
for SSH sessions now written via shlex.quote instead of sed so
values with quotes or command substitution cannot break or inject
into /etc/profile.d.
- install.ps1: honour UNSLOTH_TORCH_INDEX_FAMILY like install.sh does.
entrypoint.sh: a container started without a GPU request has no
nvidia-smi at all (the toolkit injects it), so the old check 1 reported
'CUDA runtime in this image is broken, re-pull' for the most common user
error. Fold the missing-binary case into the actionable 'No GPU visible'
message and document the CPU-only option (UNSLOTH_ALLOW_CPU=1).
run.sh / test_locally.sh: guard empty-array expansions with the
${arr[@]+...} form; bash 3.2 (macOS /bin/bash) treats "${empty[@]}"
as unbound under set -u, which broke the documented macOS CPU path.
studio_launch.sh: exclude *_TOKEN, *_API_KEY, *_PASSWORD, *_SECRET,
*_LICENSE from the env snapshot written for SSH sessions; secrets stay
in process env only, never on disk.
supervisord.conf / Dockerfile.studio: pin HOME=/root for the studio and
jupyter programs (jupyter would silently fall back to token auth if HOME
were unset), default JUPYTER_PORT and UNSLOTH_ENABLE_SSHD at the image
level so a direct supervisord invocation cannot hit a bad %(ENV_*)s
expansion, and document the root-services decision (non-root parity with
the previous production image is a tracked follow-up).
docker_confirm.ps1: mirror the bash script's GPU selector translation so
GPUS=0 / 0,1 select devices instead of silently using all GPUs.
docker-publish.yml: studio cache scope moves to mode=min; a mode=max
cache of a ~24GB image would evict everything else in the 10GB GHA
quota for no hit-rate gain.
Two failures from the first in-image Studio install, both rooted in
install.sh probing the build host:
1. setup.sh aborted on the pre-linked llama.cpp dir: 'already exists and
is not marked as a Studio-owned llama.cpp install'. The dir is the
image's baked prebuilt, provisioned exclusively for Studio, so write
the .unsloth-studio-owned marker next to the binaries.
2. With no GPU and no nvidia-smi in the build container, install.sh fell
back to cu126 torch wheels for the Studio venv (and would pick cpu
wheels on a CI runner without /proc/driver/nvidia), so the published
image's Studio venv would depend on which host built it and could not
train on Blackwell. get_torch_index_url now honours an explicit
UNSLOTH_TORCH_INDEX_FAMILY override naming the index leaf (cu128,
cu130, rocm7.2, cpu, ...). The resolved family flows into
UNSLOTH_TORCH_BACKEND, which install_python_stack.py already consumes,
so the whole downstream chain follows the pin. Dockerfile.studio sets
cu128 on amd64 and cu130 on arm64 (DGX Spark / Grace).
Base image (docker/Dockerfile):
- Install JupyterLab + notebook + ipywidgets in a separate pure-Python uv
pass so the cu128 pin set cannot move; EXPOSE 8888.
- Bake the prebuilt llama.cpp bundle into /opt/unsloth/llama.cpp at the
runtime stage using studio/install_llama_prebuilt.py from the same
UNSLOTH_REF (sha256-verified, portable CUDA bundle since the build host
has no GPU; arm64 resolves the linux-arm64-cuda13 bundle). Export
UNSLOTH_LLAMA_CPP_PATH so unsloth_zoo's save_pretrained_gguf finds it
and never reaches the interactive install prompt or a source build.
- Optional github_token BuildKit secret for the resolver's API calls on
shared CI runner IPs.
Entrypoint: UNSLOTH_ALLOW_CPU=1 degrades a missing GPU to a warning so
Docker Desktop on macOS / Windows-without-WSL2-GPU and plain CPU hosts can
run Jupyter, GGUF tooling and Studio chat; with a GPU visible the normal
pre-flight still runs.
Full image (docker/Dockerfile.studio): now mirrors the production service
set under supervisord - Studio on 8000, JupyterLab on 8888, key-only sshd
on 22 (enabled only when PUBLIC_KEY/SSH_KEY is set). Points Studio's
llama.cpp dir at the baked bundle to skip a duplicate download, accepts
any git ref via fetch+checkout (CI passes commit SHAs), and FROMs a
digest-pinned BASE_IMAGE.
Publish workflow: base image moves to the base-* tag namespace; new
build-studio/merge-studio jobs publish the full image as :latest (hub
parity with the previous production image, which shipped Studio + Jupyter
+ SSH). Studio builds FROM the exact base manifest digest published by the
same run. GPU smoke job now also boots the full image and probes Studio
/api/health and Jupyter /api.
run.sh: UNSLOTH_GPUS=none, UNSLOTH_ALLOW_CPU forwarding, UNSLOTH_PORTS
publish flags, CPU-mode and Jupyter usage examples.
Round-2 of the 12-persona reviewer.py pass found 17 issues. Address the
P1s + the regression-class P2s in this commit; the remaining nits are
left for a follow-up cleanup pass.
1. unsloth/_gpu_init.py: the `NVIDIA_VISIBLE_DEVICES in os.environ` check
triggered for every NVIDIA-runtime container including `--gpus all`
(NVIDIA_VISIBLE_DEVICES=all is the default). Gate strictly on a
non-special device list. Also drop the precondition that the env var
was absent: if the user already pinned TORCHINDUCTOR_COMPILE_THREADS=1
we should still plant the UNSLOTH_FORCE_SINGLE_COMPILE_WORKER sentinel
so the zoo-side patch knows to preserve the forcing.
2. unsloth/_gpu_init.py: after the post-`import unsloth_zoo` reassertion,
monkey-patch `unsloth_zoo.temporary_patches.common.determine_compile_threads`
to return 1, so any later `torch.compile` call that rebuilds the
options dict still sees the single-worker forcing even if a downstream
patch_torch_compile pops the env var again.
3. docker/Dockerfile: torchaudio==2.11.0 mismatched the torch==2.10.0
release pairing; pin to 2.10.0 so the ABI is correct and the audio
stack matches torch/cu128.
4. docker/Dockerfile: drop `12.1+PTX` from TORCH_CUDA_ARCH_LIST. The
cu128 toolkit compiler does not know about compute_121; the trailing
PTX entry forced nvcc to emit a `sm_121` gencode that breaks any
in-container source builds.
5. docker/smoke_test.py: the device-capability floor said `cap[0] < 8`,
rejecting Turing (sm_75) while the Dockerfile + entrypoint advertise
sm_75 as supported. Lower the smoke floor to sm_75 and print a hint
that bf16 is not available on Turing.
6. docker/run.sh: `-it` is unconditional; CI / non-TTY invocations died
with "the input device is not a TTY". Probe `[ -t 0 ] && [ -t 1 ]`
first. Also remove `set -x` which echoed the forwarded HF_TOKEN /
WANDB_API_KEY / UNSLOTH_LICENSE values to stdout.
7. docker/test_locally.sh: `-e HF_TOKEN="${HF_TOKEN:-}"` either pasted
the secret verbatim into the process arg list or shadowed any
in-container value with an empty string. Forward conditionally.
8. .github/workflows/docker-publish.yml: gate `latest` on default branch
AND on `unsloth_ref` not being overridden via workflow_dispatch.
Otherwise a maintainer testing a feature SHA from main could overwrite
`:latest` with non-main source.
9. docker/Dockerfile.studio: add an `UNSLOTH_STUDIO_REF` build-arg so
the Studio companion image is pinned to a known unsloth ref instead
of cloning `main` whenever it builds.
install.sh --local installs unsloth into the Studio venv as an editable
package keyed to the just-cloned source tree. We were rm-rf'ing that
tree in the same RUN; the resulting `unsloth_cli` import then failed at
container start with `ModuleNotFoundError: No module named 'unsloth_cli'`.
Clone the source directly under UNSLOTH_STUDIO_HOME/src so it persists
in the image layer, and strip only .git to save ~120MB.
The base unsloth-blackwell image ships the `unsloth` CLI but refuses to
start `unsloth studio` until the dedicated Studio venv is laid down under
UNSLOTH_STUDIO_HOME by install.sh. Build it once and commit the result as
an opt-in companion tag (`:studio`) instead of bloating the base image.
Build:
docker buildx build --build-arg BASE_TAG=test \
-f docker/Dockerfile.studio -t unsloth-blackwell:studio docker/
Run:
docker run --rm --gpus '"device=0"' -p 8888:8888 unsloth-blackwell:studio
Open http://localhost:8888. Inference (llama.cpp CPU + GPU) and training
are both available. First-boot admin password lands in container logs
and at /opt/unsloth-studio/auth/.bootstrap_password.
