Stacks a Colab-like JupyterLab and Studio experience on top of the
existing Blackwell image. Additive only: the training stack, CUDA/torch
pinning, and the Studio/JupyterLab/sshd service trio are unchanged.
JupyterLab labextension (prebuilt in a throwaway builder stage, so the
runtime image stays Node-free):
- Unsloth Dark (Monokai) theme, adaptive light/dark by system preference
- Colab-style ArrowDown/Up cell navigation
- top-bar Unsloth logo (stock Jupyter logo disabled and locked)
- #@title lines render as collapsible Heading-2 form bars
- Ctrl+A in a cell output selects only that output, not the whole
notebook (the old behaviour ran notebook:select-all and was laggy)
- right activity bar hidden by default
- overrides.json: per-cell run button without auto-advance, labeled
Restart and Run All, windowing off so collapsing an output does not
snap to the cell top, news/update prompts suppressed
Studio and login branding: Unsloth favicon, page logo, and a dark
Unsloth login page that rotates through the curated Studio sloth
stickers (fail-soft to the logo).
Notebook organization and Colab compatibility (base image):
- categorized folder view built from relative symlinks mirroring the
README sections, rebuilt each boot; real .ipynb files never moved,
and the symlink tree is invisible to the sync state machine
- AMD-* notebooks shown only on an AMD/HIP host (autodetected)
- Docker-only strip of the Colab "Run all on Colab" intro sentence
from unedited notebooks (upstream notebooks unchanged)
- hoist %%capture above a leading #@title form so the cell runs
- the per-cell transformers-sidecar log is silent unless
UNSLOTH_ENABLE_LOGGING=1
Dependency pinning and naming: the curated notebook extras are pinned to
their resolved versions for reproducible rebuilds; decord is split into
its own fail-soft install (no aarch64 wheel). The lean base image is
renamed from :base to :core.
Adds tests/validate_studio_features.py, a static self-test for the
labextension plugins, overrides keys, and branding wiring.
Mirror the public-link convenience Studio already has for its own UI, for
JupyterLab. Off by default; opt in two ways:
docker run -e UNSLOTH_JUPYTER_CLOUDFLARE=1 ... unsloth/unsloth
docker exec <container> unsloth-jupyter-tunnel --force
unsloth-jupyter-tunnel waits for JupyterLab, reuses a cached cloudflared (or
fetches the static binary for the arch, no account needed), and starts a
quick tunnel to the Jupyter port; the https://<name>.trycloudflare.com URL is
printed to docker logs. supervisord runs it as the jupyter-cloudflare program,
autostarted only when UNSLOTH_JUPYTER_CLOUDFLARE=1 (studio_launch.sh exports a
0 default so the autostart gate expands, matching the sshd pattern). JupyterLab
still enforces its password, so the tunnel is not an open door.
Verified: the helper fetches cloudflared and mints a working trycloudflare URL
that reaches JupyterLab (HTTP 200) inside a running container.
Parity with unsloth-studio-update: update the baked llama.cpp prebuilt in a
running container without pulling a new image.
docker exec <container> unsloth-llama-update # latest release
docker exec <container> unsloth-llama-update --check # report only
It reuses the build-time fetcher (fetch_llama_prebuilt.py, now baked at
/usr/local/lib/unsloth) rather than the host-probing installer behind the
in-app banner. The fetcher resolves the latest release via the GitHub
/releases/latest redirect (no API token, not rate-limited) and installs the
portable CUDA bundle that runs on CPU and every supported GPU, so it works the
same in a CPU-only or a --gpus container. The installer path, by contrast,
scans the GitHub API (rate-limited to 403 in practice) and probes the host GPU,
which falls back to a slow source build in a container started without --gpus.
The fetch lands in a sibling temp dir on the same filesystem and is swapped in
with an atomic rename; on any failure the existing install is left untouched.
The Studio ownership marker is preserved across the swap. Verified end to end
in a CPU-only container: b9596-mix-e6f2453 -> b9773-mix-1f1aaa4.
Updating Studio in the container previously meant pulling a fresh ~25GB image
(or at best the ~6GB fused Studio layer) for what is usually a small Python/UI
change. Add a baked helper so a running container can update in place:
docker exec <container> unsloth-studio-update
It updates only the Studio packages -- the backend code and the pre-built
frontend, which ships inside the unsloth wheel -- with `pip install -U
--no-deps unsloth unsloth_zoo`, then restarts just the studio service via
supervisor. The torch/CUDA stack is left untouched, so it is safe in both GPU
and CPU-only containers. This deliberately avoids `unsloth studio update`,
which re-runs the full installer and re-probes the GPU to pick torch wheels --
in a container started without --gpus that finds no GPU and can downgrade torch
to CPU/cu126.
Options: --ref <branch|tag|sha> installs from git (track main) instead of the
latest PyPI release; --with-deps also updates dependencies; --no-restart defers
the restart. After the swap the helper smoke-imports studio.backend.main and,
if a transitive dep is now missing, points the user at --with-deps.
The update lands in the container's writable layer (survives docker restart);
mount -v unsloth_studio_home:/opt/unsloth-studio to keep it across a recreate.
Two related changes to the baked llama.cpp prebuilt.
1. Dynamically follow the newest unslothai/llama.cpp release. build.sh resolves
the latest release tag (following the /releases/latest redirect, no API
token) to a concrete tag and passes it as LLAMA_PREBUILT_TAG, so the layer
cache busts only when upstream publishes. The Dockerfile default is now
"latest" and fetch_llama_prebuilt.py resolves it the same way, so a plain
`docker build .` also tracks latest. Pin LLAMA_PREBUILT_TAG to a concrete
tag for a reproducible, frozen build.
2. Make the in-app "newer llama.cpp available" banner work inside the image.
Studio's freshness check (utils.llama_cpp_freshness.check_prebuilt_freshness)
keys off tag / release_tag / published_repo in UNSLOTH_PREBUILT_INFO.json --
the schema install_llama_prebuilt.py writes. The image bakes the bundle
directly, so the marker was the release tarball's own, which only carries
upstream_tag / source_repo; the freshness check then bailed with
installed_tag=None and could never report "behind", hiding the banner.
fetch_llama_prebuilt.py now augments the baked marker with those keys
(setdefault, no build timestamp so the layer stays byte-identical). A fresh
build is on latest -> no banner; once upstream publishes a newer release the
banner appears, as verified against the real freshness backend.
The boot-time notebook sync now restores notebooks the user deleted, on
every boot, from the baked template (offline, even when upstream has not
advanced). It only restores files that are missing, so it never resurrects
or overwrites an edited notebook, and the GitHub refresh still bumps a
restored file to the latest upstream. Opt out with
UNSLOTH_KEEP_DELETED_NOTEBOOKS=1.
Also add unsloth_nb_content_sig.py to docker/.dockerignore's allowlist; it
was referenced by the Dockerfile COPY but excluded from the build context,
which broke the image build.
The boot-time refresh now compares only the tutorial body (the
non-boilerplate cells) when deciding whether to update an untouched
notebook. If only the install header, announcements, or footer moved
upstream, the user's file is left as-is so it is not churned. Notebooks
the user has edited or run are still kept untouched, and non-notebook
files keep the whole-file refresh. Adds unsloth_nb_content_sig.py to
segment head/middle/tail and bakes it into the image.
JupyterLab now opens with the unslothai/notebooks collection already present,
so people can open and run a notebook directly without a git clone or wget.
- Bake the repo into the image as a read-only template at /opt/unsloth-notebooks
(~206MB, .git stripped, build commit recorded). Inherited by the studio image.
- On boot the entrypoint populates /workspace/unsloth-notebooks from the template
(instant, works offline) and best-effort refreshes from GitHub, but only when
upstream has actually advanced (cheap git ls-remote gate, no download otherwise).
- The user's edits always win. We record the content hash of every file we write;
on refresh a file whose hash differs from what we last wrote is treated as
user-modified and is left untouched, so the refresh only updates files the user
has not changed and adds new ones. It never overwrites an edited notebook and
never produces merge conflicts. Verified: an edited notebook stays the user's
version across repeated upstream changes.
- Fully best-effort and gated: UNSLOTH_SKIP_NOTEBOOK_SYNC=1 disables it,
UNSLOTH_SKIP_NOTEBOOK_REFRESH=1 keeps the baked copy and never hits the network.
Offline boots keep what is there and never error.
base 18.45 -> 18.67GB, studio 24.88 -> 25.10GB (+~206MB baked notebooks).
Notebook dependency coverage (base Dockerfile):
- Bake omegaconf, einx, librosa, decord, ftfy so the TTS/STT and vision
notebooks stop dying on a silent No module named X. Installed in the
notebook-deps layer (after the torch/vLLM resolve) with an assertion that
the resolve did not move torch 2.10.0 / numpy>=2.3 / numba>=0.65.
Image size (no functional change):
- Base: prune npp to the two libs torchcodec actually dlopens
(libnppicc + libnppc), drop link-time-only .a archives and the nvshmem
device bitcode. Headers (torch/include etc) are kept so causal-conv1d /
mamba-ssm still build at notebook time with --no-build-isolation.
- Studio: pin the Studio venv to Python 3.12 (matches base) so its
nvidia-*-cu12 wheels are byte-identical to the base venv's, then symlink
the heavy arch-independent CUDA libs (cudnn/cublas/nccl/...) into the base
venv copy. cuda_nvrtc and cuda_runtime are excluded (the arm64 nvrtc swap
mutates nvrtc in place). Also remove the build-only frontend node_modules
(runtime serves the committed dist). Studio image drops ~4.8GB.
Per-notebook transformers version, run notebooks unchanged:
- Bake coherent transformers sidecars (4.57.6 default + 5.3.0/5.5.0/5.10.2),
each transformers==X with its matched huggingface_hub/tokenizers/
safetensors installed --no-deps into its own dir. Companion versions are
resolved at build time so they satisfy each transformers' requirements.
- unsloth_nb_compat.py: pick the sidecar from the notebook's pin or the
model name and activate it (prepend to sys.path) before any ML import,
without touching the base cu128 torch/vLLM/unsloth stack.
- pip/uv shim on PATH: a notebook install cell becomes safe and idempotent
inside a kernel (keeps the baked stack, records the requested transformers
for its sidecar); passthrough to the real tool everywhere else.
- IPython startup hook for manual JupyterLab, and unsloth-run for the
headless driven path.
Continuation of the notebook-dep prebaking: the in-image notebook runner
neutralises pip cells, so declared deps must be prebaked. evaluate's WER
metric imports jiwer (Whisper), DeepSeek-R1 GRPO's reward uses langid,
some vision trust_remote_code files need easydict, and sentencepiece
tokenizer conversion needs protobuf. All pure-Python; torch pin intact.
The TTS notebooks (Sesame CSM, Orpheus) read audio via soundfile, the
Whisper notebook computes WER via evaluate, and TrainingArguments
defaults report_to to tensorboard. These are declared by notebook pip
cells that the in-image notebook runner neutralises (deps are meant to be
prebaked), so without them those notebooks die on import. All are
pure-Python or self-contained wheels and never name torch, so the cu128
pin set is undisturbed.
flash-linear-attention's TileLang backend JIT-compiles CUDA kernels via
nvcc at runtime for gated-delta-rule models (Qwen3.5 family). The -base
image only ships runtime libraries, so Studio vision training of
unsloth/Qwen3.5-2B died on the first backward pass with
[Errno 2] No such file or directory: /usr/local/cuda/bin/nvcc.
Install cuda-nvcc and cuda-cudart-dev matching the image CUDA version
and assert nvcc is executable at build time. Found by driving a real
Qwen3.5-2B training run through the Studio UI in the image.
A locally built tag (test_locally.sh or docker build) is not on a
registry, so the pull phase reported hard failures on a machine that was
actually fine. Degrade to a warn when the image is present locally;
missing images still fail.
llama-quantize exits nonzero on --help/--version while still printing
usage, so a bare invocation fails the build even when the binary is
healthy. Grep for the usage banner instead; a loader failure prints
error while loading shared libraries and no usage text.
The build/bin hardlink mirror skipped symlinks, so the soname links
(libllama-common.so.0 and friends) never reached build/bin. Studio's
setup.sh relinks the root llama-quantize to build/bin/llama-quantize,
whose RUNPATH is $ORIGIN, so the loader failed with libllama-common.so.0
not found and GGUF export from Studio died with No working quantizer
found, then hit the interactive source-build prompt in a non-TTY export
subprocess (EOFError). Mirror same-directory soname symlinks into
build/bin and extend the bake sanity check to execute llama-quantize from
both the install root and build/bin. Dockerfile.studio now also runs the
studio-visible quantizer after install.sh so a regression fails the
image build instead of runtime exports.
_detect_cuda_torch_index_url now respects the explicit family override
before probing nvidia-smi, matching install.sh get_torch_index_url and
install.ps1 Get-TorchIndexUrl. Without it, a GPU-less environment falls
back to cu126 wheels which lack sm_100/sm_120 kernels and break training
on Blackwell. ROCm repair path is intentionally unchanged.
Dockerfile.studio now fails the build if the Studio venv torch local
version tag does not match the pinned TORCH_FAMILY, so a studio ref whose
installer ignores the override can never ship a silently wrong image.
Metadata-only check so QEMU arm64 builds do not need to load torch.
The notebook validation matrix caught the synthetic-data notebook dying
because the vllm server SyntheticDataKit launches never came up. Two
layers to the failure:
1. flashinfer's cpp_ext JIT shells out to ninja. The pip ninja lives in
the venv bin, which subprocesses like vllm serve do not always
inherit on PATH, so the JIT failed with exit 127. Install ninja-build
so the binary is reachable from any PATH.
2. With ninja present the JIT still cannot succeed for device code: the
runtime image deliberately ships no nvcc. Bake flashinfer-jit-cache
(cu128) so ops missing from the cubin package (fmha_gen on sm_100a
was the repro) come precompiled. In-process GRPO never hit this
because unsloth-zoo blocks the FlashInfer JIT path; standalone
vllm serve gets no zoo patches.
Fail-soft on the jit-cache for arches without a wheel; the vLLM chain
itself stays fail-loud on amd64.
Notebooks fetch sample assets with !wget; without the binary the shell
prints not-found to stderr, the cell still exits zero from Jupyter's
perspective, and the next cell crashes confusingly on the missing file.
The Whisper notebook died exactly this way in the validation matrix.
PyPI has shipped aarch64 abi3 wheels for every vLLM release since 0.17,
so the arm64 skip rested on a stale premise. With torch held at 2.10.0
the resolver lands on vllm 0.19.1 (the release pinning torch==2.10.0)
on both arches; verified by cross-resolving the exact index set for
aarch64-unknown-linux-gnu.
amd64 keeps fail-loud semantics. arm64 is fail-soft because the aarch64
wheels are newer and their GPU kernels get validated on Spark hardware
via docker_confirm.sh rather than in CI; on failure the fallback
uninstalls vllm and restores the numpy/numba floor so a partial install
cannot break import unsloth (numpy 2.2.6 ships a broken numpy.testing).
The install steps form an explicit && chain instead of a set -e
subshell: POSIX shells disable errexit inside condition contexts
(verified on dash), so a (set -e; ...) condition would mask failures.
Both confirm scripts gain a 5b vLLM phase: ok on import, bad if missing
on x86_64, warn on other arches where fast_inference=True is best-effort.
The wheel install belongs in the builder (the venv copy carries it),
but the ld.so.conf.d registration and the import check belong in the
runtime stage: the conf file does not survive the stage copy and the
import needs ffmpeg, which only the runtime stage installs.
The TTS/STT notebooks decode datasets Audio features through torchcodec,
which fails three different ways on a fresh image: the PyPI wheel pairs
with the cu13 torch line and dlopens libnvrtc.so.13; builds newer than
0.10 reference torch 2.11+ symbols; and the matching +cu128 build dlopens
torch and NVIDIA runtime libraries that live inside the venv where the
loader cannot see them. Bake ffmpeg, torchcodec==0.10.0 from the cu128
channel, nvidia-npp-cu12, and register the venv lib dirs via ld.so.conf.d
(not LD_LIBRARY_PATH, so the llama.cpp bundle keeps winning through its
own RUNPATH). Verified in-container: AudioDecoder imports and llama-server
still resolves its bundled libraries.
Running the published unslothai/notebooks set inside the image surfaced
two gaps: the Ollama export notebook installs ollama in-container and
that installer needs zstd for extraction, and DeepSeek-OCR's
trust_remote_code modeling file imports matplotlib unconditionally
(plotting is also simply expected in a Jupyter image).
- Dockerfile: lift numba past vllm's 0.61.2 pin after the numpy>=2.4
re-upgrade; 0.61.2 refuses numpy 2.3+ at import time and the stack
cannot move numpy down. Verified numba 0.65 + numpy 2.4.6 + vllm
import cleanly together.
- docker-publish.yml: resolve UNSLOTH_ZOO_REF in a step that mirrors
the pushed tag only when the tag exists in unsloth-zoo (the zoo
currently cuts no tags, so blind mirroring broke every tag publish);
falls back to main.
- Dockerfile.studio: Studio venv stays on cu128 for arm64 too, matching
the base venv (cu130 wheels would lift the driver floor to 580+), and
gets the same NVRTC cu13 swap for DGX Spark / GB10 sm_121 support.
- docker_confirm.sh: do not drop to CPU mode when docker info lacks a
nvidia runtime entry; CDI installs and Docker Desktop WSL2 expose
GPUs without one. The phase 3 --gpus probe is now the authority.
- docker_confirm.ps1: GPU selector built as an args array; comma device
lists get version-aware CSV quoting (native arg passing changed in
PowerShell 7.3).
- studio_launch.sh: no fixed Jupyter default password; generate a
random one and print it when JUPYTER_PASSWORD is unset. Env snapshot
for SSH sessions now written via shlex.quote instead of sed so
values with quotes or command substitution cannot break or inject
into /etc/profile.d.
- install.ps1: honour UNSLOTH_TORCH_INDEX_FAMILY like install.sh does.
entrypoint.sh: a container started without a GPU request has no
nvidia-smi at all (the toolkit injects it), so the old check 1 reported
'CUDA runtime in this image is broken, re-pull' for the most common user
error. Fold the missing-binary case into the actionable 'No GPU visible'
message and document the CPU-only option (UNSLOTH_ALLOW_CPU=1).
run.sh / test_locally.sh: guard empty-array expansions with the
${arr[@]+...} form; bash 3.2 (macOS /bin/bash) treats "${empty[@]}"
as unbound under set -u, which broke the documented macOS CPU path.
studio_launch.sh: exclude *_TOKEN, *_API_KEY, *_PASSWORD, *_SECRET,
*_LICENSE from the env snapshot written for SSH sessions; secrets stay
in process env only, never on disk.
supervisord.conf / Dockerfile.studio: pin HOME=/root for the studio and
jupyter programs (jupyter would silently fall back to token auth if HOME
were unset), default JUPYTER_PORT and UNSLOTH_ENABLE_SSHD at the image
level so a direct supervisord invocation cannot hit a bad %(ENV_*)s
expansion, and document the root-services decision (non-root parity with
the previous production image is a tracked follow-up).
docker_confirm.ps1: mirror the bash script's GPU selector translation so
GPUS=0 / 0,1 select devices instead of silently using all GPUs.
docker-publish.yml: studio cache scope moves to mode=min; a mode=max
cache of a ~24GB image would evict everything else in the 10GB GHA
quota for no hit-rate gain.
Studio's setup.sh provisioning runs install_llama_prebuilt.py, whose
host-probing cannot succeed inside an image build, so it fell back to a
CPU-only llama.cpp source build layered over the baked CUDA bundle.
setup.sh skips that fallback when build/bin/llama-server and
build/bin/llama-quantize are executable, so hardlink the installed bundle
into build/bin: zero extra bytes, $ORIGIN rpath still resolves, and no
symlink cycle when setup.sh later relinks the root quantizer to
build/bin/llama-quantize.
Two failures from the first in-image Studio install, both rooted in
install.sh probing the build host:
1. setup.sh aborted on the pre-linked llama.cpp dir: 'already exists and
is not marked as a Studio-owned llama.cpp install'. The dir is the
image's baked prebuilt, provisioned exclusively for Studio, so write
the .unsloth-studio-owned marker next to the binaries.
2. With no GPU and no nvidia-smi in the build container, install.sh fell
back to cu126 torch wheels for the Studio venv (and would pick cpu
wheels on a CI runner without /proc/driver/nvidia), so the published
image's Studio venv would depend on which host built it and could not
train on Blackwell. get_torch_index_url now honours an explicit
UNSLOTH_TORCH_INDEX_FAMILY override naming the index leaf (cu128,
cu130, rocm7.2, cpu, ...). The resolved family flows into
UNSLOTH_TORCH_BACKEND, which install_python_stack.py already consumes,
so the whole downstream chain follows the pin. Dockerfile.studio sets
cu128 on amd64 and cu130 on arm64 (DGX Spark / Grace).
The dockerignore uses an everything-out whitelist; fetch_llama_prebuilt.py
(base bake) and supervisord.conf + studio_launch.sh (Dockerfile.studio)
need explicit entries. docker_confirm.ps1 is the Windows Docker Desktop
counterpart of docker_confirm.sh.
The first bake attempt reused studio/install_llama_prebuilt.py, but that
resolver selects a bundle for the CURRENT host: on a GPU build host
/proc/driver/nvidia leaks into docker build and the resolver goes down the
CUDA path with no readable driver runtime (chosen_asset=none, exit 2),
while on a GPU-less CI runner it would resolve a CPU bundle instead. Both
violate the image's build-host-independence rule.
fetch_llama_prebuilt.py pins by build target only: amd64 takes the
linux-x64-cuda12-portable bundle, arm64 the linux-arm64-cuda13-portable
bundle (DGX Spark / Grace), both sha256-verified against the release's
llama-prebuilt-sha256.json. convert_hf_to_gguf.py plus gguf-py/ are
hydrated from the same release's source tarball so the converter's tensor
mappings match the binaries, mirroring unsloth_zoo's
_hydrate_converter_sources layout. LLAMA_PREBUILT_TAG build-arg overrides
the pinned release.
docker_confirm.sh: one-command confirmation script for any machine
(Linux / WSL2 / macOS) following the staging confirm-script conventions:
host + docker + GPU detection with CPU-mode auto-fallback, image pulls,
in-container torch.cuda check, 5-step LoRA training smoke, baked llama.cpp
verification, full-image boot probing Studio /api/health and JupyterLab
/api, PASS/WARN/FAIL summary with RESULT line.
Base image (docker/Dockerfile):
- Install JupyterLab + notebook + ipywidgets in a separate pure-Python uv
pass so the cu128 pin set cannot move; EXPOSE 8888.
- Bake the prebuilt llama.cpp bundle into /opt/unsloth/llama.cpp at the
runtime stage using studio/install_llama_prebuilt.py from the same
UNSLOTH_REF (sha256-verified, portable CUDA bundle since the build host
has no GPU; arm64 resolves the linux-arm64-cuda13 bundle). Export
UNSLOTH_LLAMA_CPP_PATH so unsloth_zoo's save_pretrained_gguf finds it
and never reaches the interactive install prompt or a source build.
- Optional github_token BuildKit secret for the resolver's API calls on
shared CI runner IPs.
Entrypoint: UNSLOTH_ALLOW_CPU=1 degrades a missing GPU to a warning so
Docker Desktop on macOS / Windows-without-WSL2-GPU and plain CPU hosts can
run Jupyter, GGUF tooling and Studio chat; with a GPU visible the normal
pre-flight still runs.
Full image (docker/Dockerfile.studio): now mirrors the production service
set under supervisord - Studio on 8000, JupyterLab on 8888, key-only sshd
on 22 (enabled only when PUBLIC_KEY/SSH_KEY is set). Points Studio's
llama.cpp dir at the baked bundle to skip a duplicate download, accepts
any git ref via fetch+checkout (CI passes commit SHAs), and FROMs a
digest-pinned BASE_IMAGE.
Publish workflow: base image moves to the base-* tag namespace; new
build-studio/merge-studio jobs publish the full image as :latest (hub
parity with the previous production image, which shipped Studio + Jupyter
+ SSH). Studio builds FROM the exact base manifest digest published by the
same run. GPU smoke job now also boots the full image and probes Studio
/api/health and Jupyter /api.
run.sh: UNSLOTH_GPUS=none, UNSLOTH_ALLOW_CPU forwarding, UNSLOTH_PORTS
publish flags, CPU-mode and Jupyter usage examples.
Two small fixes:
1. The fallback build-context refresh used `git pull --ff-only | tail`,
which on this script (set -uo pipefail, no -e) silently masked any
non-zero exit from pull. A failed refresh would then quietly build
from a stale clone. Wrap both clone and pull in `if ! ...; then fail`
so refresh failures abort the run with a clear message.
2. The gpt-oss-20B notebook was fetched from notebooks/main, which is
mutable. Pin to the current immutable SHA (efe20c9) via NB_REPO_REF
so reruns of this script don't silently change semantics when
notebooks/main rolls forward. Override via env when you want to
verify a newer notebook.
The header docstring advertises UNSLOTH_GPUS values like "0" and "0,1"
but Docker reads a bare integer for --gpus as a COUNT, not an INDEX.
UNSLOTH_GPUS=0 was therefore exposing zero GPUs, and the entrypoint's
GPU check refused to start. Wrap bare-int and comma-list inputs as
"device=$GPUS" so the documented values do what they say; "all" and
already-quoted device= selectors pass through unchanged.
- docker-publish.yml: add `concurrency: docker-publish-${{ github.ref }}`
(cancel-in-progress: false) so two pushes to main never race the
`:latest` retag. Don't cancel in-progress runs -- the build is
expensive and a half-built image left around is worse than a stale
:latest for a few minutes.
- Dockerfile: soften the requirements.lock.txt comment. `pip freeze`
captures versions but not wheel hashes, and several deps resolve
from VCS / nightly indexes that float, so the file is not actually
byte-reproducible. Reword as an "informational pin record".
1. Stop leaking secrets via docker run -e VAR=VALUE argv (run.sh, test_locally.sh)
`docker run ... -e HF_TOKEN=hf_xxx ...` puts the literal token in
the docker CLI's argv, which is visible to any user on the host
via `ps auxe` / `/proc/<pid>/cmdline` for the lifetime of the
process. Switch to the dash-only form `-e HF_TOKEN`, which tells
docker to read the value from the parent shell's env and never
appears in argv. Same fix for WANDB_API_KEY and UNSLOTH_LICENSE in
run.sh and HF_TOKEN in test_locally.sh.
2. Stop stripping numpy/tests/ in the runtime layer (Dockerfile)
The Dockerfile explicitly upgrades numpy >= 2.4 because numpy 2.2.6
shipped a stripped wheel where `from numpy._core.tests._natype
import pd_NA` fails. Numpy 2.4 restores `numpy/_core/tests/`, then
the existing `find ${VENV} -name tests -exec rm -rf {} +` deleted
it again -- re-introducing the same broken-import state on the
deployed image (the build-time verification at line 220 runs
BEFORE the strip so it passed). Whitelist numpy's tests directories
from the strip; keep stripping the rest.
3. Align :latest tag gate between merge and smoke-test jobs
(.github/workflows/docker-publish.yml)
merge job: enable = is-default-branch AND unsloth_ref == ''
smoke-test job: enable = is_default_branch only
On `workflow_dispatch`, `github.event.inputs.unsloth_ref` defaults to
"main" (not ""), so the merge step skipped `:latest` but the smoke
step still emitted `:latest` as tags[0]. The smoke step then
`docker pull`-ed a prior `:latest` from Docker Hub instead of the
image just merged -- so the smoke test verified the OLD image, not
the new one. Copy the merge step's exact `enable=` expression into
the smoke-test step so the two stay byte-identical and a workflow_
dispatch run validates whatever was actually merged.
Round-2 of the 12-persona reviewer.py pass found 17 issues. Address the
P1s + the regression-class P2s in this commit; the remaining nits are
left for a follow-up cleanup pass.
1. unsloth/_gpu_init.py: the `NVIDIA_VISIBLE_DEVICES in os.environ` check
triggered for every NVIDIA-runtime container including `--gpus all`
(NVIDIA_VISIBLE_DEVICES=all is the default). Gate strictly on a
non-special device list. Also drop the precondition that the env var
was absent: if the user already pinned TORCHINDUCTOR_COMPILE_THREADS=1
we should still plant the UNSLOTH_FORCE_SINGLE_COMPILE_WORKER sentinel
so the zoo-side patch knows to preserve the forcing.
2. unsloth/_gpu_init.py: after the post-`import unsloth_zoo` reassertion,
monkey-patch `unsloth_zoo.temporary_patches.common.determine_compile_threads`
to return 1, so any later `torch.compile` call that rebuilds the
options dict still sees the single-worker forcing even if a downstream
patch_torch_compile pops the env var again.
3. docker/Dockerfile: torchaudio==2.11.0 mismatched the torch==2.10.0
release pairing; pin to 2.10.0 so the ABI is correct and the audio
stack matches torch/cu128.
4. docker/Dockerfile: drop `12.1+PTX` from TORCH_CUDA_ARCH_LIST. The
cu128 toolkit compiler does not know about compute_121; the trailing
PTX entry forced nvcc to emit a `sm_121` gencode that breaks any
in-container source builds.
5. docker/smoke_test.py: the device-capability floor said `cap[0] < 8`,
rejecting Turing (sm_75) while the Dockerfile + entrypoint advertise
sm_75 as supported. Lower the smoke floor to sm_75 and print a hint
that bf16 is not available on Turing.
6. docker/run.sh: `-it` is unconditional; CI / non-TTY invocations died
with "the input device is not a TTY". Probe `[ -t 0 ] && [ -t 1 ]`
first. Also remove `set -x` which echoed the forwarded HF_TOKEN /
WANDB_API_KEY / UNSLOTH_LICENSE values to stdout.
7. docker/test_locally.sh: `-e HF_TOKEN="${HF_TOKEN:-}"` either pasted
the secret verbatim into the process arg list or shadowed any
in-container value with an empty string. Forward conditionally.
8. .github/workflows/docker-publish.yml: gate `latest` on default branch
AND on `unsloth_ref` not being overridden via workflow_dispatch.
Otherwise a maintainer testing a feature SHA from main could overwrite
`:latest` with non-main source.
9. docker/Dockerfile.studio: add an `UNSLOTH_STUDIO_REF` build-arg so
the Studio companion image is pinned to a known unsloth ref instead
of cloning `main` whenever it builds.
Two vision-notebook deps that ship by reference rather than via unsloth
extras: transformers' Gemma3N + TimmWrapperModel needs `timm`, and
DeepSeek-OCR's dynamic modeling file requires `addict`. Both are tiny
(~30MB combined). Including them in the base unified resolve avoids
hitting `ImportError: TimmWrapperModel requires the timm library`
or `ImportError: This modeling file requires the following packages
that were not found in your environment: addict` after the user has
already downloaded the model.
Repros: nb/Gemma3N_(4B)-Vision.ipynb (timm), nb/Deepseek_OCR_(3B).ipynb
(addict).
install.sh --local installs unsloth into the Studio venv as an editable
package keyed to the just-cloned source tree. We were rm-rf'ing that
tree in the same RUN; the resulting `unsloth_cli` import then failed at
container start with `ModuleNotFoundError: No module named 'unsloth_cli'`.
Clone the source directly under UNSLOTH_STUDIO_HOME/src so it persists
in the image layer, and strip only .git to save ~120MB.
Round-trip with the reviewer.py 12-persona pass surfaced four real
issues. Fix all four in this PR so the new Docker release path is
self-consistent.
1. docker/smoke_test.py used `import xformers` unconditionally, which
guarantees a failure on arm64 (built with `[huggingface]` extras to
skip xformers since it has no aarch64 cu128 wheel). Wrap the import
in try/except so the same smoke script validates both arches.
2. unsloth/_gpu_init.py forced `TORCHINDUCTOR_COMPILE_THREADS=1` before
`import unsloth_zoo`, but `patch_torch_compile` in unsloth_zoo main
pops that env var in non-debug mode. After unsloth_zoo init the
guard was effectively undone, so cgroup-pinned `docker --gpus
'"device=N"'` containers still spawned the Inductor subprocess pool
that cannot enumerate the GPU. Set `torch._inductor.config.
compile_threads = 1` directly post-import-torch and re-populate the
env var so `determine_compile_threads()` in the zoo options dict
also returns 1, regardless of whether the zoo-side fix from PR #694
has shipped yet.
3. docker-publish.yml UNSLOTH_REF build-arg defaulted to `'main'` for
tag pushes and scheduled runs, so a `v1.2.3` release image would
contain whatever `main` happened to be at build time, not v1.2.3.
Pick the tag's `github.ref_name` for tag events and `github.sha`
for branch/schedule events.
4. The smoke-test job pulled `:latest` regardless of which tag the
merge job had just published, so tag/schedule/sha publishes were
never actually validated. Re-run docker/metadata-action with the
same config the merge job used, then smoke-test the first tag from
its output.
All four changes are gated and backwards-compatible.
The base unsloth-blackwell image ships the `unsloth` CLI but refuses to
start `unsloth studio` until the dedicated Studio venv is laid down under
UNSLOTH_STUDIO_HOME by install.sh. Build it once and commit the result as
an opt-in companion tag (`:studio`) instead of bloating the base image.
Build:
docker buildx build --build-arg BASE_TAG=test \
-f docker/Dockerfile.studio -t unsloth-blackwell:studio docker/
Run:
docker run --rm --gpus '"device=0"' -p 8888:8888 unsloth-blackwell:studio
Open http://localhost:8888. Inference (llama.cpp CPU + GPU) and training
are both available. First-boot admin password lands in container logs
and at /opt/unsloth-studio/auth/.bootstrap_password.
vLLM 0.19.1 pulls numpy down to 2.2.6 whose wheel ships numpy/_core/
without the tests/ subdir, but numpy/testing/_private/utils.py imports
`from numpy._core.tests._natype import pd_NA`. Anything that hits
`from numpy import *` (scipy._lib.array_api_compat does) then crashes.
unsloth_zoo's gemma patch does `from transformers.processing_utils import
Unpack` which touches that path, so `import unsloth` blew up on every
GRPO notebook in the vLLM image.
Bump numpy to >=2.4 right after the vllm install; vllm still imports
fine on numpy 2.4.6 (verified locally).
The case-arm `auto) [ "${TARGETARCH}" = "amd64" ] && WANT_VLLM=1` exits 1
on arm64 (the [ test ] is false and nothing follows ||), which with
`set -e` aborts the entire RUN. Replace with an explicit if/then/fi so
each arch's auto branch returns 0.
Caught by ubuntu-24.04-arm CI on the staging fork.
Unsloth's GRPO notebooks (Qwen3_4B-GRPO.ipynb, Qwen3_8B_FP8_GRPO.ipynb,
Llama_FP8_GRPO.ipynb, etc.) set `fast_inference=True` which requires
vLLM to be importable in the same venv. Install vllm pre-release wheels
from https://wheels.vllm.ai/nightly alongside the cu128 pytorch index,
holding torch==2.10.0 fixed so uv refuses any vLLM build that would
yank torch out from under unsloth.
amd64 only -- vLLM does not publish aarch64 wheels yet
(vllm-project/vllm#31128 is open). On arm64 the GRPO notebooks that
need fast_inference will fail to import vllm; non-GRPO and
fast_inference=False paths are unaffected.
Gated by ARG INSTALL_VLLM=auto so the install can be disabled for
contributors who want a smaller image or are blocked by vllm/torch
resolve conflicts during iteration.
The nvidia/cuda base image already registers the CUDA apt repo with its own
Signed-By keyring. Installing cuda-keyring_1.1-1_all.deb on top adds a
duplicate sources entry with a different Signed-By value, which makes
`apt-get update` refuse the entire repo:
E: Conflicting values set for option Signed-By regarding source
https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/sbsa/
The repo URL is monolithic (every CUDA version is served from the same
path), so we can install cuda-nvrtc-13-0 + cuda-nvcc-13-0 directly without
touching the keyring. Empirically reproduced on the ubuntu-24.04-arm
GitHub Actions runner (staging-fork CI run 26360461375); fix verified via
the same staging-fork after force-push.
