- unsloth_colab_compat.py: only hoist a leading `%%` cell magic above the Colab
`#@title` form for magics whose body runs as code (capture/time/bash/python/
...). Content magics (%%writefile, %%html, %%latex, ...) are left untouched so
the form comment is never injected into the written file / rendered output.
- outputSelect.ts: stop trusting the text selection anchor to decide ownership
of Ctrl/Cmd+A. A stale selection inside an output survives a click onto a
command-mode cell or the file browser, which made select-all keep re-selecting
the old output. Gate on the keystroke target or the last pointer-down (reset to
null on any click outside an output) instead.
- unsloth_branding.py: also reject page_config.json that disables the Unsloth
labextension or any of its plugin ids via disabledExtensions (dict or list
form); that leaves the bundle on disk so the prior checks passed while the
logo/About/splash attribution was stripped at load. Lock unsloth-jupyterlab in
Dockerfile.studio as well (defense in depth), and add guard tests.
- docker-publish smoke + docker_confirm.sh probe Jupyter /login, not /api: the
launcher always configures a password hash so /api returns 403 and curl -f
would never flip the health flag (false build failure).
- entrypoint.sh CPU messaging: CPU mode covers Jupyter, GGUF tooling and
llama.cpp (GGUF) Studio chat; training AND loading an Unsloth model
(FastLanguageModel) still need a GPU, since from_pretrained runs CUDA probes.
- install_llama_prebuilt.py: rollback/activation moves used bare os.replace,
which fails with EXDEV across overlayfs in a Docker build and fell back to a
broken source build (no nvcc). Add is_cross_device_error + move_install_dir_aside
(os.replace fast path, copy+remove on EXDEV; busy errors still re-raise).
- notebooks: %pip / %uv line magics and the `!python -m pip` form bypassed the
PATH pip/uv shim and could overwrite the baked cu128 torch/vLLM stack. Add
unsloth_nb_pip_magic.py to re-point them at the shim, wired via the IPython
startup hook and installed into the venv site-packages.
Make it obvious the image is built by Unsloth and hard to white-label out with a
shallow find-and-replace, and surface the AGPLv3 license + copyright in the UI.
Visible attribution (labextension):
- Help > "About Unsloth Docker Studio" dialog (about.ts): Unsloth logo, the
AGPLv3 notice, "Copyright 2026-Present the Unsloth team", and source/website/
license links. Added to the Help menu and the command palette.
- The JupyterLab loading splash is replaced with a spinning Unsloth logo
(splash.ts, provides ISplashScreen; honors prefers-reduced-motion). The stock
@jupyterlab/apputils-extension:splash is disabled+locked at build time, like
the stock logo.
- AGPLv3 footer (license + copyright + links) on the branded login page.
- Labextension relicensed AGPL-3.0-only; SPDX headers on every source file.
Anti-tamper (no encoded/obfuscated strings -- plain readable text only; the one
data URI is the logo image):
- A canonical, plain-text attribution set lives in unsloth_branding.py with a
TypeScript mirror (branding.ts) bundled verbatim into the labextension, so the
phrase, copyright, links and plugin ids are spread across independent layers.
- unsloth_branding.py verifies all of these across the installed files (AGPLv3
text, login footer, theme, labextension package + built bundle strings, logo,
favicon) and fails loudly if any are missing. It runs at three layers:
build time (fails the image build), the whole-container launcher
(studio_launch.sh refuses to start), and as a jupyter_server extension
(refuses to serve JupyterLab).
- tests/studio/test_branding_guard.py: positive + per-marker negative coverage,
plus a check that no base64/decoder obfuscation crept into the attribution.
Two more notebook-shim gaps from review:
- A quoted PEP 508 direct reference for a protected package, e.g.
`pip install "torch @ https://.../torch.whl"` or `"unsloth @ git+https://..."`,
bypassed _KEEP: _canon hit the url guard and returned None before pulling the
distribution name, so the token was treated as a real target and reinstalled
into the base venv. _canon now extracts the name from the `name [extras] @ url`
form first, so a protected package pinned through a URL/VCS is still dropped; a
non-protected direct reference returns its name and is kept exactly as before.
- The `--requirement=reqs.txt` equals-form (pip accepts `--option=value` for any
value-taking flag) was not recognized: the token starts with `-`, so it was
kept as an opaque option, the file was never filtered, and has_target stayed
false -- a cell whose only target was that file silently no-op'd. The scan now
splits `--flag=value`, filters the requirements file for `-r`/`--requirement`,
and counts it as a target; other inline-value options stay options.
- unsloth_pip_shim.py: filter protected packages out of a notebook
`pip install -r requirements.txt`. The -r value was passed to the real pip
unchanged, so torch / transformers / vLLM / nvidia pins inside the file could
overwrite the baked cu128 stack or push transformers into the base venv.
_filter_requirements_file() applies the same _KEEP / transformers-sidecar
rules per line, writes the survivors to a temp file, keeps comments, option
lines, nested includes and urls verbatim, and records a pinned transformers
version for the sidecar.
- install.sh + Dockerfile.studio + docker-publish.yml: forward the resolved
unsloth-zoo ref into the Studio build. install.sh --local overlaid
unsloth-zoo from git main regardless of the operator-requested or base-image
ref, so the full image could run a different zoo than the base. install.sh
now honors UNSLOTH_ZOO_REF across all four --local overlays, Dockerfile.studio
passes UNSLOTH_STUDIO_ZOO_REF through to it, and the workflow resolves one zoo
ref in the prepare job and shares it with both the base and Studio builds.
- Dockerfile + docker-publish.yml: pin unslothai/notebooks to one resolved
commit. Each arch leg cloned HEAD independently, so the same tag could seed
different baked templates and .unsloth_template_commit depending on the pulled
platform. The prepare job freezes notebooks to one sha (like the llama.cpp
prebuilt tag) and the Dockerfile fetches that single ref at depth 1.
- pip shim: do not treat the value of an index-url / find-links / constraint flag
as an install target. A cell like 'pip install --extra-index-url <url> torch'
now no-ops after keeping the baked stack instead of exec'ing a bare
'pip install --extra-index-url <url>' that fails. Positional . / url / vcs and
-r/--requirement files still count as targets.
- notebook sync: on first boot, record only files we actually wrote (or that are
byte-identical to the template), never a kept pre-existing user file; and on the
GitHub refresh, treat a file present in DEST but absent from the sync state as
user-owned and keep it. Previously a bind-mounted notebook was recorded as
managed and then overwritten by upstream.
- docker-publish: add flavor latest=false to the Studio metadata steps too, so a
v* tag push cannot emit an implicit :latest via metadata-action's latest=auto;
:latest stays default-branch-only, and the smoke test pulls the published tag.
- unsloth-studio-update: resolve the unsloth-zoo ref independently of --ref (new
--zoo-ref, else use the ref only when the zoo repo has it, else fall back to
main) so 'update --ref <unsloth-tag/sha>' does not fail on a missing zoo ref.
- Dockerfile: drop 10.3 (compute_103) from TORCH_CUDA_ARCH_LIST in both the
builder and runtime stages. B300 runs sm_100 SASS, and the bundled CUDA 12.8
nvcc cannot compile compute_103 (added in 12.9), which broke arch-list-honoring
source / JIT builds.
Move the sloth-sticker fail-soft explanation above the RUN so no comment line
sits between backslash-continued commands. BuildKit strips such comments, but
keeping the RUN body a plain && chain removes the ambiguity for non-BuildKit
builders and static linters. The { ...; } fail-soft scoping is unchanged.
- pip shim: count editable/local/url/vcs targets (-e ., ., git+https, wheel
URLs) as install targets, not just canonical package names, so they are no
longer silently skipped inside notebooks
- notebook sync: never overwrite a pre-existing user notebook on first boot
(match the refresh path's ownership rule); skip .unsloth_sync_state.tmp when
recording state so it is not tracked as a managed file
- docker-publish: set flavor latest=false on the base image metadata so a v*
tag push cannot publish :latest from the base image (the Studio image owns it)
- notebook deps: pin to tested versions and install decord on its own, hard on
amd64 and fail-soft on arm64 (no aarch64 wheel) so the arm64 base build works
- studio_launch.sh: also gate the categorized-view landing URL on
UNSLOTH_SKIP_NOTEBOOK_SYNC (the entrypoint skips building the view entirely in
that mode), not just UNSLOTH_SKIP_NOTEBOOK_VIEW, so a no-sync container does not
land on a missing folder.
- Dockerfile.studio: scope the sticker-install "|| echo" fallback to only the
sticker step via a { ...; } group. It was attached to the whole branding &&
chain, so a failure in a REQUIRED step (JS resolve, favicon/logo/login copy)
was swallowed and the build continued with broken branding.
- unsloth_nb_view.py: when creating the categorized symlinks, only replace our
own stale symlinks; if a real user file already occupies that name, keep it and
skip the link instead of os.remove-ing it.
- overrides.json: drop doNotDisturbMode (it silenced ALL JupyterLab toasts,
including kernel-restart / connection-drop feedback). The news/update prompts
are already off via fetchNews / checkForUpdates.
- Dockerfile: keep decord mandatory on amd64 (fail the build on a missing or
incompatible wheel) and only fail-soft on arm64/other arches that have no wheel.
- cellNav.ts: do not hijack ArrowUp/Down when focus is in an interactive output
widget / form control, or while a completion popup is open, so ipywidgets
controls and autocomplete at cell boundaries keep working.
- unsloth_nb_view.py: rebuilding the categorized view no longer deletes
user files. The view is also JupyterLab's landing dir, so a user may
save real notebooks there; _clear_view now unlinks only the symlinks we
own and removes only folders that end up empty, leaving regular files
in place. It also tests islink before isdir, so a view that is itself a
symlink to a directory is unlinked instead of being walked into (which
would have wiped the symlink target).
- studio_launch.sh: derive the landing URL and preferred_dir from
UNSLOTH_NOTEBOOKS_VIEW_DIR / UNSLOTH_SKIP_NOTEBOOK_VIEW, the same env
the sync script uses, instead of hard-coding /workspace/Unsloth
Notebooks. A relocated or disabled view no longer opens JupyterLab on a
missing folder; it falls back to the default /lab over /workspace.
- Dockerfile.studio: the labext-builder stage now installs Node 20 from
NodeSource. Ubuntu 24.04's distro nodejs is 18, below JupyterLab 4.6's
declared Node >=20 engine. Node stays confined to the throwaway builder
stage, so the runtime image is unchanged.
- .dockerignore: explicitly allowlist jupyter/install_sloth_stickers.py
alongside its sibling jupyter assets, rather than relying on the
directory re-inclusion.
Stacks a Colab-like JupyterLab and Studio experience on top of the
existing Blackwell image. Additive only: the training stack, CUDA/torch
pinning, and the Studio/JupyterLab/sshd service trio are unchanged.
JupyterLab labextension (prebuilt in a throwaway builder stage, so the
runtime image stays Node-free):
- Unsloth Dark (Monokai) theme, adaptive light/dark by system preference
- Colab-style ArrowDown/Up cell navigation
- top-bar Unsloth logo (stock Jupyter logo disabled and locked)
- #@title lines render as collapsible Heading-2 form bars
- Ctrl+A in a cell output selects only that output, not the whole
notebook (the old behaviour ran notebook:select-all and was laggy)
- right activity bar hidden by default
- overrides.json: per-cell run button without auto-advance, labeled
Restart and Run All, windowing off so collapsing an output does not
snap to the cell top, news/update prompts suppressed
Studio and login branding: Unsloth favicon, page logo, and a dark
Unsloth login page that rotates through the curated Studio sloth
stickers (fail-soft to the logo).
Notebook organization and Colab compatibility (base image):
- categorized folder view built from relative symlinks mirroring the
README sections, rebuilt each boot; real .ipynb files never moved,
and the symlink tree is invisible to the sync state machine
- AMD-* notebooks shown only on an AMD/HIP host (autodetected)
- Docker-only strip of the Colab "Run all on Colab" intro sentence
from unedited notebooks (upstream notebooks unchanged)
- hoist %%capture above a leading #@title form so the cell runs
- the per-cell transformers-sidecar log is silent unless
UNSLOTH_ENABLE_LOGGING=1
Dependency pinning and naming: the curated notebook extras are pinned to
their resolved versions for reproducible rebuilds; decord is split into
its own fail-soft install (no aarch64 wheel). The lean base image is
renamed from :base to :core.
Adds tests/validate_studio_features.py, a static self-test for the
labextension plugins, overrides keys, and branding wiring.
Mirror the public-link convenience Studio already has for its own UI, for
JupyterLab. Off by default; opt in two ways:
docker run -e UNSLOTH_JUPYTER_CLOUDFLARE=1 ... unsloth/unsloth
docker exec <container> unsloth-jupyter-tunnel --force
unsloth-jupyter-tunnel waits for JupyterLab, reuses a cached cloudflared (or
fetches the static binary for the arch, no account needed), and starts a
quick tunnel to the Jupyter port; the https://<name>.trycloudflare.com URL is
printed to docker logs. supervisord runs it as the jupyter-cloudflare program,
autostarted only when UNSLOTH_JUPYTER_CLOUDFLARE=1 (studio_launch.sh exports a
0 default so the autostart gate expands, matching the sshd pattern). JupyterLab
still enforces its password, so the tunnel is not an open door.
Verified: the helper fetches cloudflared and mints a working trycloudflare URL
that reaches JupyterLab (HTTP 200) inside a running container.
Parity with unsloth-studio-update: update the baked llama.cpp prebuilt in a
running container without pulling a new image.
docker exec <container> unsloth-llama-update # latest release
docker exec <container> unsloth-llama-update --check # report only
It reuses the build-time fetcher (fetch_llama_prebuilt.py, now baked at
/usr/local/lib/unsloth) rather than the host-probing installer behind the
in-app banner. The fetcher resolves the latest release via the GitHub
/releases/latest redirect (no API token, not rate-limited) and installs the
portable CUDA bundle that runs on CPU and every supported GPU, so it works the
same in a CPU-only or a --gpus container. The installer path, by contrast,
scans the GitHub API (rate-limited to 403 in practice) and probes the host GPU,
which falls back to a slow source build in a container started without --gpus.
The fetch lands in a sibling temp dir on the same filesystem and is swapped in
with an atomic rename; on any failure the existing install is left untouched.
The Studio ownership marker is preserved across the swap. Verified end to end
in a CPU-only container: b9596-mix-e6f2453 -> b9773-mix-1f1aaa4.
Updating Studio in the container previously meant pulling a fresh ~25GB image
(or at best the ~6GB fused Studio layer) for what is usually a small Python/UI
change. Add a baked helper so a running container can update in place:
docker exec <container> unsloth-studio-update
It updates only the Studio packages -- the backend code and the pre-built
frontend, which ships inside the unsloth wheel -- with `pip install -U
--no-deps unsloth unsloth_zoo`, then restarts just the studio service via
supervisor. The torch/CUDA stack is left untouched, so it is safe in both GPU
and CPU-only containers. This deliberately avoids `unsloth studio update`,
which re-runs the full installer and re-probes the GPU to pick torch wheels --
in a container started without --gpus that finds no GPU and can downgrade torch
to CPU/cu126.
Options: --ref <branch|tag|sha> installs from git (track main) instead of the
latest PyPI release; --with-deps also updates dependencies; --no-restart defers
the restart. After the swap the helper smoke-imports studio.backend.main and,
if a transitive dep is now missing, points the user at --with-deps.
The update lands in the container's writable layer (survives docker restart);
mount -v unsloth_studio_home:/opt/unsloth-studio to keep it across a recreate.
Two related changes to the baked llama.cpp prebuilt.
1. Dynamically follow the newest unslothai/llama.cpp release. build.sh resolves
the latest release tag (following the /releases/latest redirect, no API
token) to a concrete tag and passes it as LLAMA_PREBUILT_TAG, so the layer
cache busts only when upstream publishes. The Dockerfile default is now
"latest" and fetch_llama_prebuilt.py resolves it the same way, so a plain
`docker build .` also tracks latest. Pin LLAMA_PREBUILT_TAG to a concrete
tag for a reproducible, frozen build.
2. Make the in-app "newer llama.cpp available" banner work inside the image.
Studio's freshness check (utils.llama_cpp_freshness.check_prebuilt_freshness)
keys off tag / release_tag / published_repo in UNSLOTH_PREBUILT_INFO.json --
the schema install_llama_prebuilt.py writes. The image bakes the bundle
directly, so the marker was the release tarball's own, which only carries
upstream_tag / source_repo; the freshness check then bailed with
installed_tag=None and could never report "behind", hiding the banner.
fetch_llama_prebuilt.py now augments the baked marker with those keys
(setdefault, no build timestamp so the layer stays byte-identical). A fresh
build is on latest -> no banner; once upstream publishes a newer release the
banner appears, as verified against the real freshness backend.
The boot-time notebook sync now restores notebooks the user deleted, on
every boot, from the baked template (offline, even when upstream has not
advanced). It only restores files that are missing, so it never resurrects
or overwrites an edited notebook, and the GitHub refresh still bumps a
restored file to the latest upstream. Opt out with
UNSLOTH_KEEP_DELETED_NOTEBOOKS=1.
Also add unsloth_nb_content_sig.py to docker/.dockerignore's allowlist; it
was referenced by the Dockerfile COPY but excluded from the build context,
which broke the image build.
The boot-time refresh now compares only the tutorial body (the
non-boilerplate cells) when deciding whether to update an untouched
notebook. If only the install header, announcements, or footer moved
upstream, the user's file is left as-is so it is not churned. Notebooks
the user has edited or run are still kept untouched, and non-notebook
files keep the whole-file refresh. Adds unsloth_nb_content_sig.py to
segment head/middle/tail and bakes it into the image.
JupyterLab now opens with the unslothai/notebooks collection already present,
so people can open and run a notebook directly without a git clone or wget.
- Bake the repo into the image as a read-only template at /opt/unsloth-notebooks
(~206MB, .git stripped, build commit recorded). Inherited by the studio image.
- On boot the entrypoint populates /workspace/unsloth-notebooks from the template
(instant, works offline) and best-effort refreshes from GitHub, but only when
upstream has actually advanced (cheap git ls-remote gate, no download otherwise).
- The user's edits always win. We record the content hash of every file we write;
on refresh a file whose hash differs from what we last wrote is treated as
user-modified and is left untouched, so the refresh only updates files the user
has not changed and adds new ones. It never overwrites an edited notebook and
never produces merge conflicts. Verified: an edited notebook stays the user's
version across repeated upstream changes.
- Fully best-effort and gated: UNSLOTH_SKIP_NOTEBOOK_SYNC=1 disables it,
UNSLOTH_SKIP_NOTEBOOK_REFRESH=1 keeps the baked copy and never hits the network.
Offline boots keep what is there and never error.
base 18.45 -> 18.67GB, studio 24.88 -> 25.10GB (+~206MB baked notebooks).
Notebook dependency coverage (base Dockerfile):
- Bake omegaconf, einx, librosa, decord, ftfy so the TTS/STT and vision
notebooks stop dying on a silent No module named X. Installed in the
notebook-deps layer (after the torch/vLLM resolve) with an assertion that
the resolve did not move torch 2.10.0 / numpy>=2.3 / numba>=0.65.
Image size (no functional change):
- Base: prune npp to the two libs torchcodec actually dlopens
(libnppicc + libnppc), drop link-time-only .a archives and the nvshmem
device bitcode. Headers (torch/include etc) are kept so causal-conv1d /
mamba-ssm still build at notebook time with --no-build-isolation.
- Studio: pin the Studio venv to Python 3.12 (matches base) so its
nvidia-*-cu12 wheels are byte-identical to the base venv's, then symlink
the heavy arch-independent CUDA libs (cudnn/cublas/nccl/...) into the base
venv copy. cuda_nvrtc and cuda_runtime are excluded (the arm64 nvrtc swap
mutates nvrtc in place). Also remove the build-only frontend node_modules
(runtime serves the committed dist). Studio image drops ~4.8GB.
Per-notebook transformers version, run notebooks unchanged:
- Bake coherent transformers sidecars (4.57.6 default + 5.3.0/5.5.0/5.10.2),
each transformers==X with its matched huggingface_hub/tokenizers/
safetensors installed --no-deps into its own dir. Companion versions are
resolved at build time so they satisfy each transformers' requirements.
- unsloth_nb_compat.py: pick the sidecar from the notebook's pin or the
model name and activate it (prepend to sys.path) before any ML import,
without touching the base cu128 torch/vLLM/unsloth stack.
- pip/uv shim on PATH: a notebook install cell becomes safe and idempotent
inside a kernel (keeps the baked stack, records the requested transformers
for its sidecar); passthrough to the real tool everywhere else.
- IPython startup hook for manual JupyterLab, and unsloth-run for the
headless driven path.
Continuation of the notebook-dep prebaking: the in-image notebook runner
neutralises pip cells, so declared deps must be prebaked. evaluate's WER
metric imports jiwer (Whisper), DeepSeek-R1 GRPO's reward uses langid,
some vision trust_remote_code files need easydict, and sentencepiece
tokenizer conversion needs protobuf. All pure-Python; torch pin intact.
The TTS notebooks (Sesame CSM, Orpheus) read audio via soundfile, the
Whisper notebook computes WER via evaluate, and TrainingArguments
defaults report_to to tensorboard. These are declared by notebook pip
cells that the in-image notebook runner neutralises (deps are meant to be
prebaked), so without them those notebooks die on import. All are
pure-Python or self-contained wheels and never name torch, so the cu128
pin set is undisturbed.
flash-linear-attention's TileLang backend JIT-compiles CUDA kernels via
nvcc at runtime for gated-delta-rule models (Qwen3.5 family). The -base
image only ships runtime libraries, so Studio vision training of
unsloth/Qwen3.5-2B died on the first backward pass with
[Errno 2] No such file or directory: /usr/local/cuda/bin/nvcc.
Install cuda-nvcc and cuda-cudart-dev matching the image CUDA version
and assert nvcc is executable at build time. Found by driving a real
Qwen3.5-2B training run through the Studio UI in the image.
A locally built tag (test_locally.sh or docker build) is not on a
registry, so the pull phase reported hard failures on a machine that was
actually fine. Degrade to a warn when the image is present locally;
missing images still fail.
llama-quantize exits nonzero on --help/--version while still printing
usage, so a bare invocation fails the build even when the binary is
healthy. Grep for the usage banner instead; a loader failure prints
error while loading shared libraries and no usage text.
The build/bin hardlink mirror skipped symlinks, so the soname links
(libllama-common.so.0 and friends) never reached build/bin. Studio's
setup.sh relinks the root llama-quantize to build/bin/llama-quantize,
whose RUNPATH is $ORIGIN, so the loader failed with libllama-common.so.0
not found and GGUF export from Studio died with No working quantizer
found, then hit the interactive source-build prompt in a non-TTY export
subprocess (EOFError). Mirror same-directory soname symlinks into
build/bin and extend the bake sanity check to execute llama-quantize from
both the install root and build/bin. Dockerfile.studio now also runs the
studio-visible quantizer after install.sh so a regression fails the
image build instead of runtime exports.
_detect_cuda_torch_index_url now respects the explicit family override
before probing nvidia-smi, matching install.sh get_torch_index_url and
install.ps1 Get-TorchIndexUrl. Without it, a GPU-less environment falls
back to cu126 wheels which lack sm_100/sm_120 kernels and break training
on Blackwell. ROCm repair path is intentionally unchanged.
Dockerfile.studio now fails the build if the Studio venv torch local
version tag does not match the pinned TORCH_FAMILY, so a studio ref whose
installer ignores the override can never ship a silently wrong image.
Metadata-only check so QEMU arm64 builds do not need to load torch.
The notebook validation matrix caught the synthetic-data notebook dying
because the vllm server SyntheticDataKit launches never came up. Two
layers to the failure:
1. flashinfer's cpp_ext JIT shells out to ninja. The pip ninja lives in
the venv bin, which subprocesses like vllm serve do not always
inherit on PATH, so the JIT failed with exit 127. Install ninja-build
so the binary is reachable from any PATH.
2. With ninja present the JIT still cannot succeed for device code: the
runtime image deliberately ships no nvcc. Bake flashinfer-jit-cache
(cu128) so ops missing from the cubin package (fmha_gen on sm_100a
was the repro) come precompiled. In-process GRPO never hit this
because unsloth-zoo blocks the FlashInfer JIT path; standalone
vllm serve gets no zoo patches.
Fail-soft on the jit-cache for arches without a wheel; the vLLM chain
itself stays fail-loud on amd64.
Notebooks fetch sample assets with !wget; without the binary the shell
prints not-found to stderr, the cell still exits zero from Jupyter's
perspective, and the next cell crashes confusingly on the missing file.
The Whisper notebook died exactly this way in the validation matrix.
PyPI has shipped aarch64 abi3 wheels for every vLLM release since 0.17,
so the arm64 skip rested on a stale premise. With torch held at 2.10.0
the resolver lands on vllm 0.19.1 (the release pinning torch==2.10.0)
on both arches; verified by cross-resolving the exact index set for
aarch64-unknown-linux-gnu.
amd64 keeps fail-loud semantics. arm64 is fail-soft because the aarch64
wheels are newer and their GPU kernels get validated on Spark hardware
via docker_confirm.sh rather than in CI; on failure the fallback
uninstalls vllm and restores the numpy/numba floor so a partial install
cannot break import unsloth (numpy 2.2.6 ships a broken numpy.testing).
The install steps form an explicit && chain instead of a set -e
subshell: POSIX shells disable errexit inside condition contexts
(verified on dash), so a (set -e; ...) condition would mask failures.
Both confirm scripts gain a 5b vLLM phase: ok on import, bad if missing
on x86_64, warn on other arches where fast_inference=True is best-effort.
The wheel install belongs in the builder (the venv copy carries it),
but the ld.so.conf.d registration and the import check belong in the
runtime stage: the conf file does not survive the stage copy and the
import needs ffmpeg, which only the runtime stage installs.
The TTS/STT notebooks decode datasets Audio features through torchcodec,
which fails three different ways on a fresh image: the PyPI wheel pairs
with the cu13 torch line and dlopens libnvrtc.so.13; builds newer than
0.10 reference torch 2.11+ symbols; and the matching +cu128 build dlopens
torch and NVIDIA runtime libraries that live inside the venv where the
loader cannot see them. Bake ffmpeg, torchcodec==0.10.0 from the cu128
channel, nvidia-npp-cu12, and register the venv lib dirs via ld.so.conf.d
(not LD_LIBRARY_PATH, so the llama.cpp bundle keeps winning through its
own RUNPATH). Verified in-container: AudioDecoder imports and llama-server
still resolves its bundled libraries.
Running the published unslothai/notebooks set inside the image surfaced
two gaps: the Ollama export notebook installs ollama in-container and
that installer needs zstd for extraction, and DeepSeek-OCR's
trust_remote_code modeling file imports matplotlib unconditionally
(plotting is also simply expected in a Jupyter image).
- Dockerfile: lift numba past vllm's 0.61.2 pin after the numpy>=2.4
re-upgrade; 0.61.2 refuses numpy 2.3+ at import time and the stack
cannot move numpy down. Verified numba 0.65 + numpy 2.4.6 + vllm
import cleanly together.
- docker-publish.yml: resolve UNSLOTH_ZOO_REF in a step that mirrors
the pushed tag only when the tag exists in unsloth-zoo (the zoo
currently cuts no tags, so blind mirroring broke every tag publish);
falls back to main.
- Dockerfile.studio: Studio venv stays on cu128 for arm64 too, matching
the base venv (cu130 wheels would lift the driver floor to 580+), and
gets the same NVRTC cu13 swap for DGX Spark / GB10 sm_121 support.
- docker_confirm.sh: do not drop to CPU mode when docker info lacks a
nvidia runtime entry; CDI installs and Docker Desktop WSL2 expose
GPUs without one. The phase 3 --gpus probe is now the authority.
- docker_confirm.ps1: GPU selector built as an args array; comma device
lists get version-aware CSV quoting (native arg passing changed in
PowerShell 7.3).
- studio_launch.sh: no fixed Jupyter default password; generate a
random one and print it when JUPYTER_PASSWORD is unset. Env snapshot
for SSH sessions now written via shlex.quote instead of sed so
values with quotes or command substitution cannot break or inject
into /etc/profile.d.
- install.ps1: honour UNSLOTH_TORCH_INDEX_FAMILY like install.sh does.
