Comment-only pass over the PR's own files. No executable line changes.
- Dockerfile / Dockerfile.studio: drop the decorative stage banner rules, the
stale "5)" / "6)" step numbering, and the entrypoint pre-flight list that
restated (and had drifted from) entrypoint.sh's own accurate header. Cut the
llama.cpp asset bullet list that repeats fetch_llama_prebuilt.py's docstring
and the structlog rationale already spelled out at the install site.
- entrypoint.sh / studio_launch.sh: fold the section banners into the
explanation lines that follow them.
- docker-publish.yml: remove the comment rule lines around the job headers.
- validate_studio_features.py: same for the numbered section headers.
- smoke_test.py: drop the stale "~125M params" note on a 1B model.
- unsloth_branding.py, unsloth_nb_view.py, unsloth_nb_pip_magic.py,
colabTitle.ts: remove comments that restate the adjacent line.
Two defects found by running the built image rather than reading it.
1. Every unsloth_cli subcommand that touches the studio backend died on
import. `unsloth list-checkpoints` on the published image:
ModuleNotFoundError: No module named 'structlog'
and the same for train / export / chat, since all four import
studio.backend.core.*. structlog is a studio backend requirement, not
an unsloth[huggingface] one, so nothing in the base install pulled it
in. Added it to the base venv, and added a build-time
`from studio.backend.core.export import ExportBackend` so a future
missing dependency in that closure fails the build instead of the
user's first CLI invocation. That guard has to live in the LAST
builder verification block: the closure also needs starlette, which
only arrives with vLLM two stages later.
2. flashinfer-jit-cache was pinned to a literal 0.6.6 while vLLM 0.26.0
resolves flashinfer-python 0.6.14. flashinfer raises at import when
the two disagree, and that exception is thrown inside the vLLM
EngineCore, so Unsloth's GRPO fast_inference path fails at engine
start with no earlier warning. A literal pin drifts again on the next
vLLM bump, so the version is now read back from the resolved
flashinfer-python, and the build proves `import flashinfer` works.
Verified on the rebuilt image: flashinfer-python 0.6.14 with
flashinfer-jit-cache 0.6.14+cu128, structlog 26.1.0, the export backend
importable, and `unsloth list-checkpoints` exiting 0.
tests/python/test_docker_llama_cuda_backend.py gains two static cases
pinning both: the jit-cache version must be derived rather than literal
and the build must import flashinfer, and the base venv must ask for
structlog with the CLI reachability guard present.
The portable llama.cpp bundle loads libggml-cuda.so with dlopen, links it
against libcublas, and does not ship libcublas. The CUDA runtime base
image only carries libcudart, and the only libcublas in the image is
torch's wheel copy under site-packages/nvidia/cublas/lib, which was not
on the loader path. So the CUDA backend failed to load, and llama.cpp
said nothing about it: `--list-devices` printed an empty list and every
GGUF request ran on the CPU.
Measured in the built image on a B200 with gemma-4-E2B-it UD-Q4_K_XL:
1.6 tok/s from llama-cli and 4.2 tok/s from llama-server. With the fix,
the same image and model report `CUDA0: NVIDIA B200` and run at 229 tok/s
and 193 tok/s. Studio's GGUF chat and the GGUF export path go through the
same bundle, so both were affected.
The venv loader config already existed for torchcodec, so cublas/lib
joins it there rather than on LD_LIBRARY_PATH: ld.so.conf.d is consulted
after DT_RUNPATH, which keeps llama.cpp resolving its own $ORIGIN libs
first. cu13/lib comes along for the arm64 bundle's layout.
A silent 140x slowdown deserves a build-time gate, so the layer after the
fetch runs ldd over libggml-cuda.so, installs the cublas major the bundle
actually asks for when it is missing, and fails the build on anything
still unresolved. The amd64 bundle wants libcublas.so.12 and torch
already provides it; the arm64 bundle is CUDA 13, and deriving the major
from ldd keeps that leg honest without hardcoding either. libcuda.so.1 is
exempt: nvidia-container-toolkit injects the driver stub at
`docker run --gpus`, so it is never resolvable at build time. ldd needs
no GPU, so the build stays host-independent.
tests/python/test_docker_llama_cuda_backend.py pins the loader entry, the
guard, the driver-stub exemption and the ordering.
The GitHub refresh phase ran synchronously in the entrypoint's notebook
sync, so an offline or slow network could hold container startup for up
to two fetch timeouts (ls-remote + clone, about two minutes at the
defaults) despite the sync being described as non-blocking. The local
template populate and the categorized view still run in the foreground;
the refresh now re-enters itself as a detached child (guarded by a flag
so it forks once), whose phase-1 pass no-ops via the hash state and
whose finalize is idempotent. Verified with an unreachable remote and
an 8 second timeout: the parent returns in under a second with the
notebooks populated while the child owns the waiting.
Three fixes from review:
unsloth-run now gives each invocation its own UNSLOTH_NB_TF_MARKER (a
temp file, cleaned up afterwards) unless the caller pinned one. The
shared default marker leaked one run's transformers pin into later or
concurrent runs in the same container: a notebook pinned to 4.57.6
left the marker behind and the next unpinned run's kernel activated
the stale sidecar. An empty marker reads as no pin, so pre-creating
the file is safe.
The vLLM startup wait in dataprep/synthetic.py capped every poll at a
full second regardless of the remaining budget, so a fractional
timeout could overshoot by up to a second. The final wait is now
clamped to the remaining time; verified empirically (timeout=1.1
elapses 1.10s).
/workspace and the default HF/Triton cache dirs were root-owned, so
docker run --user without a bind mount could not sync notebooks or
populate caches. They are now world-writable (a+rwX), matching the
documented non-root use the /opt prebuilt placement already supports.
Condense the verbose explanatory comments added by this branch to their essential
points without dropping any load-bearing rationale. Touches comments and
docstrings only, no code changes. Leaves the stable-tag gate rationale, the
byte-identical enable= sync notes, and the update-alternatives pin comment as is.
Installing cuda-nvcc-13-0 for the sm_103/sm_121 JIT tools also flips the
update-alternatives-managed /usr/local/cuda link to cuda-13.0: the package
hard-depends on cuda-toolkit-13-0-config-common, whose postinst registers
priority 130 over 12.8's 128 (reproduced in a clean
nvidia/cuda:12.8.1-base-ubuntu24.04 container; --no-install-recommends does
not help against hard Depends). TileLang JIT and torch.utils.cpp_extension
resolve nvcc through /usr/local/cuda, so on the 570-driver hosts this image
supports they would emit cu13 cubins that need driver 580 and fail at load.
Pin the alternative back to 12.8 right after the cu13 install; the cu13
tools stay reachable by absolute path, which is exactly how the entrypoint
activates them, and manual mode prevents future apt flips.
run.sh accepted the native --gpus device=0,1 form through an unquoted
passthrough, but docker requires the comma-carrying value to be quoted
(daemon rejects it with 'cannot set both Count and DeviceIDs'; reproduced
against a live daemon, and the docker GPU docs call the quoting out
explicitly). A native multi-device selector is now wrapped in the same
embedded quotes the other comma paths already use; single-device and
pre-quoted forms pass through unchanged. All eight selector forms verified
through the case block.
The amd64 base install named the cu128-ampere-torch2110 extra, which does not
exist on main yet (the CUDA extras stop at torch2100): pip/uv only warn on an
unknown extra, so plain unsloth installed without xformers and the required-
package check failed the build. Both arches now take the plain huggingface
extra and amd64 pins xformers==0.0.35 explicitly in the same resolve (it
requires torch>=2.10 without an exact pin, pairing with the baked 2.11.0;
verified on PyPI, x86_64 wheels only, matching the arm64 exclusion). This
decouples the base image from the pending extras PR.
The Studio build now receives the SAME llama.cpp tag the base image baked:
Dockerfile.studio grows a LLAMA_PREBUILT_TAG arg exported as UNSLOTH_LLAMA_TAG
to install.sh (setup.sh honours it; the "latest" default is byte-identical to
setup.sh's own default for local builds), and the publish workflow forwards
the prepare job's resolved tag in the studio build-args. Without the pin a
dispatch override or an upstream release landing between the two jobs let the
no-GPU Studio build re-resolve "latest" and replace the pinned CUDA bundle.
The Studio venv-match assertion also needs installer support for torch 2.11
on the CUDA path; that lands in a separate installer PR and is now declared
as a merge-order dependency in the PR description (the publish workflow only
runs on main pushes, so nothing builds before both are merged).
A VCS @ref can itself contain a slash (@feature/foo), and the shim split the
last path segment BEFORE dropping the ref, so
git+https://github.com/unslothai/unsloth.git@feature/foo canonicalized as
"foo" and a protected repo installed from a branch dodged _KEEP. The ref is
now stripped from the path portion first (after the authority, so an SSH
userinfo @ is never mistaken for the ref separator, matching pip's own
last-@ parsing), with regressions for slash refs, SSH userinfo, plain tags
and the no-ref form.
The help-derived value-flag drift guards were version-sensitive: repo CI runs
whatever pip/uv are current, so the next tool release turned unrelated PRs
red (pip 26 added --all-releases/--only-final/--requirements-from-script/
--uploaded-prior-to, uv added --no-editable-package/--upgrade-group; all six
now classified). The guards are opt-in for local runs
(UNSLOTH_SHIM_FLAG_DRIFT_CHECK=1) and the authoritative check now runs at
image build time via a new --unsloth-selfcheck-value-flags mode wired into
the Dockerfile verify step, where the baked pip/uv are exactly the tools the
shim fronts, so a flag added by a future baked-tool bump fails the build
instead of a user's notebook cell.
Preflight (entrypoint.sh) now scans every visible device: an unsupported device
0 stays fatal as before, an unsupported secondary GPU (mixed rig) warns at
startup with its index and the CUDA_VISIBLE_DEVICES remedy, instead of
surfacing only when a job pins to it or a multi-GPU launch fans out.
The pip shim's _VALUE_FLAGS now covers every value-taking flag of uv pip
install and pip install (generated from both tools' --help). The separated
form `uv pip install --torch-backend cu128 torch` used to drop the protected
torch but exec uv with no install target at all (uv hard-errors) instead of
no-oping like the attached `=` form, and `--extra torch peft` misread the
extra name as a protected target, leaving a dangling --extra that swallowed
peft. Adds parametrized regressions plus help-derived drift guards so a future
uv/pip value flag cannot silently reintroduce the misparse.
unsloth-llama-update now detects when the install dir is itself a mount point
(the documented -v unsloth_llama:/opt/unsloth/llama.cpp persistence recipe,
where rename(2) fails EBUSY) and swaps the bundle CONTENTS inside the mounted
tree, so the update lands in the volume and stays persistent. Work and backup
dirs live under the mount (same-fs renames), the abort trap restores the old
contents even mid-swap, and the non-mounted path keeps the whole-dir rename.
Verified: in-place swap preserves the dir inode and ownership marker, failed
fetch leaves the install untouched, simulated mid-swap abort restores fully.
Comment-only consolidation: the sm_103/sm_121 + cu13 JIT story and the
xformers-aarch64 note were each told four times across docker/Dockerfile; keep
the header telling canonical and cross-reference it elsewhere (same for the
workflow's six retellings of the resolve-refs-once rationale and
Dockerfile.studio's NVRTC block). Comments that pointed at the removed dev
scripts now name the underlying command or artifact instead. Non-comment lines
of both Dockerfiles and the workflow are byte-identical.
unsloth_sync_notebooks.sh folds the three copies of the override -> PATH ->
sibling helper resolution into one resolve_helper(), behavior verified for all
four modes including graceful absence under set -u.
unsloth_pip_shim.py collapses an if/else whose branches were identical and
merges the structurally duplicate _parse_include/_parse_editable into one
_parse_flag_line. The test suite folds 35 near-duplicate tests into 8
parametrized groups with exact case-count parity (69 collected before and
after, 81 passing including the nb-pip-magic suite).
Cuts another 144 lines with zero behavior change outside the two refactors.
Remove seven dev-only scripts that never reach the image or CI: the
.dockerignore whitelist excludes them from the build context, docker-publish.yml
runs smoke_test.py via buildx with native arm64 runners (no QEMU setup script),
and nothing else references them beyond a few comments. test_locally.sh,
docker_confirm.sh/.ps1, setup_qemu.sh, hf_pull.sh, hf_push.sh and freeze.sh can
return in a follow-up dev-tooling PR; this PR stays the image itself.
Cuts 1105 lines and 7 files from the diff.
Every new .py this PR adds now carries the same two-line SPDX header the other
new files in the branch already use (docker/jupyter/unsloth_branding.py), with
the shebang kept first where present. Matches the licensing laid out in
docker/NOTICE: the image bundles Studio (AGPL-3.0) while Unsloth Core stays
Apache-2.0.
Four follow-ups to the shim/entrypoint audit fixes:
1. unsloth_pip_shim.py let a local project directory install through: `pip
install ./transformers` / `-e ./unsloth` is not a requirement spec, so
_canon returned None and both the arg filter and the constraints file
(which only rejects a version MISMATCH) passed it, letting a same-version
local build silently replace the baked wheel. _canon now resolves the
project name from pyproject [project].name, then setup.cfg, then the
directory basename when it is an installable project, so a local checkout
of a protected package is dropped like every other artifact form. Names
match exactly after normalization, so a user dir named my-torch-utils is
untouched, and a metadata-less directory still passes through.
2. unsloth_nb_pip_magic.py only rewrote literal `!python -m pip`, so the
`!{sys.executable} -m pip ...` form notebooks use to target the running
kernel (and absolute interpreter paths) bypassed the PATH shim entirely.
Input transformers see the raw cell text before IPython expands the
braces, so the matcher now also covers {sys.executable} (quoted or bare)
and quoted/bare interpreter paths ending in python[0-9.]*(.exe) before
-m pip|uv.
3. unsloth_pip_shim.py did not strip uv's --exact, which performs an exact
sync that removes every installed package outside the kept target's
closure (vLLM, bitsandbytes, the NVIDIA libs); `uv pip install --exact
peft` would strip the baked stack after the filter kept it. --exact now
joins the resolver-wide destructive flags dropped in shim mode.
4. entrypoint.sh: the arm64 image bakes a CUDA 13 llama.cpp because upstream
(unslothai/llama.cpp) publishes no CUDA 12 arm64 asset, while the torch
stack (cu128) runs on a 570-series driver. A CUDA 13 cubin cannot load on
a 570-579 driver, so on GH200/GB200 hosts below 580 GGUF export and Studio
chat fail while training works. The entrypoint now warns up front on
aarch64 + driver < 580 instead of letting llama-server fail later.
Tests: shim + nb-pip-magic suites at 81 (18 new, including local-project
name resolution, the executable/brace forms, and --exact stripping).
Ten verified fixes from a 12-reviewer audit of the image tooling, each
reproduced before fixing:
1. install_llama_prebuilt.py move_install_dir_aside: the EXDEV fallback
copied straight into the rollback path, so a copy that died halfway
(ENOSPC, I/O error) left a partial tree that activation recovery would
later restore over the intact install while deleting the good copy.
Copy to a temp sibling and publish with one atomic rename; dst.exists()
is now a truthful complete-tree signal.
2. unsloth_run.py --out truncated the existing output before nbconvert
ran, so a timeout, missing kernel, or failed cell irreversibly
destroyed the previous result. The input copy and executed result are
staged as temp files next to the destination and published with
os.replace only on exit code 0.
3. unsloth_nb_view.py cleanup treated every symlink in the view as its
own: user-created links (and an operator's view-root routing symlink)
were deleted on every rebuild. Cleanup now removes only links that
resolve into the notebooks tree it links from, and builds inside a
view-root symlink's target instead of unlinking it.
4. unsloth_llama_update.sh: the unconditional EXIT trap deleted the .old
backup even when it was the only remaining copy (signal between the two
renames, or a failed swap whose restore also failed). The handler now
restores the backup first when the install dir is missing and removes
it only after the new tree is verifiably active; HUP/INT/TERM route
through the same handler.
5. unsloth_pip_shim.py: transitive dependencies could replace the baked
torch stack (reproduced with a wheel requiring torch==99.0). Every
forwarded install now carries a constraints file pinning the installed
protected set, turning the swap into ResolutionImpossible.
6. unsloth_pip_shim.py: ${UPPER} env references in requirements files were
classified before pip expanded them, bypassing the protected-package
filter; the shim now expands with pip's exact regex first.
7. unsloth_pip_shim.py: a failure writing the filtered requirements copy
returned the ORIGINAL file, forwarding exactly the protected pins it
had detected; it now fails closed.
8. docker-publish.yml: workflow_dispatch defaulted unsloth_ref to 'main'
while the stable-tag gates require '', so UI-default manual runs could
never advance :core/:latest/:studio; the default is now empty.
9. entrypoint.sh: the sm_103/sm_121 branch rewrote libnvrtc.so.12 to the
CUDA-13 build but the ordinary-GPU branch never restored it, so a
container moved to an older GPU kept the stale link; it is now reversed
when it points exactly at the .cu13 target.
Rejected after verification (no code change): timeout=0 semantics are
documented at the site with no zero callers, TORCHINDUCTOR_COMPILE_THREADS
override is deliberate, fetchNews is a string enum per JupyterLab's schema,
:base tag appears in no in-tree doc, install-cell digest exclusion is the
module's stated contract, transformers ceiling semantics are documented,
and the cloudflared download mirrors the pre-existing Studio downloader
(Cloudflare publishes no checksum asset). The UNSLOTH_ALLOW_CPU import
crash lives in unsloth_zoo (compiler.py / loss_utils.py capability probes),
not in this diff; the image consumes the zoo fix automatically once merged
there.
Tests: shim suite extended to 63 (constraints, env expansion, fail-closed),
jit-selector suite to 14 (NVRTC reversal transitions), plus staged-publish
and ownership repros; wider studio install suite green except failures
reproduced at the unmodified head.
Notebook pip/uv shim (docker/unsloth_pip_shim.py, active only under
UNSLOTH_NB_SHIM=1):
- Parse protected source archives (sdist/zip) by basename too, e.g.
`pip install https://.../unsloth-2026.7.1.tar.gz` or `./torch-2.11.0.tar.gz`,
mirroring the wheel-basename handling. A first-hyphen-before-digit split
keeps hyphenated names like flashinfer-python intact.
- Recognise uv's PLURAL long flags --requirements / --constraints, so those
files go through the same protected-package filter as the singular names.
- Drop --upgrade-strategy eager in shim mode so a kept target cannot eagerly
rebuild already-satisfied baked deps (falls back to pip's only-if-needed).
NVRTC default (docker/Dockerfile, docker/Dockerfile.studio, docker/entrypoint.sh):
- Make cu12.8 the immutable baked default (libnvrtc.so.12 -> .cu128.orig) with
a staged .cu13 alias, and have select_cuda_jit_tools retarget to cu13 ONLY
for sm_103/sm_121. Previously cu13 was baked as the default and restored to
cu12.8 at runtime, so a non-root `docker run --user` container that cannot
rewrite the symlink stayed on cu13 NVRTC and emitted cubins a 570-579 driver
cannot load. The safe default now needs no runtime write.
Adds regression tests for each case (tests/python/test_unsloth_pip_shim.py,
tests/sh/test_select_cuda_jit_tools.sh).
Notebook pip/uv shim (docker/unsloth_pip_shim.py), all active only under
UNSLOTH_NB_SHIM=1:
- Parse a bare wheel filename (torch-*.whl in the CWD, no ./ or / prefix) so
it is matched against _KEEP instead of passing through as an opaque
positional and reinstalling the baked torch.
- Infer the distribution from an egg-less VCS URL by repo basename
(git+https://github.com/huggingface/transformers.git -> transformers) so
the egg-less form the repo itself recommends cannot clobber the baked stack.
- Refuse remote (URL) -r/-c requirement/constraint files -- top-level and
nested includes -- since their pins cannot be inspected before the real
tool would fetch and install them.
- Strip resolver-wide reinstall/ignore-installed switches
(--force-reinstall, --ignore-installed, -I, uv --reinstall) so they cannot
rebuild already-satisfied baked deps pulled in by a kept target.
- Route uv --reinstall-package through the same _KEEP handling as
-P/--upgrade-package (both attached and separated forms; no dangling flag).
Entrypoint (docker/entrypoint.sh): select_cuda_jit_tools() now scans every
visible GPU's compute_cap instead of only the first, so a datacenter Blackwell
(sm_103/sm_121) behind an H100/B200 still enables the cu13 JIT tools it needs.
Adds regression tests for each case (tests/python/test_unsloth_pip_shim.py,
tests/sh/test_select_cuda_jit_tools.sh).
unsloth_pip_shim.py: close three more ways a protected package slipped past
_KEEP. An editable line (-e/--editable <target>) inside a -r requirements file
is a real install target, so a protected editable there is now classified and
dropped like the command-line case (new _parse_editable). pip/uv accept the
attached short forms -rreqs.txt / -cconstraints.txt / -epath / -Pname as one
token; these were falling through as opaque options (so an attached -r-only cell
no-op'd and an attached -c/-e/-P value bypassed _KEEP), so the 2-char flag is now
split from its value and routed through the separated-form handling. And a nested
-c constraint inside a -r file no longer records its transformers pin as an
install request (a constraint is not a request; mirrors the top-level -c path).
entrypoint.sh / Dockerfile: gate the CUDA 13 ptxas + NVRTC to sm_103 / sm_121 at
runtime instead of a global build-time default. A cu13 cubin needs a >= 580
driver to LOAD even when it targets an older arch (CUDA has forward, not
backward, cross-major driver compatibility), but the image supports Turing..
sm_120 on a 570+ driver, so the previous global TRITON_PTXAS_PATH ENV + cu13
NVRTC symlink would break ordinary Triton/NVRTC JIT on 570-579 driver hosts. The
build still bakes cu13 (saving the cu12.8 NVRTC as .cu128.orig); a new
select_cuda_jit_tools() in the entrypoint reads the device compute_cap and only
activates cu13 for sm_103/sm_121 (which ship >= 580 drivers), otherwise leaving
Triton on its bundled cu12.8 ptxas and restoring the cu12.8 NVRTC in both the
base and Studio venvs. The base ENTRYPOINT runs for the Studio image too.
Adds 9 pip-shim regression tests and tests/sh/test_select_cuda_jit_tools.sh
(7 device-gating cases); registers the latter in CI and tests/run_all.sh.
docker-publish.yml: freeze the requested unsloth ref to one sha in the prepare
job before the matrix fans out. UNSLOTH_REF / UNSLOTH_STUDIO_REF were raw
expressions re-evaluated per base arch leg and in the Studio build, so a mutable
branch (the workflow_dispatch default unsloth_ref=main) advancing during the run
could bake different unsloth commits under one manifest. Resolve once (same
precedence: dispatch input, else pushed tag, else triggering sha, else main;
ls-remote a branch/tag to a sha, mirroring the zoo/notebooks steps) and read
needs.prepare.outputs.unsloth_ref everywhere.
Dockerfile.studio: run the Studio venv NVRTC cu13 swap on both arches, not arm64
only. amd64 sm_103 (B300/GB300) needs cu13 NVRTC just as arm64 sm_121 does, and
the CUDA dedup never touches cuda_nvrtc, so an amd64 Studio venv would otherwise
keep its bundled cu12.8 libnvrtc and fail NVRTC/jiterator JIT on compute_103. The
base cu13 layer installs cuda-nvrtc-13-0 on both arches, so the target .so.13
exists here regardless of TARGETARCH.
unsloth_pip_shim.py: close three ways a protected package slipped past _KEEP.
Treat -e/--editable as a value-taking flag paired with its target and drop both
when the target is protected (was leaving a dangling -e that failed the cell);
filter -P/--upgrade-package values through _KEEP (a named baked package could be
refreshed while installing another target); and parse the PEP 427 distribution
name out of a wheel URL/path so a bare `pip install https://.../torch-...whl`
drops instead of reinstalling the baked torch. Non-protected editables, upgrade
selectors, and wheels are unchanged. Adds tests/python/test_unsloth_pip_shim.py
(18 regression tests, exec captured via a patched os.execv).
The Studio build symlinks the Studio venv's CUDA libs onto the base venv's
copies to reclaim ~3.7GB. That is only safe when both venvs run the same torch,
but the pre-dedup guard only checked the CUDA family (endswith('+cu128')). A
Studio venv that installed torch 2.10.0+cu128 (an installer capped below the
base's 2.11.0, or a build-time nvidia-smi fallback) would pass that check yet
mismatch the base's 2.11.0+cu128, and the dedup would link incompatible libs.
Capture the base venv's torch from its metadata and assert the Studio venv torch
equals it exactly (version and family) before the dedup runs, so a mismatch
fails the build loudly instead of silently linking skewed CUDA libs. Comparing
to the base venv also avoids hardcoding the version here. The Studio venv reaches
torch 2.11.0+cu128 via the installer's UNSLOTH_TORCH_INDEX_FAMILY=cu128 handling
and its CUDA torch spec allowing 2.11.x.
Base image (torch 2.11.0):
- amd64 unsloth extra: cu128-ampere-torch2100 -> cu128-ampere-torch2110.
The old extra pulls xformers 0.0.34, which hard-pins torch==2.10.0 and
conflicts with the torch==2.11.0 held throughout the build; the torch2110
family pulls xformers 0.0.35 (no torch pin) and resolves cleanly. This
needs an unsloth carrying the torch2110 CUDA extras on main, so merge the
torch2110 extras PR first (default UNSLOTH_REF=main).
- notebook-deps assertion: startswith('2.10.0') -> '2.11.0' so the layer
actually verifies the torch it now installs.
- refresh the torch2100/xformers 0.0.34 references in the surrounding
comments to the torch2110/0.0.35 line.
sm_103 (B300/GB300) JIT override (Codex item):
The cu13 NVRTC/ptxas override was arm64-only (sm_121), and its comment
claimed triton 3.6.0 bundles cu13 ptxas and set TRITON_PTXAS_PATH -- neither
was true: triton 3.6.0's bundled ptxas is CUDA 12.8 (V12.8.93, tops out at
sm_120) and TRITON_PTXAS_PATH was never set. So sm_103 (amd64) and even
sm_121 (arm64) Triton JIT were unfixed.
Run the cu13 install on both arches and actually wire the ptxas override:
- NVRTC swap (cu13 libnvrtc.so.13 over torch's bundled cu12.8 .so.12) now
runs on amd64 too.
- ENV TRITON_PTXAS_PATH=/usr/local/cuda-13.0/bin/ptxas routes every Triton
JIT through the cu13 ptxas. Global rather than per-arch is safe: cu13.0
ptxas spans sm_70..sm_121 (verified: Volta/Turing/Ampere/Hopper through
Blackwell), so no regression for the older GPUs in the arch list.
Verified on amd64 in the built base image: cuda-nvrtc-13-0/cuda-nvcc-13-0
install cleanly from the base's CUDA repo, ptxas lands at
/usr/local/cuda-13.0/bin/ptxas (V13.0.88) and libnvrtc.so.13 at
/usr/local/cuda-13.0/lib64/. The sm_103/sm_121 runtime path itself is not
hardware-tested (no such GPU on hand); precompiled SASS still covers both
via sm_100/sm_120 forward-compat, so only JIT-heavy paths rely on this.
* docker: Colab-grade JupyterLab and Studio UX for the Blackwell image
Stacks a Colab-like JupyterLab and Studio experience on top of the
existing Blackwell image. Additive only: the training stack, CUDA/torch
pinning, and the Studio/JupyterLab/sshd service trio are unchanged.
JupyterLab labextension (prebuilt in a throwaway builder stage, so the
runtime image stays Node-free):
- Unsloth Dark (Monokai) theme, adaptive light/dark by system preference
- Colab-style ArrowDown/Up cell navigation
- top-bar Unsloth logo (stock Jupyter logo disabled and locked)
- #@title lines render as collapsible Heading-2 form bars
- Ctrl+A in a cell output selects only that output, not the whole
notebook (the old behaviour ran notebook:select-all and was laggy)
- right activity bar hidden by default
- overrides.json: per-cell run button without auto-advance, labeled
Restart and Run All, windowing off so collapsing an output does not
snap to the cell top, news/update prompts suppressed
Studio and login branding: Unsloth favicon, page logo, and a dark
Unsloth login page that rotates through the curated Studio sloth
stickers (fail-soft to the logo).
Notebook organization and Colab compatibility (base image):
- categorized folder view built from relative symlinks mirroring the
README sections, rebuilt each boot; real .ipynb files never moved,
and the symlink tree is invisible to the sync state machine
- AMD-* notebooks shown only on an AMD/HIP host (autodetected)
- Docker-only strip of the Colab "Run all on Colab" intro sentence
from unedited notebooks (upstream notebooks unchanged)
- hoist %%capture above a leading #@title form so the cell runs
- the per-cell transformers-sidecar log is silent unless
UNSLOTH_ENABLE_LOGGING=1
Dependency pinning and naming: the curated notebook extras are pinned to
their resolved versions for reproducible rebuilds; decord is split into
its own fail-soft install (no aarch64 wheel). The lean base image is
renamed from :base to :core.
Adds tests/validate_studio_features.py, a static self-test for the
labextension plugins, overrides keys, and branding wiring.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* docker: address review feedback on the JupyterLab/Studio UX
- unsloth_nb_view.py: rebuilding the categorized view no longer deletes
user files. The view is also JupyterLab's landing dir, so a user may
save real notebooks there; _clear_view now unlinks only the symlinks we
own and removes only folders that end up empty, leaving regular files
in place. It also tests islink before isdir, so a view that is itself a
symlink to a directory is unlinked instead of being walked into (which
would have wiped the symlink target).
- studio_launch.sh: derive the landing URL and preferred_dir from
UNSLOTH_NOTEBOOKS_VIEW_DIR / UNSLOTH_SKIP_NOTEBOOK_VIEW, the same env
the sync script uses, instead of hard-coding /workspace/Unsloth
Notebooks. A relocated or disabled view no longer opens JupyterLab on a
missing folder; it falls back to the default /lab over /workspace.
- Dockerfile.studio: the labext-builder stage now installs Node 20 from
NodeSource. Ubuntu 24.04's distro nodejs is 18, below JupyterLab 4.6's
declared Node >=20 engine. Node stays confined to the throwaway builder
stage, so the runtime image is unchanged.
- .dockerignore: explicitly allowlist jupyter/install_sloth_stickers.py
alongside its sibling jupyter assets, rather than relying on the
directory re-inclusion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* docker: publish lean image as :core and full image as :studio
Complete the base->core (and "studio as studio") tag rename so the publish
workflow matches the user-facing helpers and the Dockerfile.studio header.
- The lean training image now publishes as :core (core-<tag>, core-nightly,
core-sha-*); run.sh / docker_confirm.* already told users to pull :core, but
docker-publish.yml still tagged it :base, so that pull would have 404'd. The
per-arch digest artifacts are renamed to match.
- The full Studio image keeps :latest and gains a stable :studio alias, matching
the Dockerfile.studio header.
Both the merge and post-publish smoke-test metadata blocks are updated together.
Internal "base image" wording (the layer Studio builds FROM) is left as-is.
* docker: address second-round review feedback on the JupyterLab/Studio UX
- studio_launch.sh: also gate the categorized-view landing URL on
UNSLOTH_SKIP_NOTEBOOK_SYNC (the entrypoint skips building the view entirely in
that mode), not just UNSLOTH_SKIP_NOTEBOOK_VIEW, so a no-sync container does not
land on a missing folder.
- Dockerfile.studio: scope the sticker-install "|| echo" fallback to only the
sticker step via a { ...; } group. It was attached to the whole branding &&
chain, so a failure in a REQUIRED step (JS resolve, favicon/logo/login copy)
was swallowed and the build continued with broken branding.
- unsloth_nb_view.py: when creating the categorized symlinks, only replace our
own stale symlinks; if a real user file already occupies that name, keep it and
skip the link instead of os.remove-ing it.
- overrides.json: drop doNotDisturbMode (it silenced ALL JupyterLab toasts,
including kernel-restart / connection-drop feedback). The news/update prompts
are already off via fetchNews / checkForUpdates.
- Dockerfile: keep decord mandatory on amd64 (fail the build on a missing or
incompatible wheel) and only fail-soft on arm64/other arches that have no wheel.
- cellNav.ts: do not hijack ArrowUp/Down when focus is in an interactive output
widget / form control, or while a completion popup is open, so ipywidgets
controls and autocomplete at cell boundaries keep working.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* docker: keep Studio branding RUN free of comments inside the line continuation
Move the sloth-sticker fail-soft explanation above the RUN so no comment line
sits between backslash-continued commands. BuildKit strips such comments, but
keeping the RUN body a plain && chain removes the ambiguity for non-BuildKit
builders and static linters. The { ...; } fail-soft scoping is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* docker: AGPLv3 attribution + integrity guard for the Studio/JupyterLab image
Make it obvious the image is built by Unsloth and hard to white-label out with a
shallow find-and-replace, and surface the AGPLv3 license + copyright in the UI.
Visible attribution (labextension):
- Help > "About Unsloth Docker Studio" dialog (about.ts): Unsloth logo, the
AGPLv3 notice, "Copyright 2026-Present the Unsloth team", and source/website/
license links. Added to the Help menu and the command palette.
- The JupyterLab loading splash is replaced with a spinning Unsloth logo
(splash.ts, provides ISplashScreen; honors prefers-reduced-motion). The stock
@jupyterlab/apputils-extension:splash is disabled+locked at build time, like
the stock logo.
- AGPLv3 footer (license + copyright + links) on the branded login page.
- Labextension relicensed AGPL-3.0-only; SPDX headers on every source file.
Anti-tamper (no encoded/obfuscated strings -- plain readable text only; the one
data URI is the logo image):
- A canonical, plain-text attribution set lives in unsloth_branding.py with a
TypeScript mirror (branding.ts) bundled verbatim into the labextension, so the
phrase, copyright, links and plugin ids are spread across independent layers.
- unsloth_branding.py verifies all of these across the installed files (AGPLv3
text, login footer, theme, labextension package + built bundle strings, logo,
favicon) and fails loudly if any are missing. It runs at three layers:
build time (fails the image build), the whole-container launcher
(studio_launch.sh refuses to start), and as a jupyter_server extension
(refuses to serve JupyterLab).
- tests/studio/test_branding_guard.py: positive + per-marker negative coverage,
plus a check that no base64/decoder obfuscation crept into the attribution.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* docker: address #6681 review round 2 (colab magics, output select, branding guard)
- unsloth_colab_compat.py: only hoist a leading `%%` cell magic above the Colab
`#@title` form for magics whose body runs as code (capture/time/bash/python/
...). Content magics (%%writefile, %%html, %%latex, ...) are left untouched so
the form comment is never injected into the written file / rendered output.
- outputSelect.ts: stop trusting the text selection anchor to decide ownership
of Ctrl/Cmd+A. A stale selection inside an output survives a click onto a
command-mode cell or the file browser, which made select-all keep re-selecting
the old output. Gate on the keystroke target or the last pointer-down (reset to
null on any click outside an output) instead.
- unsloth_branding.py: also reject page_config.json that disables the Unsloth
labextension or any of its plugin ids via disabledExtensions (dict or list
form); that leaves the bundle on disk so the prior checks passed while the
logo/About/splash attribution was stripped at load. Lock unsloth-jupyterlab in
Dockerfile.studio as well (defense in depth), and add guard tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* labext: pin JupyterLab extension deps; confirm.ps1 /login probe
Pin the unsloth-jupyterlab npm deps to exact versions matching the baked
jupyterlab==4.6.0 (builder stays 4.5.9, its newest release) instead of floating
^/~ ranges, so the same commit always builds the same labextension bundle.
Also probe JupyterLab /login (not /api, which 403s behind a password hash) in
the Windows confirmation script.
* docker: categorize AMD/domain notebooks and wire the feature validation into CI
unsloth_nb_view.parse_readme only reset the folder section on level-3
(###) headings. The notebooks README carries level-1 domain headers
(# AMD Notebooks, # Kaggle Notebooks) with their own nb/*.ipynb link
tables and no intervening ###, so those notebooks were mis-filed under
the previous stale section (all 148 AMD notebooks landed in Other
Notebooks on an --amd build). Reset on any heading level and strip a
leading emoji/symbol run so the domain notebooks get their own clean
folder.
Also run tests/validate_studio_features.py explicitly in the repo CPU
job. It is named validate_* (not test_*) so pytest never collected it,
which meant a regression in the notebook view, Colab compat, strip,
JupyterLab defaults or login branding failed CI only when run by hand.
* labext: use caret ranges so jlpm dedups JupyterLab/Lumino singletons
The exact pins introduced earlier (@jupyterlab/* 4.6.0, @lumino/widgets
2.8.0, @jupyterlab/builder 4.5.9) break the Dockerfile.studio
labext-builder stage. Exact-pinning the framework packages defeats
jlpm's (yarn classic) hoisting: transitive @jupyterlab deps request
caret ranges that resolve to newer patch releases (e.g. @jupyterlab/
notebook pulls @jupyterlab/cells ^4.6.0 -> a newer patch), so jlpm
installs a second nested copy alongside the exact top-level one. Two
copies of @jupyterlab/cells and @lumino/widgets in the tree produce
TS2345 "not assignable" errors (protected-member/identity mismatch)
and the build fails.
Caret ranges let jlpm collapse every @jupyterlab and @lumino package
to a single hoisted copy, which is required for a JupyterLab prebuilt
(federated) extension: at runtime those packages are shared singletons
provided by the host JupyterLab, so the build-time versions only need
to type-check against one consistent tree, not match an exact runtime
patch. This is the version set the published image was built and
validated with end to end.
Verified by building the labext in isolation against the base image
(Node 20 + bundled jlpm): caret ranges build clean (webpack compiled
successfully); the exact pins fail with the duplicate-package TS
errors.
* ci(studio-backend): trigger on docker/** so the JupyterLab feature validation guards docker-only changes
The 'Docker JupyterLab/notebook feature validation' step runs
tests/validate_studio_features.py, which checks docker/jupyter (the
labextension, overrides.json, login branding) and the docker notebook
helpers. The pull_request paths filter listed studio/unsloth/tests but
not docker/**, so a PR that only touches docker/ would skip that step
and a regression in those files could pass CI. Add docker/** so the
validation runs whenever the files it checks change.
* jupyter: center the login card and place the attribution below it
#site was a flex container using the default row direction with two children
(the login card and the AGPLv3 attribution), so they rendered side by side:
the card sat left of centre and the attribution floated up to the top-right.
Stack them in a column so the card is horizontally centred and the attribution
sits below it as a footer, matching the intended single-column layout.
* jupyter: refresh Studio attribution, About dialog and loading splash
- Attribution now reads 'Built by the Unsloth team' with a single Apache 2.0 /
AGPLv3 license link (to the repo license section) on the login page and in the
About dialog, replacing the plain 'Built by Unsloth. Licensed under the GNU
AGPLv3.' line. The integrity guard, its canonical PHRASE and the branding tests
are updated to match.
- About dialog: left-align the link rows so the labels line up instead of each
row centering independently; add an 'Unsloth Reference' link to the docs, and a
Licenses section listing Unsloth Studio (AGPLv3) and Unsloth Core (Apache 2.0)
alongside the full license link.
- Loading splash now reads 'Loading Unsloth Docker' instead of the attribution
label, via a dedicated SPLASH_LABEL constant.
* docker: document the branding attribution as an AGPLv3 Section 7 notice
Add docker/NOTICE and docker/jupyter/BRANDING.md so the Unsloth attribution that
unsloth_branding.py enforces is also a written license condition, not only a
build check. docker/NOTICE designates the attribution (the "Built by the Unsloth
team" label, the copyright line, the license notice, the logo and theme, and the
Help > About links) as required Appropriate Legal Notices under AGPLv3 Section
7(b), referencing /studio/LICENSE.AGPL-3.0 and /LICENSE. BRANDING.md is a
human-readable note next to the guard describing what must stay, where it lives
and how it is enforced.
* ci(studio-backend): restore docker/** trigger path
The docker/** pull_request path added in b558bc7d was dropped by a later
rebase, so the "Docker JupyterLab/notebook feature validation" step (which runs
tests/validate_studio_features.py against docker/jupyter branding and notebook
helpers) no longer ran on PRs that only touch docker/. Re-add docker/** so a
docker-only change is validated on the PR rather than only after merge to main.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Bump the base image torch triplet to torch==2.11.0 / torchvision==0.26.0 /
torchaudio==2.11.0 and the paired torchcodec to 0.11.0, and hold torch at
2.11.0 during the vLLM resolve so uv lands on the vLLM 0.20+ line that pins
torch 2.11.0 (the split-install rationale already anticipated the bump). Update
the build-time self-test assertion, its status line, and the test_locally.sh
log grep to match, plus the FA2 wheel note.
Also clarify the advertised architecture support: forward-compatible SASS
covers precompiled kernels on sm_103 (B300/GB300), but runtime Triton/NVRTC JIT
targets the actual device cap and the bundled cu12.8 ptxas/NVRTC cannot emit
compute_103. arm64 sm_121 is handled by the cu13 NVRTC/ptxas override; amd64
sm_103 has no cu13 override yet, so JIT-heavy paths there can fail until it
lands. Precompiled SASS still runs on sm_103 via sm_100 forward-compat.
The bundled launcher only forwarded HF/W&B/license/CPU vars, so the documented
Studio service config read by studio_launch.sh was silently dropped when running
the full image through this wrapper: JUPYTER_PASSWORD fell back to a random
password, PUBLIC_KEY/SSH_KEY never enabled sshd, and UNSLOTH_JUPYTER_CLOUDFLARE
never started the tunnel. Forward them with the same dash-only -e VAR form as the
secrets above, so the value is read from the parent env and never lands in argv.
pip shim (docker/unsloth_pip_shim.py):
- Drop protected packages named via a VCS/URL #egg=NAME fragment so
git+... #egg=torch no longer reinstalls into the baked venv.
- Filter constraint files (-c/--constraint) through the same protected
package filter as requirement files, so a pinned torch/transformers in
a constraint cannot downgrade the baked stack during resolution.
- Recursively filter nested -r/-c includes and absolutise their paths so
the filtered /tmp copy still resolves them and no protected spec deep in
the include tree slips past the keep list.
- Remove an unused subprocess import.
Notebook environment:
- Scope the transformers-request marker per kernel (UNSLOTH_NB_TF_MARKER
keyed on the kernel connection-file id) so concurrent notebooks no
longer read each other's pin.
- Install the IPython startup hook under IPYTHONDIR (set via ENV) so it
loads for any uid, including docker run --user, not just root.
- unsloth_nb_content_sig.py: only treat a %%capture / %%bash cell as
install boilerplate when it carries an install command, so substantive
captured/bash cells are hashed and upstream changes are not skipped.
- unsloth_run.py: clean up the temp dir used to materialise a downloaded
notebook.
- unsloth_sync_notebooks.sh: honor UNSLOTH_KEEP_DELETED_NOTEBOOKS across
GitHub refreshes so a deleted notebook is not restored when upstream
advances.
install_python_stack.py: the --local unsloth-zoo overlay now honors
UNSLOTH_ZOO_REF (default main), matching the install.sh overlay.
synthetic.py: preserve the timeout=None unbounded vLLM startup wait
instead of coercing it to 1200s.
A Jupyter password hash is always configured, so /api returns 403; the Windows
confirmation reported a healthy full image as a hard failure. Matches the fix
already in docker_confirm.sh and docker-publish.yml.
- docker-publish smoke + docker_confirm.sh probe Jupyter /login, not /api: the
launcher always configures a password hash so /api returns 403 and curl -f
would never flip the health flag (false build failure).
- entrypoint.sh CPU messaging: CPU mode covers Jupyter, GGUF tooling and
llama.cpp (GGUF) Studio chat; training AND loading an Unsloth model
(FastLanguageModel) still need a GPU, since from_pretrained runs CUDA probes.
- install_llama_prebuilt.py: rollback/activation moves used bare os.replace,
which fails with EXDEV across overlayfs in a Docker build and fell back to a
broken source build (no nvcc). Add is_cross_device_error + move_install_dir_aside
(os.replace fast path, copy+remove on EXDEV; busy errors still re-raise).
- notebooks: %pip / %uv line magics and the `!python -m pip` form bypassed the
PATH pip/uv shim and could overwrite the baked cu128 torch/vLLM stack. Add
unsloth_nb_pip_magic.py to re-point them at the shim, wired via the IPython
startup hook and installed into the venv site-packages.
Two more notebook-shim gaps from review:
- A quoted PEP 508 direct reference for a protected package, e.g.
`pip install "torch @ https://.../torch.whl"` or `"unsloth @ git+https://..."`,
bypassed _KEEP: _canon hit the url guard and returned None before pulling the
distribution name, so the token was treated as a real target and reinstalled
into the base venv. _canon now extracts the name from the `name [extras] @ url`
form first, so a protected package pinned through a URL/VCS is still dropped; a
non-protected direct reference returns its name and is kept exactly as before.
- The `--requirement=reqs.txt` equals-form (pip accepts `--option=value` for any
value-taking flag) was not recognized: the token starts with `-`, so it was
kept as an opaque option, the file was never filtered, and has_target stayed
false -- a cell whose only target was that file silently no-op'd. The scan now
splits `--flag=value`, filters the requirements file for `-r`/`--requirement`,
and counts it as a target; other inline-value options stay options.
- unsloth_pip_shim.py: filter protected packages out of a notebook
`pip install -r requirements.txt`. The -r value was passed to the real pip
unchanged, so torch / transformers / vLLM / nvidia pins inside the file could
overwrite the baked cu128 stack or push transformers into the base venv.
_filter_requirements_file() applies the same _KEEP / transformers-sidecar
rules per line, writes the survivors to a temp file, keeps comments, option
lines, nested includes and urls verbatim, and records a pinned transformers
version for the sidecar.
- install.sh + Dockerfile.studio + docker-publish.yml: forward the resolved
unsloth-zoo ref into the Studio build. install.sh --local overlaid
unsloth-zoo from git main regardless of the operator-requested or base-image
ref, so the full image could run a different zoo than the base. install.sh
now honors UNSLOTH_ZOO_REF across all four --local overlays, Dockerfile.studio
passes UNSLOTH_STUDIO_ZOO_REF through to it, and the workflow resolves one zoo
ref in the prepare job and shares it with both the base and Studio builds.
- Dockerfile + docker-publish.yml: pin unslothai/notebooks to one resolved
commit. Each arch leg cloned HEAD independently, so the same tag could seed
different baked templates and .unsloth_template_commit depending on the pulled
platform. The prepare job freezes notebooks to one sha (like the llama.cpp
prebuilt tag) and the Dockerfile fetches that single ref at depth 1.
- pip shim: do not treat the value of an index-url / find-links / constraint flag
as an install target. A cell like 'pip install --extra-index-url <url> torch'
now no-ops after keeping the baked stack instead of exec'ing a bare
'pip install --extra-index-url <url>' that fails. Positional . / url / vcs and
-r/--requirement files still count as targets.
- notebook sync: on first boot, record only files we actually wrote (or that are
byte-identical to the template), never a kept pre-existing user file; and on the
GitHub refresh, treat a file present in DEST but absent from the sync state as
user-owned and keep it. Previously a bind-mounted notebook was recorded as
managed and then overwritten by upstream.
- docker-publish: add flavor latest=false to the Studio metadata steps too, so a
v* tag push cannot emit an implicit :latest via metadata-action's latest=auto;
:latest stays default-branch-only, and the smoke test pulls the published tag.
- unsloth-studio-update: resolve the unsloth-zoo ref independently of --ref (new
--zoo-ref, else use the ref only when the zoo repo has it, else fall back to
main) so 'update --ref <unsloth-tag/sha>' does not fail on a missing zoo ref.
- Dockerfile: drop 10.3 (compute_103) from TORCH_CUDA_ARCH_LIST in both the
builder and runtime stages. B300 runs sm_100 SASS, and the bundled CUDA 12.8
nvcc cannot compile compute_103 (added in 12.9), which broke arch-list-honoring
source / JIT builds.
- pip shim: count editable/local/url/vcs targets (-e ., ., git+https, wheel
URLs) as install targets, not just canonical package names, so they are no
longer silently skipped inside notebooks
- notebook sync: never overwrite a pre-existing user notebook on first boot
(match the refresh path's ownership rule); skip .unsloth_sync_state.tmp when
recording state so it is not tracked as a managed file
- docker-publish: set flavor latest=false on the base image metadata so a v*
tag push cannot publish :latest from the base image (the Studio image owns it)
- notebook deps: pin to tested versions and install decord on its own, hard on
amd64 and fail-soft on arm64 (no aarch64 wheel) so the arm64 base build works
Mirror the public-link convenience Studio already has for its own UI, for
JupyterLab. Off by default; opt in two ways:
docker run -e UNSLOTH_JUPYTER_CLOUDFLARE=1 ... unsloth/unsloth
docker exec <container> unsloth-jupyter-tunnel --force
unsloth-jupyter-tunnel waits for JupyterLab, reuses a cached cloudflared (or
fetches the static binary for the arch, no account needed), and starts a
quick tunnel to the Jupyter port; the https://<name>.trycloudflare.com URL is
printed to docker logs. supervisord runs it as the jupyter-cloudflare program,
autostarted only when UNSLOTH_JUPYTER_CLOUDFLARE=1 (studio_launch.sh exports a
0 default so the autostart gate expands, matching the sshd pattern). JupyterLab
still enforces its password, so the tunnel is not an open door.
Verified: the helper fetches cloudflared and mints a working trycloudflare URL
that reaches JupyterLab (HTTP 200) inside a running container.
Parity with unsloth-studio-update: update the baked llama.cpp prebuilt in a
running container without pulling a new image.
docker exec <container> unsloth-llama-update # latest release
docker exec <container> unsloth-llama-update --check # report only
It reuses the build-time fetcher (fetch_llama_prebuilt.py, now baked at
/usr/local/lib/unsloth) rather than the host-probing installer behind the
in-app banner. The fetcher resolves the latest release via the GitHub
/releases/latest redirect (no API token, not rate-limited) and installs the
portable CUDA bundle that runs on CPU and every supported GPU, so it works the
same in a CPU-only or a --gpus container. The installer path, by contrast,
scans the GitHub API (rate-limited to 403 in practice) and probes the host GPU,
which falls back to a slow source build in a container started without --gpus.
The fetch lands in a sibling temp dir on the same filesystem and is swapped in
with an atomic rename; on any failure the existing install is left untouched.
The Studio ownership marker is preserved across the swap. Verified end to end
in a CPU-only container: b9596-mix-e6f2453 -> b9773-mix-1f1aaa4.
Updating Studio in the container previously meant pulling a fresh ~25GB image
(or at best the ~6GB fused Studio layer) for what is usually a small Python/UI
change. Add a baked helper so a running container can update in place:
docker exec <container> unsloth-studio-update
It updates only the Studio packages -- the backend code and the pre-built
frontend, which ships inside the unsloth wheel -- with `pip install -U
--no-deps unsloth unsloth_zoo`, then restarts just the studio service via
supervisor. The torch/CUDA stack is left untouched, so it is safe in both GPU
and CPU-only containers. This deliberately avoids `unsloth studio update`,
which re-runs the full installer and re-probes the GPU to pick torch wheels --
in a container started without --gpus that finds no GPU and can downgrade torch
to CPU/cu126.
Options: --ref <branch|tag|sha> installs from git (track main) instead of the
latest PyPI release; --with-deps also updates dependencies; --no-restart defers
the restart. After the swap the helper smoke-imports studio.backend.main and,
if a transitive dep is now missing, points the user at --with-deps.
The update lands in the container's writable layer (survives docker restart);
mount -v unsloth_studio_home:/opt/unsloth-studio to keep it across a recreate.
Two related changes to the baked llama.cpp prebuilt.
1. Dynamically follow the newest unslothai/llama.cpp release. build.sh resolves
the latest release tag (following the /releases/latest redirect, no API
token) to a concrete tag and passes it as LLAMA_PREBUILT_TAG, so the layer
cache busts only when upstream publishes. The Dockerfile default is now
"latest" and fetch_llama_prebuilt.py resolves it the same way, so a plain
`docker build .` also tracks latest. Pin LLAMA_PREBUILT_TAG to a concrete
tag for a reproducible, frozen build.
2. Make the in-app "newer llama.cpp available" banner work inside the image.
Studio's freshness check (utils.llama_cpp_freshness.check_prebuilt_freshness)
keys off tag / release_tag / published_repo in UNSLOTH_PREBUILT_INFO.json --
the schema install_llama_prebuilt.py writes. The image bakes the bundle
directly, so the marker was the release tarball's own, which only carries
upstream_tag / source_repo; the freshness check then bailed with
installed_tag=None and could never report "behind", hiding the banner.
fetch_llama_prebuilt.py now augments the baked marker with those keys
(setdefault, no build timestamp so the layer stays byte-identical). A fresh
build is on latest -> no banner; once upstream publishes a newer release the
banner appears, as verified against the real freshness backend.
The boot-time notebook sync now restores notebooks the user deleted, on
every boot, from the baked template (offline, even when upstream has not
advanced). It only restores files that are missing, so it never resurrects
or overwrites an edited notebook, and the GitHub refresh still bumps a
restored file to the latest upstream. Opt out with
UNSLOTH_KEEP_DELETED_NOTEBOOKS=1.
Also add unsloth_nb_content_sig.py to docker/.dockerignore's allowlist; it
was referenced by the Dockerfile COPY but excluded from the build context,
which broke the image build.
The boot-time refresh now compares only the tutorial body (the
non-boilerplate cells) when deciding whether to update an untouched
notebook. If only the install header, announcements, or footer moved
upstream, the user's file is left as-is so it is not churned. Notebooks
the user has edited or run are still kept untouched, and non-notebook
files keep the whole-file refresh. Adds unsloth_nb_content_sig.py to
segment head/middle/tail and bakes it into the image.
JupyterLab now opens with the unslothai/notebooks collection already present,
so people can open and run a notebook directly without a git clone or wget.
- Bake the repo into the image as a read-only template at /opt/unsloth-notebooks
(~206MB, .git stripped, build commit recorded). Inherited by the studio image.
- On boot the entrypoint populates /workspace/unsloth-notebooks from the template
(instant, works offline) and best-effort refreshes from GitHub, but only when
upstream has actually advanced (cheap git ls-remote gate, no download otherwise).
- The user's edits always win. We record the content hash of every file we write;
on refresh a file whose hash differs from what we last wrote is treated as
user-modified and is left untouched, so the refresh only updates files the user
has not changed and adds new ones. It never overwrites an edited notebook and
never produces merge conflicts. Verified: an edited notebook stays the user's
version across repeated upstream changes.
- Fully best-effort and gated: UNSLOTH_SKIP_NOTEBOOK_SYNC=1 disables it,
UNSLOTH_SKIP_NOTEBOOK_REFRESH=1 keeps the baked copy and never hits the network.
Offline boots keep what is there and never error.
base 18.45 -> 18.67GB, studio 24.88 -> 25.10GB (+~206MB baked notebooks).
Notebook dependency coverage (base Dockerfile):
- Bake omegaconf, einx, librosa, decord, ftfy so the TTS/STT and vision
notebooks stop dying on a silent No module named X. Installed in the
notebook-deps layer (after the torch/vLLM resolve) with an assertion that
the resolve did not move torch 2.10.0 / numpy>=2.3 / numba>=0.65.
Image size (no functional change):
- Base: prune npp to the two libs torchcodec actually dlopens
(libnppicc + libnppc), drop link-time-only .a archives and the nvshmem
device bitcode. Headers (torch/include etc) are kept so causal-conv1d /
mamba-ssm still build at notebook time with --no-build-isolation.
- Studio: pin the Studio venv to Python 3.12 (matches base) so its
nvidia-*-cu12 wheels are byte-identical to the base venv's, then symlink
the heavy arch-independent CUDA libs (cudnn/cublas/nccl/...) into the base
venv copy. cuda_nvrtc and cuda_runtime are excluded (the arm64 nvrtc swap
mutates nvrtc in place). Also remove the build-only frontend node_modules
(runtime serves the committed dist). Studio image drops ~4.8GB.
Per-notebook transformers version, run notebooks unchanged:
- Bake coherent transformers sidecars (4.57.6 default + 5.3.0/5.5.0/5.10.2),
each transformers==X with its matched huggingface_hub/tokenizers/
safetensors installed --no-deps into its own dir. Companion versions are
resolved at build time so they satisfy each transformers' requirements.
- unsloth_nb_compat.py: pick the sidecar from the notebook's pin or the
model name and activate it (prepend to sys.path) before any ML import,
without touching the base cu128 torch/vLLM/unsloth stack.
- pip/uv shim on PATH: a notebook install cell becomes safe and idempotent
inside a kernel (keeps the baked stack, records the requested transformers
for its sidecar); passthrough to the real tool everywhere else.
- IPython startup hook for manual JupyterLab, and unsloth-run for the
headless driven path.
