Both surfaced only once the staging matrix ran these suites on runners the
org queue does not cover. Neither is a product defect; both are tests
asserting something their environment cannot supply.
test_unsloth_pip_shim.py::test_forwarded_install_carries_protected_constraints
reads the ambient environment through importlib.metadata.distributions.
_protected_constraints_file correctly returns None when no protected
package is installed, so no --constraint pair is appended, and the test
then indexed execd[-2] unconditionally:
E IndexError: list index out of range
1 failed, 115 passed, 2 skipped
It failed on all four docker-test legs and in any bare venv, and passed
upstream only because studio-backend-ci installs torch and transformers
first. Its own sibling at line 93 already guards with len(execd) >= 2.
Guarding the index alone would have left the test measuring whatever
happened to be installed, so distributions() is now stubbed and the test
asserts the real contract deterministically. A second case covers the
other half of that contract, which is what a bare venv actually hits: with
nothing protected installed the install must still be forwarded, just
without the pair.
test_select_cuda_jit_tools.sh stages libnvrtc as symlinks and asserts
through readlink, because retargeting that symlink is what the function
under test does. git-bash copies instead of symlinking unless
MSYS=winsymlinks:nativestrict and the user is elevated, so readlink comes
back empty and all 14 assertions fail on both Windows runners, taking
tests/run_all.sh down with them for any Windows contributor. The code only
ever runs inside a Linux container, so probe for real symlink support and
skip when it is absent rather than assert something the filesystem cannot
represent.
Verified: the shim suite is 87 passed / 2 skipped in both a bare venv and
a full one; the shell suite still reports 14 passed on Linux and skips
with exit 0 under a simulated no-symlink filesystem.
The pip shim fronts pip/uv inside the notebook kernel so an install cell cannot
replace the baked cu128 stack, but _KEEP only covered torch/vLLM/unsloth. Across
the 433 shipped notebooks that left the training half wide open:
trl 382 pin an older release, 378 of them ending the install cell with
`pip install --no-deps trl==0.22.2`, against a baked trl 0.24.0
torchao 273 reinstall it and 2 pin 0.15.0, replacing 0.17.0+cu128
torchcodec 92 reinstall it and 26 pin 0.5 or 0.7.0, replacing the
0.11.0+cu128 wheel the Dockerfile pairs with torch 2.11
datasets 254 reinstall it, observed falling from 4.3.0 to 3.0.0
peft 225 reinstall it, observed falling from 0.19.1 to 0.14.0
accelerate 225 reinstall it
hf hub 240 reinstall it and tokenizers 64, both version-locked to
transformers and shipped in matched copies inside every sidecar
So every notebook run mutated the stack the image was validated with, while the
shim printed that it was keeping the baked versions.
The membership criterion is "replacing this invalidates the tested stack or
breaks unsloth", not "a notebook mentions it": snac, causal-conv1d, mamba-ssm,
omegaconf, protobuf, sentencepiece and the rest still install normally.
Verified in the rebuilt image by running the Gemma3 (270M) install cell verbatim:
trl 0.24.0, peft 0.19.1, datasets 4.3.0, accelerate 1.14.0, torchao 0.17.0+cu128,
transformers 5.14.1 and huggingface_hub 1.24.0 are all unchanged afterwards, the
requested transformers pin is still recorded for the sidecar, and a package the
image does not bake still installs.
The existing shim tests used peft as their "unprotected package" sentinel, so
they move to snac.
Condense the verbose explanatory comments added by this branch to their essential
points without dropping any load-bearing rationale. Touches comments and
docstrings only, no code changes. Leaves the stable-tag gate rationale, the
byte-identical enable= sync notes, and the update-alternatives pin comment as is.
A VCS @ref can itself contain a slash (@feature/foo), and the shim split the
last path segment BEFORE dropping the ref, so
git+https://github.com/unslothai/unsloth.git@feature/foo canonicalized as
"foo" and a protected repo installed from a branch dodged _KEEP. The ref is
now stripped from the path portion first (after the authority, so an SSH
userinfo @ is never mistaken for the ref separator, matching pip's own
last-@ parsing), with regressions for slash refs, SSH userinfo, plain tags
and the no-ref form.
The help-derived value-flag drift guards were version-sensitive: repo CI runs
whatever pip/uv are current, so the next tool release turned unrelated PRs
red (pip 26 added --all-releases/--only-final/--requirements-from-script/
--uploaded-prior-to, uv added --no-editable-package/--upgrade-group; all six
now classified). The guards are opt-in for local runs
(UNSLOTH_SHIM_FLAG_DRIFT_CHECK=1) and the authoritative check now runs at
image build time via a new --unsloth-selfcheck-value-flags mode wired into
the Dockerfile verify step, where the baked pip/uv are exactly the tools the
shim fronts, so a flag added by a future baked-tool bump fails the build
instead of a user's notebook cell.
Preflight (entrypoint.sh) now scans every visible device: an unsupported device
0 stays fatal as before, an unsupported secondary GPU (mixed rig) warns at
startup with its index and the CUDA_VISIBLE_DEVICES remedy, instead of
surfacing only when a job pins to it or a multi-GPU launch fans out.
The pip shim's _VALUE_FLAGS now covers every value-taking flag of uv pip
install and pip install (generated from both tools' --help). The separated
form `uv pip install --torch-backend cu128 torch` used to drop the protected
torch but exec uv with no install target at all (uv hard-errors) instead of
no-oping like the attached `=` form, and `--extra torch peft` misread the
extra name as a protected target, leaving a dangling --extra that swallowed
peft. Adds parametrized regressions plus help-derived drift guards so a future
uv/pip value flag cannot silently reintroduce the misparse.
unsloth-llama-update now detects when the install dir is itself a mount point
(the documented -v unsloth_llama:/opt/unsloth/llama.cpp persistence recipe,
where rename(2) fails EBUSY) and swaps the bundle CONTENTS inside the mounted
tree, so the update lands in the volume and stays persistent. Work and backup
dirs live under the mount (same-fs renames), the abort trap restores the old
contents even mid-swap, and the non-mounted path keeps the whole-dir rename.
Verified: in-place swap preserves the dir inode and ownership marker, failed
fetch leaves the install untouched, simulated mid-swap abort restores fully.
Comment-only consolidation: the sm_103/sm_121 + cu13 JIT story and the
xformers-aarch64 note were each told four times across docker/Dockerfile; keep
the header telling canonical and cross-reference it elsewhere (same for the
workflow's six retellings of the resolve-refs-once rationale and
Dockerfile.studio's NVRTC block). Comments that pointed at the removed dev
scripts now name the underlying command or artifact instead. Non-comment lines
of both Dockerfiles and the workflow are byte-identical.
unsloth_sync_notebooks.sh folds the three copies of the override -> PATH ->
sibling helper resolution into one resolve_helper(), behavior verified for all
four modes including graceful absence under set -u.
unsloth_pip_shim.py collapses an if/else whose branches were identical and
merges the structurally duplicate _parse_include/_parse_editable into one
_parse_flag_line. The test suite folds 35 near-duplicate tests into 8
parametrized groups with exact case-count parity (69 collected before and
after, 81 passing including the nb-pip-magic suite).
Cuts another 144 lines with zero behavior change outside the two refactors.
Every new .py this PR adds now carries the same two-line SPDX header the other
new files in the branch already use (docker/jupyter/unsloth_branding.py), with
the shebang kept first where present. Matches the licensing laid out in
docker/NOTICE: the image bundles Studio (AGPL-3.0) while Unsloth Core stays
Apache-2.0.
Four follow-ups to the shim/entrypoint audit fixes:
1. unsloth_pip_shim.py let a local project directory install through: `pip
install ./transformers` / `-e ./unsloth` is not a requirement spec, so
_canon returned None and both the arg filter and the constraints file
(which only rejects a version MISMATCH) passed it, letting a same-version
local build silently replace the baked wheel. _canon now resolves the
project name from pyproject [project].name, then setup.cfg, then the
directory basename when it is an installable project, so a local checkout
of a protected package is dropped like every other artifact form. Names
match exactly after normalization, so a user dir named my-torch-utils is
untouched, and a metadata-less directory still passes through.
2. unsloth_nb_pip_magic.py only rewrote literal `!python -m pip`, so the
`!{sys.executable} -m pip ...` form notebooks use to target the running
kernel (and absolute interpreter paths) bypassed the PATH shim entirely.
Input transformers see the raw cell text before IPython expands the
braces, so the matcher now also covers {sys.executable} (quoted or bare)
and quoted/bare interpreter paths ending in python[0-9.]*(.exe) before
-m pip|uv.
3. unsloth_pip_shim.py did not strip uv's --exact, which performs an exact
sync that removes every installed package outside the kept target's
closure (vLLM, bitsandbytes, the NVIDIA libs); `uv pip install --exact
peft` would strip the baked stack after the filter kept it. --exact now
joins the resolver-wide destructive flags dropped in shim mode.
4. entrypoint.sh: the arm64 image bakes a CUDA 13 llama.cpp because upstream
(unslothai/llama.cpp) publishes no CUDA 12 arm64 asset, while the torch
stack (cu128) runs on a 570-series driver. A CUDA 13 cubin cannot load on
a 570-579 driver, so on GH200/GB200 hosts below 580 GGUF export and Studio
chat fail while training works. The entrypoint now warns up front on
aarch64 + driver < 580 instead of letting llama-server fail later.
Tests: shim + nb-pip-magic suites at 81 (18 new, including local-project
name resolution, the executable/brace forms, and --exact stripping).
Ten verified fixes from a 12-reviewer audit of the image tooling, each
reproduced before fixing:
1. install_llama_prebuilt.py move_install_dir_aside: the EXDEV fallback
copied straight into the rollback path, so a copy that died halfway
(ENOSPC, I/O error) left a partial tree that activation recovery would
later restore over the intact install while deleting the good copy.
Copy to a temp sibling and publish with one atomic rename; dst.exists()
is now a truthful complete-tree signal.
2. unsloth_run.py --out truncated the existing output before nbconvert
ran, so a timeout, missing kernel, or failed cell irreversibly
destroyed the previous result. The input copy and executed result are
staged as temp files next to the destination and published with
os.replace only on exit code 0.
3. unsloth_nb_view.py cleanup treated every symlink in the view as its
own: user-created links (and an operator's view-root routing symlink)
were deleted on every rebuild. Cleanup now removes only links that
resolve into the notebooks tree it links from, and builds inside a
view-root symlink's target instead of unlinking it.
4. unsloth_llama_update.sh: the unconditional EXIT trap deleted the .old
backup even when it was the only remaining copy (signal between the two
renames, or a failed swap whose restore also failed). The handler now
restores the backup first when the install dir is missing and removes
it only after the new tree is verifiably active; HUP/INT/TERM route
through the same handler.
5. unsloth_pip_shim.py: transitive dependencies could replace the baked
torch stack (reproduced with a wheel requiring torch==99.0). Every
forwarded install now carries a constraints file pinning the installed
protected set, turning the swap into ResolutionImpossible.
6. unsloth_pip_shim.py: ${UPPER} env references in requirements files were
classified before pip expanded them, bypassing the protected-package
filter; the shim now expands with pip's exact regex first.
7. unsloth_pip_shim.py: a failure writing the filtered requirements copy
returned the ORIGINAL file, forwarding exactly the protected pins it
had detected; it now fails closed.
8. docker-publish.yml: workflow_dispatch defaulted unsloth_ref to 'main'
while the stable-tag gates require '', so UI-default manual runs could
never advance :core/:latest/:studio; the default is now empty.
9. entrypoint.sh: the sm_103/sm_121 branch rewrote libnvrtc.so.12 to the
CUDA-13 build but the ordinary-GPU branch never restored it, so a
container moved to an older GPU kept the stale link; it is now reversed
when it points exactly at the .cu13 target.
Rejected after verification (no code change): timeout=0 semantics are
documented at the site with no zero callers, TORCHINDUCTOR_COMPILE_THREADS
override is deliberate, fetchNews is a string enum per JupyterLab's schema,
:base tag appears in no in-tree doc, install-cell digest exclusion is the
module's stated contract, transformers ceiling semantics are documented,
and the cloudflared download mirrors the pre-existing Studio downloader
(Cloudflare publishes no checksum asset). The UNSLOTH_ALLOW_CPU import
crash lives in unsloth_zoo (compiler.py / loss_utils.py capability probes),
not in this diff; the image consumes the zoo fix automatically once merged
there.
Tests: shim suite extended to 63 (constraints, env expansion, fail-closed),
jit-selector suite to 14 (NVRTC reversal transitions), plus staged-publish
and ownership repros; wider studio install suite green except failures
reproduced at the unmodified head.
Notebook pip/uv shim (docker/unsloth_pip_shim.py, active only under
UNSLOTH_NB_SHIM=1):
- Parse protected source archives (sdist/zip) by basename too, e.g.
`pip install https://.../unsloth-2026.7.1.tar.gz` or `./torch-2.11.0.tar.gz`,
mirroring the wheel-basename handling. A first-hyphen-before-digit split
keeps hyphenated names like flashinfer-python intact.
- Recognise uv's PLURAL long flags --requirements / --constraints, so those
files go through the same protected-package filter as the singular names.
- Drop --upgrade-strategy eager in shim mode so a kept target cannot eagerly
rebuild already-satisfied baked deps (falls back to pip's only-if-needed).
NVRTC default (docker/Dockerfile, docker/Dockerfile.studio, docker/entrypoint.sh):
- Make cu12.8 the immutable baked default (libnvrtc.so.12 -> .cu128.orig) with
a staged .cu13 alias, and have select_cuda_jit_tools retarget to cu13 ONLY
for sm_103/sm_121. Previously cu13 was baked as the default and restored to
cu12.8 at runtime, so a non-root `docker run --user` container that cannot
rewrite the symlink stayed on cu13 NVRTC and emitted cubins a 570-579 driver
cannot load. The safe default now needs no runtime write.
Adds regression tests for each case (tests/python/test_unsloth_pip_shim.py,
tests/sh/test_select_cuda_jit_tools.sh).
Notebook pip/uv shim (docker/unsloth_pip_shim.py), all active only under
UNSLOTH_NB_SHIM=1:
- Parse a bare wheel filename (torch-*.whl in the CWD, no ./ or / prefix) so
it is matched against _KEEP instead of passing through as an opaque
positional and reinstalling the baked torch.
- Infer the distribution from an egg-less VCS URL by repo basename
(git+https://github.com/huggingface/transformers.git -> transformers) so
the egg-less form the repo itself recommends cannot clobber the baked stack.
- Refuse remote (URL) -r/-c requirement/constraint files -- top-level and
nested includes -- since their pins cannot be inspected before the real
tool would fetch and install them.
- Strip resolver-wide reinstall/ignore-installed switches
(--force-reinstall, --ignore-installed, -I, uv --reinstall) so they cannot
rebuild already-satisfied baked deps pulled in by a kept target.
- Route uv --reinstall-package through the same _KEEP handling as
-P/--upgrade-package (both attached and separated forms; no dangling flag).
Entrypoint (docker/entrypoint.sh): select_cuda_jit_tools() now scans every
visible GPU's compute_cap instead of only the first, so a datacenter Blackwell
(sm_103/sm_121) behind an H100/B200 still enables the cu13 JIT tools it needs.
Adds regression tests for each case (tests/python/test_unsloth_pip_shim.py,
tests/sh/test_select_cuda_jit_tools.sh).
unsloth_pip_shim.py: close three more ways a protected package slipped past
_KEEP. An editable line (-e/--editable <target>) inside a -r requirements file
is a real install target, so a protected editable there is now classified and
dropped like the command-line case (new _parse_editable). pip/uv accept the
attached short forms -rreqs.txt / -cconstraints.txt / -epath / -Pname as one
token; these were falling through as opaque options (so an attached -r-only cell
no-op'd and an attached -c/-e/-P value bypassed _KEEP), so the 2-char flag is now
split from its value and routed through the separated-form handling. And a nested
-c constraint inside a -r file no longer records its transformers pin as an
install request (a constraint is not a request; mirrors the top-level -c path).
entrypoint.sh / Dockerfile: gate the CUDA 13 ptxas + NVRTC to sm_103 / sm_121 at
runtime instead of a global build-time default. A cu13 cubin needs a >= 580
driver to LOAD even when it targets an older arch (CUDA has forward, not
backward, cross-major driver compatibility), but the image supports Turing..
sm_120 on a 570+ driver, so the previous global TRITON_PTXAS_PATH ENV + cu13
NVRTC symlink would break ordinary Triton/NVRTC JIT on 570-579 driver hosts. The
build still bakes cu13 (saving the cu12.8 NVRTC as .cu128.orig); a new
select_cuda_jit_tools() in the entrypoint reads the device compute_cap and only
activates cu13 for sm_103/sm_121 (which ship >= 580 drivers), otherwise leaving
Triton on its bundled cu12.8 ptxas and restoring the cu12.8 NVRTC in both the
base and Studio venvs. The base ENTRYPOINT runs for the Studio image too.
Adds 9 pip-shim regression tests and tests/sh/test_select_cuda_jit_tools.sh
(7 device-gating cases); registers the latter in CI and tests/run_all.sh.
docker-publish.yml: freeze the requested unsloth ref to one sha in the prepare
job before the matrix fans out. UNSLOTH_REF / UNSLOTH_STUDIO_REF were raw
expressions re-evaluated per base arch leg and in the Studio build, so a mutable
branch (the workflow_dispatch default unsloth_ref=main) advancing during the run
could bake different unsloth commits under one manifest. Resolve once (same
precedence: dispatch input, else pushed tag, else triggering sha, else main;
ls-remote a branch/tag to a sha, mirroring the zoo/notebooks steps) and read
needs.prepare.outputs.unsloth_ref everywhere.
Dockerfile.studio: run the Studio venv NVRTC cu13 swap on both arches, not arm64
only. amd64 sm_103 (B300/GB300) needs cu13 NVRTC just as arm64 sm_121 does, and
the CUDA dedup never touches cuda_nvrtc, so an amd64 Studio venv would otherwise
keep its bundled cu12.8 libnvrtc and fail NVRTC/jiterator JIT on compute_103. The
base cu13 layer installs cuda-nvrtc-13-0 on both arches, so the target .so.13
exists here regardless of TARGETARCH.
unsloth_pip_shim.py: close three ways a protected package slipped past _KEEP.
Treat -e/--editable as a value-taking flag paired with its target and drop both
when the target is protected (was leaving a dangling -e that failed the cell);
filter -P/--upgrade-package values through _KEEP (a named baked package could be
refreshed while installing another target); and parse the PEP 427 distribution
name out of a wheel URL/path so a bare `pip install https://.../torch-...whl`
drops instead of reinstalling the baked torch. Non-protected editables, upgrade
selectors, and wheels are unchanged. Adds tests/python/test_unsloth_pip_shim.py
(18 regression tests, exec captured via a patched os.execv).
