The macho check asserted a valid signature for every Mach-O under the studio
home, and failed the macos-15 mask/pipe leg on 29 files: lxml, charset_normalizer,
cygrpc, _upb, fontTools, caio, brotli and a bundled libportaudio.dylib. Those are
MH_BUNDLE and MH_DYLIB images dlopen'd into a process without library validation,
they ship unsigned in the wheels, and the same run had already installed and
imported them with the installer exiting 0.
Key the signature half off the Mach-O filetype and run it only on main
executables. Report an absent seal separately from one that fails to verify, and
capture codesign output instead of piping it into grep, which returned the
unsigned exit status through pipefail and called every unsigned binary broken.
The architecture half is unchanged and still a hard failure: it is what closes
the Rosetta 2 gap. The zero-Mach-O guard is unchanged. The .venv_t5_* sidecars
stay in scope; setup.sh creates them during a normal install and
transformers_version.py puts them on sys.path, so they are payload.
Three red checks, two of which test something this branch does not own.
desktop linux deb / appimage run the SHIPPED bundle's own install.sh, and
desktop-v0.1.50-beta was cut on 2026-07-21, before #7547 merged on 07-29. That
bundle still carries the old optional-dependency gate, so on a stripped runner it
exits 2 at [TAURI:NEED_SUDO] cmake git build-essential libcurl4-openssl-dev and
never creates a venv. Current main's _check_linux_deps runs the same set through
_SMART_APT_OPTIONAL, which suppresses every escalation path, so only a new release
can change this. The step now pins that exact outcome: the exit code must be 2 and
the log must carry exactly that package list, anything else still fails, and
finding _SMART_APT_OPTIONAL in the extracted install.sh (the guard #7547 added)
turns into a hard error saying to delete the pin. The venv and torch assertions
stay and still run whenever the installer succeeds.
win windows-11-arm gets a native ARM64 CPython, and torchaudio publishes no
win_arm64 wheel at any version, so the PyTorch step cannot resolve. The fix is in
install.ps1 on #7549, still open. Same treatment: the Install step is
continue-on-error and a new step requires all three of the PyTorch step, the
torchaudio resolution error and the missing win_arm64 platform tag, so any other
failure is red. The row leaves experimental so the job is required, and the pin
errors out as soon as the venv interpreter reports anything but win-arm64, which
is what #7549 landing looks like.
Adds the virgin Windows container lane as two jobs here rather than a sibling
workflow: same premise as the win legs, same path filters, and masked-versus-real
reads better side by side. The hosted Windows legs cannot test the VC++
2015-2022 runtime (it ships in the runner image's System32) or a Windows with no
Microsoft Store, and a servercore:ltsc2022 container on windows-2022 answers both.
The probe asserts no python, py, git, cmake, cl, winget or uv on PATH, on disk or
in the registry, and now also asserts vcruntime140.dll, vcruntime140_1.dll and
msvcp140.dll are absent, which is the one thing the hosted runner cannot un-ship.
Both container install rows stop at studio/setup.ps1's winget-only git gate on
this branch, since #7549 is what relaxes it, so both are pinned the same way. The
overlay row additionally requires the UNSLOTH_CI_SOURCE_OVERLAY hook to have
fired, unconditionally: without that it would be indistinguishable from the
released-wheel row, and the hook is this branch's own feature.
Container notes carried over from the spike: never docker pull when the image is
cached, since MCR has shipped an image ahead of the runner host before; wait for
the Docker daemon, because one leg died in 21s on npipe:////./pipe/docker_engine
and that flake misreads as "Windows containers unavailable"; drive docker from a
run: step, because the job-level container: key is Linux-only. The root CA store
is seeded after the virginity assertion, restoring what a real Windows already
has, because studio/install_node_prebuilt.py downloads Node with bare
urllib.request.urlopen and hits CERTIFICATE_VERIFY_FAILED against the empty
container ROOT store. That product bug is left alone here.
Assert arch and signature on every downloaded Mach-O. This is the one genuine
gap the simulation had: Rosetta 2 is preinstalled on hosted runners and absent
from a factory-fresh Mac, so an x86_64-only llama.cpp, whisper.cpp, Node or uv
payload runs green here and dies with "bad CPU type in executable" for the
user. llama-server launching under `assert-llama-loads.sh` does not rule that
out, because Rosetta makes it launch. The new `macho` check reads `file -b`
(`lipo` is an xcrun shim and is gone after masking, as the desktop lane already
notes) and keys the expected arch off `uname -m`, so macos-15-intel expects
x86_64. It also requires at least an ad-hoc signature on arm64, which closes
the AMFI "Killed: 9" class that uv has already been bitten by; the check is
skipped on x86_64, where unsigned code loads fine and so is not the same
defect. It fails when the scan finds nothing, since an empty scan reads exactly
like a clean one.
Make absence real rather than PATH-hidden. uv probes well-known interpreter
locations and the framework loader ignores PATH entirely, so hiding the
toolcache only hid it from `command -v`. Empty /usr/local (it EXISTS on a
factory-fresh Mac as a SIP-exempt firmlink, and is empty; it is /usr/local/bin
that is absent, so the directory itself stays), move the hosted toolcache and
/Library/Frameworks/Python.framework aside, and clear the developer dotdirs and
caches. A populated uv or pip cache can also satisfy a resolution that would
fail on a user's machine. Every removal goes through --remove and is recorded
in the generated restore.sh, guarded so a path the install recreated is not
buried inside its own restore.
Unset CI, GITHUB_* and RUNNER_* for the installer process only. An installer
branching on CI=true is a hidden dependency no consumer exercises. Scoped to
the child so the step's own $GITHUB_OUTPUT still resolves.
Record spctl --status and csrutil status. Neither is documented for these
images and both change what a binary is allowed to do.
* Tests: import bitsandbytes before the GPU-free harness spoofs CUDA
The CPU test harness patches torch.cuda.is_available to return True so
device_type.py's cache captures "cuda" on a GPU-less runner. bitsandbytes
reads the same flag at import time to decide whether to load its CUDA
backend, and that backend reads torch._C._cuda_getCurrentRawStream, which
a CPU-only torch build does not expose. An import landing inside the spoof
window therefore raises, Python drops bitsandbytes from sys.modules while
leaving its submodules cached, and every later import returns a module with
no .functional, so unsloth/kernels/utils.py dies at module scope.
Import bitsandbytes before the window so it stays on its CPU backend and
remains fully usable, rather than being degraded to unavailable.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Desktop: ask before quitting on top of a running install
This is the trigger neither #7492 nor #7490 addresses -- both start from a venv
that is already broken. Confirmed: neither PR touches cleanup_child_processes.
Quitting runs cleanup_child_processes -> install::stop_install, which SIGTERMs the
installer's process group. In the reported session that landed at "5/10 studio
deps", so the venv kept the CLI's dependencies and lost the server stack, and the
next launch died on `import structlog`. Three minutes of installing, destroyed
with no warning and no way back.
So ask. Only from the tray Quit item -- a deliberate action with a UI present. The
RunEvent::Exit path (OS shutdown, SIGTERM) is left alone: it must never block on a
dialog nobody can answer. The call already runs off the menu callback thread,
which is also what blocking_show requires.
Closing the window was already safe (it hides to tray); this closes the remaining
way to lose an install by accident.
* Tighten comments in desktop quit-during-install guard
* Condense comments in quit-during-install guard
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
* Installer: stop requiring a developer toolchain on the consumer path
A brand new Mac cannot install Studio at all. install.sh gates on
`xcode-select -p` and exits 1 with 'Xcode Command Line Tools are required',
and Linux exits 1 on any non-apt distro over cmake/gcc/git/libcurl headers.
Nothing under either gate needs a toolchain. uv is a prebuilt binary, CPython
comes from uv's managed python-build-standalone, llama.cpp and whisper.cpp are
prebuilt downloads, Node is a pinned nodejs.org archive, and triton is skipped on
macOS. unslothai/llama.cpp b10107-mix-1911198 publishes macos-arm64, macos-x64,
linux-x64 and linux-arm64 builds covering cpu, cuda12, cuda13, rocm and vulkan.
PR #6617 already dropped the Homebrew/cmake stop on macOS for this reason and
just left the CLT stop behind.
macOS: warn and continue when the CLT are absent. Linux: only a download
transport (curl or wget) is fatal; build tooling warns. Both keep a hard git
requirement for --local, which installs unsloth-zoo from a git+https URL.
Both gates move into functions so tests/sh can extract them. The old inline form
could not be reached by the tests/sh convention, which is why this shipped broken
and stayed broken. test_macos_clt_gate.sh (19 assertions) and
test_linux_deps_gate.sh (25) cover the clean machine, the CLT-stub shape where
/usr/bin/git exists but fails, the non-apt distro, and the --local paths.
Writing the Linux test caught a latent bug: the gate trimmed its list with
$(echo ... | sed ...), so on a minimal image without sed the substitution yields
empty and it reports 'all system dependencies found' on a machine with none of
them. Replaced with parameter expansion.
Also caps av<16 in the single-env constraints. av 16+ ships no cp313 macOS arm64
wheel, and it is a C extension over FFmpeg, so uv would silently fall back to a
source build needing both a compiler and FFmpeg headers.
Verified on GitHub-hosted macOS runners with /var/db/xcode_select_link,
/Library/Developer/CommandLineTools, /Applications/Xcode*.app and Homebrew moved
aside. macos-14, macos-15 and macos-26 fail on main and install cleanly with
this; the recorded tool-invocation trace for the whole install is a single
`xcode-select -p`, so nothing compiled and nothing installed a toolchain.
* Linux: auto-install git rather than dropping it, and skip triton kernels without it
Making git optional on Linux was too broad. studio/backend/requirements/
triton-kernels.txt line 2 is a git+https URL, so step 6/14 died with 'Cannot find
command git' and failed the whole setup on ubuntu2404-root, ubuntu2404-arm-root
and fedora41, all of which had been passing. The claim that nothing on the
consumer path needs git holds on macOS, where triton is skipped, but not here.
install.sh now auto-installs git through apt with the other optional tooling, so
Debian and Ubuntu are unchanged. The triton kernels step skips with a message
when git is absent instead of failing: they are a training speedup, not a boot
requirement, and a GGUF chat install has no use for them.
Six more assertions pin both halves.
* macOS Intel: skip the one package with no x86_64 wheel
The Intel clean-machine leg installed with the toolchain masked, then died in
studio setup:
subprocess.CalledProcessError: Command '['cmake', ...]' returned non-zero
ERROR: Failed building wheel for pytorch_tokenizers
pytorch_tokenizers publishes wheels for macOS arm64, linux x86_64, linux aarch64
and windows, but none for macOS x86_64 at any Python version, so uv falls back to
an sdist that shells out to cmake. Nothing passes --only-binary, so the
compiler-free property was an assumption rather than a contract, and Intel is
where it broke.
Marked so it installs everywhere except Intel macOS. Apple Silicon is unaffected.
* Stop the optional dep gate from aborting the install
_smart_apt_install exits rather than returns, and `|| true` does not catch an
exit, so a box missing cmake or git aborted at the gate added to let it
continue. Verified in sh, dash and bash. Run it in a subshell and re-raise only
code 2, the NEED_SUDO handshake install.rs answers with an elevation prompt.
install.sh treats a present-but-broken git as missing, but the Python side
tested only shutil.which, so it promised to skip the git+https triton
requirement and then fetched it anyway. Same check on both sides now.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Never elevate for optional build tools
Re-raising code 2 turned the optional set into a NEED_SUDO handshake, so a box
missing cmake or gcc got the desktop's mandatory permission dialog, whose Cancel
drops back to not-installed. That re-imposes through a prompt the build-tool
requirement this gate removes, and none of those tools are needed to run.
Suppress the handshake for optional callers; a required package still elevates.
Verified in sh, dash and bash.
Also advance the progress bar on the no-git triton skip, which otherwise ends at
14/15.
* Tighten the comments on the dependency gate
* Correct why the PyAV cap is needed
16.0.0 does ship cp313-cp313-macosx_14_0_arm64; the comment claimed no cp313
wheel exists. The actual reason is the deployment target: 15.1.0 is macosx_13_0
and 16+ is macosx_14_0, so the cap is what keeps macOS 13 off a source build.
* Tighten the installer gate comments
* Cap cryptography on x86_64 macOS so the consumer install needs no Rust
cryptography 49.0.0 (2026-06-12) dropped the macosx_10_9_universal2 wheel
and now ships macosx_11_0_arm64 only, so x86_64 macOS has no wheel and uv
falls back to the sdist. That build calls maturin, which pulls Rust and
then fails at 'linking with cc failed' on a clean Mac without the Xcode
Command Line Tools. It surfaced in the clean-machine leg mac macos-15-intel
/ mask / file, several minutes into the studio dependency step, which is
exactly the up-front toolchain requirement this branch removes.
48.0.1 is the newest release carrying a universal2 wheel, and its
cp39-abi3 / cp311-abi3 tags cover the 3.12 and 3.13 interpreters the
installer creates. The cap is marker-scoped to darwin + x86_64, so arm64
macOS and every other platform still resolve to the latest. Lift it when
cryptography ships an x86_64-capable macOS wheel again.
Resolution of studio/backend/requirements/studio.txt under this
constraints file gives 48.0.1 on x86_64-apple-darwin and 49.0.0 on
aarch64-apple-darwin and x86_64-unknown-linux-gnu, on both 3.12 and 3.13.
* Correct the av note now that cryptography also compiles on macOS
* Never escalate for optional apt packages outside Tauri mode
The optional bypass sat inside the TAURI_MODE branch, so a plain curl | sh
install on a non-root Debian or Ubuntu box still fell through to the
escalation branch and showed the default-yes permission prompt for cmake,
GCC and the libcurl headers. That is exactly the toolchain this change set
declared unnecessary on the consumer path, so the prompt asked for a
password to install packages nothing here uses, and a headless run failed
the same way instead of falling through to prebuilt llama.cpp.
Move the check above the mode split so optional callers return 2 in both
modes. Required packages such as curl still escalate unchanged.
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: free the llama-server slot when a chat stream reaches [DONE]
* Release the slot before yielding, only on a completed decode
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten the comments added by this PR
* Inline the done-sentinel check and use plain bools for the decode flags
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: show a Nudging tool calls badge while the tool-call re-prompt runs
* Guard the nudge status ordering assertion against index 0
* Tighten the nudge status comments
* Announce the nudge text instead of the generic spinner label
* Trim the nudge status comments
Collapse the multi-line notes to fewer lines and drop one that restated the assert below it. The blank-before-badge ordering reason and the keep-in-sync contract are preserved.
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
* fix(studio): show the current artifact's source after switching artifacts
The canvas source view feeds one Streamdown a fence built from the selected
artifact's code, but never keys it. Streamdown does not revise a block it has
already committed, so the panel keeps rendering the previous artifact's source.
Key the source view on the artifact ID plus a hash of its code: tool artifact
IDs are derived from the tool call, not the code, so the ID alone does not
change when a tool artifact is updated in place.
* Name the real root cause and make the source-key test load-bearing
The remount is needed because Streamdown memoizes a fenced code block on its
hast node's line/column span, which ignores the text inside the fence, so two
canvases of equal line count compare equal and the old source stays on screen.
Verified in Chromium against streamdown 2.5.0: unkeyed, 70 lines -> 70 lines
renders the previous artifact, 70 -> 71 and 70 -> 90 render correctly.
Move the key expression into the source branch so it costs nothing while the
artifact is streaming and the view is unmounted, and export the helper from
types.ts so the test exercises the shipped code instead of a local copy of the
formula (it passed before even with the key removed from the component).
* Assert the source view's Streamdown key wiring, not just the helper
The suite exercised buildArtifactSourceKey but never the component, so deleting
key={buildArtifactSourceKey(artifact)} from the Streamdown left every test
green. There is no DOM renderer available to these tests, so parse
artifact-surface.tsx with the TypeScript compiler API (already a devDependency)
and assert the source view's Streamdown carries that key.
Mutation-checked: removing the key fails 1 test, swapping it for artifact.id
fails 1, and making the helper ignore code fails 2.
* Tighten the comments added by this PR
* amd: require bitsandbytes>=0.50.0 in the amd extra
bnb <= 0.49.2 NaNs at decode shape on every AMD GPU. The ROCm 4-bit GEMV
fix (bnb PR #1887) first ships in 0.50.0, on PyPI since 2026-07-24, so the
old >=0.49.1 floor could still resolve the broken range.
Mirrors the same change made on the pip release branch in #7278.
* amd: cite the 0.50.0 ROCm work accurately in the bnb floor comment
The comment credited bnb PR #1887 as "the ROCm 4-bit GEMV fix" for every
AMD GPU. #1887 decouples blocksize from warp size and fixes a hardcoded
warp size of 32 in kgemm_4bit_inference_naive, which is a CDNA problem by
construction. The RDNA-side work is #1979 (fused 4-bit SIMT GEMM) and
#2012 (RDNA3/4 workgroup resonance). All three first ship in 0.50.0, so
the >=0.50.0 floor is unchanged; only the justification was wrong.
* amd: raise the installer bitsandbytes fallback floors to 0.50.0
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* amd: stop reporting the bitsandbytes PyPI fallback as broken
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten AMD bnb floor comments
* Keep the amd extra citation and the AMD install guide reference
* amd: do not promise aarch64 a ROCm 4-bit backend it never gets
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* amd: fall back to the PyPI bitsandbytes floor on Windows ROCm too
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Every desktop-v* release in this repo is a draft, and GitHub lists drafts only
to a token with push access, which is why resolving one needs contents: write.
A pull request from a fork receives a read-only token no matter what the
workflow declares, so on those runs the resolver cannot see any release and the
job died on "no desktop-v* release visible", accusing the repo of having no
bundle when the real cause is the trigger.
This workflow runs on pull_request for changes to itself and the stripping
scripts, so an outside contributor editing either would have hit that. Guard the
three jobs on the head repo not being a fork. A skipped job is honest here: it
does not claim to have tested a bundle it was never able to download, and it is
not reported as a pass.
All three desktop legs died at the download step with an empty REL_TAG. The
resolver passed --exclude-drafts while REL_REPO now defaults to
github.repository, and every desktop-v* release in unslothai/unsloth is a draft:
desktop-v0.1.50-beta and desktop-v0.1.471-beta are both drafts carrying the .dmg,
.deb, .AppImage and setup.exe, while only the non-desktop tags like v0.1.501-beta
are published. Excluding drafts therefore matched nothing and no leg could ever
run against a production bundle.
Drop --exclude-drafts so the newest desktop-v* release is found. A draft has no
tag ref, so releases/tags/<tag> 404s for one, but gh resolves drafts over GraphQL
and gh release download <tag> fetches their assets normally, so the download call
is unchanged. Listing drafts requires push access, which for GITHUB_TOKEN means
contents: write, so the workflow permission is raised from read and annotated.
When nothing resolves the leg still fails hard rather than skipping: with no
bundle to install there is nothing to prove, so a green run would be a lie. The
error now names both causes, no release cut yet or a token that cannot see drafts.
Also stop the restore step swallowing its own failure. `bash
.clean-machine/restore.sh || true` printed "No such file or directory" whenever an
earlier step failed before the toolchain was stripped, and hid a genuinely broken
restore just the same. Skip explicitly when the file is absent and let a real
restore failure surface. Same fix in clean-machine-install-ci.yml, which had the
identical line.
* feat(studio): share the llama-server --parallel bounds as PARALLEL_MIN/MAX
The per-load parallel-slots field needs the same 1..64 range the CLI flag
validates, but models/inference.py cannot import run.py (run.py builds the
app that imports routes that import models). Promote the bounds into this
dependency-free module, which already owns the -np/--parallel semantics, and
record the deliberate mirrors that cannot import it (run.py, the unsloth CLI,
the web UI). The denylist entry stays: the first-class field is now the single
write path for the slot count, so a pass-through would still desync the
committed bookkeeping from llama-server.
* feat(studio): note the per-load override in the --parallel help text
--parallel is now the server-wide default that a per-load n_parallel (the
Studio Parallel Slots run setting) can override, not the definitive slot
count. Point at the new control so a user does not conclude a restart is the
only way to change slots, and record the shared PARALLEL_MIN/MAX mirror
alongside the existing CLI one.
* feat(studio): add n_parallel to LoadRequest and echo the slot counts
LoadRequest.n_parallel (optional, PARALLEL_MIN..PARALLEL_MAX) lets a load pick
its own llama-server --parallel count; omitted, the server-wide launch default
applies. ValidateModelRequest carries it too so the training-coexistence
estimate sizes the KV cache like the follow-up load rather than passing on a
smaller footprint.
LoadResponse and InferenceStatusResponse gain both requested_parallel_slots
(what the load was invoked with) and parallel_slots (what llama-server
actually runs after the fitter's slot reduction), so a client can tell an
honored request from a reduced one. Both are None where --parallel has no
meaning: non-GGUF loads and the diffusion runner.
* feat(studio): record the requested parallel-slot count on the backend
The auto GPU-memory fit may launch fewer slots than requested to keep the
model fully on GPU, so the committed effective count cannot answer "is the
live server what this request asked for?". Store the invoked count separately
(mirroring the _requested_n_ctx pattern) from the pre-reduction pending
kwargs, expose it as requested_parallel_slots, and have _already_in_target_state
compare requested-vs-requested: comparing against the effective count would
reload -- and re-reduce -- forever on an identical Apply.
The comparison sits in the non-diffusion branch, since the diffusion runner
ignores --parallel entirely. The requested value shares the effective count's
lifecycle, so every unload/kill path clears it and a stale count cannot
poison the next load's dedupe.
* feat(studio): honor a per-load parallel-slot count in /load and /validate
Resolve the slot count once per load -- the request field if set, else the
server-wide launch default -- and feed it to every consumer that must agree:
the training-coexistence guard, the llama-server load kwargs, and the reload
dedupe. Without the dedupe comparison a changed slot count would be swallowed
as already_loaded; it compares requested-vs-requested and skips the diffusion
runner, which ignores --parallel.
app.state.llama_parallel_slots is deliberately never written: it stays the
launch intent and the admission-queue fallback, so one load's override cannot
leak into later loads. /validate resolves the same way so its estimate cannot
undercount what the load then allocates.
Both /load returns and /status echo the counts through one helper, which
reports None for diffusion -- its load never commits a count, so echoing the
reset placeholder would fabricate an "invoked with 1 slot".
* feat(studio): accept nParallel in the chat-preset load config
ChatPresetLoadConfig is extra="forbid", so a preset carrying the new parallel
slots knob would 422 the whole settings sync without this field. Bounds come
from the shared PARALLEL_MIN/MAX rather than literals, so a future range
change cannot start rejecting presets the UI still allows.
* test(studio): cover the per-load parallel-slots knob
Pins the behaviors a regression would silently break: the requested-vs-effective
dedupe (comparing against the reduced count would reload forever), the diffusion
skip and its None echo, the requested count's reset lifecycle, and its commit
from the pre-reduction pending kwargs.
Also pins the three bounds mirrors that cannot import PARALLEL_MIN/MAX (run.py,
the unsloth CLI, the web UI) plus the preset model that can, so a range change
cannot leave one of them clamping or rejecting at the old limit.
* test(studio): refresh the --parallel denylist comments for the UI knob
The pinned rationale said the typer flag owns the slot count and pointed users
at a Studio restart. Parallel Slots / LoadRequest.n_parallel is now the other
managed writer, and the 1..64 guard is the shared PARALLEL_MIN/MAX -- a reader
following the old comments would conclude the UI control does not exist.
* feat(studio): note the per-load override in the CLI --parallel help
Both the plain-serve and `unsloth studio run` flags now describe a server-wide
default the Studio Parallel Slots run setting can override per load, matching
the backend help text.
* feat(studio): remember a per-model Parallel Slots override
nParallel joins the per-model config with the same null-means-follow-the-default
convention as the other knobs: null keeps the server-wide --parallel count, so
a blank control never pins a number and isDefaultConfig still deletes an
otherwise-untouched config instead of storing it.
The value is re-clamped to N_PARALLEL_MIN/MAX on every localStorage read and
write (the store is user-editable), and listing it in STORED_CONFIG_FIELDS
keeps it from being dropped as an unknown key. Legacy blobs predate the knob,
so their migration carries null. No schema-version bump: an additive optional
field, like the GPU fields before it.
* feat(studio): bridge nParallel between the per-model config and the store
The config->store, store->config and equality helpers all need the new field:
without the equality arm a slots-only edit reads as unchanged, so Apply is
dropped and the dirty state never lights up.
* feat(studio): track the parallel-slot override in the chat runtime store
nParallel holds the editable override and loadedNParallel the value the last
successful load sent, which the failed-switch rollback re-sends. Both are
per-model: they clear on unload and on a model switch, unlike the standing
preferences (GPU memory mode, speculative type) that survive one.
There is deliberately no backend-echo field for the control: the echo is the
resolved count, so adopting it would pin a blank "follow the server default"
input to an explicit number.
* feat(studio): type n_parallel and the slot-count echoes
The load request gains the optional per-load slot count, and both the load
response and the status payload gain requested_parallel_slots (invoked) and
parallel_slots (actually running after the fitter's reduction). Keys stay
snake_case: the payload is serialized as-is, with no case conversion.
* feat(studio): forward n_parallel to the validate preflight
validateModel builds its own body rather than forwarding the load payload, so
the slot count has to be listed explicitly. Slots scale the KV estimate, and
the preflight exists to refuse a load the training guard would then 409 -- an
unforwarded count would validate a smaller footprint than the load allocates.
* feat(studio): include nParallel in the active model's config
The sidebar assembles the active model's config from individually subscribed
store fields; an unsubscribed field would leave the form showing a stale value
after any external change.
* feat(studio): add the Parallel Slots control to the run settings
A numeric input in the GGUF advanced section, blank meaning "follow the server
default". It clamps on change like the Draft Tokens field rather than using
NumericValueInput, so there is no blur-draft to lose when the user types a
value and immediately clicks Load.
hasNonDefaultAdvanced counts it too, so a remembered override reopens the
advanced section instead of hiding the setting that is actually in effect.
* feat(studio): key the sidebar config form on nParallel too
The signature drives the remount that re-seeds the form; without the new field
an externally changed slot count would leave the sidebar showing the old one.
* feat(studio): send the Parallel Slots override on load
performLoad snapshots the slot count at click time (staged run-settings config
first, else the store) and sends it on both the validate preflight and the
load, so the two size the same footprint. A cross-model switch re-baselines it
like the other per-model knobs -- the previous model's count must not follow
onto the next one -- and the failed-switch rollback re-sends the previous
model's value so a rescue reload cannot silently drop to the server default.
The success path keeps the click-time value rather than the response echo: the
echo is the count the fitter resolved, so adopting it would turn a blank
"follow the server default" control into an explicit pin. Slots are GGUF-only,
so a transformers load sends and records null instead of a phantom override.
* feat(studio): carry the slot override through the compare-pane load
The compare pane builds its own load request, so it needs the field explicitly
or a pane with a remembered override would load at the server default. Its
validate preflight sends the same count, matching the comment above it that
promises validation is sized exactly as the load below.
GGUF-gated on both calls, and the store adopts the pane's own click-time value
rather than the resolved echo, mirroring the single-model path.
* feat(studio): honor the remembered slot override on startup auto-load
The auto-load path reads the per-model config and forwards every other
remembered knob, so a remembered Parallel Slots value was the one setting lost
on the "load last used model" path: llama-server came back at the server-wide
default with the control showing blank, and the first manual Apply afterwards
then forced a needless reload because the counts disagreed.
* feat(studio): seed the slot baseline from the status echo
Only the rollback baseline is seeded, never the editable control: the echo is
the resolved count, so adopting it would pin a blank "follow the server
default" input to a number. Without the seed, loadedNParallel stayed null
after a tab reload or a second tab adopting the running model, and a failed
switch then rolled the previous model back at the server default while every
other knob was restored.
* feat(studio): capture Parallel Slots in chat presets
The knob joins the preset load config end to end: captured from the store,
re-clamped when read back (persisted presets are untrusted input), applied on
switch, and summarized in the preset chip. Its default is null, so
coalesceDefaultLoadKnobs keeps a default-only preset empty rather than
persisting a no-op override.
* feat(studio): re-derive the preset state when Parallel Slots changes
Both preset memos snapshot the store through capturePresetLoadConfig, so
without the new dependency a slots-only edit left the unsaved-changes flag and
the load summary showing the previous value.
* test(studio): pin the Parallel Slots wiring end to end
Source-contract coverage for the hops a refactor can silently drop: the three
/load builders (interactive, compare pane, startup auto-load) and their
validate preflights, per-model persistence and clamping, the UI row, and the
status seed -- including the negative assertion that hydration seeds only the
rollback baseline, never the control, so the resolved echo cannot pin a blank
"server default" input.
* test(studio): pin nParallel in the preset load config
Covers capture, clamped read-back and apply on the frontend, plus the backend
field itself: ChatPresetLoadConfig is extra="forbid", so a missing or drifted
field 422s every settings sync that carries a preset.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fall back to one slot when llama-server lacks --kv-unified for PR #7447
Without --kv-unified an explicit --parallel N makes llama-server give each slot -c/N, so on a build without the flag choosing N slots silently shrinks every context window for a feature that build cannot serve. Clamp to one slot and log why, placed after the requested count is captured so the echo still reports it and before the KV estimates so the fit matches what actually launches.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Clear the slot control on load paths that never send it, and size the training guard for diffusion
Four review findings on the per-load Parallel Slots knob.
The editable nParallel control means "follow the server default" when null, so
any success path that does not send a slot count has to clear it. Three paths
kept a value staged for a different model:
- chat-adapter.ts, cached non-GGUF auto-load: the interactive and compare
builders already clear both fields for a non-GGUF response, this third one
did not. The field never renders for a non-GGUF target, so the stale count
was invisible and unclearable from the UI yet still persisted, and it flips
isDefaultConfig so a user with no overrides silently gets a stored entry.
- chat-adapter.ts, fresh-model fallback: its request omits n_parallel but its
success state resynced every other knob and left the slots alone, so a staged
edit survived against a server running the default and the next Apply
reloaded at a count that load never sent.
- apply-inference-status-to-store.ts: on a model change underneath the tab
every sibling knob adopts the new model's status, but nParallel updated only
its baseline, so the previous model's explicit count followed onto the new
model and saving or reloading there pinned it. Clear the control and keep
seeding the baseline for the rollback.
The training-coexistence guard sized a diffusion GGUF with the requested slot
count. _estimate_kv_cache_bytes scales the SWA cache with slots
(swa_limit = swa * slots + ubatch), but load_model hands a diffusion target to
_start_diffusion_server before the slot plumbing, so that runner is always
single-slot. At the new default of 4 this inflated the estimate and could 409 a
load that fits. An unclassified GGUF keeps the requested count.
Backend base KV depends on -c alone, not on --parallel, which is why only the
SWA term is affected: llama.cpp PR 14363 and discussion 4130.
Tests: three training-guard cases in test_parallel_slots_per_load.py and one
source contract in test_model_picker_contracts.py, each mutation-checked.
174 passed across the backend slot/admission/training suites, 56 across the
frontend contract suites.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the slot control when re-adopting the running model, and never record slots for a diffusion load
Two follow-ups from the latest review round.
The first is a regression from c796393. That commit cleared the slot control
whenever hydratingExistingModel was set, to stop model A's count following onto
model B. But that flag is also set on the resident-model adopt path: when the
store checkpoint is an external provider id and the user re-picks the still
loaded local model, applyActiveModelStatusToStore is called with the external
id as previousCheckpoint, so the flag is unconditionally true. The clear then
wiped the config applyPerModelConfigToRuntime had restored two lines earlier,
and it was the only knob that did, because the siblings re-adopt the status
echo while this one cleared. Gate the clear on the tab's own baseline no longer
matching the running count: a genuine A to B swap still clears, re-adopting the
same model keeps its value.
The second revises an earlier call of mine. I rejected the diffusion phantom as
cosmetic because the backend ignores the value on every send. The sharpened
report is right and my rejection was wrong: capturePresetLoadConfig records
nParallel with no model gate, a Preset carries no model id, and applying one
writes nParallel for whatever model is current. So a count recorded against a
diffusion model, which the backend never applied, rides a saved preset onto a
text GGUF and becomes a real override the user never chose. Record slots only
when the load actually committed them, on all three load builders.
Tests: two source contracts in test_model_picker_contracts.py, both mutation
checked. Frontend typecheck clean, 58 passed across the contract and preset
suites.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Clear the slot baseline when status reports a model without slots
Hydrating from a GGUF to a slotless model left loadedNParallel at the previous
model's count: the seed only runs when the echo is non-null, and the control
clear added earlier touches nParallel alone. The stale baseline is what a
failed-switch rollback re-sends, and preset capture reads it, so it could claim
slots for a model that never used them.
Clear it when status describes a model that cannot have slots. /status omits
the echo entirely for non-GGUF and sends an explicit null for the diffusion
runner, so keying on is_gguf === false or an explicit null covers both while an
absent field on a GGUF, which is how an older backend reports one, still leaves
the baseline alone.
Test mutation checked; frontend typecheck clean against a fresh npm ci.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Distinguish a same-model re-adopt from a model swap, and size the training guard at the slots that launch for PR #7447
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the blank slot control across a failed-switch rollback for PR #7447
* Restore a remembered slot override when hydrating a fresh store for PR #7447
* Tighten comments for PR #7447
* Restore a remembered slot override on a model switch too for PR #7447
* Tighten comments and docstrings for PR #7447
* Take the rollback slot intent from the picker's pre-switch snapshot for PR #7447
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
The desktop job's Windows masking renamed the toolcache Python, scrubbed the
Machine and User registry PATH, and probed `py`, but nothing checked that
`python`, `git`, `cmake` or `cl` were gone. The drop list is heuristic path
fragment matching, so a runner image that moves any of those outside those
fragments leaves the bundled install.ps1 reusing hosted developer tooling while
the job still reports a clean machine. PATH written to $GITHUB_ENV only applies
to later steps, so the check has to live in a step of its own; it carries the
same event gate as the strip, exempts `py` (it lives in C:\Windows and stays,
which is why the start probe is the real evidence), and resets $LASTEXITCODE
before exiting 0 so an intentionally failing probe cannot fail a clean machine.
Also correct the no-winget matrix note: that leg is not failing for an unfixed
product reason. It stops at the unconditional git gate in setup.ps1 only on this
ref, and with that gate relaxed it passes along with every other leg, so the row
is a merge order dependency and stays required.
The Windows Install step ran `& $script` inside a `shell: pwsh` step, so
install.ps1 was executing under PowerShell 7. A genuinely clean Windows box
does not have PowerShell 7: Windows ships powershell.exe (Windows PowerShell
5.1) and pwsh is a separate install that the hosted runner image happens to
preinstall. So the one workflow whose premise is a machine that has never seen
a developer toolchain was testing the installer under a shell that machine
would not have, and no other Windows job anywhere in .github exercises
install.ps1 under 5.1.
Invoke it the way the desktop does (install.rs:325-339, and the bundled
installer step in desktop-app-clean-machine-ci.yml): powershell.exe with
-NoLogo -NoProfile -NonInteractive -ExecutionPolicy Bypass -File. The pwsh
step wrapper stays, since it is only the installer that has to be under 5.1.
Calling powershell.exe with `&` keeps the output in the pipeline, so
Tee-Object still fills logs/install.log, and $LASTEXITCODE after the pipeline
is the child's real exit code, so $rc and `exit $rc` are unchanged.
install.ps1 and studio/setup.ps1 hold no PowerShell 7-only constructs: no
`#Requires` above 5.1, no `&&`/`||` chain operators, no ternary, no
null-coalescing, no ForEach-Object -Parallel, no $IsWindows/$PSStyle, and no
6+ cmdlets or parameters. setup.ps1 declares `#Requires -Version 5.1`, and its
three $PSVersionTable branches gate a 7-only preference on the 7 side with a
5.1 fallback. Every Invoke-WebRequest already passes -UseBasicParsing, which
5.1 needs because it otherwise reaches for the IE engine.
The desktop workflow claims all three platforms are stripped, but only macOS
and Windows had a strip step and the Windows one scrubbed the process PATH
only. Both gaps let a bundle that needs a developer toolchain pass the one
workflow whose premise is that it must not.
Linux: the job ignored strip_toolchain entirely and ran the bundled install.sh
with the runner's git, gcc, cmake and make in /usr/bin. clean-machine-env.sh
now has a Linux --remove branch that moves the resolved tool binaries aside,
recorded in restore.sh, and the job calls it plus `assert absent` after the apt
step (the .deb install needs dpkg) and before the bundled installer, with a
restore step to match macOS. The loop repeats per tool so a name present in
both /usr/bin and /usr/local/bin is fully masked rather than half masked.
Windows: rewriting $env:PATH does not survive the bundled install.ps1, which
calls Refresh-SessionPath (318-337) and rebuilds $env:Path from the Machine and
User registry values, and py.exe in C:\Windows reaches the toolcache whatever
PATH says. Ported the on-disk toolcache rename, the Machine/User registry scrub
and the py -3.11/-3.12/-3.13 start probe from clean-machine-install-ci.yml, so
the strip is proven rather than assumed.
Windows preflight: the log step was Test-Path, Get-Content and Select-String,
none of which can fail, so an app that hangs before preflight passed on the
90 second liveness check alone. It now asserts a tauri.log exists and carries a
`desktop_preflight completed disposition=` line, the same unconstrained check
macOS and Linux already make. The disposition VALUE is deliberately not
constrained: ManagedReady over an unbootable venv is the reported bug.
installer_source on macOS: only the pipe delivery branched on it, so a
`published` dispatch ran the checked-out script on six of the eight macOS rows
while the run was labelled published. The script is now resolved once at the
top of the Install step and used by the file and tauri deliveries; pipe still
re-fetches through the live transport, because that is half of what it tests.
Linux, WSL and Windows already honoured the input.
Also shortened the comments across the changed files, keeping the reasoning
that says why each check exists.
Four things that let a leg go green while proving nothing.
The desktop Windows job installed the bundle and launched it, and that was all.
On a fresh profile preflight reports not_installed and the app sits on the
install screen waiting for a click, so the process happily stays alive for 90
seconds without the bundled install.ps1 ever running. A bundle that shipped no
install.ps1 resource, or a broken one, passed this job -- which is the packaged
app failure the workflow exists to catch. macOS and Linux already invoke their
bundled script directly; Windows now does the same, via the resource NSIS laid
down next to the exe, invoked the way install.rs invokes it, then asserts the
managed venv exists and can import torch. Its timeout goes to 60 minutes
because a full torch install on a Windows runner is the slowest of the three.
A manual run that selects installer_source: published only redirected the macOS
and Linux jobs. WSL kept copying the checked-out install.sh and Windows kept
running the checked-out install.ps1, so a run asking whether the script on
unsloth.ai works reported on this ref under the published label. Both now honor
the selection; install.ps1 advertises its own unsloth.ai URL, so published has a
meaning on Windows too. Both branches stay empty on pull_request and push, so
automatic runs are unchanged.
The push-to-main filter listed only install.sh, install.ps1 and this workflow,
while the PR filter also covers setup.sh, setup.ps1, install_python_stack.py and
the clean-machine helpers. A direct push touching those skipped the workflow
entirely, so the post-merge backstop never ran for the files the source overlay
was added to cover. The two lists now match.
Neither filter covered studio/backend/requirements, even though the overlay
exists precisely so a constraints change is resolved on a machine with no
compiler and no cached wheels. The update-smoke workflows cannot stand in: they
start from a preinstalled Python and full developer tooling.
* Pin the newer-mapper FP8 probe with tests that can fail
The two identity assertions added in #7478 compare the returned FP8 tables
against the installed ones, but the fixture serves the same mapper.py as both
the installed and the fetched source and exec always allocates fresh dicts, so
they pin allocation rather than provenance and hold for any new dict.
Replace them with two tests that drive get_model_name end to end: one splices an
FP8 entry into the fetched source only and asserts the upgrade error still fires,
the other serves a mapper.py with no FP8 tables and asserts the 4bit half of the
probe survives, which is the regression #7497 fixed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten the resolver stub for PR #7516
- Restore the fp8_block/fp8_row identity assert alongside the new provenance
test. It is weak, not vacuous: it still catches a probe that hands back the
installed table objects, and it costs nothing to keep.
- Bind Version and transformers_version in the stub namespace. Both are
unreached under the current gates, so a change to either would fail with a
bare NameError instead of the assertion.
Merged main, which clears the unrelated test_runtime_text_encoding failure the
branch inherited from its base.
* Cover the FP8 row-scaling path instead of duplicating the block one
The two tests this PR originally added were already covered by
tests/test_new_mapper_fetched_fp8.py from #7497. An 8-mutant matrix over
loader_utils.py found nothing they caught that the existing file did not, so
they are dropped and test_new_mapper_no_global_leak.py goes back to main.
Two real gaps were open, both on the row branch that load_in_fp8 = True plus
UNSLOTH_HAS_FBGEMM selects ahead of block:
- the FBGEMM row branch in __get_model_name could be deleted outright with
every test still green
- _resolve_with_mappers could ignore its fp8_row argument and silently fall
back to the installed row table
Adds two tests to the existing file, reusing its _load_resolver rather than a
second harness. The row-only fixture splices into the fetched row table alone,
since an entry the block table also knows lets the block branch answer and
masks the regression.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
The two ubuntu2404 root legs went red at "Assert no source build" reporting
triton-kernels. That is not a regression in what the installer does. Those
builds have always happened; they only became visible now that pip_install
stopped discarding uv's output on success, which is what finally let the
nobuild check read the dependency phase at all.
So the question was whether each build actually needs a compiler. Checked
against the real artifacts rather than assumed:
openai-whisper 20250625, randomname 0.2.1, argbind 0.3.9 -- no version of
any of the three has ever published a wheel; antlr4-python3-runtime is
pinned at 4.9.3, below the first release that ships one. All four sdists
use setuptools.build_meta, declare no ext_modules, and contain no
.c/.cpp/.pyx/.rs file. Already allowlisted, correctly.
triton-kernels is the same category and was the only name failing. It is
pinned to the triton repo's python/triton_kernels subdirectory; that tree
is 75 files of Python, a four-line pyproject.toml, no setup.py and no
native source at all. The kernels are Triton DSL compiled at runtime, not
at install time. It is also a direct URL the installer names itself rather
than something resolution picked, and only Linux reaches it. It belongs in
the allowlist, so add it with that reasoning written down.
The allowlist match is now lowercased and underscore-folded on both sides.
The requirement spells the package triton_kernels while uv prints
triton-kernels, and an allowlist that matched only one spelling would pass
by luck rather than by intent. A plain pyarrow sdist is still caught.
The two data-designer @ file:// plugin builds needed nothing: they are
in-tree local paths, already dropped by the same rule that exempts the
source overlay's own build.
Separately, the windows-11-arm leg fails for a real reason and should keep
failing. The ARM handling itself works, the log shows torchaudio being
skipped and torch plus torchvision installing from wheels. What stops it is
that pyarrow and hf-transfer publish no win_arm64 wheel at all, so uv falls
back to their sdists and they fail on CMake configure and on openssl-sys
wanting perl. That is a product gap on the platform, not a gap in the
simulation, so the leg stays experimental and keeps reporting it. Record
that above the matrix entry so the next reader does not re-diagnose it.
* fix: add XPU device support and update hardcoded CUDA selections
* fix: add XPU device support for pytest CUDA skipped tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix device handling for PR #7401
- perplexity_eval.py: use DEVICE_TYPE_TORCH, not DEVICE_TYPE. The latter can
be "hip" or "mlx", which .to() rejects, so this regressed ROCm.
- test_batched_leftpad_generation_gpu.py: XPU diverges here today, so mark it
non-strict xfail on XPU instead of reverting to a CUDA-only guard. Keeps the
real XPU gap visible and turns green once it is fixed.
- Guard torch.xpu.is_available() with hasattr, matching device_type.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Re-enable the flash varlen attention test in CI for PR #7401
attention_dispatch.py now predefines flash_attn_func / flash_attn_varlen_func
as None, so test_run_attention_flash_varlen_receives_window_and_softcap no
longer needs flash_attn importable to be monkeypatched. Verified on a runner
shaped like the CPU-only one: the test fails against main's attention_dispatch
and passes at this head, so the deselect is now dead weight.
* Tighten comments for PR #7401
Drop the hasattr rationale: torch.xpu has existed since torch 2.3 and the
dependency floor is 2.4, so no supported build predates the namespace. The
guard stays as cheap defence, but the comment claimed something untrue.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
install.sh and install.ps1 come from the ref under test, but they install
unsloth from PyPI, which is the consumer path and has to stay that way. That
left everything Python-side coming out of the released wheel: studio/setup.sh,
studio/setup.ps1, studio/install_python_stack.py, and every requirements and
constraints file those resolve through Path(__file__). A branch that changes
constraints.txt or setup.ps1 therefore got a green run that proved nothing
about the change, and some legs proved less than they looked. The Fedora
assertion was already carrying a hand-written workaround for exactly this,
tolerating a triton/git failure on the grounds that the released package lags
the ref.
Legs marked overlay: true now re-point the venv at the ref just before studio
setup runs, through UNSLOTH_CI_SOURCE_OVERLAY: a --no-deps editable install of
the checkout. That makes import studio resolve to the working tree, so the
existing setup-script lookup finds the ref's setup.sh / setup.ps1 and
install_python_stack reads the ref's constraints, with no other change to
either installer.
Not --local: --local additionally installs unsloth-zoo from a git+https URL,
which genuinely needs git, and git absence is the whole point of the masked
legs. The overlay resolves no dependencies and clones nothing, so it holds up
with git, cmake and the compilers all gone. It is not a consumer knob either:
no flag, no usage entry, ignored unless the variable names a directory with a
pyproject.toml in it.
Four legs stay on the released package deliberately, each for its own reason,
recorded in the header: the mac pipe legs keep an end-to-end signal on what a
user actually runs; the trace leg would otherwise answer its own question,
since the editable build calls git through setuptools-scm's file finder; the
non-root Linux leg dies before a venv exists; and WSL only ever receives
install.sh, not a source tree.
Two supporting fixes the overlay depends on or exposes:
install_python_stack.py discarded uv's output whenever a step succeeded, so
the nobuild assertion, which reads the install log, could not see a source
build in the dependency phase at all. That is the phase that installs
studio.txt, where an sdist-only dependency actually turns up, and it reported
"built: none" regardless. It now echoes successful output under
UNSLOTH_VERBOSE, matching what install.sh's run_install_cmd already does.
nobuild now ignores "Building <name> @ file://" lines. A local-path build is
something the caller pointed at, never a dependency resolution chose, and
index dependencies always print <name>==<version>, so a real sdist from PyPI
is still caught, including one named unsloth.
Each overlaid leg also asserts it really was overlaid, so an unset variable
cannot quietly put the whole matrix back on the released wheel.
The appimage row invoked the extractor by bare filename, and a command word
with no slash is resolved through PATH rather than the working directory, so
the extraction exited 127 and the bundled-installer assertion below it never
ran. Prefix it with ./ so the row exercises what it claims to.
The Linux log step also asserted nothing: it skipped a missing log with
continue and discarded the grep with || true. The launch step only proves the
process stayed alive for 90 seconds, and the bundled-installer checks do not
exercise the Rust preflight path, so an app that hung before preflight
completed passed both required Linux rows. Require the same
desktop_preflight completed disposition= record the macOS rows already do.
All three Windows legs failed "Verify the simulation took effect" with no
::error:: printed at all. The check itself was right: the mask step logged
"masked toolcache python: C:\hostedtoolcache\windows\Python", python/git/cmake/cl
were ABSENT, no `py -3.x` probe started an interpreter, and the winget assertions
were satisfied. The step still exited 1.
The cause is $LASTEXITCODE leaking out of the step. The last external command is
the `py -3.13` probe, which is SUPPOSED to fail; Get-Command and Write-Host are
cmdlets and never reset $LASTEXITCODE, and the runner appends
`if ((Test-Path -LiteralPath variable:\LASTEXITCODE)) { exit $LASTEXITCODE }`
to every pwsh step (actions/runner#351). So a clean machine reported failure,
and because this step runs before Install, no Windows leg has ever reached the
installer. Clear $LASTEXITCODE after the probe loop and end with an explicit
exit 0. The leak detection is untouched: a surviving python/git/cmake/cl, or a
`py -3.x` that actually starts, still exits 1.
Also print each probe's exit code and output, so the next failure here explains
itself instead of being silent, and label `py -0p` as what it is. The launcher
reads the registry, which the on-disk toolcache rename cannot rewrite, so -0p
keeps naming paths that no longer exist. Unlabelled it reads like a leak.
Accept the Fedora leg's real outcome instead of a message that can be absent
The fedora assertion only accepted the unsupported-package-manager hard exit.
That is still what this ref's install.sh does, but the pending installer change
replaces it with a warning that lets the install continue, at which point the
old grep matches nothing and the step fails for the wrong reason.
Handle both, strictly. If the log shows the newer "using prebuilt llama.cpp
(missing:" warning, the Linux gate demonstrably did not hard-stop, and the only
tolerated failure past that point is release lag: install.sh comes from this ref
while unsloth comes from PyPI, and the released studio/install_python_stack.py
has no "skip triton kernels when git is missing" guard, so it still fetches the
git+https triton_kernels requirement on a machine with no git. Anything else
after that warning fails the step. Otherwise the old hard-exit message is still
required. A missing log, a bootstrap outage or any unrecognised failure all
remain errors, and the step retires to a plain success assertion once a release
ships the no-git skip.
The manifest-removal guard added in #7492 exits with a bare 'exit 1', so in
Tauri mode the installer never emits the [TAURI:ERROR] line and the desktop
UI falls back to a generic failure instead of naming the cause. Every other
failure path in studio/setup.ps1 goes through Exit-SetupFailure, and
tests/sh/test_tauri_retry_failure_context.sh asserts that invariant, so
'Repo tests (CPU)' has been red on main since that merge.
Co-authored-by: danielhanchen <unslothai@gmail.com>
* Bound how many approvals may park, against the executor
#7455 landed parking, which is the right shape and supersedes what this branch
was carrying. It is unbounded, though, and the thing it is unbounded against is
not the GPU.
A run stopped on an approval prompt is blocked inside the to_thread(next, gen)
call that drives it, so it holds one of asyncio's default min(32, cpu + 4)
executor threads until the user answers. The slot cap used to bound that.
Parking hands the slot back, which admits another run that can park too, so the
ceiling became the wait line: 64 deep on a 1-slot backend. Long before that, the
executor is full and nothing else in the backend runs, including generation
steps for chats that already hold slots and the stream teardown that would clean
up after a disconnect.
The pool already permits `capacity` pending prompts, and each park adds one
more, so the budget is what the executor has left after the cap and a reserve of
4. On this machine (32 workers) --parallel 4 gets 8 parks and 20 free threads,
--parallel 24 gets 4 and 4, and --parallel 28 or higher gets none: there the
prompt keeps its slot and behaves exactly as it did before parking existed.
Counted process-wide rather than per queue. There is one executor, but a
per-queue budget is the same allowance again for every backend, and base_url
carries a fresh port on every model load, so a reload would mint a queue that
knows nothing about the approvals still parked on the old one. A reset clears it
too, or a leaked claim shrinks the budget for the life of the process.
park() reports whether it took the budget, and a refusal costs nothing to undo
because the slot never left its holder. The stream reads that answer rather than
recording a refused park as parked, which would make it skip the park for every
later approval in the same run even once the budget freed up.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Size the park budget from the executor's own CPU count
Two review findings, both real.
The budget read os.cpu_count(). 3.13 sizes ThreadPoolExecutor from
os.process_cpu_count(), which honours CPU affinity and cgroup quotas, and
asyncio's default executor is a plain ThreadPoolExecutor(), so a container
pinned to one core on a 64-core host got a 5-thread executor and a budget
computed from 64. The bound was then looser than no bound at all in exactly the
environment that can least afford it. It asks the same source the executor does,
and the test compares against a real ThreadPoolExecutor rather than restating
the formula, so it stays right on 3.12 as well.
The reserve was a flat 4, which on that same 5-thread executor left nothing to
budget and turned parking off entirely. Small hosts are where a chat most needs
to keep moving while another sits on a prompt. It scales now, and the ceiling
has a floor of two: a quarter of five is one, and one park cannot cover two
chats on prompts at once, which is what #7455's own two-approvals test needs.
Without that floor, that test fails on a one or two CPU runner. `spare` still
takes the budget to zero when the pool already fills the executor, so nothing
about a 32-worker machine changes: --parallel 4 still gets 8 parks, 24 gets 4,
28 gets none.
The two behavioural budget tests pin the worker count rather than reading it off
the runner, and the property test sweeps executor sizes from one CPU to 64
instead of asserting against whatever the host happens to have. The whole suite
passes with the CPU count faked to 1, 2 and 4, which is how both of these were
reproduced.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Size the park budget from every live backend, and free it on the answer
Two review findings, both real.
The budget was global but sized from one queue's capacity. A reload mints a
queue on a new port while the old one drains, so both are live, and prompts on
both park executor threads. Eight parks on an old 1-slot queue plus a new
24-slot backend is 32 threads on a 32-thread executor, with the new backend's
prompts refused and holding their slots, which is the state the reserve exists
to prevent. It sums the capacity of every backend still serving instead. Idle
queues are skipped: those are the ones the registry is about to evict, and they
are holding nothing.
The budget also outlived the wait it was paying for. unpark_async only dropped
it after reacquiring a slot, but the generator yields its post-approval event
first, so the executor thread is already back in the pool while the resume
queues. An approved chat waiting on a slot would refuse a different chat's park,
and that chat then keeps the slot the resumer is waiting for, so an unanswered
prompt strands chats that were already approved. The budget is released when the
prompt wait ends now, and the queue's parked count still runs until the slot is
back, which is what guards idle eviction and the resume ordering.
Both are separate counters on the lease as a result, and every exit from a park
drops the budget: unpark, unpark_async and release. That last one was the mutant
that came back missed, since a client disconnecting on a prompt releases
straight out of parked and would otherwise lose a budget slot for good.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten the comments on the park budget
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The `notools` check reads an absence: it passes when the trace file contains no
compiler, git or brew invocation. A shim directory that never reached PATH
produces exactly the same empty file as an installer that touched nothing, so
the single leg carrying that assertion would stay green no matter what the
installer did. "Verify the simulation actually took effect" only ran for mask
mode, which left the trace leg with nothing checking its own instrumentation.
Call git explicitly after sourcing the environment and require it to appear in
the trace, then truncate the file so the self-test entry does not count against
the install. The call has to be explicit because macOS reaches _has_working_git
only under STUDIO_LOCAL_INSTALL (install.sh:2026), so no consumer leg on that
platform probes git on its own.
Re-run `absent` after the install on the masked macOS legs. It only ran
before, so an installer that quietly selected the Xcode CLT or installed a
compiler left the leg green while every later source build could succeed,
which is the one thing clean-machine-assert.sh says `absent` guards the whole
run against.
Fail the Windows simulation when py.exe can still start an interpreter. The
launcher binary itself may stay, but Find-CompatiblePython probes `py` first
(install.ps1:1130-1153), so an interpreter registered outside the two renamed
toolcache directories gets reused and Python bootstrap is never exercised.
Exempting `py` without ever running it left that unchecked.
Propagate the WSL installer exit code. It was printed and discarded, and the
CLI check does not compensate: install.sh links the `unsloth` shim (4174-4182)
before it reports a failing studio/setup.sh (4219-4230), so a late setup
failure leaves a shim whose --version succeeds.
Run the bundled installer in the Linux desktop jobs. The launch step only
proves the process stayed alive, and on a fresh home preflight reports
not_installed and the app waits on the install screen, so both required rows
passed after 90 seconds without ever touching the shipped install.sh. Locate
the resource in the deb payload or the extracted AppImage, run it the way
install.rs does, and require a managed venv that can import torch.
REL_REPO defaulted to unsloth-test/unsloth-test, which holds one release frozen
at 2026-07-27, while release-desktop.yml publishes into github.repository. The
schedule was re-testing the same fixture forever and could never see a broken
production bundle.
The windows job carried a blanket continue-on-error, so its NSIS assertions
could not gate. lipo -archs prints and exits 0 for a thin binary and `|| true`
swallowed even that, so the architecture was never checked; fall back to file,
which survives the CLT mask. And require the preflight disposition line rather
than the mere existence of tauri.log, which setup_logging creates at process
start regardless.
Compress the comment blocks across the clean-machine workflows and
scripts. The explanations of why each check is written the way it is
stay; the padding, restatement and duplication go.
No code or workflow logic changes.
The Linux rows' only post-install gate was nobuild, a log grep, so an installer
exiting 0 having produced nothing kept a required leg green. The WSL job and the
Windows job both already check the install runs; the Linux job now does too.
The WSL detection half only printed its Select-String, and the alternation also
matches "platform linux", so a regression that skipped every WSL-specific
branch would still pass as a plain-Linux install. Assert the exact marker,
stripping ANSI first since step writes the label in reverse video. Probed against
three fixtures: real wsl log passes, platform linux fails, missing log fails.
The Windows scrub only touched PATH, so the legs were green while not clean: run
30365014702 logged "python ABSENT" and then "Python 3.13 already installed"
with uv resolving C:\hostedtoolcache\windows\Python\3.13.14\arm64\python.exe.
py.exe lives in C:\Windows and uv discovers interpreters itself, so take the
toolcache off disk and fail when tooling survives, instead of only printing it.
The Linux desktop legs never stripped anything, and the tauri.log step was all
|| true so it could not fail. Run the bundled installer the way install.rs does,
with --tauri alone, and assert torch: passing --no-torch skipped the slowest
half of first launch and let the venv check pass over it.
Pin the WSL rootfs to a dated build; current/ is a rolling alias and the digest
next to it is fixed.
* Keep `import unsloth` working when bitsandbytes is absent
device_type.py already prints "bitsandbytes is not installed - 4bit QLoRA
unallowed, but 16bit and full finetuning works" and clears
ALLOW_BITSANDBYTES / ALLOW_PREQUANTIZED_MODELS, but the import chain then
hard-required the module anyway, so `import unsloth` raised instead.
#7354 made this reachable: the gfx906 install path uninstalls the generic
bitsandbytes wheel (no gfx906 kernels in it), which leaves an MI50 / Radeon VII
host unable to import unsloth at all, not on the 16bit path the message
promises.
- kernels/utils.py: guard the bnb import; bind get_ptr and the five 4bit ctypes
handles to a stub that raises a clear message if a 4bit path is entered.
HAS_CUDA_STREAM stays False, which is the correct route.
- save.py, models/granite.py: guard Bnb_Linear4bit and peft's Linear4bit
(peft exports it only when bnb imported cleanly) with placeholder classes.
Both names only feed isinstance checks, so nothing matching is exact.
- _gpu_init.py: same degradation on the xpu branch as the cuda branch above.
Verified on a Strix Halo (gfx1151, DEVICE_TYPE=hip, torch 2.11.0+rocm7.13.0)
by blocking bitsandbytes with sys.modules["bitsandbytes"] = None, so
find_spec returns None and the import raises exactly as when the package is
absent. Before: ModuleNotFoundError at kernels/utils.py:136. After: import
succeeds, FastLanguageModel/FastModel import, ALLOW_BITSANDBYTES=False,
ALLOW_PREQUANTIZED=False, and the 4bit stub raises with the real cause. With
bitsandbytes present, every binding is unchanged.
New test walks the `import unsloth` module graph with ast and fails on any
unguarded bitsandbytes (or peft Linear4bit) import; verified it catches the
old code. Targeted suites: 702 passed, 18 skipped.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address the review on #7502: zoo coupling, non-hip flags, py3.9 collection
Three findings, each reproduced first and negative-controlled after.
1. The fix still needed an unreleased unsloth_zoo (P1). save.py imported
unsloth_zoo.saving_utils at module scope, and any zoo without the companion
#953 fix imports bitsandbytes there, so `import unsloth` kept failing for a
dependency set pyproject.toml allows. Raising the floor was not an option:
PyPI's newest zoo is 2026.7.6 and #953 is merged but unreleased, so a bump
would break every install today. Both names it pulled in are used only inside
functions, so the import is now lazy at those two call sites, matching what
determine_base_model_source in the same file already does. Verified against a
real pre-#953 zoo checkout with bitsandbytes blocked: import succeeds, and
restoring the eager import reproduces the failure at saving_utils.py:70.
This PR no longer depends on a zoo release.
2. Capability flags were only cleared on hip (P2). device_type.py probed
bitsandbytes inside its DEVICE_TYPE == "hip" branch, so a cuda or xpu host
without bnb imported fine but still reported ALLOW_BITSANDBYTES=True, and the
default load_in_4bit=True path in models/loader.py would select a 4bit
checkpoint before failing. Clear both flags whenever the module is absent, on
every backend, via find_spec so a working install pays nothing. A cuda host
with bnb blocked now reports False/False; with bnb present nothing changes.
3. The new test could not be collected on Python 3.9 (P2). `Path | None` is a
PEP 604 union and requires-python still allows 3.9, so pytest raised
TypeError at import. Added `from __future__ import annotations`. Checked in
real uv venvs on 3.9, 3.10 and 3.13: 2 passed each; removing the future
import reproduces "unsupported operand type(s) for |" on 3.9 only.
The xpu branch in _gpu_init.py needs no separate flag handling now that the
probe is backend-independent.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address the second review on #7502: guarded probe, and 8bit in the same guard
1. The capability probe used find_spec while the fallbacks in kernels/utils.py
and _gpu_init.py treat any import failure as unavailable, so an installed but
unusable wheel would leave ALLOW_BITSANDBYTES true while the kernels had
already bound the stub. Probe with the same guarded import instead, so all
three agree by construction. No new cost on any path: _gpu_init.py already
imports bnb before device_type is reached on cuda, and device_type's own hip
block imports it a few lines later.
Worth recording that the state this prevents is currently unreachable for an
unrelated reason: a broken wheel takes `import unsloth` down earlier, in
transformers/integrations/bitsandbytes.py:20 via
unsloth_zoo/patching_utils.py:680, whichever exception it raises (OSError also
escapes the zoo moe_utils `except ImportError`). So this is correctness for
when those imports get guarded, not an observable fix today.
2. Both loader guards printed for load_in_4bit or load_in_8bit but only cleared
load_in_4bit, so an explicit load_in_8bit=True survived and reached
Transformers, which builds the bnb quantizer and fails there. Clear both. The
message no longer says AMD either: the flag now goes false whenever bnb is
unusable on any backend.
Tests: the probe must not use find_spec, and an ast walk requires every
ALLOW_BITSANDBYTES guard in loader.py to clear both flags, so a third guard
cannot be added with the same omission. Dropping either fix reddens them (1 and
2 failures respectively). 4 passed on 3.9, 3.13 and the ROCm venv; absent and
healthy bnb both stay consistent across hip and cuda.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop the importlib import left over from the find_spec probe on #7502
* Address the third review on #7502: exact-name bypass and a forwarded bnb config
Both findings hold up, so both are fixed.
1. use_exact_model_name=True skipped the guard entirely. load_in_4bit defaults
to True, so on a host without bitsandbytes
FastLanguageModel.from_pretrained(name, use_exact_model_name=True) kept 4bit
set and failed downstream. That option suppresses repo-name remapping and
cannot make bitsandbytes available, so it has no business gating a capability
check. Ungated at both sites.
2. A user-supplied quantization_config survived the fallback. It sets
load_in_4bit/8bit at the top of from_pretrained and stays in kwargs, so
clearing the local flags still let Transformers rebuild the bnb quantizer.
Now dropped as part of the fallback.
One correction to the second suggestion: it cannot be dropped whenever the
fallback runs. quantization_config also carries GPTQ, AWQ, fp8 and torchao
configs, which have nothing to do with bitsandbytes and must reach the loader
untouched. The pop is gated on the config actually requesting load_in_4bit or
load_in_8bit, reusing the same dict/attr probe from the top of the function.
Behaviour, exercising the real guard block against synthetic inputs with
use_exact_model_name=True and bnb unusable:
default 4bit, no cfg 4bit=False 8bit=False
explicit 8bit, no cfg 4bit=False 8bit=False
BitsAndBytesConfig(4bit/8bit) 4bit=False 8bit=False config dropped
dict bnb config 4bit=False 8bit=False config dropped
GPTQ config 4bit=False 8bit=False config SURVIVES
fp8 dict 4bit=False 8bit=False config SURVIVES
Nothing changes when bitsandbytes works: the whole block is inside
`if not ALLOW_BITSANDBYTES`.
Tests: an ast walk requires neither guard to reference use_exact_model_name in
its test, and requires each to pop quantization_config behind a _wants_bnb
check, so an unconditional pop fails too. Re-gating one guard or removing one
pop reddens a test each. 6 passed on 3.9, 3.13 and the ROCm venv.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address the fourth review on #7502: FastModel never reached the 16bit path
Both findings are real, and the second one meant this PR did not actually
deliver what it advertises for FastModel or vision loads. Reproduced first.
1. patch_compiling_bitsandbytes() ran unguarded at the top of
FastModel.from_pretrained, and unsloth_zoo's copy imports bitsandbytes
unconditionally (patching_utils.py:40). So every FastModel call on a
bnb-less host died there, whatever the arguments:
FastModel(load_in_16bit=True) -> ModuleNotFoundError at patching_utils.py:40
FastModel(full_finetuning=True) -> ModuleNotFoundError at patching_utils.py:40
The FastLanguageModel path already wraps this call in try/except with a
warning, and its comment even says "Mirror FastModel" - FastModel was the
unwrapped one. Wrapped it the same way, so behaviour is unchanged wherever
bitsandbytes imports.
2. The mode-exclusivity check ran before the capability fallback. load_in_4bit
defaults to True, so load_in_16bit=True made
int(load_in_4bit) + int(load_in_16bit) == 2 and raised "Can only load in 4bit
or 8bit or 16bit" before the fallback could clear the unavailable 4bit
request. Moved the fallback ahead of that check.
After both, the same three calls get past every bitsandbytes gate and reach
model resolution, failing only on the deliberately fake repo name used by the
probe. Nothing changes when bitsandbytes works: the fallback is still inside
`if not ALLOW_BITSANDBYTES`, and the wrapper only swallows an import that
previously crashed the load.
Tests: the mode check must be preceded by an ALLOW_BITSANDBYTES fallback in the
same function, and no call to patch_compiling_bitsandbytes may sit outside a
try. The ordering assertion is scoped to the enclosing function on purpose - my
first version compared line numbers file-wide, so the other loader's guard
satisfied it and the negative control passed when it should have failed. With
the scoping fixed, moving the fallback back after the mode check reddens it, as
does unwrapping the patch call. 8 passed on 3.9, 3.13 and the ROCm venv.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add Unsloth desktop deep links
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address deep-link review feedback
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
studio-tauri-smoke.yml only ever built the crate, so none of its ~100 unit
tests executed. Running them surfaced two that were broken on platforms CI
never exercised:
- non_utf8_import_name_preserves_csv_extension built a filename containing a
raw 0xFF byte. Linux stores that fine, macOS enforces UTF-8 on APFS/HFS+ and
refuses to create it, so the test panicked on the unwrap. Skip when the
filesystem rejects the name; the branch under test is only reachable where
such a file can exist.
- losing_a_studio_package_changes_the_fingerprint created the posix venv
layout unconditionally, but site_packages_dirs() only walks
lib/<pyver>/site-packages on unix and looks at Lib/site-packages on Windows.
The dist-info was therefore invisible to the fingerprint there, removing it
changed nothing and the assert_ne could never hold. Build the layout the
code actually reads for the target platform.
Add the cargo test step to the existing Tauri job, where the toolchain and
WebKit dev packages are already installed.
The toolchain strip never ran on the automatic triggers: inputs exists only for
workflow_dispatch, and GitHub coerces '' and false alike to 0, so
`inputs.strip_toolchain != false` was false. Confirmed on a pull_request run
where the strip step reports skipped. Gate on the event instead.
Also: scrub the Machine and User registry PATH, since install.ps1 rebuilds
$env:Path from them mid-install and the toolchain came back; stop dropping
WindowsApps unconditionally, which removed winget on the winget=visible leg too;
fail rather than annotate when a bundle ships no installer or no CLI; run the
bundled installer, which a headless launch never reaches; resolve the newest
desktop-v* release instead of a pinned immutable tag; and give the two macOS
matrix rows distinct artifact names.
* fix(studio): reuse cached tokens while highlighting streaming code blocks
A streaming fence re-enters highlight() every animation frame with the whole
block, so Shiki re-tokenizes it from scratch each time: O(length) per frame and
O(length^2) over the message. One generation made 808 highlight() calls and
tokenized 5.5MB of text to render a 13.5KB block, putting ~50% of the renderer
main thread in the TextMate tokenizer.
Blocks under 2000 chars are unchanged. Above that, a growing fence reuses the
tokens from the last real tokenization and appends the new tail unstyled, with
a full re-tokenize at most every 250ms.
* fix(studio): render the streamed tail unstyled and always converge
Two defects found while property-testing the reuse path:
- plainLine() spread the template token, so newly streamed lines inherited the
first token's colour instead of the default foreground. Emit a bare token.
- A reused result could be the final one if the caller stopped re-rendering,
leaving the tail permanently unstyled. Schedule a trailing re-tokenize so a
reused run always converges.
* fix(studio): key the highlight cache per fence and keep tokens paired with code
Review found four real defects in the previous approach:
- entry.code advanced at dispatch time while entry.result still held the older
tokens, so a reuse could slice one against the other and drop text from the
cached run's final line.
- A finished fence re-rendered with identical code re-dispatched every frame,
keeping the per-frame cost for the rest of the stream.
- All fences of one language shared a single entry, so sibling fences evicted
each other and both were fully tokenized on every render.
- An overdue trailing timer could dispatch stale code after a newer dispatch.
Cache is now one slot per fence, matched by longest prefix. code and result
only ever move together, an exact match is served straight from cache, and a
direct dispatch or a slot eviction cancels any pending trailing refresh.
* Studio: adopt synchronous highlight results and use a monotonic throttle
@streamdown/code answers out of its own cache synchronously and never invokes
the callback in that case. dispatch() ignored that return value, so the slot
kept pointing at the older tokens. On the trailing refresh, where nothing else
consumes the return, that left the fence showing its unstyled tail until an
unrelated remount. Adopt the synchronous result on both paths and hand it to
the pending callback.
Drive the throttle off performance.now(). Date.now() is wall clock, so a
backward step from an NTP correction or a resume from sleep makes elapsed
negative, which pins the reuse branch on and schedules the trailing refresh by
the size of the step.
* Tighten code-plugin comments
---------
Co-authored-by: shimmyshimmer <info@unsloth.ai>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* fix(rocm): prefer system LLVM runtime on native Linux
* Fix/adjust the nested LLVM probe for PR #7448: lib64 hosts and non-directories
Two gaps found while simulating the fix against real ROCm layouts.
1. lib64 hosts got no LLVM dir. The candidate was built from the HSA dir, so a
host with libhsa-runtime64 under lib64 probed <root>/lib64/llvm/lib. ROCm
installs LLVM under <root>/lib/llvm regardless, so that host kept binding
system libamd_comgr to the bundle's libLLVM: exactly the bug #7446 reports.
Probe both spellings, the HSA dir's own first so a genuine lib64 layout still
wins. When lib_sub is already "lib" the seen set collapses them.
2. os.path.exists accepted a non-directory. The serve-time caller joins these
straight into LD_LIBRARY_PATH with no is-dir filter, so a file named
llvm/lib reached the loader. os.path.isdir instead.
Verified on a 27-case matrix built from real directory trees (not mocks), run on
both Windows and Linux against three revisions: main, this PR as-is, and this
commit. Zero regressions and zero reorderings of the pre-existing entries in
every case, and the installer and launcher copies never disagree. The lib64 case
goes [lib64] -> [lib64, lib/llvm/lib]; the file case drops the bogus entry; a
symlinked llvm/lib resolves correctly on Linux.
End-to-end loader check: built real ELF objects mirroring the shipped bundle
(RUNPATH=$ORIGIN, an incomplete libLLVM.so.23.0git next to llama-server, system
comgr from /opt/rocm/lib) and reproduced the reported failure verbatim, then
confirmed the prepend clears it:
before undefined symbol: LLVMInitializeSPIRVTarget -> after exit 0
Test helper now patches os.path.isdir alongside os.path.exists, else every fake
host reports its nested llvm dir as missing. New cases: lib64 finding llvm under
lib, lib64 preferring its own when both exist, and a real-filesystem check that a
non-directory is not prepended. Removing the lib fallback from one copy reddens
three tests including the two-copy parity guard.
tests/studio/install: 1361 passed on Linux, 4 pre-existing environmental
failures unchanged (3 managed-node-runtime under root, 1 the real /opt/rocm case
already covered by #7397). 30/30 on the helper suite on Windows and Linux.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: add durable Deep Research workflows
* Studio: preserve research integration after upstream updates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep research worker compatible with Python 3.11
* Studio: address Deep Research lifecycle review
* Studio: preserve durable research recovery
* Studio: preserve research stream and context
* Studio: harden research sources and limits
* Studio: align research with shared chats
* Studio: guard durable research actions
* Studio: protect durable research turns
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: deepen durable research decisions
* Studio: protect research prompts and queries
* Studio: slim research stream deltas
* Studio: preserve research evidence and citations
* Studio: harden Deep Research (CI, prompt injection, query PII, config, citations)
- Fix backend CI: add research_runs_router to the synthetic routes stub in
test_desktop_auth so studio.backend.main imports under the health-check test.
- Escape prompt-delimiter tags in the decision and synthesis prompts so gathered
web/document content cannot close an <untrusted_...> wrapper and inject
instructions into the local planner/decision/synthesis model.
- Extend the public-query sanitizer to redact Luhn-valid payment cards, phone
numbers, non-global IPs, and labeled private identifiers before a query can
reach web search.
- Reject nested credential keys in inferenceRequest and ragScope, not just
top-level keys, when persisting a durable run config.
- Treat maxSources as one budget shared across web and document sources
(collection and resume paths) instead of per type, which allowed up to 2x the
configured cap.
- Preserve document citations whose filename contains a closing bracket by
tokenizing valid citations before stripping invalid ones.
- Persist Deep Research off when switching to an external model and when enabling
Web Fetch so a refresh cannot rehydrate a mutually-exclusive state.
- Add regression tests for the query, prompt, citation, and config hardening.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make the research claims table migration atomic
The owner-scoped to global claims migration ran its RENAME, CREATE, INSERT and DROP in autocommit, so an interruption after CREATE left the new table empty, orphaned the rows in the legacy table, and never re-triggered. Wrap the rebuild in an explicit transaction so a crash rolls back cleanly and the migration re-runs on the next boot.
* Studio: block message edits and regeneration during an active research run
After a reload a durable research run is followed by the research store rather than an assistant-ui run, so thread.isRunning is false while research is still active. Message edit, refresh and the edit composer previously gated only on isRunning, which let a normal generation start alongside the running research run. Gate them on the active thread's research state as well.
* Studio: keep the plan review mounted through approval
Keying PlanReview on planRevision remounted it mid-approve when updateResearchPlan bumped the revision, resetting the local pending flag and re-enabling Start research while the approve was still in flight, which allowed a duplicate approve. Key on runId only.
* Studio: drop the redundant deep-research persistence change
setCheckpoint already persists Deep Research off for external models at the top of the function, so the added saveBool was a duplicate, and clearing Deep Research from setWebFetchToolsEnabled guarded a state that is not reachable (Deep Research is local-model only while the Web Fetch pill is external-provider only). Revert both to the pre-hardening version.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden Deep Research citations, query privacy, and message protection
Address review findings in the Deep Research backend:
- Escape an unbalanced ")" in citation destinations so a source URL cannot
close the markdown link early and inject a second link, keeping balanced
parentheses literal.
- Match raw-URL citations on whole tokens so a URL sharing another URL's
prefix is no longer partially rewritten.
- Redact non-global IPv6 addresses in public search queries, matching the
existing IPv4 handling.
- Detect credential key names after normalizing case and separators so nested
openaiApiKey, accessToken, and clientSecret values cannot be persisted.
- Reject client edits to server-managed research prompts and reports at the
storage layer; only the internal writers pass allow_research_update.
- Scope research searches to the first allowed domains instead of dropping
site scoping for large allow lists.
- Persist the same fetch evidence bound used during live synthesis so a
resumed run is not shortened.
- Scope run completion so it only replaces this run's message parts.
Add regression tests for the above.
* Studio: fix Deep Research SSE framing, source counts, and favicon privacy
- Normalize the whole SSE buffer so a CRLF split across transport chunks
still frames events.
- Count web and document sources together in the activity header so a
RAG-only run is not shown as zero sources.
- Cap the plan editor at the run's configured maxSteps instead of a
hard-coded 30.
- Add an allowRemoteIcons opt-out to the sources components and disable
third-party favicon requests for research sources so visited domains are
not leaked.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address final Deep Research review findings
* Studio: fit Deep Research synthesis evidence to loaded context, add opt-in web grounding
Size the synthesis evidence budget to the loaded model context so the prompt is not
silently truncated on small contexts. When the evidence overflowed the window the report
degenerated (it echoed the evidence tail instead of writing); the budget now reserves tokens
for the prompt scaffolding and converts the remainder to chars, keeping the full cap when the
context is unknown.
Add opt-in web grounding for auto-read: read the top search results, ingest them into an
ephemeral RAG scope, hybrid-retrieve the passages most relevant to the question with the
existing knowledge-base retriever, and fold those chunks into the step evidence. The scope is
per call and deleted afterwards, so a user's knowledge base is never touched.
Off by default; enable with UNSLOTH_RESEARCH_AUTO_SCRAPE=1. Gated per run by
budgets["maxAutoScrape"], so runs created without it keep legacy snippet-only behavior, and
grounding is skipped when the loaded context is too small for the prompt.
Add tests for the adaptive evidence budget, scraped-text cleaning, the ephemeral web-RAG
retrieval and scope cleanup, and the auto-read evidence path.
* Studio: read Deep Research synthesis context from the inference orchestrator
Make the adaptive synthesis-evidence budget actually engage in the normal Studio
architecture. _loaded_context_length read core.inference.inference, the low-level backend that
lives in the model subprocess and stays unpopulated in the main web process where the research
supervisor runs, so it returned None and the budget silently fell back to the 32000 character
cap (leaving the report exposed to the truncation this was meant to fix). Read the inference
orchestrator instead, and the llama.cpp backend for GGUF, mirroring
routes.inference._monitor_context_length so the budget sizes to the context the API layer
serves. Verified on a running server: at a 12288 token load the probe now reports 12288 and the
budget adapts to 24576 characters instead of the 32000 fallback.
Also:
- Reserve context for the generated report as well as the prompt scaffolding (raise the reserve
to 4096 tokens) so evidence does not crowd out the output on a small window.
- Honor a numeric UNSLOTH_RESEARCH_AUTO_SCRAPE by passing the per-run maxAutoScrape as the page
cap to the scraper, instead of always reading the maximum.
- Guard the web-RAG connection acquisition so a get_connection failure returns the documented
empty result rather than propagating.
- Add a synthesis-context test that patches the real backend accessor (not the probe itself) so
the production wiring is exercised, plus a scrape page-cap test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden Deep Research query redaction and research autosave
- research_runs: extend the opaque-token allowlist so unlabeled Hugging
Face (hf_) and GitLab (glpat-) tokens are redacted before a query can
reach web search, without over-redacting public model or version ids.
- runtime-provider: for a server-managed research message, echo the
backend-stored metadata verbatim on autosave. Merging the client
metadata re-added client-only fields the server never persisted, so the
server-side guard saw a diff and rejected every streamed or snapshot
update with 409.
* Studio: keep composer tool pills always accessible after merge
The merge left the composer line marked always-expanded (data-expanded
"true") while the inner pill row was still gated behind composerExpanded,
so the Search and Code toggles disappeared once the permission mode was
"off" with no other toggle set. Render the primary tool pills
unconditionally, matching the always-expanded layout, and drop the now
unused composerExpanded and permissionMode locals. Fixes the Chat UI
Playwright check that asserts the Search and Code pills stay visible.
* Studio: update Deep Research composer contract to always-expanded layout
The always-expanded composer no longer routes effectiveDeepResearchEnabled
through a composerExpanded expression, so the frontend contract now checks
that it gates the Deep Research composer button render instead.
* Studio: do not bind a research run to a populated assistant reply
create_run adopted any assistant message under the user turn whose
researchRunId was unset, including a prior answer reused by a retry. On
completion _update_assistant drops the untagged text and source parts, so
that answer was silently overwritten. Only bind to an empty placeholder or
this run's own message, and reject a reply that already carries content.
* Studio: harden Deep Research synthesis budget, prompt shielding, and message protection
- research_runs: split the synthesis evidence budget evenly across notes so a
small context still keeps a slice of every research step instead of dropping
the later steps after the earliest ones fill the budget.
- research_runs: shield the research question and approved plan before placing
them in the decision and synthesis prompts, so a closing delimiter in either
cannot escape its block and inject sibling sections.
- research_runs: redact bearer authorization tokens from public search queries.
- studio_db: include attachments in the research-message change check and guard
direct attachment deletion, so server-managed research prompts and responses
cannot be mutated through the attachment paths.
- chat_history: map the protected-message conflict on attachment deletion to 409.
* Studio: strip invalid document citations that contain brackets
The invalid-citation regex stopped at the first closing bracket, so a
citation whose filename contained brackets left its tail (".pdf, p. 9]") in
the report. Match a balanced bracketed span so the whole invalid citation is
removed; valid citations stay protected by the earlier tokenization pass.
* Studio: free the RAG search slot when a lookup times out or is cancelled
The bounded knowledge-base search held the sole admission slot in a detached
worker until the search returned, so a lookup that outlived its timeout (a
stalled embedding or blocked vector call) kept the slot forever and starved
every later lookup, disabling knowledge-base retrieval globally. Release the
slot from the caller when it stops waiting, exactly once, so a detached worker
finishes without re-holding it.
* Studio: remove Websites label from research composer
* Studio: fix Deep Research review findings (RAG slot bound, orphaned workers, hardening)
- Bound the shared RAG search slot to one running worker. The search that is
doing the embedding/index/GPU work now owns the admission slot until it
finishes, instead of freeing it on caller timeout while the detached worker
keeps running, which let a second search enter and stack concurrent work
behind the capacity-of-one semaphore.
- Cancel active research runs before deleting their thread, project, or all
history. Deleting cascade-drops the run row, but the worker only notices at
its next lease check, so it could keep doing model/web/RAG work for a run
that no longer exists; signalling cancel first shortens that window.
- Shield the planner prompt's conversation and question with _shield_untrusted,
matching the decision and synthesis prompts, so untrusted text cannot forge
planner delimiters.
- Do not let a research key-revocation failure replace a successful
non-streaming completion; log it like the streaming path does.
- Include created_at in the protected research-message guard so a client cannot
reorder server-managed prompt/response messages while leaving the body intact.
- Reject non-scalar ragScope values; a nested container evades the
sensitive-key scan when its inner keys are unlisted and would reach retrieval
code that expects a scalar scope id.
Adds regression tests for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: remove research composer globe icon
* Studio: use Hugeicons telescope in research composer
* Studio: use Telescope02 icon in research composer
* Studio: standardize Deep Research telescope icons
* Studio: move Deep Research below web and code tools
* Studio: merge grounded page excerpts with search snippets instead of replacing
When auto-scrape grounding retrieved page-body chunks, it replaced the raw
search-result text for that step. If the retrieved chunk was a distractor or
dropped the key fact, the answer-bearing search snippet was lost and grounded
runs regressed below snippet-only accuracy on factual questions (e.g. returning
Apache 2.0 instead of the Qwen License, 403 instead of 404, or a single mirror
diameter instead of the sum).
Keep the search snippets and append the grounded excerpts as supplementary
evidence via a small _merge_scraped_evidence helper. Grounding stays opt-in and
off by default, so legacy runs are unchanged. Adds regression tests.
* Studio: improve Deep Research synthesis
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden Deep Research synthesis flow
* Studio: validate Deep Research derived context
* Studio: align Deep Research synthesis evidence
* Studio: restore Deep Research synthesis state
* Improve Deep Research source queries
---------
Co-authored-by: alkinun <alkinunl@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
* Fix Windows no-torch setup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix no-torch env normalization on Windows
* Accept on for Windows no-torch mode
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep no-torch mode across studio update on Windows
Guarding the direct torch/Triton install made `install.ps1 --no-torch`
actually produce a torch-free venv, which then broke the next
`unsloth studio update`. That path exports no UNSLOTH_NO_TORCH, so
$NoTorchMode was false, the stale-venv check read the missing torch as a
broken venv, and setup tried to delete the venv it was running out of:
[ERROR] Could not remove stale venv: Access to the path 'python.exe' is denied.
That teardown can never succeed there, because setup.ps1 runs via
unsloth.exe out of that same venv. The same gap also let the shared
dependency pass reinstall torch from PyPI, unpinned, into a GGUF-only
environment.
install_python_stack.py now records the mode in the install manifest and
setup.ps1 reads it back when no env var is exported, then re-exports a
canonical value for the dependency pass (setup.ps1 drops the manifest
before invoking it, so the child cannot repeat the lookup). The key is
additive and MANIFEST_SCHEMA is unchanged, so existing manifests stay
valid and a missing key keeps today's behaviour.
Also:
- read_manifest() caught only OSError, but UnicodeDecodeError is a
ValueError. That is now on the installer's import path, so a manifest
re-saved as ANSI or truncated mid-write would abort every install.
- The env predicate now trims surrounding whitespace, matching the
Python side.
- The Windows update smoke workflow asserts the update leaves the venv
GGUF-only, which is what would have caught this.
Known follow-up, pre-existing: an install killed between the manifest
drop and the dependency pass leaves no recorded mode, so a later update
still walks the stale-venv path. Closing that needs a marker the
installer never drops.
* Persist no-torch mode in a marker the dependency pass cannot drop
The install manifest alone was not enough. Both setup.ps1 and
install_python_stack.py remove it before every dependency pass, and it is
only rewritten on success, so a no-torch install interrupted in between
left nothing recording the mode. The next update then resolved no-torch
as false, read the expected missing torch as a stale venv, and tried to
delete the environment whose python.exe was running it, which leaves the
install unrepairable from the CLI.
Add .unsloth-no-torch next to the existing .unsloth-studio-owned marker,
written before the pass and cleared when torch is wanted. setup.ps1
writes it as soon as the mode resolves, so the window between the
manifest drop and its own torch install is covered too.
Read order stays manifest key first, then marker, so migrating out of
no-torch is never blocked by a marker an earlier run left behind. Neither
present still reads as "install torch", so nothing changes for installs
made before either existed.
Also adds the AGPL-3.0 header the new test file was missing.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
* Gate the sed commands that run a shell
GNU sed executes a shell through its `e` command, both as a standalone
command (`sed -n '1e CMD' file`) and as an `s///e` flag that runs the
pattern space. It goes through popen(), so it is a literal `sh -c`, but
the terminal scan only ever saw `sed` at command position and treated the
program text as an ordinary argument.
That left `sed -n '1e rm -f victim' /etc/hosts` running with no prompt in
auto mode, and `_find_blocked_commands` returning nothing for it, so the
hard blocklist that applies in every mode missed `rm` as well.
Screens the program the same way the awk arm does. `-e` values are joined
with newlines first, since that is how sed assembles them: `sed -e '1a\'
-e 'e CMD'` appends a literal line and runs nothing, so judging the pieces
separately would prompt on a benign script. The scan then steps over every
region where `e` is data rather than a command: address and substitution
regexes, replacements, `a/i/c` text, `r`/`w` filenames, `b`/`t` labels and
comments. That keeps the common idioms silent, including `:e;N;$!be` loop
labels, `s/e/E/g`, and `s/a/b/we out.txt` where the `e` belongs to the `w`
filename and sed does not execute.
The blocklist scan recurses into a literal `e` payload the same way it
already does for `bash -c`. A bare `e` or an `s///e` can only be prompted,
since what they run is the pattern space, which is input-file text that is
not knowable statically.
Verified against real GNU sed 4.9 rather than the manual: 80 commands run
for real with a marker payload, comparing what sed actually executed
against the classifier, with no mismatches in either direction.
* Close five ways a sed program hid its shell payload
Review found five shapes the first pass missed. All five execute on GNU
sed 4.9, checked by running them rather than reading the manual.
A payload line ending in a backslash continues onto the next line, so the
scan now ends an `e` at an unescaped newline and unescapes the text the way
sed's read_text does. That is what resolves `r''m` back to `rm` for the
blocklist.
A sed comment ends at a real newline, but the terminal scan had already
replaced every newline with `;`, including newlines inside quotes, so
`# comment` swallowed the rest of the program. The sed arm now also sees a
variant where only unquoted newlines become separators, built on a
character-by-character quote scanner rather than a regex: an apostrophe in
a double-quoted word mis-pairs under a regex and inverts the state, which
opened a bypass while this was being written.
Everything attached to `-i` is a backup suffix, so reading `-ifoo` as an
attached `-f` lost the real script. Replaced the shared short-flag helper
with sed's own option grammar, which also fixes `-l 5` and
`--line-length 5` eating the script as their operand.
A sed child of `find -exec` was never recorded, so the blocklist skipped
its payload.
Substituted text splices straight into the program, and an address is as
good a place as any to open `;e CMD`, so a command substitution anywhere
in the program is treated as unresolvable. Scoped to the program: a
substitution in a file operand still runs, a `$(` or backtick inside single
quotes is literal, and parameter and arithmetic expansion are untouched.
The cost is that a substitution used to build a program now asks.
Bounding the -exec walk keeps the blocklist linear; without it a repeated
`-exec sed` line went quadratic.
Verified against real GNU sed across 103 commands run for real, no
mismatch in either direction.
* Fail closed on padded sed lines, and stop gating sed --sandbox
Four more from review, each checked by running it rather than reading the
manual.
The cap that keeps the argument walk linear was itself the bypass: padding
a line with 128 valid options pushes the script past it, and an empty
program read as proof the command only edits text. The budget is now shared
across the sed words on a line, so a lone sed reads its whole argument list
while a line packed with sed words keeps the floor that holds the walk
linear, and overflow fails closed instead of falling through.
The substitution scan counted parentheses without consulting quote state,
so a quoted paren in the substitution body left the span unterminated and
the program never matched. It now balances through the same quote scanner
used elsewhere, since a substitution body reopens quoting.
A wrapper between -exec and its child hid the child from the blocklist.
Following the wrapper also fixes the neighbouring blocked-name check, which
missed find . -exec env rm the same way. The wrapper's own name is still
screened: -exec sudo rm reports both.
sed --sandbox and --posix refuse e outright and exit 1, so gating them was
prompting for something that cannot run. They are now inert, except after
--, where the flag is an input filename and the script still executes.
env -u still hides a child from the blocklist, on this path and at top
level. That is pre-existing and left alone here.
* Resolve the sed program through find, wrappers, globs and variables
Five more from review, each run against real sed rather than read off the
manual.
find's -exec ends at + or ;, but the sed argument walk ran past it into the
next predicate, where a following -exec grep -e safe was read as sed's own
-e and discarded the real script. Stopping at the terminator also removes a
false prompt, since -exec was being parsed as -e xec and inventing a payload.
Hopping a wrapper skipped its name but not an option that takes a separate
operand, so env -u FOO sed returned FOO as the child. The table this file
already keeps for wrapper options covers it, moved up so both layers share
it. That also settles the top level: env -u PATH rm -rf x now reports rm,
as do env --unset, stdbuf -o L and xargs -I {}. Two false positives go with
it, timeout -s KILL 5 rm blaming the signal name and env -u kill blaming a
variable name, while timeout -s KILL 5 kill -9 1 still reports kill.
A program held in a variable was invisible: the assignment regex stops its
value at whitespace, so a program containing a newline never entered the
map in any pass. Resolved at the token level instead, where the value is
already whole. Both the written and the resolved program are screened,
since either can hold the e.
A command-position glob that can resolve to sed is treated as sed. The
auto gate already asks about any unresolved command glob; this is for the
blocklist, which did not know the name.
Inside double quotes a backslash makes the next character literal, so
sed "s/\$(CC)/gcc/" runs no substitution and should never have asked. The
quote scanner now reports an escaped character under its own state.
Left open: on Windows the blocklist lexer keeps quoting in its tokens, so
a multiline program held in a variable resolves there but not to a name
the blocklist reads. The prompt still fires on every platform.
* Ask when the sed program is not a literal we can read
Two from review, and the second one changes the default rather than adding
another case.
sed --sandbox and --posix were being read as disabling e for the whole
invocation. They disable exactly the scripts written after them: sed
compiles each -e as that option is parsed, and the positional script only
after the option list, so sed -e '1e CMD' input --sandbox runs the payload
with no POSIXLY_CORRECT needed. Suppression is now positional. Reading
POSIXLY_CORRECT out of the command text was considered and dropped as
unsound, since export or an outer bash -c puts it somewhere the text does
not show.
A program built by a parameter transformation was invisible: only bare
$NAME and ${NAME} were resolved, so ${p#x } passed through untouched. Rather
than add operators one at a time, a program that still holds a live
expansion after resolution is treated as unreadable and asks. Unhandled
expansion forms are now safe by default instead of silent, which also
closes ${p%Z}, array elements, printf -v, read, and p=$(...) whose binding
shlex had been truncating to a bare $.
Arithmetic is collapsed rather than exempted. It can only ever evaluate to
an integer, so it cannot spell a sed command, but leaving it as written let
"$((c+1))e CMD" read as an append-text command that swallowed the payload.
The cost is that a double-quoted program holding an unassigned variable now
asks: sed "s/$OLD/$NEW/g" f. Measured at 24 of 169 realistic invocations,
all of that one shape. Exempting it would trade enumerating expansion
operators for enumerating assignment forms, and four of the bypasses above
sit outside the assignment pattern, so the blanket rule stays.
Left open: -f prog.sed is still unscreened, since the program is in a file.
* Decide where a sed scan stops by context, not by token text
Four from review, two of them exploiting fixes from earlier rounds.
Stopping the sed walk at a + or ; token read the text after shlex had
already removed its quoting, so a quoted file operand looked exactly like
a find terminator and the scan gave up before the -e that followed. sed
still compiles that -e, because getopt permutes. Termination is now decided
by token index: a separator counts only if it was unquoted, and + or ; only
while a find or fd exec action is open, which is the only place quoting
does not matter. The same shape works with & | ( ) and }, so all of them
are covered.
The assignment map kept the first binding for a name, but the shell uses
the most recent one before the command. Bindings are now ordered and only
those preceding a given sed are folded in, with a later one replacing an
earlier. A value that is not itself literal clears the name rather than
leaving the older literal standing, which would otherwise have dressed an
unread program up as a safe one.
Exhausting the wrapper budget under find -exec returned the same answer as
finding no child at all, so a long enough chain of wrappers hid whatever
followed. It now reports overflow and blocks the chain word. This was
hiding more than sed: the same shape hid a plain rm.
fd spells its exec flags -x, -X, --exec and --exec-batch, none of which
were routed into the nested scan. They are now, but only while a find or
fd word is in scope and no action is already open, so a -x that belongs to
a child command is left alone.
Prompt rate is unchanged at 45 of 169 realistic invocations; this round
adds no new prompts.
* Drop the words the shell removes before a command runs
Two from review, both verified to run for real.
A redirection is performed by the shell and never reaches the command, but
the words stayed in the token list and the first of them was taken for
sed's positional script, so the real one behind it was never read.
`sed </dev/null '1e touch MARKER' input` creates the file, and so do the
`>`, `2>`, `2>&1`, `&>`, `>|` and here-string spellings. Redirections are
now recognised as spans and skipped: the target may be glued on, be the
next word, or sit one further along when a punctuation character splits
the operator. A skip is honoured only where sed would take the word as an
argument, so a pending -e/-f/-l value is still read.
The same words also hid a command outright. `> out.txt rm -rf victim` and
`2>&1 rm -rf victim` both really delete, because the redirection target
was read as the command word and the rm behind it landed in argument
position, where the always-on blocklist does not look.
shlex emits a RUN of punctuation characters as one token, so bash's `|&`
matched no separator and a sed scan ran on into the NEXT command, taking
its `-e safe` for the real script and dropping the payload. Any token
built only from those characters now ends an invocation, and a quoted one
is excluded the same way a quoted `';'` already was.
The third item from that review, `-l N` eating the script as its length
operand, was already closed in 9a5cfddb.
Prompt rate is unchanged at 45 of 169 realistic invocations; this round
adds no new prompts.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Read a sed program from what the shell really hands it
Five from an independent review pass, each verified by executing it.
sed joins its -e and -f sources with newlines, but a source boundary also
closes a line continuation open across it. Reading every -e as one
uninterrupted text let an unreadable -f in the middle hide the piece
behind it: `sed -e '1a\' -f /dev/null -e 'e CMD' input` runs CMD while the
same line without the -f only appends text.
A program flag ahead of the positional script makes that word an input
file. One behind it does so only while getopt permutes, and
POSIXLY_CORRECT turns permutation off from outside the command text, so
the positional is now read as a script as well. The suppression that a
flag written first performs is unchanged.
xargs builds the argv of the command behind it, appending what it reads on
stdin and substituting it into an -I placeholder, so the program need not
be in the text at all. A sed whose program is empty or is only the
placeholder is failed closed. The ordinary idioms are untouched: their
program is present and the placeholder stands where the file goes.
Only a word that really changes shell state rebinds a program held in a
variable. An assignment-shaped argument, one inside a subshell and one
used as a command's environment prefix all leave the variable alone, and
recording them replaced a payload with a value bash never assigned. A
conditional assignment after && or || may or may not run, so it clears the
name rather than being guessed at.
Exec-flag forwarding now starts only at a command word. Any token spelled
fd or find used to turn it on, so a -x or -exec in the text after one was
read as an exec flag and its neighbour hard-blocked; `echo fd -x rm` and
`grep fd -x rm file` were refused outright. A command-position glob bash
resolves to find is still recognised.
Prompt rate is unchanged at 45 of 169 realistic invocations.
* Judge a sed program against what getopt and find really do
Seven from review, each verified by executing it.
A redirection is removed wherever it stands, including where an option
value goes, so `sed -n -e >out '1e CMD' input` takes the word behind it as
the script. The skip is now honoured ahead of a pending value rather than
after it. The target of a detached redirection may itself look like an
option or a quoted operator, and the shell hands it to open() either way,
so `sed > --sandbox '1e CMD' input` and its `> ';'` twin no longer leave
that word standing as a sed flag or script. Only a bare operator is
refused, which is a malformed line.
A program flag written behind the positional script and the positional
itself are ALTERNATIVES, since permutation decides which sed compiles and
nothing in the text settles it. They were joined into one program, where an
unterminated command in the one swallowed the other: `-e safe` is an `s`
with delimiter `a` and no closing one, and it ate the payload behind it.
Each source is now scanned on its own.
find closes its batched form at `{} +` only, so a `+` anywhere else is an
ordinary argument it hands the child. Stopping at one threw away the script
behind it. The `;` spellings need no such test: a quoted `';'` and an
escaped `\;` reach find as the same word and it stops at either, which the
`;` twin of that line confirms by not executing.
An `-f` naming a stream (`-`, /dev/stdin, /dev/fd/N) takes the script off
stdin, which the same command line may well supply through a heredoc. That
is ignorance rather than safety, so the sed fails closed. A named program
file is unreadable in a different way and is unchanged.
bash expands the program word before sed is started, so in a directory
holding a suitably named file `sed *` runs whatever that file contains.
A program word carrying an unexpanded glob now fails closed. Quoted
programs expand nothing and a glob among the file operands is not the
program, so ordinary work is untouched.
Prompt rate is unchanged at 45 of 169 realistic invocations.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep command position and quoting intact through the sed scan
Six from review, two of them regressions the previous commit introduced.
Scoping exec-flag forwarding to a command word lost that position at a
shell keyword and across a wrapper's own operands, so `if true; then find
. -exec rm ...` and the `env -u FOO find ...` and `timeout 5 find ...`
shapes stopped blocking rm entirely. Keywords now keep the position and
wrapper options and their operands are stepped over, the way the command
walk already does.
Reading any operator-shaped token as a separator did the opposite: a
QUOTED one is data the command receives, so `printf '%s' '|&' rm` and
`grep '|&' rm file` were refused although they run nothing. The walk now
applies the same quoted-index exclusion the layout pass does, which also
clears the older `printf '%s' ';' rm` false positive.
ANSI-C decoding flattened the word's whitespace, and a sed program ends
its comment at exactly the newline that flattening destroyed. The decoded
text is re-quoted instead, keeping the spaces and the `#` around it, with
the newline standing as a mark so it stays data for whatever command
receives it rather than a place a new one begins.
An assignment inside a function body has not run and may never run, so it
is no longer recorded as the current value; the name is cleared instead,
which is right whether or not the function is later called.
An `-f` taking a process substitution is a generated /dev/fd/N script, and
the lexer ends the invocation at the `(` before the operand is read at
all. A still-pending program operand now fails the sed closed.
Live expansions were compared against the raw command spelling while the
sed program carried the post-lex one, so an escaped expansion read as
already resolved. Both sides are keyed without their escaping, which can
only make a spelling match and so errs closed.
Prompt rate is unchanged at 45 of 169 realistic invocations.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Read the sed program from the word the shell actually passes
Six from review, four of them bypasses and two false alarms.
find rewrites `{}` with the pathname it found before the child ever starts,
so a sed whose whole program is that placeholder was never read. Nested
under xargs it really runs whatever a suitably named file contains. A `{}`
among the file operands, which is the ordinary idiom, is not the program
and is untouched.
A quoted redirection is a word the command receives rather than something
the shell performs, and it was being removed either way, so a `-f` script
file named `>prog` disappeared and took the `-e` behind it out of view.
Quoting is now read from the operator the token opens with, which leaves
`2>'/dev/null'` a redirection with a quoted target.
An apostrophe in an ANSI-C word sent it down the flattening path, which
destroys the newline a sed comment ends at. The apostrophe is re-quoted
the way a shell does it instead.
fd takes the command attached to its short exec option, and only the exact
`-x` and `-X` spellings opened an action, so `-xrm` reached neither layer.
Conversely nothing behind a bare `--` is an option at all, and reading one
there refused `fd -- -x rm`, which merely lists a file.
The set of live expansions covers the whole command, so matching a sed
program against it by text alone attributed an expansion another command
performs to a program that only spells the same thing. Which occurrence it
was decides it now, and single quoting keeps its meaning while double
quoting does not.
Prompt rate is unchanged at 45 of 169 realistic invocations.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten the comments this PR added
Every comment kept says why a rule exists and, where the reason is a
real tool behaviour, names the one command that proves it. What went is
narration of the code, the history of how each fix evolved, and the same
mechanism re-explained at each site that uses it: it is stated once at
the definition now and referred to from there.
Docstrings on the private helpers give what they return and the one fact
that is not obvious; the worked examples they carried are in the tests,
which already run them. The longest block is 8 lines, from 19.
229 lines off the diff. No code changed.
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
findCodeBlockRegions scanned every region found so far for each inline code
match, and accepted inline spans were appended to the same array, making it
quadratic in the number of inline spans. preprocessLaTeX runs on the full
message text every animation frame while streaming and calls it twice.
Fenced and inline matches are both ascending and non-overlapping, so walk the
fenced list with a cursor instead. Only fenced regions can contain an inline
span, so previously accepted inline regions never needed checking.
34,670 chars with 2,100 inline spans: 5.51ms per call to 0.12ms.
Co-authored-by: shimmyshimmer <info@unsloth.ai>
test_shipping_code_names_an_encoding is red on main. #7373 added
sync_marker_llama_backend, whose read_text/write_text pair does not name
an encoding, so both fall back to locale.getencoding():
AssertionError: 2 text read/write call sites in shipping code let the
operator's locale decide the encoding, so they crash or silently
produce mojibake on Windows. Pass encoding = "utf-8":
['studio/install_llama_prebuilt.py:5656: write_text()',
'studio/install_llama_prebuilt.py:5647: read_text()']
Reproduced on a clean checkout of main at 7917c7828: 1 failed, 7 passed.
That guard landed in #7486 a few commits earlier, so the rule predates
these call sites; nothing about the Vulkan work is wrong beyond the
missing kwarg. The create path that writes the same file, 26 lines above
at 5621, already passes encoding = "utf-8", so main is also internally
inconsistent about one file: written as utf-8, read back under the
operator locale.
Scope, stated honestly: json.dumps defaults to ensure_ascii = True, so
the marker this module writes is pure ASCII and round-trips under cp1252
as well as utf-8. The exposure is a marker produced or edited by
something else. A decode failure on the read would not even surface,
because UnicodeDecodeError subclasses ValueError and the surrounding
except (OSError, ValueError) swallows it into the early return, leaving
the backend silently unsynced. So this restores a green suite and makes
the file self-consistent rather than fixing a live crash.
Verified: tests/test_runtime_text_encoding.py 1 failed / 7 passed before,
8 passed after; tests/test_source_read_encoding.py still passes.
* Studio: keep grouped Python scripts visible and save them natively
* Studio: render the executed Python script outside the card collapsible
Ungrouping the aggregate tool group was not enough on its own. Each Python
card still mounts with defaultOpen={isRunning}, so on a reopened turn the
script and its Copy/Download controls stayed hidden behind the card's own
chevron and the reported issue persisted.
Render ToolCodeCell outside ToolFallbackContent for Python, restoring the
behaviour from #7240 that #7455 folded back inside when it unified the code
cell. Status, output and images still collapse. Terminal keeps its command
inside the collapsible: a one-line command is not the artifact a user reopens
a thread to retrieve, a script is.
Verified against a running Studio: reopening a persisted turn with two
adjacent Python calls now shows both scripts and both Download controls with
no clicks, and Download still saves byte-exact script.py.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
test_tensor_abort_cache_invalidated_on_binary_mtime_change bumped mtime by a
single nanosecond. NTFS stores timestamps as 64-bit FILETIME values in 100ns
ticks, so on Windows that bump rounds away, st_mtime_ns reads back unchanged,
the cache key is identical and the stale abort is inherited, and the assertion
sees True where it wants False.
1ms is still a same-second, sub-second change and is exactly representable, so
the case the test exists to cover actually runs. Skip when the filesystem cannot
record any sub-second change at all rather than asserting product behaviour the
platform cannot exercise.
Not caught before because both jobs in studio-backend-ci.yml are
runs-on: ubuntu-latest, so the studio backend tests only ever run on Linux.
Daniel Han