Static classifier:
- fix scope-local walker so a nested def/class reassigning an alias name no longer inflates the outer single-assignment count and drops a real module-level sink alias
- track non-bare compile aliases (builtins.compile, from builtins import compile as comp) for the types.FunctionType(c) code-object gadget
- block child-process file writers at shell command position (touch/tee/cp/mv/mkdir/install/truncate/mkfifo/mknod/shred/unlink): a spawned child runs without the in-process write guard
- expand a literal **{...} unpack in the read scanner so open(**{'file': '../../etc/passwd'}) is resolved
- resolve a pathlib expression bound to a single-assignment name before read methods (p = Path('..')/'etc'/'passwd'; p.read_text())
- keep a wrapper's separated option argument in command position so stdbuf -o L python -c ... still detects the interpreter (env -i rm still caught; no FP on grep patterns)
- treat object.__getattribute__ / type.__getattribute__ as attribute obfuscation, covering gadget dunders and sensitive-module attrs (also closes __closure__ recovery of a guarded wrapper's original callable)
- block runpy.run_path / runpy.run_module execution sinks
- treat shutil.copy*/move SOURCE as a read callee so a .. traversal source is caught
Runtime backstop:
- normalize a bytes realpath (fsdecode) before the workdir prefix compare so a legitimate in-workdir bytes write is not denied by a TypeError; outside bytes writes still denied
Static classifier:
- resolve pathlib expressions passed to open()/read callees so open(Path('/etc') / 'passwd') blocks like open('/etc/passwd')
- flag getattr()/setattr() of an introspection gadget dunder (__globals__, __subclasses__, ...) regardless of receiver
- add the .get() twin of the globals()/locals()/vars() namespace-dict subscript guard
- constant-fold sys.modules[...] and sys.modules.get(...) keys so a concatenated key is caught
- track 'from builtins import __import__ as imp' as a dynamic import alias
- treat deserializer module aliases (pickle, dill, ...) as sensitive targets for getattr/vars/__dict__
- flag code objects executed through types.FunctionType(compile(src, ...), ...), including the c = compile(src); FunctionType(c) two-step
- block language interpreters (python/perl/ruby/node/...) at shell command position: a spawned child runs without the in-process write guard
Runtime backstop:
- pin os.fspath/os.path.realpath to captured originals inside _within so a sandboxed reassignment of os.fspath cannot make realpath resolve an outside write target to an in-workdir path
Scope-aware assignment aliases (extends the per-scope index):
- Resolve single-assignment aliases of dangerous callables in the call's own scope:
e = builtins.eval, im = importlib.import_module (and imp = __import__), and
l = pickle.loads (incl. aliased modules). Previously only bare-name and from-import
aliases were recognized.
- Count function parameters as local bindings so a parameter lexically shadows an
outer sink alias of the same name (fixes a false positive where def f(s): s(...)
with a module-level s = os.system flagged the parameter call).
Sensitive-read scanner:
- Resolve pathlib join receivers -- (Path('/etc') / 'passwd').read_text() and
Path('/etc').joinpath('passwd') -- not just a bare Path(...) constructor.
- Normalize path spellings (collapse redundant separators / '.' and resolve '..')
before the exact / dir checks, so /etc//passwd, /etc/./passwd and
/tmp/../etc/passwd are matched.
- Fold function-local single-assignment path constants (def f(): p = '/etc/passwd';
open(p)), not only module-level constants.
- Flag sys.modules.get('os') as the method-call twin of sys.modules['os'].
Runtime realpath backstop:
- Path.open coerces a str-subclass mode through the base str (matching the other open
wrappers) so a lying __contains__ cannot skip the write check.
- Path.rename/replace/link materialize the target once so a stateful __fspath__ cannot
return an in-workdir path for the check and an outside one for the real call (the
pre-3.11 accessor path where this wrapper is the only confinement).
Adds regression tests across the classifier, aliasing and runtime-backstop suites.
Scope-aware alias resolution (replaces the flat, module-wide alias maps):
- A new per-scope index resolves shell-sink, exec-builtin and compiled-code aliases
with Python lexical scoping. This fixes two problems the flat maps had: a safe
`c = compile('1+1')` in one function no longer shadows a dynamic `exec(c)` in
another (a real bypass), and a `s = os.system` in one function no longer makes a
benign `s = print` call in another look like a shell sink (a false positive), while
still catching a genuine function-local sink and honoring local shadowing of a
module-level alias.
Runtime realpath backstop:
- io.FileIO now passes the MATERIALIZED fspath to the real constructor (a stateful
__fspath__ could otherwise return an outside path to the C constructor).
- Deny an integer fd path for the mutating single-path wrappers (os.chmod(fd) etc.):
a read-only fd opened on an outside file could otherwise mutate host metadata.
Constant-folder allocation DoS:
- Refuse dynamic printf widths/precisions ('%*s', '%.*f') that draw their size from a
runtime argument.
- Bound str.replace / str.join output before it allocates (a long replacement over
many occurrences, or joining many long parts, can build a multi-gigabyte string).
Static classifier:
- Flag builtins / a sensitive module reached through the namespace dict:
globals()['__builtins__'], locals()[...] and globals()['os'].
Adds regression tests across the aliasing, runtime-backstop, const-fold and classifier
suites for every item above.
Runtime realpath backstop:
- Guard the low-level posix / nt module mutators (os re-exports from them, so
posix.open / posix.rename / ... stayed reachable with the originals).
- Guard io.FileIO / _io.FileIO constructors for write modes (a C constructor that
opens a file without routing through open()).
- Add os.mkfifo / os.utime / os.setxattr / os.removexattr (and lchflags) to the
guarded single-path mutators.
- Materialize fspath ONCE per call so a stateful __fspath__ cannot return a workdir
path for the check and an outside path for the syscall (TOCTOU).
- Coerce open() mode through the base str and os.open flags through the base int, so
a str-subclass __contains__ or an int-subclass __and__ cannot lie to the guard.
Constant-folder allocation DoS:
- Refuse str.format templates with a nested width field ({:{}}) driven by an
oversized numeric argument before format() allocates.
Static classifier:
- Reconstruct the full pathlib receiver path (all constructor args, joined) and
accept module-qualified pathlib.Path so Path('/etc', 'passwd').read_text() and
pathlib.Path(...) reads are inspected, not just single-arg bare Path(...).
- Treat builtins.__import__ / __builtins__.__import__ as a dynamic import.
- Count alias single-assignment per function scope instead of tree-wide, so two
functions binding the same local name no longer cancel out and miss a real sink.
Adds regression tests across the runtime-backstop, const-fold, aliasing and
classifier suites for every item above.
Fixes a further batch of P1 bypasses and analysis-time DoS vectors found in review.
Static classifier:
- Decode exec/compile bytes payloads the way CPython does (PEP 263 coding cookie
via tokenize.detect_encoding), then analyze the real source. A bytes payload whose
UTF-8 view is pure comments but whose utf-7 decode runs hidden code no longer slips
through; a payload decoding to a blocked op blocks, a benign one stays allowed.
- Resolve exec-builtin aliases assigned in nested scopes (def f(): e = exec; e(...)),
matching the shell-sink aliasing (stored-once guard keeps it low false-positive).
- Treat deserializer modules (pickle/marshal/dill/...) as dangerous dynamic-import
targets so __import__('pickle').loads(blob) is caught.
- Flag vars(os) / vars(__builtins__) as a module-__dict__ obfuscation, like os.__dict__.
- Inspect the pathlib receiver path for read methods: Path('../../.ssh/id_rsa').read_text()
/ read_bytes() / open() now check the constructor path, not only call args.
- Fold literal os.path.join / posixpath.join so open(os.path.join('/etc','passwd')).read()
is seen by the sensitive-read scanner instead of treated as opaque.
Constant-folder allocation DoS (folding runs in-process, before subprocess rlimits):
- Reject oversized f-string / str.format / %-format widths and precisions before
format() allocates the padded string.
- Reject oversized str padding-method widths (ljust/rjust/center/zfill).
- Cap list/tuple repetition (seq * n) as str/bytes repetition already was.
Runtime realpath backstop:
- Do not publish __wrapped__ on the guard wrappers (functools.wraps would expose the
original unguarded callable, e.g. open.__wrapped__(outside, 'w')).
- Guard the low-level _io.open entry point (io.open / builtins.open originate there).
- Confine os.chdir to the workdir and deny os.fchdir so a cwd escape cannot turn a
later relative read/write into a host-path access.
- Deny fd-based metadata mutators (os.fchmod / os.fchown) that could reuse a read-only
descriptor opened on an outside file.
Adds regression tests across the const-fold, aliasing, exec-recursion and runtime-
backstop suites for every item above.
Harden the code-exec sandbox against bypasses raised in review, keeping the
static gate a pure classifier (it never executes the tool call):
- Shell scan: `timeout` duration/float args (5m, 0.5, 2h) no longer drop the
following command out of command position, and `find -exec CMD ... ;` rescans
the whole slice so a wrapped `env`/`timeout`/`sh -c` target is still caught.
- exec/eval/compile of a bytes payload that is not valid UTF-8 Python now blocks:
those sinks honor PEP 263 coding cookies (e.g. utf-7) that the static UTF-8 view
cannot see; plain ASCII bytes payloads stay allowed.
- Resolve aliased/indirect reaches to exec, dynamic import, sensitive modules and
deserialization sinks: builtins.eval / __builtins__.exec, from builtins import
exec as e, importlib aliases, sys.modules[...] (and the getattr form), os.__dict__,
posix/nt, and pickle/marshal module-and-symbol aliases plus the *.load variants.
- Shell-sink aliasing walks the whole tree, so a function-local `s = os.system`
alias is resolved (the stored-once guard keeps it low false-positive).
- Drop __mro__ / __code__ from the introspection-gadget dunders: on their own they
do not reach an execution primitive and are read by ordinary ML/debug code.
- Refuse folding oversized `bytes(n)` / `bytearray(n)` so static analysis cannot OOM.
Runtime realpath backstop:
- Fail closed on a mutating dir_fd / src_dir_fd / dst_dir_fd (an fd-relative path
cannot be confined by a string realpath) for os and shutil mutators.
- Confine Path.rename/replace/symlink_to/hardlink_to when the destination is passed
as the `target=` keyword, not only positionally.
- Splice the guard after a leading docstring and `from __future__` imports instead
of prepending it, so future-import programs no longer raise SyntaxError while the
sandbox is still established before the first real statement.
Adds regression tests for each gap across the shell, const-fold, exec-recursion,
aliasing and runtime-backstop suites.
On Python <= 3.11, pathlib._NormalAccessor captures io.open (and os.* mutators)
into class attributes at pathlib import time. A C builtin captured there does not
bind on instance access, but a Python wrapper does: self shifts into the next
positional, so Path.open / Path.write_text raised 'open() argument mode must be
str, not PosixPath' once the guard had replaced io.open with a Python wrapper
before pathlib was imported. (3.12+ dropped the accessor, which is why it only
failed on the 3.10 CI leg.)
Import io and pathlib at the top of the guard, before any patching, so the
accessor captures the original builtins, and confine Path.open by wrapping the
public method directly (mode-aware) rather than relying on the io.open patch to
reach it. Direct io.open() writers are still guarded for the zipfile-based cases.
The static filesystem write-confinement (the LOCAL/ESCAPE/UNKNOWN path resolver
_resolve_path / _resolve_path_call plus the _FS_* mutating-op inventory) was the
largest and most complex part of the classifier, and for writes it duplicated the
runtime realpath backstop, which is strictly more robust: it resolves the true
realpath at the syscall boundary, so it also catches dynamic paths, pre-existing
symlinks, and library writers the static pass could not prove.
Make the runtime backstop the single filesystem-write boundary and delete the
static resolver:
- Harden the backstop to close the gaps the static layer used to cover: guard the
low-level os.open (any mutating flag confines the target; a mutating dir_fd fails
closed) and io.open (which also carries pathlib.Path.open('w')), and add
os.mknod / lchmod / lchown / chflags and shutil.chown / copymode / copystat to
the wrapped set. Native-C writers (cv2.imwrite) and the realpath TOCTOU window
remain documented residuals that only OS-level isolation can close.
- Remove _resolve_path, _resolve_path_call, _resolve_join, _classify_path_string,
_is_pathlib_expr and the _FS_* / _PATHLIB_CTORS / _PATH_DEPTH_CAP constants, and
the write half of the filesystem visitor plus the FS_READ_STRICT knob.
- Reads are not confined by the backstop, so keep a small static sensitive-read
scanner (_is_sensitive_abs_path) that still blocks host-secret reads via a
sensitive absolute / ~ literal in any call arg (covers open, os.open, and library
loaders such as pandas.read_csv('/etc/passwd')) and .. / ~ traversal on the
dedicated open/read callees.
Net: about 300 fewer lines in tools.py and one fewer concept to audit; static
analysis now scopes to exec, shell, network and sensitive-reads while writes are
confined at runtime. Rework the filesystem tests around the new contract and add
os.open / io.open / Path.open / dir_fd escape cases to the backstop suite.
* Fix Windows installer torch index override
* Clear inherited uv index env vars for pinned installs in studio/setup.ps1 (#6898)
* Harden setup.ps1 index-var clearing to truly remove vars (#6898)
* Apply UV_DEFAULT_INDEX torch index fix to Linux/Mac install.sh (#6898)
* Neutralize all uv index env vars for pinned torch installs (#6898)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: add Vulkan llama.cpp support
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address gemini's feedback
* Studio: move the Vulkan VRAM probe into a standalone script
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Improve Vulkan probe error reporting
* Resolve llama-server symlink so Vulkan build is detected
* Drop unreachable Vulkan fallback in GPU free-memory dispatcher
* Skip the Intel GPU probe when NVIDIA or ROCm is present
* Reserve host RAM headroom for Vulkan integrated GPUs
* Add a `UNSLOTH_FORCE_VULKAN` environment variable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor GGML_VK_VISIBLE_DEVICES, reserve discrete Vulkan VRAM headroom, and clear Intel GPU on --cpu-fallback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Route Intel and forced-Vulkan hosts to the upstream Vulkan prebuilt, add arm64 Vulkan, keep Vulkan out of RAG auto-detect
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Clear the fork release pin when routing a Vulkan host to the upstream repo
* Gate auto-Vulkan routing on no physical NVIDIA so hidden CUDA devices aren't used
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Pin Vulkan launches with --device Vulkan<i> instead of the raw GGML_VK_VISIBLE_DEVICES index space
* Let user --device override the Vulkan pin, and gate direct Vulkan asset picks on no physical NVIDIA
* Update RAG auto-backend test mocks for the _resolve_auto binary and Vulkan probes
* Keep the add_dll_directory handle alive through the Vulkan probe DLL loads
* Revert RAG auto Vulkan guard, guard multi-backend Vulkan detection, and preserve forced Vulkan across updates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use getattr for RTLD_GLOBAL in the Vulkan probe CDLL mode
* Skip CUDA/ROCm APU and datacenter GPU tuning on Vulkan builds
On a Vulkan llama.cpp build gpu_indices are ggml compact ordinals, not
CUDA/ROCm physical ids, so _amd_apu_wants_unified_memory and
_apply_datacenter_env were reading the wrong device. On a mixed AMD APU
plus discrete GPU host that could raise a spurious system-RAM shortfall
and block a valid discrete-GPU load. Gate all three call sites on
not is_vulkan_backend; the Vulkan path already reserves iGPU host
headroom and the backend ignores GGML_CUDA_* anyway.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten Vulkan-guard comment in load_model
* Reduce comments in Vulkan support to be more succinct
* Resolve shell-wrapper llama-server entrypoint to the real lib dir
create_exec_entrypoint falls back to a #!/bin/sh wrapper at the install
root when it cannot symlink into build/bin. _find_llama_server_binary
returns that root entrypoint, but Path.resolve() does not follow a shell
wrapper, so _llama_lib_dir returned the install root and _is_vulkan_backend
missed libggml-vulkan.so -- silently skipping the Vulkan probe and --device
pin on an otherwise valid Vulkan install. Follow the wrapper's exec target
to build/bin. Regression test: test_shell_wrapper_entrypoint_resolves_to_real_lib_dir.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Studio /v1/messages: accept thinking and unknown content blocks
The Anthropic-compatible /v1/messages endpoint modeled a message's content as
Union[str, list[{text|image|tool_use|tool_result}]], so any other block type
made Pydantic reject the whole request with
`messages.N.content.str: Input should be a valid string`. Resuming a Claude
session commonly replays assistant turns that carry `thinking` (extended
thinking) blocks, and sometimes a null content for a tool-only turn, both of
which tripped this and returned a 400.
Accept them:
- Add a permissive AnthropicUnknownBlock fallback (any block whose type is not
one of the four known ones), so thinking/redacted_thinking/provider-specific/
future blocks validate. A validator keeps known types on their typed models,
so a malformed known block (e.g. a tool_use without id) still fails cleanly.
- Coerce a null message (and tool_result) content to "" so the converter's
`for block in content` stays safe.
The converter already drops block types it does not translate, so a thinking
block is not forwarded to the model.
* Studio /v1/messages: keep user content validation strict
Make the thinking/null leniency role-aware so it never silently drops real
user input. Assistant turns (replayed history) still accept unknown/thinking
blocks and coerce a null tool-only turn to empty. User turns keep the strict
boundary: a null user content is rejected, and a content block the converter
cannot translate is rejected instead of being dropped into an empty prompt.
Also remove an empty file committed by accident.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio /v1/messages: coalesce resumed user turns and tighten content checks
- The /v1/messages count and generation paths now coalesce the adjacent user
turns that dropping an empty or null assistant turn can leave behind, so a
strict GGUF chat template no longer 400s on non-alternating roles.
- A user content block with a non-string type (list / dict) is rejected as a
clean 400 instead of raising TypeError and escaping as a 500.
- The assistant null-to-empty coercion only applies to an explicit null; an
assistant turn that omits content entirely still fails required-field
validation instead of being silently coerced to an empty string.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio /v1/messages: tighten comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Tighten the eval/exec/compile dynamic policy so an executing sink (eval/exec/
runpy) applied to a payload that cannot be statically recovered is refused
unconditionally, not only when an RCE-core module happens to be imported in the
snippet. An un-analyzable executing payload can synthesize any shell, network,
or filesystem escape at runtime, so the prior in-scope-import heuristic left
exec(input()) and eval(user_var) allowed whenever no such import was present.
compile() of the same payload stays allowed since it does not run.
A payload that is fully recovered as a constant but is invalid Python for the
sink's mode (for example eval("data = 1") or eval("not python !!")) is now
allowed: its exact source is known and it raises SyntaxError at runtime, so it
is not an execution vector. Only genuinely opaque, non-recoverable payloads
(for example eval of a runtime-computed f-string) reach the block.
Remove the now-unused _RCE_CORE_MODULES set and _scope_imported_roots helper,
and move the opaque-f-string case in the tests to the blocked set.
* Stabilize floating monitor drag
* Restore floating monitor exit animation
* Harden Windows Studio smoke checks
* Keep API menu badge removed
* Apply no-build-tools env overrides in-script
The runner does not apply step-level env keys containing parentheses,
so ProgramFiles(x86) kept its real value and Find-VsBuildTools still
detected VS through vswhere. Set the overrides inside each pwsh step
instead; child processes inherit them. The resolver step moves to pwsh
because bash cannot export a variable named ProgramFiles(x86).
* Reset chat UI session without a second browser context
macOS runs Chromium with --single-process, where closing the last
context tears down the whole browser, so the shutdown re-login died
with TargetClosedError on new_page. Clear cookies and swap pages
inside the same context instead, opening the replacement page before
closing the old one.
* Keep the no-build-tools Path filtered across session refreshes
install.ps1's Refresh-SessionPath and setup.ps1's Refresh-Environment
rebuild the session Path from the Machine and User registry scopes, so
the process-level filter could be undone mid-install and re-expose
CMake. Filter those scopes in the Prepare step with normalized dir
matching and restore them in cleanup.
* Drop stale localStorage auth tokens before re-login
Auth tokens live in localStorage, not cookies, and the login guest
guard redirects on their mere presence. Remove them during the session
reset so the /login navigation is deterministic instead of relying on
the tolerated redirect bounce.
Add ln to the bash command denylist so a symlink escape cannot be created from
the terminal tool. In the sandboxed (non-bypass) _python_exec path, prepend a
one-line guard to the generated temp module that monkeypatches only MUTATING file
ops (builtins.open in write/append/x/+ modes, os remove/unlink/rmdir/removedirs/
rename/renames/replace/truncate/chmod/chown/mkdir/makedirs/symlink/link, shutil
rmtree/move/copy/copy2/copyfile/copytree, pathlib write_text/write_bytes/unlink/
rename/replace/mkdir/rmdir/chmod/symlink_to/hardlink_to/touch) to resolve the true
os.path.realpath of the target and raise PermissionError unless it lands inside
the injected session workdir. Reads are left unpatched. The guard runs in its own
namespace so helper names never leak into user globals, and it is skipped
entirely under disable_sandbox. This catches what the static gate cannot prove:
pre-existing symlink escapes and dynamic library-writer paths that funnel through
builtins.open. Benign in-workdir relative writes and library imports are
unaffected (importlib swallows out-of-workdir bytecode-cache write failures).
Add pragmatic aliasing so an aliased shell sink with a dangerous argument is
caught: a name stored exactly once and bound to a resolved os/subprocess sink
(s = os.system; s('rm -rf /')) and inline literal-container indexing
([os.system][0](...), (os.system,)[0](...), {'k': os.system}['k'](...)) both feed
the existing _find_blocked_commands argument check. Resolution is deliberately
low-false-positive: only unambiguous single assignments and inline literal
containers, never a flow-insensitive union, so s = os.system; s = print; s('hi')
is not aliased. The shell-sink set is lifted to module scope (_SHELL_SINK_FUNCS)
so the alias pre-pass and the visitor share one definition. Interprocedural and
flow-sensitive taint remain out of scope (deferred to a full fixpoint).
Add a filesystem_violations category backed by _resolve_path, a LOCAL / ESCAPE /
UNKNOWN classifier that constant-folds strings and understands os.path.join,
pathlib Path()/'/'/joinpath, and f-strings with real join plus absolute-reset
semantics. expanduser / expandvars / os.environ / getcwd / dynamic parts collapse
to UNKNOWN. A new _FilesystemPolicyVisitor inventories destructive and mutating
ops (open write/append/x/+, os remove/unlink/rmdir/rename/replace/truncate/chmod/
chown/mkdir/makedirs/mknod/symlink/link/chdir, shutil rmtree/move/copy*, pathlib
write_text/write_bytes/unlink/rename/replace/mkdir/rmdir/chmod/symlink_to/touch,
tempfile dir=, and a curated numpy/pandas/torch/joblib/PIL/matplotlib/cv2 writer
set) and applies prove-or-block: mutating LOCAL allows, UNKNOWN/ESCAPE blocks.
rename/move check src and dst; symlink/link check both target and link path;
chdir must be LOCAL; tempfile dir= must be LOCAL. Reads block only on a provable
escape (sensitive absolute path or ..'/~ traversal), with an FS_READ_STRICT knob
for prove-or-block reads. A callee-independent literal-sensitive-path scan blocks
loaders like pandas.read_csv('/etc/shadow'). Library writers block only on a
provable escape so in-memory buffers are not over-blocked; the Stage 5 runtime
backstop covers the dynamic residual.
Replace the blanket dynamic_exec block with a recursive payload analyzer gated by
UNSLOTH_STUDIO_SINK_ANALYZER (default on; =0 restores the legacy ban). For eval /
exec / compile (and single-assignment aliases like e = exec), constant-fold the
first argument; a recovered source string is bracket-depth pre-scanned, size and
recursion-depth bounded, then re-classified through the full analyzer. An inner
sink blocks and surfaces the inner reason; a clean inner payload allows; a bound
or budget breach fails closed. Non-foldable payloads follow a low-false-positive
dynamic policy: block when assembled from decode / fetch / runtime-assembly
primitives (including nested exec and large string repetition) or when an
RCE-core module is imported in scope, else allow.
Keep the gadget-dunder and dynamic-import blocks but constant-fold import names
so __import__('hugging'+'face_hub') resolves to a real module. Refine getattr /
setattr on a sensitive module so a benign constant attribute (getattr(os,
'getpid')) is allowed while a dynamic or dangerous constant attribute blocks. Add
pickle/marshal/dill.loads as unverifiable code-deserialization sinks. eval('2+2'),
compile('a+b','<s>','eval') and ast.literal_eval now pass; base64/hex/rot13/chr
and gadget-obfuscated escapes still block. Wire a filesystem_violations category
through is_safe, the info dict, and the reason assembly (populated in a later
stage). The three legacy tests that asserted the blanket ban are updated to the
new recurse-the-payload behavior.
Introduce _const_fold, a whitelist-only, bounded, side-effect-free partial
evaluator plus a single-assignment const-prop environment builder. It recomputes
pure transforms on literals only (concat, repeat, join, format, f-strings,
slice/reverse, chr/ord, base64/hex/rot13/zlib decode, pure builtins and string
methods) and never executes, imports, or reflects on user code. Depth, op, size,
and sequence caps guarantee it can only fail to recover a value, never crash or
hang. This is the foundation the later eval/exec unwrapping and filesystem path
resolver build on.
* unstructured block removal
* Enhance unstructured block handling
* Restrict block cleanup to upload UIDs
* cleanup for seed block uploads
* upload cleanup queue for unstructured blocks in recipe studio
* Fix unstructured upload cleanup edge cases
* Fix unstructured upload import ownership
* Fix-unstructured-import-path-ownership
* Guard failed-delete restore against stale block in unstructured drop zone
* Drain queued upload cleanups when autosave is skipped
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: render thinking blocks for safetensors inference with prefilled <think> templates
Reasoning templates like Qwen3.6 end the generation prompt with an open
<think> tag. skip_prompt streaming drops it, so the frontend never sees
the opening tag and shows reasoning as plain text. Detect the prefill
and re-emit it at the start of the stream on the transformers and MLX
paths. Also stop stripping think tags in _clean_generated_text when a
tokenizer marks them special.
* Studio: guard think re-emit for special close tags, yield prefill early
Address review feedback:
- Guard: skip re-emitting the open <think> when the tokenizer marks </think>
as a special token, since skip_special_tokens would strip the model's close
tag and leave an unclosed block that swallows the answer. Falls back to
plain text (pre-fix behaviour) for those tokenizers.
- Yield the prefilled <think> before the first token so the thinking block
renders during prompt prefill instead of after the first generated token.
- Drop the now-unnecessary _clean_generated_text think-tag exemption; the
guard handles the special-token case at the source.
No mainstream reasoning model (Qwen3.6, Qwen3, DeepSeek-R1, QwQ, GLM-4.6)
marks think tags special, so behaviour is unchanged for them.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lyxot <longyixing331@gmail.com>
* Studio: allow CPU-only DiffusionGemma by granting the diffusion runner the CPU device
* Studio: mark CPU-only DiffusionGemma as non-GPU-resident for training VRAM preflight
* Studio: keep the CPU DiffusionGemma change minimal (revert VRAM-flag tweak; Metal hosts still hold unified memory)
* Studio: keep CPU DiffusionGemma fallback fully CPU-masked so a masked GPU host does not re-expose GPU 0
worker.py imports has_blackwell_gpu from utils.wheel_utils, but _load_worker_module
stubs utils.wheel_utils with a fixed name tuple that omitted it, so loading the worker
raised ImportError (cannot import name 'has_blackwell_gpu') and Backend CI could not
collect test_mlx_training_worker_config.py. Add the name to the stub so it matches
worker.py's imports.
* fix: Remove moot has_blackwell_gpu() function
Fixes unslothai/unsloth#6961. This function skipped flash-attn on Blackwell GPUs because no prebuilt wheel existed;
Dao-AILab now ships one and url_exists() already gates resolution.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: use torchao 0.17.0 for Blackwell
Fixes#6961. Torchao 0.16.0's cpp extensions are built against CUDA 12, so on a CUDA-13
torch (cu130 / Blackwell) they fail to load with "libcudart.so.12: cannot
open shared object file". Select 0.17.0 there instead: its cpp targets torch
2.11, so it is skipped cleanly rather than crashing. CUDA-12 / ROCm / CPU
torch 2.10 keeps 0.16.0 and its working kernels.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Condense torchao version-selection comments (no behavior change)
* Support torch 2.11 in the Studio installer via the torch2.10 prebuilt wheels
Map torch 2.11 to the torch2.10 prebuilt wheels for flash-attn, causal-conv1d,
and mamba through wheel_utils.prebuilt_wheel_torch_mm, applied in direct_wheel_url
(filename) and flash_attn_wheel_url (version). Those torch2.10 CUDA wheels load and
pass each project's own test suite on torch 2.11 (verified on B200), so a torch 2.11
environment gets the prebuilt accelerators instead of skipping or building from source.
Raise _CUDA_TORCH_PKG_SPEC to <2.12.0 (torchvision <0.27.0, torchaudio <2.12.0) so
the CUDA torch repair path can install torch 2.11, where torchao 0.17's cpp kernels
load cleanly. Add tests for the mapping.
* Keep has_blackwell_gpu as a False stub for future arch gating
* Restore has_blackwell_gpu as a return-False probe kept for future arch gating
Keep the nvidia-smi compute_cap detection and its two call sites, but short-circuit
with return False at the top so flash-attn is no longer skipped on Blackwell (sm_100+
now has prebuilt wheels and url_exists gates resolution). Drop the early return to
re-enable arch-based detection later.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(studio/hub): apply repo_id length limit per segment, not whole string
is_valid_repo_id() applied the 96-char limit to the full "namespace/repo_name"
string, so a repo with a valid (<=96 char) name but a long combined id was
falsely rejected. Match huggingface_hub.validate_repo_id by checking the length
per segment instead. Fixes#6946.
* Fix long repo id state filenames
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: source CPU llama.cpp prebuilts from the unslothai fork
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reject unknown Linux CPU arches and keep ROCm-tooling hosts off the CPU prebuilt
* Studio: extend the resolve-prebuilt ROCm-tooling guard to Windows
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: let ROCm-SDK-only CPU hosts take the fork CPU prebuilt
* Studio: accept windows-arm64 prebuilt kind and refresh stale fork-routing comments
* Studio: correct stale fork-routing comments and --resolve-prebuilt help
* Refresh stale ggml-org routing comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: keep transformers off sys.modules until the training worker activates the sidecar
The training worker (core/training/worker.py:run_training_process) decides the per-worker
Xet env flip during preflight by importing utils/hf_xet_fallback.py, which eagerly imported
unsloth_zoo at module load. unsloth_zoo's __init__ imports transformers, so the default
transformers 4.57.x was cached in sys.modules before activate_transformers_for_subprocess
prepended the 5.x sidecar to sys.path. Since activation only edits sys.path, the already
cached module won, and 5.x models failed to load their tokenizer or config:
- Qwen3.5 / GLM-4.7 (tokenizer_class TokenizersBackend): "Tokenizer class TokenizersBackend
does not exist or is not currently imported."
- gemma-4: "... is not supported yet in transformers==4.57.6."
Fix: load the shared unsloth_zoo backend lazily (only when a heavy download helper is first
used, which is after activation). child_should_disable_xet and the DEFAULT_* constants are
defined locally so importing the shim stays light. The download wrappers, the DownloadStallError
class, start_watchdog and get_hf_download_state resolve the shared backend on first use, and the
degraded no-unsloth_zoo fallback is preserved.
Tests:
- test_hf_xet_fallback.py: existing suite kept green via the restored _shared_* seam; the
GPU-init retry test now triggers the lazy load explicitly; new guard asserts importing
child_should_disable_xet does not import transformers/unsloth_zoo.
- test_training_worker_import_discipline.py: new invariant test that the worker preflight
imports leave transformers unimported, so this class of regression cannot return silently.
Runs in studio-backend-ci (CPU only, no network/GPU/weights).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: CPU-only guard that activation switches transformers to the model's sidecar version
Adds test_worker_activates_correct_transformers.py: runs the real worker preflight
(from utils.hf_xet_fallback import child_should_disable_xet) plus the real tier
detection and activate_transformers_for_subprocess for a transformers-5.x model
(Qwen3.5, tier 530), then asserts the in-process transformers actually switched to
the 5.x sidecar. A stale pre-activation import leaves 4.57.x pinned and fails the
assertion, which is exactly the TokenizersBackend regression (#6951).
Self-contained CUDA spoof (mirrors tests/_zoo_aggressive_cuda_spoof.py) forces
unsloth_zoo down its full, transformers-importing init path on a GPU-less runner;
without it unsloth_zoo degrades and never preloads transformers, masking the bug.
A one-line stub sidecar stands in for the 5.x venv, so no GPU, network, weights, or
real sidecar are needed. Passes on this fix, fails on buggy main.
* Studio: load the repo's canonical CUDA spoof in the correct-version guard
Load tests/_zoo_aggressive_cuda_spoof.py (the committed spoof the consolidated CI
already relies on) as the single source of truth so the guard matches CI and stays
robust on a CPU-only torch wheel, where a partial hand-rolled spoof could miss a
torch.cuda call and let the unsloth_zoo import raise (masking the bug). Falls back to
a minimal inline spoof for a standalone studio checkout. Verified: passes on this fix,
fails on buggy main, and the fallback path passes when the spoof file is absent.
* Studio: declare the lazily-resolved xet names so ruff F822 stays green
DownloadStallError, start_watchdog and get_hf_download_state are provided via the
module __getattr__ (PEP 562), so ruff F822 flagged them as undefined names in __all__
and the Source-lint / pre-commit checks went red. Add annotation-only declarations
(no value bound, so __getattr__ still resolves them lazily to the shared unsloth_zoo
backend) to mark them defined for the linter while keeping F822 active for the rest
of __all__.
* Studio: tighten comments on the sidecar-activation fix and its tests
* Studio: mirror the new MLX-dispatch preflight import in the import-discipline guard
The worker preflight now also runs 'from core.training.training import
is_apple_silicon_training_platform, should_use_mlx_training_backend' before it
activates the transformers sidecar. Add that import (guarded) to the guard's
preflight snippet so the invariant test stays a faithful mirror: a future change
that makes core.training.training pull transformers/unsloth_zoo eagerly would then
be caught too. Verified clean on the current tree (no leak).
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* feat: detect installed coding agent CLIs in Studio settings
The API-keys panel only ever showed the "claude" flavor of the
`unsloth start` command, so anyone using Codex, OpenCode, OpenClaw,
Hermes, or Pi had to manually rewrite the copied command by hand.
Add a backend check that looks for each agent's CLI binary on PATH
(shutil.which, mirroring the pattern already used elsewhere in
studio/backend/utils) and expose it as GET /api/settings/coding-agents.
The API-keys panel now renders a picker for all six supported agents,
marks the ones it finds installed, and defaults to one of those instead
of always falling back to claude.
Includes unit tests for the detection helper.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address review feedback on coding-agent detection
Three fixes from PR review:
- detect_installed_coding_agents now treats a PATH lookup failure as
"not installed" instead of letting it bubble up and break the
settings endpoint; added a regression test for it.
- CodingAgentsResponse.agents is now typed as an immutable tuple
instead of a list built from one, matching CODING_AGENTS itself.
- Fixed a race in the API-keys panel: picking an agent while the
installed-CLI check is still in flight could get silently overwritten
once that check resolved. A ref now tracks whether the user has made
a manual choice, so the auto-detected default only applies before
that happens.
* Address Codex feedback: GGUF gating and remote-detection scope
- codex refuses to launch against a non-GGUF (transformers-backed) model
(unsloth_cli's _require_gguf_for_codex), so auto-defaulting to it produced
a copy-pasteable command that fails immediately whenever the loaded model
isn't GGUF. Add useActiveModelIsGguf() (looks up the active checkpoint in
the chat runtime store) and a correction effect that steers the auto-pick
away from codex unless the loaded model qualifies, without ever touching a
choice the user made by hand.
- Detection runs via shutil.which on the Studio backend host, which isn't
the same machine as the browser in a tunnel/remote session. Reword the
'installed'/'detected' copy to say so explicitly when the tunnel URL is
in use, instead of implying the check ran on the viewer's own device.
* Rework auto-default per review: loopback gating + inline GGUF check
Replaces the previous approach with the exact shape discussed on the PR:
- Export isLoopbackHost/normalizeHost from agent-command.ts. The detection
endpoint runs shutil.which on the Studio backend, which only describes the
browser's own machine when the base this panel targets resolves to
loopback. For a LAN or tunnel/remote base, gate the whole thing off --
don't mark anything as "detected" and don't let it drive the default --
instead of just relabeling the copy.
- Drop the separate GGUF-correction effect and useActiveModelIsGguf hook.
Read useChatRuntimeStore.getState().activeGgufVariant inline inside the
existing detection effect's .then() (so it doesn't need to sit in the
effect's deps), and pick the first detected agent that isn't codex unless
the loaded model is GGUF, leaving the existing default untouched when no
compatible agent is detected.
Verified both branches (loopback vs LAN/tunnel base, gguf vs non-gguf,
manual pick preserved, no-compatible-agent fallback) with a standalone
port of the .then() logic.
* Address latest Codex findings: stale detection, model swap, cache
- Clear detectedAgents (and skip the network call entirely) when the panel
leaves a loopback base, instead of leaving a previous loopback detection
result marked 'installed' for a command that now targets a LAN/tunnel/
remote host.
- Add a separate, network-free correction effect keyed on the live
activeGgufVariant: if codex was auto-picked while a GGUF model was loaded
and the user then switches to a transformers-backed model while this panel
stays mounted, steer away from codex instead of leaving a command that
unsloth_cli's _require_gguf_for_codex will now reject. Never touches a
manual pick.
- Drop coding-agents.ts's module-lifetime cache. Installed-CLI detection is
environment state, not a persisted setting, so a stale positive/negative
from before the user installed something (or reopened the tab) is worse
than one extra cheap local API call per mount; keep only the in-flight
de-dupe for concurrent callers.
Verified the correction-effect logic (gguf->non-gguf swap with/without a
fallback, still-gguf no-op, manual pick never overridden) with a standalone
port of the effect.
* Make the codex/GGUF auto-pick symmetric in both directions
The correction effect only steered away from codex when the model stopped
being GGUF; it never steered back toward codex if the model became GGUF
*after* a non-GGUF-gated fallback had already picked something else (e.g.
codex is the only detected CLI, a transformers model is loaded so the
selection correctly falls back to the claude default, then the user loads a
GGUF model while the panel stays mounted -- codex never gets reconsidered).
Consolidate into one effect that re-derives the preferred detected agent
from scratch whenever detectedAgents or activeGgufVariant changes, in either
direction, instead of only reacting to the codex-specific downgrade case.
The fetch effect now only populates detectedAgents/availableAgents; this
effect is the single source of truth for what gets auto-picked from that
list. Never overrides a manual choice.
Verified both transition directions plus the manual-pick-survives and
initial-detection cases with a standalone port of the derivation logic.
* Reset the auto-pick to the default when it stops being trustworthy
Two more real gaps from the latest Codex pass on d988f52:
- The unified derivation effect only handled the case where a *different*
detected agent could take over. If codex was the only detected agent and
auto-picked while a GGUF model was loaded, then the model stopped being
GGUF, 'preferred' came back undefined and the effect silently left the
selection on codex -- exactly the command unsloth_cli's
_require_gguf_for_codex now rejects. Fall back to DEFAULT_AGENT in that
case instead of leaving it untouched.
- Leaving a loopback base cleared detectedAgents (so the 'installed' badges
correctly disappear) but left whatever agent had been auto-picked from
that now-stale, server-side-only detection still selected. Reset to
DEFAULT_AGENT there too, unless the user picked by hand.
Introduces a shared DEFAULT_AGENT constant instead of repeating the "claude"
literal at each reset site. Verified all five cases (both new resets, both
manual-pick-survives variants, and the existing multi-detected-agent
fallback still preferring another compatible agent over resetting) with a
standalone port of the effects.
* Derive GGUF-ness from the actual loaded state, not just the variant string
activeGgufVariant only covers an HF-repo GGUF pick (a specific quant
variant string). A direct local .gguf file -- custom folder, LM
Studio, or drag-drop -- is just as much a GGUF the codex preflight
(unsloth_cli's _require_gguf_for_codex) would accept, but it never has
a "variant" to report, so it read as non-GGUF here even though
/api/inference/status correctly reports is_gguf: true for it. That
mismatch could leave a Codex-only install not auto-selected, or reset
an auto-picked Codex, for a model that actually supports it.
Combined activeGgufVariant with activeNativePathToken (covers the
drag-drop/picked-file case) and ggufContextLength (only ever populated
when the backend last reported is_gguf: true for the active model, see
applyActiveModelStatusToStore) so all three paths a model can be GGUF
through are covered, matching the same is_gguf-or-equivalent check
hasGgufSource already applies to a staged pick elsewhere in this
codebase.
* Clear stale native-path token on a non-GGUF status refresh
When a native (drag-dropped or picked) GGUF was loaded and the backend later
switches to a transformers model outside the UI load path, refresh() adopts the
new /api/inference/status via setCheckpoint and applyActiveModelStatusToStore.
Those reset activeGgufVariant and ggufContextLength but never clear
activeNativePathToken, so the isGguf OR stays true after the switch and a
Codex-only detection auto-selects unsloth start codex for a non-GGUF model its
preflight rejects.
Drop activeNativePathToken in applyActiveModelStatusToStore whenever the status
is non-GGUF. A real GGUF load reports is_gguf: true, so its token is preserved
(the load path owns it); only a non-GGUF status clears it.
* Add the AGPL-3.0 header to the new studio contract test
* Fix/adjust agent detection for PR #6909
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
The python tool's static safety analysis (_check_signal_escape_patterns) was
purely name-based with no attribute visitor, so obfuscated routes to the shell
/ network / file policies it already enforces slipped past: eval / exec /
compile, __import__ / importlib with a computed or dangerous module name,
getattr / setattr aimed at os / subprocess / sys / builtins, and dunder gadget
chains (().__class__.__bases__[0].__subclasses__()).
Add a dynamic_exec category covering those, surfaced through _check_code_safety
alongside the existing categories. Dynamic import stays allowed for a benign
literal module name (huggingface_hub, json, numpy) so real workflows and the HF
upload gate keep working; ordinary getattr(obj, "field") and __class__ access
stay benign. Bypass Permissions (disable_sandbox) still skips the check.
Tests: TestDynamicExecObfuscation in test_sandbox_tools.py with matching benign
cases, giving _check_signal_escape_patterns its first direct coverage.
* feat(cli): detect MLX distributed launch context
* feat(mlx): wire distributed inference backend
* feat(cli): broadcast MLX distributed chat turns
* fix(cli): wait indefinitely for distributed chat turns
* fix(cli): report MLX distributed load errors cleanly
* fix(mlx): route distributed vlm through loader
* fix(cli): detect inline MLX host JSON
* fix(studio): harden distributed object sharing
* fix(studio): select JACCL distributed backend
* fix(cli): abort distributed error paths
* Distinguish real stream errors from model text via GenStreamError in distributed CLI
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fail loud when MLX distributed init returns a singleton group
The worker only reaches this block when distributed was explicitly
requested. A singleton (size 1) group means the launch failed to form a
real group (MLX built without distributed support, or an invalid launch
env/hostfile); silently continuing leaves nonzero ranks looping forever
on share_distributed_object. Raise instead so the surrounding handler
returns a clear load error.
* Tighten MLX distributed inference comments
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat(studio): route CLI trainer to MLX backend
* fix(studio): harden MLX trainer routing
* fix(studio): harden MLX trainer adapter routing
* test(studio): assert MLX CLI activation order
* fix(studio): address MLX CLI review feedback
* feat(cli): support MLX in legacy script
* fix(cli): adapt MLX tokenizer for raw text
* fix(cli): omit unsupported MLX eval batch arg
* fix(cli): feed raw text to MLX trainer
* Fix CLI MLX routing and Python 3.9 annotations
Route the MLX backend through create_mlx_trainer_adapter so the torch-free
Apple Silicon path never imports trainer.py (torch/unsloth/trl). Replace
from __future__ import annotations with typing.Optional/Union so the CLI
annotations stay Python 3.9 compatible without the unused-import lint hit.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Strip return_tensors from MLX raw-text tokenizer proxy
On a torch-free MLX install, RawTextDataLoader calls the tokenizer with
return_tensors='pt'; the callable proxy forwarded that to the HF
tokenizer, which tried to build torch tensors and failed before
training. Drop return_tensors so the MLX path returns plain token ids.
* Tighten CLI MLX-backend comments
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: fix link, currency and indentation edge cases in LaTeX rendering
Follow-up to #6914. Three fixes to studio/frontend/src/lib/latex.ts:
- Skip reference-link definition URLs ([id]: url) during delimiter
conversion, so escaped parens in such URLs are not rewritten as math.
- Preserve the opener line's indentation when emitting a display $$ block,
so a \[...\] inside a list item stays part of the list.
- Stop a currency amount from pairing with a converted span's opening $,
which swallowed the price into math (for example $5 + x \(y\)).
* Exclude GFM footnote definitions from the reference-URL skip
A footnote definition like [^1]: \(x\) had its body treated as a link
destination, so leading math was left literal. Skip [^...] labels.
* Merge overlapping link destination regions
A reference-def token can nest inline-link spans (for example
[1]: http://h/[a](b)/foo\(x\)), so the combined spans could overlap and
isInRegion's binary search missed the outer one, rewriting the URL. Merge
overlapping spans before the search.
* Guard lineStart when the display opener is at index 0
Behavior is unchanged (lastIndexOf clamps a negative fromIndex to 0), but
the explicit guard avoids relying on that implicit clamp.
* Scope to indentation and currency fixes
Drop the reference-link URL protection added earlier. It guards a case
models effectively never emit (escaped parens in a reference-style URL),
and approximating CommonMark reference definitions with a regex needs
open-ended special-casing. Keep the two high-value fixes: preserve display
math indentation (including multi-line bodies) inside a list item, and stop
a currency amount from pairing with a converted span's opening dollar sign.
* Studio: show Hugging Face address on hover for Hub and online model rows
The model selector already shows an on-disk path tooltip on local rows,
but Hub and online rows showed only the bare repo id, and nothing at all
when there was no VRAM estimate. Add an optional hubUrl prop and a
hubRepoUrl helper that mirrors localPathTooltip, and surface
huggingface.co/<repo_id> on hover for the Discover, search, and
downloaded Hub rows. Local and VRAM tooltips are unchanged; the VRAM
tooltip now also appends the address line.
Closes#6382
* Studio: use a 700ms hover delay before the model-row tooltip
Give the model-row hover tooltip (the Hugging Face address, plus the VRAM
and local-path lines it shares) a 700ms open delay instead of showing it
instantly, so it does not flash while sweeping the mouse down the list.
* Fix/adjust GGUF tooltips for PR #6928
---------
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* fix: handle case-variant GGUF cache hits for unsloth start
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* gguf cache: keep split shards co-located and isolate cache tests properly
When a cached main shard was reused from an older snapshot, the extra shards
were resolved independently and could come from a different snapshot dir (or a
fresh download into the current ref), leaving llama.cpp unable to load a
multi-shard GGUF whose pieces are split across directories. Only reuse a cached
main shard when every sibling shard sits in the same snapshot; otherwise fetch
the whole set together so they stay co-located.
Also patch huggingface_hub.constants.HF_HUB_CACHE (not just the HF_HUB_CACHE env
var) in the two cache tests that seeded a temp cache: the snapshot lookup reads
the module constant, so the env-only override let the real cache leak in and
skip an asserted download.
* Do not let a companion-only cache snapshot shadow real GGUF variants
When listing GGUF variants from the local HF cache, a newer snapshot may
contain only a companion file (for example a vision projector fetched on
demand) while the actual quant files live in an older snapshot. The prior
scan returned the first snapshot whose vision flag was set, yielding an
empty variant list and hiding the real quants. Keep scanning older
snapshots for actual variants and carry the vision flag across snapshots.
Also record the disk-space fallback variant's size in expected_sizes so
the later cache-reuse probe can size-verify the fallback main shard
instead of only checking for its existence.
* Propagate cached repo casing to companions and preflight split co-location
Two fixes to the case-variant GGUF cache reuse:
- Resolve the requested repo id to its cached canonical casing once in
load_model, up front, and pass it to the main GGUF and its companions
(mmproj / MTP drafter). Previously only _download_gguf resolved the
casing internally, so a case-variant request loaded the main file from
the canonical cache dir while the companions kept the requested casing
and missed the cached vision projector / drafter offline. Extracted the
resolution into a shared _resolve_repo_id_casing helper.
- Apply the split-shard co-location check in the disk-space preflight. When
a split GGUF's shards are cached across different snapshots the whole set
is refetched later, so counting them as cached made the preflight read 0
bytes to download, skip the smaller-variant fallback, and then fail the
full download on a low-disk machine.
* Reuse a co-located split GGUF snapshot and fix split fallback size probe
- When reusing a cached split GGUF, scan snapshots for one that holds the
whole set co-located instead of taking the newest snapshot's first shard.
A newer snapshot with only the first shard no longer shadows an older
complete snapshot, so an already-cached split model is reused rather than
refetched (which would fail offline).
- The disk-space fallback records its size in expected_sizes only for a
single-file fallback. _find_smallest_fitting_variant returns the whole
variant size, so using it as the first shard's expected size rejected a
valid cached first shard of a split fallback and forced a re-download.
* Scan for a complete split snapshot in the preflight; require a loaded catalog hit
- The disk-space preflight now uses the same co-located snapshot scan as the
download path (_cached_colocated_split_main) instead of the newest-snapshot
probe, so a newer snapshot holding only the first shard no longer masks an
older complete one and trips the smaller-variant fallback for a fully cached
split model.
- _resolve_model only attaches to a /v1/models entry that is actually loaded
(loaded != False). /v1/models also lists cached-but-unloaded catalog entries,
and matching one by case skipped /api/inference/load and left the agent
pointed at a model that is not resident.
* Restrict cross-snapshot GGUF cache reuse to offline
Reusing a same-name blob from an older or case-variant snapshot bypasses the
Hub revision/etag check, so a repo that updates a GGUF in place could serve
stale weights online. Gate the cross-snapshot and case-variant reuse (both the
disk-space preflight accounting and the download path) on HF_HUB_OFFLINE.
Online, hf_hub_download fetches the current revision and resumes a partial
download, so the reuse is unnecessary there; offline it remains the resilience
fallback. Marked the two reuse regression tests as the offline scenarios they
represent and added an online test asserting a fresh fetch.
* Harden offline cache reuse and hub-id detection
Three follow-ups on the case-variant GGUF cache path:
- Honor every truthy HF_HUB_OFFLINE spelling (1/true/yes/on), not just "1", when
gating the cross-snapshot and case-variant cache reuse. With HF_HUB_OFFLINE=true
the Hub calls are already offline, so the reuse must trigger or the cached GGUF
fails to load; route both the preflight accounting and the download path through
the same offline parse the rest of the backend uses.
- Resolve mmproj/MTP companions from the actual cached snapshot when offline.
resolve_cached_repo_id_case can keep a partial lower-case spelling when any dir
exists under the requested casing, so an hf_hub_download on that casing misses the
canonical companion; scan every case-variant snapshot and return the cached path.
- Restrict the case-insensitive model-id match to syntactically valid hub ids
(a single namespace/name over the HF charset). A server-side relative path such
as models/Llama/Foo.gguf is no longer treated as a hub id, so it cannot
casefold-match a differently cased path on a case-sensitive filesystem. This is
host independent, unlike the local-existence probe which cannot see a server path.
* Only casefold-match model ids against a loopback Studio
A two-segment string like Models/Foo is indistinguishable from a hub id, and the
local Path.exists() probe in _is_hub_model_id cannot see a path that exists only
on a remote Studio host. So against a remote server, casefolding could attach to
a distinct server-side path (Models/Foo vs models/foo) on a case-sensitive
filesystem. Gate the case-insensitive match on is_loopback_url(base): only a
local Studio, where the existence probe is authoritative, casefolds. For a remote
Studio the match is exact and a case-mismatched request falls through to
/api/inference/load, whose already-loaded dedup resolves it correctly.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* Studio: heal DiffusionGemma tool calls into structured tool_calls
* Fall back to supports_tools for backends without the passthrough capability
* Route DiffusionGemma client tools through passthrough when enable_tools is on
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop orphaned strip_tool_call_markup import after syncing with main
* Tighten supports_tool_passthrough comment
* Re-run CI on current main
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Move New badge to System settings tab
Show the "New" badge on the System tab and drop it from Connections.
* Stabilize refresh revocation UI test
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Polish assistant message actions menu
Use the circle question mark (HelpCircleIcon) for the "See response
details" action instead of the file-database icon, and lowercase the
"Export as markdown" label.
* Align response details sheet icon
* Speed up Studio startup path
* Studio: recheck managed binary executability on preflight cache hit and ignore stale unauthenticated platform fetches
Preflight: a matching capability cache fingerprint no longer skips the
runnability check when the managed binary's executable bit was cleared
(size and mtime unchanged, since chmod bumps ctime not mtime). The cache
fast path now confirms the binary is still executable, otherwise it falls
back to the CLI help probe so preflight reports Stale and can repair,
instead of returning Ready and failing later at backend start. Adds a
regression test.
Frontend: now that first render is no longer gated on fetchDeviceType,
the initial unauthenticated health call can resolve after an
authenticated platform fetch. Guard the store so a late unauthenticated
or failed non-forced response cannot overwrite an already authoritative
device type, tunnel URL, or secure flag. Forced refreshes and the first
unauthenticated load are unaffected.
* Studio: use access(X_OK) for the preflight cache executability guard
A mode bitmask treats any execute bit as launchable, but the executable
bits can be set only for another owner or group, or be denied by an ACL,
so the current user could still hit PermissionDenied at launch and the
cached fast path would wrongly return Ready. access(X_OK) checks real
executability for the calling user, so an ownership or permission change
correctly falls back to the CLI help probe and the Stale repair path.
* Studio: ignore any stale non-forced platform fetch once authoritative
Extend the platform store guard so a non-forced health response never
overwrites an already authoritative result, not only unauthenticated
ones. With a saved token the post-render non-forced request can be
authenticated but older than a later forced refresh that already picked
up the tunnel URL and secure flag; if that earlier request resolves last
it would null those fields. Now any non-forced response is dropped once
the store holds a server-reported platform. Forced refreshes and the
first authoritative write are unaffected.
* Studio: run the managed CLI help probe before trusting the preflight cache
Restore running the managed CLI help probe before returning Ready from
the desktop capability cache, so a managed install whose venv interpreter
or a runtime dependency is broken (while path, size, mtime, and markers
are unchanged) is reported Stale for repair rather than proceeding to a
backend start that cannot spawn. The capability cache still skips the
heavier desktop-capabilities probe on a hit, so a warm cache runs one
probe instead of two. Removes the executable-access shortcut, which the
help probe now subsumes.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix: match qwen3-thinking chat template double-newline in response pattern
The Qwen3-thinking chat template generates `<think>\n\n` (double newline)
after the think tag, but `train_on_responses_only` was looking for
`<think>\n` (single newline).
`\n\n` is token 271 while `\n` is token 198 -- different tokens, so the
pattern match in `train_on_responses_only` fails, masking ALL tokens and
dropping 100% of training samples.
Update the response pattern from `<think>\n` to `<think>\n\n` to match
what the actual qwen3-thinking template generates.
Fixes#6919
* fix qwen3 thinking response marker
---------
Co-authored-by: Ayushman Paul <ayushman@HP>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* show chat by by last activity
* Update chat thread updated_at logic and enhance sidebar chat item handling
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: account for DeepSeek-V4 compute buffer in context auto-fit
DeepSeek-V4-Flash's lightning indexer plus compressed sparse attention reserve a
large context-scaling compute buffer that _compute_buffer_ctx_bytes did not model
(the KQ-mask and dequant-scratch rates both miss it, even with an f16 cache).
Measured on UD-Q4_K_XL at ub 512 it is about 65.5 GiB at 1M context, which the
mask estimate puts near 1.5 GiB, so the auto-fit kept the full 1M train context
and llama-server OOM'd allocating the ~70 GB buffer, then spilled to CPU (~4
tok/s). Add a deepseek4-gated flat plus per-token term so the fit caps the context
(about 256k on a B200) and the model stays fully on GPU.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: add assistant response details panel
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Hide model badge by default, show on hover/focus
Wrap MessageResponseModelBadge in a span with hidden/group-hover visibility classes to reduce visual clutter. The badge now only displays when hovering or focusing on the assistant message, improving the UI presentation. Updated corresponding tests to verify the new CSS classes.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
