Collapse the remaining multi-line comment blocks in the attention, cache, LoRA, prequant, precision and compile-cache modules, the sd.cpp arg builder and engine, the video routes, the Ideogram 4 assembly, the model picker, and the diffusion test suites. Comments only, no code or behaviour changes.
Collapse the remaining multi-line comment blocks in the video page, training routes and service, sd.cpp server and installer, memory and speed planners, and the shared request models. Comments only, no code or behaviour changes.
Collapse the multi-line comment blocks across the image, video, sd.cpp and diffusion-training code to one or two lines each, and drop comments that only restate the statement below them. Comments only, no code or behaviour changes.
Five items from the latest review; four were real.
An unload or arbiter eviction only cancels the generation holding
_generate_lock. A second request queued behind it holds no cancel event yet,
and Python locks are not FIFO, so it could take the lock the instant the
active denoise released it, still see a loaded pipeline, and run a whole new
denoise after the model was told to go away: the eviction then waits minutes
for it and an image lands after the eject. Unload and a superseding load now
raise a fence under _lock before they queue, and a generation that wins the
lock while one is pending refuses instead.
The cached-model scan judged pipeline completeness across every revision, so
a repo holding an older complete snapshot plus a newer companion-only one
read as complete while the snapshot from_pretrained actually opens has no
transformer. Both scans now look at the revision the loader will open.
Deleting a dataset image deleted its caption sidecar unconditionally, which
for cat.jpg alongside cat.png removed the caption the survivor still resolves
to. The sidecar now goes only with the last image of that stem, matching what
the thumbnail cleanup beside it already did.
Importing an example into a folder that holds no images but does hold files
fell back to promoting the staging dir one file at a time, so an interruption
left a partial dataset that the image_count check accepts as complete on
retry. Those files are folded into the staging dir instead and the promotion
stays a single atomic rename.
The MPS generator report does not apply: torch.Generator(device="mps") has
worked since PyTorch 2.0 (pytorch/pytorch#91348) and the studio installer
pins torch>=2.4.
/images/download-plan always asked the diffusers backend, while /images/load
picks the engine per host: a GGUF pick on a machine with no usable GPU routes
to native sd.cpp, which reads a single-file VAE plus text encoders and never
opens the base repo's sharded components.
Measured on unsloth/FLUX.2-klein-4B-GGUF (Q2_K): the plan staged 7.66 GB of
FLUX.2-klein-4B components the native load discards, and the 7.80 GB sd-cli
actually needs was then fetched inline by the loader, outside the download
manager's progress and its disk preflight. Z-Image-Turbo is the same shape.
The plan now asks whichever engine the load will select. predict_engine()
applies the selection policy without any side effect: it activates nothing
(staging a download must not unload the resident model) and only locates the
binary rather than installing it, but still counts an installable binary as
available, since that is what the load does on a fresh host. The native
backend gains a download_plan built from the same _asset_specs the loader
fetches, returning the same envelope, so the manager stages exactly the files
sd-cli opens.
Four fixes from the latest review pass.
The video gallery downloaded each clip into a blob before it could play, so
playback waited on the whole file (tens to hundreds of MB), seeking was
limited to what had arrived, and every viewed clip stayed pinned in the
webview. The file route already streams and serves ranges; it just could not
be a <video src> because it is bearer-gated. Mint a short-lived signed link
instead (its own HMAC secret, 12 hour TTL, separate from the image links) and
hand it to the element, which then fetches only the ranges it plays. That
removes the blob budget, its LRU and every revoke on this page.
The sd.cpp readiness probe accepted any process answering on the port, so a
foreign server that grabbed the port between the bind check and the spawn was
adopted as ours. Confirm the listener is our child before reporting ready,
and stay best-effort (psutil missing, an unknown owner, or any probe error
still passes) so the check can only reject a definitely foreign process.
Dataset import held its lock for the extract but not for the upload path, so
two concurrent uploads into the same folder interleaved; take the same lock
and return 409. And reject Windows device names (CON, NUL, COM1..9, LPT1..9,
with or without an extension) plus trailing periods in dataset names, which
are unopenable on Windows.
The active engine is module state, and several tests set it by plain assignment
because what _activate does to it is the thing under test, so monkeypatch could
not undo it. A leaked ENGINE_SD_CPP left get_active_diffusion_engine() handing
back the sd.cpp backend for the rest of the process, and every later route that
reads the active engine then saw an unloaded model: eight tests in
test_openai_images_generations_route.py returned 503 in a full-suite run while
passing on their own. The autouse fixture now snapshots and restores it.
The newer families (Z-Image, Krea 2, FLUX.2, LTX-2, HunyuanImage) exist only
from diffusers 0.39, and 0.39 cannot be installed on Python 3.9 at all --
diffusers dropped 3.9 in 0.38, so the requirement is conditional or the whole
extra becomes unresolvable. On such an environment the picker still offered
those rows, every pick failed deterministically, and the error's advice to run
pip install -U diffusers could not fix it without also upgrading Python.
The cached-repo picker now applies the same availability check
validate_load_request does, which is keyed on the pipeline class actually
present rather than on the Python version, so it is also right for an
intentionally pinned older diffusers on 3.10+. Fails open when diffusers cannot
be imported at all: that is a different problem and the load path reports it.
Staging does not set busy, so while a multi-gigabyte download runs the user can
still change precision, memory mode, speed or the baked LoRA selection. The
pending record held only the repo and artifact, and the completed download fired
a load that read the CURRENT state: the staged file set could then be missing
files that load needs (fetched inline, with no progress and no disk preflight)
or hold gigabytes it no longer uses.
One snapshot of every Advanced control is now taken when the plan is built, and
it travels with the pending record into the load, so the load that runs is the
one the download was planned for.
An fp8 text-encoder request loads a hosted PRE-CAST checkpoint, but the image
download plan never received text_encoder_quant, so the manager staged the base
repo's dense encoder (FLUX.2-dev's Mistral-24B is ~48 GB, Qwen-Image's
Qwen2.5-VL ~16.6 GB) and the load then pulled the pre-cast file inline, outside
the manager's progress and disk preflight. The plan now takes the field,
resolves the hosted artifact with the same resolver the injection uses, stages
that file, and drops only those components' dense weight shards. The load's own
prefetch takes the same treatment, since it paid the same cost. Only a
checkpoint that really resolves on the Hub earns the drop, so a gated or renamed
artifact still stages the dense encoder the load will fall back to.
The two trainers admitted each other with independent check-then-act guards:
the diffusion route checks the LLM backend several network-bound preflights
before it reserves, and the LLM route checks the diffusion service well before
it spawns, so two near-simultaneous starts could both pass and train on one GPU.
reserve() now re-tests the LLM backend under its own lock, and the LLM route
holds the diffusion service's gpu_load_admission across its spawn, so exactly
one of the two wins. Both halves fail open, so a chat-only install still
trains.
sd-server does not interrupt an in-flight job, so when it ignores a cancel the
grace branch abandoned the poll and reported cancellation while the native job
kept a core (or the GPU) busy to completion and held the server's job slot. The
comment said the caller stops the server, but only unload does that
immediately: a superseding load stops it after its multi-gigabyte download, and
a load that then fails never gets there. Stop it here, as the deadline branch
already does.
DELETE /api/models/delete-finetuned checked only the LLM trainer, so it could
rmtree the output directory a live diffusion LoRA run was about to write its
adapter into. Consult the diffusion training service too, like the dataset
mutation and model-load routes.
find_sd_*_binary only checks is_file(), so an interrupted extraction (or a
prebuilt for the wrong CPU) left a present-but-unrunnable binary the installer
never retried: every load probed it, fell back to diffusers, and native
inference stayed off until the directory was deleted by hand. Probe it and
reinstall, but only for a copy under the installer-owned root -- SD_CLI_PATH,
UNSLOTH_SD_CPP_PATH, an in-tree build and anything on PATH are the user's.
The 3.10 CI leg resolves PyAV 17, where av.container.OutputContainer is an
immutable C type, so the no-libopus export test died on "cannot set
'add_stream' attribute of immutable type" before it asserted anything. Inject
the refusal by wrapping the container av.open() returns instead; modules stay
patchable on every build. Removing the injection makes the test fail again, so
it still covers the branch it is named for.
The Xet shim's degraded-path tests drop utils.hf_xet_fallback from sys.modules
and import a throwaway copy. Restoring only the sys.modules entry left the
utils package attribute bound to the throwaway, and the two disagreed for the
rest of the process: a later monkeypatch of the dotted target patched one copy
while the code under test imported the other, so the patch did nothing and
test_fetch_te_prequant_only_reports_what_it_downloaded reached the real Hub and
got a 401. Restore both bindings.
ggml checks every node against the device's supports_op and calls GGML_ABORT
when one is not implemented, because a single-backend graph has nowhere else to
put it: there is no per-op CPU fallback. The whole sd-server dies with SIGABRT
mid-generation and the user gets "the native image renderer stopped
unexpectedly" with no way forward.
Seen on macos-14 arm64 with FLUX.2-klein-4B Q2_K through the cross-platform CI:
the text encoder is already pinned to CPU, and the abort moved into the denoise
loop instead.
ggml_metal_op_encode_impl: error: unsupported op 'MUL_MAT' -> ggml_abort
StableDiffusionGGML::sample -> sample_k_diffusion
A retry on the same backend would abort identically, so the load is restarted
once with --backend cpu (the only flag that changes which backend executes the
graph; --offload-to-cpu moves parameters, not compute) and the generation is
re-submitted. The same checkpoint then renders slower rather than not at all.
Strictly bounded: the signature must carry both the unsupported-op line and
ggml_abort, the device must not already be CPU, and it happens once per load,
so an OOM kill or a genuine crash still surfaces as itself.
tsc -b failed on the branch head, so npm run build produced no dist and every
platform job fell back to --api-only:
blob-url-cache.ts(29,15): TS1294: This syntax is not allowed when
'erasableSyntaxOnly' is enabled
dataset-labeling-grid.tsx / dataset-showcase.tsx: Argument of type
'{ url: string; bytes: number; }' is not assignable to parameter of type 'string'
The cache took its budget as a constructor parameter property, which the
project's tsconfig forbids, and fetchGalleryObjectUrl now returns the blob size
alongside the URL for that budget, which the two dataset thumbnail components
still consumed as a bare string. Declare the field explicitly and destructure
the URL at both call sites. tsc -b is clean and vite build emits dist again.
A cancelled video load could resume: begin_load cleared the shared cancel
event, and unload() drops _loading without waiting for the worker, so the
next load cleared the very object the cancelled worker was watching and its
multi-gigabyte pull ran on alongside the replacement until the token check
at the end. Each load now gets its own threading.Event, passed down through
_fetch_te_prequant and _predownload_base, so a cancelled worker stays
cancelled.
A cached repo with a model_index.json was advertised as text-to-image on the
trust rule alone, but validate_load_request also requires a detected image
family, so a trusted pipeline of an unsupported class produced a picker row
that deterministically 400s. The picker now applies both gates, mirroring the
video branch.
cond_cache_dir was accepted for sdxl and then ignored: only the DiT trainer
reads it, while the SDXL trainer builds a per-run in-memory latent cache, so
the promised cross-run reuse never happened. The route now refuses it with a
400 that names the families which do support it, checked against the resolved
family so an omitted model_family with an SDXL base is caught too.
Three fixes from review.
The 0.39-only pipeline classes (Flux2Klein, Z-Image, Krea 2, LTX-2,
HunyuanImage) were resolved by getattr deep in the load, so on the older
diffusers that packaging still allows on Python 3.9 -- diffusers dropped 3.9 in
0.38 and this project still supports it, so the 0.39 floor has to be conditional
or the extra becomes unresolvable -- an advertised model failed with a bare
AttributeError after its checkpoint had already been downloaded. Krea 2 already
guarded itself this way; assert_pipeline_class_available now runs the same check
for every image and video family from validation, before any fetch, and names
the version and the fix.
WebM export accumulated the whole VP9 output in a BytesIO and returned it as one
bytes object that the response held again. The request caps allow 2048x2048 for
1024 frames, so an export runs to hundreds of MB and concurrent clicks could
exhaust the process, while the MP4 route beside it already streamed from disk.
transcode_to_file encodes to a temp file and the route returns a FileResponse
with a background unlink, so nothing large is resident.
A conditioned generation's recipe carried only the txt2img fields, so the
gallery presented an inpaint or upscale result as a complete Create recipe and
restoring it replayed an unrelated text-to-image request. The images themselves
are still not persisted (user uploads with their own lifetime), but the workflow
and its scalars are, restore reapplies them, and the toast now names the inputs
that have to be supplied again instead of silently landing on Create.
Reported by Codex.
The gate still let the retry run on hosts unsloth does not support, which is
where it is most harmful: a 7 GB macOS runner lost the Studio server 26 s into a
load, and the Linux runner was torn down mid-generation. Neither MPS nor plain
CPU can complete the import, so the retry there pays the cost and fails anyway.
Require an accelerator unsloth actually supports (CUDA/ROCm via torch.cuda, or
XPU), with UNSLOTH_ALLOW_CPU as the documented override, and hoist the predicate
to module level so it is tested directly rather than through the import system.
On a CPU-only host the retry no longer fires at all; on CUDA the clean-environment
case it was added for still passes 29/29.
The retry added for the clean-environment patch failures is not free: importing
unsloth pulls torch in behind it, which costs ~940 MB of RSS measured in a
process that had neither, and on a host with no accelerator it fails anyway. A
cross-platform CI job that had generated fine at ~900 s later died 19 s in with
SIGTERM and every 'if: always()' step skipped, which is the runner being torn
down rather than a step failing.
Retry only when torch is already imported (true of the server and of anything
patching a real module, and the condition that stops the retry from being what
loads torch), unsloth is installed but not yet imported, and the first failure
was the ImportError the sentinel guard raises. The clean-environment case it was
added for still passes 29/29.
macos-14 loads FLUX.2-klein-4B Q2_K natively on mps and then dies on the first
generation with exit code -6:
ggml_metal_op_encode_impl: error: unsupported op 'RMS_NORM' -> ggml_abort
LLMEmbedder::encode_prompt -> LLMRunner::compute -> GGMLRunner::compute
ggml's Metal backend gates RMS_NORM on contiguous rows and aborts the process
when that does not hold, with no per-op CPU fallback, so any LLM text encoder
(Qwen3 for FLUX.2 and Z-Image, T5 for FLUX.1) takes sd-server down. The encoder
runs once per prompt while the DiT runs every step, so pinning only the encoder
keeps Metal for the part that matters. UNSLOTH_DIFFUSION_SD_CPP_METAL_TE_GPU=1
opts back in once ggml grows the kernel.
fp8 auto precision defaulted to precise accumulate on any non-consumer GPU,
which made fp8 2.05x slower than int8 on RTX 6000 Ada and slower than not
quantising at all. NVIDIA's professional whitepapers do publish equal FP8 rates
for both accumulate modes there, so the hardware premise held, but the cost is
in the cuBLAS path rather than the published rate. Default to fast accumulate:
measured on B200 the flag is a no-op (4096^3 _scaled_mm at 3023.8 vs 3041.8
TFLOP/s, bitwise-identical output, 1.213 s vs 1.230 s end to end), so it is a
large win where it bites and free where it does not. Precise accumulate stays
available via transformer_quant_fast_accum.
Z-Image's DiT is a Lumina2 derivative, so unsloth/Z-Image-GGUF and
unsloth/Z-Image-Turbo-GGUF both declare general.architecture = "lumina2" and the
whole line was tagged image-diffusion-unsupported and hidden from the Images "On
Device" list, though validate_load_request loads them. Resolve shared archs from
the repo/file name like bare "wan" already does, with a test asserting the picker
and the loader agree for every family.
The sage attention on-demand install ran an unpinned `pip install sageattention`,
but PyPI's newest wheel is 1.0.6 and diffusers refuses anything below 2.1.1: the
install always "succeeded", wrote an unusable version into the running venv, and
was rejected on the next line. Carry the dispatcher's floor so pip resolves
nothing instead.
The dense-quant disk gate sized the download from the bf16-RESIDENT table. The
fp32 families download twice that (Z-Image: 23,479 MiB against a 21,970 MiB
gate), leaving a window where the check passed and the download filled the disk;
Ideogram 4 ships fp8 and was overcharged the other way. Size the gate by
published bytes, verified against HF sibling metadata for all 12 families.
Patch installs went through unsloth_zoo, which refuses to import unless
UNSLOTH_IS_PRESENT is set, and that is set by unsloth itself. The server imports
unsloth at boot so it never showed there, but any other process ran silently
unpatched with every install returning False, which is 13 test failures on a
clean environment. Import unsloth and retry once, memoised per process.
Also: the GGUF+LoRA refusal pointed at the native engine without saying a GPU
host only selects it under UNSLOTH_DIFFUSION_ENGINE=sd_cpp, so the suggestion was
unreachable; the gallery recipe recorded loras from the generate request alone,
losing a load-time bake; load-progress claimed "40.07 GB downloaded" for a fully
cached load; and pickers.tsx imported three catalog-group helpers it never used.
Reported by oobabooga.
* Studio: add durable Deep Research workflows
* Studio: preserve research integration after upstream updates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep research worker compatible with Python 3.11
* Studio: address Deep Research lifecycle review
* Studio: preserve durable research recovery
* Studio: preserve research stream and context
* Studio: harden research sources and limits
* Studio: align research with shared chats
* Studio: guard durable research actions
* Studio: protect durable research turns
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: deepen durable research decisions
* Studio: protect research prompts and queries
* Studio: slim research stream deltas
* Studio: preserve research evidence and citations
* Studio: harden Deep Research (CI, prompt injection, query PII, config, citations)
- Fix backend CI: add research_runs_router to the synthetic routes stub in
test_desktop_auth so studio.backend.main imports under the health-check test.
- Escape prompt-delimiter tags in the decision and synthesis prompts so gathered
web/document content cannot close an <untrusted_...> wrapper and inject
instructions into the local planner/decision/synthesis model.
- Extend the public-query sanitizer to redact Luhn-valid payment cards, phone
numbers, non-global IPs, and labeled private identifiers before a query can
reach web search.
- Reject nested credential keys in inferenceRequest and ragScope, not just
top-level keys, when persisting a durable run config.
- Treat maxSources as one budget shared across web and document sources
(collection and resume paths) instead of per type, which allowed up to 2x the
configured cap.
- Preserve document citations whose filename contains a closing bracket by
tokenizing valid citations before stripping invalid ones.
- Persist Deep Research off when switching to an external model and when enabling
Web Fetch so a refresh cannot rehydrate a mutually-exclusive state.
- Add regression tests for the query, prompt, citation, and config hardening.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make the research claims table migration atomic
The owner-scoped to global claims migration ran its RENAME, CREATE, INSERT and DROP in autocommit, so an interruption after CREATE left the new table empty, orphaned the rows in the legacy table, and never re-triggered. Wrap the rebuild in an explicit transaction so a crash rolls back cleanly and the migration re-runs on the next boot.
* Studio: block message edits and regeneration during an active research run
After a reload a durable research run is followed by the research store rather than an assistant-ui run, so thread.isRunning is false while research is still active. Message edit, refresh and the edit composer previously gated only on isRunning, which let a normal generation start alongside the running research run. Gate them on the active thread's research state as well.
* Studio: keep the plan review mounted through approval
Keying PlanReview on planRevision remounted it mid-approve when updateResearchPlan bumped the revision, resetting the local pending flag and re-enabling Start research while the approve was still in flight, which allowed a duplicate approve. Key on runId only.
* Studio: drop the redundant deep-research persistence change
setCheckpoint already persists Deep Research off for external models at the top of the function, so the added saveBool was a duplicate, and clearing Deep Research from setWebFetchToolsEnabled guarded a state that is not reachable (Deep Research is local-model only while the Web Fetch pill is external-provider only). Revert both to the pre-hardening version.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden Deep Research citations, query privacy, and message protection
Address review findings in the Deep Research backend:
- Escape an unbalanced ")" in citation destinations so a source URL cannot
close the markdown link early and inject a second link, keeping balanced
parentheses literal.
- Match raw-URL citations on whole tokens so a URL sharing another URL's
prefix is no longer partially rewritten.
- Redact non-global IPv6 addresses in public search queries, matching the
existing IPv4 handling.
- Detect credential key names after normalizing case and separators so nested
openaiApiKey, accessToken, and clientSecret values cannot be persisted.
- Reject client edits to server-managed research prompts and reports at the
storage layer; only the internal writers pass allow_research_update.
- Scope research searches to the first allowed domains instead of dropping
site scoping for large allow lists.
- Persist the same fetch evidence bound used during live synthesis so a
resumed run is not shortened.
- Scope run completion so it only replaces this run's message parts.
Add regression tests for the above.
* Studio: fix Deep Research SSE framing, source counts, and favicon privacy
- Normalize the whole SSE buffer so a CRLF split across transport chunks
still frames events.
- Count web and document sources together in the activity header so a
RAG-only run is not shown as zero sources.
- Cap the plan editor at the run's configured maxSteps instead of a
hard-coded 30.
- Add an allowRemoteIcons opt-out to the sources components and disable
third-party favicon requests for research sources so visited domains are
not leaked.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address final Deep Research review findings
* Studio: fit Deep Research synthesis evidence to loaded context, add opt-in web grounding
Size the synthesis evidence budget to the loaded model context so the prompt is not
silently truncated on small contexts. When the evidence overflowed the window the report
degenerated (it echoed the evidence tail instead of writing); the budget now reserves tokens
for the prompt scaffolding and converts the remainder to chars, keeping the full cap when the
context is unknown.
Add opt-in web grounding for auto-read: read the top search results, ingest them into an
ephemeral RAG scope, hybrid-retrieve the passages most relevant to the question with the
existing knowledge-base retriever, and fold those chunks into the step evidence. The scope is
per call and deleted afterwards, so a user's knowledge base is never touched.
Off by default; enable with UNSLOTH_RESEARCH_AUTO_SCRAPE=1. Gated per run by
budgets["maxAutoScrape"], so runs created without it keep legacy snippet-only behavior, and
grounding is skipped when the loaded context is too small for the prompt.
Add tests for the adaptive evidence budget, scraped-text cleaning, the ephemeral web-RAG
retrieval and scope cleanup, and the auto-read evidence path.
* Studio: read Deep Research synthesis context from the inference orchestrator
Make the adaptive synthesis-evidence budget actually engage in the normal Studio
architecture. _loaded_context_length read core.inference.inference, the low-level backend that
lives in the model subprocess and stays unpopulated in the main web process where the research
supervisor runs, so it returned None and the budget silently fell back to the 32000 character
cap (leaving the report exposed to the truncation this was meant to fix). Read the inference
orchestrator instead, and the llama.cpp backend for GGUF, mirroring
routes.inference._monitor_context_length so the budget sizes to the context the API layer
serves. Verified on a running server: at a 12288 token load the probe now reports 12288 and the
budget adapts to 24576 characters instead of the 32000 fallback.
Also:
- Reserve context for the generated report as well as the prompt scaffolding (raise the reserve
to 4096 tokens) so evidence does not crowd out the output on a small window.
- Honor a numeric UNSLOTH_RESEARCH_AUTO_SCRAPE by passing the per-run maxAutoScrape as the page
cap to the scraper, instead of always reading the maximum.
- Guard the web-RAG connection acquisition so a get_connection failure returns the documented
empty result rather than propagating.
- Add a synthesis-context test that patches the real backend accessor (not the probe itself) so
the production wiring is exercised, plus a scrape page-cap test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden Deep Research query redaction and research autosave
- research_runs: extend the opaque-token allowlist so unlabeled Hugging
Face (hf_) and GitLab (glpat-) tokens are redacted before a query can
reach web search, without over-redacting public model or version ids.
- runtime-provider: for a server-managed research message, echo the
backend-stored metadata verbatim on autosave. Merging the client
metadata re-added client-only fields the server never persisted, so the
server-side guard saw a diff and rejected every streamed or snapshot
update with 409.
* Studio: keep composer tool pills always accessible after merge
The merge left the composer line marked always-expanded (data-expanded
"true") while the inner pill row was still gated behind composerExpanded,
so the Search and Code toggles disappeared once the permission mode was
"off" with no other toggle set. Render the primary tool pills
unconditionally, matching the always-expanded layout, and drop the now
unused composerExpanded and permissionMode locals. Fixes the Chat UI
Playwright check that asserts the Search and Code pills stay visible.
* Studio: update Deep Research composer contract to always-expanded layout
The always-expanded composer no longer routes effectiveDeepResearchEnabled
through a composerExpanded expression, so the frontend contract now checks
that it gates the Deep Research composer button render instead.
* Studio: do not bind a research run to a populated assistant reply
create_run adopted any assistant message under the user turn whose
researchRunId was unset, including a prior answer reused by a retry. On
completion _update_assistant drops the untagged text and source parts, so
that answer was silently overwritten. Only bind to an empty placeholder or
this run's own message, and reject a reply that already carries content.
* Studio: harden Deep Research synthesis budget, prompt shielding, and message protection
- research_runs: split the synthesis evidence budget evenly across notes so a
small context still keeps a slice of every research step instead of dropping
the later steps after the earliest ones fill the budget.
- research_runs: shield the research question and approved plan before placing
them in the decision and synthesis prompts, so a closing delimiter in either
cannot escape its block and inject sibling sections.
- research_runs: redact bearer authorization tokens from public search queries.
- studio_db: include attachments in the research-message change check and guard
direct attachment deletion, so server-managed research prompts and responses
cannot be mutated through the attachment paths.
- chat_history: map the protected-message conflict on attachment deletion to 409.
* Studio: strip invalid document citations that contain brackets
The invalid-citation regex stopped at the first closing bracket, so a
citation whose filename contained brackets left its tail (".pdf, p. 9]") in
the report. Match a balanced bracketed span so the whole invalid citation is
removed; valid citations stay protected by the earlier tokenization pass.
* Studio: free the RAG search slot when a lookup times out or is cancelled
The bounded knowledge-base search held the sole admission slot in a detached
worker until the search returned, so a lookup that outlived its timeout (a
stalled embedding or blocked vector call) kept the slot forever and starved
every later lookup, disabling knowledge-base retrieval globally. Release the
slot from the caller when it stops waiting, exactly once, so a detached worker
finishes without re-holding it.
* Studio: remove Websites label from research composer
* Studio: fix Deep Research review findings (RAG slot bound, orphaned workers, hardening)
- Bound the shared RAG search slot to one running worker. The search that is
doing the embedding/index/GPU work now owns the admission slot until it
finishes, instead of freeing it on caller timeout while the detached worker
keeps running, which let a second search enter and stack concurrent work
behind the capacity-of-one semaphore.
- Cancel active research runs before deleting their thread, project, or all
history. Deleting cascade-drops the run row, but the worker only notices at
its next lease check, so it could keep doing model/web/RAG work for a run
that no longer exists; signalling cancel first shortens that window.
- Shield the planner prompt's conversation and question with _shield_untrusted,
matching the decision and synthesis prompts, so untrusted text cannot forge
planner delimiters.
- Do not let a research key-revocation failure replace a successful
non-streaming completion; log it like the streaming path does.
- Include created_at in the protected research-message guard so a client cannot
reorder server-managed prompt/response messages while leaving the body intact.
- Reject non-scalar ragScope values; a nested container evades the
sensitive-key scan when its inner keys are unlisted and would reach retrieval
code that expects a scalar scope id.
Adds regression tests for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: remove research composer globe icon
* Studio: use Hugeicons telescope in research composer
* Studio: use Telescope02 icon in research composer
* Studio: standardize Deep Research telescope icons
* Studio: move Deep Research below web and code tools
* Studio: merge grounded page excerpts with search snippets instead of replacing
When auto-scrape grounding retrieved page-body chunks, it replaced the raw
search-result text for that step. If the retrieved chunk was a distractor or
dropped the key fact, the answer-bearing search snippet was lost and grounded
runs regressed below snippet-only accuracy on factual questions (e.g. returning
Apache 2.0 instead of the Qwen License, 403 instead of 404, or a single mirror
diameter instead of the sum).
Keep the search snippets and append the grounded excerpts as supplementary
evidence via a small _merge_scraped_evidence helper. Grounding stays opt-in and
off by default, so legacy runs are unchanged. Adds regression tests.
* Studio: fix stale website access assertion in Deep Research contract test
The dialog heading was renamed to a DialogTitle, so the contract test still
asserted a <span>Websites</span> that no longer exists and failed on every
branch built on this one. Assert the current heading instead.
* Add AGPL-3.0 SPDX header to the two new test files for PR #7219
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix citation loss, effort clamping and nested inferenceRequest for PR #7219
Three review findings, each with a regression test that fails without the fix.
Citation dropped for a bare URL in prose parentheses. _RAW_URL swallows the
closing paren and the old trim set only stripped ".,;:!?", so the catalog
lookup missed and the validator deleted the whole citation, leaving an
unbalanced "(" in the report. New _trim_url_tail follows GFM extended autolink
path validation: one right-to-left pass that interleaves punctuation and
unmatched-")" trimming. Both rules must run in the same loop, else
"https://x/y.)" keeps a stray dot. Balanced parens inside a URL
(Wikipedia-style) still survive. Output verified against cmark-gfm on nine
cases, including "https://x/foo)bar)" which must keep ")bar".
Research runs forwarded reasoningEffort unclamped. The local chat path clamps
to the loaded model's advertised levels; the research branch did not, and the
backend only validates enum membership, so llama.cpp dropped a level the model
lacks and the whole durable run silently fell back to the template default.
Now uses the same helper and the same levels as normal chat. Note this makes
"max" on a gpt-oss low|medium|high model resolve to "low" rather than falling
through to the template default, matching normal chat exactly; the divergence
between the two paths was the bug.
Nested inferenceRequest values were persisted. Every allowed field is a scalar
and the numeric/bool/enum ones reject a container while coercing, but "model"
is stringified with str(), which never raises, so {"auth": "sk-..."} slipped
past the sensitive-key scan ("auth" is not on the list) into the durable run
config as the model id. Mirrors the ragScope guard already in this PR.
Verified: 542 passed across the research/web/sandbox/chat-history backend
suites, frontend contract 10 passed, tsc --noEmit clean.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix report-stalling regex, uncataloged KB evidence and bracketed titles for PR #7219
Catastrophic backtracking in _DOCUMENT_CITATION. The alternation
(?:[^\[\]]+|\[[^\[\]]*\])* backtracks exponentially on an unterminated
"[Document:" with no later bare "]", which is ordinary malformed model output
and exactly what this sanitizer exists to handle. Runtime quadrupled every two
characters; one realistic 76-char line did not finish in 90s. It runs
synchronously inside async _research (the line below it uses asyncio.to_thread),
so a single bad report pins the event loop and stalls all of Studio, not just
the run. Replaced with the language-equivalent unrolled form, verified identical
on well-formed inputs including bracketed filenames, and linear: a 20,000-char
tail now takes 0.4ms. Not using possessive quantifiers or atomic groups, which
need Python 3.11 while this package declares >=3.9.
Uncataloged knowledge base evidence reached synthesis. When maxSources is
already full, every returned chunk hits the continue, so accepted_rag_sources
stays empty, the "if accepted_rag_sources" rebuild no-ops and rag_result keeps
the raw KB text. That text has no document_source_catalog entry, so the
validator strips any citation to it and synthesis is left building claims on
private KB chunks it cannot attribute. Cleared, gated on rag_sources so a
text-only KB reply is still passed through. The resume branch built rag_evidence
from all restored sources with the same hole, so it now mirrors the live loop.
Bracketed source titles destroyed their own citation. The catalog gave the model
the raw title while the citation writer stripped brackets. Search titles
routinely carry one ("[PDF] Annual Report"), and the prompt tells the model to
copy the title verbatim, producing a label the validator cannot match. Both
sides now share _citation_title.
Verified: 756 passed across the research/web/sandbox/chat-history/rag backend
suites. Each fix has a regression test that fails without it.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep a durable run alive when no model is loaded for PR #7219
A durable run is claimable within the supervisor's poll interval of startup
(main.py starts it in the lifespan, and claim_next takes any 'running' run whose
lease expired), Studio has no startup model auto-load, and the browser is not
connected yet. So restarting Studio mid-run reliably lands the next model call
on the local endpoint's HTTP 400 "No model loaded". That 400 is not retryable:
_completion retries only >= 500, and _stream_completion, which serves both
planning and synthesis, has no retry at all. The run is marked failed, and the
only recovery is retry, which sets report_text NULL and deletes every
research_plan_step, research_source and research_document_source. Up to an hour
of scraping and synthesis is lost on a plain restart, on the feature whose whole
point is surviving one.
Treat only that refusal as transient: wait up to the run's own
modelTimeoutSeconds for a model to come back, then re-send. Any other 400 still
fails immediately, so no behaviour changes on the happy path. The wait polls
_check_active, so cancellation and lease loss are still honoured, and the model
probe fails open, so a probe error can only send a request, never withhold one.
Each wait is bounded by the run timeout and the number of waits per call is
capped, so a model that keeps disappearing cannot re-send forever.
Deliberately not pinning or restoring the model, which the review comment also
suggested. Auto-switch is opt-in, default off, and GGUF-only, so restoring
would silently evict the model the user just loaded from a background worker,
and comparing the configured name to the loaded id is fragile across variant
suffixes and advertised aliases, so it would break working runs.
Verified: 853 passed across the research/web/sandbox/chat-history/rag/inference
backend suites. Eight of the nine new tests fail without the fix.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make website-policy search reach the whole allowlist and refill past blocks for PR #7219
Two review findings on the website access policy.
Domains past the site: filter cap were undiscoverable. The policy accepts up to
100 allowed domains and the prompt tells the model all of them are searchable,
but scope_search_query always scoped to allowed[:8], so a source in the ninth or
later domain could never be found, and an undiscovered URL cannot be fetched
either. The cap itself is right, search engines stop honouring long OR chains,
so the window now rotates by a hash of the query instead of being a fixed head.
Every allowed domain is reachable across a multi-step run, the same query is
always scoped the same way, and lists at or under the cap are unchanged.
A page of blocked results returned nothing. The policy filters after the search
while DDGS was asked for exactly max_results candidates, so if those happened to
be disallowed the tool reported no results even when valid ones ranked just
below, wasting a research step. Ask for a deeper pool when a policy is set and
stop at max_results allowed entries. No policy means no over-fetch, so ordinary
searches are unchanged.
Verified: 2324 passed across the research/web/sandbox/chat-history/rag/tool
backend suites. The 8 test_studio_api.py failures are pre-existing and need live
OpenAI/Anthropic credentials; they fail identically with these changes stashed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Only overfetch search results when the website policy restricts for PR #7219
Follow-up to 8be0b3699. Every run stores normalize_website_policy(...), which
returns {"allowedDomains": [], "blockedDomains": []} and is truthy even when
nothing is restricted, so the default unrestricted path asked DDGS for four
times as many results on every step. That is pure added latency and timeout
risk, since the filter passes everything and only max_results entries are
returned either way. Test the domain lists rather than the dict.
* Budget the whole research prompt against the loaded context for PR #7219
Only the synthesis evidence was budgeted, so the budget could not prevent the
overflow it existed to prevent.
Measured at head with a realistic prompt (40-source catalog, 12-step plan): the
untrimmable scaffolding is about 7,900 chars and the conversation context adds
up to 12,000 more. On a 4096-token context, which is the GGUF auto-fit floor and
the transformers default, the synthesis request came to about 1.7x the window.
Worse, _synthesis_evidence_budget computed usable_tokens = 0 at or below the
4,096-token reserve and then returned the 1,500-char floor anyway, so it added
evidence to a prompt that already did not fit. The decision prompt had no
context awareness at all: a fixed evidence[-60000:], roughly ten times a small
window, on every step rather than once at the end.
Overflow is not cosmetic here. It either silently truncates and degenerates the
report, as the comment above these constants already warned, or fails the run,
and a failed run is only recoverable via retry, which deletes every plan step,
source and document source and nulls the report.
Both paths now share _prompt_char_budget plus _trimmable_budget: each trimmable
section is measured against what the rest of the prompt leaves, and can reach 0
instead of a floor, because a shorter report beats a destroyed run. Evidence is
budgeted before the chat history, since the evidence is the report. Unknown
context still keeps the full cap.
At 4096 tokens the synthesis prompt now fits (0.6x). Below that it is still
over, since a 40-source catalog alone exceeds the window; that needs a smaller
maxSources, and the context box does accept values down to 128.
test_synthesis_evidence_budget_tracks_loaded_context asserted the old floor at
2048 tokens, which is the bug, so it now asserts 0 and that the rest of the
prompt counts against the same budget.
Verified: 2325 passed across the research/web/sandbox/chat-history/rag/tool
suites. The test_mcp_stdio_sessions failure is pre-existing and fails
identically with these changes stashed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope replayed research history to its own attempt for PR #7219
A retry deletes the previous attempt's research_plan_steps, research_sources
and research_document_sources rows but keeps its events, and the SSE route
attaches one live run snapshot to every event it emits, replayed history
included. The step.completed payload carries only position, title, action,
input and sourceCount, so that snapshot is the sole source of the excerpt and
evidence.
On any refresh after a retry, a replayed attempt-0 step was therefore matched
against attempt-1's step row by position alone, and start_position resets to 0
after the delete, so the positions line up exactly. The preserved attempt-0
activity then showed attempt-1's excerpt and evidence, or lost them entirely
when attempt 1 had not yet reached that position, under a banner that says
previous activity is preserved. The run.started resumed branch read the same
cross-attempt snapshot and spliced those activities out.
Both are gated on the event's attempt matching the snapshot's retryCount, which
is the same attempt scoping get_reasoning_text already applies server-side. The
excerpt and evidence fall back to what the activity already holds, so a mismatch
is non-destructive rather than blanking it.
Verified: frontend contract 11 passed, tsc -b exit 0, and the new test fails
without the store change.
* Retry pre-stream failures in the research stream for PR #7219
_stream_completion serves planning, every decision step and synthesis, and it
had no transport retry: a connection error or a 5xx raised before any response
byte failed the durable run, and retry then deletes every gathered source,
document source and plan step. _completion already treats the identical
failures on the identical endpoint as retryable, so the two paths disagreed.
This is partly a hole my own 689b06535 opened. After the no-model 400 the body
is read, the connection returns to the pool, and _wait_for_local_model then
sleeps for up to modelTimeoutSeconds before re-sending on the same client.
Uvicorn's keep-alive is 5s, so that pooled connection is essentially always
server-closed by then, and losing the has_expired race raises
RemoteProtocolError, killing the run the wait existed to save. Also reachable
via a read timeout waiting for headers under prompt-eval load.
Retrying is safe only because nothing has been consumed at that point, and that
is structural rather than a convention: with stream=True httpx returns on the
response headers without calling aread(), and raise_for_status() reads no body,
both verified against the installed 0.28.1. The handler is scoped to the inner
try that ends at break, and _iter_stream_lines sits outside the loop with no
path back to send, so a re-send cannot duplicate report text.
Bounded and mirrors _completion: same >= 500 predicate, same 3 attempts, same
2**attempt backoff, lease and cancellation re-checked before re-sending. The
transport counter and the model-wait counter are independent, so they cannot
multiply. The response is closed before every re-send, as manual stream mode
requires.
Note HTTPStatusError is not a TransportError in httpx, so both are caught
explicitly.
Verified: 2330 passed. Five of the new tests fail without the fix; the three
that pass either way are the invariants that must not change (fail fast on a
real 400, never retry once the report has streamed, existing model-wait path).
* Bound the planning prompt to the loaded context for PR #7219
Completes dc16598a4, which budgeted the decision and synthesis prompts but left
planning unbounded. The question reaches the planner verbatim (a pasted document
arrives here as-is) and the history is capped only at the fixed 12,000 chars,
so on a small context planning could overflow before any plan was persisted,
failing the run without doing any research at all.
Same helpers as the other two paths. The question is budgeted before the
history, since the question is the request.
A test now asserts all three prompt paths hold their own context budget, so a
fourth path cannot be added later without one.
Verified: 2331 passed; the new test fails without the change.
* Keep prompt inputs non-empty and fit the source catalog for PR #7219
Two follow-ups to the prompt budgeting, the first a regression I introduced in
dc16598a4.
The output reserve was a flat 4096 tokens, so on any context at or below that,
including the documented 4096-token GGUF floor, the whole prompt budget came out
as 0. Every trimmable section then sliced to nothing: planning_question became
the empty string, so the planner never saw the request at all, and synthesis
dropped all its evidence. Removing the old floor outright went too far; an empty
prompt is worse than the overflow it was avoiding. The reserve is now capped at
half the window, and the question and the evidence each keep a floor, since one
carries the request and the other carries the answer. A truncated completion is
recoverable, a confidently empty report is not.
The source catalog was the one section still inserted whole. It holds up to
maxSources entries with snippets persisted at up to 4000 chars each, so on a
smaller context it alone could exceed the budget while the code responded only
by zeroing the evidence and history. It is now fitted first, dropping whole
entries from the tail rather than slicing mid-entry, because a half-truncated
URL is worse than an absent one: the validator would strip it and the claim
would be left uncited.
Verified: 2333 passed. All three new tests fail without the change; the question
now keeps 1072 chars at a 2048-token context and 4144 at 4096, where both were
previously 0.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten Deep Research comments for PR #7219
Post-convergence comment pass over the 40 source files in the PR diff, limited
to lines the PR itself adds so untouched upstream code in the same files is left
alone. 15 files, 110 insertions, 141 deletions.
The reduction is deliberately small. Almost every comment here records why
something non-obvious is done, a measured result, a spec rule, or the exact bug
it prevents, and those are worth more than the lines they cost, so nearly every
edit is a same-meaning compression rather than a deletion. Kept in full: the GFM
autolink citation for the URL trim, the catastrophic-backtracking note on
_DOCUMENT_CITATION, the prompt-budget notes recording that a reserve at or above
the context leaves nothing, the two measured site: filter findings, and the
remount note on the activity panel key.
Verified comment-only three ways: comment_tools.py reports 15/15 code-unchanged,
and an independent ast.dump comparison with docstrings stripped shows zero of the
12 Python files differing. 421 backend tests and the 11 frontend contract tests
pass, and the phrase the contract test asserts on is still present on one line.
* Harden Deep Research model streams
* Fit Deep Research decision prompts
* Preserve Deep Research follow-up context
* Redact composite credentials from research queries
* Scale Deep Research UI typography
* Address Deep Research refinement review
* Harden Deep Research refinement edge cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* tests: read checked-in files as UTF-8 instead of the platform default
Path.read_text() with no encoding uses locale.getpreferredencoding(), which
is UTF-8 on the Linux runners and cp1252 on a stock Windows install. Nine
module-level reads of checked-in source files were relying on that default.
studio/backend/routes/inference.py carries the DeepSeek tool-call token
regexes, so it holds U+FF5C and U+2581. Under cp1252 that read raised
UnicodeDecodeError on byte 0x81 at position 97806, and because the reads run
at import time it took test_cancel_atomicity.py and test_cancel_id_wiring.py
out at collection, not as failures. Green on CI, permanently broken for a
Windows contributor running the suite locally.
Adds a guard: at module scope there is no tmp_path fixture, so a bare
read_text()/write_text()/open() there is always touching a checked-in file.
That makes the rule mechanical enough to enforce with no allowlist, while
staying quiet about temp-dir I/O inside test bodies where the platform
default is harmless.
The repo already spells this correctly in 464 other places; this only stops
the stragglers coming back.
* tests: cover import-time helper reads and keep the guard py3.9-safe
Follows up on the Codex review:
- add `from __future__ import annotations`, since `str | None` in
`_offender` is evaluated at import on Python 3.9 and pyproject declares
requires-python ">=3.9,<3.15".
- widen the guard from module scope to import time. Class bodies and the
bodies of module-level helpers called from an executing statement run
during collection too, so `CODE = _extract_mixed_precision_code()` was
the same hazard as an inline read. `if __name__ == "__main__":` blocks
are skipped: pytest never executes them.
- scan studio/backend/tests/ as well as tests/. Both trees are collected
on Windows by separate CI jobs, and the offender that started this,
test_tool_xml_strip.py reading routes/inference.py, lives there.
Widening it surfaced seven more import-time reads of checked-in sources;
all now name utf-8.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden the import-time encoding guard for PR #7438
Close the detector gaps raised in review, all of which I reproduced against
the actual AST before changing anything.
False negatives (the guard let a real hazard through):
- _is_main_guard ignored the comparison operator, so if __name__ != "__main__"
counted as script-only even though its body runs at import.
- The else arm of a main guard was discarded with the rest of the If node.
- Decorators and argument defaults on a module-level def were skipped with the
body, though both are evaluated when the def executes.
- Path.open() in text mode was invisible; only builtin open() was matched.
- encoding = None and encoding = "locale" both re-select the platform default,
but the keyword merely being present counted as pinned.
False positives (the guard would have blocked a compliant contributor):
- A non-literal mode fell through to the "r" default, so open(p, mode) was
flagged even when mode is "rb", where adding encoding= is a ValueError and
there is no edit that satisfies the rule.
- Same for open(*args) and a **kwargs splat, which hide the mode and can hide
an encoding.
- Lambda bodies and comprehension elements were walked even though neither runs
at definition.
Verified: still reports the same 22 offenders on unpatched main, green on this
branch and on the tree merged with latest main (557 files), and an adversarial
corpus of 33 cases now scores zero false positives and zero false negatives.
Also corrected two docstring claims: neither collecting job runs on Windows,
and the read is governed by locale.getencoding().
* Walk eager comprehensions and treat io.open as the builtin
Two regressions from the previous commit, both reproduced against the AST
before changing anything.
Lumping list, set and dict comprehensions in with generator expressions was
wrong. Only a genexp is lazy; the other three run their element expression,
their filters and their nested iterators immediately, so
CONTENTS = [p.read_text() for p in PATHS] at module scope is an import-time
read the guard was silently missing. Comprehensions are now walked in full and
only the genexp keeps the outermost-iterable-only treatment.
io was also in the not-a-path-opener list, but io.open is the builtin, with the
same mode position and the same platform default. io.open(CHECKED_IN_FILE) is
exactly the hazard this guard exists for, so it is matched now, with binary
modes and a pinned encoding still exempt. tarfile.open and fitz.open stay
exempt since neither has an encoding to name.
Verified: 13 targeted cases covering all five eager comprehension forms and
io.open in text, binary and pinned shapes all classify correctly; still 22
offenders on unpatched main; green on this branch and on the tree merged with
latest main.
* Close three more walker gaps in the import-time guard
All three reproduced against the AST first.
A generator expression handed straight to a call is consumed there, so
DATA = "".join(p.read_text() for p in paths) runs its element at import. Only
an unconsumed genexp bound to a name stays lazy, so the walker now follows the
consumed ones in full and keeps the outermost-iterable-only treatment for the
rest.
if "__main__" == __name__ is an equivalent and accepted spelling of the main
guard, but requiring __name__ on the left meant its body was treated as
import-time code. That is a false positive on a block pytest never runs, so
both operand orders are recognised now.
The helper table was built from module-level defs only, so a def in a class
body invoked while the class is constructed was never followed, contradicting
the walker's stated coverage of class bodies. Helpers are now collected from
the module body and from class bodies at any nesting.
Verified: 15 targeted cases including all three fixes and the earlier ones
still classify correctly; still 22 offenders on unpatched main; green on this
branch and on the tree merged with latest main.
* Handle positional read_text encodings, lazy generators and nested helpers
* Guard reads reached from test bodies, unbound Path calls and __file__ paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Follow derived paths, skip lazy generator helpers, cover compressed openers
* Guard the CLI tests, helper parameters and unbound Path arguments
* Discover test roots and follow literal, in-place and tuple-derived paths
* Identify module openers by import, unwrap starred paths, pin subprocess snippets
* Resolve import origins, seed helper locals, follow named generators and parametrize
* Scope imports lexically, list tracked test files, bind unpacked names
* Resolve aliased openers, keyword-only params, destructured targets, next()
* Pin the encoding on subprocess snippets, workflow lint and CLI output for PR #7438
* Harden the CLI encoding guard against detached streams for PR #7438
* Tighten the encoding guard's path and scope analysis for PR #7438
* Resolve path provenance more precisely and keep POSIX stream encodings for PR #7438
* Resolve qualified path classes and scope conditional imports for PR #7438
* Scope CLI stream setup to the entry point and align two encoding pairs for PR #7438
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* fix(studio): report Vulkan GPUs in system UI
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): separate Vulkan inference GPU reporting
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): keep retrying Vulkan probe refreshes
* fix(studio): preserve known zero GPU budgets
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(studio): reject Vulkan diffusion gpu_ids before Phase 1 teardown
Classify local GGUF paths (and cached HF downloads when available) for
diffusion before _kill_process() so unsupported gpu_ids requests return
400 without tearing down the active model. Fixes#7205.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): always pre-download HF GGUF before Vulkan diffusion preflight
Reverts the cached-path shortcut so partial split caches still run
_download_gguf before Phase 1 teardown. Header-only classification from
resolve_local_gguf_path() does not prove the variant is complete.
* Fix inaccurate shared-constant comment and cover the local pre-teardown branch for PR #7415
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add a regression test for the pre-teardown GGUF download for PR #7415
* Tighten the Vulkan diffusion preflight comments for PR #7415
* Trim the Vulkan diffusion preflight comments for PR #7415
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
* Bypass fast_generate for flash_attention_2 models (frozen KV / gibberish)
unsloth_base_fast_generate forces cache_implementation="static", which
pre-allocates the full prompt+max_new_tokens KV buffer. With SDPA the
not-yet-filled slots are masked out; flash_attention_2 does not receive such
a mask, so decoding attends over uninitialized cache memory and produces
incoherent output (observed: coherent prompt echo followed by gibberish
rollouts on Phi-4-mini-instruct during TRL GRPO training; the KV length
appears frozen at the pre-allocated size). Note that on transformers >=
4.56 UNSLOTH_DISABLE_STATIC_GENERATION=1 still selects the static cache, so
the env-var escape hatch does not help either.
Fall back to the wrapped model's original generate when the config reports
_attn_implementation == "flash_attention_2" - plain HF generate is correct
with FA2 (validated: prefill q=13/kv=13, cache grows 14, 15, ..., coherent
output; equivalent to UNSLOTH_DISABLE_FAST_GENERATION=1 but scoped to FA2).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix FA2 vision generation fallback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Detect FA2 in VLM llm configs
* Fix default FlashAttention config detection
* Honor language attention overrides
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle nested FA2 configs and cache cleanup
* Pin a dynamic cache on the FlashAttention fallback for PR #7429
* Cover the explicit cache kwarg and caller caches in the FA2 fallback for PR #7429
* Tighten the FlashAttention fallback comments for PR #7429
---------
Co-authored-by: Piotr Wąsiewicz <piotrwasiewicz72@mail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
DiffusionLoraConfig has carried cond_cache_dir for a while and the DiT trainer
acts on it, but DiffusionTrainingStartRequest omitted the field, so Pydantic
dropped it silently and every API-driven run fell back to the in-memory cache
that is rebuilt from scratch each time. The warm path skips loading the VAE and
the multi-GB text encoders on a rerun whose images, captions and resolution are
unchanged, so this was a real capability that could not be reached.
Contained like output_dir rather than left to the trainer subprocess's cwd,
since it is another directory the trainer writes to. Blank or omitted still
means the in-memory cache, so it must not resolve to the outputs root.
- The image and video load guards read is_active() and only then selected an
engine, acquired the arbiter and registered the load. A /train/diffusion/start
reserving inside that window freed residents the load had not registered yet,
so the trainer came up beside a brand-new pipeline. The service already had
exactly the right pattern for this in dataset_mutation, so gpu_load_admission
mirrors it: reserve() refuses while an admission is open, an admission refuses
once a start is reserved, both decided under the one lock. The span is only the
registration, since begin_load returns as soon as the load is registered and
_free_gpu_for_diffusion_training preempts an in-flight load from that point.
Chat is deliberately not covered: its load spans an eviction plus a multi-minute
GGUF load, and it admits models that fit beside training by design, which is a
different contract from the diffusion pipeline's all-or-nothing one.
- Hugging Face gives the LTX-2 family the image-to-video pipeline_tag (both
Lightricks/LTX-2 and unsloth/LTX-2.3-GGUF report it), so a text-to-video-only
filter dropped the flagship audio family out of Video Hub search while the rest
of the app routed it to Video.
- Task-scoped quant fit sized picks against the LARGEST visible device while
resolve_diffusion_device_target returns a bare "cuda" and torch places on the
current one. On a heterogeneous host that recommended a checkpoint sized for the
bigger card and then loaded it onto the smaller one. Fit now uses the device the
load actually lands on; identical on a homogeneous host.
Four review findings, all reproduced first:
- The gallery object-URL caches were unbounded. A clip runs from a few MB to a
few hundred, both pages stay mounted after their first visit, and entries were
only dropped on delete, so scrolling pinned everything for the session. Both
pages now share a byte-budgeted LRU (512 MB video / 192 MB images) keyed off
the visibility signal the near-viewport fetching already provides. On-screen
media, the selected clip or image, and the item just fetched are never
evicted, so eviction is invisible and a single item larger than the whole
budget cannot evict itself into a refetch loop.
- The image, video and chat load guards ran two independent training probes but
returned early when the FIRST one raised, so an unreadable LLM backend
disabled the diffusion interlock and a load could proceed straight into an
active diffusion trainer on the same GPU. The probes are independent now.
- An engine switch swallowed a failed teardown and published the new engine
anyway, which is exactly the leak the unload exists to prevent: the arbiter's
evictor, /images/unload and the next load all resolve through
get_active_diffusion_engine(), so the still-resident pipeline (or a live
sd-server) became unreachable and the next load allocated on top of it. The
switch now fails and leaves the old engine published, so it stays reclaimable.
- The native generation timeout was 30 minutes while the Images page waits up to
6 hours (SETTLE_MAX_MS), so slow-but-progressing CPU jobs died deterministically
at the deadline. Measured on GPU-less runners, a 512x512 4-step Q2_K generation
took 900 s on Linux and 1465 s on Windows, so larger images or step counts clear
half an hour easily. The ceiling now matches the page's window and applies to
the whole request: chunks of a split batch share one deadline instead of each
getting a full budget. Cancellation is unchanged.
Declined: gating the huggingfacenotorch extra off Python 3.9 over the
conditional diffusers marker. The marker is deliberate and its comment says why:
diffusers dropped 3.9 in 0.38, so pinning >=0.39 outright leaves pip no candidate
and the whole extra unresolvable there. The pipelines it names live in
studio/backend, which cannot install on 3.9 anyway (studio.txt pins
matplotlib==3.10.9 and fastmcp>=3.0.2, both requires_python >=3.10), and the
extra is the general core one, so the alternative drops 3.9 for library users who
never touch Studio.
models/training.py annotates with typing constructs throughout (105 Optional[...],
no Union), and flow_shift was the one place using PEP 604. Union[] reads like the
rest of the file, and it also drops the runtime evaluation that would raise on
Python 3.9.
Two halves of the same gap, found while measuring the LTX-2.3 download plan:
- The video download plan and the scoped pre-download never saw
text_encoder_quant. An fp8 request loads a hosted pre-cast encoder, so
asking for one still staged and downloaded the base repo's dense Gemma3
(48.79 GB of Lightricks/LTX-2 on the 2.3 distilled pick) that the pipeline
then never opened. The plan now drops those shards and stages the pre-cast
checkpoint instead; their configs stay, since the pre-cast loader still
meta-inits the encoder from the base repo's component config.
- The LTX-2.3 assembly builds every component itself, so pipe_kwargs (which
carries the pre-cast encoder for from_pretrained) never reached it and an
fp8 request silently loaded the dense encoder anyway. It is passed across
explicitly now.
The dense skip is earned, not assumed: only a pre-cast checkpoint that
resolves on the Hub lets the plan drop the dense shards, and only one already
fetched to disk lets the pull drop them, so an unpublished or gated artifact
leaves both exactly as they were. If injection still fails after that, the
load tops the dense weights back up rather than handing from_pretrained a
snapshot with no encoder in it.
Measured against the real Hub on the 2.3 distilled Q4_K_M pick: 67.24 GB
before, 18.92 GB with a 0.43 GB stand-in for the pre-cast artifact (the base
entry drops from 24 files / 48.79 GB to 13 files / 0.04 GB).
A ggml abort prints its cause first and then a stack trace, so reporting the last twenty captured
lines gave twenty addresses and nothing about the failure: on the macOS runner the native server
died on an unimplemented Metal op and the message carried only frame pointers. The captured tail
now leads with the lines that name a cause and keeps recent context after them, for both the
startup failure and the mid-request death.
Uploads store .txt and .caption sidecars as raw bytes, so one can hold invalid UTF-8. The
trainer treats any existing sidecar, decodable or not, as an empty tombstone and never falls
back to the metadata row for that image. The labeling grid and the dataset summary read an
undecodable sidecar as absent instead, so both showed a metadata caption that the run would
silently replace with the instance prompt, and counted the image as captioned. Both now track
sidecar presence separately, so what the user reviews is what the run trains on.
