Lifecycle rows are shared so a load or a download shows up in everyone's
monitor list, which is deliberate. Since they started carrying via_api_key
they also carry the flag the floating panel auto-opens on, and that reached
every authenticated subject: another logged-in browser sprang open for API
traffic it had nothing to do with. The row now records the caller that drove
it and reports the attribution only to them. Visibility is untouched, so the
row still appears for everybody, and a subject-scoped Clear hides a shared row
it owns rather than deleting it out of everyone else's history.
Auto-download had the flag hardcoded on, reasoning that only an API request
gets that far. Only a /v1 request does, which is not the same thing: Studio's
own chat calls those same endpoints with a session JWT, so a chat that named a
model this server does not have popped the panel open mid-chat, which is
exactly what via_api_key exists to prevent. The attribution now comes from the
request that asked for the download.
Two ways the Hub's per-model settings page could offer a resident model's
saved or default values as if they were live, and then write them back over
the running config on Apply.
ModelConfigPage seeds its editable state from loadedConfig in a useState
initializer, so it reads that prop once per mounted instance. The sidebar
entry keys its instance on a signature of the live config; the Hub's keyed on
the model and quant only. Open the page before /api/inference/status has
hydrated, or while that same target is still loading, and loadedConfig flips
from null to the live config after mount with nothing to remount on: the
editor keeps the values it seeded from and Apply reloads the model with them.
Both hosts now mount under one shared key that includes the live config, so
the arrival of that config re-seeds the editor and a repeated poll of the same
values does not.
The live config itself comes out of the chat runtime store, and landing
straight on /hub is the one entry point where nothing has applied the status
yet: useChatModelRuntime has no mount sync and the chat page is a different
route. The Hub's own status effect pinned the checkpoint and stopped there, so
the resident check passed while kv cache, speculative decoding, tensor
parallel and every GPU placement field still held their defaults. It now
applies the whole status, the same call the chat runtime's refresh makes, and
holds off when a load owns the store or an external provider is selected so it
cannot fight either.
A loose .gguf keys its settings by the bare path with no variant, since the path
already names the one file. The loader still derives a label from the filename
and /status reports it, so the settings page compared a derived quant against a
deliberate null, never matched, and withheld the live launch config from the very
file that was loaded. Applying from that page could then write the saved values
over what the resident model is running with.
The variant equality is skipped for that case only. Every other target, a repo
row or a directory, still has to agree on the quant, because there the variant is
what tells two loaded copies apart.
The overlay opens on API-key traffic only, and record_lifecycle never set that
flag. Auto-switch and auto-download run before the endpoint opens its request
row, so a switch or a download that is refused never reaches api_monitor.start
and its lifecycle row is the whole trace of the request. The monitor therefore
stayed shut on exactly the failures automatic observability is for.
The row now carries the attribution: a load takes it from the request that drove
it, and auto-download passes it directly, since only an API request reaches that
path at all. A manual unload and an idle unload are not API traffic and stay
unattributed, so neither pops the overlay.
Three follow-ups on the last round.
The auto-switch load tried the advertised id before the concrete load path, so an
entry under the alias shadowed the settings the user had just saved and kept
shadowing them. The settings page keys every local row by the path being loaded,
while the alias is derived: the /v1/models name a hand-written overrides PUT is
written against, and for a loose .gguf only its filename stem. Each pair now
reads the path first, and the alias, the bare ids and the older filename-label
key are all still read after it, so a cached repo (keyed by its repo id, which is
the alias) resolves exactly as before.
publicModelId collapses two paths that share a filename or a directory basename
onto one id, so the resident check added last round could mark the wrong catalog
row as loaded and seed its editor with another model's live launch config, then
save that under this model's key. The Hub page now records the loadable
identifier /status reports, as every other status reader already does, so the
literal comparison names one row; the public-id pass only accepts a namespaced
repo id, the one collapse that cannot name two models.
The override map shipped before this browser mirror did, holding only
llama_extra_args and max_seq_length, so an upgraded install can have a server
entry for a model whose context, KV cache, speculative and GPU settings live only
in localStorage. The backfill read key presence as done, skipped exactly those
models and then marked itself complete, so their API loads lost the settings for
good. Filling them in from the browser would reopen the race the conditional
write just closed, so the merge is the server's: the PUT flag is now
fill_absent_fields, and studio_db merges field by field under the write's own
transaction, where a stored value always wins. A fill with an entry already there
also stops replaying that entry's stored flags through validation, so one
denylisted since it was saved cannot 400 the one-time migration.
Tests: the key order and the fill in studio/backend/tests, the collapse in
studio/frontend/tests, and the wiring in tests/studio/test_model_picker_contracts.py.
Five follow-ups on the per-model settings map.
The one-time localStorage backfill read the override map once and then wrote
each model in turn, so a save by another tab during that pass was replaced by
this browser's older copy, against the migration's own "never overwrites"
contract. Re-fetching per model would cost a round trip each; instead the PUT
takes only_if_absent and the server tests and writes under one transaction.
gpu_ids arrived unbounded and normalize_model_override de-duplicated it by
scanning the list it was building, so a large authenticated array cost roughly
20x what the same work costs with a set (4.5s against 0.27s for a million
entries). The payload now bounds the field to the number of ids the normalizer
can store, and the dedupe uses a set.
A settings target opened from the Chat model picker carried no apiLoadable, so
the isGguf fallback mirrored an Ollama GGUF to the server. Ollama's blobs reach
that picker as custom-folder GGUFs under a .studio_links / ollama_links dir,
which local_model_resolver refuses to index, so the mirror advertised a load the
API can never make. The picker, the sidebar editor and the backfill now all use
the same classification.
The Hub settings page compared the loaded model to settingsTarget.id, but a GGUF
loaded from an inactive HF cache or straight off disk loads by path while
/status reports the clean public id, so the page ignored the live launch config
and showed saved or default values. It now also matches the settings identity
and the public id the backend would report.
A standalone .gguf gets a filename-derived format_variant from the inventory, so
the Hub row menu stored its settings under <path>:Q4_K_M while the Chat picker,
the detail card and the backfill all used the bare path. The row menu now uses
the bare path too.
Tests: publicModelId / residentModelIdMatches / isOllamaLinkPath /
settingsGgufVariantForRow in studio/frontend/tests, the create-only write and
the gpu_ids bound in studio/backend/tests, and the wiring in
tests/studio/test_model_picker_contracts.py.
Comment-only pass over the quant-suffix split, the storage-shape normalization
and the wall-clock deadline in the idle-unload test. Same points, fewer lines.
savePerModelConfig normalizes before deciding whether a config is default, and
the runtime hands the settings page Speculative Decoding "auto", which
canonicalizes to null. The page judged the raw object instead, so a model sitting
at defaults looked non-default: turning on "Remember for this model" reported
saved while the local write had dropped the entry, reopening showed it as not
remembered, and the mirror sent the server a speculative_type "auto" override the
browser did not have. That disagreement between the two is the one thing the
mirror is written to avoid.
The page now normalizes once and uses that object for the default check, the
local write and the server mirror alike. Driving the real module under node: the
raw object reads as non-default, storage stores nothing, and only the normalized
reading agrees with storage.
The backfill took a key's identity by splitting on its last colon, so anything
else that ends in one was read as a quant separator. A Windows path made
"C:\models\foo.gguf" into model "C" with variant "\models\foo.gguf", and an
ordinary colon inside a POSIX filename folded "/models/foo:Bar.gguf" and
"/models/foo:bar.gguf" onto one key, so whichever of the two was already on the
server made the other look migrated and left its API loads on defaults.
splitQuantSuffix now mirrors split_quant_suffix on the backend: the suffix has to
be a known quant label, with or without a bits-per-weight modifier, or the head
has to be a .gguf carrying a stem label. Checked against the backend over twelve
keys, including every case above, with identical answers on both sides.
Settings also opens from the on-device detail card, and that constructor never
set apiLoadable, so an Ollama model reached the server mirror and the "API loads
use these settings" line from that entry point even though the auto-switch
resolver skips Ollama's scanner. It now reads the same source the row menu does.
The drive loop waited a fixed 200 iterations of a 10 ms sleep. Windows rounds
that sleep up to the roughly 15.6 ms scheduler tick, on this loop and on the
idle loop under test alike, so the unload got far less real time there than the
count suggests and the test failed on the Windows runner for being slow rather
than wrong. It now waits on time.monotonic with a generous ceiling and still
breaks as soon as the KV file is gone, so the fast path costs nothing.
The test is older than this PR; the deadline is the only change to it.
_is_abs_path_id decides whether an id is a host path that must not be published
through /v1/models, and it asked pathlib.Path, which follows the running OS. A
Windows backend therefore read "/home/me/x.gguf" as a relative name and a POSIX
one read "C:\models\x.gguf" the same way, and in both cases the path was
advertised verbatim as a model id.
Ids outlive the machine that wrote them: settings sync, a WSL session and a
copied config all carry the other platform's spelling, which is why the
model-override identity in this PR already folds Windows drive, UNC and WSL
paths. The backend now agrees with it, reading the value as both a POSIX and a
Windows path. Neither reading can misfire on a repo id, which has no leading
separator, drive letter or UNC prefix.
The two tests this fixes on Windows are older than this PR; the new one pins the
contract in both directions.
Pass over every comment this branch touches. Collapse the multi-paragraph
rationales to the point they were making, drop prop docs that only restated
the prop name, and reflow the rest onto fewer lines. No code changes.
A repo cached outside the active HF cache reports load_id as its snapshot path
(cache_inventory), which is what the loader needs, but the chat picker and the
auto-switch index both name that repo by repo_id. The new Hub settings page was
saving under the load id, so the settings landed on a key no other load reads:
the picker, an auto-load and an OpenAI-compatible request all fell back to
defaults, and a server override already stored under the repo id could win
against the save.
ModelPickTarget now carries configId for the case where the storage identity is
not the loadable one. Every read, write and server mirror in ModelConfigPage
uses it; the chat template and GGUF header probes keep target.id, since they
have to open the model. The Hub sets it for cache rows and resolves its own
load through the same helper, so a config saved from the settings page is the
one a later load finds. Rows whose load id is already their identity, which is
every local row and every repo in the active cache, are unaffected.
Verified against the real per-model-config module under node: saved under the
snapshot path, a picker read reports remembered=false and the default max
sequence length; saved under the repo id it reports remembered=true and 8192.
The backfill wrote the snapshot it took before fetchModelOverrides resolved, so
a save or a forget during that round trip was undone: the write is queued behind
the interactive one and commits last, leaving the browser showing the new
settings while an API load applied the old ones. Each write now re-reads the
model's current local config and skips it if it has gone or gone back to
defaults. Verified against the real module under node: the write carries
maxSeqLength 9999 where it previously carried the stale 1000.
target.isGguf was also standing in for "an API request can load this". It
cannot for an Ollama model: local_model_resolver skips Ollama's scanner on
purpose, so those models are never in the auto-switch index, yet the mirror ran
and the settings page told the user the API would apply them. The target now
carries apiLoadable, set from the inventory source the row already has, and both
the mirror and that sentence read it.
Extracting the labels out of the monitor page left two contracts asserting they
were still in it, so the staged run went red on all three platforms. They read
the new module now, and the page contract additionally pins that it imports from
there rather than redefining them.
Three separate reports, all confirmed against head.
listPerModelConfigs reported future-schema records. loadPerModelConfig refuses to
apply one and eviction refuses to drop one, because this client cannot interpret
that schema, so handing it to the backfill would persist a partial reading of it
server-side and let an API-triggered load apply settings the same client will not
apply locally. It is skipped there now, matching the other two paths.
The detail view's on-device card passes a null variant while its own lookup is
pending or after it failed, and this entry point opened the editor anyway. That
saves a bare-model config, which the picker never finds because it matches
variants exactly, while the API's bare-key fallback would apply it.
openModelSettings already refuses with a toast for exactly that reason; this
path now refuses the same way.
A failed detail fetch was never retried. The revision is recorded when the fetch
starts, and on failure the entry stays missing, so selectedIsMissing does not
change and a terminal row's updated_at does not advance: nothing was left to
re-run the effect, and the full prompt and reply stayed unavailable until
another row was selected. The in-flight flag settling is the trigger now, and
the attempt count bounds it, because the usual failure is an entry that has aged
out of the ring buffer and will never arrive however often it is asked for.
Two holes in the previous two commits, both found by the same review round.
The per-model write queue keyed on the literal spelling, so the backfill's
legacy casing and a UI save's normalized one opened two queues for one model and
raced exactly as before. It now keys on the folded identity, which is what the
backend resolves by.
A .gguf with no recognizable quant token is labelled by its filename stem, and
v2 storage lowercases that label while the scanner probes with the filename's
own casing. Folding only recognized quant labels therefore left the migrated
entry unreachable for precisely the files that need the stem fallback. The
suffix rule now also accepts a case-insensitive match against the label the
scanner derives for that filename, which keeps an ordinary colon out because the
head still has to be a .gguf. _bare_model_id drops onto the same shared rule
rather than repeating half of it.
Saving twice quickly, or saving while the one-time backfill is still running,
started independent requests with no sequencing, so the older response could
commit last and resurrect the entry the newer one meant to replace or remove. An
API-driven load then applies context or GPU settings the user has already
changed, with nothing in the UI showing it.
Writes now chain per override key. The chain hangs off the settled tail, so a
failed write cannot cancel the next one, and only the last writer clears the
slot so a queue that is still building keeps its order. Different models still
overlap. Verified against the real module under node: two saves for one model
with the first made slow commit oldest-first and never overlap, where the
previous version committed them in the wrong order; a rejected write still lets
the next succeed; and two models still run concurrently.
The overlay is mounted from __root.tsx and imported two label helpers from the
page, so the page and its dependency graph were pulled into the eagerly loaded
bundle and the route's lazyRouteComponent bought nothing: every route paid for
the monitor page even when it was never opened. Measured on a production vite
build, the async api-monitor chunk was 0.20 kB, meaning the implementation had
landed in the main bundle.
The helpers now live in their own module. The same build gives an 18.83 kB
api-monitor chunk and a main bundle 18 kB smaller (3.9 kB gzipped).
An explicit remove ran the launch-flag validation first, so a form still
carrying a rejected flag raised a 400 and left the override in place. Nothing is
stored on that path, so there is nothing to validate; remove now short-circuits
it, which is what the branch below already claims to do.
Clear log dropped only the caller's own rows, but a lifecycle row is shared: it
is visible to everyone and owned by no one, so those rows survived and the
reload straight after the click brought them back, leaving the button visibly
ineffective. Deleting them is not an option either, since that erases another
caller's history. They are now hidden per subject, so the clear is true for that
caller and harmless to the rest. A shared row that is still running is live
state rather than history, so it stays visible, and the hidden ids are pruned
against the ring buffer so they cannot accumulate.
Three gaps the previous round left, all in the same key-resolution rule, so the
rule now lives in one place as split_quant_suffix.
A quant label may carry a bits-per-weight modifier, because two files at the
same base quant are kept distinct by it: utils/models/model_config.py preserves
IQ4_XS-3.53bpw while hub/utils/gguf.py strips it, and both forms reach the
override keys. The known-quant pattern accepted neither, so _bare_model_id
missed the bare entry and the first qualified save dropped its launch flags.
On POSIX the browser lowercases the quant but keeps the path casing, so a
migrated "/models/Foo:q4_k_m" was unreachable from the scanner's
"/models/Foo:Q4_K_M". Only the quant suffix folds now, and only when it really
is a quant, so "/models/foo:Bar.gguf" stays a distinct filename and the path
itself stays case-sensitive.
A standalone .gguf picked directly has no quant to choose between and is stored
with a null variant. The backfill filter read that as safetensors and skipped
it, and the done flag is set on the same pass, so those settings stayed
browser-only for good while auto-switch kept loading the model with defaults.
A load, unload or download is recorded in the monitor but is not an HTTP call.
It reads as running for as long as the load takes, so it was counted as an
in-flight request with no client waiting, and a multi-minute download was folded
into Avg latency and the error rate. The backend already excludes these rows
from active_count for the same reason, so the page was also disagreeing with the
number the API itself reports. Requests counted them too, so that is now the
non-lifecycle count rather than the raw entry count.
Also limit the API-reach sentence on the Hub settings page to GGUF models. The
Hub opens that page for every downloaded model, but ModelConfigPage mirrors
settings to the server only when target.isGguf, because auto-switch indexes
GGUFs only, so a safetensors user was told the settings apply to an API request
that cannot reach them.
A .gguf whose filename holds no recognizable quant token still gets a label
from the scanner, which falls back to the filename stem, so the UI stores keys
like "/models/custom.gguf:custom". _bare_model_id accepted only known quant
tokens, so the first per-quant save did not carry over llama_extra_args stored
under the bare id, and auto-switch prefers the qualified entry, so those flags
were silently dropped with no UI able to restore them.
Accept a suffix that is exactly the label the scanner derives for that filename.
Requiring the head to be a .gguf and the suffix to match exactly is what keeps
an arbitrary colon-containing POSIX path out: "/models/foo:bar.gguf" splits to
a head that is not a .gguf. The filename is taken by splitting on both
separators, since a "C:\\..." key is written on Windows but may be read back
by a backend that is not.
The merge that brought main into this branch dropped the closing brace, so the
function body ran straight into the interface declared below it and the frontend
did not compile at all: tsc reports TS1005 at the end of the file and vite fails
the build. Reproduced against the pushed head and clean with the brace restored.
resolve_model_override_key refused the case fallback for every filesystem path,
but only a POSIX path is case-sensitive. A Windows drive path, a UNC share and a
WSL drive path each name one file whatever the casing, and the browser folds
exactly those three before storing. A Windows user's migrated entry was
therefore keyed lowercase while an API auto-switch resolved the same file with
its on-disk casing, so the lookup missed and the saved launch flags silently
stopped applying until the settings were saved again.
Fold those three shapes here too, normalizing the separator as the browser does
so C:/Models/Foo.gguf and c:\models\foo.gguf agree. POSIX stays case-sensitive,
/mnt/data stays an ordinary mount rather than a WSL drive, and an ambiguous fold
still matches nothing so a load takes defaults instead of guessing.
The existing Windows test asserted the opposite. It carried no rationale, unlike
its POSIX sibling, and get_model_override's docstring already scopes the rule to
POSIX, so it read as an over-generalisation of the POSIX case.
app_settings carries no schema version, so an install that predates identity
normalization holds rows keyed by whatever id was typed, such as
Unsloth/Repo-GGUF:Q4_K_M, while this browser only ever stores the folded form.
The exact property lookup therefore reported "not on the server" for a row that
is, and the backfill PUT over it, replacing server settings the file documents
as the newer authority. The migration runs once on every existing profile, so
this lands on exactly the upgrades it was written to protect.
Fold both sides before comparing, splitting on the last colon because a quant
label never contains one. A repo id and a Windows path fold, a POSIX path does
not, which is the same rule the backend resolves by. Verified with the real
module under node: the legacy-casing, variant-casing and Windows-path cases go
from overwriting to skipping, a second run stays clean, and a genuinely new
model, a different quant of the same repo, a POSIX path differing only in case,
and a bare legacy key all still migrate.
The remove branch already resolved the key a load would use, but the save
branch wrote payload.model_id literally. The browser normalizes casing before
storing, so a backfilled key and a later UI save left two entries for one
model; with two equivalent keys present resolve_model_override_key finds no
unique match, so any third casing resolved to no override at all and the model
silently loaded with defaults.
A standalone .gguf gets variants=() from the resolver, so variant is None and
only the bare ids were tried. The picker keys the same file by the quant label
it derives from the filename, which is never empty, so those settings lived
under <path>:LABEL and nothing reached them. Try the filename-derived key after
the variant-qualified ones and before the bare ones, so older bare entries
still work.
* Studio: add durable Deep Research workflows
* Studio: preserve research integration after upstream updates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep research worker compatible with Python 3.11
* Studio: address Deep Research lifecycle review
* Studio: preserve durable research recovery
* Studio: preserve research stream and context
* Studio: harden research sources and limits
* Studio: align research with shared chats
* Studio: guard durable research actions
* Studio: protect durable research turns
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: deepen durable research decisions
* Studio: protect research prompts and queries
* Studio: slim research stream deltas
* Studio: preserve research evidence and citations
* Studio: harden Deep Research (CI, prompt injection, query PII, config, citations)
- Fix backend CI: add research_runs_router to the synthetic routes stub in
test_desktop_auth so studio.backend.main imports under the health-check test.
- Escape prompt-delimiter tags in the decision and synthesis prompts so gathered
web/document content cannot close an <untrusted_...> wrapper and inject
instructions into the local planner/decision/synthesis model.
- Extend the public-query sanitizer to redact Luhn-valid payment cards, phone
numbers, non-global IPs, and labeled private identifiers before a query can
reach web search.
- Reject nested credential keys in inferenceRequest and ragScope, not just
top-level keys, when persisting a durable run config.
- Treat maxSources as one budget shared across web and document sources
(collection and resume paths) instead of per type, which allowed up to 2x the
configured cap.
- Preserve document citations whose filename contains a closing bracket by
tokenizing valid citations before stripping invalid ones.
- Persist Deep Research off when switching to an external model and when enabling
Web Fetch so a refresh cannot rehydrate a mutually-exclusive state.
- Add regression tests for the query, prompt, citation, and config hardening.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make the research claims table migration atomic
The owner-scoped to global claims migration ran its RENAME, CREATE, INSERT and DROP in autocommit, so an interruption after CREATE left the new table empty, orphaned the rows in the legacy table, and never re-triggered. Wrap the rebuild in an explicit transaction so a crash rolls back cleanly and the migration re-runs on the next boot.
* Studio: block message edits and regeneration during an active research run
After a reload a durable research run is followed by the research store rather than an assistant-ui run, so thread.isRunning is false while research is still active. Message edit, refresh and the edit composer previously gated only on isRunning, which let a normal generation start alongside the running research run. Gate them on the active thread's research state as well.
* Studio: keep the plan review mounted through approval
Keying PlanReview on planRevision remounted it mid-approve when updateResearchPlan bumped the revision, resetting the local pending flag and re-enabling Start research while the approve was still in flight, which allowed a duplicate approve. Key on runId only.
* Studio: drop the redundant deep-research persistence change
setCheckpoint already persists Deep Research off for external models at the top of the function, so the added saveBool was a duplicate, and clearing Deep Research from setWebFetchToolsEnabled guarded a state that is not reachable (Deep Research is local-model only while the Web Fetch pill is external-provider only). Revert both to the pre-hardening version.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden Deep Research citations, query privacy, and message protection
Address review findings in the Deep Research backend:
- Escape an unbalanced ")" in citation destinations so a source URL cannot
close the markdown link early and inject a second link, keeping balanced
parentheses literal.
- Match raw-URL citations on whole tokens so a URL sharing another URL's
prefix is no longer partially rewritten.
- Redact non-global IPv6 addresses in public search queries, matching the
existing IPv4 handling.
- Detect credential key names after normalizing case and separators so nested
openaiApiKey, accessToken, and clientSecret values cannot be persisted.
- Reject client edits to server-managed research prompts and reports at the
storage layer; only the internal writers pass allow_research_update.
- Scope research searches to the first allowed domains instead of dropping
site scoping for large allow lists.
- Persist the same fetch evidence bound used during live synthesis so a
resumed run is not shortened.
- Scope run completion so it only replaces this run's message parts.
Add regression tests for the above.
* Studio: fix Deep Research SSE framing, source counts, and favicon privacy
- Normalize the whole SSE buffer so a CRLF split across transport chunks
still frames events.
- Count web and document sources together in the activity header so a
RAG-only run is not shown as zero sources.
- Cap the plan editor at the run's configured maxSteps instead of a
hard-coded 30.
- Add an allowRemoteIcons opt-out to the sources components and disable
third-party favicon requests for research sources so visited domains are
not leaked.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address final Deep Research review findings
* Studio: fit Deep Research synthesis evidence to loaded context, add opt-in web grounding
Size the synthesis evidence budget to the loaded model context so the prompt is not
silently truncated on small contexts. When the evidence overflowed the window the report
degenerated (it echoed the evidence tail instead of writing); the budget now reserves tokens
for the prompt scaffolding and converts the remainder to chars, keeping the full cap when the
context is unknown.
Add opt-in web grounding for auto-read: read the top search results, ingest them into an
ephemeral RAG scope, hybrid-retrieve the passages most relevant to the question with the
existing knowledge-base retriever, and fold those chunks into the step evidence. The scope is
per call and deleted afterwards, so a user's knowledge base is never touched.
Off by default; enable with UNSLOTH_RESEARCH_AUTO_SCRAPE=1. Gated per run by
budgets["maxAutoScrape"], so runs created without it keep legacy snippet-only behavior, and
grounding is skipped when the loaded context is too small for the prompt.
Add tests for the adaptive evidence budget, scraped-text cleaning, the ephemeral web-RAG
retrieval and scope cleanup, and the auto-read evidence path.
* Studio: read Deep Research synthesis context from the inference orchestrator
Make the adaptive synthesis-evidence budget actually engage in the normal Studio
architecture. _loaded_context_length read core.inference.inference, the low-level backend that
lives in the model subprocess and stays unpopulated in the main web process where the research
supervisor runs, so it returned None and the budget silently fell back to the 32000 character
cap (leaving the report exposed to the truncation this was meant to fix). Read the inference
orchestrator instead, and the llama.cpp backend for GGUF, mirroring
routes.inference._monitor_context_length so the budget sizes to the context the API layer
serves. Verified on a running server: at a 12288 token load the probe now reports 12288 and the
budget adapts to 24576 characters instead of the 32000 fallback.
Also:
- Reserve context for the generated report as well as the prompt scaffolding (raise the reserve
to 4096 tokens) so evidence does not crowd out the output on a small window.
- Honor a numeric UNSLOTH_RESEARCH_AUTO_SCRAPE by passing the per-run maxAutoScrape as the page
cap to the scraper, instead of always reading the maximum.
- Guard the web-RAG connection acquisition so a get_connection failure returns the documented
empty result rather than propagating.
- Add a synthesis-context test that patches the real backend accessor (not the probe itself) so
the production wiring is exercised, plus a scrape page-cap test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden Deep Research query redaction and research autosave
- research_runs: extend the opaque-token allowlist so unlabeled Hugging
Face (hf_) and GitLab (glpat-) tokens are redacted before a query can
reach web search, without over-redacting public model or version ids.
- runtime-provider: for a server-managed research message, echo the
backend-stored metadata verbatim on autosave. Merging the client
metadata re-added client-only fields the server never persisted, so the
server-side guard saw a diff and rejected every streamed or snapshot
update with 409.
* Studio: keep composer tool pills always accessible after merge
The merge left the composer line marked always-expanded (data-expanded
"true") while the inner pill row was still gated behind composerExpanded,
so the Search and Code toggles disappeared once the permission mode was
"off" with no other toggle set. Render the primary tool pills
unconditionally, matching the always-expanded layout, and drop the now
unused composerExpanded and permissionMode locals. Fixes the Chat UI
Playwright check that asserts the Search and Code pills stay visible.
* Studio: update Deep Research composer contract to always-expanded layout
The always-expanded composer no longer routes effectiveDeepResearchEnabled
through a composerExpanded expression, so the frontend contract now checks
that it gates the Deep Research composer button render instead.
* Studio: do not bind a research run to a populated assistant reply
create_run adopted any assistant message under the user turn whose
researchRunId was unset, including a prior answer reused by a retry. On
completion _update_assistant drops the untagged text and source parts, so
that answer was silently overwritten. Only bind to an empty placeholder or
this run's own message, and reject a reply that already carries content.
* Studio: harden Deep Research synthesis budget, prompt shielding, and message protection
- research_runs: split the synthesis evidence budget evenly across notes so a
small context still keeps a slice of every research step instead of dropping
the later steps after the earliest ones fill the budget.
- research_runs: shield the research question and approved plan before placing
them in the decision and synthesis prompts, so a closing delimiter in either
cannot escape its block and inject sibling sections.
- research_runs: redact bearer authorization tokens from public search queries.
- studio_db: include attachments in the research-message change check and guard
direct attachment deletion, so server-managed research prompts and responses
cannot be mutated through the attachment paths.
- chat_history: map the protected-message conflict on attachment deletion to 409.
* Studio: strip invalid document citations that contain brackets
The invalid-citation regex stopped at the first closing bracket, so a
citation whose filename contained brackets left its tail (".pdf, p. 9]") in
the report. Match a balanced bracketed span so the whole invalid citation is
removed; valid citations stay protected by the earlier tokenization pass.
* Studio: free the RAG search slot when a lookup times out or is cancelled
The bounded knowledge-base search held the sole admission slot in a detached
worker until the search returned, so a lookup that outlived its timeout (a
stalled embedding or blocked vector call) kept the slot forever and starved
every later lookup, disabling knowledge-base retrieval globally. Release the
slot from the caller when it stops waiting, exactly once, so a detached worker
finishes without re-holding it.
* Studio: remove Websites label from research composer
* Studio: fix Deep Research review findings (RAG slot bound, orphaned workers, hardening)
- Bound the shared RAG search slot to one running worker. The search that is
doing the embedding/index/GPU work now owns the admission slot until it
finishes, instead of freeing it on caller timeout while the detached worker
keeps running, which let a second search enter and stack concurrent work
behind the capacity-of-one semaphore.
- Cancel active research runs before deleting their thread, project, or all
history. Deleting cascade-drops the run row, but the worker only notices at
its next lease check, so it could keep doing model/web/RAG work for a run
that no longer exists; signalling cancel first shortens that window.
- Shield the planner prompt's conversation and question with _shield_untrusted,
matching the decision and synthesis prompts, so untrusted text cannot forge
planner delimiters.
- Do not let a research key-revocation failure replace a successful
non-streaming completion; log it like the streaming path does.
- Include created_at in the protected research-message guard so a client cannot
reorder server-managed prompt/response messages while leaving the body intact.
- Reject non-scalar ragScope values; a nested container evades the
sensitive-key scan when its inner keys are unlisted and would reach retrieval
code that expects a scalar scope id.
Adds regression tests for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: remove research composer globe icon
* Studio: use Hugeicons telescope in research composer
* Studio: use Telescope02 icon in research composer
* Studio: standardize Deep Research telescope icons
* Studio: move Deep Research below web and code tools
* Studio: merge grounded page excerpts with search snippets instead of replacing
When auto-scrape grounding retrieved page-body chunks, it replaced the raw
search-result text for that step. If the retrieved chunk was a distractor or
dropped the key fact, the answer-bearing search snippet was lost and grounded
runs regressed below snippet-only accuracy on factual questions (e.g. returning
Apache 2.0 instead of the Qwen License, 403 instead of 404, or a single mirror
diameter instead of the sum).
Keep the search snippets and append the grounded excerpts as supplementary
evidence via a small _merge_scraped_evidence helper. Grounding stays opt-in and
off by default, so legacy runs are unchanged. Adds regression tests.
* Studio: improve Deep Research synthesis
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden Deep Research synthesis flow
* Studio: validate Deep Research derived context
* Studio: align Deep Research synthesis evidence
* Studio: restore Deep Research synthesis state
* Improve Deep Research source queries
---------
Co-authored-by: alkinun <alkinunl@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
* Fix Windows no-torch setup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix no-torch env normalization on Windows
* Accept on for Windows no-torch mode
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep no-torch mode across studio update on Windows
Guarding the direct torch/Triton install made `install.ps1 --no-torch`
actually produce a torch-free venv, which then broke the next
`unsloth studio update`. That path exports no UNSLOTH_NO_TORCH, so
$NoTorchMode was false, the stale-venv check read the missing torch as a
broken venv, and setup tried to delete the venv it was running out of:
[ERROR] Could not remove stale venv: Access to the path 'python.exe' is denied.
That teardown can never succeed there, because setup.ps1 runs via
unsloth.exe out of that same venv. The same gap also let the shared
dependency pass reinstall torch from PyPI, unpinned, into a GGUF-only
environment.
install_python_stack.py now records the mode in the install manifest and
setup.ps1 reads it back when no env var is exported, then re-exports a
canonical value for the dependency pass (setup.ps1 drops the manifest
before invoking it, so the child cannot repeat the lookup). The key is
additive and MANIFEST_SCHEMA is unchanged, so existing manifests stay
valid and a missing key keeps today's behaviour.
Also:
- read_manifest() caught only OSError, but UnicodeDecodeError is a
ValueError. That is now on the installer's import path, so a manifest
re-saved as ANSI or truncated mid-write would abort every install.
- The env predicate now trims surrounding whitespace, matching the
Python side.
- The Windows update smoke workflow asserts the update leaves the venv
GGUF-only, which is what would have caught this.
Known follow-up, pre-existing: an install killed between the manifest
drop and the dependency pass leaves no recorded mode, so a later update
still walks the stale-venv path. Closing that needs a marker the
installer never drops.
* Persist no-torch mode in a marker the dependency pass cannot drop
The install manifest alone was not enough. Both setup.ps1 and
install_python_stack.py remove it before every dependency pass, and it is
only rewritten on success, so a no-torch install interrupted in between
left nothing recording the mode. The next update then resolved no-torch
as false, read the expected missing torch as a stale venv, and tried to
delete the environment whose python.exe was running it, which leaves the
install unrepairable from the CLI.
Add .unsloth-no-torch next to the existing .unsloth-studio-owned marker,
written before the pass and cleared when torch is wanted. setup.ps1
writes it as soon as the mode resolves, so the window between the
manifest drop and its own torch install is covered too.
Read order stays manifest key first, then marker, so migrating out of
no-torch is never blocked by a marker an earlier run left behind. Neither
present still reads as "install torch", so nothing changes for installs
made before either existed.
Also adds the AGPL-3.0 header the new test file was missing.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
* Gate the sed commands that run a shell
GNU sed executes a shell through its `e` command, both as a standalone
command (`sed -n '1e CMD' file`) and as an `s///e` flag that runs the
pattern space. It goes through popen(), so it is a literal `sh -c`, but
the terminal scan only ever saw `sed` at command position and treated the
program text as an ordinary argument.
That left `sed -n '1e rm -f victim' /etc/hosts` running with no prompt in
auto mode, and `_find_blocked_commands` returning nothing for it, so the
hard blocklist that applies in every mode missed `rm` as well.
Screens the program the same way the awk arm does. `-e` values are joined
with newlines first, since that is how sed assembles them: `sed -e '1a\'
-e 'e CMD'` appends a literal line and runs nothing, so judging the pieces
separately would prompt on a benign script. The scan then steps over every
region where `e` is data rather than a command: address and substitution
regexes, replacements, `a/i/c` text, `r`/`w` filenames, `b`/`t` labels and
comments. That keeps the common idioms silent, including `:e;N;$!be` loop
labels, `s/e/E/g`, and `s/a/b/we out.txt` where the `e` belongs to the `w`
filename and sed does not execute.
The blocklist scan recurses into a literal `e` payload the same way it
already does for `bash -c`. A bare `e` or an `s///e` can only be prompted,
since what they run is the pattern space, which is input-file text that is
not knowable statically.
Verified against real GNU sed 4.9 rather than the manual: 80 commands run
for real with a marker payload, comparing what sed actually executed
against the classifier, with no mismatches in either direction.
* Close five ways a sed program hid its shell payload
Review found five shapes the first pass missed. All five execute on GNU
sed 4.9, checked by running them rather than reading the manual.
A payload line ending in a backslash continues onto the next line, so the
scan now ends an `e` at an unescaped newline and unescapes the text the way
sed's read_text does. That is what resolves `r''m` back to `rm` for the
blocklist.
A sed comment ends at a real newline, but the terminal scan had already
replaced every newline with `;`, including newlines inside quotes, so
`# comment` swallowed the rest of the program. The sed arm now also sees a
variant where only unquoted newlines become separators, built on a
character-by-character quote scanner rather than a regex: an apostrophe in
a double-quoted word mis-pairs under a regex and inverts the state, which
opened a bypass while this was being written.
Everything attached to `-i` is a backup suffix, so reading `-ifoo` as an
attached `-f` lost the real script. Replaced the shared short-flag helper
with sed's own option grammar, which also fixes `-l 5` and
`--line-length 5` eating the script as their operand.
A sed child of `find -exec` was never recorded, so the blocklist skipped
its payload.
Substituted text splices straight into the program, and an address is as
good a place as any to open `;e CMD`, so a command substitution anywhere
in the program is treated as unresolvable. Scoped to the program: a
substitution in a file operand still runs, a `$(` or backtick inside single
quotes is literal, and parameter and arithmetic expansion are untouched.
The cost is that a substitution used to build a program now asks.
Bounding the -exec walk keeps the blocklist linear; without it a repeated
`-exec sed` line went quadratic.
Verified against real GNU sed across 103 commands run for real, no
mismatch in either direction.
* Fail closed on padded sed lines, and stop gating sed --sandbox
Four more from review, each checked by running it rather than reading the
manual.
The cap that keeps the argument walk linear was itself the bypass: padding
a line with 128 valid options pushes the script past it, and an empty
program read as proof the command only edits text. The budget is now shared
across the sed words on a line, so a lone sed reads its whole argument list
while a line packed with sed words keeps the floor that holds the walk
linear, and overflow fails closed instead of falling through.
The substitution scan counted parentheses without consulting quote state,
so a quoted paren in the substitution body left the span unterminated and
the program never matched. It now balances through the same quote scanner
used elsewhere, since a substitution body reopens quoting.
A wrapper between -exec and its child hid the child from the blocklist.
Following the wrapper also fixes the neighbouring blocked-name check, which
missed find . -exec env rm the same way. The wrapper's own name is still
screened: -exec sudo rm reports both.
sed --sandbox and --posix refuse e outright and exit 1, so gating them was
prompting for something that cannot run. They are now inert, except after
--, where the flag is an input filename and the script still executes.
env -u still hides a child from the blocklist, on this path and at top
level. That is pre-existing and left alone here.
* Resolve the sed program through find, wrappers, globs and variables
Five more from review, each run against real sed rather than read off the
manual.
find's -exec ends at + or ;, but the sed argument walk ran past it into the
next predicate, where a following -exec grep -e safe was read as sed's own
-e and discarded the real script. Stopping at the terminator also removes a
false prompt, since -exec was being parsed as -e xec and inventing a payload.
Hopping a wrapper skipped its name but not an option that takes a separate
operand, so env -u FOO sed returned FOO as the child. The table this file
already keeps for wrapper options covers it, moved up so both layers share
it. That also settles the top level: env -u PATH rm -rf x now reports rm,
as do env --unset, stdbuf -o L and xargs -I {}. Two false positives go with
it, timeout -s KILL 5 rm blaming the signal name and env -u kill blaming a
variable name, while timeout -s KILL 5 kill -9 1 still reports kill.
A program held in a variable was invisible: the assignment regex stops its
value at whitespace, so a program containing a newline never entered the
map in any pass. Resolved at the token level instead, where the value is
already whole. Both the written and the resolved program are screened,
since either can hold the e.
A command-position glob that can resolve to sed is treated as sed. The
auto gate already asks about any unresolved command glob; this is for the
blocklist, which did not know the name.
Inside double quotes a backslash makes the next character literal, so
sed "s/\$(CC)/gcc/" runs no substitution and should never have asked. The
quote scanner now reports an escaped character under its own state.
Left open: on Windows the blocklist lexer keeps quoting in its tokens, so
a multiline program held in a variable resolves there but not to a name
the blocklist reads. The prompt still fires on every platform.
* Ask when the sed program is not a literal we can read
Two from review, and the second one changes the default rather than adding
another case.
sed --sandbox and --posix were being read as disabling e for the whole
invocation. They disable exactly the scripts written after them: sed
compiles each -e as that option is parsed, and the positional script only
after the option list, so sed -e '1e CMD' input --sandbox runs the payload
with no POSIXLY_CORRECT needed. Suppression is now positional. Reading
POSIXLY_CORRECT out of the command text was considered and dropped as
unsound, since export or an outer bash -c puts it somewhere the text does
not show.
A program built by a parameter transformation was invisible: only bare
$NAME and ${NAME} were resolved, so ${p#x } passed through untouched. Rather
than add operators one at a time, a program that still holds a live
expansion after resolution is treated as unreadable and asks. Unhandled
expansion forms are now safe by default instead of silent, which also
closes ${p%Z}, array elements, printf -v, read, and p=$(...) whose binding
shlex had been truncating to a bare $.
Arithmetic is collapsed rather than exempted. It can only ever evaluate to
an integer, so it cannot spell a sed command, but leaving it as written let
"$((c+1))e CMD" read as an append-text command that swallowed the payload.
The cost is that a double-quoted program holding an unassigned variable now
asks: sed "s/$OLD/$NEW/g" f. Measured at 24 of 169 realistic invocations,
all of that one shape. Exempting it would trade enumerating expansion
operators for enumerating assignment forms, and four of the bypasses above
sit outside the assignment pattern, so the blanket rule stays.
Left open: -f prog.sed is still unscreened, since the program is in a file.
* Decide where a sed scan stops by context, not by token text
Four from review, two of them exploiting fixes from earlier rounds.
Stopping the sed walk at a + or ; token read the text after shlex had
already removed its quoting, so a quoted file operand looked exactly like
a find terminator and the scan gave up before the -e that followed. sed
still compiles that -e, because getopt permutes. Termination is now decided
by token index: a separator counts only if it was unquoted, and + or ; only
while a find or fd exec action is open, which is the only place quoting
does not matter. The same shape works with & | ( ) and }, so all of them
are covered.
The assignment map kept the first binding for a name, but the shell uses
the most recent one before the command. Bindings are now ordered and only
those preceding a given sed are folded in, with a later one replacing an
earlier. A value that is not itself literal clears the name rather than
leaving the older literal standing, which would otherwise have dressed an
unread program up as a safe one.
Exhausting the wrapper budget under find -exec returned the same answer as
finding no child at all, so a long enough chain of wrappers hid whatever
followed. It now reports overflow and blocks the chain word. This was
hiding more than sed: the same shape hid a plain rm.
fd spells its exec flags -x, -X, --exec and --exec-batch, none of which
were routed into the nested scan. They are now, but only while a find or
fd word is in scope and no action is already open, so a -x that belongs to
a child command is left alone.
Prompt rate is unchanged at 45 of 169 realistic invocations; this round
adds no new prompts.
* Drop the words the shell removes before a command runs
Two from review, both verified to run for real.
A redirection is performed by the shell and never reaches the command, but
the words stayed in the token list and the first of them was taken for
sed's positional script, so the real one behind it was never read.
`sed </dev/null '1e touch MARKER' input` creates the file, and so do the
`>`, `2>`, `2>&1`, `&>`, `>|` and here-string spellings. Redirections are
now recognised as spans and skipped: the target may be glued on, be the
next word, or sit one further along when a punctuation character splits
the operator. A skip is honoured only where sed would take the word as an
argument, so a pending -e/-f/-l value is still read.
The same words also hid a command outright. `> out.txt rm -rf victim` and
`2>&1 rm -rf victim` both really delete, because the redirection target
was read as the command word and the rm behind it landed in argument
position, where the always-on blocklist does not look.
shlex emits a RUN of punctuation characters as one token, so bash's `|&`
matched no separator and a sed scan ran on into the NEXT command, taking
its `-e safe` for the real script and dropping the payload. Any token
built only from those characters now ends an invocation, and a quoted one
is excluded the same way a quoted `';'` already was.
The third item from that review, `-l N` eating the script as its length
operand, was already closed in 9a5cfddb.
Prompt rate is unchanged at 45 of 169 realistic invocations; this round
adds no new prompts.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Read a sed program from what the shell really hands it
Five from an independent review pass, each verified by executing it.
sed joins its -e and -f sources with newlines, but a source boundary also
closes a line continuation open across it. Reading every -e as one
uninterrupted text let an unreadable -f in the middle hide the piece
behind it: `sed -e '1a\' -f /dev/null -e 'e CMD' input` runs CMD while the
same line without the -f only appends text.
A program flag ahead of the positional script makes that word an input
file. One behind it does so only while getopt permutes, and
POSIXLY_CORRECT turns permutation off from outside the command text, so
the positional is now read as a script as well. The suppression that a
flag written first performs is unchanged.
xargs builds the argv of the command behind it, appending what it reads on
stdin and substituting it into an -I placeholder, so the program need not
be in the text at all. A sed whose program is empty or is only the
placeholder is failed closed. The ordinary idioms are untouched: their
program is present and the placeholder stands where the file goes.
Only a word that really changes shell state rebinds a program held in a
variable. An assignment-shaped argument, one inside a subshell and one
used as a command's environment prefix all leave the variable alone, and
recording them replaced a payload with a value bash never assigned. A
conditional assignment after && or || may or may not run, so it clears the
name rather than being guessed at.
Exec-flag forwarding now starts only at a command word. Any token spelled
fd or find used to turn it on, so a -x or -exec in the text after one was
read as an exec flag and its neighbour hard-blocked; `echo fd -x rm` and
`grep fd -x rm file` were refused outright. A command-position glob bash
resolves to find is still recognised.
Prompt rate is unchanged at 45 of 169 realistic invocations.
* Judge a sed program against what getopt and find really do
Seven from review, each verified by executing it.
A redirection is removed wherever it stands, including where an option
value goes, so `sed -n -e >out '1e CMD' input` takes the word behind it as
the script. The skip is now honoured ahead of a pending value rather than
after it. The target of a detached redirection may itself look like an
option or a quoted operator, and the shell hands it to open() either way,
so `sed > --sandbox '1e CMD' input` and its `> ';'` twin no longer leave
that word standing as a sed flag or script. Only a bare operator is
refused, which is a malformed line.
A program flag written behind the positional script and the positional
itself are ALTERNATIVES, since permutation decides which sed compiles and
nothing in the text settles it. They were joined into one program, where an
unterminated command in the one swallowed the other: `-e safe` is an `s`
with delimiter `a` and no closing one, and it ate the payload behind it.
Each source is now scanned on its own.
find closes its batched form at `{} +` only, so a `+` anywhere else is an
ordinary argument it hands the child. Stopping at one threw away the script
behind it. The `;` spellings need no such test: a quoted `';'` and an
escaped `\;` reach find as the same word and it stops at either, which the
`;` twin of that line confirms by not executing.
An `-f` naming a stream (`-`, /dev/stdin, /dev/fd/N) takes the script off
stdin, which the same command line may well supply through a heredoc. That
is ignorance rather than safety, so the sed fails closed. A named program
file is unreadable in a different way and is unchanged.
bash expands the program word before sed is started, so in a directory
holding a suitably named file `sed *` runs whatever that file contains.
A program word carrying an unexpanded glob now fails closed. Quoted
programs expand nothing and a glob among the file operands is not the
program, so ordinary work is untouched.
Prompt rate is unchanged at 45 of 169 realistic invocations.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep command position and quoting intact through the sed scan
Six from review, two of them regressions the previous commit introduced.
Scoping exec-flag forwarding to a command word lost that position at a
shell keyword and across a wrapper's own operands, so `if true; then find
. -exec rm ...` and the `env -u FOO find ...` and `timeout 5 find ...`
shapes stopped blocking rm entirely. Keywords now keep the position and
wrapper options and their operands are stepped over, the way the command
walk already does.
Reading any operator-shaped token as a separator did the opposite: a
QUOTED one is data the command receives, so `printf '%s' '|&' rm` and
`grep '|&' rm file` were refused although they run nothing. The walk now
applies the same quoted-index exclusion the layout pass does, which also
clears the older `printf '%s' ';' rm` false positive.
ANSI-C decoding flattened the word's whitespace, and a sed program ends
its comment at exactly the newline that flattening destroyed. The decoded
text is re-quoted instead, keeping the spaces and the `#` around it, with
the newline standing as a mark so it stays data for whatever command
receives it rather than a place a new one begins.
An assignment inside a function body has not run and may never run, so it
is no longer recorded as the current value; the name is cleared instead,
which is right whether or not the function is later called.
An `-f` taking a process substitution is a generated /dev/fd/N script, and
the lexer ends the invocation at the `(` before the operand is read at
all. A still-pending program operand now fails the sed closed.
Live expansions were compared against the raw command spelling while the
sed program carried the post-lex one, so an escaped expansion read as
already resolved. Both sides are keyed without their escaping, which can
only make a spelling match and so errs closed.
Prompt rate is unchanged at 45 of 169 realistic invocations.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Read the sed program from the word the shell actually passes
Six from review, four of them bypasses and two false alarms.
find rewrites `{}` with the pathname it found before the child ever starts,
so a sed whose whole program is that placeholder was never read. Nested
under xargs it really runs whatever a suitably named file contains. A `{}`
among the file operands, which is the ordinary idiom, is not the program
and is untouched.
A quoted redirection is a word the command receives rather than something
the shell performs, and it was being removed either way, so a `-f` script
file named `>prog` disappeared and took the `-e` behind it out of view.
Quoting is now read from the operator the token opens with, which leaves
`2>'/dev/null'` a redirection with a quoted target.
An apostrophe in an ANSI-C word sent it down the flattening path, which
destroys the newline a sed comment ends at. The apostrophe is re-quoted
the way a shell does it instead.
fd takes the command attached to its short exec option, and only the exact
`-x` and `-X` spellings opened an action, so `-xrm` reached neither layer.
Conversely nothing behind a bare `--` is an option at all, and reading one
there refused `fd -- -x rm`, which merely lists a file.
The set of live expansions covers the whole command, so matching a sed
program against it by text alone attributed an expansion another command
performs to a program that only spells the same thing. Which occurrence it
was decides it now, and single quoting keeps its meaning while double
quoting does not.
Prompt rate is unchanged at 45 of 169 realistic invocations.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten the comments this PR added
Every comment kept says why a rule exists and, where the reason is a
real tool behaviour, names the one command that proves it. What went is
narration of the code, the history of how each fix evolved, and the same
mechanism re-explained at each site that uses it: it is stated once at
the definition now and referred to from there.
Docstrings on the private helpers give what they return and the one fact
that is not obvious; the worked examples they carried are in the tests,
which already run them. The longest block is 8 lines, from 19.
229 lines off the diff. No code changed.
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
findCodeBlockRegions scanned every region found so far for each inline code
match, and accepted inline spans were appended to the same array, making it
quadratic in the number of inline spans. preprocessLaTeX runs on the full
message text every animation frame while streaming and calls it twice.
Fenced and inline matches are both ascending and non-overlapping, so walk the
fenced list with a cursor instead. Only fenced regions can contain an inline
span, so previously accepted inline regions never needed checking.
34,670 chars with 2,100 inline spans: 5.51ms per call to 0.12ms.
Co-authored-by: shimmyshimmer <info@unsloth.ai>
test_shipping_code_names_an_encoding is red on main. #7373 added
sync_marker_llama_backend, whose read_text/write_text pair does not name
an encoding, so both fall back to locale.getencoding():
AssertionError: 2 text read/write call sites in shipping code let the
operator's locale decide the encoding, so they crash or silently
produce mojibake on Windows. Pass encoding = "utf-8":
['studio/install_llama_prebuilt.py:5656: write_text()',
'studio/install_llama_prebuilt.py:5647: read_text()']
Reproduced on a clean checkout of main at 7917c7828: 1 failed, 7 passed.
That guard landed in #7486 a few commits earlier, so the rule predates
these call sites; nothing about the Vulkan work is wrong beyond the
missing kwarg. The create path that writes the same file, 26 lines above
at 5621, already passes encoding = "utf-8", so main is also internally
inconsistent about one file: written as utf-8, read back under the
operator locale.
Scope, stated honestly: json.dumps defaults to ensure_ascii = True, so
the marker this module writes is pure ASCII and round-trips under cp1252
as well as utf-8. The exposure is a marker produced or edited by
something else. A decode failure on the read would not even surface,
because UnicodeDecodeError subclasses ValueError and the surrounding
except (OSError, ValueError) swallows it into the early return, leaving
the backend silently unsynced. So this restores a green suite and makes
the file self-consistent rather than fixing a live crash.
Verified: tests/test_runtime_text_encoding.py 1 failed / 7 passed before,
8 passed after; tests/test_source_read_encoding.py still passes.
* Studio: keep grouped Python scripts visible and save them natively
* Studio: render the executed Python script outside the card collapsible
Ungrouping the aggregate tool group was not enough on its own. Each Python
card still mounts with defaultOpen={isRunning}, so on a reopened turn the
script and its Copy/Download controls stayed hidden behind the card's own
chevron and the reported issue persisted.
Render ToolCodeCell outside ToolFallbackContent for Python, restoring the
behaviour from #7240 that #7455 folded back inside when it unified the code
cell. Status, output and images still collapse. Terminal keeps its command
inside the collapsible: a one-line command is not the artifact a user reopens
a thread to retrieve, a script is.
Verified against a running Studio: reopening a persisted turn with two
adjacent Python calls now shows both scripts and both Download controls with
no clicks, and Download still saves byte-exact script.py.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
test_tensor_abort_cache_invalidated_on_binary_mtime_change bumped mtime by a
single nanosecond. NTFS stores timestamps as 64-bit FILETIME values in 100ns
ticks, so on Windows that bump rounds away, st_mtime_ns reads back unchanged,
the cache key is identical and the stale abort is inherited, and the assertion
sees True where it wants False.
1ms is still a same-second, sub-second change and is exactly representable, so
the case the test exists to cover actually runs. Skip when the filesystem cannot
record any sub-second change at all rather than asserting product behaviour the
platform cannot exercise.
Not caught before because both jobs in studio-backend-ci.yml are
runs-on: ubuntu-latest, so the studio backend tests only ever run on Linux.
