Studio: improve Deep Research synthesis (#7393)
* Studio: add durable Deep Research workflows * Studio: preserve research integration after upstream updates * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: keep research worker compatible with Python 3.11 * Studio: address Deep Research lifecycle review * Studio: preserve durable research recovery * Studio: preserve research stream and context * Studio: harden research sources and limits * Studio: align research with shared chats * Studio: guard durable research actions * Studio: protect durable research turns * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: deepen durable research decisions * Studio: protect research prompts and queries * Studio: slim research stream deltas * Studio: preserve research evidence and citations * Studio: harden Deep Research (CI, prompt injection, query PII, config, citations) - Fix backend CI: add research_runs_router to the synthetic routes stub in test_desktop_auth so studio.backend.main imports under the health-check test. - Escape prompt-delimiter tags in the decision and synthesis prompts so gathered web/document content cannot close an <untrusted_...> wrapper and inject instructions into the local planner/decision/synthesis model. - Extend the public-query sanitizer to redact Luhn-valid payment cards, phone numbers, non-global IPs, and labeled private identifiers before a query can reach web search. - Reject nested credential keys in inferenceRequest and ragScope, not just top-level keys, when persisting a durable run config. - Treat maxSources as one budget shared across web and document sources (collection and resume paths) instead of per type, which allowed up to 2x the configured cap. - Preserve document citations whose filename contains a closing bracket by tokenizing valid citations before stripping invalid ones. - Persist Deep Research off when switching to an external model and when enabling Web Fetch so a refresh cannot rehydrate a mutually-exclusive state. - Add regression tests for the query, prompt, citation, and config hardening. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: make the research claims table migration atomic The owner-scoped to global claims migration ran its RENAME, CREATE, INSERT and DROP in autocommit, so an interruption after CREATE left the new table empty, orphaned the rows in the legacy table, and never re-triggered. Wrap the rebuild in an explicit transaction so a crash rolls back cleanly and the migration re-runs on the next boot. * Studio: block message edits and regeneration during an active research run After a reload a durable research run is followed by the research store rather than an assistant-ui run, so thread.isRunning is false while research is still active. Message edit, refresh and the edit composer previously gated only on isRunning, which let a normal generation start alongside the running research run. Gate them on the active thread's research state as well. * Studio: keep the plan review mounted through approval Keying PlanReview on planRevision remounted it mid-approve when updateResearchPlan bumped the revision, resetting the local pending flag and re-enabling Start research while the approve was still in flight, which allowed a duplicate approve. Key on runId only. * Studio: drop the redundant deep-research persistence change setCheckpoint already persists Deep Research off for external models at the top of the function, so the added saveBool was a duplicate, and clearing Deep Research from setWebFetchToolsEnabled guarded a state that is not reachable (Deep Research is local-model only while the Web Fetch pill is external-provider only). Revert both to the pre-hardening version. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: harden Deep Research citations, query privacy, and message protection Address review findings in the Deep Research backend: - Escape an unbalanced ")" in citation destinations so a source URL cannot close the markdown link early and inject a second link, keeping balanced parentheses literal. - Match raw-URL citations on whole tokens so a URL sharing another URL's prefix is no longer partially rewritten. - Redact non-global IPv6 addresses in public search queries, matching the existing IPv4 handling. - Detect credential key names after normalizing case and separators so nested openaiApiKey, accessToken, and clientSecret values cannot be persisted. - Reject client edits to server-managed research prompts and reports at the storage layer; only the internal writers pass allow_research_update. - Scope research searches to the first allowed domains instead of dropping site scoping for large allow lists. - Persist the same fetch evidence bound used during live synthesis so a resumed run is not shortened. - Scope run completion so it only replaces this run's message parts. Add regression tests for the above. * Studio: fix Deep Research SSE framing, source counts, and favicon privacy - Normalize the whole SSE buffer so a CRLF split across transport chunks still frames events. - Count web and document sources together in the activity header so a RAG-only run is not shown as zero sources. - Cap the plan editor at the run's configured maxSteps instead of a hard-coded 30. - Add an allowRemoteIcons opt-out to the sources components and disable third-party favicon requests for research sources so visited domains are not leaked. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: address final Deep Research review findings * Studio: fit Deep Research synthesis evidence to loaded context, add opt-in web grounding Size the synthesis evidence budget to the loaded model context so the prompt is not silently truncated on small contexts. When the evidence overflowed the window the report degenerated (it echoed the evidence tail instead of writing); the budget now reserves tokens for the prompt scaffolding and converts the remainder to chars, keeping the full cap when the context is unknown. Add opt-in web grounding for auto-read: read the top search results, ingest them into an ephemeral RAG scope, hybrid-retrieve the passages most relevant to the question with the existing knowledge-base retriever, and fold those chunks into the step evidence. The scope is per call and deleted afterwards, so a user's knowledge base is never touched. Off by default; enable with UNSLOTH_RESEARCH_AUTO_SCRAPE=1. Gated per run by budgets["maxAutoScrape"], so runs created without it keep legacy snippet-only behavior, and grounding is skipped when the loaded context is too small for the prompt. Add tests for the adaptive evidence budget, scraped-text cleaning, the ephemeral web-RAG retrieval and scope cleanup, and the auto-read evidence path. * Studio: read Deep Research synthesis context from the inference orchestrator Make the adaptive synthesis-evidence budget actually engage in the normal Studio architecture. _loaded_context_length read core.inference.inference, the low-level backend that lives in the model subprocess and stays unpopulated in the main web process where the research supervisor runs, so it returned None and the budget silently fell back to the 32000 character cap (leaving the report exposed to the truncation this was meant to fix). Read the inference orchestrator instead, and the llama.cpp backend for GGUF, mirroring routes.inference._monitor_context_length so the budget sizes to the context the API layer serves. Verified on a running server: at a 12288 token load the probe now reports 12288 and the budget adapts to 24576 characters instead of the 32000 fallback. Also: - Reserve context for the generated report as well as the prompt scaffolding (raise the reserve to 4096 tokens) so evidence does not crowd out the output on a small window. - Honor a numeric UNSLOTH_RESEARCH_AUTO_SCRAPE by passing the per-run maxAutoScrape as the page cap to the scraper, instead of always reading the maximum. - Guard the web-RAG connection acquisition so a get_connection failure returns the documented empty result rather than propagating. - Add a synthesis-context test that patches the real backend accessor (not the probe itself) so the production wiring is exercised, plus a scrape page-cap test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: harden Deep Research query redaction and research autosave - research_runs: extend the opaque-token allowlist so unlabeled Hugging Face (hf_) and GitLab (glpat-) tokens are redacted before a query can reach web search, without over-redacting public model or version ids. - runtime-provider: for a server-managed research message, echo the backend-stored metadata verbatim on autosave. Merging the client metadata re-added client-only fields the server never persisted, so the server-side guard saw a diff and rejected every streamed or snapshot update with 409. * Studio: keep composer tool pills always accessible after merge The merge left the composer line marked always-expanded (data-expanded "true") while the inner pill row was still gated behind composerExpanded, so the Search and Code toggles disappeared once the permission mode was "off" with no other toggle set. Render the primary tool pills unconditionally, matching the always-expanded layout, and drop the now unused composerExpanded and permissionMode locals. Fixes the Chat UI Playwright check that asserts the Search and Code pills stay visible. * Studio: update Deep Research composer contract to always-expanded layout The always-expanded composer no longer routes effectiveDeepResearchEnabled through a composerExpanded expression, so the frontend contract now checks that it gates the Deep Research composer button render instead. * Studio: do not bind a research run to a populated assistant reply create_run adopted any assistant message under the user turn whose researchRunId was unset, including a prior answer reused by a retry. On completion _update_assistant drops the untagged text and source parts, so that answer was silently overwritten. Only bind to an empty placeholder or this run's own message, and reject a reply that already carries content. * Studio: harden Deep Research synthesis budget, prompt shielding, and message protection - research_runs: split the synthesis evidence budget evenly across notes so a small context still keeps a slice of every research step instead of dropping the later steps after the earliest ones fill the budget. - research_runs: shield the research question and approved plan before placing them in the decision and synthesis prompts, so a closing delimiter in either cannot escape its block and inject sibling sections. - research_runs: redact bearer authorization tokens from public search queries. - studio_db: include attachments in the research-message change check and guard direct attachment deletion, so server-managed research prompts and responses cannot be mutated through the attachment paths. - chat_history: map the protected-message conflict on attachment deletion to 409. * Studio: strip invalid document citations that contain brackets The invalid-citation regex stopped at the first closing bracket, so a citation whose filename contained brackets left its tail (".pdf, p. 9]") in the report. Match a balanced bracketed span so the whole invalid citation is removed; valid citations stay protected by the earlier tokenization pass. * Studio: free the RAG search slot when a lookup times out or is cancelled The bounded knowledge-base search held the sole admission slot in a detached worker until the search returned, so a lookup that outlived its timeout (a stalled embedding or blocked vector call) kept the slot forever and starved every later lookup, disabling knowledge-base retrieval globally. Release the slot from the caller when it stops waiting, exactly once, so a detached worker finishes without re-holding it. * Studio: remove Websites label from research composer * Studio: fix Deep Research review findings (RAG slot bound, orphaned workers, hardening) - Bound the shared RAG search slot to one running worker. The search that is doing the embedding/index/GPU work now owns the admission slot until it finishes, instead of freeing it on caller timeout while the detached worker keeps running, which let a second search enter and stack concurrent work behind the capacity-of-one semaphore. - Cancel active research runs before deleting their thread, project, or all history. Deleting cascade-drops the run row, but the worker only notices at its next lease check, so it could keep doing model/web/RAG work for a run that no longer exists; signalling cancel first shortens that window. - Shield the planner prompt's conversation and question with _shield_untrusted, matching the decision and synthesis prompts, so untrusted text cannot forge planner delimiters. - Do not let a research key-revocation failure replace a successful non-streaming completion; log it like the streaming path does. - Include created_at in the protected research-message guard so a client cannot reorder server-managed prompt/response messages while leaving the body intact. - Reject non-scalar ragScope values; a nested container evades the sensitive-key scan when its inner keys are unlisted and would reach retrieval code that expects a scalar scope id. Adds regression tests for each. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: remove research composer globe icon * Studio: use Hugeicons telescope in research composer * Studio: use Telescope02 icon in research composer * Studio: standardize Deep Research telescope icons * Studio: move Deep Research below web and code tools * Studio: merge grounded page excerpts with search snippets instead of replacing When auto-scrape grounding retrieved page-body chunks, it replaced the raw search-result text for that step. If the retrieved chunk was a distractor or dropped the key fact, the answer-bearing search snippet was lost and grounded runs regressed below snippet-only accuracy on factual questions (e.g. returning Apache 2.0 instead of the Qwen License, 403 instead of 404, or a single mirror diameter instead of the sum). Keep the search snippets and append the grounded excerpts as supplementary evidence via a small _merge_scraped_evidence helper. Grounding stays opt-in and off by default, so legacy runs are unchanged. Adds regression tests. * Studio: improve Deep Research synthesis * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: harden Deep Research synthesis flow * Studio: validate Deep Research derived context * Studio: align Deep Research synthesis evidence * Studio: restore Deep Research synthesis state * Improve Deep Research source queries --------- Co-authored-by: alkinun <alkinunl@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothai@gmail.com>
This commit is contained in:
oobabooga
GitHub
parent
d7594ec10f
commit
20006dbce7
4 changed files with 722 additions and 42 deletions
|
|
@ -194,9 +194,11 @@ function reduceActivity(
|
|||
const title =
|
||||
phase === "planning"
|
||||
? "Planning an approach"
|
||||
: phase === "synthesis"
|
||||
? "Connecting the findings"
|
||||
: "Choosing the next step";
|
||||
: phase === "synthesis_audit"
|
||||
? "Checking the evidence"
|
||||
: phase === "synthesis" || phase === "synthesis_recovery"
|
||||
? "Connecting the findings"
|
||||
: "Choosing the next step";
|
||||
if (existingIndex >= 0) {
|
||||
const existing = next[existingIndex];
|
||||
next[existingIndex] = {
|
||||
|
|
|
|||
|
|
@ -11,7 +11,13 @@ export type ResearchRunStatus =
|
|||
| "completed"
|
||||
| "failed";
|
||||
|
||||
export type ResearchPhase = "planning" | "decision" | "synthesis" | "unknown";
|
||||
export type ResearchPhase =
|
||||
| "planning"
|
||||
| "decision"
|
||||
| "synthesis_audit"
|
||||
| "synthesis"
|
||||
| "synthesis_recovery"
|
||||
| "unknown";
|
||||
export type ResearchAction = "search" | "fetch";
|
||||
|
||||
export interface ResearchPlanStep {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue