unsloth/studio/backend/models
Nilay 601155114d
Studio: persistent stdio MCP sessions so server state survives across tool calls (#7080)
* Studio: persistent stdio MCP sessions so server state survives across tool calls

call_tool_sync spawned a fresh stdio subprocess per tool call
(keep_alive=False) and tore it down when the call returned, so any stateful
MCP server lost its state between calls: with @playwright/mcp,
browser_navigate opened the page in one subprocess and
browser_take_screenshot ran in a brand-new one, screenshotting about:blank.

Keep one connected client per (command, env) on a dedicated event-loop
thread and reuse it across calls:

- idle sessions are reaped after 5 minutes (in-flight calls excluded) and
  everything closes at exit, preserving the old design's no-orphans property
- a dead subprocess is detected via is_connected() and retried once on a
  fresh session; tool-level errors leave the session alone
- cancel and timeout semantics are unchanged, and a timed-out call does not
  tear the session down
- updating a server's endpoint/env/enabled state or deleting it closes its
  live session
- HTTP/SSE servers stay one-shot per call

* address review feedback

* fix stdio session cleanup

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* address review: per-thread MCP scope, close-during-connect and abort races

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* address review: unblock no-limit calls on close, drain borrowers before close, scope closes to url+env

* don't retry sessions closed by config changes, re-verify server row before caching, keep env secrets out of generation keys

* fail fast on connect errors and make the stdio key-lock wait cancellable

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* quote MCP scope parts so IDs with colons can't collide

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* serialize per-session stdio calls, span one timeout budget across connect and call, hash urls in generation keys

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Harden persistent stdio MCP sessions: crash recovery, concurrency, scoping

- Evict a stdio session on any transport-level (non-ToolError) call failure and
  do not replay it, so a mid-call subprocess crash can no longer poison the scope.
  Never gate liveness on Client.is_connected() (it only reports that a session
  object exists, not that the subprocess is alive); add a version-adaptive
  dead-transport probe that works on fastmcp 3.0.2 and newer.
- Re-check closed/defunct/config and transport liveness after acquiring the call
  lock, and retire a session before releasing the lock, so a queued same-scope
  caller never reuses a session that another caller's timeout already retired.
- Force a ProactorEventLoop on Windows so the stdio transport can always spawn
  subprocesses regardless of the active event-loop policy.
- Scope stdio sessions per conversation: require thread_id to persist, and tag
  the fields so a session_id and a thread_id with the same value cannot collide.
  A session_id alone is project-wide, so it now falls back to a safe one-shot
  session instead of sharing browser/DB/REPL state across conversations.
- Forward thread_id on the Anthropic Messages path.
- Treat timeout=None as unlimited on connect and the key lock (was capped at 60s).
- Bound the session cache (default 32, override via
  UNSLOTH_STUDIO_MAX_STDIO_MCP_SESSIONS) with LRU eviction of idle sessions.
- Run config_check on cache hits, and log a redacted exe#digest label instead of
  the raw command so credentials in argv never reach the logs.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trim the stdio MCP session cache on release and skip close-generation for HTTP servers

Two fixes from review of the persistent stdio session lifecycle:

- Re-enforce the session cap when a session goes idle. A concurrent burst of
  distinct-scope calls can overshoot the cap while every cached session is busy
  (insert-time eviction only reclaims idle sessions), and the overshoot used to
  persist until the 5-minute idle reaper. _release_stdio_session now trims the
  idle overshoot back within the cap, without ever evicting an in-flight call.
- close_stdio_sessions() now no-ops for a specific non-stdio (HTTP/SSE) url.
  Those transports are never cached as stdio sessions, so calling it on every
  HTTP server update or delete used to accrue an unbounded close-generation entry.

Both are covered by regression tests that fail before the change and pass after.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep the live stdio MCP session across a display-name rename

The edit dialog resends url, headers, and use_oauth unchanged whenever a
server is saved, so gating the tool-cache invalidation and stdio session
close on field presence dropped the persistent process on a plain rename
or any no-op edit. Gate on a real value change against the stored row so
only a genuine endpoint, auth, or enable change closes the session.

Regression tests: a rename that resends unchanged url/headers/oauth keeps
the session; a real command change still closes it.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments in the stdio MCP session lifecycle

Collapse a few verbose comments to fewer lines with the wording preserved,
and drop one that restated the clear_oauth_tokens_async docstring. Comments
only; no code change.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-14 02:28:43 -07:00
..
.gitkeep fix: restore models directory files deleted during restructure 2026-02-02 19:36:30 +00:00
__init__.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
auth.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
data_recipe.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
datasets.py Studio fix recipe dataset preview (#6031) 2026-06-09 14:02:00 +02:00
export.py Studio: multi-select export formats, portable FP8/INT8, GGUF LoRA, and source parity (#6767) 2026-07-03 08:25:10 -07:00
inference.py Studio: persistent stdio MCP sessions so server state survives across tool calls (#7080) 2026-07-14 02:28:43 -07:00
mcp_servers.py studio: show MCP "Import config" on the add-server form (#6030) 2026-06-11 16:17:22 +01:00
models.py studio: allow updating HF models through UI (#5388) 2026-07-01 01:54:57 +03:00
providers.py Studio: Add custom provider option to Connections (#6112) 2026-06-12 13:09:35 +02:00
responses.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
training.py (feat) Add project names to studio training runs (#6512) 2026-06-29 16:06:36 +02:00
users.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00