Commit graph

6 commits

Author SHA1 Message Date
danielhanchen
4f501e53e9 Update vulnerable dependencies to patched versions
Clears the safe set of Dependabot advisories.

Frontend (overrides, all transitive; npm audit now 0):
  mermaid 11.14.0->11.15.0, hono 4.12.17->4.12.18, qs 6.15.1->6.15.2,
  ip-address 10.1.0->10.1.1 (also clears express-rate-limit), and
  brace-expansion 5.0.5->5.0.6 (scoped, the 1.1.14 line is untouched).

Desktop: tauri 2.10.3->2.11.1, pulling the runtime crates it requires
(tao, wry, tauri-runtime, tauri-build, tray-icon).

Backend (test-only): pytest <9.0 -> >=9.0.3,<10, and the pinned
pytest-rerunfailures==15.1 -> >=16.2,<17 (16.2 is the first release with
pytest 9 support). Verified pytest 9.0.3 + rerunfailures 16.3 +
pytest-json-report + pytest-xdist run reruns, fixtures, json reports and
xdist together.

Left out intentionally: transformers (stays 4.x for unsloth compat; patch
is 5.0.0rc3), sqlfluff (major 3->4), glib/rand (stack-coupled transitive).
2026-06-03 05:08:00 -07:00
Nilay
9a907a8acb
Studio: add remote MCP server support (#5750)
* added remote MCP server support

* trim

* added tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* increased timeout

* disabling MCP chat toggle

* Fix MCP OpenAI function-name validation + cancel propagation for PR #5750

OpenAI requires function.name to match ^[a-zA-Z0-9_-]{1,64}$ before
streaming starts. The existing 64-char length check is necessary but
not sufficient: MCP servers can return tool names containing '.', '/',
spaces, etc. that would 400 the whole chat request. Validate the
composed mcp__<server_id>__<tool> name against the regex, skip + warn
on miss, and drop duplicate tool names from the same server (which
would also 400 the request as "duplicates").

Also propagate the agentic-loop cancel_event into MCP tool execution
so a /cancel POST during a long-running MCP call (e.g. GitHub MCP
search across a large repo) actually interrupts the in-flight HTTP
call instead of waiting out the 300 s timeout. The watcher polls the
threading.Event at 50 ms cadence inside the asyncio loop (matches
routes/inference.py's existing cancel-watcher cadence) and races
against the call task with asyncio.wait FIRST_COMPLETED.

Tests added:
  - test_mcp_specs_skip_invalid_openai_function_names: drops bad chars
  - test_mcp_specs_skip_empty_tool_name
  - test_mcp_specs_drops_duplicate_names
  - test_call_tool_sync_respects_pre_set_cancel_event

Also fix test_desktop_auth.py's router stub that listed every existing
router but missed mcp_servers_router, so importing main.py fails after
this PR adds it to routes/__init__.py.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* PR #5750 round 2: OAuth cleanup on delete/url-change + mcp_enabled standalone

Round 2 of cross-platform validation surfaced two more P1 findings:

1. OAuth tokens never get cleared. fastmcp keys tokens by MCP URL, not by
   server row, and delete / URL change / use_oauth toggle only updated
   the SQLite row. Re-registering the same URL would silently reuse the
   old account's credentials. Adds clear_oauth_tokens_async() in
   mcp_client.py and calls it from the delete + put route handlers when
   the row had use_oauth=True and either the URL changes or OAuth is
   turned off.

2. mcp_enabled=true was ignored unless the caller also sent
   enable_tools=true. The frontend always sends both together so the UI
   path was fine, but a direct API caller sending only mcp_enabled would
   silently get no MCP tools, which contradicts the field's documented
   "append tools from every enabled MCP server" behavior. Loosens the
   use_tools gate in both the GGUF and safetensors paths so mcp_enabled
   opens the tool loop on its own; when the caller did not also opt
   into built-ins, the built-in list starts empty.

Tests added:
  - test_clear_oauth_tokens_async_no_op_safe
  - test_delete_server_calls_oauth_cleanup_when_oauth_was_on
  - test_delete_server_skips_oauth_cleanup_when_oauth_off
  - test_update_server_clears_oauth_on_url_change
  - test_update_server_clears_oauth_when_oauth_disabled

26 backend MCP tests pass; full studio/backend suite 1710 passed locally.
Cross-platform CI (Linux, macOS, Windows) green on staging fork.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* PR #5750 round 3: reject null bool updates + /test surfaces 400

Round 3 of cross-platform validation:

1. PUT /api/mcp/servers/<id> would 500 with TypeError when the body
   explicitly set is_enabled or use_oauth to null. Pydantic accepts
   None for an Optional[bool] and _changes_from_payload then passed
   None into mcp_servers_db.update_server, which int(None)d. Reject
   explicit null at the validation layer with 400 instead.

2. POST /api/mcp/servers/test caught HTTPException under
   "except Exception", so an invalid URL came back as HTTP 200 with
   {"ok": false, "error": "400: ..."} instead of a real 400. The
   create + update paths return 400 for the same input. Move
   validation outside the transport try/except so it surfaces 400.

Tests added:
  - test_changes_from_payload_rejects_null_is_enabled
  - test_changes_from_payload_rejects_null_use_oauth
  - test_test_endpoint_surfaces_url_validation_as_400

* PR #5750 round 4: hyphenated MCP tool names + empty-tool-list gate

Round 4 surfaces two more interaction bugs between the new MCP path
and existing safetensors tool plumbing:

1. OpenAI accepts ^[a-zA-Z0-9_-]{1,64}$ for function.name, and round 1
   widened the MCP regex to that set, so MCP tools can now be advertised
   as `mcp__srv__list-issues`. But the XML tool-call parser in
   tool_call_parser.py used `\w+` (no hyphen), so the model could call
   the tool but Studio could not parse the call. Same in
   routes/inference.py's `_TOOL_XML_RE` stripper, which would leave
   hyphenated tool-call XML in the visible content. Both regexes now
   use `[\w-]+`.

2. safetensors_agentic treats `tools=[]` as "allow all" (documented
   contract, exercised by test_empty_tools_list_does_not_enforce_allowlist).
   When a caller sends `enable_tools=true` + `enabled_tools=[]` +
   `mcp_enabled=true` and MCP discovery returns 0, the resolved tool
   list is genuinely empty and built-in tools (web_search / python /
   terminal) could execute via the model's emitted call. Fix at the
   route gate instead of breaking the documented contract: set
   `use_tools=False` when the resolved list is empty, in both GGUF and
   safetensors paths. Existing callers who omit `enabled_tools` still
   get ALL_TOOLS and are unaffected.

Tests added (32 total):
  - test_tool_xml_parser_handles_hyphenated_function_names
  - test_tool_xml_strip_handles_hyphenated_function_names
  - test_safetensors_agentic_empty_allowlist_still_means_allow_all
    (documents the contract round 4 preserved)

1716 passed locally; cross-platform CI on staging fork still green.

* PR #5750 round 5: GGUF allow-list + CLI policy + hyphenated params + cancel race

Round 5 of parallel-reviewer aggregation surfaced six additional
findings; five are real and fixed here:

1. Hyphenated MCP parameter names (`<parameter=issue-number>`) were
   dropped by the XML parser's `\w+` regex. Extended to `[\w-]+` in
   both core/inference/tool_call_parser.py and core/tool_healing.py.
   The latter is GGUF's own copy of the parser/strip patterns and was
   missed by round 4.

2. core/tool_healing.py's `strip_tool_call_markup` still used
   `<function=\w+>` so hyphenated MCP tool-call XML leaked into the
   GGUF visible content even after round 4 fixed the shared parser.

3+4. `mcp_enabled` re-opened the tool loop even when the operator
   passed `unsloth run --disable-tools` (CLI policy False). Round 2's
   `(_tools_on or payload.mcp_enabled)` gate ignored the raw process
   policy. Now reads `state.tool_policy.get_tool_policy()` and gates
   mcp_enabled on `_cli_policy is not False`. Applied to both GGUF
   and safetensors paths.

5. GGUF's agentic loop called `execute_tool(tool_name, ...)` without
   checking the model-emitted name against the per-request tool list,
   while the safetensors loop already enforces this. Added the same
   allow-list check so a model that hallucinates a filtered MCP name
   or a built-in the caller opted out of returns "not enabled" instead
   of executing.

Bonus P2 fixes:
  - `call_tool_sync` now checks `cancel_event.is_set()` BEFORE
    creating the call task, so a pre-set cancellation does not open
    the HTTP transport.
  - `clear_oauth_tokens_async` moved the OAuth import + construction
    inside the protected try block; a fastmcp.client.auth load error
    used to escape and 500 the delete / update route.

NOT fixed (verified false or out of scope):
  - finding #10 "structured_content vs structuredContent": fastmcp's
    CallToolResult dataclass uses snake_case (verified live against
    structured-only tool result; fields are
    `dict_keys(['content', 'structured_content', 'meta', 'data', 'is_error'])`).
  - finding #11 "asyncio.run from running loop": call_tool_sync is
    invoked from `asyncio.to_thread` worker threads which have no
    event loop; asyncio.run() is safe there.

Tests added (37 total): hyphenated param names, tool_healing strip,
GGUF allow-list gate, cancel pre-set short-circuit, OAuth cleanup
constructor-error swallowing. 1721 passed locally, no regressions.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-05-27 07:01:11 -07:00
Manan Shah
72b768e0be
Fixing Qwen3.5 bug and adding Outetts dependencies (#4459)
* Fixing Qwen3.5 bug and adding Outetts dependencies

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Apply suggestion from @danielhanchen

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-03-19 01:52:07 -07:00
Daniel Han
3a28446a54
Trim ~255 MB of unused packages from Studio setup (#4395)
* Comment out large unused packages from Studio setup requirements

Audited all packages installed by `unsloth studio setup` against actual
imports in unsloth, unsloth_zoo, and studio/backend. The following have
zero imports anywhere and are the largest offenders by disk size:

- gradio (148 MB) in studio.txt -- Studio uses React + FastAPI, not Gradio
- executorch (41.5 MB) in extras-no-deps.txt -- no imports found
- scikit-learn (31.8 MB) in extras.txt -- no imports found
- MeCab (19.9 MB) in extras.txt -- Japanese tokenizer, no imports found
- coremltools (10.2 MB) in extras.txt -- Apple CoreML, no imports found
- uroman (4.0 MB) in extras.txt -- romanization tool, no imports found

Total savings: ~255 MB (~32% of the 805 MB installed by setup).

Each line is commented out with the package size annotated so they can be
re-enabled easily if needed in the future.

* Restore scikit-learn -- needed by sentence_transformers

sentence_transformers is installed with --no-deps in extras-no-deps.txt,
so its sklearn dependency is not auto-resolved. Multiple modules in
sentence_transformers import sklearn at the top level (evaluation,
util/similarity), so removing scikit-learn would break embedding jobs.
2026-03-17 21:32:38 -07:00
Shine1i
c8c844a4d6 feat: introduce single-env Python dependency management for streamlined compatibility
- Added constrained dependency files for single-env installations: `constraints.txt`, `data-designer.txt`, and `data-designer-deps.txt`.
- Implemented a `patch_metadata.py` script to resolve metadata conflicts between dependency versions.
- Updated `setup.sh` to integrate single-env setup, including dependency installation and metadata patching.
- Upgraded `fastmcp` and `websockets` versions in `extras.txt` for compatibility.
- Commented out unused "Start Tutorial" button in `data-recipes-page.tsx`.
2026-02-24 07:45:40 +01:00
Roland Tannous
39b072d2ee move requirements/ to studio/backend/ and update paths in setup.sh 2026-02-17 19:02:25 +00:00
Renamed from requirements/extras.txt (Browse further)