unsloth/studio/backend/core/inference
Daniel Han 0c1127cb08
Studio: Bypass Permissions menu fix, decimal GB sizes, and GLM-5.2 high/max/disabled thinking (#6444)
* Studio: fix Bypass Permissions menu freeze and show decimal GB for model sizes

Bypass Permissions freeze: the warning dialog lived inside the composer
"+"/More dropdown and kept the menu mounted via onSelect preventDefault,
so confirming or cancelling the dialog left both popovers frozen open.
Lift the dialog out of the menu into a store-driven
BypassPermissionsConfirmDialog mounted at a stable spot in the composer.
The menu item now closes normally on select and just toggles a new
bypassConfirmOpen store flag, so the popovers dismiss as expected.

Model search sizes: formatBytes divided bytes by 1024 but labelled the
result "GB", so unsloth/GLM-5.2-GGUF:UD-IQ1_S showed 201.8 GB where
Hugging Face reports 217 GB. Switch the search display to decimal
(base-1000) units to match what Hugging Face reports. The GPU-fit math
stays base-1024 since VRAM capacity is binary.

* Studio: address review feedback and add GLM-5.2 high/max/disabled thinking

Review feedback on the Bypass Permissions and size-format changes:

- Mount the Bypass Permissions warning dialog once at the chat-page root
  instead of inside each Composer. It is driven by global store state, so
  the per-composer mount meant Compare mode (multiple composers) rendered
  duplicate dialogs and the shared-composer menu had none. A single root
  mount fixes both.
- Defer opening the dialog past Radix's menu-close focus restoration with
  setTimeout(0), so the dropdown does not steal focus back and break the
  dialog's focus trap.
- Clamp the unit index in formatBytes so units[i] cannot go out of bounds
  past TB (and to absorb log() float error at exact powers of 1000).

GLM-5.2 reasoning levels:

GLM-5.2's template gates thinking with enable_thinking and also reads a
reasoning_effort level ('high' or 'max'), so it needs high / max /
disabled rather than the binary toggle it got before (its style was
detected as enable_thinking, which made 'high' unreachable). Add a new
reasoning style 'enable_thinking_effort' that reuses the effort dropdown
but, unlike gpt-oss, can be fully disabled:

- detect_reasoning_flags classifies a template that has both
  enable_thinking and reasoning_effort, extracting the discrete levels
  from the quoted effort literals it branches on. Templates with only one
  of the two (gpt-oss, Qwen3, DeepSeek, GLM-4.6) are unchanged.
- _request_reasoning_kwargs maps the new style to enable_thinking plus an
  in-range reasoning_effort; disabling sends enable_thinking=false. The
  gpt-oss reasoning_effort path is left untouched.
- The backend reports reasoning_effort_levels on the load/status response;
  the frontend carries them through to the effort dropdown and sends
  enable_thinking + reasoning_effort for this style.

Verified: backend reasoning kwargs render the real GLM-5.2 template to
"Reasoning Effort: High/Max" (thinking) and an empty <think></think>
(disabled); tsc, eslint, i18n parity and the production build all pass.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: address review feedback on reasoning effort and formatBytes

- chat-adapter localReasoningEffort: accept 'minimal' so a template that
  branches on it (extracted into reasoning_effort_levels) is sent through
  instead of being coerced to 'low' and then dropped by the backend.
- formatBytes: return '0 B' for non-finite / non-positive sizes (missing
  metadata -> NaN, Infinity, negatives) and clamp the unit index lower
  bound to 0, so sub-1-byte values can't produce a negative index.

* Studio: hybrid reasoning none gate and decimal GB in load progress

- _request_reasoning_kwargs: for enable_thinking_effort models, treat a
  raw reasoning_effort='none' (OpenAI 'no reasoning' sentinel) as the
  enable_thinking=false off gate, so a direct API caller can disable
  thinking even without passing enable_thinking. The frontend already
  sends enable_thinking=false; this only affects raw API callers.
- use-chat-model-runtime: the download / 'X of Y GB in memory' load
  progress divided bytes by 1024**3 but labelled GB, so it disagreed with
  the model picker and Hugging Face. Use decimal GB (1e9) to match.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: carry hybrid reasoning levels on all load paths and harden formatBytes

Review follow-ups on the enable_thinking_effort work:

- Every model-load path now copies reasoning_effort_levels and derives
  supportsReasoningOff, via a shared reasoningCapsFromLoad() helper. The
  shared/Compare composer load and the three chat-adapter auto-load paths
  previously set only reasoningStyle, so a GLM-style hybrid model loaded
  through Compare or first-chat auto-load fell back to the default
  low|medium|high and lost its Max / Off controls.
- The local send path clamps the effort to the loaded model's advertised
  levels (clampReasoningEffortToLevels) instead of a hard-coded list. A
  stale "max" carried over from an external provider no longer reaches a
  pure reasoning_effort (gpt-oss) model that only accepts none|low|medium|
  high, where the backend would have dropped it.
- formatBytes divides iteratively instead of via Math.log, which has float
  error at exact powers of 1000 (log(1e12)/log(1000) = 3.9999... would
  label 1 TB as "1000 GB"). Keeps the non-finite/non-positive guard.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-18 10:10:01 -07:00
..
__init__.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
_html_to_md.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
anthropic_compat.py Studio: improve OpenAI- and Anthropic-compatible API spec compliance (#6010) 2026-06-09 17:13:25 +02:00
api_monitor.py Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
audio_codecs.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
chat_template_helpers.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
chat_templates.py Studio: bundle Gemma 4 chat templates (E2B/E4B + larger) and auto-apply to unsloth/gemma-4-*-GGUF (#6245) 2026-06-12 05:49:39 -07:00
defaults.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
external_provider.py Studio: ignore unsupported env proxy during Studio startup (#6102) 2026-06-11 05:13:27 -07:00
inference.py Expose runtime context length for hub models (#6154) 2026-06-11 22:13:53 +03:00
key_exchange.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
llama_cpp.py Studio: Bypass Permissions menu fix, decimal GB sizes, and GLM-5.2 high/max/disabled thinking (#6444) 2026-06-18 10:10:01 -07:00
llama_http.py Studio: serialize non-streaming responses once and pool the proxy client (#6393) 2026-06-17 22:38:02 -07:00
llama_server_args.py studio: deterministic VRAM auto-fit for GGUF (MTP reserve, compute buffer, total-based budget) (#6312) 2026-06-17 03:10:22 -07:00
llama_stats.py Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
mcp_client.py Studio: enable stdio MCP servers on a loopback bind (#6295) 2026-06-15 03:02:32 +01:00
mcp_config_import.py studio: show MCP "Import config" on the add-server form (#6030) 2026-06-11 16:17:22 +01:00
mlx_inference.py Expose runtime context length for hub models (#6154) 2026-06-11 22:13:53 +03:00
orchestrator.py Reap Studio child processes when the parent dies abnormally (#6425) 2026-06-18 05:51:22 -07:00
pricing.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
providers.py Studio: Add custom provider option to Connections (#6112) 2026-06-12 13:09:35 +02:00
runtime_context.py Expose runtime context length for hub models (#6154) 2026-06-11 22:13:53 +03:00
safetensors_agentic.py Studio: Bypass Permissions (skip confirmation, disable tool sandbox) (#5895) 2026-06-15 04:04:22 -07:00
tensor_fallback.py studio: deterministic VRAM auto-fit for GGUF (MTP reserve, compute buffer, total-based budget) (#6312) 2026-06-17 03:10:22 -07:00
tool_call_parser.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
tool_loop_controller.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
tools.py Rename chat artifacts copy to canvas (#6298) 2026-06-15 13:38:39 +02:00
worker.py Harden model fetching (#6391) 2026-06-18 05:39:52 -07:00