unsloth/studio/backend
Long Yixing 2a6abe2ff5
feat(cli): support MLX distributed inference (#6845)
* feat(cli): detect MLX distributed launch context

* feat(mlx): wire distributed inference backend

* feat(cli): broadcast MLX distributed chat turns

* fix(cli): wait indefinitely for distributed chat turns

* fix(cli): report MLX distributed load errors cleanly

* fix(mlx): route distributed vlm through loader

* fix(cli): detect inline MLX host JSON

* fix(studio): harden distributed object sharing

* fix(studio): select JACCL distributed backend

* fix(cli): abort distributed error paths

* Distinguish real stream errors from model text via GenStreamError in distributed CLI

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fail loud when MLX distributed init returns a singleton group

The worker only reaches this block when distributed was explicitly
requested. A singleton (size 1) group means the launch failed to form a
real group (MLX built without distributed support, or an invalid launch
env/hostfile); silently continuing leaves nonzero ranks looping forever
on share_distributed_object. Raise instead so the surrounding handler
returns a clear load error.

* Tighten MLX distributed inference comments

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-08 03:25:39 -07:00
..
assets Add DeepSeek-V4-Flash-GGUF to Studio with none/high/max reasoning (#6908) 2026-07-07 06:13:43 -07:00
auth Studio: opt-in OpenAI /v1 model auto-switch and idle keep-warm (#6392) 2026-07-01 06:42:23 -07:00
core feat(cli): support MLX distributed inference (#6845) 2026-07-08 03:25:39 -07:00
hub Fix Studio custom folders on Linux external drives (#6799) 2026-07-03 19:10:04 +01:00
loggers Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
models Studio: apply presence_penalty on the safetensors and MLX inference paths (#6923) 2026-07-06 22:24:47 -07:00
plugins Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
requirements Studio: exclude mlx-lm 0.31.3 (broke gemma4/qwen3_5 QK-norm load on Apple Silicon) (#6803) 2026-07-06 19:40:06 -07:00
routes feat(cli): support MLX distributed inference (#6845) 2026-07-08 03:25:39 -07:00
state Studio: shareable per-checkpoint preview links (#6486) 2026-06-24 06:31:53 -07:00
storage Sort chat recents by last activity (#6844) 2026-07-07 17:54:32 +01:00
tests feat(cli): support MLX distributed inference (#6845) 2026-07-08 03:25:39 -07:00
utils Fix case-variant model matching and GGUF cache reuse in unsloth start (#6900) 2026-07-08 02:32:06 -07:00
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
_platform_compat.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
cloudflare_tunnel.py Reap Studio child processes when the parent dies abnormally (#6425) 2026-06-18 05:51:22 -07:00
colab.py Studio Colab: opt-in shareable Cloudflare tunnel link (#6684) 2026-06-26 00:56:23 -07:00
main.py Studio: multi-select export formats, portable FP8/INT8, GGUF LoRA, and source parity (#6767) 2026-07-03 08:25:10 -07:00
run.py Studio: fix torch CUDA undefined-symbol errors from a conflicting LD_LIBRARY_PATH (#6905) 2026-07-06 23:48:23 -07:00
startup_banner.py Fix Windows Studio UTF-8 startup handling (#6614) 2026-07-01 13:47:33 +01:00