unsloth/studio/backend
Daniel Han 80d84a5b5f studio: optimize llama-server flags for single-user studio
Refactor command building (deduplicate HF/local paths) and add
flags for better performance:

- --parallel 1: studio is single-user, so only 1 inference slot
  is needed. The previous auto-detect picked 4 slots, wasting
  VRAM on 3 unused KV caches.
- --flash-attn on: force flash attention for faster inference.
  Default is "auto" which may not always enable it.
- --fit on: auto-adjust parameters to fit in available device
  memory. Already the default but now explicit.

Also cleaned up the duplicated command building for HF vs local
mode into a single block.
2026-03-15 05:24:06 -07:00
..
assets Final cleanup 2026-03-12 18:28:04 +00:00
auth fix: remove old comments (#4292) 2026-03-14 16:50:13 +04:00
core studio: optimize llama-server flags for single-user studio 2026-03-15 05:24:06 -07:00
loggers Final cleanup 2026-03-12 18:28:04 +00:00
models studio: don't cap max_tokens for GGUF inference 2026-03-15 05:24:06 -07:00
plugins Final cleanup 2026-03-12 18:28:04 +00:00
requirements fix: remove old comments (#4292) 2026-03-14 16:50:13 +04:00
routes studio: don't cap max_tokens for GGUF inference 2026-03-15 05:24:06 -07:00
state Final cleanup 2026-03-12 18:28:04 +00:00
tests fix(seed): disable remote code execution in seed inspect dataset loads (#4275) 2026-03-13 19:37:43 +04:00
utils fix: prevent ai-assist model config RCE via untrusted Hugging Face repos (#4274) 2026-03-13 19:29:11 +04:00
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
colab.py Final cleanup 2026-03-12 18:28:04 +00:00
main.py fix: remove old comments (#4292) 2026-03-14 16:50:13 +04:00
run.py Final cleanup 2026-03-12 18:28:04 +00:00