15d70a1d7b* fix: honor --ctx-size and other forwarded args from `unsloth studio run` in Studio's context-fit logic * refactor: extract resolve_requested_ctx as single source of truth The test helper was reimplementing the two-line 'ctx_override = parse_ctx_override(...); requested_ctx = ctx_override if ctx_override is not None else n_ctx' pattern locally, so the test asserted against its own reimplementation rather than production logic. Extract the conditional into resolve_requested_ctx and have both the production caller and the test use it. * fix(studio): honor pass-through cache type flags in KV VRAM estimate Studio's KV cache VRAM estimate computed from the first-class cache_type_kv even when the user passed -ctk/--cache-type-k/-ctv/ --cache-type-v via extras. Those flags reached llama-server fine (last-wins on the CLI) but the pre-launch estimate kept using the default f16 bytes-per-element, so GPU placement decisions could be off when the user lowered cache precision via pass-through. Adds parse_cache_override + resolve_cache_type_kv in llama_server_args.py (mirroring parse_ctx_override / resolve_requested_ctx), wires both into load_model alongside the existing ctx resolution, and adds focused unit tests for the parser + resolver. Follow-up to @rolandtannous review on #5815. --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| backend | ||
| frontend | ||
| src-tauri | ||
| __init__.py | ||
| install_llama_prebuilt.py | ||
| install_python_stack.py | ||
| LICENSE.AGPL-3.0 | ||
| package-lock.json | ||
| package.json | ||
| setup.bat | ||
| setup.ps1 | ||
| setup.sh | ||
| Unsloth_Studio_Colab.ipynb | ||