15d70a1d7b* fix: honor --ctx-size and other forwarded args from `unsloth studio run` in Studio's context-fit logic * refactor: extract resolve_requested_ctx as single source of truth The test helper was reimplementing the two-line 'ctx_override = parse_ctx_override(...); requested_ctx = ctx_override if ctx_override is not None else n_ctx' pattern locally, so the test asserted against its own reimplementation rather than production logic. Extract the conditional into resolve_requested_ctx and have both the production caller and the test use it. * fix(studio): honor pass-through cache type flags in KV VRAM estimate Studio's KV cache VRAM estimate computed from the first-class cache_type_kv even when the user passed -ctk/--cache-type-k/-ctv/ --cache-type-v via extras. Those flags reached llama-server fine (last-wins on the CLI) but the pre-launch estimate kept using the default f16 bytes-per-element, so GPU placement decisions could be off when the user lowered cache precision via pass-through. Adds parse_cache_override + resolve_cache_type_kv in llama_server_args.py (mirroring parse_ctx_override / resolve_requested_ctx), wires both into load_model alongside the existing ctx resolution, and adds focused unit tests for the parser + resolver. Follow-up to @rolandtannous review on #5815. --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| assets | ||
| auth | ||
| core | ||
| loggers | ||
| models | ||
| plugins | ||
| requirements | ||
| routes | ||
| state | ||
| storage | ||
| tests | ||
| utils | ||
| __init__.py | ||
| _platform_compat.py | ||
| colab.py | ||
| main.py | ||
| run.py | ||
| startup_banner.py | ||