15d70a1d7b* fix: honor --ctx-size and other forwarded args from `unsloth studio run` in Studio's context-fit logic * refactor: extract resolve_requested_ctx as single source of truth The test helper was reimplementing the two-line 'ctx_override = parse_ctx_override(...); requested_ctx = ctx_override if ctx_override is not None else n_ctx' pattern locally, so the test asserted against its own reimplementation rather than production logic. Extract the conditional into resolve_requested_ctx and have both the production caller and the test use it. * fix(studio): honor pass-through cache type flags in KV VRAM estimate Studio's KV cache VRAM estimate computed from the first-class cache_type_kv even when the user passed -ctk/--cache-type-k/-ctv/ --cache-type-v via extras. Those flags reached llama-server fine (last-wins on the CLI) but the pre-launch estimate kept using the default f16 bytes-per-element, so GPU placement decisions could be off when the user lowered cache precision via pass-through. Adds parse_cache_override + resolve_cache_type_kv in llama_server_args.py (mirroring parse_ctx_override / resolve_requested_ctx), wires both into load_model alongside the existing ctx resolution, and adds focused unit tests for the parser + resolver. Follow-up to @rolandtannous review on #5815. --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| _html_to_md.py | ||
| anthropic_compat.py | ||
| audio_codecs.py | ||
| chat_template_helpers.py | ||
| defaults.py | ||
| external_provider.py | ||
| inference.py | ||
| key_exchange.py | ||
| llama_cpp.py | ||
| llama_server_args.py | ||
| mcp_client.py | ||
| mlx_inference.py | ||
| orchestrator.py | ||
| pricing.py | ||
| providers.py | ||
| safetensors_agentic.py | ||
| tool_call_parser.py | ||
| tools.py | ||
| worker.py | ||