Studio: track requested n_ctx so Auto-slider flips trigger a reload

Review feedback on PR #5427 from gemini-code-assist.

The original short-circuit compared ``request.max_seq_length`` against
``llama_backend.context_length`` (the effective context). VRAM-fit
logic can cap the running server below what the caller asked for, so
this comparison incorrectly returns ``already_loaded`` when the user
flips the slider from an explicit length (e.g. 8192) back to "Auto"
(0): the explicit request was capped to, say, 4096, and the new "Auto"
request reads ``backend.context_length == 4096`` and decides nothing
changed.

Track the originally requested ``n_ctx`` on the backend instead and
compare against that. ``requested_n_ctx == 0`` means the last load
asked for the model's native length; ``request.max_seq_length == 0``
matches it.

Verified in the sandbox suite (now 90 tests):

- ``test_explicit_to_auto_triggers_reload`` -- loaded with explicit
  8192, then Apply with ``max_seq_length=0`` falls through to a real
  reload and the new server runs at the native 40960.
- ``test_auto_to_explicit_triggers_reload`` -- inverse direction.
- ``test_explicit_to_same_explicit_short_circuits`` -- re-Apply with
  the same explicit value still short-circuits (no needless reload).
- Existing scenarios (kv change, spec change, template change, extra
  args inherit, parallel-load stress, frontend Apply flow) unchanged.

``pytest studio/backend/tests`` still green on the same set of tests;
the pre-existing ``test_help_output`` failure and ``test_studio_api``
fixture errors are unaffected.
This commit is contained in:
Daniel Han 2026-05-15 02:15:50 +00:00
commit dd0b1d58a7
2 changed files with 36 additions and 4 deletions

View file

@ -486,6 +486,15 @@ class LlamaCppBackend:
# round-trip those flags) can inherit the values the CLI / first
# load supplied. See issue #5401.
self._extra_args: Optional[List[str]] = None
# The n_ctx the most recent load_model() was *invoked with* (NOT
# the effective context the server is running at -- that one
# lives in ``_effective_context_length`` and may have been capped
# by VRAM-fit logic). The route layer compares against this so an
# Apply that flips the slider between "Auto" (0) and an explicit
# length is detected as a settings change even when the running
# server happens to be at the same effective context. See issue
# #5401 and the gemini-code-assist review on PR #5427.
self._requested_n_ctx: int = 0
self._stdout_lines: list[str] = []
self._stdout_thread: Optional[threading.Thread] = None
self._cancel_event = threading.Event()
@ -533,6 +542,21 @@ class LlamaCppBackend:
"""
return list(self._extra_args) if self._extra_args is not None else None
@property
def requested_n_ctx(self) -> int:
"""The ``n_ctx`` value the last ``load_model()`` was *invoked
with* (not the effective context the server is running at).
Returned as an int -- 0 means the caller asked for the model's
native length. The route layer compares an incoming
``request.max_seq_length`` against this so a slider flip from
explicit (e.g. 8192) to "Auto" (0) is detected as a real
settings change even when the running server's effective context
happens to match the explicit value (because VRAM-fit may have
capped it). See issue #5401.
"""
return self._requested_n_ctx
@property
def context_length(self) -> Optional[int]:
"""Return the effective context length the server is running at."""
@ -2033,6 +2057,11 @@ class LlamaCppBackend:
# See issue #5401.
if extra_args is not None:
self._extra_args = list(extra_args)
# Track the requested n_ctx so the route comparator can
# distinguish "user picked Auto" (0) from "user picked an
# explicit length that happens to equal the running effective
# context". See requested_n_ctx property docstring.
self._requested_n_ctx = int(n_ctx)
self._cancel_event.clear()

View file

@ -423,10 +423,13 @@ def _request_matches_loaded_settings(
runtime setting falls through to a real reload instead of silently
returning ``status="already_loaded"`` (issue #5401).
"""
# max_seq_length == 0 means "use model default" -- treat as a match
# against the running effective context length.
backend_ctx = llama_backend.context_length or 0
if request.max_seq_length and request.max_seq_length != backend_ctx:
# Compare against the *requested* n_ctx, not the effective context.
# The effective context can be lower than the request if VRAM-fit
# capped it, so comparing against the effective value would
# incorrectly call an explicit-to-Auto slider flip a "match".
# ``requested_n_ctx == 0`` means the last load asked for the model's
# native length; an incoming ``max_seq_length == 0`` matches that.
if request.max_seq_length != llama_backend.requested_n_ctx:
return False
if _normalise_settings_str(request.cache_type_kv) != _normalise_settings_str(
llama_backend.cache_type_kv