e8db1cecff* studio: show the active run's saved config in the Training Progress popover The Training Config popover on the live Training Progress page read the editable form store (useTrainingConfigStore), so it showed stale/static values whenever the form changed after the run started; only the History view read the run's saved config snapshot, which is why re-opening the same run from Recents showed the correct values (#6853). Wire the live view to the same authoritative source History already uses: - Extract History's field mapping into sections/run-config-override.ts (mapRunConfigToOverride) so both views share one mapper over GET /api/train/runs/{id} config. - LiveTrainingView fetches the run record as soon as the job id is known and passes the mapped override to ProgressSection; the fetched config is keyed by job id, and until it loads (or if the fetch fails) the form store remains the fallback. The run record is created at job start, so it is available while the run is live. - ProgressSection prefers configOverride whenever one is present instead of only when isHistorical, so the live override takes effect. Adds a source-level regression test pinning the wiring and the mapper's backend config keys. Fixes #6853 * studio: retry the run-config fetch after the first step, carry the saved method Two review fixes on the live Training Config popover source: 1. The backend creates the run row only on the first progress event, so the fetch issued as soon as the job id appeared commonly 404'd during model/dataset preparation and never retried -- leaving the popover on the form store for the whole run. The effect is now also keyed on firstStepReceived (and skips once resolved for the job), so it re-fetches exactly when the row is guaranteed to exist. 2. The popover's method label and LoRA-row visibility came from viewData.trainingMethod, still read from the editable form store; changing the form (e.g. LoRA -> Full) after starting a run relabeled it and hid its saved LoRA rows. The run-config mapper now derives trainingMethod from the snapshot's training_type/load_in_4bit (via parseBackendTrainingMethod, now exported from the feature index) and the live view prefers it. * studio: fetch the run config on a terminal phase too, not just the first step The live config-popover fetch was keyed on firstStepReceived, which the runtime store sets only when step > 0. A run that fails or completes during preparation (before step 1) creates and finalizes its row from the terminal error/complete event, but neither the job id nor firstStepReceived changed, so the fetch never ran and the popover stayed on the editable form store -- showing the wrong config/method if the form was edited afterward (Configure re-enables on failure). Gate the fetch on a runRowReady signal = firstStepReceived OR a terminal phase (completed/error/stopped), the states in which the backend guarantees the row exists. This also stops the earlier fetch-then-404 churn during preparation and lets the effect depend only on values it reads (no lint suppression needed). * studio: retry the run-config lookup and accept a hydrated step as row-ready Two ways the popover could stay stuck on the editable form store for a whole run: - The backend publishes the progress event that reveals the run before create_run commits, so the first lookup can lose that race and 404. The catch changed neither runRowReady nor fetchedRunConfig, leaving every effect dependency identical, so no further attempt was ever made for that job. The failure path now schedules an explicit retry, bounded and keyed by job id, so a genuinely absent row falls back to the form store instead of polling. - A run recovered through status/metrics polling (SSE unavailable or blocked) has currentStep restored by applyStatus/applyMetrics but never firstStepReceived, and the phase stays training, so the row was treated as not ready even at step > 0. currentStep > 0 is now a readiness signal of its own. * studio: fetch the saved run config as soon as the job id exists start_training() inserts the run row before the pump can consume any event -- deliberately, so the run appears in history during model loading -- and /status exposes the job id throughout the pre-step phases. Gating the lookup on a first step or a terminal phase therefore held the popover on the editable form store for the whole configuring/loading/downloading window, which on a long model or dataset load is minutes, and indefinitely for a run adopted from another client. The job id is now the entire readiness condition; the existing bounded retry still covers the instant before the insert commits. * Fix Training Config popover fallback for history runs without a saved config; tighten popover comments --------- Co-authored-by: danielhanchen <unslothai@gmail.com>
109 lines
4.5 KiB
Python
109 lines
4.5 KiB
Python
# SPDX-License-Identifier: AGPL-3.0-only
|
|
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
|
|
|
"""Source-level regression guards for the Training Config popover data source
|
|
(#6853).
|
|
|
|
The live Training Progress popover used to read the editable form store
|
|
(useTrainingConfigStore) while a run was active, so it showed stale/static
|
|
values whenever the user touched the form after starting the run; only the
|
|
History view read the run's saved config snapshot. These guards pin the fixed
|
|
wiring: both views feed ProgressSection a config override mapped from
|
|
GET /api/train/runs/{id}, and ProgressSection prefers that override whenever
|
|
one is present -- not only for historical views.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from pathlib import Path
|
|
|
|
_STUDIO_FRONTEND = Path(__file__).resolve().parents[2] / "frontend" / "src" / "features" / "studio"
|
|
|
|
|
|
def _read(rel: str) -> str:
|
|
return (_STUDIO_FRONTEND / rel).read_text(encoding = "utf-8")
|
|
|
|
|
|
def test_progress_section_prefers_override_over_form_store():
|
|
src = _read("sections/progress-section.tsx")
|
|
# Fields key on the override's presence, not isHistorical: a live view passing
|
|
# an override wins over the store; without one, live keeps the store while
|
|
# History shows blanks rather than unrelated live form values.
|
|
assert "const cfg = configOverride ?? (isHistorical ? undefined : config)" in src
|
|
assert "const cfgEpochs = cfg?.epochs" in src
|
|
assert "isHistorical ? configOverride?.epochs" not in src
|
|
|
|
|
|
def test_live_view_fetches_the_active_run_config():
|
|
src = _read("live-training-view.tsx")
|
|
# Live view resolves the run's saved config snapshot by job id...
|
|
assert "getTrainingRun(" in src
|
|
assert "mapRunConfigToOverride(" in src
|
|
# ...and hands it to the popover.
|
|
assert "configOverride={runConfigOverride}" in src
|
|
|
|
|
|
def test_live_view_fetches_as_soon_as_the_job_id_exists():
|
|
# start_training() inserts the run row BEFORE the pump consumes any event, so
|
|
# the saved config is available during configuring/loading/downloading. The
|
|
# job id is therefore the whole readiness condition: gating on a first step
|
|
# or a terminal phase would show the wrong config for the entire pre-step
|
|
# window of a long load, or for a run adopted from another client.
|
|
src = _read("live-training-view.tsx")
|
|
assert "if (!runtime.jobId) {" in src
|
|
assert "[runtime.jobId, fetchedRunConfig, fetchAttempt]" in src
|
|
# No step/phase readiness gate may creep back in.
|
|
assert "runRowReady" not in src
|
|
|
|
|
|
def test_live_view_retries_the_transient_row_miss():
|
|
# start_training() creates the row before the pump, but a lookup racing that
|
|
# commit can still 404. Nothing else in the effect deps changes on failure, so
|
|
# the retry must be explicit and bounded, else a genuinely absent row would
|
|
# poll forever instead of falling back to the form store.
|
|
src = _read("live-training-view.tsx")
|
|
assert "RUN_CONFIG_FETCH_RETRIES" in src
|
|
assert "RUN_CONFIG_FETCH_RETRY_MS" in src
|
|
assert "setFetchAttempt(" in src
|
|
assert "attempts >= RUN_CONFIG_FETCH_RETRIES" in src
|
|
# The budget is keyed by job so a new run always starts fresh.
|
|
assert "fetchAttempt?.jobId === jobId ? fetchAttempt.count : 0" in src
|
|
# The pending retry must be cancelled with the effect.
|
|
assert "clearTimeout(retryTimer)" in src
|
|
|
|
|
|
def test_live_view_prefers_saved_training_method():
|
|
# The method label / LoRA-row visibility must come from the run snapshot,
|
|
# not the editable form (which may have changed since the run started).
|
|
src = _read("live-training-view.tsx")
|
|
assert "runConfigOverride?.trainingMethod ?? config.trainingMethod" in src
|
|
|
|
|
|
def test_history_view_uses_the_shared_mapper():
|
|
src = _read("historical-training-view.tsx")
|
|
# Shared mapper, not a re-inlined field-by-field copy that could drift.
|
|
assert "mapRunConfigToOverride(detail.config)" in src
|
|
assert "num_epochs" not in src
|
|
|
|
|
|
def test_shared_mapper_matches_backend_config_keys():
|
|
src = _read("sections/run-config-override.ts")
|
|
# The mapper reads the run config JSON the backend snapshots at job start;
|
|
# keep the key set pinned so a silent rename breaks loudly here.
|
|
for key in (
|
|
"training_type",
|
|
"load_in_4bit",
|
|
"num_epochs",
|
|
"batch_size",
|
|
"learning_rate",
|
|
"max_steps",
|
|
"max_seq_length",
|
|
"warmup_steps",
|
|
"optim",
|
|
"lora_r",
|
|
"lora_alpha",
|
|
"lora_dropout",
|
|
"use_rslora",
|
|
"use_loftq",
|
|
):
|
|
assert key in src, f"run-config mapper lost backend key {key}"
|