* Measure where Studio's startup time actually goes
Nothing measured this. studio/backend/main.py logs 'lifespan startup completed in
X ms' but no test or CI job ever asserted a budget, a repo-wide grep for
startup_ms or time_to_ready matches only that one file, and studio_test_kit polls
/healthz in a loop that discards the elapsed time it already computes. Its
default healthz_timeout_s of 180 was the only recorded expectation.
scripts/profile_startup.py breaks a launch into phases: import cost via
python -X importtime in a subprocess (top cumulative contributors), process spawn
to first output, and spawn to /healthz 200, over N repeats with median and p90.
First numbers on Linux: importing the backend module costs 5.7 to 6.6 seconds
before the server can even bind, and it dominates everything else. That is eager
module-level imports pulled in by the routes package, not the hardware detection
I first suspected: utils.hardware is 23ms and does not pull torch.
--max-healthz-seconds exists so a budget can be enforced once per-platform
numbers are agreed. It is not wired into a gate yet, deliberately: a threshold
picked before the data is in would either be meaningless or flaky.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Profile the code under test, and let the profile fail
Both installer calls omitted --local, so every phase measured the published PyPI
backend and could not move when a PR edits main.py, run.py or routes. t_first_byte
was a dead local, advertised in the docstring but never returned, and the reader
could deadlock once the child filled the pipe. A failed launch and an impossible
budget both produced a warning and exit 0, and the importtime parse reported the
largest cumulative row, which is site, not main, so a raising import published a
number as success. Pin the controller to the profiled venv's interpreter.
* Stop the startup summary hiding failed launches
The aggregates cover only the runs that reached healthz, so two dead launches
and one fast one rendered as a normal fast startup, and an all-failed phase
printed nothing at all. With continue-on-error and no budget wired, that summary
is the only thing anyone sees. Say how many launches the number is made of, and
say so explicitly when none came up.
* Reject --repeats below 1
range(0) launches nothing, so the empty runs list reached the budget check as
"no healthz measurement", warned and exited 0: a gate that cannot fail. The
value comes straight from a dispatch input, so reject it loudly instead.
* Run the startup profile when the imported startup tree changes
The path filter listed main.py, run.py and routes/**, but the graph the
profiler measures is far wider: main.py imports auth, core, hub, loggers,
models, picker and utils at module scope, and routes/models.py imports
utils.utils and utils.hidden_models. A change to any of those moved
`import main` without ever running this job, so the regressions the
workflow exists to catch went unmeasured.
Cover studio/backend/** (tests excluded) and unsloth_cli/**, since the
launch phase spawns `unsloth studio --api-only` and the CLI is on the
process-to-healthz path.
* Read the labelled main row and kill the Windows launcher tree
total_seconds took by_cum[0], the largest cumulative row in -X importtime
output. That output also carries the interpreter's own startup graph (site,
encodings, whatever a venv sitecustomize pulls in), which is not part of
import main, and the two are not ordered by construction. With a trivial main
the old code reported site's 0.027s as "import main" while main actually cost
0.000249s. Today's backend dwarfs site so the published figures are unchanged,
but the headline number must not silently become another module's cost once
the backend imports get optimized, so read the row named main.
profile_launch spawned Scripts/unsloth.exe on Windows. A pip console-script
.exe is a distlib launcher stub that CreateProcess's the venv python and waits,
so terminate() reaped the stub and left the backend holding the inherited
stdout handle: the reader thread never saw EOF and burned the full 10s join,
and with --repeats each iteration stranded another server on the shared
UNSLOTH_STUDIO_HOME. Walk the tree with taskkill /T, matching the cleanup in
unsloth_cli/commands/start.py and unsloth/dataprep/synthetic.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fail the startup budget when nothing was measured and fall back when taskkill fails
* Trigger on installer inputs and harden the startup gate tests
* Tighten comments in the startup profiler and its workflow
* Trigger the startup profile on the Studio setup scripts
install.sh --local runs the checkout's studio/setup.sh, install.ps1 reaches
studio/setup.ps1 through the editable install, and both call
install_python_stack.py, which decides the dependency set that gets imported.
Editing any of them could change startup time with no measurement taken.
* Shorten the startup profiler comments
Comments and docstrings only.
* Reject non-finite startup budgets and profile when the desktop argv changes
--max-healthz-seconds nan or inf parses as a float but compares False against
any median, so the gate reported success without bounding anything. Require a
finite value.
The profiler hardcodes the argv that process.rs::backend_args builds, but that
file was not in the trigger paths, so a change to the desktop launch command
scheduled no measurement. Add it, and anchor the two argv lists with a test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
bd3972804d