unsloth/.github/workflows/studio-windows-update-smoke.yml
Daniel Han 9f39cc2c39
Studio: use an isolated Node.js for the frontend build instead of replacing the system Node/npm (#6533)
* Studio: use an isolated Node.js for the frontend build instead of replacing the system Node/npm

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: address Node isolation review (no-Node probe crash, PATH refresh, OXC provisioning, venv python, runtime node resolver)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix/adjust Node isolation for PR #6533

* Studio Node: don't cache a negative node resolution; accept Node metadata in setup.sh ownership guard

- node_runtime: memoize only a version-adequate executable so a Node installed
  by a separate-process 'studio update' is picked up without a backend restart.
- setup.sh: _studio_owned_adoptable also accepts UNSLOTH_NODE_PREBUILT_INFO.json,
  matching the setup.ps1 Node ownership guard (custom-home parity).

* Studio setup.ps1: skip OXC npm install gracefully when npm is absent

Mirror setup.sh's `command -v npm` guard so a pip-installed Studio with no
system Node skips the OXC runtime install (validator degrades at runtime) instead
of exit 1 aborting the whole setup. Tighten test_node_probe_guard.ps1's probe
regex so it only matches the two system-version probes, not this new npm guard.

* Wire test_node_probe_guard.ps1 into Windows CI for PR #6533

* Harden isolated Node install and probes for PR #6533

- install_node_prebuilt.py: keep an existing, still-usable isolated Node
  when nodejs.org's dist index is unreachable instead of aborting the
  update on a transient outage (existing_install_usable + tolerant fetch).
- install_node_prebuilt.py: pin NPM_CONFIG_PREFIX/npm_config_prefix and
  drop NODE_PATH in _run_node so any npm -g stays inside the isolated
  prefix; Windows npm otherwise writes to %APPDATA%\npm.
- install_node_prebuilt.py: resolve tar hard-link targets against the
  archive root (symlink targets stay link-parent relative).
- setup.ps1: wrap the system node/npm probes in try/catch so a present
  but broken shim degrades to the bundled Node instead of aborting setup.
- setup.ps1: run the isolated Node install with the handed-off/venv Python
  (ReusedSetupPython); the main resolver runs later and bare python may be
  a Store stub this early.
- setup.sh: log when the OXC validator runtime is skipped for missing npm,
  matching setup.ps1.
- node_runtime.py: move the version-floor comment onto _version_meets_floor.
- Tests for the offline-reuse and broken-shim paths.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trim verbose comments across the Studio Node installer for PR #6533

Comments-only pass: collapse the multi-line section banners to single lines,
drop comments that restate obvious code, and tighten the remaining docstrings
and "why" notes without losing intent. No code changes (verified with an AST
comment-only check on the Python files and a non-comment-diff scan on setup.sh
and setup.ps1). Net 109 fewer lines; the install, decision, and probe-guard
suites stay green.

* Harden Node install from review: validated Python, version floor, legacy home, lock race

For PR #6533, addressing the latest review pass:

- setup.ps1: run the isolated Node install with the validated reused/venv Python.
  An incompatible reused interpreter (old venv, conda, stale UNSLOTH_SETUP_PYTHON)
  is no longer used; fall back to the resolved python instead.
- setup.ps1: a STUDIO_HOME/UNSLOTH_STUDIO_HOME override equal to the legacy default
  now uses the legacy sibling node dir (~/.unsloth/node), matching the runtime
  resolver and setup.sh, so OXC can find the Node it installed.
- install_node_prebuilt.py: reject an explicit --node-version below the floor
  (^20.19 || >=22.12 || >=23) instead of installing a Node the build cannot use.
- install_node_prebuilt.py: atomically rename a stale install lock before unlinking
  so two concurrent runs without filelock cannot both acquire it.

Tests added for the version floor (parametrized + explicit-below-floor rejection).
Full install suite: 937 passed, 1 skipped; setup.ps1 parses; decision tests green.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address latest review: armv7l + later-fetch offline reuse for PR #6533

- install_node_prebuilt.py: reject 32-bit ARM (armv7l) up front. Node 24 LTS
  ships no linux-armv7l build, so the old path failed late with a confusing
  "no sha256"; it now fails fast with a clear unsupported-architecture error.
- install_node_prebuilt.py: extend the offline-reuse fallback to the SHASUMS and
  archive fetches. If index.json resolves a newer Node but a later download fails
  and a usable isolated Node is already on disk, keep it instead of aborting a
  non-force update.

Tests added: armv7l/armhf are unsupported; a SHASUMS failure keeps an existing
usable Node and re-raises when none is present. Full install suite: 941 passed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add UNSLOTH_STUDIO_HOME node-dir tests (install side + resolver) for PR #6533

* Add regression tests pinning the reuse path read-only and isolating installer writes

Lock in the two invariants behind the isolated-Node design: reusing a good
system Node never mutates the user's Node/npm, and the installer's own npm
calls only ever write inside its install_dir.

- tests/studio/install/test_install_node_prebuilt_logic.py: assert _run_node
  redirects NPM_CONFIG_PREFIX/npm_config_prefix into install_dir and drops an
  inherited NODE_PATH; assert _ensure_npm_floor scopes the npm self-upgrade to
  install_dir (never -g against the system) and is a no-op once npm meets the floor.
- tests/sh/test_system_node_readonly.sh (new, wired into studio-backend-ci.yml):
  the setup.sh NODE_SOURCE=system arm runs no global install and sets no
  NPM_CONFIG_PREFIX, with a positive control that the bundled arm does.
- tests/studio/test_node_decision.ps1: symmetric structural guard that the prefix
  pin and the only global install (bun) live in the bundled branch, not the system arm.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
2026-06-21 21:17:29 -07:00

292 lines
12 KiB
YAML

# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
# Windows counterpart to studio-update-smoke.yml /
# studio-mac-update-smoke.yml. Verifies that on the FREE
# windows-latest runner:
#
# 1. install.ps1 --local --no-torch installs Studio AND auto-fetches
# the prebuilt llama.cpp Windows binary (llama-bNNNN-bin-win-cpu-
# x64 from ggml-org/llama.cpp). Hitting the source-build fallback
# is treated as an Unsloth bug -- Studio must always pick the
# prebuilt on Windows.
# 2. unsloth studio update --local is idempotent. Two consecutive
# runs both report "prebuilt up to date and validated", no
# source-build fallback. The CLI's _find_setup_script picks
# setup.ps1 on Windows automatically.
# 3. The installed Studio still boots and /api/health returns
# healthy after the update path.
name: Windows Studio Update CI
on:
pull_request:
paths:
- 'install.ps1'
- 'scripts/uninstall.ps1'
- 'studio/setup.ps1'
- 'studio/setup.bat'
- 'studio/install_python_stack.py'
- 'studio/install_llama_prebuilt.py'
- 'studio/backend/requirements/**'
- 'unsloth_cli/commands/studio.py'
- 'pyproject.toml'
- '.github/workflows/studio-windows-update-smoke.yml'
push:
branches: [main, pip]
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
update-idempotency:
name: Studio Updating Tests
runs-on: windows-latest
timeout-minutes: 30
defaults:
run:
shell: bash
env:
# Force UTF-8 for stdio (Windows defaults to cp1252; hf
# download / Studio CLI print "✓" checkmarks and crash
# otherwise).
PYTHONIOENCODING: utf-8
PYTHONUTF8: '1'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.12'
# Don't cache pip: install.ps1 + setup.ps1 go through uv
# and never populate ~/.cache/pip; setup-python's post-step
# then fatal-errors with "Cache folder path is retrieved
# for pip but doesn't exist on disk".
- name: Pre-install Windows tweaks (npm 11 + Defender exclusions)
shell: pwsh
# Two surgical fixes against measured Windows-only install
# waste (vs Mac/Linux on the same SHA):
#
# (1) npm. setup.ps1's Get-NodeDecision requires Node 22.12+
# (or 20.19+ / 23+) AND npm >=11 because Vite 8 needs both.
# actions/setup-node@v4 with `node-version: '22'` lands
# Node 22.22.2 + the npm 10.9.7 it bundles, so the decision
# is "bundled" and setup.ps1 downloads an isolated Node (~30
# MB) we don't need on a runner that already has a fine Node.
# `npm install -g npm@^11` updates the runner's npm in-place
# in ~5 s, flipping the decision to "system" so setup.ps1
# reuses the existing Node with no download.
#
# (2) Defender. windows-latest's real-time scan opens / hashes
# every file Studio writes during install (Vite output =
# thousands of small chunks, uv pip = wheel-extraction =
# thousands of small files). The latency dominates the
# 200 s frontend build and the 90 s deps install. Adding
# ExclusionPath entries for the directories the install
# writes to drops per-file open latency from ~ms to ~us.
# Add-MpPreference needs admin; the runneradmin user has
# it, but wrap in try/catch so a permission flake leaves
# the install otherwise unaffected.
run: |
$ProgressPreference = 'SilentlyContinue'
Write-Host "npm version before upgrade: $(npm -v)"
npm install -g 'npm@^11' 2>&1 | Out-Host
Write-Host "npm version after upgrade: $(npm -v)"
# NOTE: do NOT pre-create these directories before adding the
# exclusion -- creating an empty studio/frontend/dist trips
# setup.ps1 line 1281-1296's mtime-based "is the frontend
# stale?" check into "up to date, skip rebuild", because the
# newly-created dist's mtime is younger than every source
# file. Studio then boots with an empty dist and 500s on
# GET / with FileNotFoundError: dist\index.html. See run
# 25546676715 / job 74984469728.
# Add-MpPreference accepts paths that do not yet exist; the
# exclusion is registered and applies when the path
# materialises.
foreach ($p in @(
"$env:USERPROFILE\.unsloth",
"$env:USERPROFILE\AppData\Local\uv",
"$env:GITHUB_WORKSPACE\studio\frontend\node_modules",
"$env:GITHUB_WORKSPACE\studio\frontend\dist"
)) {
try {
Add-MpPreference -ExclusionPath $p -ErrorAction Stop
Write-Host "Defender exclusion added: $p"
} catch {
Write-Host "Defender exclusion skipped ($($_.Exception.Message)): $p"
}
}
- name: Install Studio (--local, --no-torch)
shell: pwsh
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
New-Item -ItemType Directory -Force -Path logs | Out-Null
# *>&1 captures Write-Host (Information stream) output;
# plain 2>&1 does not. setup.ps1 emits "prebuilt installed
# and validated" via Write-Host, and we grep for that.
$ProgressPreference = 'SilentlyContinue'
& ./install.ps1 --local --no-torch *>&1 | Tee-Object -FilePath logs/install.log
- name: Assert install.ps1 used the Windows llama.cpp prebuilt
run: |
# Filesystem-based check (setup.ps1's stream output isn't
# captured back through the parent pipeline).
LLAMA_DIR=~/.unsloth/llama.cpp
INFO="$LLAMA_DIR/UNSLOTH_PREBUILT_INFO.json"
BIN="$LLAMA_DIR/build/bin/Release/llama-server.exe"
if grep -q "falling back to source build" logs/install.log; then
echo "::error::install.ps1 fell back to source-build llama.cpp on Windows."
grep -E "llama-prebuilt|llama.cpp" logs/install.log | tail -60
exit 1
fi
if [ ! -f "$INFO" ]; then
echo "::error::no UNSLOTH_PREBUILT_INFO.json at $INFO."
ls -la "$LLAMA_DIR" || true
exit 1
fi
if [ ! -f "$BIN" ]; then
echo "::error::no llama-server.exe at $BIN."
ls -la "$LLAMA_DIR/build/bin" || true
exit 1
fi
echo "install.ps1 installed the Windows prebuilt llama.cpp:"
cat "$INFO"
- name: Add Studio shim to GITHUB_PATH
run: |
SHIM_DIR=~/.unsloth/studio/bin
if [ ! -f "$SHIM_DIR/unsloth.exe" ]; then
echo "::error::unsloth.exe shim not found at $SHIM_DIR"
ls -la ~/.unsloth/studio/ || true
exit 1
fi
cygpath -w "$SHIM_DIR" >> "$GITHUB_PATH"
- name: First update should be a no-op (prebuilt already validated)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
set -o pipefail
unsloth studio update --local 2>&1 | tee logs/update.log
if grep -q "falling back to source build" logs/update.log; then
echo "::error::studio update fell back to source-build llama.cpp on Windows."
grep -E "llama-prebuilt|llama.cpp" logs/update.log | tail -60
exit 1
fi
if ! grep -qE "prebuilt up to date and validated|prebuilt installed and validated" logs/update.log; then
echo "::error::no prebuilt up-to-date marker in update.log."
grep -E "llama-prebuilt|llama.cpp" logs/update.log | tail -60
exit 1
fi
echo "update path took the prebuilt fast path"
- name: Second update must also be a no-op
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
set -o pipefail
unsloth studio update --local 2>&1 | tee logs/update2.log
grep -q "falling back to source build" logs/update2.log && {
echo "::error::second update fell back to source build on Windows"
tail -60 logs/update2.log; exit 1; } || true
grep -qE "prebuilt up to date and validated|prebuilt installed and validated" logs/update2.log
echo "second update was clean"
- name: Boot Studio briefly to confirm the install is still usable
run: |
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p 18891 \
> logs/studio.log 2>&1 &
PID=$!
HEALTHY=""
# Use jq (a Git Bash builtin) instead of `python -c
# open('/tmp/health.json')` to read the saved health
# response. Bash on windows-latest is MSYS Git Bash, which
# resolves `/tmp/...` against the MSYS root, while the
# python interpreter is Windows-native and resolves it
# against the current drive's root. The two paths don't
# agree, so python never finds the file curl just wrote.
# jq reads through MSYS, so the path matches. Mirrors what
# studio-windows-api-smoke.yml and the other Windows smoke
# workflows already do.
for i in $(seq 1 60); do
if curl -fs http://127.0.0.1:18891/api/health > /tmp/health.json; then
if jq -e '.status == "healthy"' /tmp/health.json >/dev/null; then
HEALTHY=1
break
fi
fi
sleep 1
done
if [ -z "$HEALTHY" ]; then
echo "Studio failed to come up after \`update\`"
tail -200 logs/studio.log
kill "$PID" 2>/dev/null || true
exit 1
fi
kill "$PID" 2>/dev/null || true
echo "post-update Studio /api/health OK"
- name: Uninstall and verify clean
# Round-trip through scripts/uninstall.ps1 against the default
# install tree at %USERPROFILE%\.unsloth\studio. Catches
# regressions where install.ps1 starts writing under a new key
# (registry, Start Menu, %APPDATA%) and scripts/uninstall.ps1 has
# not been updated to match. Skips gracefully if
# scripts/uninstall.ps1 has not landed yet (lets this workflow
# merge before #5513).
shell: pwsh
run: |
New-Item -ItemType Directory -Force -Path logs | Out-Null
if (-not (Test-Path "$PWD\scripts\uninstall.ps1")) {
Write-Host "scripts/uninstall.ps1 not present in this tree; skipping round-trip"
"" | Set-Content logs/uninstall.log
exit 0
}
pwsh -NoProfile -File "$PWD\scripts\uninstall.ps1" *>&1 | Tee-Object -FilePath logs/uninstall.log
$leak = 0
foreach ($p in @(
"$env:USERPROFILE\.unsloth\studio",
"$env:USERPROFILE\.unsloth\studio\unsloth_studio",
"$env:USERPROFILE\.unsloth\studio\bin\unsloth.exe"
)) {
if (Test-Path -LiteralPath $p) {
Write-Host "::error::leak: $p"
$leak++
}
}
if ($leak -gt 0) { exit 1 }
# Idempotency.
pwsh -NoProfile -File "$PWD\scripts\uninstall.ps1" *>&1 | Select-Object -Last 5
pwsh -NoProfile -File "$PWD\scripts\uninstall.ps1" *>&1 | Select-Object -Last 5
Write-Host "PASS: windows install -> update -> uninstall round-trip clean"
- name: Upload update logs
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: windows-studio-update-log
path: |
logs/install.log
logs/update.log
logs/update2.log
logs/studio.log
logs/uninstall.log
retention-days: 7