From 2843e2a9bbf2ebaf527b6b8029924c10d8790626 Mon Sep 17 00:00:00 2001 From: Daniel Han Date: Fri, 8 May 2026 08:54:50 +0000 Subject: [PATCH] ci(windows): pre-upgrade npm to 11 + Defender exclusions for ~/.unsloth + frontend Side-by-side substep timing (Update CI, same SHA, post cache-revert): Mac Linux Windows install uv 1s 1s 12s uv pip install unsloth 8s 10s 29s Node setup 4s 4s 35s <- winget reinstall frontend build 20s 22s 204s <- 10x slower 9-step uv pip deps 15s 20s 92s <- 5x slower llama.cpp validate 38s 21s 13s ------------------------------------------------- total 96s 93s 400s Two Windows-specific time sinks have nothing to do with the install logic itself; they are runner-environment friction: (1) `setup.ps1` line 1109-1145 requires Node 22.12+ AND npm >=11 (Vite 8 hard requirement). actions/setup-node@v4 with `node-version: '22'` lands Node 22.22.2 + the npm 10.9.7 it bundles, so the npm check fails and setup.ps1 falls into the "winget install Node.js LTS" branch (~35 s) for a Node reinstall we do not actually need. `npm install -g npm@^11` upgrades the bundled npm in-place in ~5 s, which lets setup.ps1 short-circuit on the existing Node 22. (2) windows-latest's Windows Defender real-time scanning opens and hashes every file the install writes. Vite/Tailwind/TSC produce thousands of small chunks during the frontend build, and uv pip extracts thousands of small files per wheel. The scan latency dominates both. Adding Add-MpPreference -ExclusionPath entries for the four directories Studio writes to drops per-file open latency from ~ms to ~us. The runneradmin user has the privilege needed; wrap each call in try/catch so a permission flake leaves the install otherwise unaffected. Excluded paths: $env:USERPROFILE\.unsloth (Studio venv + llama.cpp) $env:USERPROFILE\AppData\Local\uv (uv wheel cache + extracts) $env:GITHUB_WORKSPACE\studio\frontend\node_modules $env:GITHUB_WORKSPACE\studio\frontend\dist Six Windows jobs touched (4 workflows, with the inference workflow fanning out to 3 jobs): studio-windows-update-smoke.yml (1 job) studio-windows-api-smoke.yml (1 job) studio-windows-ui-smoke.yml (1 job) studio-windows-inference-smoke.yml (3 jobs: openai-anthropic, tool-calling, json-images) The new "Pre-install Windows tweaks" step is identical across every Windows job; the rationale is described once in studio-windows-update-smoke.yml and cross-referenced from the others. Expected savings per Windows job: - npm fix: ~35 s saved (winget Node reinstall skipped) - Defender exclusions: ~30-90 s saved (frontend / uv-pip-extract) - Combined: ~60-120 s per job, or ~6-12 min CPU per PR push across all 6 Windows jobs. Not addressed (out of scope for this commit): - The fundamental Vite/TSC/Tailwind frontend build cost on NTFS. Optimising that would mean changing the build pipeline (e.g. skipping `tsc -b` and relying on type-check elsewhere), which is much more invasive. - The uv pip extraction cost. The actions/setup-python@v5 cache already caches pip wheels; uv has its own cache that we could cache separately, but the cache restore overhead on Windows (76 s for the venv we tried and reverted) tends to eat the savings -- the Defender exclusion above goes after the same cost via a different lever. --- .../workflows/studio-windows-api-smoke.yml | 26 +++++++++++ .github/workflows/studio-windows-ui-smoke.yml | 26 +++++++++++ .../workflows/studio-windows-update-smoke.yml | 45 +++++++++++++++++++ 3 files changed, 97 insertions(+) diff --git a/.github/workflows/studio-windows-api-smoke.yml b/.github/workflows/studio-windows-api-smoke.yml index 0208b0f5d8..429627cc7d 100644 --- a/.github/workflows/studio-windows-api-smoke.yml +++ b/.github/workflows/studio-windows-api-smoke.yml @@ -78,6 +78,32 @@ jobs: HF_HUB_ENABLE_HF_TRANSFER=1 \ hf download "$GGUF_REPO" "$GGUF_FILE" + - name: Pre-install Windows tweaks (npm 11 + Defender exclusions) + shell: pwsh + # See studio-windows-update-smoke.yml for the full rationale. + # tl;dr: setup.ps1 needs npm >=11 to skip a 35 s winget Node + # reinstall, and Defender's real-time scan dominates the + # frontend / uv-pip-extract steps. + run: | + $ProgressPreference = 'SilentlyContinue' + Write-Host "npm version before upgrade: $(npm -v)" + npm install -g 'npm@^11' 2>&1 | Out-Host + Write-Host "npm version after upgrade: $(npm -v)" + foreach ($p in @( + "$env:USERPROFILE\.unsloth", + "$env:USERPROFILE\AppData\Local\uv", + "$env:GITHUB_WORKSPACE\studio\frontend\node_modules", + "$env:GITHUB_WORKSPACE\studio\frontend\dist" + )) { + try { + if (-not (Test-Path $p)) { New-Item -ItemType Directory -Force -Path $p | Out-Null } + Add-MpPreference -ExclusionPath $p -ErrorAction Stop + Write-Host "Defender exclusion added: $p" + } catch { + Write-Host "Defender exclusion skipped ($($_.Exception.Message)): $p" + } + } + - name: Install Studio (--local, --no-torch) shell: pwsh env: diff --git a/.github/workflows/studio-windows-ui-smoke.yml b/.github/workflows/studio-windows-ui-smoke.yml index 2ceb99fa0c..fc62b2c5a8 100644 --- a/.github/workflows/studio-windows-ui-smoke.yml +++ b/.github/workflows/studio-windows-ui-smoke.yml @@ -87,6 +87,32 @@ jobs: HF_HUB_ENABLE_HF_TRANSFER=1 \ hf download "$GGUF_REPO" "$GGUF_FILE" + - name: Pre-install Windows tweaks (npm 11 + Defender exclusions) + shell: pwsh + # See studio-windows-update-smoke.yml for the full rationale. + # tl;dr: setup.ps1 needs npm >=11 to skip a 35 s winget Node + # reinstall, and Defender's real-time scan dominates the + # frontend / uv-pip-extract steps. + run: | + $ProgressPreference = 'SilentlyContinue' + Write-Host "npm version before upgrade: $(npm -v)" + npm install -g 'npm@^11' 2>&1 | Out-Host + Write-Host "npm version after upgrade: $(npm -v)" + foreach ($p in @( + "$env:USERPROFILE\.unsloth", + "$env:USERPROFILE\AppData\Local\uv", + "$env:GITHUB_WORKSPACE\studio\frontend\node_modules", + "$env:GITHUB_WORKSPACE\studio\frontend\dist" + )) { + try { + if (-not (Test-Path $p)) { New-Item -ItemType Directory -Force -Path $p | Out-Null } + Add-MpPreference -ExclusionPath $p -ErrorAction Stop + Write-Host "Defender exclusion added: $p" + } catch { + Write-Host "Defender exclusion skipped ($($_.Exception.Message)): $p" + } + } + - name: Install Studio (--local, --no-torch) # install.ps1 is the supported Windows installer. install.sh # has no Windows branch (apt-get / brew calls). The PS1 diff --git a/.github/workflows/studio-windows-update-smoke.yml b/.github/workflows/studio-windows-update-smoke.yml index 86957f2b36..209c7a540b 100644 --- a/.github/workflows/studio-windows-update-smoke.yml +++ b/.github/workflows/studio-windows-update-smoke.yml @@ -73,6 +73,51 @@ jobs: # then fatal-errors with "Cache folder path is retrieved # for pip but doesn't exist on disk". + - name: Pre-install Windows tweaks (npm 11 + Defender exclusions) + shell: pwsh + # Two surgical fixes against measured Windows-only install + # waste (vs Mac/Linux on the same SHA): + # + # (1) npm. setup.ps1 line 1109-1145 requires Node 22.12+ (or + # 20.19+ / 23+) AND npm >=11 because Vite 8 needs both. + # actions/setup-node@v4 with `node-version: '22'` lands + # Node 22.22.2 + the npm 10.9.7 it bundles, so the npm + # check fails and setup.ps1 falls through to the + # "winget install Node.js LTS" branch -- a ~35 s reinstall + # of Node we don't need. `npm install -g npm@^11` updates + # the bundled npm in-place in ~5 s, which makes setup.ps1 + # short-circuit on the existing Node. + # + # (2) Defender. windows-latest's real-time scan opens / hashes + # every file Studio writes during install (Vite output = + # thousands of small chunks, uv pip = wheel-extraction = + # thousands of small files). The latency dominates the + # 200 s frontend build and the 90 s deps install. Adding + # ExclusionPath entries for the directories the install + # writes to drops per-file open latency from ~ms to ~us. + # Add-MpPreference needs admin; the runneradmin user has + # it, but wrap in try/catch so a permission flake leaves + # the install otherwise unaffected. + run: | + $ProgressPreference = 'SilentlyContinue' + Write-Host "npm version before upgrade: $(npm -v)" + npm install -g 'npm@^11' 2>&1 | Out-Host + Write-Host "npm version after upgrade: $(npm -v)" + foreach ($p in @( + "$env:USERPROFILE\.unsloth", + "$env:USERPROFILE\AppData\Local\uv", + "$env:GITHUB_WORKSPACE\studio\frontend\node_modules", + "$env:GITHUB_WORKSPACE\studio\frontend\dist" + )) { + try { + if (-not (Test-Path $p)) { New-Item -ItemType Directory -Force -Path $p | Out-Null } + Add-MpPreference -ExclusionPath $p -ErrorAction Stop + Write-Host "Defender exclusion added: $p" + } catch { + Write-Host "Defender exclusion skipped ($($_.Exception.Message)): $p" + } + } + - name: Install Studio (--local, --no-torch) shell: pwsh env: