Three rename + one substantial test rewrite:
- "tool calling tests" -> "Tool calling Tests"
- "Chat UI smoke (Playwright + Chromium)" -> "Chat UI Tests"
- "install.sh + `unsloth studio update --local`" -> "Studio Updating Tests"
Chat UI Tests was a 4-second pass-through (fill new password, send one
message, reload). Rewrote into a 15-section flow that runs ~30 seconds
locally and exercises the full Studio chat surface a real user touches:
1. Login form (username is hardcoded HIDDEN_LOGIN_USERNAME in
auth-form.tsx, so we only fill #password)
2. Composer mounts after auth
3. Composer toolbar (Send + Add Attachment)
4. Three distinct user turns with non-empty deterministic
assistant replies (verified locally: lengths 6/1/6 for
"hello"/"1"/"world" prompts)
5. Assistant action bar: Copy + Regenerate
6. Settings sheet open + close
7. Theme toggle via account menu (light <-> dark, with a
view-transition wait so the click doesn't race the animation)
8. Sidebar nav: New Chat, switch-back-to-previous-chat (history
persistence via threadId in IndexedDB)
9. Sidebar Search dialog
10. Sidebar collapse/expand
11. Reload + verify session JWT survives (the 2026.5.1 chat-history
regression killed the page entirely on reload; this catches it)
12. Post-reload turn proves inference still works
13. /api/health stays healthy
14. Negative-auth: old bootstrap pw -> 401, rotated pw -> 200
15. Zero pageerror events captured
The CI step that boots Studio + loads the model now rotates the
bootstrap password BEFORE calling /api/inference/load. /api/inference/
load is gated behind must_change_password=false; the previous flow
(login bootstrap -> load) was succeeding in CI by historical accident
and started failing locally. New flow:
bootstrap login -> change-password -> rotated login -> load model
Both passwords are exposed to the Playwright step via env, so the
test can drive /login with the rotated password AND assert the old
one is now 401.
Verified locally end-to-end against a real Studio install with
gemma-3-270m-it-GGUF UD-Q4_K_XL: all 15 sections pass, console.error
count = 0, total runtime ~30s.
Last security-audit run revealed 4 step-level errors hidden by
continue-on-error (the job reported pass but each fix is real):
1. OSV-Scanner curl 404 -> tar exit 2. v2.x ships a raw binary
(`osv-scanner_linux_amd64`), not a tarball. Drop tar -xzf,
curl -o the binary directly + chmod +x.
2. cargo audit `parse error: TOML parse error at line 5 col 8`
on RUSTSEC-2026-0073.md. cargo-audit 0.21 doesn't parse the
CVSS 4.0 schema used in 2026 advisories. Bump pin to ^0.22.
3. TruffleHog `flag 'no-update' cannot be repeated`. The
trufflesecurity/trufflehog action passes --no-update
internally already; remove our duplicate from extra_args.
4. cyclonedx-py `unrecognized arguments: --schema-version 1.6
--outfile ...`. cyclonedx-bom 4.x renamed to `--sv` for spec
version and `-o` for the output file.
Plus pin every remaining mutable-ref action to a 40-char SHA. The
new GHA pinning verifier flagged 4 third-party + 40 first-party
mutable refs; this commit pins all 44 to the latest SHA *within
the existing major version* (no auto-upgrades). Mappings:
actions/checkout @v4 -> 34e114876b... (v4.3.1)
actions/setup-node @v4 -> 49933ea528... (v4.4.0)
actions/setup-python @v5 -> a26af69be9... (v5.6.0)
actions/stale @v10 -> b5d41d4e1d... (v10.2.0)
actions/upload-artifact @v4 -> ea165f8d65... (v4.6.2)
actions/cache @v4 -> 0057852bfa... (v4.3.0)
swatinem/rust-cache @v2 -> 23869a5bd6... (v2.9.1)
dtolnay/rust-toolchain @stable-> 29eef336d9... (stable @ 2026-05-07)
44 pins applied across 11 workflow files. The pin verifier now
reports zero unpinned `uses:`. Dependabot's github-actions
ecosystem (already configured in .github/dependabot.yml) will
auto-bump these SHAs in weekly batches.
This closes the same attack class that hit litellm 1.82.7: an
attacker who hijacks a tag (as in the aquasecurity/trivy-action
March 2026 incident) cannot redirect our workflows because we no
longer follow tag refs.
studio/install_llama_prebuilt.py lists releases on
ggml-org/llama.cpp via the GitHub API. Unauthenticated calls get
60/hr per source IP, which is fine for one install per workflow but
the new Studio Update CI does install + update + update back-to-back
on the same runner, blowing past the limit and falling back to a
source build (which then fails the idempotency assertion).
Surfaced on the Studio Update CI run with:
failed to inspect published releases in ggml-org/llama.cpp:
GitHub API returned 403 ...
set GH_TOKEN or GITHUB_TOKEN to avoid GitHub API rate limits.
GITHUB_TOKEN with the existing `permissions: contents: read` is more
than enough for unauthenticated read API access (1000/hr, scoped to
the repo). Wired into every install.sh and `unsloth studio update`
step across studio-update-smoke.yml, studio-inference-smoke.yml, and
studio-ui-smoke.yml so a busy runner can't trip the same fallback.
Two new PR-time gates that the existing inference / wheel jobs miss.
Studio Update CI:
- Runs install.sh --local --no-torch, then `unsloth studio update
--local` twice, asserting both invocations take the prebuilt
"up to date and validated" code path with no source-build
fallback.
- Boots Studio to /api/health afterwards so a broken update that
nukes the venv or the llama-server binary surfaces immediately.
- Triggers when install.sh, studio/setup.sh, the python_stack /
llama_prebuilt installers, the requirements files, or
unsloth_cli/commands/studio.py change.
Studio UI CI:
- Drives the actual frontend bundle in headless Chromium via
Playwright with the smallest GGUF (gemma-3-270m-it UD-Q4_K_XL).
- Covers: bootstrap login, must_change_password gate + change form,
chat composer becomes interactive after model load, sending a
message produces an assistant bubble with non-empty text, full
page reload re-hydrates the conversation, configuration sheet
opens and closes cleanly, and the rotated password is the only
one that logs in afterwards.
- This is the first workflow that catches the class of bug 2026.5.1
shipped: backend healthy + frontend builds, but assistant-ui
runtime wiring or chat-history persistence broken so the actual
UI was unusable. Backend-only or wheel-only gates do not see it.
