Three rename + one substantial test rewrite:
- "tool calling tests" -> "Tool calling Tests"
- "Chat UI smoke (Playwright + Chromium)" -> "Chat UI Tests"
- "install.sh + `unsloth studio update --local`" -> "Studio Updating Tests"
Chat UI Tests was a 4-second pass-through (fill new password, send one
message, reload). Rewrote into a 15-section flow that runs ~30 seconds
locally and exercises the full Studio chat surface a real user touches:
1. Login form (username is hardcoded HIDDEN_LOGIN_USERNAME in
auth-form.tsx, so we only fill #password)
2. Composer mounts after auth
3. Composer toolbar (Send + Add Attachment)
4. Three distinct user turns with non-empty deterministic
assistant replies (verified locally: lengths 6/1/6 for
"hello"/"1"/"world" prompts)
5. Assistant action bar: Copy + Regenerate
6. Settings sheet open + close
7. Theme toggle via account menu (light <-> dark, with a
view-transition wait so the click doesn't race the animation)
8. Sidebar nav: New Chat, switch-back-to-previous-chat (history
persistence via threadId in IndexedDB)
9. Sidebar Search dialog
10. Sidebar collapse/expand
11. Reload + verify session JWT survives (the 2026.5.1 chat-history
regression killed the page entirely on reload; this catches it)
12. Post-reload turn proves inference still works
13. /api/health stays healthy
14. Negative-auth: old bootstrap pw -> 401, rotated pw -> 200
15. Zero pageerror events captured
The CI step that boots Studio + loads the model now rotates the
bootstrap password BEFORE calling /api/inference/load. /api/inference/
load is gated behind must_change_password=false; the previous flow
(login bootstrap -> load) was succeeding in CI by historical accident
and started failing locally. New flow:
bootstrap login -> change-password -> rotated login -> load model
Both passwords are exposed to the Playwright step via env, so the
test can drive /login with the rotated password AND assert the old
one is now 401.
Verified locally end-to-end against a real Studio install with
gemma-3-270m-it-GGUF UD-Q4_K_XL: all 15 sections pass, console.error
count = 0, total runtime ~30s.
148 lines
5.7 KiB
YAML
148 lines
5.7 KiB
YAML
# SPDX-License-Identifier: AGPL-3.0-only
|
|
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
|
|
|
|
# Verifies that `unsloth studio update --local` is idempotent: a fresh
|
|
# install via install.sh, followed by `unsloth studio update --local`,
|
|
# succeeds and is a no-op for the llama.cpp prebuilt (it should report
|
|
# "prebuilt up to date and validated", not re-run the source build).
|
|
#
|
|
# This catches regressions in setup.sh's update path that the existing
|
|
# GGUF / wheel jobs would miss because they only invoke install.sh once.
|
|
|
|
name: Studio Update CI
|
|
|
|
on:
|
|
pull_request:
|
|
paths:
|
|
- 'install.sh'
|
|
- 'studio/setup.sh'
|
|
- 'studio/install_python_stack.py'
|
|
- 'studio/install_llama_prebuilt.py'
|
|
- 'studio/backend/requirements/**'
|
|
- 'unsloth_cli/commands/studio.py'
|
|
- 'pyproject.toml'
|
|
- '.github/workflows/studio-update-smoke.yml'
|
|
push:
|
|
branches: [main, pip]
|
|
workflow_dispatch:
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
update-idempotency:
|
|
name: Studio Updating Tests
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 15
|
|
steps:
|
|
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
|
|
|
|
- name: Linux deps for llama.cpp prebuilt
|
|
run: |
|
|
sudo apt-get update
|
|
sudo apt-get install -y --no-install-recommends \
|
|
libcurl4-openssl-dev libssl-dev jq
|
|
|
|
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
|
with:
|
|
node-version: '22'
|
|
cache: 'npm'
|
|
cache-dependency-path: studio/frontend/package-lock.json
|
|
|
|
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
|
|
with:
|
|
python-version: '3.12'
|
|
cache: 'pip'
|
|
|
|
- name: Install Studio (--local, --no-torch)
|
|
# Pass the workflow token so the llama.cpp prebuilt installer's
|
|
# GitHub-API call to list releases isn't rate-limited (60/hr
|
|
# unauthenticated). Without this, three consecutive install +
|
|
# update + update calls in this job exceed the limit and the
|
|
# prebuilt path falls back to source build.
|
|
env:
|
|
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
run: |
|
|
mkdir -p logs
|
|
set -o pipefail
|
|
bash install.sh --local --no-torch 2>&1 | tee logs/install.log
|
|
|
|
- name: First update should be a no-op (prebuilt already validated)
|
|
# `unsloth studio update --local` runs studio/setup.sh against
|
|
# the local repo. Right after install.sh the llama.cpp prebuilt
|
|
# has just been installed and validated, so the second run must
|
|
# take the "prebuilt up to date and validated" code path. Any
|
|
# source-build fallback or re-download here means setup.sh's
|
|
# idempotency regressed.
|
|
env:
|
|
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
run: |
|
|
set -o pipefail
|
|
unsloth studio update --local 2>&1 | tee logs/update.log
|
|
if grep -q "falling back to source build" logs/update.log; then
|
|
echo "::error::studio update fell back to source-build llama.cpp on a fresh install. setup.sh idempotency regressed."
|
|
grep -E "llama-prebuilt|llama.cpp" logs/update.log | tail -60
|
|
exit 1
|
|
fi
|
|
if ! grep -qE "prebuilt up to date and validated|prebuilt installed and validated" logs/update.log; then
|
|
echo "::error::no prebuilt up-to-date marker in update.log. Did setup.sh skip the prebuilt path on update?"
|
|
grep -E "llama-prebuilt|llama.cpp" logs/update.log | tail -60
|
|
exit 1
|
|
fi
|
|
echo "update path took the prebuilt fast path"
|
|
|
|
- name: Second update must also be a no-op
|
|
# Two consecutive `update`s back-to-back is the usual desktop
|
|
# flow (auto-update, then user-triggered update). Asserting the
|
|
# second run is also clean rules out hidden state changes from
|
|
# the first one.
|
|
env:
|
|
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
run: |
|
|
set -o pipefail
|
|
unsloth studio update --local 2>&1 | tee logs/update2.log
|
|
grep -q "falling back to source build" logs/update2.log && {
|
|
echo "::error::second update fell back to source build"
|
|
tail -60 logs/update2.log; exit 1; } || true
|
|
grep -qE "prebuilt up to date and validated|prebuilt installed and validated" logs/update2.log
|
|
echo "second update was clean"
|
|
|
|
- name: Boot Studio briefly to confirm the install is still usable
|
|
# If `update --local` accidentally broke the venv or wiped the
|
|
# llama-server binary, the server would fail to start here.
|
|
run: |
|
|
mkdir -p logs
|
|
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p 18891 \
|
|
> logs/studio.log 2>&1 &
|
|
PID=$!
|
|
for i in $(seq 1 60); do
|
|
if curl -fs http://127.0.0.1:18891/api/health > /tmp/health.json; then
|
|
jq -e '.status == "healthy"' /tmp/health.json
|
|
break
|
|
fi
|
|
sleep 1
|
|
done
|
|
if ! jq -e '.status == "healthy"' /tmp/health.json 2>/dev/null; then
|
|
echo "Studio failed to come up after `update`"
|
|
tail -200 logs/studio.log
|
|
kill "$PID" 2>/dev/null || true
|
|
exit 1
|
|
fi
|
|
kill "$PID" 2>/dev/null || true
|
|
echo "post-update Studio /api/health OK"
|
|
|
|
- name: Upload update logs on failure
|
|
if: failure()
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: studio-update-log
|
|
path: |
|
|
logs/install.log
|
|
logs/update.log
|
|
logs/update2.log
|
|
logs/studio.log
|
|
retention-days: 7
|