unsloth/.github/workflows/studio-api-smoke.yml
Daniel Han 8ca0455be4
studio/ci: sweep actions/cache v5 hardening across sibling smoke workflows (#5399)
* studio/ci: sweep actions/cache@v5 hardening across sibling smoke workflows

Follow-up to PR 5396, which fixed the same flake in
studio-windows-inference-smoke.yml. actions/cache@v5 has a recurring
mode where it logs `Cache hit for: <key>` and then exits non-zero
without extracting the archive (see actions/cache#1621 and github
community discussion #163260). 12 cache blocks across 8 sibling Studio
smoke workflows remained on the vulnerable one-step pattern and would
abort before priming HF_HOME / installing Studio on the same flake.

Apply the same restore + save split mechanically to every block:

  - actions/cache/restore@<v5.0.5 sha> with continue-on-error: true
  - Prime/Download gate widened to also fire on outcome != 'success'
    so the silent-restore-failure path re-downloads
  - actions/cache/save@<v5.0.5 sha> with continue-on-error: true,
    gated on the Prime/Download outcome so we only write a fresh entry
    when we actually rebuilt the directory

Same SHA-pinned action, same cache keys (character-identical), same
paths. Existing cache entries keep matching. Only behavior change is
that a transient restore-side or save-side failure now falls through
to a re-download instead of failing the job.

Files touched (12 cache blocks total):
  studio-api-smoke.yml             (1 block)
  studio-mac-api-smoke.yml         (1 block)
  studio-mac-ui-smoke.yml          (1 block)
  studio-ui-smoke.yml              (1 block)
  studio-windows-api-smoke.yml     (1 block)
  studio-windows-ui-smoke.yml      (1 block)
  studio-inference-smoke.yml       (3 blocks: HF, GGUF flat, HF+mmproj)
  studio-mac-inference-smoke.yml   (3 blocks: HF, GGUF flat, HF+mmproj)

Verification: all 12 single-step actions/cache@ uses removed, replaced
by 12 restore@ + 12 save@; every file parses as valid YAML.

* studio/ci: drop continue-on-error from cache/save steps

Reverting the save-side continue-on-error addition. Defensive masking
of save failures was correct in principle but loses signal:

  - cache/save@v5.0.5 already swallows ReserveCacheError (the most
    common save flake) as a non-fatal core.info, so the mask was
    rarely doing anything today.
  - A real save-side failure (sustained cache backend outage, blob
    server 5xx storm) is something we want to see, not hide. Without
    the signal we would see slow CI for days without knowing the cache
    layer is broken.
  - If save flakes start showing up in practice we add this back with
    concrete evidence.

The restore-side continue-on-error stays -- that is the actual fix
for actions/cache#1621 silent-restore-failures and removing it would
re-introduce the bug.
2026-05-12 05:47:41 -07:00

167 lines
5.9 KiB
YAML

# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
# Studio API & Auth Tests -- HTTP-level integration tests for the
# FastAPI surface. No Playwright, no model UI; tests/studio/test_studio_api_smoke.py
# runs ~30 s and asserts:
# - CORS hardening (no wildcard + credentials, no bootstrap leak)
# - /api/system + /api/system/hardware require auth
# - Auth state machine + JWT expiry
# - API key lifecycle E2E (create / list / use / delete / reject)
# - Auth file-mode hardening (Linux only)
# - Inference lifecycle (force reload, bogus variant, /v1/models, /v1/embeddings, /v1/responses)
# - Endpoint-by-endpoint auth audit
#
# Reuses the GGUF cache key from studio-ui-smoke.yml so the model
# download is one cache-hit on the second job.
name: Studio API CI
on:
pull_request:
paths:
- 'studio/**'
- 'unsloth/**'
- 'unsloth_cli/**'
- 'install.sh'
- 'pyproject.toml'
- 'tests/studio/**'
- '.github/workflows/studio-api-smoke.yml'
push:
branches: [main, pip]
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
api-smoke:
name: Studio API & Auth Tests
runs-on: ubuntu-latest
timeout-minutes: 12
env:
GGUF_REPO: unsloth/gemma-3-270m-it-GGUF
GGUF_VARIANT: UD-Q4_K_XL
GGUF_FILE: gemma-3-270m-it-UD-Q4_K_XL.gguf
STUDIO_PORT: '18893'
HF_HOME: ${{ github.workspace }}/hf-cache
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Linux deps
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
libcurl4-openssl-dev libssl-dev jq
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.12'
cache: 'pip'
- name: Restore HF_HOME for ${{ env.GGUF_REPO }}
id: cache-hf
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
continue-on-error: true
with:
path: hf-cache
# Same key as studio-ui-smoke.yml so the two jobs share a
# single GGUF download across CI.
key: ${{ runner.os }}-hf-${{ env.GGUF_REPO }}-${{ env.GGUF_VARIANT }}-v1
- name: Prime HF_HOME with the GGUF
id: prime-hf
if: steps.cache-hf.outputs.cache-hit != 'true' || steps.cache-hf.outcome != 'success'
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
mkdir -p hf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE"
- name: Save HF_HOME for ${{ env.GGUF_REPO }}
if: always() && steps.prime-hf.outcome == 'success'
uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: hf-cache
key: ${{ runner.os }}-hf-${{ env.GGUF_REPO }}-${{ env.GGUF_VARIANT }}-v1
- name: Install Studio (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
mkdir -p logs
set -o pipefail
bash install.sh --local --no-torch 2>&1 | tee logs/install.log
- name: Install pyjwt for the JWT-expiry forge test
run: pip install 'pyjwt>=2.6'
- name: Reset auth + boot Studio (API-only)
run: |
unsloth studio reset-password
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
echo "STUDIO_PID=$!" >> "$GITHUB_ENV"
- name: Wait for /api/health
run: |
for i in $(seq 1 180); do
if curl -fs "http://127.0.0.1:${STUDIO_PORT}/api/health" > /tmp/health.json; then
jq -e '.status == "healthy"' /tmp/health.json && break
fi
sleep 1
done
jq -e '.status == "healthy"' /tmp/health.json
- name: Pass bootstrap password + rotated targets to the test
# The test does its own bootstrap-login + rotation to exercise
# the auth state machine; we just pre-mint two random rotated
# passwords for it. Mask them so the log is clean.
run: |
OLD=$(cat ~/.unsloth/studio/auth/.bootstrap_password)
NEW="ApiSmoke-$(python -c 'import secrets; print(secrets.token_urlsafe(16))')"
NEW2="ApiSmoke-$(python -c 'import secrets; print(secrets.token_urlsafe(16))')"
echo "::add-mask::$OLD"
echo "::add-mask::$NEW"
echo "::add-mask::$NEW2"
echo "STUDIO_OLD_PW=$OLD" >> "$GITHUB_ENV"
echo "STUDIO_NEW_PW=$NEW" >> "$GITHUB_ENV"
echo "STUDIO_NEW2_PW=$NEW2" >> "$GITHUB_ENV"
- name: Run Studio API & Auth tests
# The script is named WITHOUT a `test_` prefix so it isn't
# auto-collected by pytest in Backend CI's `tests/` walk
# (which doesn't set BASE_URL and would crash at import).
env:
BASE_URL: http://127.0.0.1:18893
STUDIO_AUTH_DIR: /home/runner/.unsloth/studio/auth
run: python tests/studio/studio_api_smoke.py
- name: Stop Studio
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true
sleep 2
- name: Upload API smoke logs
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: studio-api-smoke-log
path: |
logs/install.log
logs/studio.log
retention-days: 7