Studio: stop handing CI/user secrets to downloaded llama.cpp binaries (#6696)
* Studio: stop handing CI/user secrets to downloaded llama.cpp binaries The macOS prebuilt path installs llama.cpp from the unslothai/llama.cpp fork's latest (unpinned, mutable) release and then executes the downloaded llama-server / llama-quantize binaries during install-time validation. binary_env() built that child environment from a full os.environ.copy(), so a compromised or tampered prebuilt would inherit every secret in the process: HF_TOKEN and the workflow GitHub tokens in CI, and HF / cloud credentials for end users running install.sh / setup.sh. We publish prebuilts daily, so pinning a release tag is not workable. Instead, neutralise the impact: these binaries have no reason to read any token, so strip secret-bearing variables (exact names plus TOKEN/SECRET/PASSWORD/CREDENTIAL/PRIVATE_KEY/API_KEY markers) before handing the env to a downloaded binary. The installer's own GitHub and Hugging Face API calls read os.environ directly, so authentication and release-API rate limiting are unaffected; PATH, LD_LIBRARY_PATH, DYLD_LIBRARY_PATH and CUDA/ROCm vars are preserved. One change covers the install-time validation path for all six macOS workflows and end users. Follow-up (separate, sequenced): publish build-provenance attestations from the fork's prebuilt workflows and verify them in CI, so a forged release is rejected rather than merely starved of secrets. * Strip KUBECONFIG, SSH_AUTH_SOCK, and PASSPHRASE-marked vars from binary env Extend the deny-list per PR review: KUBECONFIG and SSH_AUTH_SOCK are credential pointers/capabilities a downloaded binary never needs, and a PASSPHRASE marker catches SSH_PASSPHRASE / GPG_PASSPHRASE. Tests updated. * Studio: also scrub proxy/index env vars and URL-embedded credentials before running prebuilt binaries * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope mlx-ci secrets to the install + download commands for PR #6696 Drop the ambient step-level env block and pass GH/GITHUB/HF tokens only on the installer and GGUF-download commands, so the directly invoked llama-quantize / llama-server smoke runs see no secrets. The installer still reads tokens from os.environ for the releases API and probe fetch. * Trim verbose comments around the secret-env scrubber for PR #6696 Comment-only: condense the block comments added across this PR. Logic unchanged (comment_tools.py check confirms code-only signature equal). * Redirect HOME / cache pointers to an empty dir for prebuilt binaries (PR #6696) Address Codex P2: stripping token env vars still let a tampered binary read on-disk token stores (~/.cache/huggingface/token, ~/.aws/credentials, ~/.config/gh) through $HOME and the cache/config pointers. Point HOME plus the HF / XDG / Windows home pointers at a single empty throwaway dir for the downloaded-binary env. Defense in depth: a binary resolving the real home via getpwuid is out of scope and needs OS sandboxing. * Close residual credential-probe gaps for PR #6696 Address the latest Codex review: - Strip token-only URL userinfo too (scheme://ghp_token@host), not just the user:pass form. - Redirect HOMEDRIVE/HOMEPATH alongside USERPROFILE so a Windows binary cannot reconstruct the real profile from %HOMEDRIVE%%HOMEPATH%. - Drop explicit credential-file pointers (NETRC, PIP_CONFIG_FILE, DOCKER_CONFIG, GIT_CONFIG_GLOBAL) that live outside HOME. - Probe ldd with a secret-free env: linux_runtime_dirs ran ldd on the untrusted prebuilt with the inherited os.environ, and ldd may execute the binary, so it could observe HF_TOKEN/GITHUB_TOKEN during the probe. Factored the shared scrub into secret_free_environ(). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Separate token-bearing install from binary smoke; drop CI command files (PR #6696) Address the two P1s in the latest review: - mlx-ci: GitHub bakes secrets into the run-script text, so inline token assignments in a step that later runs the prebuilt let a tampered binary read them from the script. Split into a token-bearing install + download step that never launches a binary, and a secret-free smoke step that runs llama-quantize / llama-server. - secret_free_environ now drops the GitHub Actions command files (GITHUB_ENV, GITHUB_PATH, GITHUB_OUTPUT, GITHUB_STEP_SUMMARY, BASH_ENV) and the smoke step unsets them, so a tampered prebuilt cannot inject PATH/env into the later token-bearing MLX steps. * Run the prebuilt smoke last, after all token-bearing steps (PR #6696) Address the P1 workspace-poisoning vector: even with no secrets in its env, a tampered prebuilt could edit the checkout or installed modules, and the later HF_TOKEN MLX steps would then execute that poisoned code on push builds. Move the prebuilt install + smoke to the end of the job so the untrusted binary runs after every token-bearing step, leaving nothing for it to corrupt. The MLX GGUF reload uses a source-built llama-cli, not this prebuilt, so nothing depends on the earlier position. * Trim comments around the secret-env scrubber and prebuilt CI steps (PR #6696) Comment-only: condense the security-rationale block comments and merge the duplicated prebuilt-step description in mlx-ci. Logic unchanged (comment_tools.py check confirms the code-only signature is equal; install suite still passes). * Authenticate the GGUF export release-API lookup with the read-only GITHUB_TOKEN (PR #6696) * Rename env scrubber off the secret-named identifier CodeQL flags as a clear-text sink (PR #6696) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Daniel Han
GitHub
parent
98a01e70cd
commit
4c72e09480
3 changed files with 482 additions and 95 deletions
181
.github/workflows/mlx-ci.yml
vendored
181
.github/workflows/mlx-ci.yml
vendored
|
|
@ -231,99 +231,6 @@ jobs:
|
|||
tests/studio/test_is_mlx_dispatch_gate.py \
|
||||
tests/studio/test_mlx_training_worker_behaviors.py
|
||||
|
||||
# Studio prebuilt llama.cpp install + GGUF inference. Mirrors the
|
||||
# path Studio's setup.sh takes on macOS since #5963: plan against
|
||||
# the unslothai/llama.cpp fork's latest release, which ships the
|
||||
# bin-macos-arm64 bundle plus the llama-prebuilt-manifest.json the
|
||||
# default policy reads. After install, downloads a small published
|
||||
# GGUF (unsloth/gemma-3-270m-it-GGUF, Q4_K_M) and validates
|
||||
# llama-server /completion end to end. An install failure or a
|
||||
# non-zero binary exit is an Unsloth/Studio bug.
|
||||
- name: Studio prebuilt llama.cpp install + GGUF inference (Mac M1)
|
||||
env:
|
||||
# Withheld on PR: this step runs checked-out PR code; public GGUF still downloads.
|
||||
HF_TOKEN: ${{ github.event_name != 'pull_request' && secrets.HF_TOKEN || '' }}
|
||||
# install_llama_prebuilt.py hits the GitHub releases API to
|
||||
# resolve the asset URL. Anonymous calls share the runner-IP
|
||||
# rate-limit bucket and 403 quickly -- pass the workflow's
|
||||
# automatic GITHUB_TOKEN to bump us to the 5000/hr authenticated
|
||||
# bucket.
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
INSTALL_DIR="$HOME/.unsloth-studio-prebuilt-test/llama.cpp"
|
||||
rm -rf "$INSTALL_DIR"
|
||||
# Mirror studio/setup.sh on macOS (the install.sh user path):
|
||||
# it plans against the unslothai/llama.cpp fork's latest
|
||||
# release with no policy or tag flags.
|
||||
python studio/install_llama_prebuilt.py \
|
||||
--install-dir "$INSTALL_DIR" \
|
||||
--published-repo unslothai/llama.cpp
|
||||
|
||||
# Studio bundles only llama-server + llama-quantize from the
|
||||
# prebuilt (not llama-cli) -- inference goes through
|
||||
# llama-server's HTTP /completion endpoint. Validate both:
|
||||
# llama-quantize --help proves the dynamic libs link, then
|
||||
# spin up llama-server and POST a /completion request on a
|
||||
# tiny published GGUF.
|
||||
LLAMA_SERVER="$INSTALL_DIR/build/bin/llama-server"
|
||||
LLAMA_QUANT="$INSTALL_DIR/build/bin/llama-quantize"
|
||||
[ -x "$LLAMA_SERVER" ] || { echo "::error::llama-server missing at $LLAMA_SERVER"; find "$INSTALL_DIR/build" -type f | head -40; exit 1; }
|
||||
[ -x "$LLAMA_QUANT" ] || { echo "::error::llama-quantize missing at $LLAMA_QUANT"; exit 1; }
|
||||
echo "llama-server : $LLAMA_SERVER"
|
||||
echo "llama-quantize: $LLAMA_QUANT"
|
||||
"$LLAMA_QUANT" --help >/dev/null && echo " llama-quantize loads OK"
|
||||
|
||||
mkdir -p /tmp/ggufs
|
||||
bash .github/scripts/hf-download-with-retry.sh \
|
||||
'unsloth/gemma-3-270m-it-GGUF' \
|
||||
'gemma-3-270m-it-Q4_K_M.gguf' \
|
||||
/tmp/ggufs
|
||||
|
||||
PORT=18080
|
||||
echo "=== starting llama-server on 127.0.0.1:$PORT ==="
|
||||
"$LLAMA_SERVER" \
|
||||
-m /tmp/ggufs/gemma-3-270m-it-Q4_K_M.gguf \
|
||||
--host 127.0.0.1 \
|
||||
--port "$PORT" \
|
||||
-c 256 \
|
||||
-n 16 \
|
||||
--no-warmup \
|
||||
> /tmp/llama-server.log 2>&1 &
|
||||
SERVER_PID=$!
|
||||
trap 'kill "$SERVER_PID" 2>/dev/null || true' EXIT
|
||||
|
||||
# Wait for /health to come up
|
||||
for i in $(seq 1 30); do
|
||||
if curl -sf "http://127.0.0.1:$PORT/health" >/dev/null 2>&1; then
|
||||
echo " server up after ${i}s"
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
if ! curl -sf "http://127.0.0.1:$PORT/health" >/dev/null 2>&1; then
|
||||
echo "::error::llama-server never became healthy"
|
||||
tail -40 /tmp/llama-server.log
|
||||
exit 1
|
||||
fi
|
||||
|
||||
PROMPT="Hello, my name is"
|
||||
echo "=== POST /completion ==="
|
||||
RESP=$(curl -sf -X POST "http://127.0.0.1:$PORT/completion" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "{\"prompt\":\"$PROMPT\",\"n_predict\":16,\"temperature\":0,\"seed\":3407}")
|
||||
echo "raw response (head): $(echo "$RESP" | head -c 600)"
|
||||
CONTENT=$(echo "$RESP" | python -c "import json,sys; print(json.loads(sys.stdin.read()).get('content',''))")
|
||||
echo "completion content: $CONTENT"
|
||||
|
||||
if [ -z "$CONTENT" ]; then
|
||||
echo "::error::llama-server /completion returned empty content"
|
||||
tail -40 /tmp/llama-server.log
|
||||
exit 1
|
||||
fi
|
||||
echo "OK: Studio prebuilt llama.cpp on Mac M1 + GGUF /completion works"
|
||||
|
||||
# Real MLX training + inference smoke test. Trains
|
||||
# unsloth/gemma-3-270m-it for 7 deterministic LoRA steps
|
||||
# (batch_size=2, gradient_accumulation_steps=3) on a single
|
||||
|
|
@ -338,6 +245,9 @@ jobs:
|
|||
UNSLOTH_COMPILE_DISABLE: '1'
|
||||
run: |
|
||||
mkdir -p mlx_workdir
|
||||
# Authenticate llama.cpp's release-API lookup (anonymous 403s on rate-limit);
|
||||
# read-only GITHUB_TOKEN scoped here only, never to steps that run binaries.
|
||||
GH_TOKEN="${{ secrets.GITHUB_TOKEN }}" GITHUB_TOKEN="${{ secrets.GITHUB_TOKEN }}" \
|
||||
python tests/studio/run_real_mlx_smoke.py train \
|
||||
--workdir "$PWD/mlx_workdir"
|
||||
|
||||
|
|
@ -406,3 +316,88 @@ jobs:
|
|||
cat "$f" 2>/dev/null || echo "(missing)"
|
||||
echo
|
||||
done
|
||||
|
||||
# Validates the macOS prebuilt path Studio's setup.sh uses (#5963): install the
|
||||
# unslothai/llama.cpp fork's latest release, download a small public GGUF, and
|
||||
# check llama-server /completion end to end. Split and placed last so the
|
||||
# untrusted binary runs only in the final smoke step, after every HF_TOKEN step,
|
||||
# leaving no token-bearing step or shared workspace for a tampered prebuilt to
|
||||
# corrupt. GH_TOKEN: releases API; HF_TOKEN (withheld on PR): probe + GGUF fetch.
|
||||
- name: Studio prebuilt llama.cpp install + GGUF download (Mac M1)
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
HF_TOKEN: ${{ github.event_name != 'pull_request' && secrets.HF_TOKEN || '' }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
INSTALL_DIR="$HOME/.unsloth-studio-prebuilt-test/llama.cpp"
|
||||
rm -rf "$INSTALL_DIR"
|
||||
# Download only -- no llama-quantize / llama-server launch in this step.
|
||||
python studio/install_llama_prebuilt.py \
|
||||
--install-dir "$INSTALL_DIR" \
|
||||
--published-repo unslothai/llama.cpp
|
||||
mkdir -p /tmp/ggufs
|
||||
bash .github/scripts/hf-download-with-retry.sh \
|
||||
'unsloth/gemma-3-270m-it-GGUF' \
|
||||
'gemma-3-270m-it-Q4_K_M.gguf' \
|
||||
/tmp/ggufs
|
||||
|
||||
# Final step: runs the downloaded binaries with no secrets present, and clears
|
||||
# the GitHub Actions command files so a tampered prebuilt cannot influence the job.
|
||||
- name: Studio prebuilt llama.cpp GGUF inference smoke (Mac M1)
|
||||
run: |
|
||||
set -euo pipefail
|
||||
unset GITHUB_ENV GITHUB_PATH GITHUB_OUTPUT GITHUB_STEP_SUMMARY
|
||||
INSTALL_DIR="$HOME/.unsloth-studio-prebuilt-test/llama.cpp"
|
||||
# Studio bundles only llama-server + llama-quantize (not llama-cli);
|
||||
# inference goes through llama-server's HTTP /completion endpoint.
|
||||
LLAMA_SERVER="$INSTALL_DIR/build/bin/llama-server"
|
||||
LLAMA_QUANT="$INSTALL_DIR/build/bin/llama-quantize"
|
||||
[ -x "$LLAMA_SERVER" ] || { echo "::error::llama-server missing at $LLAMA_SERVER"; find "$INSTALL_DIR/build" -type f | head -40; exit 1; }
|
||||
[ -x "$LLAMA_QUANT" ] || { echo "::error::llama-quantize missing at $LLAMA_QUANT"; exit 1; }
|
||||
echo "llama-server : $LLAMA_SERVER"
|
||||
echo "llama-quantize: $LLAMA_QUANT"
|
||||
"$LLAMA_QUANT" --help >/dev/null && echo " llama-quantize loads OK"
|
||||
|
||||
PORT=18080
|
||||
echo "=== starting llama-server on 127.0.0.1:$PORT ==="
|
||||
"$LLAMA_SERVER" \
|
||||
-m /tmp/ggufs/gemma-3-270m-it-Q4_K_M.gguf \
|
||||
--host 127.0.0.1 \
|
||||
--port "$PORT" \
|
||||
-c 256 \
|
||||
-n 16 \
|
||||
--no-warmup \
|
||||
> /tmp/llama-server.log 2>&1 &
|
||||
SERVER_PID=$!
|
||||
trap 'kill "$SERVER_PID" 2>/dev/null || true' EXIT
|
||||
|
||||
# Wait for /health to come up
|
||||
for i in $(seq 1 30); do
|
||||
if curl -sf "http://127.0.0.1:$PORT/health" >/dev/null 2>&1; then
|
||||
echo " server up after ${i}s"
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
if ! curl -sf "http://127.0.0.1:$PORT/health" >/dev/null 2>&1; then
|
||||
echo "::error::llama-server never became healthy"
|
||||
tail -40 /tmp/llama-server.log
|
||||
exit 1
|
||||
fi
|
||||
|
||||
PROMPT="Hello, my name is"
|
||||
echo "=== POST /completion ==="
|
||||
RESP=$(curl -sf -X POST "http://127.0.0.1:$PORT/completion" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "{\"prompt\":\"$PROMPT\",\"n_predict\":16,\"temperature\":0,\"seed\":3407}")
|
||||
echo "raw response (head): $(echo "$RESP" | head -c 600)"
|
||||
CONTENT=$(echo "$RESP" | python -c "import json,sys; print(json.loads(sys.stdin.read()).get('content',''))")
|
||||
echo "completion content: $CONTENT"
|
||||
|
||||
if [ -z "$CONTENT" ]; then
|
||||
echo "::error::llama-server /completion returned empty content"
|
||||
tail -40 /tmp/llama-server.log
|
||||
exit 1
|
||||
fi
|
||||
echo "OK: Studio prebuilt llama.cpp on Mac M1 + GGUF /completion works"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue