9723d72aa4
Previously, UNSLOTH_REF was pinned to the triggering tag (e.g. v2026.5.8) but UNSLOTH_ZOO_REF was hardcoded to main. That made release-tag images ship a zoo from whatever was on main at build time rather than the zoo release cut alongside that unsloth tag, so a 2026.5.8 tag image could install a zoo from days later. Mirror the tag branch of UNSLOTH_REF. SHA-based branch pushes still fall through to main because the unsloth SHA does not exist in the unsloth-zoo repo. workflow_dispatch still honours the unsloth_zoo_ref input.
262 lines
11 KiB
YAML
262 lines
11 KiB
YAML
# Builds and publishes the Blackwell-compatible Unsloth Docker image.
|
|
#
|
|
# The build runs on free GitHub-hosted Ubuntu runners with NO GPU attached.
|
|
# This is possible because:
|
|
# 1. cu128 PyTorch wheels are fat binaries -- they already ship sm_70 through
|
|
# sm_120 SASS on amd64 (and sm_80;90;100;120 on aarch64), cross-compiled
|
|
# upstream by the PyTorch team.
|
|
# 2. The Dockerfile pins explicit wheel URLs (no --torch-backend=auto, no
|
|
# install.sh that introspects the host driver).
|
|
# 3. The build-time sanity check uses torch._C._cuda_getArchFlags(), which
|
|
# reads compiled wheel metadata and does NOT require a CUDA device.
|
|
# 4. UNSLOTH_COMPILE_DISABLE=1 prevents Unsloth from JIT-compiling a Triton
|
|
# kernel cache keyed to the (non-existent) build-host GPU.
|
|
#
|
|
# Multi-arch: build amd64 and arm64 in parallel on NATIVE GitHub runners
|
|
# (`ubuntu-latest` and `ubuntu-24.04-arm`, both free on public repos since
|
|
# Aug-2025), then merge the per-arch digests into a single multi-platform
|
|
# manifest. Native arm64 is ~3x faster than building aarch64 under QEMU,
|
|
# and avoids QEMU's occasional flakiness on long-running cu* installs.
|
|
# End users on DGX Spark / Grace pull the arm64 child natively; CUDA works
|
|
# as normal (no runtime emulation).
|
|
#
|
|
# Required repository secrets:
|
|
# DOCKERHUB_USERNAME, DOCKERHUB_TOKEN
|
|
#
|
|
# Optional repository variable (gates the smoke-test job):
|
|
# HAS_GPU_RUNNER = 'true' if a self-hosted GPU runner is available
|
|
|
|
name: Publish Blackwell Docker image
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
tags: ['v*']
|
|
schedule:
|
|
- cron: '17 4 * * 1' # weekly Mon 04:17 UTC (off-the-hour on purpose)
|
|
workflow_dispatch:
|
|
inputs:
|
|
unsloth_ref:
|
|
description: 'unsloth git ref to bake in'
|
|
required: false
|
|
default: 'main'
|
|
unsloth_zoo_ref:
|
|
description: 'unsloth-zoo git ref to bake in'
|
|
required: false
|
|
default: 'main'
|
|
|
|
env:
|
|
REGISTRY: docker.io
|
|
IMAGE_NAME: unsloth/unsloth
|
|
|
|
# Serialise per-ref runs so two pushes to main (or two scheduled
|
|
# fires racing a manual dispatch) don't both retag `:latest` from
|
|
# different commits. Don't cancel in-progress runs -- the build is
|
|
# expensive and a half-built image left around in Docker Hub is
|
|
# worse than a slightly stale `:latest` for a few minutes.
|
|
concurrency:
|
|
group: docker-publish-${{ github.ref }}
|
|
cancel-in-progress: false
|
|
|
|
jobs:
|
|
# ---------------------------------------------------------------------------
|
|
# Per-arch build. The matrix fans out two parallel jobs on the matching
|
|
# native runner. Each pushes a single-arch image *by digest* (no human-
|
|
# readable tag), and the merge job below stitches the two digests into one
|
|
# multi-arch manifest under the real tags. This is the canonical pattern
|
|
# from docker/build-push-action's docs and avoids the "last push wins" race
|
|
# that you get when two jobs push the same tag separately.
|
|
# ---------------------------------------------------------------------------
|
|
build:
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- platform: linux/amd64
|
|
runner: ubuntu-latest
|
|
- platform: linux/arm64
|
|
runner: ubuntu-24.04-arm
|
|
runs-on: ${{ matrix.runner }}
|
|
timeout-minutes: 90
|
|
permissions:
|
|
contents: read
|
|
packages: write
|
|
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
# Free up ~20GB on the runner so cu128 wheels + cudnn fit. Layout is
|
|
# similar between the amd64 and arm64 runners but not identical -- the
|
|
# arm64 image lacks /usr/share/dotnet, hence `|| true`.
|
|
- name: Reclaim disk
|
|
run: |
|
|
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc \
|
|
/opt/hostedtoolcache/CodeQL "$AGENT_TOOLSDIRECTORY" || true
|
|
df -h /
|
|
|
|
- uses: docker/setup-buildx-action@v3
|
|
|
|
- name: Log in to Docker Hub
|
|
uses: docker/login-action@v3
|
|
with:
|
|
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
|
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
|
|
|
# Pull the image label/annotation set we'll attach to the FINAL manifest.
|
|
# We don't apply tags at this layer because each per-arch build pushes by
|
|
# digest only; tags get attached by the merge job.
|
|
- name: Resolve labels
|
|
id: meta
|
|
uses: docker/metadata-action@v5
|
|
with:
|
|
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
|
|
|
- name: Build and push (per-arch by digest)
|
|
id: build
|
|
uses: docker/build-push-action@v6
|
|
with:
|
|
context: ./docker
|
|
file: ./docker/Dockerfile
|
|
platforms: ${{ matrix.platform }}
|
|
labels: ${{ steps.meta.outputs.labels }}
|
|
# Per-arch build cache. Keying on the platform suffix lets the two
|
|
# matrix legs reuse their own caches without colliding.
|
|
cache-from: type=gha,scope=build-${{ matrix.platform }}
|
|
cache-to: type=gha,scope=build-${{ matrix.platform }},mode=max
|
|
outputs: type=image,name=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=true
|
|
build-args: |
|
|
CUDA_VERSION=12.8.1
|
|
UBUNTU_VERSION=24.04
|
|
PYTHON_VERSION=3.12
|
|
# Workflow-dispatch: honour the explicit input. Tag pushes:
|
|
# bake the tag's source ref (e.g. v1.2.3) so the published
|
|
# tag image actually contains that release. Branch pushes and
|
|
# scheduled runs: bake the triggering commit SHA. Falls back
|
|
# to `main` for any other event class.
|
|
UNSLOTH_REF=${{ github.event.inputs.unsloth_ref || (startsWith(github.ref, 'refs/tags/') && github.ref_name) || github.sha || 'main' }}
|
|
# UNSLOTH_ZOO_REF mirrors the tag case (unsloth-zoo cuts the same
|
|
# release tag, e.g. 2026.5.8, alongside unsloth) so release-tag
|
|
# images install a matched zoo. SHA-based branch pushes can't be
|
|
# mirrored -- the SHA doesn't exist in the zoo repo -- so they
|
|
# fall through to `main`. Workflow-dispatch can override.
|
|
UNSLOTH_ZOO_REF=${{ github.event.inputs.unsloth_zoo_ref || (startsWith(github.ref, 'refs/tags/') && github.ref_name) || 'main' }}
|
|
|
|
# Stash the per-arch digest as an artifact for the merge job to pick up.
|
|
# Filenames need to be unique across the matrix; `platform` contains a
|
|
# slash so substitute it for a dash.
|
|
- name: Export digest
|
|
run: |
|
|
mkdir -p /tmp/digests
|
|
digest='${{ steps.build.outputs.digest }}'
|
|
touch "/tmp/digests/${digest#sha256:}"
|
|
|
|
- name: Upload digest
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: digests-${{ matrix.platform == 'linux/amd64' && 'amd64' || 'arm64' }}
|
|
path: /tmp/digests/*
|
|
if-no-files-found: error
|
|
retention-days: 1
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Merge the two per-arch digests into a multi-platform manifest under the
|
|
# real, user-facing tag(s). This job runs only after both `build` matrix
|
|
# legs finish successfully.
|
|
# ---------------------------------------------------------------------------
|
|
merge:
|
|
runs-on: ubuntu-latest
|
|
needs: build
|
|
timeout-minutes: 15
|
|
permissions:
|
|
contents: read
|
|
packages: write
|
|
steps:
|
|
- uses: actions/download-artifact@v4
|
|
with:
|
|
path: /tmp/digests
|
|
pattern: digests-*
|
|
merge-multiple: true
|
|
|
|
- uses: docker/setup-buildx-action@v3
|
|
|
|
- name: Log in to Docker Hub
|
|
uses: docker/login-action@v3
|
|
with:
|
|
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
|
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
|
|
|
- name: Resolve tags
|
|
id: meta
|
|
uses: docker/metadata-action@v5
|
|
with:
|
|
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
|
tags: |
|
|
# Only tag :latest when the workflow ran on the default branch
|
|
# AND the operator did NOT override unsloth_ref on dispatch.
|
|
# Without the second condition a maintainer testing a feature
|
|
# SHA from main could overwrite :latest with non-main source.
|
|
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && github.event.inputs.unsloth_ref == '' }}
|
|
type=ref,event=tag
|
|
type=schedule,pattern=nightly
|
|
type=sha,prefix=sha-,format=short
|
|
|
|
- name: Create multi-arch manifest
|
|
working-directory: /tmp/digests
|
|
run: |
|
|
docker buildx imagetools create \
|
|
$(jq -cr '.tags | map("-t " + .) | join(" ")' <<<"$DOCKER_METADATA_OUTPUT_JSON") \
|
|
$(printf '${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@sha256:%s ' *)
|
|
|
|
- name: Inspect the result
|
|
run: |
|
|
for tag in $(jq -r '.tags[]' <<<"$DOCKER_METADATA_OUTPUT_JSON"); do
|
|
echo "=== $tag ==="
|
|
docker buildx imagetools inspect "$tag"
|
|
done
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Optional: pull the freshly published image onto a self-hosted GPU runner
|
|
# and run smoke_test.py. Skipped automatically when no GPU runner is
|
|
# registered. Architecture matches whatever the runner is.
|
|
# ---------------------------------------------------------------------------
|
|
smoke-test:
|
|
needs: merge
|
|
if: ${{ vars.HAS_GPU_RUNNER == 'true' }}
|
|
runs-on: [self-hosted, gpu]
|
|
timeout-minutes: 20
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
# Re-compute the tag list deterministically from the same metadata-action
|
|
# config the merge job used, so tag/schedule/SHA runs pull the image
|
|
# they just published instead of an unrelated `:latest` from a prior run.
|
|
# IMPORTANT: keep this `enable=` expression byte-identical to the merge
|
|
# job's :latest gate above. The two used to differ
|
|
# (merge: ref + unsloth_ref guard; smoke: is_default_branch only),
|
|
# which meant workflow_dispatch with unsloth_ref defaulting to "main"
|
|
# would skip :latest on merge but still emit :latest as tags[0] on
|
|
# smoke -- so docker pull would fetch a previously-published :latest
|
|
# from Docker Hub, not the image just merged.
|
|
- name: Resolve published tag
|
|
id: meta
|
|
uses: docker/metadata-action@v5
|
|
with:
|
|
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
|
tags: |
|
|
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && github.event.inputs.unsloth_ref == '' }}
|
|
type=ref,event=tag
|
|
type=schedule,pattern=nightly
|
|
type=sha,prefix=sha-,format=short
|
|
|
|
- name: Pull and smoke-test
|
|
run: |
|
|
# Use the first tag from the metadata output -- that is the image we
|
|
# just published. Falls back to :latest only when the metadata is
|
|
# empty (defensive; should not happen on default-branch runs).
|
|
TAG="$(jq -r '.tags[0] // ""' <<<"$DOCKER_METADATA_OUTPUT_JSON")"
|
|
if [ -z "$TAG" ]; then
|
|
TAG="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest"
|
|
fi
|
|
echo "smoke-testing $TAG"
|
|
docker pull "$TAG"
|
|
docker run --rm --gpus all "$TAG" python /workspace/smoke_test.py
|