docker: ship Jupyter, Studio and prebuilt llama.cpp out of the box
Base image (docker/Dockerfile): - Install JupyterLab + notebook + ipywidgets in a separate pure-Python uv pass so the cu128 pin set cannot move; EXPOSE 8888. - Bake the prebuilt llama.cpp bundle into /opt/unsloth/llama.cpp at the runtime stage using studio/install_llama_prebuilt.py from the same UNSLOTH_REF (sha256-verified, portable CUDA bundle since the build host has no GPU; arm64 resolves the linux-arm64-cuda13 bundle). Export UNSLOTH_LLAMA_CPP_PATH so unsloth_zoo's save_pretrained_gguf finds it and never reaches the interactive install prompt or a source build. - Optional github_token BuildKit secret for the resolver's API calls on shared CI runner IPs. Entrypoint: UNSLOTH_ALLOW_CPU=1 degrades a missing GPU to a warning so Docker Desktop on macOS / Windows-without-WSL2-GPU and plain CPU hosts can run Jupyter, GGUF tooling and Studio chat; with a GPU visible the normal pre-flight still runs. Full image (docker/Dockerfile.studio): now mirrors the production service set under supervisord - Studio on 8000, JupyterLab on 8888, key-only sshd on 22 (enabled only when PUBLIC_KEY/SSH_KEY is set). Points Studio's llama.cpp dir at the baked bundle to skip a duplicate download, accepts any git ref via fetch+checkout (CI passes commit SHAs), and FROMs a digest-pinned BASE_IMAGE. Publish workflow: base image moves to the base-* tag namespace; new build-studio/merge-studio jobs publish the full image as :latest (hub parity with the previous production image, which shipped Studio + Jupyter + SSH). Studio builds FROM the exact base manifest digest published by the same run. GPU smoke job now also boots the full image and probes Studio /api/health and Jupyter /api. run.sh: UNSLOTH_GPUS=none, UNSLOTH_ALLOW_CPU forwarding, UNSLOTH_PORTS publish flags, CPU-mode and Jupyter usage examples.
This commit is contained in:
danielhanchen
Daniel Han
parent
6448587483
commit
f1a63db6fa
7 changed files with 481 additions and 51 deletions
221
.github/workflows/docker-publish.yml
vendored
221
.github/workflows/docker-publish.yml
vendored
|
|
@ -124,6 +124,11 @@ jobs:
|
|||
cache-from: type=gha,scope=build-${{ matrix.platform }}
|
||||
cache-to: type=gha,scope=build-${{ matrix.platform }},mode=max
|
||||
outputs: type=image,name=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=true
|
||||
# The llama.cpp prebuilt bake reads GITHUB_TOKEN (BuildKit secret,
|
||||
# never a layer) so the resolver's GitHub API calls are not subject
|
||||
# to the anonymous per-IP rate limit shared across Actions runners.
|
||||
secrets: |
|
||||
github_token=${{ github.token }}
|
||||
build-args: |
|
||||
CUDA_VERSION=12.8.1
|
||||
UBUNTU_VERSION=24.04
|
||||
|
|
@ -153,7 +158,7 @@ jobs:
|
|||
- name: Upload digest
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: digests-${{ matrix.platform == 'linux/amd64' && 'amd64' || 'arm64' }}
|
||||
name: digests-base-${{ matrix.platform == 'linux/amd64' && 'amd64' || 'arm64' }}
|
||||
path: /tmp/digests/*
|
||||
if-no-files-found: error
|
||||
retention-days: 1
|
||||
|
|
@ -170,11 +175,17 @@ jobs:
|
|||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
outputs:
|
||||
# Multi-arch manifest digest of the just-published base image. The
|
||||
# build-studio job FROMs this exact digest so the Studio image always
|
||||
# layers on the bits published by THIS run, not whatever `base`
|
||||
# happens to point at when the job is scheduled.
|
||||
digest: ${{ steps.manifest_digest.outputs.digest }}
|
||||
steps:
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
path: /tmp/digests
|
||||
pattern: digests-*
|
||||
pattern: digests-base-*
|
||||
merge-multiple: true
|
||||
|
||||
- uses: docker/setup-buildx-action@v3
|
||||
|
|
@ -191,10 +202,148 @@ jobs:
|
|||
with:
|
||||
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
||||
tags: |
|
||||
# Only tag :latest when the workflow ran on the default branch
|
||||
# The lean training image publishes under the base- prefix; the
|
||||
# full Studio image (build-studio/merge-studio below) owns
|
||||
# :latest, matching what the previous production image shipped.
|
||||
# Only tag :base when the workflow ran on the default branch
|
||||
# AND the operator did NOT override unsloth_ref on dispatch.
|
||||
# Without the second condition a maintainer testing a feature
|
||||
# SHA from main could overwrite :latest with non-main source.
|
||||
# SHA from main could overwrite :base with non-main source.
|
||||
type=raw,value=base,enable=${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && github.event.inputs.unsloth_ref == '' }}
|
||||
type=ref,event=tag,prefix=base-
|
||||
type=schedule,pattern=base-nightly
|
||||
type=sha,prefix=base-sha-,format=short
|
||||
|
||||
- name: Create multi-arch manifest
|
||||
working-directory: /tmp/digests
|
||||
run: |
|
||||
docker buildx imagetools create \
|
||||
$(jq -cr '.tags | map("-t " + .) | join(" ")' <<<"$DOCKER_METADATA_OUTPUT_JSON") \
|
||||
$(printf '${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@sha256:%s ' *)
|
||||
|
||||
- name: Inspect the result
|
||||
run: |
|
||||
for tag in $(jq -r '.tags[]' <<<"$DOCKER_METADATA_OUTPUT_JSON"); do
|
||||
echo "=== $tag ==="
|
||||
docker buildx imagetools inspect "$tag"
|
||||
done
|
||||
|
||||
- name: Export manifest digest
|
||||
id: manifest_digest
|
||||
run: |
|
||||
TAG="$(jq -r '.tags[0]' <<<"$DOCKER_METADATA_OUTPUT_JSON")"
|
||||
DIGEST="$(docker buildx imagetools inspect "$TAG" --format '{{json .Manifest.Digest}}' | tr -d '"')"
|
||||
test -n "$DIGEST"
|
||||
echo "digest=${DIGEST}" >> "$GITHUB_OUTPUT"
|
||||
echo "base manifest: ${TAG} @ ${DIGEST}"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Full image: base + Unsloth Studio + JupyterLab + sshd (Dockerfile.studio).
|
||||
# This is what :latest points at, matching the service set of the previous
|
||||
# production image. Same by-digest build + manifest-merge pattern as the
|
||||
# base. FROMs the exact base manifest digest published by the merge job.
|
||||
# The arm64 leg builds Studio's vite frontend natively on the arm runner;
|
||||
# that is the long pole, hence the larger timeout.
|
||||
# ---------------------------------------------------------------------------
|
||||
build-studio:
|
||||
needs: merge
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- platform: linux/amd64
|
||||
runner: ubuntu-latest
|
||||
- platform: linux/arm64
|
||||
runner: ubuntu-24.04-arm
|
||||
runs-on: ${{ matrix.runner }}
|
||||
timeout-minutes: 150
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Reclaim disk
|
||||
run: |
|
||||
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc \
|
||||
/opt/hostedtoolcache/CodeQL "$AGENT_TOOLSDIRECTORY" || true
|
||||
df -h /
|
||||
|
||||
- uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Log in to Docker Hub
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Resolve labels
|
||||
id: meta
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
||||
|
||||
- name: Build and push (per-arch by digest)
|
||||
id: build
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: ./docker
|
||||
file: ./docker/Dockerfile.studio
|
||||
platforms: ${{ matrix.platform }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
cache-from: type=gha,scope=studio-${{ matrix.platform }}
|
||||
cache-to: type=gha,scope=studio-${{ matrix.platform }},mode=max
|
||||
outputs: type=image,name=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=true
|
||||
build-args: |
|
||||
BASE_IMAGE=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ needs.merge.outputs.digest }}
|
||||
# Mirror of the base job's UNSLOTH_REF resolution so the Studio
|
||||
# tree matches the unsloth baked into the base venv.
|
||||
UNSLOTH_STUDIO_REF=${{ github.event.inputs.unsloth_ref || (startsWith(github.ref, 'refs/tags/') && github.ref_name) || github.sha || 'main' }}
|
||||
|
||||
- name: Export digest
|
||||
run: |
|
||||
mkdir -p /tmp/digests
|
||||
digest='${{ steps.build.outputs.digest }}'
|
||||
touch "/tmp/digests/${digest#sha256:}"
|
||||
|
||||
- name: Upload digest
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: digests-studio-${{ matrix.platform == 'linux/amd64' && 'amd64' || 'arm64' }}
|
||||
path: /tmp/digests/*
|
||||
if-no-files-found: error
|
||||
retention-days: 1
|
||||
|
||||
merge-studio:
|
||||
runs-on: ubuntu-latest
|
||||
needs: build-studio
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
steps:
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
path: /tmp/digests
|
||||
pattern: digests-studio-*
|
||||
merge-multiple: true
|
||||
|
||||
- uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Log in to Docker Hub
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Resolve tags
|
||||
id: meta
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
||||
tags: |
|
||||
# The full Studio image owns the unprefixed namespace, headed by
|
||||
# :latest. Same :latest gating rationale as the base job.
|
||||
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && github.event.inputs.unsloth_ref == '' }}
|
||||
type=ref,event=tag
|
||||
type=schedule,pattern=nightly
|
||||
|
|
@ -220,25 +369,51 @@ jobs:
|
|||
# registered. Architecture matches whatever the runner is.
|
||||
# ---------------------------------------------------------------------------
|
||||
smoke-test:
|
||||
needs: merge
|
||||
needs: [merge, merge-studio]
|
||||
if: ${{ vars.HAS_GPU_RUNNER == 'true' }}
|
||||
runs-on: [self-hosted, gpu]
|
||||
timeout-minutes: 20
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
# Re-compute the tag list deterministically from the same metadata-action
|
||||
# config the merge job used, so tag/schedule/SHA runs pull the image
|
||||
# they just published instead of an unrelated `:latest` from a prior run.
|
||||
# IMPORTANT: keep this `enable=` expression byte-identical to the merge
|
||||
# job's :latest gate above. The two used to differ
|
||||
# they just published instead of an unrelated tag from a prior run.
|
||||
# IMPORTANT: keep the `enable=` expressions byte-identical to the
|
||||
# corresponding merge jobs' gates above. The two used to differ
|
||||
# (merge: ref + unsloth_ref guard; smoke: is_default_branch only),
|
||||
# which meant workflow_dispatch with unsloth_ref defaulting to "main"
|
||||
# would skip :latest on merge but still emit :latest as tags[0] on
|
||||
# smoke -- so docker pull would fetch a previously-published :latest
|
||||
# from Docker Hub, not the image just merged.
|
||||
- name: Resolve published tag
|
||||
id: meta
|
||||
- name: Resolve published base tag
|
||||
id: meta_base
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
||||
tags: |
|
||||
type=raw,value=base,enable=${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && github.event.inputs.unsloth_ref == '' }}
|
||||
type=ref,event=tag,prefix=base-
|
||||
type=schedule,pattern=base-nightly
|
||||
type=sha,prefix=base-sha-,format=short
|
||||
|
||||
- name: Pull and smoke-test the base image
|
||||
run: |
|
||||
# Use the first tag from the metadata output -- that is the image we
|
||||
# just published. Falls back to :base only when the metadata is
|
||||
# empty (defensive; should not happen on default-branch runs).
|
||||
TAG="$(jq -r '.tags[0] // ""' <<<"$STEPS_META_BASE_JSON")"
|
||||
if [ -z "$TAG" ]; then
|
||||
TAG="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:base"
|
||||
fi
|
||||
echo "smoke-testing $TAG"
|
||||
docker pull "$TAG"
|
||||
docker run --rm --gpus all "$TAG" python /workspace/smoke_test.py
|
||||
env:
|
||||
STEPS_META_BASE_JSON: ${{ steps.meta_base.outputs.json }}
|
||||
|
||||
- name: Resolve published studio tag
|
||||
id: meta_studio
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
||||
|
|
@ -248,15 +423,25 @@ jobs:
|
|||
type=schedule,pattern=nightly
|
||||
type=sha,prefix=sha-,format=short
|
||||
|
||||
- name: Pull and smoke-test
|
||||
- name: Boot the full image and probe Studio + Jupyter
|
||||
run: |
|
||||
# Use the first tag from the metadata output -- that is the image we
|
||||
# just published. Falls back to :latest only when the metadata is
|
||||
# empty (defensive; should not happen on default-branch runs).
|
||||
TAG="$(jq -r '.tags[0] // ""' <<<"$DOCKER_METADATA_OUTPUT_JSON")"
|
||||
TAG="$(jq -r '.tags[0] // ""' <<<"$STEPS_META_STUDIO_JSON")"
|
||||
if [ -z "$TAG" ]; then
|
||||
TAG="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest"
|
||||
fi
|
||||
echo "smoke-testing $TAG"
|
||||
echo "booting $TAG"
|
||||
docker pull "$TAG"
|
||||
docker run --rm --gpus all "$TAG" python /workspace/smoke_test.py
|
||||
CID="$(docker run -d --gpus all -p 18000:8000 -p 18888:8888 "$TAG")"
|
||||
trap 'docker logs --tail 100 "$CID"; docker rm -f "$CID"' EXIT
|
||||
ok_studio=0; ok_jupyter=0
|
||||
for i in $(seq 1 60); do
|
||||
if curl -fsS http://localhost:18000/api/health >/dev/null 2>&1; then ok_studio=1; fi
|
||||
if curl -fsS http://localhost:18888/api >/dev/null 2>&1; then ok_jupyter=1; fi
|
||||
[ "$ok_studio" = 1 ] && [ "$ok_jupyter" = 1 ] && break
|
||||
sleep 5
|
||||
done
|
||||
[ "$ok_studio" = 1 ] || { echo "Studio /api/health never went healthy"; exit 1; }
|
||||
[ "$ok_jupyter" = 1 ] || { echo "Jupyter /api never responded"; exit 1; }
|
||||
echo "Studio + Jupyter healthy"
|
||||
env:
|
||||
STEPS_META_STUDIO_JSON: ${{ steps.meta_studio.outputs.json }}
|
||||
|
|
|
|||
|
|
@ -222,6 +222,18 @@ RUN set -eux \
|
|||
echo ">> vLLM skipped (INSTALL_VLLM=${INSTALL_VLLM}, TARGETARCH=${TARGETARCH:-amd64})"; \
|
||||
fi
|
||||
|
||||
# JupyterLab so the published image runs unslothai/notebooks out of the box:
|
||||
# docker run --gpus all -p 8888:8888 unsloth/unsloth \
|
||||
# jupyter lab --ip 0.0.0.0 --port 8888 --allow-root --no-browser
|
||||
# Installed as a separate pass AFTER the torch-pinned resolves on purpose:
|
||||
# jupyterlab's dependency closure is pure-Python (tornado, jinja2, nbconvert,
|
||||
# nbclient, ipykernel, ...) and never names torch, so uv cannot disturb the
|
||||
# cu128 pin set here. Naming torch in this pass would be actively dangerous:
|
||||
# without the cu128 extra index uv could swap in the PyPI CPU wheel.
|
||||
RUN ${VENV}/bin/uv pip install \
|
||||
--python ${VENV}/bin/python \
|
||||
jupyterlab notebook ipywidgets
|
||||
|
||||
# 5) Emit an informational pin record so downstream consumers can see exactly
|
||||
# what was resolved. This is NOT a byte-reproducible lockfile -- `pip freeze`
|
||||
# captures version strings but not wheel hashes, and several deps (unsloth,
|
||||
|
|
@ -408,9 +420,51 @@ RUN if [ "${TARGETARCH:-amd64}" = "arm64" ]; then \
|
|||
fi; \
|
||||
fi
|
||||
|
||||
# Prebuilt llama.cpp so GGUF export works out of the box.
|
||||
#
|
||||
# unsloth_zoo's save_pretrained_gguf() calls check_llama_cpp(), which looks
|
||||
# for llama-quantize + convert_hf_to_gguf.py in $UNSLOTH_LLAMA_CPP_PATH
|
||||
# (default ~/.unsloth/llama.cpp). Without a baked install the first GGUF
|
||||
# export inside the container would hit install_llama_cpp()'s interactive
|
||||
# prompt and then a slow source build. We reuse Studio's own resolver
|
||||
# (studio/install_llama_prebuilt.py at the same UNSLOTH_REF baked into the
|
||||
# venv) to fetch the matching prebuilt from unslothai/llama.cpp releases:
|
||||
# * sha256-verified against the release's llama-prebuilt-sha256.json
|
||||
# * no GPU on the build host -> the resolver picks the PORTABLE CUDA
|
||||
# bundle, which carries its own CUDA runtime libs and runs on every
|
||||
# supported arch at container runtime (same reasoning as the wheels)
|
||||
# * amd64 -> app-<tag>-linux-x64-cuda12-portable.tar.gz
|
||||
# arm64 -> the linux-arm64-cuda13 bundle (DGX Spark / Grace)
|
||||
# * the binaries + convert script land at the install dir ROOT, which is
|
||||
# exactly the layout check_llama_cpp() expects
|
||||
# /opt (not /root) so the install survives a `docker run --user` override;
|
||||
# UNSLOTH_LLAMA_CPP_PATH makes zoo find it regardless of $HOME.
|
||||
#
|
||||
# The optional BuildKit secret raises the GitHub API rate limit on busy CI
|
||||
# runners (the resolver reads GITHUB_TOKEN); local builds work without it.
|
||||
ARG UNSLOTH_REF=main
|
||||
ADD https://raw.githubusercontent.com/unslothai/unsloth/${UNSLOTH_REF}/studio/install_llama_prebuilt.py /tmp/install_llama_prebuilt.py
|
||||
RUN --mount=type=secret,id=github_token \
|
||||
set -eux \
|
||||
&& if [ -s /run/secrets/github_token ]; then \
|
||||
export GITHUB_TOKEN="$(cat /run/secrets/github_token)"; \
|
||||
fi \
|
||||
&& /opt/unsloth-venv/bin/python /tmp/install_llama_prebuilt.py \
|
||||
--install-dir /opt/unsloth/llama.cpp \
|
||||
&& rm -f /tmp/install_llama_prebuilt.py \
|
||||
&& test -x /opt/unsloth/llama.cpp/llama-quantize \
|
||||
&& test -x /opt/unsloth/llama.cpp/llama-server \
|
||||
&& test -f /opt/unsloth/llama.cpp/convert_hf_to_gguf.py \
|
||||
&& cat /opt/unsloth/llama.cpp/UNSLOTH_PREBUILT_INFO.json
|
||||
ENV UNSLOTH_LLAMA_CPP_PATH=/opt/unsloth/llama.cpp
|
||||
|
||||
WORKDIR /workspace
|
||||
RUN mkdir -p ${HF_HOME} ${TRITON_CACHE_DIR}
|
||||
|
||||
# JupyterLab lives in the venv (see builder stage). Persistent notebooks
|
||||
# should be bind-mounted onto /workspace.
|
||||
EXPOSE 8888
|
||||
|
||||
COPY smoke_test.py /workspace/smoke_test.py
|
||||
COPY entrypoint.sh /usr/local/bin/unsloth-entrypoint
|
||||
RUN chmod +x /usr/local/bin/unsloth-entrypoint
|
||||
|
|
|
|||
|
|
@ -1,40 +1,48 @@
|
|||
# Unsloth Studio variant of the Blackwell image.
|
||||
# Full Unsloth image: base training stack + Studio + JupyterLab + sshd.
|
||||
#
|
||||
# Builds on top of unsloth-blackwell:<tag> (default `test`) and runs the
|
||||
# upstream `install.sh --local` so the Studio CLI can re-exec into its
|
||||
# own venv under $UNSLOTH_STUDIO_HOME. The base image already ships the
|
||||
# `unsloth` Python CLI, but `unsloth studio` refuses to start until that
|
||||
# venv exists; install.sh is the canonical way to lay it down.
|
||||
# This is the image published as docker.io/unsloth/unsloth:latest. It layers
|
||||
# Unsloth Studio on top of the lean base image (Dockerfile, published under
|
||||
# the `base` tags) and runs the same service trio as the previous production
|
||||
# image: Studio on 8000, JupyterLab on 8888, key-only sshd on 22.
|
||||
#
|
||||
# Build:
|
||||
# Build (local):
|
||||
# docker buildx build \
|
||||
# --build-arg BASE_TAG=test \
|
||||
# --build-arg BASE_IMAGE=unsloth-blackwell:test \
|
||||
# -f docker/Dockerfile.studio \
|
||||
# -t unsloth-blackwell:studio docker/
|
||||
#
|
||||
# Run:
|
||||
# docker run --rm --gpus '"device=0"' -p 8888:8888 \
|
||||
# docker run --rm --gpus all -p 8000:8000 -p 8888:8888 \
|
||||
# -v $HOME/.cache/huggingface:/workspace/.cache/huggingface \
|
||||
# unsloth-blackwell:studio
|
||||
#
|
||||
# Open http://localhost:8888 . First-boot admin password is printed in the
|
||||
# container logs and persisted under /opt/unsloth-studio/auth/.bootstrap_password.
|
||||
# Open http://localhost:8000 for Studio (first-boot admin password is printed
|
||||
# in the container logs and persisted under /opt/unsloth-studio/auth/) and
|
||||
# http://localhost:8888 for JupyterLab (password: JUPYTER_PASSWORD env,
|
||||
# default `unsloth`). On hosts without GPU passthrough (Docker Desktop on
|
||||
# macOS, Windows without WSL2 GPU) add -e UNSLOTH_ALLOW_CPU=1: training is
|
||||
# unavailable but Studio chat / Data Recipes / GGUF tooling / Jupyter work.
|
||||
#
|
||||
# CI pins BASE_IMAGE to the just-published multi-arch base digest so the two
|
||||
# images always ship the same stack.
|
||||
|
||||
ARG BASE_TAG=test
|
||||
FROM unsloth-blackwell:${BASE_TAG}
|
||||
ARG BASE_IMAGE=unsloth-blackwell:test
|
||||
FROM ${BASE_IMAGE}
|
||||
|
||||
# Studio source ref to clone. Defaults to `main`, but a CI publish pipeline
|
||||
# that pins BASE_TAG to a tag/SHA should pin this too so the published
|
||||
# `:studio` companion image is reproducible against a known unsloth ref.
|
||||
# that pins BASE_IMAGE to a digest should pin this too (same UNSLOTH_REF as
|
||||
# the base) so the published image is reproducible against a known ref.
|
||||
ARG UNSLOTH_STUDIO_REF=main
|
||||
|
||||
USER root
|
||||
ENV UNSLOTH_STUDIO_HOME=/opt/unsloth-studio \
|
||||
DEBIAN_FRONTEND=noninteractive
|
||||
|
||||
# install.sh needs curl + git; the base image already has python + uv + pip.
|
||||
# install.sh needs curl + git; supervisor + openssh-server run the service
|
||||
# trio. The base image already has python + uv + pip.
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends curl git ca-certificates \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
curl git ca-certificates supervisor openssh-server \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Clone + install Studio into a dedicated venv under $UNSLOTH_STUDIO_HOME.
|
||||
|
|
@ -43,16 +51,32 @@ RUN apt-get update \
|
|||
# entrypoint to keep resolving. Move it under $UNSLOTH_STUDIO_HOME/src
|
||||
# (already inside the persistent layer) instead of deleting it. Strip
|
||||
# .git to save ~120MB.
|
||||
#
|
||||
# The llama.cpp symlink BEFORE install.sh points Studio's prebuilt dir at
|
||||
# the bundle already baked into the base image (validated, sha256-checked,
|
||||
# UNSLOTH_PREBUILT_INFO.json present), so the installer's prebuilt step
|
||||
# recognises it and skips a second ~400MB download.
|
||||
# fetch+checkout FETCH_HEAD instead of `clone --branch` because the CI
|
||||
# pipeline passes a commit SHA as the ref (clone --branch only accepts
|
||||
# branch/tag names).
|
||||
RUN mkdir -p "${UNSLOTH_STUDIO_HOME}" \
|
||||
&& git clone --depth 1 --branch "${UNSLOTH_STUDIO_REF}" https://github.com/unslothai/unsloth "${UNSLOTH_STUDIO_HOME}/src" \
|
||||
&& ln -s /opt/unsloth/llama.cpp "${UNSLOTH_STUDIO_HOME}/llama.cpp" \
|
||||
&& git init -q "${UNSLOTH_STUDIO_HOME}/src" \
|
||||
&& cd "${UNSLOTH_STUDIO_HOME}/src" \
|
||||
&& git remote add origin https://github.com/unslothai/unsloth \
|
||||
&& git fetch -q --depth 1 origin "${UNSLOTH_STUDIO_REF}" \
|
||||
&& git checkout -q FETCH_HEAD \
|
||||
&& UNSLOTH_STUDIO_HOME="${UNSLOTH_STUDIO_HOME}" bash install.sh --local \
|
||||
&& rm -rf "${UNSLOTH_STUDIO_HOME}/src/.git" /root/.cache
|
||||
|
||||
# Expose Studio's HTTP port. Default CMD binds 0.0.0.0 because containers
|
||||
# isolate the namespace; the operator publishes it explicitly with `-p`.
|
||||
EXPOSE 8888
|
||||
COPY supervisord.conf /etc/supervisor/supervisord.conf
|
||||
COPY studio_launch.sh /usr/local/bin/unsloth-studio-launch
|
||||
RUN chmod +x /usr/local/bin/unsloth-studio-launch
|
||||
|
||||
# Use the Studio launcher in the dedicated venv; -H 0.0.0.0 binds inside
|
||||
# the container only and is fine for typical local docker workflows.
|
||||
CMD ["sh", "-c", "${UNSLOTH_STUDIO_HOME}/bin/unsloth studio -H 0.0.0.0 -p 8888"]
|
||||
# Studio web UI, JupyterLab, sshd. All bind 0.0.0.0 inside the container's
|
||||
# network namespace; the operator publishes them explicitly with -p.
|
||||
EXPOSE 8000 8888 22
|
||||
|
||||
# The base ENTRYPOINT (unsloth-entrypoint) still runs its GPU pre-flight
|
||||
# first, then hands off to the service launcher.
|
||||
CMD ["/usr/local/bin/unsloth-studio-launch"]
|
||||
|
|
|
|||
|
|
@ -32,6 +32,22 @@ fi
|
|||
err() { printf "\033[1;31mERROR:\033[0m %s\n" "$*" >&2; }
|
||||
warn() { printf "\033[1;33mWARN:\033[0m %s\n" "$*" >&2; }
|
||||
|
||||
# CPU mode for hosts that cannot pass a GPU into a Linux container at all:
|
||||
# Docker Desktop on macOS (no Metal passthrough), Docker Desktop on Windows
|
||||
# without WSL2 GPU support, plain CPU Linux boxes, and CI runners. Training
|
||||
# needs an NVIDIA GPU, but Jupyter, GGUF tooling (the baked llama.cpp), and
|
||||
# Studio chat / Data Recipes all work on CPU. With UNSLOTH_ALLOW_CPU=1 a
|
||||
# missing GPU degrades to a warning instead of the hard pre-flight failure;
|
||||
# when a GPU IS visible the normal checks below still run so a broken GPU
|
||||
# setup is not silently ignored.
|
||||
if [[ "${UNSLOTH_ALLOW_CPU:-0}" == "1" ]]; then
|
||||
if ! command -v nvidia-smi >/dev/null 2>&1 || ! nvidia-smi -L 2>/dev/null | grep -q '^GPU'; then
|
||||
warn "UNSLOTH_ALLOW_CPU=1 and no GPU visible -- continuing on CPU."
|
||||
warn "Training requires an NVIDIA GPU. CPU mode covers Jupyter, GGUF tooling and Studio chat."
|
||||
exec "$@"
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- Check 1: nvidia-smi present and can enumerate at least one GPU ---------
|
||||
if ! command -v nvidia-smi >/dev/null 2>&1; then
|
||||
err "nvidia-smi not found inside the container."
|
||||
|
|
|
|||
|
|
@ -23,9 +23,25 @@
|
|||
# ($PWD is mounted at
|
||||
# /workspace/host)
|
||||
#
|
||||
# The full image (unsloth/unsloth:latest) starts Studio (8000) + JupyterLab
|
||||
# (8888) by default; publish the ports when you want them:
|
||||
# UNSLOTH_PORTS="-p 8000:8000 -p 8888:8888" bash docker/run.sh
|
||||
# JupyterLab on the lean base image (unsloth/unsloth:base):
|
||||
# UNSLOTH_PORTS="-p 8888:8888" UNSLOTH_IMAGE=unsloth/unsloth:base \
|
||||
# bash docker/run.sh jupyter lab --ip 0.0.0.0 --port 8888 --allow-root
|
||||
# CPU-only hosts (Docker Desktop on macOS, Windows without WSL2 GPU, plain
|
||||
# CPU Linux): no --gpus and set UNSLOTH_ALLOW_CPU=1. Training is unavailable
|
||||
# but Studio chat / Data Recipes, Jupyter and GGUF tooling work:
|
||||
# UNSLOTH_GPUS=none UNSLOTH_ALLOW_CPU=1 \
|
||||
# UNSLOTH_PORTS="-p 8000:8000 -p 8888:8888" bash docker/run.sh
|
||||
#
|
||||
# Overridable env:
|
||||
# UNSLOTH_IMAGE=unsloth/unsloth:latest image and tag to pull/run
|
||||
# UNSLOTH_GPUS=all GPUs to expose ("all" | "0" | "0,1")
|
||||
# UNSLOTH_GPUS=all GPUs to expose ("all" | "0" | "0,1"
|
||||
# | "none" to run without GPU)
|
||||
# UNSLOTH_ALLOW_CPU= set to 1 to allow GPU-less runs
|
||||
# UNSLOTH_PORTS= extra -p publish flags, e.g.
|
||||
# "-p 8000:8000 -p 8888:8888"
|
||||
# HF_HOME=$HOME/.cache/huggingface host HF cache dir to mount
|
||||
# TRITON_CACHE_DIR=$HOME/.cache/unsloth-triton
|
||||
# host Triton cache dir to mount
|
||||
|
|
@ -39,11 +55,14 @@ GPUS="${UNSLOTH_GPUS:-all}"
|
|||
# integer for --gpus as a COUNT, not an INDEX, so `UNSLOTH_GPUS=0` would
|
||||
# expose zero GPUs and the entrypoint would refuse to start. `all` and
|
||||
# already-quoted `device=...` / `"device=..."` selectors pass through.
|
||||
# "none" omits --gpus entirely (CPU mode; pair with UNSLOTH_ALLOW_CPU=1).
|
||||
GPU_FLAG=(--gpus "$GPUS")
|
||||
case "$GPUS" in
|
||||
all|"") ;;
|
||||
\"device=*|device=*) ;;
|
||||
*[!0-9]*) GPUS="\"device=${GPUS}\"" ;; # contains a non-digit (comma, UUID-prefix, etc.)
|
||||
*) GPUS="\"device=${GPUS}\"" ;; # bare integer: treat as an INDEX, per docstring
|
||||
none) GPU_FLAG=() ;;
|
||||
all|"") ;;
|
||||
\"device=*|device=*) ;;
|
||||
*[!0-9]*) GPU_FLAG=(--gpus "\"device=${GPUS}\"") ;; # comma list / UUID
|
||||
*) GPU_FLAG=(--gpus "\"device=${GPUS}\"") ;; # bare integer index
|
||||
esac
|
||||
HF_CACHE="${HF_HOME:-$HOME/.cache/huggingface}"
|
||||
TRITON_CACHE="${TRITON_CACHE_DIR:-$HOME/.cache/unsloth-triton}"
|
||||
|
|
@ -68,9 +87,17 @@ fi
|
|||
# `ps auxe` / `/proc/<pid>/cmdline` for the lifetime of the docker CLI
|
||||
# process.
|
||||
declare -a ENV_FORWARD=(-e HF_HUB_ENABLE_HF_TRANSFER=1)
|
||||
[[ -n "${HF_TOKEN:-}" ]] && ENV_FORWARD+=(-e HF_TOKEN)
|
||||
[[ -n "${WANDB_API_KEY:-}" ]] && ENV_FORWARD+=(-e WANDB_API_KEY)
|
||||
[[ -n "${UNSLOTH_LICENSE:-}" ]] && ENV_FORWARD+=(-e UNSLOTH_LICENSE)
|
||||
[[ -n "${HF_TOKEN:-}" ]] && ENV_FORWARD+=(-e HF_TOKEN)
|
||||
[[ -n "${WANDB_API_KEY:-}" ]] && ENV_FORWARD+=(-e WANDB_API_KEY)
|
||||
[[ -n "${UNSLOTH_LICENSE:-}" ]] && ENV_FORWARD+=(-e UNSLOTH_LICENSE)
|
||||
[[ -n "${UNSLOTH_ALLOW_CPU:-}" ]] && ENV_FORWARD+=(-e UNSLOTH_ALLOW_CPU)
|
||||
|
||||
# Extra publish flags for the service ports (Studio 8000, Jupyter 8888).
|
||||
declare -a PORT_FLAGS=()
|
||||
if [[ -n "${UNSLOTH_PORTS:-}" ]]; then
|
||||
# shellcheck disable=SC2206 # intentional word splitting of "-p X -p Y"
|
||||
PORT_FLAGS=(${UNSLOTH_PORTS})
|
||||
fi
|
||||
|
||||
# Only attach -t when our own stdin/stdout are a TTY; CI / piped invocations
|
||||
# otherwise hit `the input device is not a TTY` and never reach the entrypoint.
|
||||
|
|
@ -83,7 +110,7 @@ fi
|
|||
# values do not get echoed to stdout/CI logs. The forwarded env vars are
|
||||
# already in ENV_FORWARD; printing them again was a secret leak.
|
||||
exec docker run --rm "${TTY_FLAG[@]}" \
|
||||
--gpus "$GPUS" \
|
||||
"${GPU_FLAG[@]}" \
|
||||
--ipc=host \
|
||||
--ulimit memlock=-1 \
|
||||
--ulimit stack=67108864 \
|
||||
|
|
@ -91,4 +118,5 @@ exec docker run --rm "${TTY_FLAG[@]}" \
|
|||
-v "$TRITON_CACHE":/workspace/.cache/triton \
|
||||
-v "$WORK_DIR":/workspace/host \
|
||||
"${ENV_FORWARD[@]}" \
|
||||
"${PORT_FLAGS[@]}" \
|
||||
"$IMAGE" "$@"
|
||||
|
|
|
|||
65
docker/studio_launch.sh
Normal file
65
docker/studio_launch.sh
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
#!/usr/bin/env bash
|
||||
# Default CMD of the full Unsloth image (Dockerfile.studio).
|
||||
#
|
||||
# Bootstraps the three services managed by supervisord:
|
||||
# studio port 8000 first-boot admin password printed in `docker logs`
|
||||
# jupyter port 8888 password from JUPYTER_PASSWORD (default: unsloth)
|
||||
# sshd port 22 key-only; enabled when PUBLIC_KEY / SSH_KEY is set
|
||||
#
|
||||
# Environment:
|
||||
# JUPYTER_PORT Jupyter port inside the container (default 8888)
|
||||
# JUPYTER_PASSWORD Jupyter login password (default unsloth)
|
||||
# PUBLIC_KEY/SSH_KEY OpenSSH public key for root login; sshd stays disabled
|
||||
# when neither is set (nothing to authenticate with --
|
||||
# password login is never enabled for root)
|
||||
set -euo pipefail
|
||||
|
||||
export JUPYTER_PORT="${JUPYTER_PORT:-8888}"
|
||||
export UNSLOTH_STUDIO_HOME="${UNSLOTH_STUDIO_HOME:-/opt/unsloth-studio}"
|
||||
|
||||
# Make the runtime env visible to SSH sessions, which get a fresh login shell
|
||||
# without the `docker run -e` vars. Same pattern as the production image.
|
||||
printenv | grep -E '^(HF_|CUDA_|NCCL_|JUPYTER_|UNSLOTH_|WANDB_|PATH=|TRITON_)' | \
|
||||
sed 's/^\([^=]*\)=\(.*\)$/export \1="\2"/' > /etc/profile.d/unsloth_env.sh || true
|
||||
|
||||
# --- Jupyter -----------------------------------------------------------------
|
||||
# Hash the password with jupyter's own helper; never store the plaintext.
|
||||
JUPYTER_CONFIG_DIR=/root/.jupyter
|
||||
if [[ ! -f "${JUPYTER_CONFIG_DIR}/jupyter_lab_config.py" ]]; then
|
||||
mkdir -p "${JUPYTER_CONFIG_DIR}"
|
||||
HASH=$(python - <<PY
|
||||
from jupyter_server.auth import passwd
|
||||
import os
|
||||
print(passwd(os.environ.get("JUPYTER_PASSWORD", "unsloth")))
|
||||
PY
|
||||
)
|
||||
cat > "${JUPYTER_CONFIG_DIR}/jupyter_lab_config.py" <<EOF
|
||||
c.ServerApp.ip = "0.0.0.0"
|
||||
c.ServerApp.open_browser = False
|
||||
c.ServerApp.root_dir = "/workspace"
|
||||
c.PasswordIdentityProvider.hashed_password = "${HASH}"
|
||||
EOF
|
||||
fi
|
||||
|
||||
# --- sshd (opt-in) -----------------------------------------------------------
|
||||
# Enabled only when a public key is provided; root password login is never
|
||||
# allowed. Cloud GPU platforms (e.g. runpod-style hosts) inject PUBLIC_KEY.
|
||||
PUBLIC_SSH_KEY="${SSH_KEY:-${PUBLIC_KEY:-}}"
|
||||
export UNSLOTH_ENABLE_SSHD=false
|
||||
if [[ -n "${PUBLIC_SSH_KEY}" ]] && command -v sshd >/dev/null 2>&1; then
|
||||
mkdir -p /root/.ssh && chmod 700 /root/.ssh
|
||||
echo "${PUBLIC_SSH_KEY}" > /root/.ssh/authorized_keys
|
||||
chmod 600 /root/.ssh/authorized_keys
|
||||
ssh-keygen -A
|
||||
mkdir -p /run/sshd
|
||||
export UNSLOTH_ENABLE_SSHD=true
|
||||
fi
|
||||
|
||||
mkdir -p /workspace
|
||||
echo "Unsloth Studio -> http://localhost:8000 (first-boot password below)"
|
||||
echo "JupyterLab -> http://localhost:${JUPYTER_PORT} (password: JUPYTER_PASSWORD env, default 'unsloth')"
|
||||
if [[ "${UNSLOTH_ENABLE_SSHD}" == "true" ]]; then
|
||||
echo "sshd -> port 22 (key-only)"
|
||||
fi
|
||||
|
||||
exec supervisord -c /etc/supervisor/supervisord.conf
|
||||
58
docker/supervisord.conf
Normal file
58
docker/supervisord.conf
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
# Service manager for the full Unsloth image (Dockerfile.studio).
|
||||
#
|
||||
# Mirrors the service set of the production docker.io/unsloth/unsloth image:
|
||||
# studio Unsloth Studio web UI port 8000
|
||||
# jupyter JupyterLab for the notebooks port $JUPYTER_PORT (default 8888)
|
||||
# sshd key-only SSH for cloud hosts port 22
|
||||
#
|
||||
# All three log to the container's stdout/stderr (the Docker-native pattern)
|
||||
# so `docker logs` shows everything, including Studio's first-boot password
|
||||
# and Jupyter's startup line.
|
||||
|
||||
[unix_http_server]
|
||||
file=/run/supervisor.sock
|
||||
chmod=0700
|
||||
|
||||
[supervisorctl]
|
||||
serverurl=unix:///run/supervisor.sock
|
||||
|
||||
[rpcinterface:supervisor]
|
||||
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
|
||||
|
||||
[supervisord]
|
||||
nodaemon=true
|
||||
pidfile=/run/supervisord.pid
|
||||
logfile=/dev/null
|
||||
logfile_maxbytes=0
|
||||
loglevel=info
|
||||
|
||||
[program:studio]
|
||||
command=%(ENV_UNSLOTH_STUDIO_HOME)s/bin/unsloth studio -H 0.0.0.0 -p 8000
|
||||
directory=/workspace
|
||||
autostart=true
|
||||
autorestart=true
|
||||
startretries=3
|
||||
startsecs=5
|
||||
stdout_logfile=/dev/stdout
|
||||
stdout_logfile_maxbytes=0
|
||||
stderr_logfile=/dev/stderr
|
||||
stderr_logfile_maxbytes=0
|
||||
|
||||
[program:jupyter]
|
||||
command=jupyter lab --no-browser --ip=0.0.0.0 --port=%(ENV_JUPYTER_PORT)s --allow-root --notebook-dir=/workspace
|
||||
directory=/workspace
|
||||
autostart=true
|
||||
autorestart=true
|
||||
stdout_logfile=/dev/stdout
|
||||
stdout_logfile_maxbytes=0
|
||||
stderr_logfile=/dev/stderr
|
||||
stderr_logfile_maxbytes=0
|
||||
|
||||
[program:sshd]
|
||||
command=/usr/sbin/sshd -D -e
|
||||
autostart=%(ENV_UNSLOTH_ENABLE_SSHD)s
|
||||
autorestart=true
|
||||
stdout_logfile=/dev/stdout
|
||||
stdout_logfile_maxbytes=0
|
||||
stderr_logfile=/dev/stderr
|
||||
stderr_logfile_maxbytes=0
|
||||
Loading…
Add table
Add a link
Reference in a new issue