58693c4c73
When someone launches the unsloth container, the common failure modes are not unsloth bugs -- they're Docker / nvidia-container-toolkit / driver issues that surface as cryptic CUDA errors deep in torch. The entrypoint catches the three that cover ~95% of "it doesn't work" reports up front: 1. nvidia-smi inside the container sees no GPU -> user forgot --gpus all, or host is missing nvidia-container-toolkit -> entrypoint prints the exact docker run flag and the toolkit install URL 2. nvidia-smi works but torch.cuda.is_available() is False -> host driver is older than CUDA 12.8 supports -> entrypoint prints the minimum driver version per architecture 3. compute capability < sm_80 -> entrypoint prints the supported architecture table and exits Each check fails with a clear, actionable message rather than a stack trace. Set UNSLOTH_SKIP_GPU_CHECK=1 to bypass (for docs builds, offline tooling, CI). run.sh wraps `docker run` with the flags people most often forget: --gpus all (without it, the new entrypoint refuses to start) --ipc=host (DataLoader workers need >64MB shm) --ulimit memlock=-1 (NCCL + CUDA pinned host buffers) --ulimit stack=64MB (some torch kernels OOM the default 8MB stack) Plus it mounts the host HF cache + Triton JIT cache so model downloads and compiled kernels persist across container runs, and forwards HF_TOKEN / WANDB_API_KEY / UNSLOTH_LICENSE only when they are set on the host. Usage: bash docker/run.sh # interactive python REPL bash docker/run.sh bash # shell in container bash docker/run.sh python /workspace/smoke_test.py bash docker/run.sh python /workspace/host/train.py # $PWD mounted at /workspace/host Verified locally: - No GPU visible: entrypoint refuses with driver-version message, exit 1 - B200 sm_100 visible: entrypoint prints GPU banner, exits cleanly into the user command (rc=0)
69 lines
3.4 KiB
Bash
Executable file
69 lines
3.4 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# Convenience wrapper for `docker run unsloth/unsloth`. Sets the flags that
|
|
# people most often forget and that cause the most confusing failures:
|
|
#
|
|
# --gpus all Without this, no GPU is attached and the container's
|
|
# entrypoint will refuse to start.
|
|
# --ipc=host PyTorch DataLoader workers need ample /dev/shm. The
|
|
# default 64MB causes "DataLoader worker (pid X) exited
|
|
# unexpectedly" on any non-trivial dataset.
|
|
# --ulimit memlock=-1 Unlimited pinned memory for NCCL / CUDA pinned host
|
|
# buffers. Without this, multi-GPU training stalls.
|
|
# --ulimit stack=64MB Larger thread stack for libtorch (some kernels OOM
|
|
# the default 8MB stack).
|
|
#
|
|
# Plus mounts the host Hugging Face cache and Triton JIT cache so model
|
|
# downloads and compiled kernels persist across container runs.
|
|
#
|
|
# Usage:
|
|
# bash docker/run.sh # interactive python REPL
|
|
# bash docker/run.sh bash # shell in the container
|
|
# bash docker/run.sh python /workspace/smoke_test.py # run the smoke test
|
|
# bash docker/run.sh python /workspace/host/train.py # run your training script
|
|
# ($PWD is mounted at
|
|
# /workspace/host)
|
|
#
|
|
# Overridable env:
|
|
# UNSLOTH_IMAGE=unsloth/unsloth:latest image and tag to pull/run
|
|
# UNSLOTH_GPUS=all GPUs to expose ("all" | "0" | "0,1")
|
|
# HF_HOME=$HOME/.cache/huggingface host HF cache dir to mount
|
|
# TRITON_CACHE_DIR=$HOME/.cache/unsloth-triton
|
|
# host Triton cache dir to mount
|
|
# UNSLOTH_WORKDIR=$PWD host dir mounted at /workspace/host
|
|
set -euo pipefail
|
|
|
|
IMAGE="${UNSLOTH_IMAGE:-unsloth/unsloth:latest}"
|
|
GPUS="${UNSLOTH_GPUS:-all}"
|
|
HF_CACHE="${HF_HOME:-$HOME/.cache/huggingface}"
|
|
TRITON_CACHE="${TRITON_CACHE_DIR:-$HOME/.cache/unsloth-triton}"
|
|
WORK_DIR="${UNSLOTH_WORKDIR:-$PWD}"
|
|
|
|
mkdir -p "$HF_CACHE" "$TRITON_CACHE"
|
|
|
|
# Warn early if the host doesn't have the nvidia runtime registered.
|
|
# We let `docker run` fail loudly rather than abort here -- some setups
|
|
# (rootless docker, custom runtimes) report runtimes differently.
|
|
if ! docker info 2>/dev/null | grep -qi 'Runtimes:.*nvidia'; then
|
|
printf "\033[1;33mWARN:\033[0m 'docker info' does not list 'nvidia' as a runtime.\n" >&2
|
|
printf " If --gpus all fails below, install nvidia-container-toolkit:\n" >&2
|
|
printf " https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html\n\n" >&2
|
|
fi
|
|
|
|
# Forward common secrets only if they're set in the host environment.
|
|
# Empty strings would shadow whatever is already inside the image.
|
|
declare -a ENV_FORWARD=(-e HF_HUB_ENABLE_HF_TRANSFER=1)
|
|
[[ -n "${HF_TOKEN:-}" ]] && ENV_FORWARD+=(-e "HF_TOKEN=${HF_TOKEN}")
|
|
[[ -n "${WANDB_API_KEY:-}" ]] && ENV_FORWARD+=(-e "WANDB_API_KEY=${WANDB_API_KEY}")
|
|
[[ -n "${UNSLOTH_LICENSE:-}" ]] && ENV_FORWARD+=(-e "UNSLOTH_LICENSE=${UNSLOTH_LICENSE}")
|
|
|
|
set -x
|
|
exec docker run --rm -it \
|
|
--gpus "$GPUS" \
|
|
--ipc=host \
|
|
--ulimit memlock=-1 \
|
|
--ulimit stack=67108864 \
|
|
-v "$HF_CACHE":/workspace/.cache/huggingface \
|
|
-v "$TRITON_CACHE":/workspace/.cache/triton \
|
|
-v "$WORK_DIR":/workspace/host \
|
|
"${ENV_FORWARD[@]}" \
|
|
"$IMAGE" "$@"
|