0d574d8161
Round-2 of the 12-persona reviewer.py pass found 17 issues. Address the
P1s + the regression-class P2s in this commit; the remaining nits are
left for a follow-up cleanup pass.
1. unsloth/_gpu_init.py: the `NVIDIA_VISIBLE_DEVICES in os.environ` check
triggered for every NVIDIA-runtime container including `--gpus all`
(NVIDIA_VISIBLE_DEVICES=all is the default). Gate strictly on a
non-special device list. Also drop the precondition that the env var
was absent: if the user already pinned TORCHINDUCTOR_COMPILE_THREADS=1
we should still plant the UNSLOTH_FORCE_SINGLE_COMPILE_WORKER sentinel
so the zoo-side patch knows to preserve the forcing.
2. unsloth/_gpu_init.py: after the post-`import unsloth_zoo` reassertion,
monkey-patch `unsloth_zoo.temporary_patches.common.determine_compile_threads`
to return 1, so any later `torch.compile` call that rebuilds the
options dict still sees the single-worker forcing even if a downstream
patch_torch_compile pops the env var again.
3. docker/Dockerfile: torchaudio==2.11.0 mismatched the torch==2.10.0
release pairing; pin to 2.10.0 so the ABI is correct and the audio
stack matches torch/cu128.
4. docker/Dockerfile: drop `12.1+PTX` from TORCH_CUDA_ARCH_LIST. The
cu128 toolkit compiler does not know about compute_121; the trailing
PTX entry forced nvcc to emit a `sm_121` gencode that breaks any
in-container source builds.
5. docker/smoke_test.py: the device-capability floor said `cap[0] < 8`,
rejecting Turing (sm_75) while the Dockerfile + entrypoint advertise
sm_75 as supported. Lower the smoke floor to sm_75 and print a hint
that bf16 is not available on Turing.
6. docker/run.sh: `-it` is unconditional; CI / non-TTY invocations died
with "the input device is not a TTY". Probe `[ -t 0 ] && [ -t 1 ]`
first. Also remove `set -x` which echoed the forwarded HF_TOKEN /
WANDB_API_KEY / UNSLOTH_LICENSE values to stdout.
7. docker/test_locally.sh: `-e HF_TOKEN="${HF_TOKEN:-}"` either pasted
the secret verbatim into the process arg list or shadowed any
in-container value with an empty string. Forward conditionally.
8. .github/workflows/docker-publish.yml: gate `latest` on default branch
AND on `unsloth_ref` not being overridden via workflow_dispatch.
Otherwise a maintainer testing a feature SHA from main could overwrite
`:latest` with non-main source.
9. docker/Dockerfile.studio: add an `UNSLOTH_STUDIO_REF` build-arg so
the Studio companion image is pinned to a known unsloth ref instead
of cloning `main` whenever it builds.
78 lines
3.8 KiB
Bash
Executable file
78 lines
3.8 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# Convenience wrapper for `docker run unsloth/unsloth`. Sets the flags that
|
|
# people most often forget and that cause the most confusing failures:
|
|
#
|
|
# --gpus all Without this, no GPU is attached and the container's
|
|
# entrypoint will refuse to start.
|
|
# --ipc=host PyTorch DataLoader workers need ample /dev/shm. The
|
|
# default 64MB causes "DataLoader worker (pid X) exited
|
|
# unexpectedly" on any non-trivial dataset.
|
|
# --ulimit memlock=-1 Unlimited pinned memory for NCCL / CUDA pinned host
|
|
# buffers. Without this, multi-GPU training stalls.
|
|
# --ulimit stack=64MB Larger thread stack for libtorch (some kernels OOM
|
|
# the default 8MB stack).
|
|
#
|
|
# Plus mounts the host Hugging Face cache and Triton JIT cache so model
|
|
# downloads and compiled kernels persist across container runs.
|
|
#
|
|
# Usage:
|
|
# bash docker/run.sh # interactive python REPL
|
|
# bash docker/run.sh bash # shell in the container
|
|
# bash docker/run.sh python /workspace/smoke_test.py # run the smoke test
|
|
# bash docker/run.sh python /workspace/host/train.py # run your training script
|
|
# ($PWD is mounted at
|
|
# /workspace/host)
|
|
#
|
|
# Overridable env:
|
|
# UNSLOTH_IMAGE=unsloth/unsloth:latest image and tag to pull/run
|
|
# UNSLOTH_GPUS=all GPUs to expose ("all" | "0" | "0,1")
|
|
# HF_HOME=$HOME/.cache/huggingface host HF cache dir to mount
|
|
# TRITON_CACHE_DIR=$HOME/.cache/unsloth-triton
|
|
# host Triton cache dir to mount
|
|
# UNSLOTH_WORKDIR=$PWD host dir mounted at /workspace/host
|
|
set -euo pipefail
|
|
|
|
IMAGE="${UNSLOTH_IMAGE:-unsloth/unsloth:latest}"
|
|
GPUS="${UNSLOTH_GPUS:-all}"
|
|
HF_CACHE="${HF_HOME:-$HOME/.cache/huggingface}"
|
|
TRITON_CACHE="${TRITON_CACHE_DIR:-$HOME/.cache/unsloth-triton}"
|
|
WORK_DIR="${UNSLOTH_WORKDIR:-$PWD}"
|
|
|
|
mkdir -p "$HF_CACHE" "$TRITON_CACHE"
|
|
|
|
# Warn early if the host doesn't have the nvidia runtime registered.
|
|
# We let `docker run` fail loudly rather than abort here -- some setups
|
|
# (rootless docker, custom runtimes) report runtimes differently.
|
|
if ! docker info 2>/dev/null | grep -qi 'Runtimes:.*nvidia'; then
|
|
printf "\033[1;33mWARN:\033[0m 'docker info' does not list 'nvidia' as a runtime.\n" >&2
|
|
printf " If --gpus all fails below, install nvidia-container-toolkit:\n" >&2
|
|
printf " https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html\n\n" >&2
|
|
fi
|
|
|
|
# Forward common secrets only if they're set in the host environment.
|
|
# Empty strings would shadow whatever is already inside the image.
|
|
declare -a ENV_FORWARD=(-e HF_HUB_ENABLE_HF_TRANSFER=1)
|
|
[[ -n "${HF_TOKEN:-}" ]] && ENV_FORWARD+=(-e "HF_TOKEN=${HF_TOKEN}")
|
|
[[ -n "${WANDB_API_KEY:-}" ]] && ENV_FORWARD+=(-e "WANDB_API_KEY=${WANDB_API_KEY}")
|
|
[[ -n "${UNSLOTH_LICENSE:-}" ]] && ENV_FORWARD+=(-e "UNSLOTH_LICENSE=${UNSLOTH_LICENSE}")
|
|
|
|
# Only attach -t when our own stdin/stdout are a TTY; CI / piped invocations
|
|
# otherwise hit `the input device is not a TTY` and never reach the entrypoint.
|
|
TTY_FLAG=()
|
|
if [ -t 0 ] && [ -t 1 ]; then
|
|
TTY_FLAG=(-it)
|
|
fi
|
|
|
|
# Avoid `set -x` here so the literal HF_TOKEN / WANDB_API_KEY / UNSLOTH_LICENSE
|
|
# values do not get echoed to stdout/CI logs. The forwarded env vars are
|
|
# already in ENV_FORWARD; printing them again was a secret leak.
|
|
exec docker run --rm "${TTY_FLAG[@]}" \
|
|
--gpus "$GPUS" \
|
|
--ipc=host \
|
|
--ulimit memlock=-1 \
|
|
--ulimit stack=67108864 \
|
|
-v "$HF_CACHE":/workspace/.cache/huggingface \
|
|
-v "$TRITON_CACHE":/workspace/.cache/triton \
|
|
-v "$WORK_DIR":/workspace/host \
|
|
"${ENV_FORWARD[@]}" \
|
|
"$IMAGE" "$@"
|