ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Dockerfile: install gcc + g++ + python3-dev in runtime stage

Browse source 
Triton's nvidia backend lazily JIT-compiles a small C extension
(CudaUtils, in triton/backends/nvidia/driver.py) on first GPU access.
Without a C compiler and Python headers in the runtime image, the
very first forward pass of any Unsloth model dies with:

  RuntimeError: Failed to find C compiler.
                Please specify via CC environment variable.

The builder stage has build-essential and python3.12-dev so this
worked during the build's verification step (no GPU = no Triton kernel
call = no C extension build). But the runtime stage stripped those
out for size, so the failure only surfaces when a real user runs
training inside the container.

Add gcc + g++ + python3.12-dev to the runtime stage. Increases the
runtime image by ~250MB, which is the cost of letting Triton JIT
correctly. Pre-compiling CudaUtils at build time would need a real
CUDA device (the constructor calls cuda runtime functions), so
shipping the toolchain is the right trade-off.
This commit is contained in:
Daniel Han 2026-05-24 08:22:31 +00:00
commit 1cdc5f1720
1 changed files with 10 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

11
docker/Dockerfile
View file

@ -184,12 +184,21 @@ ENV DEBIAN_FRONTEND=noninteractive \
RUN apt-get update && apt-get install -y --no-install-recommends \
software-properties-common ca-certificates curl git libgomp1 \
gcc g++ \
&& add-apt-repository -y ppa:deadsnakes/ppa \
&& apt-get update && apt-get install -y --no-install-recommends \
python${PYTHON_VERSION} python${PYTHON_VERSION}-venv \
python${PYTHON_VERSION} python${PYTHON_VERSION}-venv python${PYTHON_VERSION}-dev \
&& ln -sf /usr/bin/python${PYTHON_VERSION} /usr/local/bin/python \
&& ln -sf /usr/bin/python${PYTHON_VERSION} /usr/local/bin/python3 \
&& rm -rf /var/lib/apt/lists/*
# Why gcc + g++ + python3.12-dev in the RUNTIME stage:
# Triton's nvidia backend lazily compiles a small C extension (CudaUtils) on
# first GPU access. Without a C compiler + Python headers the very first
# forward pass of any Unsloth model dies with:
# RuntimeError: Failed to find C compiler. Please specify via CC env var.
# Adds ~250MB to the runtime image, which is the cost of letting every kernel
# JIT correctly. (Pre-compiling CudaUtils at build time would need a GPU, so
# shipping the toolchain is the right trade-off.)
COPY --from=builder /opt/unsloth-venv /opt/unsloth-venv