ba209f8539
The image did not build. bitsandbytes pins scikit-build-core unbounded in build-system.requires, so the builder always resolves the newest release. 1.0.0 (2026-07-06) added a PEP 660 gate to the setuptools plugin backend that bitsandbytes uses, rejecting editable installs unless pyproject sets editable.mode = "inplace". The pin here is from March and predates upstream adding that line, so pip install -e . dies in prepare_metadata_for_build_editable with "setuptools editable installs require editable.mode = 'inplace'". Verified by version bisection in the image: 0.11.6 and 0.12.2 install cleanly, 1.0.0, 1.0.1 and 1.0.3 all fail. Install non-editable instead. Upstream's own install docs annotate the -e as being for bitsandbytes development and say to leave it out otherwise, and an editable checkout buys nothing in an image while tying the build to a scikit-build-core knob that keeps moving. CMakeLists writes the library straight into the package dir, so setuptools package-data ships exactly what cmake built: the installed libbitsandbytes_rocm64.so is md5 identical to the compiled one and its .hip_fatbin still holds a single gfx942 bundle. The verification step could not fail. pip show passes on dist-info alone, so a missing or unloadable HIP library left it green on precisely the broken image it exists to catch. Load the artifacts instead. bitsandbytes cannot self-report here: with no AMD GPU on the builder get_cuda_specs returns None, it loads libbitsandbytes_cpu.so and hands back a healthy BNBNativeLibrary even after the ROCm library is deleted, so the check globs for libbitsandbytes_rocm*.so beside the installed package and ctypes.CDLL loads it. unsloth_zoo raises NotImplementedError on import without a torch accelerator, so it and unsloth are resolved with importlib.util.find_spec rather than executed. Deleting the library now fails the check; the intact image passes. The amdgpu-arch warning at configure time is benign and left alone. It comes from hip-config-amd.cmake, pulled in by find_package(hipblas), and only governs --offload-arch on the hip::device interface for CXX sources. bitsandbytes links hip::host and compiles its kernels as HIP, taking the arch from CMAKE_HIP_ARCHITECTURES, which BNB_ROCM_ARCH sets before enable_language(HIP). The built library contains gfx942 and no gfx906.
69 lines
2.8 KiB
Text
69 lines
2.8 KiB
Text
ARG BASE_DOCKER=rocm/vllm-dev:nightly_main_20250914
|
|
FROM $BASE_DOCKER
|
|
|
|
WORKDIR /unsloth-workspace
|
|
|
|
# Single RUN so the apt index can't go stale; cleanup keeps the layer small.
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends zip unzip wget build-essential cmake \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
ARG COMPUTE_BACKEND="hip"
|
|
ARG BNB_ROCM_ARCHS="gfx942"
|
|
# bnb PR #1887 floor (0.50.0.dev0); earlier builds NaN at 4-bit GEMV decode on AMD.
|
|
ARG BNB_COMMIT="713a3b83e405d32449e1cb392e5e25985ab927c6"
|
|
ARG BNB_REPO="https://github.com/bitsandbytes-foundation/bitsandbytes.git"
|
|
|
|
# Plain install, not `-e`: upstream reserves editable for bnb development, and
|
|
# scikit-build-core 1.0.0 rejects it unless pyproject opts into
|
|
# editable.mode = "inplace", which this pin predates. cmake already dropped the
|
|
# .so into the package dir, so setuptools package-data ships it as built.
|
|
RUN mkdir bitsandbytes \
|
|
&& cd bitsandbytes \
|
|
&& git init \
|
|
&& git remote add origin ${BNB_REPO} \
|
|
&& git fetch --depth 1 origin ${BNB_COMMIT} \
|
|
&& git checkout FETCH_HEAD \
|
|
&& rm -rf .git \
|
|
&& cmake -S . -B build \
|
|
-DCOMPUTE_BACKEND=${COMPUTE_BACKEND} \
|
|
-DBNB_ROCM_ARCH="${BNB_ROCM_ARCHS}" \
|
|
&& cmake --build build -j"$(nproc)" \
|
|
&& pip install . \
|
|
&& rm -rf build
|
|
|
|
ARG UNSLOTH_REPO="https://github.com/unslothai/unsloth.git"
|
|
ARG UNSLOTH_BRANCH="main"
|
|
|
|
|
|
# huggingfacenotorch skips torch/torchvision so the base image's ROCm torch survives; bare `-e .` can't train.
|
|
RUN git clone --branch ${UNSLOTH_BRANCH} --depth 1 ${UNSLOTH_REPO} unsloth \
|
|
&& cd unsloth \
|
|
&& rm -rf .git \
|
|
&& pip install -e '.[huggingfacenotorch]'
|
|
|
|
|
|
ARG UNSLOTH_ZOO_REPO="https://github.com/unslothai/unsloth-zoo.git"
|
|
ARG UNSLOTH_ZOO_BRANCH="main"
|
|
|
|
RUN git clone --branch ${UNSLOTH_ZOO_BRANCH} --depth 1 ${UNSLOTH_ZOO_REPO} unsloth-zoo \
|
|
&& cd unsloth-zoo \
|
|
&& rm -rf .git \
|
|
&& pip install -e .
|
|
|
|
|
|
# pip show reads dist-info only, so it stayed green on an image whose freshly
|
|
# compiled HIP library was missing. bitsandbytes is no help either: with no AMD
|
|
# GPU on the builder it falls back to libbitsandbytes_cpu.so and reports
|
|
# success, so load the ROCm library explicitly. unsloth_zoo refuses to import
|
|
# without a torch accelerator, so only resolve it and unsloth.
|
|
# Still uncaught: a wrong-arch build loads fine here and only fails once a
|
|
# 4-bit GEMV runs on real AMD silicon.
|
|
RUN python3 -c "import ctypes, importlib.util, pathlib; \
|
|
import bitsandbytes; \
|
|
pkg = pathlib.Path(bitsandbytes.__file__).parent; \
|
|
libs = sorted(pkg.glob('libbitsandbytes*.so')); \
|
|
assert any('_rocm' in so.name for so in libs), f'no libbitsandbytes_rocm*.so in {pkg}'; \
|
|
[ctypes.CDLL(str(so)) for so in libs]; \
|
|
missing = [m for m in ('unsloth', 'unsloth_zoo') if importlib.util.find_spec(m) is None]; \
|
|
assert not missing, f'not importable: {missing}'"
|