Two defects found by running the built image rather than reading it.
1. Every unsloth_cli subcommand that touches the studio backend died on
import. `unsloth list-checkpoints` on the published image:
ModuleNotFoundError: No module named 'structlog'
and the same for train / export / chat, since all four import
studio.backend.core.*. structlog is a studio backend requirement, not
an unsloth[huggingface] one, so nothing in the base install pulled it
in. Added it to the base venv, and added a build-time
`from studio.backend.core.export import ExportBackend` so a future
missing dependency in that closure fails the build instead of the
user's first CLI invocation. That guard has to live in the LAST
builder verification block: the closure also needs starlette, which
only arrives with vLLM two stages later.
2. flashinfer-jit-cache was pinned to a literal 0.6.6 while vLLM 0.26.0
resolves flashinfer-python 0.6.14. flashinfer raises at import when
the two disagree, and that exception is thrown inside the vLLM
EngineCore, so Unsloth's GRPO fast_inference path fails at engine
start with no earlier warning. A literal pin drifts again on the next
vLLM bump, so the version is now read back from the resolved
flashinfer-python, and the build proves `import flashinfer` works.
Verified on the rebuilt image: flashinfer-python 0.6.14 with
flashinfer-jit-cache 0.6.14+cu128, structlog 26.1.0, the export backend
importable, and `unsloth list-checkpoints` exiting 0.
tests/python/test_docker_llama_cuda_backend.py gains two static cases
pinning both: the jit-cache version must be derived rather than literal
and the build must import flashinfer, and the base venv must ask for
structlog with the CLI reachability guard present.
The portable llama.cpp bundle loads libggml-cuda.so with dlopen, links it
against libcublas, and does not ship libcublas. The CUDA runtime base
image only carries libcudart, and the only libcublas in the image is
torch's wheel copy under site-packages/nvidia/cublas/lib, which was not
on the loader path. So the CUDA backend failed to load, and llama.cpp
said nothing about it: `--list-devices` printed an empty list and every
GGUF request ran on the CPU.
Measured in the built image on a B200 with gemma-4-E2B-it UD-Q4_K_XL:
1.6 tok/s from llama-cli and 4.2 tok/s from llama-server. With the fix,
the same image and model report `CUDA0: NVIDIA B200` and run at 229 tok/s
and 193 tok/s. Studio's GGUF chat and the GGUF export path go through the
same bundle, so both were affected.
The venv loader config already existed for torchcodec, so cublas/lib
joins it there rather than on LD_LIBRARY_PATH: ld.so.conf.d is consulted
after DT_RUNPATH, which keeps llama.cpp resolving its own $ORIGIN libs
first. cu13/lib comes along for the arm64 bundle's layout.
A silent 140x slowdown deserves a build-time gate, so the layer after the
fetch runs ldd over libggml-cuda.so, installs the cublas major the bundle
actually asks for when it is missing, and fails the build on anything
still unresolved. The amd64 bundle wants libcublas.so.12 and torch
already provides it; the arm64 bundle is CUDA 13, and deriving the major
from ldd keeps that leg honest without hardcoding either. libcuda.so.1 is
exempt: nvidia-container-toolkit injects the driver stub at
`docker run --gpus`, so it is never resolvable at build time. ldd needs
no GPU, so the build stays host-independent.
tests/python/test_docker_llama_cuda_backend.py pins the loader entry, the
guard, the driver-stub exemption and the ordering.
