From afbdaa09b7d9bffc3bcf315065300ad8d4fde9b2 Mon Sep 17 00:00:00 2001 From: danielhanchen Date: Tue, 28 Jul 2026 22:47:33 +0000 Subject: [PATCH] Allowlist the triton-kernels pure-Python sdist, and record why Windows on ARM is red The two ubuntu2404 root legs went red at "Assert no source build" reporting triton-kernels. That is not a regression in what the installer does. Those builds have always happened; they only became visible now that pip_install stopped discarding uv's output on success, which is what finally let the nobuild check read the dependency phase at all. So the question was whether each build actually needs a compiler. Checked against the real artifacts rather than assumed: openai-whisper 20250625, randomname 0.2.1, argbind 0.3.9 -- no version of any of the three has ever published a wheel; antlr4-python3-runtime is pinned at 4.9.3, below the first release that ships one. All four sdists use setuptools.build_meta, declare no ext_modules, and contain no .c/.cpp/.pyx/.rs file. Already allowlisted, correctly. triton-kernels is the same category and was the only name failing. It is pinned to the triton repo's python/triton_kernels subdirectory; that tree is 75 files of Python, a four-line pyproject.toml, no setup.py and no native source at all. The kernels are Triton DSL compiled at runtime, not at install time. It is also a direct URL the installer names itself rather than something resolution picked, and only Linux reaches it. It belongs in the allowlist, so add it with that reasoning written down. The allowlist match is now lowercased and underscore-folded on both sides. The requirement spells the package triton_kernels while uv prints triton-kernels, and an allowlist that matched only one spelling would pass by luck rather than by intent. A plain pyarrow sdist is still caught. The two data-designer @ file:// plugin builds needed nothing: they are in-tree local paths, already dropped by the same rule that exempts the source overlay's own build. Separately, the windows-11-arm leg fails for a real reason and should keep failing. The ARM handling itself works, the log shows torchaudio being skipped and torch plus torchvision installing from wheels. What stops it is that pyarrow and hf-transfer publish no win_arm64 wheel at all, so uv falls back to their sdists and they fail on CMake configure and on openssl-sys wanting perl. That is a product gap on the platform, not a gap in the simulation, so the leg stays experimental and keeps reporting it. Record that above the matrix entry so the next reader does not re-diagnose it. --- .github/scripts/clean-machine-assert.sh | 22 ++++++++++++++++--- .../workflows/clean-machine-install-ci.yml | 11 ++++++++++ 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/.github/scripts/clean-machine-assert.sh b/.github/scripts/clean-machine-assert.sh index c2c65708cf..22b29e090a 100755 --- a/.github/scripts/clean-machine-assert.sh +++ b/.github/scripts/clean-machine-assert.sh @@ -94,13 +94,28 @@ for check in "$@"; do ;; nobuild) - # "Built an sdist" is NOT "needed a compiler". Four packages on the macOS path - # are sdist-only PURE PYTHON (verified against cp313/macos-arm64): + # "Built an sdist" is NOT "needed a compiler". Every name below was checked + # against its actual sdist: setuptools.build_meta backend, no ext_modules, + # and not one .c/.cpp/.pyx/.rs file in the archive, so the PEP 517 build is + # a pure-Python metadata-and-copy step that completes with no compiler. # openai-whisper, argbind, randomname -- no version ever ships a wheel # antlr4-python3-runtime==4.9.3 -- pinned below the 4.13.2 wheel + # triton-kernels -- studio/backend/requirements/ + # triton-kernels.txt pins it to a git URL under the triton repo's + # python/triton_kernels subdirectory. That tree is 75 files of Python + # with a four-line pyproject.toml and no setup.py; the kernels are + # Triton DSL compiled at runtime, never at install time. It is also a + # direct URL the installer names itself, not something resolution + # chose, and only the Linux legs reach it (install_python_stack.py + # skips the step on Windows and macOS). # Failing on those is a false alarm, so the contract is "nothing needing a # COMPILER was built". UNSLOTH_ALLOW_SDIST extends the allowlist. - _allow="openai-whisper argbind randomname antlr4-python3-runtime ${UNSLOTH_ALLOW_SDIST:-}" + # + # Lowercased and underscore-folded on both sides, because a project's + # distribution name and the name uv prints can disagree on the separator: + # the requirement says triton_kernels, the build line says triton-kernels, + # and an allowlist that matched only one spelling would silently miss. + _allow="$(printf '%s' "openai-whisper argbind randomname antlr4-python3-runtime triton-kernels ${UNSLOTH_ALLOW_SDIST:-}" | tr 'A-Z_' 'a-z-')" if [ ! -f "$LOG" ]; then fail "nobuild requested but $LOG is missing" else @@ -123,6 +138,7 @@ for check in "$@"; do | grep -oiE "building wheel for [a-z0-9._-]+|building [a-z0-9._-]+(==| @ )" \ | tr 'A-Z' 'a-z' \ | sed -E -e 's/^building wheel for //' -e 's/^building //' -e 's/(==| @ )$//' \ + | tr '_' '-' \ | sort -u || true)" _bad="" for pkg in $_built; do diff --git a/.github/workflows/clean-machine-install-ci.yml b/.github/workflows/clean-machine-install-ci.yml index cd630f7b3b..44b29e472b 100644 --- a/.github/workflows/clean-machine-install-ci.yml +++ b/.github/workflows/clean-machine-install-ci.yml @@ -678,6 +678,17 @@ jobs: winget: 'masked' experimental: false overlay: true + # Windows on ARM gets as far as the dependency install and then stops on + # two packages that publish no win_arm64 wheel at all: + # pyarrow==25.0.0 (pulled in by datasets) -- PyPI has win_amd64 only, + # so uv falls back to the sdist and its CMake configure fails + # hf-transfer==0.1.9 -- a maturin/Rust sdist whose openssl-sys build + # script wants perl, which the image does not have + # The ARM-specific handling added for this platform is working: the log + # shows "windows on arm: skipping torchaudio", and torch 2.10.0+cpu and + # torchvision both install from wheels. The redness that remains is a + # real product gap on this platform, not a gap in the simulation, so the + # leg stays experimental and keeps reporting it rather than hiding it. - os: windows-11-arm winget: 'visible' experimental: true