* disable_torchcodec_if_broken: also patch datasets and clean sys.modules (#5446)
transformers's _torchcodec_available was being flipped to False already,
but datasets keeps its own datasets.config.TORCHCODEC_AVAILABLE flag
(datasets >= 4.0) that gates every torchcodec call site inside
datasets/features/{audio,video}.py, datasets/features/features.py and the
three datasets formatters. Without flipping that flag, transformers
falls back to librosa but datasets still routes through torchcodec and
re-raises the same RuntimeError, which is what users hit on Colab when
libavutil is missing.
Also pops half-loaded torchcodec submodules + datasets.features._torchcodec
from sys.modules so a later re-import does not re-trigger the failed
native-library dlopen.
Verified live state after the patch on a Colab-like broken-torchcodec env:
transformers._torchcodec_available = False
transformers.is_torchcodec_available() = False
datasets.config.TORCHCODEC_AVAILABLE = False
stale torchcodec entries in sys.modules = []
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* disable_torchcodec_if_broken: seat sys.modules[torchcodec]=None sentinel
The first commit on this branch flipped the transformers and datasets
availability flags, but a few unconditional torchcodec call sites in
datasets / torchaudio (e.g. Audio.encode_example does
"from torchcodec.encoders import AudioEncoder" outside the
TORCHCODEC_AVAILABLE gate) still hit a cryptic RuntimeError from the
broken native library load.
Seating sys.modules["torchcodec"] = None makes any subsequent
"import torchcodec" / "from torchcodec.X import Y" raise
ModuleNotFoundError (subclass of ImportError) which the existing
try/except ImportError blocks in datasets / torchaudio catch and
re-raise as the clean "please install torchcodec" message users get
when they uninstall torchcodec manually. This was the workaround in
issue #5446.
Also makes find_spec("torchcodec") return None on re-entry, so the
function is a strict no-op on the second call.
Verified across 12 scenarios in temp/torchcodec_test/: healthy
torchcodec untouched, broken torchcodec sentinel-blocked, datasets 3.x
without the flag handled, no-datasets-installed handled, py3.11 Colab-
exact env (transformers==4.56.2 + trl==0.22.2) handled. Side-by-side
on the user's exact stack: Audio.encode_example switches from
"RuntimeError: Could not load libtorchcodec" to
"ImportError: To support encoding audio data, please install 'torchcodec'.".
* disable_torchcodec_if_broken: trim comments
Same behaviour, shorter docstring and inline comments.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
aba57d0872