Commit graph

1 commit

Author SHA1 Message Date
Daniel Han
6e057ffebe
Studio: training survives a non-writable HF datasets cache (#6148)
* Studio: training survives a non-writable HF datasets cache

A shared HF datasets cache can contain subtrees owned by another user
(for example populated by an earlier root-run job). datasets then dies
with "[Errno 13] Permission denied: ..._builder.lock" while locking
the cached builder and the training run fails. load_dataset in the
training worker and trainer now goes through a wrapper that catches the
EACCES and rebuilds the dataset in a Studio-owned cache under
cache_root()/hf-datasets, logging the fallback.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scope the HF_DATASETS_CACHE override to the fallback load

* Route non-streaming dataset preview loads through the cache-safe wrapper

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-10 08:22:47 -07:00