6e057ffebe* Studio: training survives a non-writable HF datasets cache A shared HF datasets cache can contain subtrees owned by another user (for example populated by an earlier root-run job). datasets then dies with "[Errno 13] Permission denied: ..._builder.lock" while locking the cached builder and the training run fails. load_dataset in the training worker and trainer now goes through a wrapper that catches the EACCES and rebuilds the dataset in a Studio-owned cache under cache_root()/hf-datasets, logging the fallback. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope the HF_DATASETS_CACHE override to the fallback load * Route non-streaming dataset preview loads through the cache-safe wrapper --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| cache_safe.py | ||
| chat_templates.py | ||
| data_collators.py | ||
| dataset_none_detect.py | ||
| dataset_utils.py | ||
| format_conversion.py | ||
| format_detection.py | ||
| llm_assist.py | ||
| model_mappings.py | ||
| raw_text.py | ||
| vlm_processing.py | ||