unsloth/studio/backend/core/export
Daniel Han 3fd948eb95
Pin utf-8 on shipping-code text I/O instead of the operator locale (#7486)
* Pin utf-8 on shipping-code text I/O instead of the operator locale

113 read_text/write_text/open call sites across unsloth, studio and
unsloth_cli let locale.getencoding() decide the encoding. That is utf-8 on
the Linux and macOS runners and cp1252 on a stock Windows install, so the
same file decodes differently for a Windows user and silently produces
mojibake or raises UnicodeDecodeError.

Adds tests/test_runtime_text_encoding.py to keep it that way. It resolves
openers through each file's own imports rather than a fixed list of module
names, so an aliased tarfile.open or a local from PIL.Image import open is
not asked for an encoding it does not take.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scan tracked files only and resolve the unbound Path calling forms

* Honour PEP 263 when scanning sources and migrate a legacy JSONL before appending

* Scope guard imports lexically and only migrate a legacy file when it round-trips

* Leave a legacy JSONL untouched and resolve path aliases in the foreign-opener check

* Tighten comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-27 02:14:20 -07:00
..
__init__.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
export.py Pin utf-8 on shipping-code text I/O instead of the operator locale (#7486) 2026-07-27 02:14:20 -07:00
orchestrator.py Studio: add configurable model download location (#7274) 2026-07-23 01:34:38 -07:00
worker.py Studio: quiet noisy logs, log real progress, and speed up Windows/macOS dataset prep (#7087) 2026-07-15 06:49:52 -07:00