ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

tests: read checked-in files as UTF-8 instead of the platform default (#7438)

Browse source 
* tests: read checked-in files as UTF-8 instead of the platform default

Path.read_text() with no encoding uses locale.getpreferredencoding(), which
is UTF-8 on the Linux runners and cp1252 on a stock Windows install. Nine
module-level reads of checked-in source files were relying on that default.

studio/backend/routes/inference.py carries the DeepSeek tool-call token
regexes, so it holds U+FF5C and U+2581. Under cp1252 that read raised
UnicodeDecodeError on byte 0x81 at position 97806, and because the reads run
at import time it took test_cancel_atomicity.py and test_cancel_id_wiring.py
out at collection, not as failures. Green on CI, permanently broken for a
Windows contributor running the suite locally.

Adds a guard: at module scope there is no tmp_path fixture, so a bare
read_text()/write_text()/open() there is always touching a checked-in file.
That makes the rule mechanical enough to enforce with no allowlist, while
staying quiet about temp-dir I/O inside test bodies where the platform
default is harmless.

The repo already spells this correctly in 464 other places; this only stops
the stragglers coming back.

* tests: cover import-time helper reads and keep the guard py3.9-safe

Follows up on the Codex review:

- add `from __future__ import annotations`, since `str | None` in
  `_offender` is evaluated at import on Python 3.9 and pyproject declares
  requires-python ">=3.9,<3.15".
- widen the guard from module scope to import time. Class bodies and the
  bodies of module-level helpers called from an executing statement run
  during collection too, so `CODE = _extract_mixed_precision_code()` was
  the same hazard as an inline read. `if __name__ == "__main__":` blocks
  are skipped: pytest never executes them.
- scan studio/backend/tests/ as well as tests/. Both trees are collected
  on Windows by separate CI jobs, and the offender that started this,
  test_tool_xml_strip.py reading routes/inference.py, lives there.

Widening it surfaced seven more import-time reads of checked-in sources;
all now name utf-8.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Harden the import-time encoding guard for PR #7438

Close the detector gaps raised in review, all of which I reproduced against
the actual AST before changing anything.

False negatives (the guard let a real hazard through):
- _is_main_guard ignored the comparison operator, so if __name__ != "__main__"
  counted as script-only even though its body runs at import.
- The else arm of a main guard was discarded with the rest of the If node.
- Decorators and argument defaults on a module-level def were skipped with the
  body, though both are evaluated when the def executes.
- Path.open() in text mode was invisible; only builtin open() was matched.
- encoding = None and encoding = "locale" both re-select the platform default,
  but the keyword merely being present counted as pinned.

False positives (the guard would have blocked a compliant contributor):
- A non-literal mode fell through to the "r" default, so open(p, mode) was
  flagged even when mode is "rb", where adding encoding= is a ValueError and
  there is no edit that satisfies the rule.
- Same for open(*args) and a **kwargs splat, which hide the mode and can hide
  an encoding.
- Lambda bodies and comprehension elements were walked even though neither runs
  at definition.

Verified: still reports the same 22 offenders on unpatched main, green on this
branch and on the tree merged with latest main (557 files), and an adversarial
corpus of 33 cases now scores zero false positives and zero false negatives.
Also corrected two docstring claims: neither collecting job runs on Windows,
and the read is governed by locale.getencoding().

* Walk eager comprehensions and treat io.open as the builtin

Two regressions from the previous commit, both reproduced against the AST
before changing anything.

Lumping list, set and dict comprehensions in with generator expressions was
wrong. Only a genexp is lazy; the other three run their element expression,
their filters and their nested iterators immediately, so
CONTENTS = [p.read_text() for p in PATHS] at module scope is an import-time
read the guard was silently missing. Comprehensions are now walked in full and
only the genexp keeps the outermost-iterable-only treatment.

io was also in the not-a-path-opener list, but io.open is the builtin, with the
same mode position and the same platform default. io.open(CHECKED_IN_FILE) is
exactly the hazard this guard exists for, so it is matched now, with binary
modes and a pinned encoding still exempt. tarfile.open and fitz.open stay
exempt since neither has an encoding to name.

Verified: 13 targeted cases covering all five eager comprehension forms and
io.open in text, binary and pinned shapes all classify correctly; still 22
offenders on unpatched main; green on this branch and on the tree merged with
latest main.

* Close three more walker gaps in the import-time guard

All three reproduced against the AST first.

A generator expression handed straight to a call is consumed there, so
DATA = "".join(p.read_text() for p in paths) runs its element at import. Only
an unconsumed genexp bound to a name stays lazy, so the walker now follows the
consumed ones in full and keeps the outermost-iterable-only treatment for the
rest.

if "__main__" == __name__ is an equivalent and accepted spelling of the main
guard, but requiring __name__ on the left meant its body was treated as
import-time code. That is a false positive on a block pytest never runs, so
both operand orders are recognised now.

The helper table was built from module-level defs only, so a def in a class
body invoked while the class is constructed was never followed, contradicting
the walker's stated coverage of class bodies. Helpers are now collected from
the module body and from class bodies at any nesting.

Verified: 15 targeted cases including all three fixes and the earlier ones
still classify correctly; still 22 offenders on unpatched main; green on this
branch and on the tree merged with latest main.

* Handle positional read_text encodings, lazy generators and nested helpers

* Guard reads reached from test bodies, unbound Path calls and __file__ paths

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Follow derived paths, skip lazy generator helpers, cover compressed openers

* Guard the CLI tests, helper parameters and unbound Path arguments

* Discover test roots and follow literal, in-place and tuple-derived paths

* Identify module openers by import, unwrap starred paths, pin subprocess snippets

* Resolve import origins, seed helper locals, follow named generators and parametrize

* Scope imports lexically, list tracked test files, bind unpacked names

* Resolve aliased openers, keyword-only params, destructured targets, next()

* Pin the encoding on subprocess snippets, workflow lint and CLI output for PR #7438

* Harden the CLI encoding guard against detached streams for PR #7438

* Tighten the encoding guard's path and scope analysis for PR #7438

* Resolve path provenance more precisely and keep POSIX stream encodings for PR #7438

* Resolve qualified path classes and scope conditional imports for PR #7438

* Scope CLI stream setup to the entry point and align two encoding pairs for PR #7438

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
This commit is contained in:
Leo Borcherding 2026-07-27 01:31:56 -05:00 • committed by GitHub
commit 1dd2fc4583
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
84 changed files with 1652 additions and 352 deletions
Download patch file Download diff file Expand all files Collapse all files

4
tests/python/test_cpo_processor_text_tokenizer.py
View file

@ -40,7 +40,7 @@ def _registrations(source):
def test_cpo_registration_matches_orpo():
regs = _registrations(open(RL_PATH).read())
regs = _registrations(open(RL_PATH, encoding = "utf-8").read())
shared = {"orpo_trainer_text_tokenizer", "orpo_trainer_processor_pad_token"}
assert shared <= set(regs.get("orpo_trainer", []))
assert shared <= set(regs.get("cpo_trainer", []))
@ -48,7 +48,7 @@ def test_cpo_registration_matches_orpo():
def _load_pad_rewriter():
"""Exec orpo_trainer_processor_pad_token (+ _PAD_FALLBACK) without importing unsloth."""
tree = ast.parse(open(RL_PATH).read())
tree = ast.parse(open(RL_PATH, encoding = "utf-8").read())
nodes = []
for n in tree.body:
if isinstance(n, ast.Assign) and any(

2
tests/python/test_dpo_vision_processor_passthrough.py
View file

@ -11,7 +11,7 @@ RL_PATH = os.path.join(REPO_ROOT, "unsloth", "models", "rl_replacements.py")
def _load_helpers():
src = open(RL_PATH).read()
src = open(RL_PATH, encoding = "utf-8").read()
tree = ast.parse(src)
import torch as _torch

30
tests/python/test_e2e_no_torch_sandbox.py
View file

@ -193,7 +193,7 @@ class TestBeforeAfterImportChain:
mm = types.ModuleType('model_mappings')
mm.MODEL_TO_TEMPLATE_MAPPER = {{}}
sys.modules['model_mappings'] = mm
source = open({str(before_file)!r}).read()
source = open({str(before_file)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .model_mappings import', 'from model_mappings import')
exec(source)
@ -215,7 +215,7 @@ class TestBeforeAfterImportChain:
loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers
exec(open({str(before_file)!r}).read())
exec(open({str(before_file)!r}, encoding = "utf-8").read())
""")
result = _run_in_sandbox(no_torch_venv, code)
assert result.returncode != 0, "BEFORE data_collators.py should crash without torch"
@ -284,7 +284,7 @@ class TestBeforeAfterImportChain:
it = types.ModuleType('iterable')
it.is_streaming_dataset = lambda *a, **k: False
sys.modules['iterable'] = it
source = open({str(CHAT_TEMPLATES)!r}).read()
source = open({str(CHAT_TEMPLATES)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .model_mappings import', 'from model_mappings import')
source = source.replace('from .iterable import', 'from iterable import')
@ -304,7 +304,7 @@ class TestBeforeAfterImportChain:
loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read())
exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
print("OK")
""")
result = _run_in_sandbox(no_torch_venv, code)
@ -382,7 +382,7 @@ class TestDataclassInstantiation:
loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read())
exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
obj = DataCollatorSpeechSeq2SeqWithPadding(processor=None)
assert obj.processor is None
print("OK")
@ -397,7 +397,7 @@ class TestDataclassInstantiation:
loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read())
exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
obj = DeepSeekOCRDataCollator(processor=None)
assert obj.processor is None
assert obj.max_length == 2048
@ -414,7 +414,7 @@ class TestDataclassInstantiation:
loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read())
exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
obj = VLMDataCollator(processor=None)
assert obj.processor is None
assert obj.max_length == 2048
@ -441,7 +441,7 @@ class TestDataclassInstantiation:
it.is_streaming_dataset = lambda *a, **k: False
sys.modules['iterable'] = it
ns = {{}}
source = open({str(CHAT_TEMPLATES)!r}).read()
source = open({str(CHAT_TEMPLATES)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .model_mappings import', 'from model_mappings import')
source = source.replace('from .iterable import', 'from iterable import')
@ -473,7 +473,7 @@ class TestEdgeCasesBrokenTorch:
code = textwrap.dedent(f"""\
import sys
sys.path.insert(0, {str(sandbox_dir)!r})
exec(open({str(sandbox_dir / 'data_collators.py')!r}).read())
exec(open({str(sandbox_dir / 'data_collators.py')!r}, encoding = "utf-8").read())
obj = DataCollatorSpeechSeq2SeqWithPadding(processor=None)
print("OK: data_collators works despite broken torch on sys.path")
""")
@ -495,7 +495,7 @@ class TestEdgeCasesBrokenTorch:
code = textwrap.dedent(f"""\
import sys
sys.path.insert(0, {str(sandbox_dir)!r})
source = open({str(HARDWARE_PY)!r}).read()
source = open({str(HARDWARE_PY)!r}, encoding = "utf-8").read()
ns = {{'__name__': '__test__'}}
exec(source, ns)
result = ns['detect_hardware']()
@ -530,7 +530,7 @@ class TestEdgeCasesBrokenTorch:
code = textwrap.dedent(f"""\
import sys
sys.path.insert(0, {str(sandbox_dir)!r})
source = open({str(HARDWARE_PY)!r}).read()
source = open({str(HARDWARE_PY)!r}, encoding = "utf-8").read()
ns = {{'__name__': '__test__'}}
exec(source, ns)
result = ns['detect_hardware']()
@ -559,7 +559,7 @@ class TestEdgeCasesBrokenTorch:
sys.modules['iterable'] = it
ns = {{}}
source = open({str(CHAT_TEMPLATES)!r}).read()
source = open({str(CHAT_TEMPLATES)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .model_mappings import', 'from model_mappings import')
source = source.replace('from .iterable import', 'from iterable import')
@ -604,7 +604,7 @@ class TestHardwareDetectionNoTorch:
code = textwrap.dedent(f"""\
import sys
sys.path.insert(0, {str(sandbox_dir)!r})
source = open({str(HARDWARE_PY)!r}).read()
source = open({str(HARDWARE_PY)!r}, encoding = "utf-8").read()
ns = {{'__name__': '__test__'}}
exec(source, ns)
device = ns['detect_hardware']()
@ -624,7 +624,7 @@ class TestHardwareDetectionNoTorch:
code = textwrap.dedent(f"""\
import sys
sys.path.insert(0, {str(sandbox_dir)!r})
source = open({str(HARDWARE_PY)!r}).read()
source = open({str(HARDWARE_PY)!r}, encoding = "utf-8").read()
ns = {{'__name__': '__test__'}}
exec(source, ns)
versions = ns['get_package_versions']()
@ -651,7 +651,7 @@ class TestHardwareDetectionNoTorch:
code = textwrap.dedent(f"""\
import sys
sys.path.insert(0, {str(sandbox_dir)!r})
source = open({str(hw_sandbox / 'hardware.py')!r}).read()
source = open({str(hw_sandbox / 'hardware.py')!r}, encoding = "utf-8").read()
ns = {{'__name__': '__test__'}}
exec(source, ns)
assert callable(ns['detect_hardware'])

2
tests/python/test_fast_language_model_text_only.py
View file

@ -14,7 +14,7 @@ UTILS_PATH = REPO_ROOT / "unsloth" / "models" / "_utils.py"
def _source(path):
return path.read_text()
return path.read_text(encoding = "utf-8")
def _class_method(tree, class_name, method_name):

2
tests/python/test_fast_model_config_passthrough.py
View file

@ -12,7 +12,7 @@ LLAMA_PATH = REPO_ROOT / "unsloth" / "models" / "llama.py"
def _source(path):
return path.read_text()
return path.read_text(encoding = "utf-8")
def _class_method(tree, class_name, method_name):

6
tests/python/test_gpu_init_ldconfig_guard.py
View file

@ -17,13 +17,13 @@ def _find_geteuid_guard(tree: ast.AST):
def test_gpu_init_has_geteuid_guard():
tree = ast.parse(GPU_INIT.read_text())
tree = ast.parse(GPU_INIT.read_text(encoding = "utf-8"))
guard = _find_geteuid_guard(tree)
assert guard is not None, "_gpu_init.py must guard ldconfig recovery on os.geteuid()"
def test_ldconfig_calls_only_inside_geteuid_guard():
src = GPU_INIT.read_text()
src = GPU_INIT.read_text(encoding = "utf-8")
tree = ast.parse(src)
guard = _find_geteuid_guard(tree)
assert guard is not None
@ -39,6 +39,6 @@ def test_ldconfig_calls_only_inside_geteuid_guard():
def test_non_root_branch_warns_when_bnb_present():
src = GPU_INIT.read_text()
src = GPU_INIT.read_text(encoding = "utf-8")
assert "elif bnb is not None" in src
assert "sudo ldconfig" in src

2
tests/python/test_grpo_ddp_model_config.py
View file

@ -9,7 +9,7 @@ SOURCE_PATH = os.path.join(REPO_ROOT, "unsloth", "models", "rl_replacements.py")
def _read_source() -> str:
with open(SOURCE_PATH, "r") as fh:
with open(SOURCE_PATH, "r", encoding = "utf-8") as fh:
return fh.read()

2
tests/python/test_orpo_processor_text_tokenizer.py
View file

@ -10,7 +10,7 @@ RL_PATH = os.path.join(REPO_ROOT, "unsloth", "models", "rl_replacements.py")
def _load_orpo_rewriter(name = "orpo_trainer_text_tokenizer"):
src = open(RL_PATH).read()
src = open(RL_PATH, encoding = "utf-8").read()
tree = ast.parse(src)
ns = {"re": re}
# Materialise sibling module-level _-prefixed assignments the rewriter may reference.

2
tests/python/test_pad_token_fix.py
View file

@ -21,7 +21,7 @@ WANTED = {
def _load_pad_helpers():
"""Exec only the pad-token helpers with a stub logger (no heavy imports)."""
tree = ast.parse(open(TOK_PATH).read())
tree = ast.parse(open(TOK_PATH, encoding = "utf-8").read())
nodes = []
for node in tree.body:
if isinstance(node, ast.Assign):

18
tests/python/test_studio_import_no_torch.py
View file

@ -148,7 +148,7 @@ class TestDataCollatorsNoTorchVenv:
loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read())
exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
print("OK: exec succeeded")
""")
result = subprocess.run(
@ -168,7 +168,7 @@ class TestDataCollatorsNoTorchVenv:
loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read())
exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
obj = DataCollatorSpeechSeq2SeqWithPadding(processor=None)
assert obj.processor is None, "processor should be None"
print("OK: DataCollatorSpeechSeq2SeqWithPadding instantiated")
@ -190,7 +190,7 @@ class TestDataCollatorsNoTorchVenv:
loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read())
exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
obj = DeepSeekOCRDataCollator(processor=None)
assert obj.processor is None, "processor should be None"
assert obj.max_length == 2048, "default max_length should be 2048"
@ -212,7 +212,7 @@ class TestDataCollatorsNoTorchVenv:
loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read())
exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
obj = VLMDataCollator(processor=None)
assert obj.processor is None
assert obj.mask_input_tokens is True, "default mask_input_tokens should be True"
@ -259,7 +259,7 @@ class TestChatTemplatesNoTorchVenv:
sys.modules['iterable'] = iterable
# Read and transform the source: replace relative imports with absolute
source = open({str(CHAT_TEMPLATES)!r}).read()
source = open({str(CHAT_TEMPLATES)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .model_mappings import', 'from model_mappings import')
source = source.replace('from .iterable import', 'from iterable import')
@ -305,7 +305,7 @@ class TestChatTemplatesNoTorchVenv:
sys.modules['iterable'] = iterable
ns = {{}}
source = open({str(CHAT_TEMPLATES)!r}).read()
source = open({str(CHAT_TEMPLATES)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .model_mappings import', 'from model_mappings import')
source = source.replace('from .iterable import', 'from iterable import')
@ -402,7 +402,7 @@ class TestFormatConversionNoTorchVenv:
sys.modules['utils.hardware'] = hardware_mod
# Read and exec format_conversion.py
source = open({str(FORMAT_CONVERSION)!r}).read()
source = open({str(FORMAT_CONVERSION)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .iterable import', 'from iterable import')
ns = {{'__name__': '__test__'}}
@ -463,7 +463,7 @@ class TestFormatConversionNoTorchVenv:
sys.modules['utils'] = utils_mod
sys.modules['utils.hardware'] = hardware_mod
source = open({str(FORMAT_CONVERSION)!r}).read()
source = open({str(FORMAT_CONVERSION)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .iterable import', 'from iterable import')
ns = {{'__name__': '__test__'}}
@ -517,7 +517,7 @@ class TestNegativeControls:
loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers
exec(open({temp_file!r}).read())
exec(open({temp_file!r}, encoding = "utf-8").read())
""")
result = subprocess.run(
[no_torch_venv, "-c", code],

2
tests/python/test_v100_fullft_precision.py
View file

@ -29,7 +29,7 @@ RL_PY = Path(__file__).resolve().parents[2] / "unsloth" / "models" / "rl.py"
def _extract_mixed_precision_code() -> str:
lines = RL_PY.read_text().split("\n")
lines = RL_PY.read_text(encoding = "utf-8").split("\n")
try:
start = next(i for i, l in enumerate(lines) if "mixed_precision = (" in l)
except StopIteration:

2
tests/python/test_vision_lora_targeting.py
View file

@ -37,7 +37,7 @@ def test_vlm_lora_regex_respects_language_only_with_explicit_targets():
def test_fast_vision_model_wraps_explicit_targets_when_layer_filters_are_used():
source = Path("unsloth/models/vision.py").read_text()
source = Path("unsloth/models/vision.py").read_text(encoding = "utf-8")
assert "target_modules = get_peft_regex(" in source
assert "target_modules = list(target_modules)" in source

4
tests/saving/test_fix_sentencepiece_gguf_robustness.py
View file

@ -82,7 +82,7 @@ def test_entry_with_non_int_id_is_skipped(tmp_path):
def test_save_py_except_clause_is_broad_exception():
with open(_SAVE_PY) as f:
with open(_SAVE_PY, encoding = "utf-8") as f:
tree = ast.parse(f.read())
for node in ast.walk(tree):
if isinstance(node, ast.FunctionDef) and node.name == "unsloth_save_pretrained_gguf":
@ -102,7 +102,7 @@ def test_save_py_except_clause_is_broad_exception():
def test_tokenizer_utils_uses_import_protobuf_fallback_pattern():
with open(_TOK_PY) as f:
with open(_TOK_PY, encoding = "utf-8") as f:
src = f.read()
tree = ast.parse(src)
for node in ast.walk(tree):

10
tests/security/test_scan_packages.py
View file

@ -35,9 +35,11 @@ def test_fixture_bytes_are_deterministic(tmp_path):
rebuild_dir = tmp_path / "rebuild"
rebuild_dir.mkdir()
# The build helper writes to its own dir; copy + patch HERE.
builder_src = (FIXTURES / "_build.py").read_text()
builder_src = (FIXTURES / "_build.py").read_text(encoding = "utf-8")
rebuilt_helper = rebuild_dir / "_build.py"
rebuilt_helper.write_text(builder_src)
# builder_src came out of a checked-in file, so it carries whatever
# non-ASCII that file holds and cp1252 cannot encode it back out.
rebuilt_helper.write_text(builder_src, encoding = "utf-8")
# Run with SOURCE_DATE_EPOCH=0 and HERE override via a shim.
shim = rebuild_dir / "run.py"
shim.write_text(
@ -1260,7 +1262,7 @@ def test_committed_baseline_suppresses_known_but_not_a_new_payload():
import json
baseline_path = REPO_ROOT / "scripts" / "scan_packages_baseline.json"
entries = json.loads(baseline_path.read_text())["entries"]
entries = json.loads(baseline_path.read_text(encoding = "utf-8"))["entries"]
target = next(
e
for e in entries
@ -1296,7 +1298,7 @@ def test_committed_baseline_entries_all_carry_evidence_hash():
import json
baseline_path = REPO_ROOT / "scripts" / "scan_packages_baseline.json"
entries = json.loads(baseline_path.read_text())["entries"]
entries = json.loads(baseline_path.read_text(encoding = "utf-8"))["entries"]
assert entries, "committed baseline should not be empty"
missing = [
f"{e['package']}:{e['file']}:{e['check']}" for e in entries if not e.get("evidence_hash")

4
tests/studio/install/test_llama_pr_force_and_source.py
View file

@ -346,7 +346,7 @@ class TestSourcePatternsSh:
@pytest.fixture(autouse = True)
def _load_source(self):
self.content = SETUP_SH.read_text()
self.content = SETUP_SH.read_text(encoding = "utf-8")
def test_has_default_pr_force(self):
assert '_DEFAULT_LLAMA_PR_FORCE=""' in self.content
@ -412,7 +412,7 @@ class TestSourcePatternsPs1:
@pytest.fixture(autouse = True)
def _load_source(self):
self.content = SETUP_PS1.read_text()
self.content = SETUP_PS1.read_text(encoding = "utf-8")
def test_has_default_pr_force(self):
assert '$DefaultLlamaPrForce = ""' in self.content

6
tests/studio/install/test_managed_node_runtime.py
View file

@ -125,7 +125,7 @@ def test_resolve_falls_back_to_managed_when_no_system(monkeypatch, tmp_path):
monkeypatch.setenv("UNSLOTH_STUDIO_HOME", str(tmp_path))
managed = nr.managed_node_binary()
managed.parent.mkdir(parents = True, exist_ok = True)
managed.write_text("#!/bin/sh\necho v24.17.0\n")
managed.write_text("#!/bin/sh\necho v24.17.0\n", encoding = "utf-8")
monkeypatch.setattr(nr.shutil, "which", lambda name: None)
monkeypatch.setattr(nr, "_node_version_ok", lambda exe: str(exe) == str(managed))
assert nr.resolve_node_executable() == str(managed)
@ -136,7 +136,7 @@ def test_resolve_prefers_managed_over_unsuitable_system(monkeypatch, tmp_path):
monkeypatch.setenv("UNSLOTH_STUDIO_HOME", str(tmp_path))
managed = nr.managed_node_binary()
managed.parent.mkdir(parents = True, exist_ok = True)
managed.write_text("fake")
managed.write_text("fake", encoding = "utf-8")
monkeypatch.setattr(nr.shutil, "which", lambda name: "/old/node")
monkeypatch.setattr(nr, "_node_version_ok", lambda exe: str(exe) == str(managed))
assert nr.resolve_node_executable() == str(managed)
@ -167,7 +167,7 @@ def test_negative_result_is_not_cached(monkeypatch, tmp_path):
managed = nr.managed_node_binary()
managed.parent.mkdir(parents = True, exist_ok = True)
managed.write_text("now-installed")
managed.write_text("now-installed", encoding = "utf-8")
monkeypatch.setattr(nr, "_node_version_ok", lambda exe: str(exe) == str(managed))
assert nr.resolve_node_executable() == str(managed)

46
tests/studio/install/test_pr4562_bugfixes.py
View file

@ -617,7 +617,7 @@ class TestSourceCodePatterns:
def test_setup_sh_no_rm_before_prereq_check(self):
"""rm -rf must appear AFTER cmake/git checks, not before."""
content = SETUP_SH.read_text()
content = SETUP_SH.read_text(encoding = "utf-8")
# Anchor on the source-build cmake check block.
idx_block = content.find("command -v cmake")
assert idx_block != -1
@ -630,7 +630,7 @@ class TestSourceCodePatterns:
def test_setup_sh_clone_uses_branch_tag(self):
"""git clone in source-build should use --branch via the clone args array."""
content = SETUP_SH.read_text()
content = SETUP_SH.read_text(encoding = "utf-8")
assert "_CLONE_ARGS=(git clone --depth 1)" in content
assert (
'_CLONE_ARGS+=(--branch "$_RESOLVED_SOURCE_REF")' in content
@ -642,7 +642,7 @@ class TestSourceCodePatterns:
def test_setup_sh_source_build_uses_helper_latest_tag_only(self):
"""Shell source fallback should only use helper latest-tag resolution."""
content = SETUP_SH.read_text()
content = SETUP_SH.read_text(encoding = "utf-8")
assert "--resolve-source-build" not in content
assert "--resolve-install-tag" not in content
assert '--resolve-llama-tag latest --published-repo "ggml-org/llama.cpp"' in content
@ -653,7 +653,7 @@ class TestSourceCodePatterns:
def test_setup_sh_prebuilt_install_entrypoint(self):
"""Shell prebuilt path uses the helper install entrypoint, not the old releases-latest flow."""
content = SETUP_SH.read_text()
content = SETUP_SH.read_text(encoding = "utf-8")
assert "--resolve-install-tag" not in content
assert "_HELPER_RELEASE_REPO}/releases/latest" not in content
assert "ggml-org/llama.cpp/releases/latest" not in content
@ -663,7 +663,7 @@ class TestSourceCodePatterns:
fork like every other host, so the release-repo decision is unconditional.
Guards against a silent reintroduction of a ggml-org CPU routing branch.
GPU usability detection (used for PyTorch / source decisions) must stay."""
content = SETUP_SH.read_text()
content = SETUP_SH.read_text(encoding = "utf-8")
assert '_HELPER_RELEASE_REPO="unslothai/llama.cpp"' in content
assert '_HELPER_RELEASE_REPO="ggml-org/llama.cpp"' not in content
# Usability gating (not routing) still distinguishes a hidden GPU.
@ -676,14 +676,14 @@ class TestSourceCodePatterns:
def test_setup_sh_reports_installed_prebuilt_release(self):
"""Shell wrapper should report the installed prebuilt release from metadata."""
content = SETUP_SH.read_text()
content = SETUP_SH.read_text(encoding = "utf-8")
assert "UNSLOTH_PREBUILT_INFO.json" in content
assert "installed release:" in content
assert 'print_installed_llama_prebuilt_release "$LLAMA_CPP_DIR"' in content
def test_setup_sh_macos_arm64_uses_metal_flags(self):
"""Apple Silicon source builds should explicitly enable Metal like upstream."""
content = SETUP_SH.read_text()
content = SETUP_SH.read_text(encoding = "utf-8")
assert "_IS_MACOS_ARM64=true" in content
assert 'if [ "$_IS_MACOS_ARM64" = true ]; then' in content
assert "-DGGML_METAL=ON" in content
@ -695,7 +695,7 @@ class TestSourceCodePatterns:
def test_setup_sh_macos_metal_configure_has_cpu_fallback(self):
"""GPU configure/build failure retries a CPU build. Stays label-agnostic
(PR #5826 generalised the Metal-only wording via $_FB_LABEL)."""
content = SETUP_SH.read_text()
content = SETUP_SH.read_text(encoding = "utf-8")
assert "_TRY_METAL_CPU_FALLBACK=true" in content
assert 'configure failed; retrying CPU build..." "$C_WARN"' in content
assert 'build failed; retrying CPU build..." "$C_WARN"' in content
@ -714,7 +714,7 @@ class TestSourceCodePatterns:
"""PR #5826: a fresh CUDA toolkit's host-compiler whitelist lags distro gcc/clang
(nvcc "#error -- unsupported GNU version"). setup.sh exports
NVCC_PREPEND_FLAGS=-allow-unsupported-compiler via env, not CMAKE_ARGS (word-splitting safety)."""
content = SETUP_SH.read_text()
content = SETUP_SH.read_text(encoding = "utf-8")
assert "-allow-unsupported-compiler" in content
# Via NVCC_PREPEND_FLAGS (covers the configure-time probe too), not CMAKE_ARGS.
assert "export NVCC_PREPEND_FLAGS=" in content
@ -726,7 +726,7 @@ class TestSourceCodePatterns:
def test_setup_ps1_exports_allow_unsupported_compiler(self):
"""Windows parity for PR #5826: CUDA toolkit whitelist lags MSVC. setup.ps1 sets
NVCC_PREPEND_FLAGS=-allow-unsupported-compiler in the CUDA branch via env, out of $CmakeArgs."""
content = SETUP_PS1.read_text()
content = SETUP_PS1.read_text(encoding = "utf-8")
assert "-allow-unsupported-compiler" in content
# Via process env, not $CmakeArgs, so it reaches both the configure probe and `cmake --build`.
assert "$env:NVCC_PREPEND_FLAGS" in content
@ -763,7 +763,7 @@ class TestSourceCodePatterns:
def test_setup_sh_does_not_enable_metal_for_intel_macos(self):
"""Intel macOS should stay on the existing non-Metal path in this patch."""
content = SETUP_SH.read_text()
content = SETUP_SH.read_text(encoding = "utf-8")
assert 'if [ "$_IS_MACOS_ARM64" = true ]; then' in content
assert (
'Darwin" ] && { [ "$_HOST_MACHINE" = "arm64" ] || [ "$_HOST_MACHINE" = "aarch64" ]; }'
@ -778,20 +778,20 @@ class TestSourceCodePatterns:
def test_setup_ps1_uses_checkout_b(self):
"""PS1 should use checkout -B, not checkout --force FETCH_HEAD."""
content = SETUP_PS1.read_text()
content = SETUP_PS1.read_text(encoding = "utf-8")
assert "checkout -B unsloth-llama-build" in content
assert "checkout --force FETCH_HEAD" not in content
def test_setup_ps1_clone_uses_branch_tag(self):
"""PS1 clone should use --branch with the resolved tag."""
content = SETUP_PS1.read_text()
content = SETUP_PS1.read_text(encoding = "utf-8")
assert "--branch" in content and "$ResolvedSourceRef" in content
# The old commented-out clone line should be gone.
assert "# git clone --depth 1 --branch" not in content
def test_setup_ps1_no_git_pull(self):
"""PS1 should use fetch, not pull (which fails in detached HEAD)."""
content = SETUP_PS1.read_text()
content = SETUP_PS1.read_text(encoding = "utf-8")
# No "git pull" in the source-build section (only valid on a branch).
lines = content.splitlines()
for i, line in enumerate(lines):
@ -800,18 +800,18 @@ class TestSourceCodePatterns:
# Allowed elsewhere; fail only in the llama.cpp build section.
context = "\n".join(lines[max(0, i - 5) : i + 5])
if "LlamaCppDir" in context:
pytest.fail(f"Found 'git pull' in llama.cpp build section at line {i+1}")
pytest.fail(f"Found 'git pull' in llama.cpp build section at line {i + 1}")
def test_setup_ps1_prebuilt_install_entrypoint(self):
"""PS1 prebuilt path uses the helper install entrypoint, not the old releases-latest flow."""
content = SETUP_PS1.read_text()
content = SETUP_PS1.read_text(encoding = "utf-8")
assert "--resolve-install-tag" not in content
assert "$HelperReleaseRepo/releases/latest" not in content
assert "ggml-org/llama.cpp/releases/latest" not in content
def test_setup_ps1_reports_installed_prebuilt_release(self):
"""PS1 wrapper should report the installed prebuilt release from metadata."""
content = SETUP_PS1.read_text()
content = SETUP_PS1.read_text(encoding = "utf-8")
assert "Get-InstalledLlamaPrebuiltRelease" in content
assert "UNSLOTH_PREBUILT_INFO.json" in content
assert "installed release:" in content
@ -822,7 +822,7 @@ class TestSourceCodePatterns:
def test_setup_ps1_source_build_uses_helper_latest_tag_only(self):
"""PS1 source fallback should only use helper latest-tag resolution."""
content = SETUP_PS1.read_text()
content = SETUP_PS1.read_text(encoding = "utf-8")
assert "--resolve-source-build" not in content
assert "--resolve-install-tag" not in content
assert (
@ -835,7 +835,7 @@ class TestSourceCodePatterns:
def test_setup_ps1_prebuilt_install_disables_native_error_abort(self):
"""PS1 prebuilt install should not abort setup on helper stderr."""
content = SETUP_PS1.read_text()
content = SETUP_PS1.read_text(encoding = "utf-8")
install_idx = content.index("& python @prebuiltArgs 2>&1")
block = content[max(0, install_idx - 800) : install_idx + 800]
assert "$PSNativeCommandUseErrorActionPreference = $false" in block
@ -844,7 +844,7 @@ class TestSourceCodePatterns:
def test_setup_ps1_helper_disables_error_action_abort(self):
"""Helper resolution should suppress terminating NativeCommandError on PS 5.1."""
content = SETUP_PS1.read_text()
content = SETUP_PS1.read_text(encoding = "utf-8")
helper_idx = content.index("function Invoke-LlamaHelper")
block = content[helper_idx : helper_idx + 2200]
assert "$previousErrorActionPreference = $ErrorActionPreference" in block
@ -853,19 +853,19 @@ class TestSourceCodePatterns:
def test_setup_ps1_uses_local_tempfile_helper(self):
"""PS1 should not depend on New-TemporaryFile being available anywhere."""
content = SETUP_PS1.read_text()
content = SETUP_PS1.read_text(encoding = "utf-8")
assert "function New-UnslothTemporaryFile" in content
assert "$resolveErrorLog = New-TemporaryFile" not in content
def test_setup_ps1_find_nvcc_uses_version_sort_for_latest_toolkit(self):
"""The unconstrained nvcc fallback should not sort toolkit dirs lexicographically."""
content = SETUP_PS1.read_text()
content = SETUP_PS1.read_text(encoding = "utf-8")
assert "Sort-Object Name | Select-Object -Last 1" not in content
assert "Sort-Object { [version]($_.Name -replace '^v','') } -Descending" in content
def test_binary_env_linux_has_binary_parent(self):
"""The Linux branch of binary_env should include binary_path.parent."""
content = MODULE_PATH.read_text()
content = MODULE_PATH.read_text(encoding = "utf-8")
in_func = False
in_linux = False
found = False

6
tests/studio/load_freeze/test_load_orchestrator.py
View file

@ -441,7 +441,7 @@ def test_load_model_caches_audio_type_inside_serial_load_lock():
"""Audio-type detection must run inside load_model under _serial_load_lock,
else a concurrent /load can replace the backend mid-probe (review on #5669)."""
f = _REPO_ROOT / "studio" / "backend" / "core" / "inference" / "llama_cpp.py"
text = f.read_text()
text = f.read_text(encoding = "utf-8")
assert (
"with self._serial_load_lock" in text
), "LlamaCppBackend.load_model must hold self._serial_load_lock"
@ -462,7 +462,7 @@ def test_routes_inference_reads_cached_audio_type_not_calls_detect():
"""routes/inference.py must read cached _audio_type/_is_audio, not call
detect_audio_type / init_audio_codec directly (both moved into load_model)."""
f = _REPO_ROOT / "studio" / "backend" / "routes" / "inference.py"
text = f.read_text()
text = f.read_text(encoding = "utf-8")
assert "llama_backend.detect_audio_type(" not in text, (
"routes/inference.py should not call detect_audio_type directly; "
"load_model already cached it under the lock."
@ -485,7 +485,7 @@ def test_no_other_async_route_calls_detect_audio_type_unwrapped():
# function helper is excluded below.
pattern = re.compile(r"\b\w+\.detect_audio_type\s*\(")
for path in routes_dir.rglob("*.py"):
for i, line in enumerate(path.read_text().splitlines(), start = 1):
for i, line in enumerate(path.read_text(encoding = "utf-8").splitlines(), start = 1):
m = pattern.search(line)
if not m:
continue

10
tests/studio/playwright_chat_ime_i18n.py
View file

@ -241,10 +241,12 @@ with sync_playwright() as p:
# Source-level guard: grep the unmounted edit/compare composers' JSX for dir="auto".
_repo_root = Path(__file__).resolve().parents[2]
_thread_src = (
_repo_root / "studio/frontend/src/components/assistant-ui/thread.tsx"
).read_text()
_shared_src = (_repo_root / "studio/frontend/src/features/chat/shared-composer.tsx").read_text()
_thread_src = (_repo_root / "studio/frontend/src/components/assistant-ui/thread.tsx").read_text(
encoding = "utf-8"
)
_shared_src = (_repo_root / "studio/frontend/src/features/chat/shared-composer.tsx").read_text(
encoding = "utf-8"
)
_edit_idx = _thread_src.find("aui-edit-composer-input")
if _edit_idx == -1 or 'dir="auto"' not in _thread_src[_edit_idx : _edit_idx + 600]:
soft_fail('edit composer source is missing dir="auto"')

2
tests/studio/playwright_chat_ui.py
View file

@ -98,7 +98,7 @@ def expected_default_model():
/ "defaults.py"
)
try:
tree = ast.parse(defaults_path.read_text())
tree = ast.parse(defaults_path.read_text(encoding = "utf-8"))
except Exception as exc:
fail(f"could not read {defaults_path}: {exc}")
models = None

2
tests/studio/studio_api_smoke.py
View file

@ -142,7 +142,7 @@ except Exception as exc:
# GET / cross-origin must NOT leak the bootstrap password in the served HTML.
boot_path = AUTH_DIR / ".bootstrap_password"
if boot_path.exists():
bootstrap_pw = boot_path.read_text().strip()
bootstrap_pw = boot_path.read_text(encoding = "utf-8").strip()
if bootstrap_pw:
req = urllib.request.Request(
f"{BASE}/",

28
tests/studio/test_auth_form_input_count.py
View file

@ -51,7 +51,7 @@ def _conditional_extent(src: str) -> tuple[int, int]:
def test_hasbootstrappassword_constant_is_derived_from_bootstrap_window_value():
"""The guard must read from window.__UNSLOTH_BOOTSTRAP__, matching the backend's
bootstrap-injection contract in studio/backend/main.py::_inject_bootstrap."""
src = AUTH_FORM.read_text()
src = AUTH_FORM.read_text(encoding = "utf-8")
assert "const hasBootstrapPassword = Boolean(window.__UNSLOTH_BOOTSTRAP__?.password);" in src, (
"hasBootstrapPassword constant missing or its derivation drifted; "
"this is the gate that hides the Current password input on first boot"
@ -61,7 +61,7 @@ def test_hasbootstrappassword_constant_is_derived_from_bootstrap_window_value():
def test_exactly_one_hasBootstrapPassword_conditional_exists():
"""Only one `!hasBootstrapPassword` JSX check is allowed; a second would split
rendering into branches and likely hide or duplicate the New / Confirm inputs."""
src = AUTH_FORM.read_text()
src = AUTH_FORM.read_text(encoding = "utf-8")
count = src.count("!hasBootstrapPassword")
assert count == 1, (
f"expected exactly one !hasBootstrapPassword usage, found {count}; "
@ -72,7 +72,7 @@ def test_exactly_one_hasBootstrapPassword_conditional_exists():
def test_current_password_input_is_inside_the_hasBootstrapPassword_conditional():
"""`id="current-password"` must sit inside `{!hasBootstrapPassword && (...)}`,
else it renders on first boot too, regressing the pre-#5490 UX that PR #5545 restores."""
src = AUTH_FORM.read_text()
src = AUTH_FORM.read_text(encoding = "utf-8")
s, e = _conditional_extent(src)
idx = src.find('id="current-password"')
assert idx != -1, "the Current password input was removed entirely"
@ -86,7 +86,7 @@ def test_current_password_input_is_inside_the_hasBootstrapPassword_conditional()
def test_new_password_input_is_outside_the_hasBootstrapPassword_conditional():
"""`id="new-password"` must sit outside `{!hasBootstrapPassword && (...)}`,
else it disappears on admin-forced resets, regressing PR #5490."""
src = AUTH_FORM.read_text()
src = AUTH_FORM.read_text(encoding = "utf-8")
s, e = _conditional_extent(src)
idx = src.find('id="new-password"')
assert idx != -1, "the New password input was removed entirely"
@ -99,7 +99,7 @@ def test_new_password_input_is_outside_the_hasBootstrapPassword_conditional():
def test_confirm_password_input_is_outside_the_hasBootstrapPassword_conditional():
"""Same as New password, for `id="confirm-password"`."""
src = AUTH_FORM.read_text()
src = AUTH_FORM.read_text(encoding = "utf-8")
s, e = _conditional_extent(src)
idx = src.find('id="confirm-password"')
assert idx != -1, "the Confirm password input was removed entirely"
@ -114,7 +114,7 @@ def test_change_password_jsx_declares_exactly_three_password_inputs():
"""The change-password JSX block (`{!isLoginMode && (...)}`) must declare exactly
current/new/confirm; a fourth would break the 2-input first-boot contract (the
conditional only hides Current)."""
src = AUTH_FORM.read_text()
src = AUTH_FORM.read_text(encoding = "utf-8")
start = src.find("{!isLoginMode && (")
assert start != -1, (
"the change-password JSX subtree marker {!isLoginMode && (...)} "
@ -147,7 +147,7 @@ def test_change_password_jsx_declares_exactly_three_password_inputs():
def test_login_jsx_declares_exactly_one_password_input():
"""The login JSX block (`isLoginMode && (...)`) must declare exactly one password
input (the bootstrap password pasted from the CLI); a second breaks the per-mode matrix."""
src = AUTH_FORM.read_text()
src = AUTH_FORM.read_text(encoding = "utf-8")
start = src.find("{isLoginMode && (")
assert start != -1, "the login JSX subtree marker is missing"
depth = 1
@ -163,18 +163,20 @@ def test_login_jsx_declares_exactly_one_password_input():
ids = re.findall(r'id="([a-z-]+)"', subtree)
# Lock the count, not the spelling, so a rename does not falsely fail.
pw_ids = [x for x in ids if "password" in x]
assert len(pw_ids) == 1, (
f"login JSX must declare exactly one password-typed input; " f"found {pw_ids!r}"
)
assert (
len(pw_ids) == 1
), f"login JSX must declare exactly one password-typed input; found {pw_ids!r}"
def test_auth_flow_routes_do_not_mount_global_settings():
root = (FRONTEND / "app/routes/__root.tsx").read_text()
root = (FRONTEND / "app/routes/__root.tsx").read_text(encoding = "utf-8")
assert "{!isAuthFlowRoute && <SettingsDialog />}" in root
assert "useSettingsDialogStore.getState().closeDialog();" in root
assert "if (isAuthFlowRoute) return;" in root
for route in ("login", "change-password", "onboarding"):
assert "isAuthFlow: true" in (FRONTEND / f"app/routes/{route}.tsx").read_text()
assert "isAuthFlow: true" in (FRONTEND / f"app/routes/{route}.tsx").read_text(
encoding = "utf-8"
)
def test_auth_redirect_targets_are_idempotent_and_concurrent(tmp_path: Path):
@ -190,7 +192,7 @@ def test_auth_redirect_targets_are_idempotent_and_concurrent(tmp_path: Path):
pytest.skip("node --experimental-strip-types not available")
source = (
AUTH_API.read_text()
AUTH_API.read_text(encoding = "utf-8")
.replace('from "@/lib/api-base"', 'from "./stubs.mjs"')
.replace('from "./session"', 'from "./stubs.mjs"')
)

2
tests/studio/test_cancel_atomicity.py
View file

@ -9,7 +9,7 @@ from pathlib import Path
SOURCE_PATH = Path(__file__).resolve().parents[2] / "studio" / "backend" / "routes" / "inference.py"
_SRC = SOURCE_PATH.read_text()
_SRC = SOURCE_PATH.read_text(encoding = "utf-8")
_TREE = ast.parse(_SRC)

12
tests/studio/test_cancel_id_wiring.py
View file

@ -13,10 +13,14 @@ from pathlib import Path
WORKSPACE = Path(__file__).resolve().parents[2]
MODELS_SRC = (WORKSPACE / "studio/backend/models/inference.py").read_text()
ROUTES_SRC = (WORKSPACE / "studio/backend/routes/inference.py").read_text()
ADAPTER_SRC = (WORKSPACE / "studio/frontend/src/features/chat/api/chat-adapter.ts").read_text()
API_TYPES_SRC = (WORKSPACE / "studio/frontend/src/features/chat/types/api.ts").read_text()
MODELS_SRC = (WORKSPACE / "studio/backend/models/inference.py").read_text(encoding = "utf-8")
ROUTES_SRC = (WORKSPACE / "studio/backend/routes/inference.py").read_text(encoding = "utf-8")
ADAPTER_SRC = (WORKSPACE / "studio/frontend/src/features/chat/api/chat-adapter.ts").read_text(
encoding = "utf-8"
)
API_TYPES_SRC = (WORKSPACE / "studio/frontend/src/features/chat/types/api.ts").read_text(
encoding = "utf-8"
)
def _find_class(tree: ast.AST, name: str) -> ast.ClassDef | None:

8
tests/studio/test_chat_preset_builtin_invariants.py
View file

@ -40,13 +40,15 @@ def _require_node():
def _ensure_harness():
TEMP.mkdir(parents = True, exist_ok = True)
(TEMP / "register.mjs").write_text(
"import { register } from 'node:module';\nregister('./loader.mjs', import.meta.url);\n"
"import { register } from 'node:module';\nregister('./loader.mjs', import.meta.url);\n",
encoding = "utf-8",
)
(TEMP / "loader.mjs").write_text(
"export function resolve(specifier, context, next) {\n"
" if (specifier.endsWith('/types/runtime')) return next(specifier + '.ts', context);\n"
" return next(specifier, context);\n"
"}\n"
"}\n",
encoding = "utf-8",
)
@ -54,7 +56,7 @@ def _run(script: str):
_require_node()
_ensure_harness()
script_path = TEMP / "run.mts"
script_path.write_text(script)
script_path.write_text(script, encoding = "utf-8")
env = dict(os.environ, NODE_NO_WARNINGS = "1")
result = subprocess.run(
[

4
tests/studio/test_chat_prompt_variables.py
View file

@ -6,7 +6,9 @@ from pathlib import Path
WORKSPACE = Path(__file__).resolve().parents[2]
ADAPTER_SRC = (WORKSPACE / "studio/frontend/src/features/chat/api/chat-adapter.ts").read_text()
ADAPTER_SRC = (WORKSPACE / "studio/frontend/src/features/chat/api/chat-adapter.ts").read_text(
encoding = "utf-8"
)
def _function_source(name: str) -> str:

20
tests/studio/test_chat_response_details_ui_contract.py
View file

@ -20,14 +20,14 @@ CHAT_TAB_TSX = REPO / "studio/frontend/src/features/settings/tabs/chat-tab.tsx"
def test_assistant_more_menu_exposes_response_details_action():
src = THREAD_TSX.read_text()
src = THREAD_TSX.read_text(encoding = "utf-8")
assert "MessageResponseDetailsSheet" in src
assert "See response details" in src
assert "setDetailsOpen(true)" in src
def test_response_details_sheet_uses_unsloth_sheet_and_key_sections():
src = DETAILS_TSX.read_text()
src = DETAILS_TSX.read_text(encoding = "utf-8")
assert "SheetContent" in src
assert "Response details" in src
assert "MessageResponseModelBadge" in src
@ -45,17 +45,17 @@ def test_response_details_sheet_uses_unsloth_sheet_and_key_sections():
def test_response_model_badge_is_user_configurable_and_rendered_once_per_message():
prefs_src = CHAT_PREFS_TS.read_text()
chat_tab_src = CHAT_TAB_TSX.read_text()
thread_src = THREAD_TSX.read_text()
reasoning_src = REASONING_TSX.read_text()
prefs_src = CHAT_PREFS_TS.read_text(encoding = "utf-8")
chat_tab_src = CHAT_TAB_TSX.read_text(encoding = "utf-8")
thread_src = THREAD_TSX.read_text(encoding = "utf-8")
reasoning_src = REASONING_TSX.read_text(encoding = "utf-8")
assert "showResponseModel: boolean" in prefs_src
assert "showResponseModel: false" in prefs_src
assert "showResponseModel: saved?.showResponseModel ?? false" in prefs_src
assert "Show response model" in chat_tab_src
assert "setShowResponseModel" in chat_tab_src
details_src = DETAILS_TSX.read_text()
details_src = DETAILS_TSX.read_text(encoding = "utf-8")
assert (
"aui-response-model-badge pointer-events-none relative inline-flex min-h-5" in details_src
)
@ -76,7 +76,7 @@ def test_response_model_badge_is_user_configurable_and_rendered_once_per_message
def test_reasoning_keeps_streaming_height_cap_through_automatic_collapse():
src = REASONING_TSX.read_text()
src = REASONING_TSX.read_text(encoding = "utf-8")
assert "const [retainStreamingHeight, setRetainStreamingHeight]" in src
assert "setRetainStreamingHeight(false)" in src
@ -91,7 +91,7 @@ def test_reasoning_clears_manual_open_on_a_new_stream():
isOpen is `(streaming && !dismissed) || manualOpen` and manualOpen is only
settable while idle, so the new-stream reset has to clear it too.
"""
src = REASONING_TSX.read_text()
src = REASONING_TSX.read_text(encoding = "utf-8")
marker = "setDismissedWhileStreaming(false)"
start = src.find(marker)
@ -101,7 +101,7 @@ def test_reasoning_clears_manual_open_on_a_new_stream():
def test_response_details_metadata_is_persisted_without_backend_schema_change():
src = ADAPTER_TS.read_text()
src = ADAPTER_TS.read_text(encoding = "utf-8")
assert "interface ResponseDetailsMetadata" in src
assert "buildResponseDetails" in src
assert "responseDetails: buildResponseDetails(finishedAt)" in src

14
tests/studio/test_chat_title_generation.py
View file

@ -41,7 +41,7 @@ def _balanced_block(src: str, anchor: str) -> str:
def test_title_model_prompt_targets_conversation_topic():
block = _source_until(
RUNTIME_TSX.read_text(),
RUNTIME_TSX.read_text(encoding = "utf-8"),
"async function generateTitleWithModel",
"\nconst inflightTitleByKey",
)
@ -54,7 +54,7 @@ def test_title_model_prompt_targets_conversation_topic():
def test_title_model_payload_includes_optional_assistant_reply():
block = _source_until(
RUNTIME_TSX.read_text(),
RUNTIME_TSX.read_text(encoding = "utf-8"),
"async function generateTitleWithModel",
"\nconst inflightTitleByKey",
)
@ -71,7 +71,7 @@ def test_title_model_payload_includes_optional_assistant_reply():
def test_generate_title_passes_first_assistant_reply_after_first_user():
block = _balanced_block(
RUNTIME_TSX.read_text(),
RUNTIME_TSX.read_text(encoding = "utf-8"),
"async generateTitle(remoteId",
)
@ -84,7 +84,7 @@ def test_generate_title_passes_first_assistant_reply_after_first_user():
def test_tool_call_only_first_assistant_still_uses_first_user_message():
source = RUNTIME_TSX.read_text()
source = RUNTIME_TSX.read_text(encoding = "utf-8")
extract_block = " ".join(_balanced_block(source, "function extractTextParts").split())
generate_block = " ".join(_balanced_block(source, "async generateTitle(remoteId").split())
@ -104,7 +104,7 @@ def test_tool_call_only_first_assistant_still_uses_first_user_message():
def test_auto_title_disabled_uses_deterministic_user_text_fallback():
block = _balanced_block(
RUNTIME_TSX.read_text(),
RUNTIME_TSX.read_text(encoding = "utf-8"),
"async generateTitle(remoteId",
)
auto_title_off = _balanced_block(block, "if (!autoTitle)")
@ -114,7 +114,7 @@ def test_auto_title_disabled_uses_deterministic_user_text_fallback():
def test_model_failure_still_falls_back_to_user_text():
source = RUNTIME_TSX.read_text()
source = RUNTIME_TSX.read_text(encoding = "utf-8")
model_block = _source_until(
source,
"async function generateTitleWithModel",
@ -130,7 +130,7 @@ def test_model_failure_still_falls_back_to_user_text():
def test_title_normalizer_still_enforces_output_constraints():
block = _source_until(
RUNTIME_TSX.read_text(),
RUNTIME_TSX.read_text(encoding = "utf-8"),
"async function generateTitleWithModel",
"\nconst inflightTitleByKey",
)

4
tests/studio/test_cli_run_alias.py
View file

@ -17,7 +17,7 @@ def _module_calls(source: str):
def test_top_level_run_alias_registered():
"""`app.command("run", ...)` must be invoked with studio_run as its target."""
source = _CLI_INIT.read_text()
source = _CLI_INIT.read_text(encoding = "utf-8")
# Find ``app.command("run", ...)`` call -- the decorator-call form.
found_decorator_call = False
@ -46,7 +46,7 @@ def test_top_level_run_alias_registered():
def test_studio_run_imported_for_alias():
"""The alias must wire up to the studio.run function, not redefine it."""
source = _CLI_INIT.read_text()
source = _CLI_INIT.read_text(encoding = "utf-8")
tree = ast.parse(source)
has_import = False
for node in ast.walk(tree):

13
tests/studio/test_cli_studio_defaults.py
View file

@ -48,20 +48,19 @@ def _find_typer_option_default(source: str, func_name: str, long_option: str):
def test_studio_default_host_is_loopback():
"""`unsloth studio` (studio_default) --host default must be 127.0.0.1."""
source = _STUDIO_CMD_PY.read_text()
source = _STUDIO_CMD_PY.read_text(encoding = "utf-8")
host_default = _find_typer_option_default(source, "studio_default", "--host")
assert (
host_default is not None
), "Could not find --host typer.Option default in studio_default()"
assert host_default == "127.0.0.1", (
f"studio_default() --host default must be '127.0.0.1' (loopback) "
f"but got '{host_default}'."
)
assert (
host_default == "127.0.0.1"
), f"studio_default() --host default must be '127.0.0.1' (loopback) but got '{host_default}'."
def test_studio_run_host_is_loopback():
"""`unsloth studio run` --host default must be 127.0.0.1."""
source = _STUDIO_CMD_PY.read_text()
source = _STUDIO_CMD_PY.read_text(encoding = "utf-8")
host_default = _find_typer_option_default(source, "run", "--host")
assert host_default is not None, "Could not find --host typer.Option default in run()"
assert host_default == "127.0.0.1", (
@ -71,7 +70,7 @@ def test_studio_run_host_is_loopback():
def test_dns_pinning_opt_out_is_registered_safe_by_default():
source = _STUDIO_CMD_PY.read_text()
source = _STUDIO_CMD_PY.read_text(encoding = "utf-8")
for func_name in ("studio_default", "run"):
default = _find_typer_option_default(source, func_name, "--disable-dns-pinning")
assert default is False, f"{func_name} must keep DNS pinning enabled by default"

42
tests/studio/test_composer_rtl_bidi_attribute.py
View file

@ -26,22 +26,22 @@ def _block_around(
def test_main_composer_has_dir_auto():
# PR #5784 turned the attribute into a JSX conditional; anchor on the inner
# "Message input" literal, which survives both spellings.
block = _block_around(THREAD_TSX.read_text(), '"Message input"')
block = _block_around(THREAD_TSX.read_text(encoding = "utf-8"), '"Message input"')
assert 'dir="auto"' in block, 'main composer is missing dir="auto"'
def test_edit_composer_has_dir_auto():
block = _block_around(THREAD_TSX.read_text(), "aui-edit-composer-input")
block = _block_around(THREAD_TSX.read_text(encoding = "utf-8"), "aui-edit-composer-input")
assert 'dir="auto"' in block, 'edit composer is missing dir="auto"'
def test_compare_composer_has_dir_auto():
block = _block_around(SHARED_TSX.read_text(), "Send to both models")
block = _block_around(SHARED_TSX.read_text(encoding = "utf-8"), "Send to both models")
assert 'dir="auto"' in block, 'compare composer is missing dir="auto"'
def test_ime_workflow_step_does_not_set_studio_old_pw():
yml = WORKFLOW_YML.read_text()
yml = WORKFLOW_YML.read_text(encoding = "utf-8")
drive_idx = yml.find("Drive IME + multilingual paste regression")
assert drive_idx != -1, "IME drive step not found in workflow"
next_step_idx = yml.find("- name:", drive_idx + 1)
@ -53,7 +53,7 @@ def test_ime_workflow_step_does_not_set_studio_old_pw():
def test_ime_pass_password_step_does_not_export_old_pw():
yml = WORKFLOW_YML.read_text()
yml = WORKFLOW_YML.read_text(encoding = "utf-8")
pass_idx = yml.find("Pass bootstrap pw for IME / i18n test")
assert pass_idx != -1, "IME password setup step not found"
next_step_idx = yml.find("- name:", pass_idx + 1)
@ -65,7 +65,7 @@ def test_ime_pass_password_step_does_not_export_old_pw():
def test_ime_playwright_script_does_not_read_studio_old_pw():
src = IME_PY.read_text()
src = IME_PY.read_text(encoding = "utf-8")
code_only = re.sub(r'""".*?"""', "", src, flags = re.DOTALL)
assert (
"STUDIO_OLD_PW" not in code_only
@ -76,7 +76,7 @@ def test_ime_playwright_script_does_not_read_studio_old_pw():
def test_main_composer_has_stuck_compositionend_watchdog():
"""Issue #5546: WSL Chrome never emits compositionend after IME commit, so the
composer needs a watchdog releasing the composing flag or Send stays disabled."""
src = THREAD_TSX.read_text()
src = THREAD_TSX.read_text(encoding = "utf-8")
assert (
"IME_STUCK_TIMEOUT_MS" in src
), "main composer is missing the stuck-compositionend watchdog (issue #5546)"
@ -87,7 +87,7 @@ def test_main_composer_has_stuck_compositionend_watchdog():
def test_compare_composer_has_stuck_compositionend_watchdog():
src = SHARED_TSX.read_text()
src = SHARED_TSX.read_text(encoding = "utf-8")
assert (
"IME_STUCK_TIMEOUT_MS" in src
), "compare composer is missing the stuck-compositionend watchdog (issue #5546)"
@ -97,7 +97,7 @@ def test_compare_composer_has_stuck_compositionend_watchdog():
def test_main_composer_keydown_repins_composing_during_ime():
"""Issue #5546: the keydown IME gate must re-pin composingRef so a follow-up
Enter does not submit preedit text after the watchdog clears it."""
src = THREAD_TSX.read_text()
src = THREAD_TSX.read_text(encoding = "utf-8")
assert "onKeyDown" in src, "main composer is missing onKeyDown IME gate"
assert "e.nativeEvent.isComposing" in src and "keyCode === 229" in src, (
"main composer keydown gate must check both nativeEvent.isComposing "
@ -108,7 +108,7 @@ def test_main_composer_keydown_repins_composing_during_ime():
def test_compare_composer_keydown_repins_composing_during_ime():
"""Compare composer onKeyDown re-pins composingRef on IME keypress so a
follow-up click-Send during the watchdog window does not slip preedit text."""
src = SHARED_TSX.read_text()
src = SHARED_TSX.read_text(encoding = "utf-8")
assert "composingRef.current = true" in src, (
"compare composer keydown gate must re-pin composingRef when the "
"browser still considers the IME active"
@ -142,7 +142,7 @@ def _extract_block(
def test_main_composer_keydown_rearms_watchdog():
"""After keydown re-pins composingRef the watchdog must re-arm, else the
WSL+Chrome no-compositionend path locks Send after any IME keypress (#5546)."""
src = THREAD_TSX.read_text()
src = THREAD_TSX.read_text(encoding = "utf-8")
block = _extract_block(src, "const onKeyDown = useCallback")
assert "refreshStuckTimer" in block, (
"main composer keydown gate must call refreshStuckTimer after "
@ -159,12 +159,11 @@ def test_main_composer_keydown_rearms_watchdog():
def test_compare_composer_keydown_rearms_watchdog():
"""Same re-arm contract for the compare-mode composer."""
src = SHARED_TSX.read_text()
src = SHARED_TSX.read_text(encoding = "utf-8")
block = _extract_block(src, "function onKeyDown", opener = "{", closer = "}")
assert "refreshStuckImeTimer" in block, (
"compare composer keydown gate must call refreshStuckImeTimer "
"after re-pinning composingRef"
)
assert (
"refreshStuckImeTimer" in block
), "compare composer keydown gate must call refreshStuckImeTimer after re-pinning composingRef"
def _assert_enter_guard_before_immediate_recovery(block: str, refresh_call: str) -> None:
@ -177,10 +176,9 @@ def _assert_enter_guard_before_immediate_recovery(block: str, refresh_call: str)
"composingRef; candidate-confirming Enter must not submit"
)
guard_block = block[enter_idx:recovery_idx]
assert "preventDefault()" in guard_block, (
"Enter while composingRef is stuck must prevent the same key from "
"falling through to submit"
)
assert (
"preventDefault()" in guard_block
), "Enter while composingRef is stuck must prevent the same key from falling through to submit"
assert (
refresh_call in guard_block
), "Enter while composingRef is stuck must keep the watchdog armed"
@ -190,12 +188,12 @@ def _assert_enter_guard_before_immediate_recovery(block: str, refresh_call: str)
def test_main_composer_stuck_enter_does_not_clear_before_submit():
src = THREAD_TSX.read_text()
src = THREAD_TSX.read_text(encoding = "utf-8")
block = _extract_block(src, "const onKeyDown = useCallback")
_assert_enter_guard_before_immediate_recovery(block, "refreshStuckTimer")
def test_compare_composer_stuck_enter_does_not_clear_before_submit():
src = SHARED_TSX.read_text()
src = SHARED_TSX.read_text(encoding = "utf-8")
block = _extract_block(src, "function onKeyDown", opener = "{", closer = "}")
_assert_enter_guard_before_immediate_recovery(block, "refreshStuckImeTimer")

12
tests/studio/test_export_output_path_contract.py
View file

@ -32,7 +32,7 @@ def _return_tuple_arity(fn):
def test_export_methods_return_three_tuple_annotation():
tree = ast.parse(EXPORT.read_text())
tree = ast.parse(EXPORT.read_text(encoding = "utf-8"))
for fn_name in EXPORT_FNS:
fn = _find_method(tree, "ExportBackend", fn_name)
assert fn is not None, f"missing ExportBackend.{fn_name}"
@ -46,7 +46,7 @@ def test_export_methods_return_three_tuple_annotation():
def test_export_methods_return_three_element_tuples():
tree = ast.parse(EXPORT.read_text())
tree = ast.parse(EXPORT.read_text(encoding = "utf-8"))
for fn_name in EXPORT_FNS:
fn = _find_method(tree, "ExportBackend", fn_name)
assert fn is not None
@ -57,7 +57,7 @@ def test_export_methods_return_three_element_tuples():
def test_local_save_assigns_output_path():
tree = ast.parse(EXPORT.read_text())
tree = ast.parse(EXPORT.read_text(encoding = "utf-8"))
for fn_name in EXPORT_FNS:
fn = _find_method(tree, "ExportBackend", fn_name)
assert fn is not None
@ -74,7 +74,7 @@ def test_local_save_assigns_output_path():
def test_gpu_save_method_bound_for_hub_only():
tree = ast.parse(EXPORT.read_text())
tree = ast.parse(EXPORT.read_text(encoding = "utf-8"))
fn = _find_method(tree, "ExportBackend", "export_merged_model")
assert fn is not None
found_pre_save_method = False
@ -103,7 +103,7 @@ def test_gpu_save_method_bound_for_hub_only():
def test_mlx_hub_only_uses_temp_directory():
src = EXPORT.read_text()
src = EXPORT.read_text(encoding = "utf-8")
assert (
src.count("tempfile.TemporaryDirectory") >= 3
), "expected TemporaryDirectory in merged, base, and lora hub-push paths"
@ -111,7 +111,7 @@ def test_mlx_hub_only_uses_temp_directory():
def test_is_mlx_imported_from_unsloth():
src = EXPORT.read_text()
src = EXPORT.read_text(encoding = "utf-8")
assert "from unsloth import" in src
head = src.split("class ExportBackend")[0]
assert "_IS_MLX" in head

18
tests/studio/test_frontend_dep_removal.py
View file

@ -53,8 +53,7 @@ CASES: list[Case] = [
),
Case(
"C3",
"removing katex is safe: streamdown/math, mermaid, "
"rehype-katex all keep it at top level",
"removing katex is safe: streamdown/math, mermaid, rehype-katex all keep it at top level",
["katex"],
"PASS",
[],
@ -69,8 +68,7 @@ CASES: list[Case] = [
),
Case(
"C6",
"removing @radix-ui/react-slot is safe: pulled by "
"radix-ui umbrella + @assistant-ui/react",
"removing @radix-ui/react-slot is safe: pulled by radix-ui umbrella + @assistant-ui/react",
["@radix-ui/react-slot"],
"PASS",
[],
@ -852,7 +850,7 @@ ADV_CASES: list[AdvCase] = [
"A12",
"JSDoc @import of removed pkg should FAIL",
"adv12.ts",
'/** @type {import("__adv_only_pkg_l__").Foo} */\n' "const x = null;\n",
'/** @type {import("__adv_only_pkg_l__").Foo} */\nconst x = null;\n',
"__adv_only_pkg_l__",
"FAIL",
["__adv_only_pkg_l__"],
@ -1047,7 +1045,7 @@ PKG_FIELD_CASES: list[PkgFieldCase] = [
def run_pkg_field_cases() -> int:
head_pkg = json.loads(HEAD_PKG.read_text())
head_pkg = json.loads(HEAD_PKG.read_text(encoding = "utf-8"))
passed = 0
for pc in PKG_FIELD_CASES:
synth_head = json.loads(json.dumps(head_pkg))
@ -1110,13 +1108,13 @@ def run_pkg_field_cases() -> int:
def run_adversarial_cases() -> int:
ADVERSARIAL_TMP_DIR.mkdir(parents = True, exist_ok = True)
head_pkg = json.loads(HEAD_PKG.read_text())
head_pkg = json.loads(HEAD_PKG.read_text(encoding = "utf-8"))
passed = 0
for ac in ADV_CASES:
# Drop the synthetic file.
fpath = ADVERSARIAL_TMP_DIR / ac.filename
try:
fpath.write_text(ac.content)
fpath.write_text(ac.content, encoding = "utf-8")
# Base adds the target pkg; real head lacks it, so the script
# treats it as removed and scans the repo (now with our file).
synth_base = json.loads(json.dumps(head_pkg))
@ -1259,7 +1257,7 @@ ENUM_CASES: list[EnumCase] = [
def run_enum_cases() -> int:
head_pkg = json.loads(HEAD_PKG.read_text())
head_pkg = json.loads(HEAD_PKG.read_text(encoding = "utf-8"))
passed = 0
ADVERSARIAL_TMP_DIR.mkdir(parents = True, exist_ok = True)
for ec in ENUM_CASES:
@ -1508,7 +1506,7 @@ def run_wrapper_cases() -> int:
def main() -> int:
head_pkg = json.loads(HEAD_PKG.read_text())
head_pkg = json.loads(HEAD_PKG.read_text(encoding = "utf-8"))
print(f"Running {len(CASES)} edge cases against {SCRIPT.relative_to(REPO)}")
print()
results: list[tuple[Case, bool, str]] = []

2
tests/studio/test_is_mlx_dispatch_gate.py
View file

@ -27,7 +27,7 @@ UNSLOTH_INIT = REPO_ROOT / "unsloth" / "__init__.py"
def test_is_mlx_gate_uses_three_required_predicates():
"""_IS_MLX must AND Darwin+arm64+importable-mlx; dropping any breaks dispatch."""
tree = ast.parse(UNSLOTH_INIT.read_text())
tree = ast.parse(UNSLOTH_INIT.read_text(encoding = "utf-8"))
target = None
for node in ast.walk(tree):

2
tests/studio/test_llama_cpp_wall_clock_cap.py
View file

@ -14,7 +14,7 @@ SOURCE_PATH = (
/ "inference"
/ "llama_cpp.py"
)
SRC = SOURCE_PATH.read_text()
SRC = SOURCE_PATH.read_text(encoding = "utf-8")
TREE = ast.parse(SRC)

14
tests/studio/test_mlx_training_worker_behaviors.py
View file

@ -15,7 +15,7 @@ def _find_func(tree, name):
def test_run_mlx_training_passes_token_to_from_pretrained():
tree = ast.parse(WORKER.read_text())
tree = ast.parse(WORKER.read_text(encoding = "utf-8"))
fn = _find_func(tree, "_run_mlx_training")
assert fn is not None
found = False
@ -36,7 +36,7 @@ def test_run_mlx_training_passes_token_to_from_pretrained():
def test_wandb_init_strips_secret_keys():
src = WORKER.read_text()
src = WORKER.read_text(encoding = "utf-8")
assert "_wandb_sensitive" in src, "expected a sensitive-key set near wandb.init"
assert '"hf_token"' in src and '"wandb_token"' in src
assert (
@ -45,26 +45,26 @@ def test_wandb_init_strips_secret_keys():
def test_local_dataset_loader_uses_load_dataset_path():
src = WORKER.read_text()
src = WORKER.read_text(encoding = "utf-8")
assert "_resolve_mlx_local_dataset_files" in src
assert "_mlx_local_dataset_loader_for_files" in src
assert "data_files = all_files" in src or "data_files=all_files" in src
def test_send_aliases_status_message_to_message():
src = WORKER.read_text()
src = WORKER.read_text(encoding = "utf-8")
assert 'kwargs["message"] = sm' in src or 'kwargs["message"]=sm' in src
def test_slice_uses_inclusive_end_and_handles_zero():
src = WORKER.read_text()
src = WORKER.read_text(encoding = "utf-8")
assert "min(end + 1, len(ds))" in src or "min(end+1, len(ds))" in src
assert "slice_start if slice_start is not None else 0" in src
assert "slice_end if slice_end is not None else len(ds) - 1" in src
def test_poll_stop_returns_on_broken_pipe():
src = WORKER.read_text()
src = WORKER.read_text(encoding = "utf-8")
assert "except (EOFError, OSError)" in src
lines = src.splitlines()
for i, line in enumerate(lines):
@ -83,7 +83,7 @@ def test_poll_stop_returns_on_broken_pipe():
def test_unsloth_zoo_mlx_imports_have_friendly_error():
src = WORKER.read_text()
src = WORKER.read_text(encoding = "utf-8")
assert "from unsloth_zoo.mlx.loader import FastMLXModel" in src
assert "from unsloth_zoo.mlx.trainer import" in src
assert "raise ImportError" in src

2
tests/studio/test_model_picker_contracts.py
View file

@ -23,7 +23,7 @@ FRONTEND = WORKDIR / "studio" / "frontend" / "src"
def _read(rel: str) -> str:
path = FRONTEND / rel
assert path.exists(), f"missing source file: {path}"
return path.read_text()
return path.read_text(encoding = "utf-8")
def test_models_api_sends_token_via_header_not_query():

2
tests/studio/test_studio_gguf_export_script_pin.py
View file

@ -12,7 +12,7 @@ from pathlib import Path
SOURCE_PATH = (
Path(__file__).resolve().parents[2] / "studio" / "backend" / "core" / "export" / "export.py"
)
SRC = SOURCE_PATH.read_text()
SRC = SOURCE_PATH.read_text(encoding = "utf-8")
TREE = ast.parse(SRC)

2
tests/studio/test_studio_text_descender_clipping.py
View file

@ -24,7 +24,7 @@ APP_SIDEBAR = WORKDIR / "studio" / "frontend" / "src" / "components" / "app-side
def _read(path: Path) -> str:
assert path.exists(), f"missing source file: {path}"
return path.read_text()
return path.read_text(encoding = "utf-8")
def test_model_selector_trigger_label_uses_leading_tight():

2
tests/test_fast_generate_slow_guard.py
View file

@ -13,7 +13,7 @@ UTILS = os.path.join(HERE, "unsloth", "models", "_utils.py")
def _load_factory():
src = open(UTILS).read()
src = open(UTILS, encoding = "utf-8").read()
for node in ast.parse(src).body:
if isinstance(node, ast.FunctionDef) and node.name == "make_fast_generate_wrapper":
ns = {"functools": functools}

4
tests/test_fp8_device_context.py
View file

@ -78,7 +78,7 @@ class _LaunchVisitor(ast.NodeVisitor):
def _load_device_context_helper(fake_torch: _FakeTorch):
source = FP8_SOURCE.read_text()
source = FP8_SOURCE.read_text(encoding = "utf-8")
tree = ast.parse(source)
for node in tree.body:
if isinstance(node, ast.FunctionDef) and node.name == "_fp8_triton_device_context":
@ -144,7 +144,7 @@ def test_fp8_device_context_is_noop_for_non_cuda_tensor() -> None:
def test_fp8_triton_launches_enter_tensor_device_context() -> None:
tree = ast.parse(FP8_SOURCE.read_text())
tree = ast.parse(FP8_SOURCE.read_text(encoding = "utf-8"))
function_names = {node.name for node in ast.walk(tree) if isinstance(node, ast.FunctionDef)}
assert "_fp8_triton_device_context" in function_names

2
tests/test_gemma4_chat_template.py
View file

@ -14,7 +14,7 @@ CHAT_TEMPLATES_PATH = os.path.join(
def _extract_template(name):
src = open(CHAT_TEMPLATES_PATH).read()
src = open(CHAT_TEMPLATES_PATH, encoding = "utf-8").read()
pattern = rf'{re.escape(name)}\s*=\s*\\\n"""(.*?)"""'
m = re.search(pattern, src, flags = re.DOTALL)
assert m, f"Could not extract {name} from chat_templates.py"

2
tests/test_gemma_2b_mapper_key.py
View file

@ -18,7 +18,7 @@ MAPPER_PATH = os.path.join(os.path.dirname(__file__), os.pardir, "unsloth", "mod
def _load_mappers():
with open(MAPPER_PATH) as f:
with open(MAPPER_PATH, encoding = "utf-8") as f:
source = f.read()
namespace = {}
exec(compile(source, MAPPER_PATH, "exec"), namespace)

2
tests/test_generate_kwarg_gate.py
View file

@ -9,7 +9,7 @@ VISION = os.path.join(HERE, "unsloth", "models", "vision.py")
def _load_helper():
src = open(VISION).read()
src = open(VISION, encoding = "utf-8").read()
mod = ast.parse(src)
for node in mod.body:
if isinstance(node, ast.FunctionDef) and node.name == "_unsloth_generate_accepts_kwarg":

6
tests/test_gradient_checkpointing_restore.py
View file

@ -28,8 +28,8 @@ import re
from pathlib import Path
_ROOT = Path(__file__).resolve().parent.parent / "unsloth" / "models"
_RL = (_ROOT / "rl.py").read_text()
_RL_REPLACEMENTS = (_ROOT / "rl_replacements.py").read_text()
_RL = (_ROOT / "rl.py").read_text(encoding = "utf-8")
_RL_REPLACEMENTS = (_ROOT / "rl_replacements.py").read_text(encoding = "utf-8")
# The single-line ternary form used at the trainer call sites:
# <obj>._unsloth_gradient_checkpointing if hasattr(<obj>, '...') else getattr(<args>, 'gradient_checkpointing', True)
@ -162,7 +162,7 @@ def test_recording_sites_are_real_module_code():
# string. Assert it's present at the choke point (patch_peft_model, so loaded adapters
# are covered) and at the pre-wrapped pass-through, both of which bypass the old
# get_peft_model-only recording.
llama = (_ROOT / "llama.py").read_text()
llama = (_ROOT / "llama.py").read_text(encoding = "utf-8")
tree = ast.parse(llama)
def assigns_marker(node):

2
tests/test_import_fixes_drift.py
View file

@ -704,7 +704,7 @@ def test_accelerate_find_device_skips_empty_logits():
def test_accelerate_patch_wired_into_gpu_init():
"""The patch must be installed at startup, not only importable."""
source = Path(__file__).resolve().parent.parent / "unsloth" / "_gpu_init.py"
source = source.read_text()
source = source.read_text(encoding = "utf-8")
assert "patch_accelerate_recursively_apply()" in source, (
"DRIFT DETECTED: patch_accelerate_recursively_apply is defined but "
"never called in _gpu_init.py, so real imports never install it."

2
tests/test_loader_glob_skip.py
View file

@ -116,7 +116,7 @@ class TestLoaderSourceHasGuard(unittest.TestCase):
loader_path = os.path.join(
os.path.dirname(__file__), os.pardir, "unsloth", "models", "loader.py"
)
with open(loader_path) as f:
with open(loader_path, encoding = "utf-8") as f:
source = f.read()
lines = source.splitlines()

2
tests/test_multi_image_grpo_chunking.py
View file

@ -12,7 +12,7 @@ SOURCE_PATH = os.path.join(REPO_ROOT, "unsloth", "models", "rl_replacements.py")
def _read_source() -> str:
with open(SOURCE_PATH, "r") as fh:
with open(SOURCE_PATH, "r", encoding = "utf-8") as fh:
return fh.read()

2
tests/test_offload_embedding_hooks.py
View file

@ -11,7 +11,7 @@ VISION = os.path.join(HERE, "unsloth", "models", "vision.py")
def _load_installer():
src = open(VISION).read()
src = open(VISION, encoding = "utf-8").read()
mod = ast.parse(src)
for node in mod.body:
if isinstance(node, ast.FunctionDef) and node.name == "_install_offload_embedding_hooks":

2
tests/test_offload_tied_guard.py
View file

@ -11,7 +11,7 @@ VISION = os.path.join(HERE, "unsloth", "models", "vision.py")
def _load_fn():
src = open(VISION).read()
src = open(VISION, encoding = "utf-8").read()
mod = ast.parse(src)
for node in mod.body:
if isinstance(node, ast.FunctionDef) and node.name == "_embeddings_are_tied":

1252
tests/test_source_read_encoding.py Normal file
View file

File diff suppressed because it is too large Load diff

89
tests/test_studio_install_workspace_guard.py
View file

@ -15,16 +15,13 @@ SETUP_SH = REPO_ROOT / "studio" / "setup.sh"
# Stubs for helpers the extracted guard block calls; mv-based replacement reproduces the venv-gone
# effect without the full rollback machinery.
_INSTALL_GUARD_STUBS = (
"substep() { :; }\n"
"_start_studio_venv_replacement() {\n"
' mv -- "$1" "$1.replaced"\n'
"}\n"
'substep() { :; }\n_start_studio_venv_replacement() {\n mv -- "$1" "$1.replaced"\n}\n'
)
def _extract_install_sh_guard_block() -> str:
"""Extract install.sh's venv guard block (up to the first elif) as a self-contained snippet."""
src = INSTALL_SH.read_text()
src = INSTALL_SH.read_text(encoding = "utf-8")
m = re.search(
r'(if \[ -x "\$VENV_DIR/bin/python" \]; then\n.*?)elif \[ "\$_STUDIO_HOME_REDIRECT" != "env"',
src,
@ -119,7 +116,7 @@ def test_default_mode_skips_sentinel_check(tmp_path):
def test_install_ps1_has_matching_env_mode_guard():
src = INSTALL_PS1.read_text()
src = INSTALL_PS1.read_text(encoding = "utf-8")
block_start = src.index("if (Test-Path -LiteralPath $VenvPython)")
block = src[block_start : block_start + 2000]
assert (
@ -131,7 +128,7 @@ def test_install_ps1_has_matching_env_mode_guard():
def test_setup_ps1_has_writability_probe():
src = SETUP_PS1.read_text()
src = SETUP_PS1.read_text(encoding = "utf-8")
idx = src.index("if (Test-Path -LiteralPath $_studioOverride -PathType Container)")
block = src[idx : idx + 2000]
assert (
@ -193,7 +190,7 @@ def test_env_mode_passes_when_bin_unsloth_is_a_symlink(tmp_path):
def test_install_ps1_sentinel_uses_pathtype_leaf():
"""Remove-Item $VenvDir gate must use -PathType Leaf so a sentinel-path directory cannot satisfy it."""
src = INSTALL_PS1.read_text()
src = INSTALL_PS1.read_text(encoding = "utf-8")
block_start = src.index("if (Test-Path -LiteralPath $VenvPython)")
block = src[block_start : block_start + 2000]
assert (
@ -206,7 +203,7 @@ def test_install_ps1_sentinel_uses_pathtype_leaf():
def test_setup_ps1_stale_venv_has_env_mode_guard():
"""setup.ps1 stale-venv branch must gate Remove-Item $VenvDir on a custom-root Unsloth sentinel."""
src = SETUP_PS1.read_text()
src = SETUP_PS1.read_text(encoding = "utf-8")
idx = src.index("Stale venv detected")
block = src[idx : idx + 1500]
assert (
@ -226,7 +223,7 @@ def test_setup_ps1_stale_venv_has_env_mode_guard():
def test_setup_sh_prebuilt_llama_cpp_has_ownership_guard():
"""setup.sh prebuilt llama.cpp path must _assert_studio_owned_or_absent before install_llama_prebuilt.py."""
src = SETUP_SH.read_text()
src = SETUP_SH.read_text(encoding = "utf-8")
idx = src.index("installing prebuilt llama.cpp...")
block = src[idx : idx + 2000]
assert (
@ -240,7 +237,7 @@ def test_setup_sh_prebuilt_llama_cpp_has_ownership_guard():
def test_setup_ps1_prebuilt_llama_cpp_has_ownership_guard():
"""setup.ps1 prebuilt llama.cpp path must Assert-StudioOwnedOrAbsent before install_llama_prebuilt.py."""
src = SETUP_PS1.read_text()
src = SETUP_PS1.read_text(encoding = "utf-8")
idx = src.index("installing prebuilt llama.cpp bundle (preferred path)")
block = src[idx : idx + 2000]
assert (
@ -266,9 +263,9 @@ def test_env_mode_passes_when_venv_marker_present(tmp_path):
"""install.sh env-mode guard must accept the in-VENV .unsloth-studio-owned marker as a sentinel."""
studio_home = tmp_path / "ws"
res = _run_install_guard(studio_home, redirect = "env", create_venv_marker = True)
assert res.returncode == 0, (
f"in-VENV marker must allow cleanup; " f"stdout={res.stdout!r} stderr={res.stderr!r}"
)
assert (
res.returncode == 0
), f"in-VENV marker must allow cleanup; stdout={res.stdout!r} stderr={res.stderr!r}"
assert "RESULT=ok" in res.stdout
assert not (studio_home / "unsloth_studio").exists()
@ -318,16 +315,15 @@ def test_env_mode_blocks_when_bin_unsloth_is_broken_symlink(tmp_path):
text = True,
capture_output = True,
)
assert res.returncode != 0, (
"broken symlink at bin/unsloth must NOT pass; "
f"stdout={res.stdout!r} stderr={res.stderr!r}"
)
assert (
res.returncode != 0
), f"broken symlink at bin/unsloth must NOT pass; stdout={res.stdout!r} stderr={res.stderr!r}"
assert (venv / "important.txt").is_file()
def test_install_sh_writes_venv_marker_after_uv_venv():
"""install.sh must write .unsloth-studio-owned into $VENV_DIR right after `uv venv` succeeds."""
src = INSTALL_SH.read_text()
src = INSTALL_SH.read_text(encoding = "utf-8")
create_idx = src.index('run_install_cmd "create venv" uv venv "$VENV_DIR"')
tail = src[create_idx : create_idx + 600]
assert (
@ -337,7 +333,7 @@ def test_install_sh_writes_venv_marker_after_uv_venv():
def test_install_ps1_writes_venv_marker_after_uv_venv():
"""install.ps1 must write .unsloth-studio-owned into $VenvDir after `uv venv` succeeds."""
src = INSTALL_PS1.read_text()
src = INSTALL_PS1.read_text(encoding = "utf-8")
venv_create = src.index("uv venv $VenvDir --python")
tail = src[venv_create : venv_create + 1500]
assert (
@ -347,7 +343,7 @@ def test_install_ps1_writes_venv_marker_after_uv_venv():
def test_install_ps1_guard_accepts_venv_marker():
"""install.ps1 env-mode guard must accept the in-VENV .unsloth-studio-owned marker as a sentinel."""
src = INSTALL_PS1.read_text()
src = INSTALL_PS1.read_text(encoding = "utf-8")
block_start = src.index("if (Test-Path -LiteralPath $VenvPython)")
block = src[block_start : block_start + 2000]
assert (
@ -357,7 +353,7 @@ def test_install_ps1_guard_accepts_venv_marker():
def test_setup_helpers_gate_on_canonical_custom_root():
"""setup.sh/setup.ps1 ownership guards must gate on a canonical custom-vs-legacy root comparison."""
sh_src = SETUP_SH.read_text()
sh_src = SETUP_SH.read_text(encoding = "utf-8")
sh_idx = sh_src.index("_assert_studio_owned_or_absent() {")
sh_func = sh_src[sh_idx : sh_idx + 600]
assert (
@ -369,7 +365,7 @@ def test_setup_helpers_gate_on_canonical_custom_root():
and "_STUDIO_HOME_IS_CUSTOM=" in sh_src
), "setup.sh must compute the canonical custom-root flag"
ps_src = SETUP_PS1.read_text()
ps_src = SETUP_PS1.read_text(encoding = "utf-8")
ps_idx = ps_src.index("function Assert-StudioOwnedOrAbsent")
ps_func = ps_src[ps_idx : ps_idx + 800]
assert (
@ -382,7 +378,7 @@ def test_setup_helpers_gate_on_canonical_custom_root():
def test_setup_ps1_inplace_git_sync_marks_studio_owned():
"""setup.ps1 in-place git-sync branch must Mark-StudioOwned after a successful sync."""
src = SETUP_PS1.read_text()
src = SETUP_PS1.read_text(encoding = "utf-8")
inplace_idx = src.index('Test-Path -LiteralPath (Join-Path $LlamaCppDir ".git")')
# The in-place branch ends just before the temp-dir clone branch.
clone_idx = src.index("Cloning llama.cpp @", inplace_idx)
@ -397,7 +393,7 @@ def test_setup_ps1_inplace_git_sync_marks_studio_owned():
def test_setup_ps1_inplace_git_sync_asserts_studio_owned_before_mutation():
"""setup.ps1 in-place git-sync must Assert-StudioOwnedOrAbsent before any destructive git op."""
src = SETUP_PS1.read_text()
src = SETUP_PS1.read_text(encoding = "utf-8")
inplace_idx = src.index('Test-Path -LiteralPath (Join-Path $LlamaCppDir ".git")')
clone_idx = src.index("Cloning llama.cpp @", inplace_idx)
inplace_block = src[inplace_idx:clone_idx]
@ -410,7 +406,7 @@ def test_setup_ps1_inplace_git_sync_asserts_studio_owned_before_mutation():
def _extract_check_health_function() -> str:
src = INSTALL_SH.read_text()
src = INSTALL_SH.read_text(encoding = "utf-8")
fn_start = src.index("_check_health() {")
fn_end = src.index("\n}\n", fn_start) + 2
return src[fn_start:fn_end]
@ -498,7 +494,7 @@ def test_check_health_handles_arbitrary_id_token():
def test_install_ps1_test_studio_health_verifies_studio_root_id():
"""install.ps1 Test-StudioHealth must compare studio_root_id against baked $_ExpectedStudioRootId."""
src = INSTALL_PS1.read_text()
src = INSTALL_PS1.read_text(encoding = "utf-8")
fn_start = src.index("function Test-StudioHealth")
fn_end = src.index("\n}\n", fn_start) + 2
fn = src[fn_start:fn_end]
@ -510,7 +506,7 @@ def test_install_ps1_test_studio_health_verifies_studio_root_id():
def test_install_ps1_bakes_studio_root_id_into_launcher():
"""install.ps1 must persist a CSPRNG id at share/studio_install_id and bake it as $_ExpectedStudioRootId."""
src = INSTALL_PS1.read_text()
src = INSTALL_PS1.read_text(encoding = "utf-8")
assert "$_studioRootId" in src, "install.ps1 must compute $_studioRootId for the launcher"
assert (
'"share"' in src and "studio_install_id" in src
@ -526,7 +522,7 @@ def test_install_ps1_bakes_studio_root_id_into_launcher():
def test_health_endpoint_exposes_studio_root_id_not_raw_path():
"""/api/health must expose studio_root_id (hex digest), NOT the raw path (info disclosure on -H 0.0.0.0)."""
main_py = REPO_ROOT / "studio" / "backend" / "main.py"
src = main_py.read_text()
src = main_py.read_text(encoding = "utf-8")
health_idx = src.index('@app.get("/api/health")')
# Slice up to the next top-level @app. so a growing body stays in scope.
next_app_idx = src.find("\n@app.", health_idx + 1)
@ -542,7 +538,7 @@ def test_health_endpoint_exposes_studio_root_id_not_raw_path():
def test_install_sh_bakes_studio_root_id_into_launcher():
"""install.sh must persist the id at share/studio_install_id and bake it into the launcher for ALL modes."""
src = INSTALL_SH.read_text()
src = INSTALL_SH.read_text(encoding = "utf-8")
assert (
"_css_studio_root_id" in src
), "install.sh must compute _css_studio_root_id for the launcher"
@ -568,8 +564,10 @@ def test_tauri_preflight_scrubs_studio_home_env():
preflight_root / "preflight.rs",
*(preflight_root / "preflight").glob("*.rs"),
]
preflight = "\n".join(p.read_text() for p in preflight_paths if p.exists())
commands = (REPO_ROOT / "studio" / "src-tauri" / "src" / "commands.rs").read_text()
preflight = "\n".join(p.read_text(encoding = "utf-8") for p in preflight_paths if p.exists())
commands = (REPO_ROOT / "studio" / "src-tauri" / "src" / "commands.rs").read_text(
encoding = "utf-8"
)
# Expect 2 scrubs in preflight (run_cli_probe + probe_cli_capability), 1 in commands.
assert (
preflight.count('cmd.env_remove("UNSLOTH_STUDIO_HOME")') >= 2
@ -587,7 +585,7 @@ def test_tauri_preflight_scrubs_studio_home_env():
def test_install_sh_shim_uses_atomic_replace():
"""install.sh shim install must use ln -sfn for atomic replace (rm+ln left a missing-shim window)."""
src = INSTALL_SH.read_text()
src = INSTALL_SH.read_text(encoding = "utf-8")
shim_idx = src.index('_shim_path="$_LOCAL_BIN/unsloth"')
block = src[shim_idx : shim_idx + 1500]
assert (
@ -600,7 +598,7 @@ def test_install_sh_shim_uses_atomic_replace():
def test_install_sh_create_shortcuts_seeds_id_from_csprng_with_python_fallback(tmp_path):
"""_create_shortcuts seeds ids from /dev/urandom (python3 secrets fallback) and is re-run idempotent."""
src = INSTALL_SH.read_text()
src = INSTALL_SH.read_text(encoding = "utf-8")
fn_start = src.index('_css_data_dir="$DATA_DIR"')
block = src[fn_start : fn_start + 3000]
urandom_idx = block.index("od -An -N32 -tx1 /dev/urandom")
@ -645,7 +643,7 @@ def test_install_sh_create_shortcuts_seeds_id_from_csprng_with_python_fallback(t
def test_install_sh_create_shortcuts_fails_fast_when_no_entropy():
"""With no entropy source, _create_shortcuts must `return 1` not bake an empty studio_root_id."""
src = INSTALL_SH.read_text()
src = INSTALL_SH.read_text(encoding = "utf-8")
fn_start = src.index('_css_data_dir="$DATA_DIR"')
block = src[fn_start : fn_start + 3000]
assert (
@ -661,7 +659,7 @@ def test_install_sh_create_shortcuts_fails_fast_when_no_entropy():
def test_install_sh_bakes_installed_is_env_mode_flag_in_launcher():
"""install.sh must bake the install-time mode into the launcher so a sourced studio.conf can't flip it."""
src = INSTALL_SH.read_text()
src = INSTALL_SH.read_text(encoding = "utf-8")
assert (
"_INSTALLED_IS_ENV_MODE='@@INSTALLED_IS_ENV_MODE@@'" in src
), "launcher heredoc must declare _INSTALLED_IS_ENV_MODE='@@INSTALLED_IS_ENV_MODE@@'"
@ -676,7 +674,7 @@ def test_install_sh_bakes_installed_is_env_mode_flag_in_launcher():
def test_install_sh_launcher_gates_port_file_on_baked_flag_not_runtime_env():
"""Launcher PORT_FILE/LOCK_DIR must gate on baked $_INSTALLED_IS_ENV_MODE, not runtime $UNSLOTH_STUDIO_HOME."""
src = INSTALL_SH.read_text()
src = INSTALL_SH.read_text(encoding = "utf-8")
heredoc_start = src.index("cat > \"$_css_launcher\" << 'LAUNCHER_EOF'")
heredoc_end = src.index("LAUNCHER_EOF\n", heredoc_start)
heredoc = src[heredoc_start:heredoc_end]
@ -724,7 +722,7 @@ def test_install_sh_launcher_gates_port_file_on_baked_flag_not_runtime_env():
def test_main_py_studio_root_id_caches_at_module_load():
"""_studio_root_id() must read the id once at module load and reuse it (no per-poll FS/hash work)."""
main_py = (REPO_ROOT / "studio" / "backend" / "main.py").read_text()
main_py = (REPO_ROOT / "studio" / "backend" / "main.py").read_text(encoding = "utf-8")
assert (
"_STUDIO_ROOT_ID_CACHE: str = _read_studio_install_id()" in main_py
), "main.py must populate _STUDIO_ROOT_ID_CACHE from _read_studio_install_id() at module load"
@ -785,7 +783,7 @@ def test_llama_cpp_search_roots_handles_studio_root_oserror():
holds the handler so the two never disagree on which root is legacy."""
llama_cpp = (
REPO_ROOT / "studio" / "backend" / "core" / "inference" / "llama_cpp.py"
).read_text()
).read_text(encoding = "utf-8")
def _method_body(name: str) -> str:
# Whole method body (def to next sibling def) so the check survives growth.
@ -830,7 +828,7 @@ def test_install_sh_install_id_survives_symlinked_studio_home(tmp_path):
def test_install_sh_substitutes_root_id_before_data_dir():
"""sed must bake the non-user-controlled placeholders before @@DATA_DIR@@ so a crafted $DATA_DIR isn't mutated."""
src = INSTALL_SH.read_text()
src = INSTALL_SH.read_text(encoding = "utf-8")
root_id_idx = src.index("s|@@STUDIO_ROOT_ID@@|$_css_studio_root_id|g")
env_mode_idx = src.index("s|@@INSTALLED_IS_ENV_MODE@@|$_css_is_env_mode|g")
data_dir_idx = src.index("s|@@DATA_DIR@@|$_sed_safe|g")
@ -845,13 +843,15 @@ def test_install_sh_substitutes_root_id_before_data_dir():
def test_install_sh_root_id_pass_does_not_mutate_user_data_dir(tmp_path):
"""A $DATA_DIR containing the literal @@STUDIO_ROOT_ID@@ must survive the placeholder-first sed passes."""
src = INSTALL_SH.read_text()
src = INSTALL_SH.read_text(encoding = "utf-8")
heredoc_start = src.index("cat > \"$_css_launcher\" << 'LAUNCHER_EOF'")
heredoc_body_start = src.index("\n", heredoc_start) + 1
heredoc_body_end = src.index("LAUNCHER_EOF\n", heredoc_start)
template = src[heredoc_body_start:heredoc_body_end]
launcher_path = tmp_path / "launch.sh"
launcher_path.write_text(template)
# template comes out of install.sh, so it carries whatever non-ASCII that
# file holds and cp1252 cannot encode it back out.
launcher_path.write_text(template, encoding = "utf-8")
# sed order: root-id first, then data-dir.
weird_data_dir = "/tmp/with-@@STUDIO_ROOT_ID@@/share"
root_id = "deadbeef" * 8
@ -866,7 +866,8 @@ sed "s|@@DATA_DIR@@|$_sed_safe|g" "{launcher_path}" > "{launcher_path}.tmp" \\
&& mv "{launcher_path}.tmp" "{launcher_path}"
"""
subprocess.run(["bash", "-c", script], check = True)
final = launcher_path.read_text()
# written as utf-8 just above, and the template carries U+2500.
final = launcher_path.read_text(encoding = "utf-8")
assert (
f"DATA_DIR='{weird_data_dir}'" in final
), f"DATA_DIR must be preserved verbatim (no @@STUDIO_ROOT_ID@@ mutation); got: {final[:500]}"
@ -877,7 +878,7 @@ sed "s|@@DATA_DIR@@|$_sed_safe|g" "{launcher_path}" > "{launcher_path}.tmp" \\
def test_install_ps1_install_id_file_layout_matches_backend_read_path():
"""install.ps1 must write the id at share/studio_install_id where the backend reads it, idempotently."""
src = INSTALL_PS1.read_text()
src = INSTALL_PS1.read_text(encoding = "utf-8")
id_idx = src.index('$_studioIdDir = Join-Path $StudioHome "share"')
context = src[id_idx : id_idx + 1500]
assert (

4
tests/test_studio_root_resilience.py
View file

@ -64,7 +64,7 @@ def test_kill_orphan_catches_oserror_from_studio_root():
"""Cleanup must not crash when studio_root() raises. _kill_orphaned_servers
resolves the install root through the shared _resolved_studio_root_and_is_legacy()
classifier, which swallows (ImportError, OSError, ValueError) on the probe."""
src = LLAMA_CPP.read_text()
src = LLAMA_CPP.read_text(encoding = "utf-8")
# Cleanup delegates to the shared classifier rather than importing studio_root inline.
assert "LlamaCppBackend._resolved_studio_root_and_is_legacy()" in _method_body(
src, "_kill_orphaned_servers"
@ -85,7 +85,7 @@ def _exec_search_roots_block(
"""Run _find_llama_server_binary's search_roots derivation -- plus the shared
_resolved_studio_root_and_is_legacy() classifier it delegates to -- with a
controlled studio_root() and resolve(), without importing the heavy module."""
src = LLAMA_CPP.read_text()
src = LLAMA_CPP.read_text(encoding = "utf-8")
# Shared root classifier (holds the defensive try/except for studio_root()).
# End the slice at the next sibling def/decorator at the same indent rather
# than the literal "@staticmethod" string, so a future docstring mentioning a

2
tests/test_tool_mask_zoo_compat.py
View file

@ -15,7 +15,7 @@ RL_REPLACEMENTS_SOURCE_PATH = os.path.join(REPO_ROOT, "unsloth", "models", "rl_r
def _read(path: str) -> str:
with open(path, "r") as fh:
with open(path, "r", encoding = "utf-8") as fh:
return fh.read()

4
tests/utils/test_prepare_inputs_leftpad.py
View file

@ -46,7 +46,7 @@ WIRED_MODEL_FILES = [
def _load_function():
tree = ast.parse(LLAMA_PY.read_text())
tree = ast.parse(LLAMA_PY.read_text(encoding = "utf-8"))
for node in ast.walk(tree):
if isinstance(node, ast.FunctionDef) and node.name == FUNC_NAME:
return node
@ -207,7 +207,7 @@ def test_model_families_stay_wired_to_shared_prepare_inputs():
path = REPO_ROOT / "unsloth" / "models" / fname
if not path.exists():
continue
if "fix_prepare_inputs_for_generation(" not in path.read_text():
if "fix_prepare_inputs_for_generation(" not in path.read_text(encoding = "utf-8"):
missing.append(fname)
assert not missing, (
"these model files no longer call fix_prepare_inputs_for_generation, "

6
tests/utils/test_rope_scaling_drift.py
View file

@ -52,7 +52,7 @@ MAX_POS = 131072
def _load_class_init():
tree = ast.parse(LLAMA_PY.read_text())
tree = ast.parse(LLAMA_PY.read_text(encoding = "utf-8"))
for node in ast.walk(tree):
if isinstance(node, ast.ClassDef) and node.name == CLASS_NAME:
for sub in node.body:
@ -96,7 +96,7 @@ def _iter_names_and_calls(node):
def _find_method(source_path, class_name, method_name):
for node in ast.walk(ast.parse(source_path.read_text())):
for node in ast.walk(ast.parse(source_path.read_text(encoding = "utf-8"))):
if isinstance(node, ast.ClassDef) and node.name == class_name:
for sub in node.body:
if isinstance(sub, ast.FunctionDef) and sub.name == method_name:
@ -105,7 +105,7 @@ def _find_method(source_path, class_name, method_name):
def _find_function(source_path, function_name):
for node in ast.walk(ast.parse(source_path.read_text())):
for node in ast.walk(ast.parse(source_path.read_text(encoding = "utf-8"))):
if isinstance(node, ast.FunctionDef) and node.name == function_name:
return node
return None