ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

tests: read checked-in files as UTF-8 instead of the platform default (#7438)

Browse source 
* tests: read checked-in files as UTF-8 instead of the platform default

Path.read_text() with no encoding uses locale.getpreferredencoding(), which
is UTF-8 on the Linux runners and cp1252 on a stock Windows install. Nine
module-level reads of checked-in source files were relying on that default.

studio/backend/routes/inference.py carries the DeepSeek tool-call token
regexes, so it holds U+FF5C and U+2581. Under cp1252 that read raised
UnicodeDecodeError on byte 0x81 at position 97806, and because the reads run
at import time it took test_cancel_atomicity.py and test_cancel_id_wiring.py
out at collection, not as failures. Green on CI, permanently broken for a
Windows contributor running the suite locally.

Adds a guard: at module scope there is no tmp_path fixture, so a bare
read_text()/write_text()/open() there is always touching a checked-in file.
That makes the rule mechanical enough to enforce with no allowlist, while
staying quiet about temp-dir I/O inside test bodies where the platform
default is harmless.

The repo already spells this correctly in 464 other places; this only stops
the stragglers coming back.

* tests: cover import-time helper reads and keep the guard py3.9-safe

Follows up on the Codex review:

- add `from __future__ import annotations`, since `str | None` in
  `_offender` is evaluated at import on Python 3.9 and pyproject declares
  requires-python ">=3.9,<3.15".
- widen the guard from module scope to import time. Class bodies and the
  bodies of module-level helpers called from an executing statement run
  during collection too, so `CODE = _extract_mixed_precision_code()` was
  the same hazard as an inline read. `if __name__ == "__main__":` blocks
  are skipped: pytest never executes them.
- scan studio/backend/tests/ as well as tests/. Both trees are collected
  on Windows by separate CI jobs, and the offender that started this,
  test_tool_xml_strip.py reading routes/inference.py, lives there.

Widening it surfaced seven more import-time reads of checked-in sources;
all now name utf-8.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Harden the import-time encoding guard for PR #7438

Close the detector gaps raised in review, all of which I reproduced against
the actual AST before changing anything.

False negatives (the guard let a real hazard through):
- _is_main_guard ignored the comparison operator, so if __name__ != "__main__"
  counted as script-only even though its body runs at import.
- The else arm of a main guard was discarded with the rest of the If node.
- Decorators and argument defaults on a module-level def were skipped with the
  body, though both are evaluated when the def executes.
- Path.open() in text mode was invisible; only builtin open() was matched.
- encoding = None and encoding = "locale" both re-select the platform default,
  but the keyword merely being present counted as pinned.

False positives (the guard would have blocked a compliant contributor):
- A non-literal mode fell through to the "r" default, so open(p, mode) was
  flagged even when mode is "rb", where adding encoding= is a ValueError and
  there is no edit that satisfies the rule.
- Same for open(*args) and a **kwargs splat, which hide the mode and can hide
  an encoding.
- Lambda bodies and comprehension elements were walked even though neither runs
  at definition.

Verified: still reports the same 22 offenders on unpatched main, green on this
branch and on the tree merged with latest main (557 files), and an adversarial
corpus of 33 cases now scores zero false positives and zero false negatives.
Also corrected two docstring claims: neither collecting job runs on Windows,
and the read is governed by locale.getencoding().

* Walk eager comprehensions and treat io.open as the builtin

Two regressions from the previous commit, both reproduced against the AST
before changing anything.

Lumping list, set and dict comprehensions in with generator expressions was
wrong. Only a genexp is lazy; the other three run their element expression,
their filters and their nested iterators immediately, so
CONTENTS = [p.read_text() for p in PATHS] at module scope is an import-time
read the guard was silently missing. Comprehensions are now walked in full and
only the genexp keeps the outermost-iterable-only treatment.

io was also in the not-a-path-opener list, but io.open is the builtin, with the
same mode position and the same platform default. io.open(CHECKED_IN_FILE) is
exactly the hazard this guard exists for, so it is matched now, with binary
modes and a pinned encoding still exempt. tarfile.open and fitz.open stay
exempt since neither has an encoding to name.

Verified: 13 targeted cases covering all five eager comprehension forms and
io.open in text, binary and pinned shapes all classify correctly; still 22
offenders on unpatched main; green on this branch and on the tree merged with
latest main.

* Close three more walker gaps in the import-time guard

All three reproduced against the AST first.

A generator expression handed straight to a call is consumed there, so
DATA = "".join(p.read_text() for p in paths) runs its element at import. Only
an unconsumed genexp bound to a name stays lazy, so the walker now follows the
consumed ones in full and keeps the outermost-iterable-only treatment for the
rest.

if "__main__" == __name__ is an equivalent and accepted spelling of the main
guard, but requiring __name__ on the left meant its body was treated as
import-time code. That is a false positive on a block pytest never runs, so
both operand orders are recognised now.

The helper table was built from module-level defs only, so a def in a class
body invoked while the class is constructed was never followed, contradicting
the walker's stated coverage of class bodies. Helpers are now collected from
the module body and from class bodies at any nesting.

Verified: 15 targeted cases including all three fixes and the earlier ones
still classify correctly; still 22 offenders on unpatched main; green on this
branch and on the tree merged with latest main.

* Handle positional read_text encodings, lazy generators and nested helpers

* Guard reads reached from test bodies, unbound Path calls and __file__ paths

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Follow derived paths, skip lazy generator helpers, cover compressed openers

* Guard the CLI tests, helper parameters and unbound Path arguments

* Discover test roots and follow literal, in-place and tuple-derived paths

* Identify module openers by import, unwrap starred paths, pin subprocess snippets

* Resolve import origins, seed helper locals, follow named generators and parametrize

* Scope imports lexically, list tracked test files, bind unpacked names

* Resolve aliased openers, keyword-only params, destructured targets, next()

* Pin the encoding on subprocess snippets, workflow lint and CLI output for PR #7438

* Harden the CLI encoding guard against detached streams for PR #7438

* Tighten the encoding guard's path and scope analysis for PR #7438

* Resolve path provenance more precisely and keep POSIX stream encodings for PR #7438

* Resolve qualified path classes and scope conditional imports for PR #7438

* Scope CLI stream setup to the entry point and align two encoding pairs for PR #7438

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
This commit is contained in:
Leo Borcherding 2026-07-27 01:31:56 -05:00 • committed by GitHub
commit 1dd2fc4583
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
84 changed files with 1652 additions and 352 deletions
Download patch file Download diff file Expand all files Collapse all files

6
scripts/lint_workflow_triggers.py
View file

@ -52,14 +52,14 @@ def _normalise_on(on_field):
def _load_workflow(path: Path): def _load_workflow(path: Path):
try: try:
return yaml.safe_load(path.read_text()) return yaml.safe_load(path.read_text(encoding = "utf-8"))
except Exception as exc: except Exception as exc:
print(f"ERROR: failed to parse {path}: {exc}", file = sys.stderr) print(f"ERROR: failed to parse {path}: {exc}", file = sys.stderr)
sys.exit(2) sys.exit(2)
def _extract_cache_keys(path: Path) -> list[str]: def _extract_cache_keys(path: Path) -> list[str]:
text = path.read_text() text = path.read_text(encoding = "utf-8")
keys: list[str] = [] keys: list[str] = []
for m in re.finditer(r"(?:^|\n)\s*key:\s*([^\n]+)", text): for m in re.finditer(r"(?:^|\n)\s*key:\s*([^\n]+)", text):
keys.append(m.group(1).strip()) keys.append(m.group(1).strip())
@ -104,7 +104,7 @@ def main() -> int:
for t in RESTRICTED_TRIGGERS: for t in RESTRICTED_TRIGGERS:
if t in triggers: if t in triggers:
text = path.read_text() text = path.read_text(encoding = "utf-8")
if "lint:workflow_triggers-allow-workflow_run" not in text: if "lint:workflow_triggers-allow-workflow_run" not in text:
findings.append( findings.append(
f"{path.name}: RESTRICTED trigger '{t}' requires an " f"{path.name}: RESTRICTED trigger '{t}' requires an "

10
studio/backend/tests/test_cloudflare_tunnel.py
View file

@ -915,17 +915,17 @@ def _argparse_default(source, option):
def test_run_server_cloudflare_default_off(): def test_run_server_cloudflare_default_off():
defaults = _func_param_defaults(_RUN_PY.read_text(), "run_server") defaults = _func_param_defaults(_RUN_PY.read_text(encoding = "utf-8"), "run_server")
assert "cloudflare" in defaults assert "cloudflare" in defaults
assert defaults["cloudflare"] is None assert defaults["cloudflare"] is None
def test_argparse_cloudflare_default_off(): def test_argparse_cloudflare_default_off():
assert _argparse_default(_RUN_PY.read_text(), "--cloudflare") is None assert _argparse_default(_RUN_PY.read_text(encoding = "utf-8"), "--cloudflare") is None
def test_verify_global_reachability_marks_private_address_unreachable(): def test_verify_global_reachability_marks_private_address_unreachable():
src = _RUN_PY.read_text() src = _RUN_PY.read_text(encoding = "utf-8")
tree = ast.parse(src) tree = ast.parse(src)
func_src = next( func_src = next(
ast.get_source_segment(src, n) ast.get_source_segment(src, n)
@ -949,7 +949,7 @@ def test_verify_global_reachability_marks_private_address_unreachable():
def test_run_server_registers_tunnel_atexit_backstop(): def test_run_server_registers_tunnel_atexit_backstop():
# An abnormal exit (exception after startup -> sys.exit) bypasses # An abnormal exit (exception after startup -> sys.exit) bypasses
# _graceful_shutdown; an atexit backstop must still stop the tunnel. # _graceful_shutdown; an atexit backstop must still stop the tunnel.
src = _RUN_PY.read_text() src = _RUN_PY.read_text(encoding = "utf-8")
assert "atexit.register(stop_studio_tunnel)" in src assert "atexit.register(stop_studio_tunnel)" in src
@ -965,7 +965,7 @@ def _run_print_cloudflare_line(
color = False, color = False,
): ):
"""Exec _print_cloudflare_line without importing run.py's heavy deps.""" """Exec _print_cloudflare_line without importing run.py's heavy deps."""
src = _RUN_PY.read_text() src = _RUN_PY.read_text(encoding = "utf-8")
tree = ast.parse(src) tree = ast.parse(src)
func_src = next( func_src = next(
ast.get_source_segment(src, n) ast.get_source_segment(src, n)

20
studio/backend/tests/test_consent_gate.py
View file

@ -402,7 +402,7 @@ class TestWorkersWireTheGate:
], ],
) )
def test_worker_invokes_gate(self, rel): def test_worker_invokes_gate(self, rel):
src = (Path(__file__).resolve().parent.parent / rel).read_text() src = (Path(__file__).resolve().parent.parent / rel).read_text(encoding = "utf-8")
assert "evaluate_remote_code_consent" in src assert "evaluate_remote_code_consent" in src
assert "remote_code_blocked" in src assert "remote_code_blocked" in src
assert ".blocked" in src assert ".blocked" in src
@ -410,14 +410,14 @@ class TestWorkersWireTheGate:
def test_mlx_training_path_gates_before_load(self): def test_mlx_training_path_gates_before_load(self):
# The Apple-Silicon path returns before run_training_process's gate, so it must # The Apple-Silicon path returns before run_training_process's gate, so it must
# scan before FastMLXModel.from_pretrained runs repo code. # scan before FastMLXModel.from_pretrained runs repo code.
src = (_BACKEND / "core/training/worker.py").read_text() src = (_BACKEND / "core/training/worker.py").read_text(encoding = "utf-8")
head = src[: src.index("FastMLXModel.from_pretrained(")] head = src[: src.index("FastMLXModel.from_pretrained(")]
assert "evaluate_remote_code_consent" in head assert "evaluate_remote_code_consent" in head
def test_lora_base_model_is_gated(self): def test_lora_base_model_is_gated(self):
# Inference + export expand the consent scan to the LoRA base model's code. # Inference + export expand the consent scan to the LoRA base model's code.
for rel in ("core/inference/worker.py", "core/export/worker.py"): for rel in ("core/inference/worker.py", "core/export/worker.py"):
src = (_BACKEND / rel).read_text() src = (_BACKEND / rel).read_text(encoding = "utf-8")
assert "evaluate_remote_code_consent" in src assert "evaluate_remote_code_consent" in src
assert "get_base_model_from_lora" in src or "mc.base_model" in src assert "get_base_model_from_lora" in src or "mc.base_model" in src
@ -431,12 +431,12 @@ class TestWorkersWireTheGate:
"core/training/worker.py", "core/training/worker.py",
"core/export/worker.py", "core/export/worker.py",
): ):
src = (_BACKEND / rel).read_text() src = (_BACKEND / rel).read_text(encoding = "utf-8")
assert "get_base_model_from_lora_identifier" in src, rel assert "get_base_model_from_lora_identifier" in src, rel
def test_embedding_training_path_gates_before_load(self): def test_embedding_training_path_gates_before_load(self):
# The embedding pipeline must run the malware + consent gates before loading, like the other paths. # The embedding pipeline must run the malware + consent gates before loading, like the other paths.
src = (_BACKEND / "core/training/worker.py").read_text() src = (_BACKEND / "core/training/worker.py").read_text(encoding = "utf-8")
start = src.index("def _run_embedding_training(") start = src.index("def _run_embedding_training(")
end = src.index("FastSentenceTransformer.from_pretrained(", start) end = src.index("FastSentenceTransformer.from_pretrained(", start)
region = src[start:end] region = src[start:end]
@ -505,7 +505,9 @@ class TestStructuredFindingsForDialog:
assert d.findings and d.fingerprint # structured findings for the UI assert d.findings and d.fingerprint # structured findings for the UI
def test_scan_route_uses_preflight(self): def test_scan_route_uses_preflight(self):
src = (Path(__file__).resolve().parent.parent / "routes/models.py").read_text() src = (Path(__file__).resolve().parent.parent / "routes/models.py").read_text(
encoding = "utf-8"
)
assert "remote-code-scan" in src assert "remote-code-scan" in src
# The scan route pins one combined fingerprint over adapter + base, so adapter code is reviewed and approvable too. # The scan route pins one combined fingerprint over adapter + base, so adapter code is reviewed and approvable too.
assert "preflight_remote_code_consent_for_targets" in src assert "preflight_remote_code_consent_for_targets" in src
@ -636,7 +638,7 @@ class TestStructuredFindingsForDialog:
], ],
) )
def test_fingerprint_threaded_to_worker(self, rel): def test_fingerprint_threaded_to_worker(self, rel):
src = (Path(__file__).resolve().parent.parent / rel).read_text() src = (Path(__file__).resolve().parent.parent / rel).read_text(encoding = "utf-8")
assert "approved_remote_code_fingerprint" in src assert "approved_remote_code_fingerprint" in src
# The per-user approval cache rides the same path as the fingerprint. # The per-user approval cache rides the same path as the fingerprint.
assert "subject" in src assert "subject" in src
@ -738,7 +740,7 @@ class TestNemotronGateUsesTrustCheck:
], ],
) )
def test_worker_nemotron_block_calls_trust_check(self, rel): def test_worker_nemotron_block_calls_trust_check(self, rel):
src = (_BACKEND / rel).read_text() src = (_BACKEND / rel).read_text(encoding = "utf-8")
assert "_NEMOTRON_TRUST_SUBSTRINGS" in src assert "_NEMOTRON_TRUST_SUBSTRINGS" in src
assert "is_trusted_org_repo(" in src assert "is_trusted_org_repo(" in src
@ -1525,6 +1527,6 @@ class TestDiscardRemoteCodeDownload:
assert res == {"deleted": False, "reason": "not_cached"} assert res == {"deleted": False, "reason": "not_cached"}
def test_route_source_reports_created_by_scan(self): def test_route_source_reports_created_by_scan(self):
src = (_BACKEND / "routes/models.py").read_text() src = (_BACKEND / "routes/models.py").read_text(encoding = "utf-8")
assert "created_by_scan" in src assert "created_by_scan" in src
assert "discard-remote-code" in src assert "discard-remote-code" in src

2
studio/backend/tests/test_cpu_threads.py
View file

@ -120,7 +120,7 @@ def _ast_line_of_platform_compat_import(source: str) -> int:
# run.py and main.py. Robust to formatting / line shifts. # run.py and main.py. Robust to formatting / line shifts.
@pytest.mark.parametrize("entry_point", [_RUN_PY, _MAIN_PY]) @pytest.mark.parametrize("entry_point", [_RUN_PY, _MAIN_PY])
def test_cpu_thread_configuration_runs_before_backend_imports(entry_point): def test_cpu_thread_configuration_runs_before_backend_imports(entry_point):
source = entry_point.read_text() source = entry_point.read_text(encoding = "utf-8")
call_line = _ast_line_of_configure_call(source) call_line = _ast_line_of_configure_call(source)
compat_line = _ast_line_of_platform_compat_import(source) compat_line = _ast_line_of_platform_compat_import(source)
assert call_line < compat_line, ( assert call_line < compat_line, (

2
studio/backend/tests/test_data_recipe_seed.py
View file

@ -11,7 +11,7 @@ import pytest
def _seed_route_source() -> str: def _seed_route_source() -> str:
return ( return (
Path(__file__).resolve().parent.parent / "routes" / "data_recipe" / "seed.py" Path(__file__).resolve().parent.parent / "routes" / "data_recipe" / "seed.py"
).read_text() ).read_text(encoding = "utf-8")
def test_seed_inspect_load_kwargs_disables_remote_code_execution(): def test_seed_inspect_load_kwargs_disables_remote_code_execution():

8
studio/backend/tests/test_desktop_auth.py
View file

@ -123,7 +123,7 @@ def test_ensure_default_admin_does_not_recreate_bootstrap_for_existing_admin():
def test_ensure_default_admin_loads_existing_bootstrap_after_restart(monkeypatch): def test_ensure_default_admin_loads_existing_bootstrap_after_restart(monkeypatch):
created = storage.ensure_default_admin() created = storage.ensure_default_admin()
bootstrap_pw = storage._BOOTSTRAP_PW_PATH.read_text().strip() bootstrap_pw = storage._BOOTSTRAP_PW_PATH.read_text(encoding = "utf-8").strip()
monkeypatch.setattr(storage, "_bootstrap_password", None) monkeypatch.setattr(storage, "_bootstrap_password", None)
created_again = storage.ensure_default_admin() created_again = storage.ensure_default_admin()
@ -136,12 +136,12 @@ def test_ensure_default_admin_loads_existing_bootstrap_after_restart(monkeypatch
def test_ensure_default_admin_does_not_generate_for_empty_existing_bootstrap(): def test_ensure_default_admin_does_not_generate_for_empty_existing_bootstrap():
seed_user() seed_user()
storage._BOOTSTRAP_PW_PATH.write_text(" \n") storage._BOOTSTRAP_PW_PATH.write_text(" \n", encoding = "utf-8")
created = storage.ensure_default_admin() created = storage.ensure_default_admin()
assert created is False assert created is False
assert storage._BOOTSTRAP_PW_PATH.read_text() == " \n" assert storage._BOOTSTRAP_PW_PATH.read_text(encoding = "utf-8") == " \n"
assert storage.get_bootstrap_password() is None assert storage.get_bootstrap_password() is None
@ -649,7 +649,7 @@ def test_desktop_auth_provision_has_bounded_timeout():
rs_path = ( rs_path = (
Path(__file__).resolve().parents[3] / "studio" / "src-tauri" / "src" / "desktop_auth.rs" Path(__file__).resolve().parents[3] / "studio" / "src-tauri" / "src" / "desktop_auth.rs"
) )
src = rs_path.read_text() src = rs_path.read_text(encoding = "utf-8")
start = src.index("async fn provision_desktop_auth(") start = src.index("async fn provision_desktop_auth(")
depth = 0 depth = 0
body_start = src.index("{", start) body_start = src.index("{", start)

6
studio/backend/tests/test_gguf_load_cache_reuse.py
View file

@ -809,7 +809,9 @@ class TestLoadHubDownloadExclusion:
asyncio.run(scenario()) asyncio.run(scenario())
def test_load_marker_precedes_hub_guard_and_unload(self): def test_load_marker_precedes_hub_guard_and_unload(self):
source = (Path(__file__).resolve().parent.parent / "routes" / "inference.py").read_text() source = (Path(__file__).resolve().parent.parent / "routes" / "inference.py").read_text(
encoding = "utf-8"
)
# _load_model_impl has more than one `if config.is_gguf:`, so anchor on # _load_model_impl has more than one `if config.is_gguf:`, so anchor on
# the branch that actually owns the load marker rather than the first # the branch that actually owns the load marker rather than the first
# one in the file, which belongs to an earlier check. # one in the file, which belongs to an earlier check.
@ -832,7 +834,7 @@ class TestLoadHubDownloadExclusion:
) )
llama_source = ( llama_source = (
Path(__file__).resolve().parent.parent / "core" / "inference" / "llama_cpp.py" Path(__file__).resolve().parent.parent / "core" / "inference" / "llama_cpp.py"
).read_text() ).read_text(encoding = "utf-8")
assert "@_with_gguf_load_marker\n def load_model(" in llama_source assert "@_with_gguf_load_marker\n def load_model(" in llama_source
def _capture_hub_guard_require_mmproj( def _capture_hub_guard_require_mmproj(

4
studio/backend/tests/test_host_defaults.py
View file

@ -64,7 +64,7 @@ def test_run_server_default_host_is_loopback():
0.0.0.0 exposes the service on all interfaces; loopback is the 0.0.0.0 exposes the service on all interfaces; loopback is the
least-permissive default. Users needing network access pass -H 0.0.0.0. least-permissive default. Users needing network access pass -H 0.0.0.0.
""" """
source = _RUN_PY.read_text() source = _RUN_PY.read_text(encoding = "utf-8")
defaults = _parse_function_param_defaults(source, "run_server") defaults = _parse_function_param_defaults(source, "run_server")
assert "host" in defaults, "run_server() must have a 'host' parameter with a default" assert "host" in defaults, "run_server() must have a 'host' parameter with a default"
host_default = defaults["host"] host_default = defaults["host"]
@ -81,7 +81,7 @@ def test_argparse_default_host_is_loopback():
When run.py is invoked directly (python run.py), the argparse default When run.py is invoked directly (python run.py), the argparse default
must match the function default so direct execution is equally safe. must match the function default so direct execution is equally safe.
""" """
source = _RUN_PY.read_text() source = _RUN_PY.read_text(encoding = "utf-8")
host_default = _parse_argparse_add_argument_default(source, "--host") host_default = _parse_argparse_add_argument_default(source, "--host")
assert host_default is not None, "Could not find add_argument('--host', ...) in run.py" assert host_default is not None, "Could not find add_argument('--host', ...) in run.py"
assert ( assert (

4
studio/backend/tests/test_mcp_servers.py
View file

@ -599,7 +599,9 @@ def test_tool_xml_strip_handles_hyphenated_function_names():
from core.inference.tool_call_parser import _DEEPSEEK_OPEN_RE_SRC as _DS_OPEN_SRC from core.inference.tool_call_parser import _DEEPSEEK_OPEN_RE_SRC as _DS_OPEN_SRC
src = (Path(__file__).resolve().parent.parent / "routes/inference.py").read_text() src = (Path(__file__).resolve().parent.parent / "routes/inference.py").read_text(
encoding = "utf-8"
)
m = _re.search(r"_TOOL_XML_RE = _re\.compile\((.*?)\n\)", src, _re.DOTALL) m = _re.search(r"_TOOL_XML_RE = _re\.compile\((.*?)\n\)", src, _re.DOTALL)
assert m, "could not extract _TOOL_XML_RE" assert m, "could not extract _TOOL_XML_RE"
ns: dict = {"_re": _re, "_DS_OPEN_SRC": _DS_OPEN_SRC} ns: dict = {"_re": _re, "_DS_OPEN_SRC": _DS_OPEN_SRC}

8
studio/backend/tests/test_mlx_training_worker_config.py
View file

@ -86,7 +86,9 @@ def test_mlx_studio_rejects_unknown_scheduler():
def test_mlx_studio_keeps_hf_style_tokenizer_dual_purpose(): def test_mlx_studio_keeps_hf_style_tokenizer_dual_purpose():
source = (Path(__file__).resolve().parents[1] / "core" / "training" / "worker.py").read_text() source = (Path(__file__).resolve().parents[1] / "core" / "training" / "worker.py").read_text(
encoding = "utf-8"
)
assert "tokenizer = tokenizer" in source assert "tokenizer = tokenizer" in source
assert "processor = tokenizer if is_vlm else None" not in source assert "processor = tokenizer if is_vlm else None" not in source
@ -96,7 +98,9 @@ def test_mlx_wandb_run_config_excludes_subject_and_secrets():
# The MLX W&B run config uploads the whole config minus a sensitive set. The owner's # The MLX W&B run config uploads the whole config minus a sensitive set. The owner's
# subject (authenticated username / API-key id) must be filtered alongside the secrets, # subject (authenticated username / API-key id) must be filtered alongside the secrets,
# otherwise it lands in W&B run config even though DB history already strips it. # otherwise it lands in W&B run config even though DB history already strips it.
source = (Path(__file__).resolve().parents[1] / "core" / "training" / "worker.py").read_text() source = (Path(__file__).resolve().parents[1] / "core" / "training" / "worker.py").read_text(
encoding = "utf-8"
)
assert ( assert (
'_wandb_sensitive = {"hf_token", "wandb_token", "s3_config", "subject"}' in source '_wandb_sensitive = {"hf_token", "wandb_token", "s3_config", "subject"}' in source

36
studio/backend/tests/test_mtp_vram_budget.py
View file

@ -320,7 +320,7 @@ class TestFitContextWithMtp:
def _fit_backend(self, kv_per_token = 325_000): def _fit_backend(self, kv_per_token = 325_000):
b = _make_backend() b = _make_backend()
b._can_estimate_kv = lambda: True b._can_estimate_kv = lambda: True
b._estimate_kv_cache_bytes = lambda n, _t = None, **_k: (0 if n <= 0 else n * kv_per_token) b._estimate_kv_cache_bytes = lambda n, _t = None, **_k: 0 if n <= 0 else n * kv_per_token
return b return b
def test_overhead_fn_lowers_context(self): def test_overhead_fn_lowers_context(self):
@ -347,19 +347,23 @@ class TestFitContextWithMtp:
131072, 131072,
avail_mib, avail_mib,
model, model,
mtp_overhead_fn = lambda c: b._estimate_mtp_overhead_bytes( mtp_overhead_fn = lambda c: (
c, draft_cache_type_k = "f16", draft_cache_type_v = "f16" b._estimate_mtp_overhead_bytes(
) c, draft_cache_type_k = "f16", draft_cache_type_v = "f16"
or 0, )
or 0
),
) )
q4 = b._fit_context_to_vram( q4 = b._fit_context_to_vram(
131072, 131072,
avail_mib, avail_mib,
model, model,
mtp_overhead_fn = lambda c: b._estimate_mtp_overhead_bytes( mtp_overhead_fn = lambda c: (
c, draft_cache_type_k = "q4_0", draft_cache_type_v = "q4_0" b._estimate_mtp_overhead_bytes(
) c, draft_cache_type_k = "q4_0", draft_cache_type_v = "q4_0"
or 0, )
or 0
),
) )
assert 0 < q4 == f16 assert 0 < q4 == f16
@ -818,9 +822,9 @@ class TestExtraArgsMtpDetection:
# helper, or an env-driven tensor server (or its layer downgrade) is # helper, or an env-driven tensor server (or its layer downgrade) is
# needlessly reloaded (#6312). Read from disk (importing routes.inference # needlessly reloaded (#6312). Read from disk (importing routes.inference
# drags in heavy deps). # drags in heavy deps).
routes_src = ( routes_src = (Path(__file__).resolve().parent.parent / "routes" / "inference.py").read_text(
Path(__file__).resolve().parent.parent / "routes" / "inference.py" encoding = "utf-8"
).read_text() )
start = routes_src.index("def _request_matches_loaded_settings") start = routes_src.index("def _request_matches_loaded_settings")
end = routes_src.index("\ndef ", start + 1) end = routes_src.index("\ndef ", start + 1)
body = "".join(routes_src[start:end].split()) body = "".join(routes_src[start:end].split())
@ -832,9 +836,9 @@ class TestExtraArgsMtpDetection:
def test_route_matcher_retries_after_drafter_not_found(self): def test_route_matcher_retries_after_drafter_not_found(self):
# drafter_not_found must not report "already loaded" or the reload never # drafter_not_found must not report "already loaded" or the reload never
# retries the download (#6459). Read source: importing routes pulls deps. # retries the download (#6459). Read source: importing routes pulls deps.
routes_src = ( routes_src = (Path(__file__).resolve().parent.parent / "routes" / "inference.py").read_text(
Path(__file__).resolve().parent.parent / "routes" / "inference.py" encoding = "utf-8"
).read_text() )
start = routes_src.index("def _request_matches_loaded_settings") start = routes_src.index("def _request_matches_loaded_settings")
end = routes_src.index("\ndef ", start + 1) end = routes_src.index("\ndef ", start + 1)
body = "".join(routes_src[start:end].split()) body = "".join(routes_src[start:end].split())
@ -990,7 +994,7 @@ def test_qwen36_class_regression_picks_lower_ctx_with_mtp():
strictly lower one once the MTP draft reserve is accounted for.""" strictly lower one once the MTP draft reserve is accounted for."""
b = _make_backend() b = _make_backend()
b._can_estimate_kv = lambda: True b._can_estimate_kv = lambda: True
b._estimate_kv_cache_bytes = lambda n, _t = None, **_k: (0 if n <= 0 else int(n * 66_000)) b._estimate_kv_cache_bytes = lambda n, _t = None, **_k: 0 if n <= 0 else int(n * 66_000)
avail_mib = 24_000 avail_mib = 24_000
model = int(17.9 * GIB) # UD-Q4_K_XL weights model = int(17.9 * GIB) # UD-Q4_K_XL weights
no_mtp = b._fit_context_to_vram(262144, avail_mib, model) no_mtp = b._fit_context_to_vram(262144, avail_mib, model)

2
studio/backend/tests/test_native_context_length.py
View file

@ -374,7 +374,7 @@ class TestRouteCompleteness:
def _load_source(self): def _load_source(self):
"""Read routes/inference.py source once.""" """Read routes/inference.py source once."""
routes_path = Path(__file__).resolve().parent.parent / "routes" / "inference.py" routes_path = Path(__file__).resolve().parent.parent / "routes" / "inference.py"
self._source = routes_path.read_text() self._source = routes_path.read_text(encoding = "utf-8")
def _find_construction_blocks(self, class_name: str) -> list[str]: def _find_construction_blocks(self, class_name: str) -> list[str]:
"""Extract all code blocks that construct a given response class.""" """Extract all code blocks that construct a given response class."""

6
studio/backend/tests/test_native_template_trust_remote_code.py
View file

@ -170,7 +170,9 @@ def test_backend_model_info_persists_trust_remote_code():
"""Both backends must store ``trust_remote_code`` on their per-model info dict so """Both backends must store ``trust_remote_code`` on their per-model info dict so
``render_native_template`` can source the consent value. Guards against the read ``render_native_template`` can source the consent value. Guards against the read
landing on a key ``load_model`` never sets (which would silently no-op the fix).""" landing on a key ``load_model`` never sets (which would silently no-op the fix)."""
inf = (Path(_BACKEND_DIR) / "core" / "inference" / "inference.py").read_text() inf = (Path(_BACKEND_DIR) / "core" / "inference" / "inference.py").read_text(encoding = "utf-8")
mlx = (Path(_BACKEND_DIR) / "core" / "inference" / "mlx_inference.py").read_text() mlx = (Path(_BACKEND_DIR) / "core" / "inference" / "mlx_inference.py").read_text(
encoding = "utf-8"
)
assert '"trust_remote_code": trust_remote_code,' in inf assert '"trust_remote_code": trust_remote_code,' in inf
assert '"trust_remote_code": trust_remote_code,' in mlx assert '"trust_remote_code": trust_remote_code,' in mlx

2
studio/backend/tests/test_offline_inference_parent.py
View file

@ -205,7 +205,7 @@ class TestTrainingWorkerProbeNoGlobalTimeout:
import re import re
from pathlib import Path from pathlib import Path
src = Path(_BACKEND_DIR, "core", "training", "worker.py").read_text() src = Path(_BACKEND_DIR, "core", "training", "worker.py").read_text(encoding = "utf-8")
m = re.search( m = re.search(
r'if\s+"HF_HUB_OFFLINE"\s+not\s+in\s+os\.environ\s*:.*?' r'if\s+"HF_HUB_OFFLINE"\s+not\s+in\s+os\.environ\s*:.*?'
r"print\([^)]*HF_HUB_OFFLINE=1[^)]*\)", r"print\([^)]*HF_HUB_OFFLINE=1[^)]*\)",

2
studio/backend/tests/test_recommended_folders_has_model.py
View file

@ -32,7 +32,7 @@ def _load_has_downloaded_model():
"""Return the real ``_dir_has_downloaded_model`` (plus its ``_safe_is_dir`` """Return the real ``_dir_has_downloaded_model`` (plus its ``_safe_is_dir``
and ``_is_weight_bin`` deps, and the ``_WEIGHT_BIN_PREFIXES`` constant the and ``_is_weight_bin`` deps, and the ``_WEIGHT_BIN_PREFIXES`` constant the
latter reads) without importing the heavy module.""" latter reads) without importing the heavy module."""
tree = ast.parse(_models_src.read_text()) tree = ast.parse(_models_src.read_text(encoding = "utf-8"))
wanted = {"_safe_is_dir", "_dir_has_downloaded_model", "_is_weight_bin"} wanted = {"_safe_is_dir", "_dir_has_downloaded_model", "_is_weight_bin"}
body = [] body = []
for node in tree.body: for node in tree.body:

2
studio/backend/tests/test_recommended_folders_permission.py
View file

@ -36,7 +36,7 @@ _models_src = _backend_root / "routes" / "models.py"
def _load_safe_is_dir(): def _load_safe_is_dir():
"""Return the real ``_safe_is_dir`` from routes/models.py without """Return the real ``_safe_is_dir`` from routes/models.py without
importing the dependency-laden module.""" importing the dependency-laden module."""
tree = ast.parse(_models_src.read_text()) tree = ast.parse(_models_src.read_text(encoding = "utf-8"))
fn = next( fn = next(
node node
for node in tree.body for node in tree.body

8
studio/backend/tests/test_sandbox_tools.py
View file

@ -558,24 +558,24 @@ class TestSandboxCpuRlimitDefault:
"""Pin the default so a regression below 600s without opt-in is caught.""" """Pin the default so a regression below 600s without opt-in is caught."""
def test_default_cpu_s_is_600(self): def test_default_cpu_s_is_600(self):
src = (_BACKEND_ROOT / "core" / "inference" / "tools.py").read_text() src = (_BACKEND_ROOT / "core" / "inference" / "tools.py").read_text(encoding = "utf-8")
assert 'UNSLOTH_STUDIO_SANDBOX_CPU_S", "600"' in src assert 'UNSLOTH_STUDIO_SANDBOX_CPU_S", "600"' in src
def test_clone_newnet_removed(self): def test_clone_newnet_removed(self):
src = (_BACKEND_ROOT / "core" / "inference" / "tools.py").read_text() src = (_BACKEND_ROOT / "core" / "inference" / "tools.py").read_text(encoding = "utf-8")
assert "_libc.unshare(0x40000000)" not in src assert "_libc.unshare(0x40000000)" not in src
# Explanatory comment retained. # Explanatory comment retained.
assert "CLONE_NEWNET" in src assert "CLONE_NEWNET" in src
def test_nofile_env_tunable(self): def test_nofile_env_tunable(self):
src = (_BACKEND_ROOT / "core" / "inference" / "tools.py").read_text() src = (_BACKEND_ROOT / "core" / "inference" / "tools.py").read_text(encoding = "utf-8")
# Parity with the other rlimits: must come from the env, not be hardcoded. # Parity with the other rlimits: must come from the env, not be hardcoded.
assert "UNSLOTH_STUDIO_SANDBOX_NOFILE" in src assert "UNSLOTH_STUDIO_SANDBOX_NOFILE" in src
class TestMaxBodyDefault: class TestMaxBodyDefault:
def test_default_is_500_mb(self): def test_default_is_500_mb(self):
src = (_BACKEND_ROOT / "utils" / "upload_limits.py").read_text() src = (_BACKEND_ROOT / "utils" / "upload_limits.py").read_text(encoding = "utf-8")
assert "DEFAULT_UPLOAD_LIMIT_MB = 500" in src assert "DEFAULT_UPLOAD_LIMIT_MB = 500" in src
assert "UNSLOTH_STUDIO_MAX_BODY_MB" in src assert "UNSLOTH_STUDIO_MAX_BODY_MB" in src

10
studio/backend/tests/test_security_gate_consistency.py
View file

@ -43,7 +43,7 @@ def test_capability_probes_thread_the_hf_token():
offenders = [] offenders = []
for path in _iter_caller_files(): for path in _iter_caller_files():
try: try:
tree = ast.parse(path.read_text()) tree = ast.parse(path.read_text(encoding = "utf-8"))
except SyntaxError: except SyntaxError:
continue continue
for node in ast.walk(tree): for node in ast.walk(tree):
@ -60,7 +60,7 @@ def test_capability_probes_thread_the_hf_token():
def test_gguf_trust_remote_code_reported_inert_not_from_yaml(): def test_gguf_trust_remote_code_reported_inert_not_from_yaml():
"""GGUF never executes auto_map, so requires_trust_remote_code is reported via the """GGUF never executes auto_map, so requires_trust_remote_code is reported via the
resolver or False, never the raw YAML bool() (the round-6 regression).""" resolver or False, never the raw YAML bool() (the round-6 regression)."""
src = (_BACKEND / "routes" / "inference.py").read_text() src = (_BACKEND / "routes" / "inference.py").read_text(encoding = "utf-8")
assert "requires_trust_remote_code = bool(" not in src, ( assert "requires_trust_remote_code = bool(" not in src, (
"Report requires_trust_remote_code via _resolve_loaded_trust_remote_code " "Report requires_trust_remote_code via _resolve_loaded_trust_remote_code "
"(non-GGUF) or set it False (GGUF); never bool(inference_config.get(...))." "(non-GGUF) or set it False (GGUF); never bool(inference_config.get(...))."
@ -70,7 +70,7 @@ def test_gguf_trust_remote_code_reported_inert_not_from_yaml():
def test_capability_detection_caches_are_token_aware(): def test_capability_detection_caches_are_token_aware():
"""Every capability cache is keyed by (model, token_fingerprint) so an unauthenticated """Every capability cache is keyed by (model, token_fingerprint) so an unauthenticated
miss cannot poison a later authenticated lookup (the audio-cache regression).""" miss cannot poison a later authenticated lookup (the audio-cache regression)."""
src = (_BACKEND / "utils" / "models" / "model_config.py").read_text() src = (_BACKEND / "utils" / "models" / "model_config.py").read_text(encoding = "utf-8")
offenders = [] offenders = []
for line in src.splitlines(): for line in src.splitlines():
stripped = line.strip() stripped = line.strip()
@ -93,7 +93,7 @@ def test_malware_and_consent_gates_cover_the_lora_base():
] ]
offenders = [] offenders = []
for rel in gated_workers: for rel in gated_workers:
src = (_BACKEND / rel).read_text() src = (_BACKEND / rel).read_text(encoding = "utf-8")
runs_gate = "evaluate_file_security(" in src or "evaluate_remote_code_consent" in src runs_gate = "evaluate_file_security(" in src or "evaluate_remote_code_consent" in src
resolves_base = "get_base_model_from_lora_identifier(" in src or "base_model" in src resolves_base = "get_base_model_from_lora_identifier(" in src or "base_model" in src
if runs_gate and not resolves_base: if runs_gate and not resolves_base:
@ -107,7 +107,7 @@ def test_rag_embedding_path_runs_the_malware_gate():
or a flagged repo loads unscanned (bypassing the normal model-load protections).""" or a flagged repo loads unscanned (bypassing the normal model-load protections)."""
offenders = [] offenders = []
for rel in ("routes/settings.py", "core/rag/embeddings.py"): for rel in ("routes/settings.py", "core/rag/embeddings.py"):
if "evaluate_file_security(" not in (_BACKEND / rel).read_text(): if "evaluate_file_security(" not in (_BACKEND / rel).read_text(encoding = "utf-8"):
offenders.append( offenders.append(
f"{rel} loads/persists an embedding model without evaluate_file_security" f"{rel} loads/persists an embedding model without evaluate_file_security"
) )

14
studio/backend/tests/test_ssm_runtime.py
View file

@ -401,13 +401,13 @@ def test_hip_uv_source_build_uses_no_cache(monkeypatch):
def test_inference_worker_calls_ensure_ssm_runtime(): def test_inference_worker_calls_ensure_ssm_runtime():
src = (_BACKEND / "core" / "inference" / "worker.py").read_text() src = (_BACKEND / "core" / "inference" / "worker.py").read_text(encoding = "utf-8")
assert "from utils.ssm_runtime import ensure_ssm_runtime" in src assert "from utils.ssm_runtime import ensure_ssm_runtime" in src
assert "ensure_ssm_runtime(" in src assert "ensure_ssm_runtime(" in src
def test_inference_worker_skips_ssm_on_mlx_and_checks_lora_base(): def test_inference_worker_skips_ssm_on_mlx_and_checks_lora_base():
src = (_BACKEND / "core" / "inference" / "worker.py").read_text() src = (_BACKEND / "core" / "inference" / "worker.py").read_text(encoding = "utf-8")
# MLX (Apple Silicon) must not try to build CUDA/ROCm SSM kernels. # MLX (Apple Silicon) must not try to build CUDA/ROCm SSM kernels.
assert 'getattr(backend, "device", None) != "mlx"' in src assert 'getattr(backend, "device", None) != "mlx"' in src
# A LoRA load must also check its base model, not just the adapter id. # A LoRA load must also check its base model, not just the adapter id.
@ -417,12 +417,12 @@ def test_inference_worker_skips_ssm_on_mlx_and_checks_lora_base():
def test_inference_worker_resolves_remote_lora_base_pre_import(): def test_inference_worker_resolves_remote_lora_base_pre_import():
# A remote LoRA's base (from the Hub adapter_config.json) must be resolved before the # A remote LoRA's base (from the Hub adapter_config.json) must be resolved before the
# transformers import so its SSM kernels are pre-installed, not too late in _handle_load. # transformers import so its SSM kernels are pre-installed, not too late in _handle_load.
src = (_BACKEND / "core" / "inference" / "worker.py").read_text() src = (_BACKEND / "core" / "inference" / "worker.py").read_text(encoding = "utf-8")
assert "_remote_lora_base" in src assert "_remote_lora_base" in src
def test_inference_worker_tiers_on_base_and_gates_lora_base_only(): def test_inference_worker_tiers_on_base_and_gates_lora_base_only():
src = (_BACKEND / "core" / "inference" / "worker.py").read_text() src = (_BACKEND / "core" / "inference" / "worker.py").read_text(encoding = "utf-8")
# Tier activation runs on the resolved base, not the raw adapter id (remote-LoRA fix). # Tier activation runs on the resolved base, not the raw adapter id (remote-LoRA fix).
assert "_activate_transformers_version(_base" in src assert "_activate_transformers_version(_base" in src
# The gate only adds a genuine LoRA base, never a full fine-tune's recorded (unloaded) base. # The gate only adds a genuine LoRA base, never a full fine-tune's recorded (unloaded) base.
@ -432,7 +432,7 @@ def test_inference_worker_tiers_on_base_and_gates_lora_base_only():
def test_inference_worker_probes_base_for_ssm_kernels(): def test_inference_worker_probes_base_for_ssm_kernels():
# Both the pre-import path and _handle_load must derive SSM targets from a real model id # Both the pre-import path and _handle_load must derive SSM targets from a real model id
# via ssm_probe_identifier, not the raw adapter id / local checkpoint path. # via ssm_probe_identifier, not the raw adapter id / local checkpoint path.
src = (_BACKEND / "core" / "inference" / "worker.py").read_text() src = (_BACKEND / "core" / "inference" / "worker.py").read_text(encoding = "utf-8")
assert src.count("ssm_probe_identifier(") >= 2 assert src.count("ssm_probe_identifier(") >= 2
@ -484,7 +484,7 @@ def test_pre_import_gate_is_transformers_free():
def test_pre_import_gate_skips_subdir_computation(): def test_pre_import_gate_skips_subdir_computation():
# The worker's pre-import preflight must call the gate with compute_subdirs=False so it # The worker's pre-import preflight must call the gate with compute_subdirs=False so it
# never imports model_config/transformers before the SSM kernels are installed. # never imports model_config/transformers before the SSM kernels are installed.
src = (_BACKEND / "core" / "inference" / "worker.py").read_text() src = (_BACKEND / "core" / "inference" / "worker.py").read_text(encoding = "utf-8")
assert "compute_subdirs = False" in src assert "compute_subdirs = False" in src
@ -506,7 +506,7 @@ def test_security_gates_run_before_ssm_install():
# The SSM install is name-based and can source-build native packages, so a malware / # The SSM install is name-based and can source-build native packages, so a malware /
# blocked-code model must be refused first -- in both the pre-import path and _handle_load. # blocked-code model must be refused first -- in both the pre-import path and _handle_load.
import ast import ast
tree = ast.parse((_BACKEND / "core" / "inference" / "worker.py").read_text()) tree = ast.parse((_BACKEND / "core" / "inference" / "worker.py").read_text(encoding = "utf-8"))
for fn in ("run_inference_process", "_handle_load"): for fn in ("run_inference_process", "_handle_load"):
gates = _call_linenos(tree, fn, "_run_security_gates") gates = _call_linenos(tree, fn, "_run_security_gates")
ssm = _call_linenos(tree, fn, "_ensure_ssm_kernels") ssm = _call_linenos(tree, fn, "_ensure_ssm_kernels")

27
studio/backend/tests/test_studio_api.py
View file

@ -403,9 +403,9 @@ def test_openai_tools_stream(base_url: str, api_key: str):
) )
assert status == 200, f"Expected 200, got {status}" assert status == 200, f"Expected 200, got {status}"
assert len(chunks) > 0, "No SSE chunks received" assert len(chunks) > 0, "No SSE chunks received"
assert _final_finish_reason(chunks) == "tool_calls", ( assert (
f"Expected final finish_reason='tool_calls', got " f"{_final_finish_reason(chunks)!r}" _final_finish_reason(chunks) == "tool_calls"
) ), f"Expected final finish_reason='tool_calls', got {_final_finish_reason(chunks)!r}"
assembled = _collect_streamed_tool_calls(chunks) assembled = _collect_streamed_tool_calls(chunks)
assert len(assembled) >= 1, "No tool_calls reassembled from stream" assert len(assembled) >= 1, "No tool_calls reassembled from stream"
first = assembled[0] first = assembled[0]
@ -486,16 +486,16 @@ def test_openai_sdk_tool_calling(base_url: str, api_key: str):
tool_choice = "required", tool_choice = "required",
stream = False, stream = False,
) )
assert resp.choices[0].finish_reason == "tool_calls", ( assert (
f"Expected finish_reason='tool_calls', got " f"{resp.choices[0].finish_reason!r}" resp.choices[0].finish_reason == "tool_calls"
) ), f"Expected finish_reason='tool_calls', got {resp.choices[0].finish_reason!r}"
tool_calls = resp.choices[0].message.tool_calls tool_calls = resp.choices[0].message.tool_calls
assert tool_calls and len(tool_calls) >= 1, "No tool_calls from SDK" assert tool_calls and len(tool_calls) >= 1, "No tool_calls from SDK"
tc = tool_calls[0] tc = tool_calls[0]
assert tc.function.name == "get_weather" assert tc.function.name == "get_weather"
parsed = json.loads(tc.function.arguments) parsed = json.loads(tc.function.arguments)
assert "city" in parsed assert "city" in parsed
print(f" PASS openai SDK tool calling: " f"tool={tc.function.name}, args={parsed}") print(f" PASS openai SDK tool calling: tool={tc.function.name}, args={parsed}")
def test_invalid_key_rejected(base_url: str): def test_invalid_key_rejected(base_url: str):
@ -783,12 +783,17 @@ def _start_server(model: str, variant: str | None) -> tuple[subprocess.Popen, st
cmd.extend(["--gguf-variant", variant]) cmd.extend(["--gguf-variant", variant])
LOG_FILE.parent.mkdir(parents = True, exist_ok = True) LOG_FILE.parent.mkdir(parents = True, exist_ok = True)
log_fh = open(LOG_FILE, "w") log_fh = open(LOG_FILE, "w", encoding = "utf-8")
# The child writes to this descriptor itself, so the parent's encoding does
# not transcode anything: tell the child to emit utf-8 or the reads below
# decode its locale bytes as utf-8 and raise on the first non-ASCII glyph.
child_env = {**os.environ, "PYTHONIOENCODING": "utf-8", "PYTHONUTF8": "1"}
proc = subprocess.Popen( proc = subprocess.Popen(
cmd, cmd,
stdout = log_fh, stdout = log_fh,
stderr = subprocess.STDOUT, stderr = subprocess.STDOUT,
preexec_fn = os.setsid, preexec_fn = os.setsid,
env = child_env,
) )
# Wait for the banner containing the API key # Wait for the banner containing the API key
@ -798,16 +803,16 @@ def _start_server(model: str, variant: str | None) -> tuple[subprocess.Popen, st
time.sleep(2) time.sleep(2)
if proc.poll() is not None: if proc.poll() is not None:
log_fh.flush() log_fh.flush()
log_text = LOG_FILE.read_text() log_text = LOG_FILE.read_text(encoding = "utf-8")
raise RuntimeError(f"Server exited early (code {proc.returncode}):\n{log_text[-2000:]}") raise RuntimeError(f"Server exited early (code {proc.returncode}):\n{log_text[-2000:]}")
log_text = LOG_FILE.read_text() log_text = LOG_FILE.read_text(encoding = "utf-8")
m = re.search(r"API Key:\s+(sk-unsloth-[a-f0-9]+)", log_text) m = re.search(r"API Key:\s+(sk-unsloth-[a-f0-9]+)", log_text)
if m: if m:
api_key = m.group(1) api_key = m.group(1)
break break
if not api_key: if not api_key:
log_text = LOG_FILE.read_text() log_text = LOG_FILE.read_text(encoding = "utf-8")
_kill_server(proc) _kill_server(proc)
raise RuntimeError(f"Timed out waiting for API key in server output:\n{log_text[-2000:]}") raise RuntimeError(f"Timed out waiting for API key in server output:\n{log_text[-2000:]}")

16
studio/backend/tests/test_tool_call_parser_strict.py
View file

@ -64,9 +64,7 @@ class TestFunctionStyleTrailingText:
# The real closing </function> is the last one; the literal inside # The real closing </function> is the last one; the literal inside
# the code argument must survive (rfind, not the first match). # the code argument must survive (rfind, not the first match).
text = ( text = (
"<function=python><parameter=code>" '<function=python><parameter=code>print("</function>")</parameter></function> all done'
'print("</function>")'
"</parameter></function> all done"
) )
call = _only(text) call = _only(text)
assert call == {"name": "python", "arguments": {"code": 'print("</function>")'}} assert call == {"name": "python", "arguments": {"code": 'print("</function>")'}}
@ -146,9 +144,7 @@ class TestParityWithJsonStyle:
class TestGemmaNativeStyle: class TestGemmaNativeStyle:
def test_closed_native_call_with_trailing_prose_is_accepted(self): def test_closed_native_call_with_trailing_prose_is_accepted(self):
text = ( text = '<|tool_call>call:terminal{command:"ls -la",workdir:"."}<tool_call|> running it now'
'<|tool_call>call:terminal{command:"ls -la",workdir:"."}<tool_call|>' " running it now"
)
calls = parse_tool_calls_from_text(text, allow_incomplete = False) calls = parse_tool_calls_from_text(text, allow_incomplete = False)
assert len(calls) == 1 assert len(calls) == 1
assert calls[0]["function"]["name"] == "terminal" assert calls[0]["function"]["name"] == "terminal"
@ -792,7 +788,7 @@ def test_tool_call_parser_declares_future_annotations_for_py39_import():
from pathlib import Path from pathlib import Path
src = ( src = (
Path(__file__).resolve().parent.parent / "core" / "inference" / "tool_call_parser.py" Path(__file__).resolve().parent.parent / "core" / "inference" / "tool_call_parser.py"
).read_text() ).read_text(encoding = "utf-8")
assert "from __future__ import annotations" in src assert "from __future__ import annotations" in src
@ -1069,8 +1065,7 @@ class TestBareJsonOuterOverXmlLiteral:
def test_bare_json_code_arg_quoting_function_xml(self): def test_bare_json_code_arg_quoting_function_xml(self):
text = ( text = (
'{"name": "python", "arguments": ' '{"name": "python", "arguments": {"code": "run() # <function=terminal>ls</function>"}}'
'{"code": "run() # <function=terminal>ls</function>"}}'
) )
calls = parse_tool_calls_from_text(text, enabled_tool_names = {"python"}) calls = parse_tool_calls_from_text(text, enabled_tool_names = {"python"})
assert [c["function"]["name"] for c in calls] == ["python"] assert [c["function"]["name"] for c in calls] == ["python"]
@ -1300,8 +1295,7 @@ class TestLeadingWrapperlessGemmaOverEmbeddedMarkers:
def test_leading_gemma_wins_over_quoted_xml_literal(self): def test_leading_gemma_wins_over_quoted_xml_literal(self):
text = ( text = (
'call:web_search{query:"explain <tool_call>' 'call:web_search{query:"explain <tool_call>{"name":"evil","arguments":{}}</tool_call>"}'
'{"name":"evil","arguments":{}}</tool_call>"}'
) )
calls = parse_tool_calls_from_text(text, enabled_tool_names = {"web_search", "evil"}) calls = parse_tool_calls_from_text(text, enabled_tool_names = {"web_search", "evil"})
assert [c["function"]["name"] for c in calls] == ["web_search"] assert [c["function"]["name"] for c in calls] == ["web_search"]

2
studio/backend/tests/test_tool_xml_strip.py
View file

@ -21,7 +21,7 @@ if _BACKEND_DIR not in sys.path:
# Extract the regex from source (routes module needs heavy stubbing to import). # Extract the regex from source (routes module needs heavy stubbing to import).
import re as _re import re as _re
_src = (Path(_BACKEND_DIR) / "routes" / "inference.py").read_text() _src = (Path(_BACKEND_DIR) / "routes" / "inference.py").read_text(encoding = "utf-8")
_m = _re.search(r"_TOOL_XML_RE = _re\.compile\((.*?)\n\)", _src, _re.DOTALL) _m = _re.search(r"_TOOL_XML_RE = _re\.compile\((.*?)\n\)", _src, _re.DOTALL)
assert _m, "could not extract _TOOL_XML_RE source" assert _m, "could not extract _TOOL_XML_RE source"
# The lazy ``(.*?)\n\)`` could grab a shorter expression if an arm is ever wrapped; # The lazy ``(.*?)\n\)`` could grab a shorter expression if an arm is ever wrapped;

8
studio/backend/tests/test_tp_vision_regression.py
View file

@ -450,7 +450,7 @@ def test_fallback_hint_uses_effective_tensor_request_not_just_toggle():
"""Tensor intent keys off _effective_tensor_parallel (toggle + extras + env), not """Tensor intent keys off _effective_tensor_parallel (toggle + extras + env), not
just the toggle, so extra/env-driven tensor users keep multi-GPU (#6659).""" just the toggle, so extra/env-driven tensor users keep multi-GPU (#6659)."""
route = Path(_BACKEND_DIR) / "routes" / "inference.py" route = Path(_BACKEND_DIR) / "routes" / "inference.py"
src = route.read_text() src = route.read_text(encoding = "utf-8")
idx = src.find("_tensor_intent_overall = _effective_tensor_parallel(") idx = src.find("_tensor_intent_overall = _effective_tensor_parallel(")
assert idx != -1, "the GGUF load closure must compute tensor intent" assert idx != -1, "the GGUF load closure must compute tensor intent"
block = src[idx : idx + 300] block = src[idx : idx + 300]
@ -482,7 +482,7 @@ def test_preserved_fallback_carried_across_non_drop_reload():
gated on the same model loaded, so a ctx-only reload keeps multi-GPU but a model gated on the same model loaded, so a ctx-only reload keeps multi-GPU but a model
switch / explicit drop doesn't inherit it (#6659).""" switch / explicit drop doesn't inherit it (#6659)."""
route = Path(_BACKEND_DIR) / "routes" / "inference.py" route = Path(_BACKEND_DIR) / "routes" / "inference.py"
src = route.read_text() src = route.read_text(encoding = "utf-8")
idx = src.find("_tensor_intent_overall = _effective_tensor_parallel(") idx = src.find("_tensor_intent_overall = _effective_tensor_parallel(")
assert idx != -1 assert idx != -1
block = src[idx : idx + 400] block = src[idx : idx + 400]
@ -499,7 +499,7 @@ def test_same_model_guard_checks_path_and_variant():
repo), so a reload keeps the carry-forward and a different variant doesn't inherit repo), so a reload keeps the carry-forward and a different variant doesn't inherit
the prior one's preserved tensor intent (#6659).""" the prior one's preserved tensor intent (#6659)."""
route = Path(_BACKEND_DIR) / "routes" / "inference.py" route = Path(_BACKEND_DIR) / "routes" / "inference.py"
src = route.read_text() src = route.read_text(encoding = "utf-8")
idx = src.find("_same_model_loaded = (") idx = src.find("_same_model_loaded = (")
assert idx != -1 assert idx != -1
block = src[idx : idx + 1300] block = src[idx : idx + 1300]
@ -748,7 +748,7 @@ def test_explicit_tensor_drop_uses_shared_helper_in_both_readers():
_is_explicit_tensor_drop, so they agree on what counts as a drop -- a reload for _is_explicit_tensor_drop, so they agree on what counts as a drop -- a reload for
an unrelated extra still carries the preserved intent rather than collapsing to one an unrelated extra still carries the preserved intent rather than collapsing to one
GPU (Codex #6659).""" GPU (Codex #6659)."""
src = (Path(_BACKEND_DIR) / "routes" / "inference.py").read_text() src = (Path(_BACKEND_DIR) / "routes" / "inference.py").read_text(encoding = "utf-8")
# Dedup reader (the preserved-fallback reload guard). # Dedup reader (the preserved-fallback reload guard).
assert "layer_preserves_tensor_intent and _is_explicit_tensor_drop(request)" in src assert "layer_preserves_tensor_intent and _is_explicit_tensor_drop(request)" in src
# Load carry-forward reader feeds the same decision into the carry-forward. # Load carry-forward reader feeds the same decision into the carry-forward.

8
studio/backend/tests/test_training_raw_support.py
View file

@ -163,13 +163,13 @@ class TestTrainingRawSupport(unittest.TestCase):
def test_route_forwards_all_grad_clipping_fields(self): def test_route_forwards_all_grad_clipping_fields(self):
# The HTTP route builds the config dict by hand; a schema field that # The HTTP route builds the config dict by hand; a schema field that
# is not forwarded here is silently dropped for REST callers. # is not forwarded here is silently dropped for REST callers.
source = (_BACKEND_ROOT / "routes" / "training.py").read_text() source = (_BACKEND_ROOT / "routes" / "training.py").read_text(encoding = "utf-8")
self.assertIn('"max_grad_norm": request.max_grad_norm', source) self.assertIn('"max_grad_norm": request.max_grad_norm', source)
self.assertIn('"max_grad_value": request.max_grad_value', source) self.assertIn('"max_grad_value": request.max_grad_value', source)
self.assertIn('"max_grad_leaf_norm": request.max_grad_leaf_norm', source) self.assertIn('"max_grad_leaf_norm": request.max_grad_leaf_norm', source)
def test_mlx_worker_falls_back_init_seeds_to_random_seed(self): def test_mlx_worker_falls_back_init_seeds_to_random_seed(self):
source = (_BACKEND_ROOT / "core" / "training" / "worker.py").read_text() source = (_BACKEND_ROOT / "core" / "training" / "worker.py").read_text(encoding = "utf-8")
# random_seed itself is normalized first so explicit None coming # random_seed itself is normalized first so explicit None coming
# from a raw / backend caller does not propagate through the chain. # from a raw / backend caller does not propagate through the chain.
@ -198,7 +198,7 @@ class TestTrainingRawSupport(unittest.TestCase):
self.assertIn("seed = random_seed,", source) self.assertIn("seed = random_seed,", source)
def test_mlx_worker_preserves_null_max_grad_value_for_trainer_default(self): def test_mlx_worker_preserves_null_max_grad_value_for_trainer_default(self):
source = (_BACKEND_ROOT / "core" / "training" / "worker.py").read_text() source = (_BACKEND_ROOT / "core" / "training" / "worker.py").read_text(encoding = "utf-8")
# None must survive to the MLX trainer so it picks its own runtime # None must survive to the MLX trainer so it picks its own runtime
# default, and any other value must coerce to float without # default, and any other value must coerce to float without
@ -251,7 +251,7 @@ class TestTrainingRawSupport(unittest.TestCase):
# unsloth-zoo update. Until that floor is in place, the # unsloth-zoo update. Until that floor is in place, the
# worker must gate them so releases that predate those fields can # worker must gate them so releases that predate those fields can
# still construct MLXTrainingConfig without TypeError. # still construct MLXTrainingConfig without TypeError.
source = (_BACKEND_ROOT / "core" / "training" / "worker.py").read_text() source = (_BACKEND_ROOT / "core" / "training" / "worker.py").read_text(encoding = "utf-8")
self.assertIn( self.assertIn(
'getattr(MLXTrainingConfig, "__dataclass_fields__", {})', 'getattr(MLXTrainingConfig, "__dataclass_fields__", {})',

2
studio/backend/tests/test_transformers_version.py
View file

@ -2672,7 +2672,7 @@ class TestLatestTierForces16Bit:
def _read(self, rel): def _read(self, rel):
backend_dir = Path(__file__).resolve().parent.parent backend_dir = Path(__file__).resolve().parent.parent
return (backend_dir / rel).read_text() return (backend_dir / rel).read_text(encoding = "utf-8")
def test_worker_guard_present(self): def test_worker_guard_present(self):
src = self._read("core/inference/worker.py") src = self._read("core/inference/worker.py")

6
studio/backend/tests/test_yaml_trust_remote_code_removed.py
View file

@ -19,7 +19,7 @@ _MODEL_DEFAULTS = _CONFIGS / "model_defaults"
def test_no_model_default_yaml_sets_trust_remote_code(): def test_no_model_default_yaml_sets_trust_remote_code():
offenders = [] offenders = []
for f in _MODEL_DEFAULTS.rglob("*.yaml"): for f in _MODEL_DEFAULTS.rglob("*.yaml"):
doc = yaml.safe_load(f.read_text()) or {} doc = yaml.safe_load(f.read_text(encoding = "utf-8")) or {}
if not isinstance(doc, dict): if not isinstance(doc, dict):
continue continue
for section, body in doc.items(): for section, body in doc.items():
@ -37,7 +37,7 @@ def test_no_model_default_yaml_has_empty_or_none_section():
# A bare `inference:` header (no keys) parses to None and crashes the .get() loaders. # A bare `inference:` header (no keys) parses to None and crashes the .get() loaders.
offenders = [] offenders = []
for f in _MODEL_DEFAULTS.rglob("*.yaml"): for f in _MODEL_DEFAULTS.rglob("*.yaml"):
doc = yaml.safe_load(f.read_text()) doc = yaml.safe_load(f.read_text(encoding = "utf-8"))
if not isinstance(doc, dict): if not isinstance(doc, dict):
offenders.append(f"{f.relative_to(_CONFIGS)} (not a mapping)") offenders.append(f"{f.relative_to(_CONFIGS)} (not a mapping)")
continue continue
@ -96,7 +96,7 @@ def test_all_model_yamls_load_for_training_and_inference():
def test_base_templates_have_no_trust_remote_code(): def test_base_templates_have_no_trust_remote_code():
for name in ("full_finetune.yaml", "lora_text.yaml", "vision_lora.yaml"): for name in ("full_finetune.yaml", "lora_text.yaml", "vision_lora.yaml"):
doc = yaml.safe_load((_CONFIGS / name).read_text()) or {} doc = yaml.safe_load((_CONFIGS / name).read_text(encoding = "utf-8")) or {}
flat = yaml.safe_dump(doc) flat = yaml.safe_dump(doc)
assert "trust_remote_code" not in flat, f"{name} should not set trust_remote_code" assert "trust_remote_code" not in flat, f"{name} should not set trust_remote_code"

4
tests/python/test_cpo_processor_text_tokenizer.py
View file

@ -40,7 +40,7 @@ def _registrations(source):
def test_cpo_registration_matches_orpo(): def test_cpo_registration_matches_orpo():
regs = _registrations(open(RL_PATH).read()) regs = _registrations(open(RL_PATH, encoding = "utf-8").read())
shared = {"orpo_trainer_text_tokenizer", "orpo_trainer_processor_pad_token"} shared = {"orpo_trainer_text_tokenizer", "orpo_trainer_processor_pad_token"}
assert shared <= set(regs.get("orpo_trainer", [])) assert shared <= set(regs.get("orpo_trainer", []))
assert shared <= set(regs.get("cpo_trainer", [])) assert shared <= set(regs.get("cpo_trainer", []))
@ -48,7 +48,7 @@ def test_cpo_registration_matches_orpo():
def _load_pad_rewriter(): def _load_pad_rewriter():
"""Exec orpo_trainer_processor_pad_token (+ _PAD_FALLBACK) without importing unsloth.""" """Exec orpo_trainer_processor_pad_token (+ _PAD_FALLBACK) without importing unsloth."""
tree = ast.parse(open(RL_PATH).read()) tree = ast.parse(open(RL_PATH, encoding = "utf-8").read())
nodes = [] nodes = []
for n in tree.body: for n in tree.body:
if isinstance(n, ast.Assign) and any( if isinstance(n, ast.Assign) and any(

2
tests/python/test_dpo_vision_processor_passthrough.py
View file

@ -11,7 +11,7 @@ RL_PATH = os.path.join(REPO_ROOT, "unsloth", "models", "rl_replacements.py")
def _load_helpers(): def _load_helpers():
src = open(RL_PATH).read() src = open(RL_PATH, encoding = "utf-8").read()
tree = ast.parse(src) tree = ast.parse(src)
import torch as _torch import torch as _torch

30
tests/python/test_e2e_no_torch_sandbox.py
View file

@ -193,7 +193,7 @@ class TestBeforeAfterImportChain:
mm = types.ModuleType('model_mappings') mm = types.ModuleType('model_mappings')
mm.MODEL_TO_TEMPLATE_MAPPER = {{}} mm.MODEL_TO_TEMPLATE_MAPPER = {{}}
sys.modules['model_mappings'] = mm sys.modules['model_mappings'] = mm
source = open({str(before_file)!r}).read() source = open({str(before_file)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import') source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .model_mappings import', 'from model_mappings import') source = source.replace('from .model_mappings import', 'from model_mappings import')
exec(source) exec(source)
@ -215,7 +215,7 @@ class TestBeforeAfterImportChain:
loggers = types.ModuleType('loggers') loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers sys.modules['loggers'] = loggers
exec(open({str(before_file)!r}).read()) exec(open({str(before_file)!r}, encoding = "utf-8").read())
""") """)
result = _run_in_sandbox(no_torch_venv, code) result = _run_in_sandbox(no_torch_venv, code)
assert result.returncode != 0, "BEFORE data_collators.py should crash without torch" assert result.returncode != 0, "BEFORE data_collators.py should crash without torch"
@ -284,7 +284,7 @@ class TestBeforeAfterImportChain:
it = types.ModuleType('iterable') it = types.ModuleType('iterable')
it.is_streaming_dataset = lambda *a, **k: False it.is_streaming_dataset = lambda *a, **k: False
sys.modules['iterable'] = it sys.modules['iterable'] = it
source = open({str(CHAT_TEMPLATES)!r}).read() source = open({str(CHAT_TEMPLATES)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import') source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .model_mappings import', 'from model_mappings import') source = source.replace('from .model_mappings import', 'from model_mappings import')
source = source.replace('from .iterable import', 'from iterable import') source = source.replace('from .iterable import', 'from iterable import')
@ -304,7 +304,7 @@ class TestBeforeAfterImportChain:
loggers = types.ModuleType('loggers') loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read()) exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
print("OK") print("OK")
""") """)
result = _run_in_sandbox(no_torch_venv, code) result = _run_in_sandbox(no_torch_venv, code)
@ -382,7 +382,7 @@ class TestDataclassInstantiation:
loggers = types.ModuleType('loggers') loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read()) exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
obj = DataCollatorSpeechSeq2SeqWithPadding(processor=None) obj = DataCollatorSpeechSeq2SeqWithPadding(processor=None)
assert obj.processor is None assert obj.processor is None
print("OK") print("OK")
@ -397,7 +397,7 @@ class TestDataclassInstantiation:
loggers = types.ModuleType('loggers') loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read()) exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
obj = DeepSeekOCRDataCollator(processor=None) obj = DeepSeekOCRDataCollator(processor=None)
assert obj.processor is None assert obj.processor is None
assert obj.max_length == 2048 assert obj.max_length == 2048
@ -414,7 +414,7 @@ class TestDataclassInstantiation:
loggers = types.ModuleType('loggers') loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read()) exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
obj = VLMDataCollator(processor=None) obj = VLMDataCollator(processor=None)
assert obj.processor is None assert obj.processor is None
assert obj.max_length == 2048 assert obj.max_length == 2048
@ -441,7 +441,7 @@ class TestDataclassInstantiation:
it.is_streaming_dataset = lambda *a, **k: False it.is_streaming_dataset = lambda *a, **k: False
sys.modules['iterable'] = it sys.modules['iterable'] = it
ns = {{}} ns = {{}}
source = open({str(CHAT_TEMPLATES)!r}).read() source = open({str(CHAT_TEMPLATES)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import') source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .model_mappings import', 'from model_mappings import') source = source.replace('from .model_mappings import', 'from model_mappings import')
source = source.replace('from .iterable import', 'from iterable import') source = source.replace('from .iterable import', 'from iterable import')
@ -473,7 +473,7 @@ class TestEdgeCasesBrokenTorch:
code = textwrap.dedent(f"""\ code = textwrap.dedent(f"""\
import sys import sys
sys.path.insert(0, {str(sandbox_dir)!r}) sys.path.insert(0, {str(sandbox_dir)!r})
exec(open({str(sandbox_dir / 'data_collators.py')!r}).read()) exec(open({str(sandbox_dir / 'data_collators.py')!r}, encoding = "utf-8").read())
obj = DataCollatorSpeechSeq2SeqWithPadding(processor=None) obj = DataCollatorSpeechSeq2SeqWithPadding(processor=None)
print("OK: data_collators works despite broken torch on sys.path") print("OK: data_collators works despite broken torch on sys.path")
""") """)
@ -495,7 +495,7 @@ class TestEdgeCasesBrokenTorch:
code = textwrap.dedent(f"""\ code = textwrap.dedent(f"""\
import sys import sys
sys.path.insert(0, {str(sandbox_dir)!r}) sys.path.insert(0, {str(sandbox_dir)!r})
source = open({str(HARDWARE_PY)!r}).read() source = open({str(HARDWARE_PY)!r}, encoding = "utf-8").read()
ns = {{'__name__': '__test__'}} ns = {{'__name__': '__test__'}}
exec(source, ns) exec(source, ns)
result = ns['detect_hardware']() result = ns['detect_hardware']()
@ -530,7 +530,7 @@ class TestEdgeCasesBrokenTorch:
code = textwrap.dedent(f"""\ code = textwrap.dedent(f"""\
import sys import sys
sys.path.insert(0, {str(sandbox_dir)!r}) sys.path.insert(0, {str(sandbox_dir)!r})
source = open({str(HARDWARE_PY)!r}).read() source = open({str(HARDWARE_PY)!r}, encoding = "utf-8").read()
ns = {{'__name__': '__test__'}} ns = {{'__name__': '__test__'}}
exec(source, ns) exec(source, ns)
result = ns['detect_hardware']() result = ns['detect_hardware']()
@ -559,7 +559,7 @@ class TestEdgeCasesBrokenTorch:
sys.modules['iterable'] = it sys.modules['iterable'] = it
ns = {{}} ns = {{}}
source = open({str(CHAT_TEMPLATES)!r}).read() source = open({str(CHAT_TEMPLATES)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import') source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .model_mappings import', 'from model_mappings import') source = source.replace('from .model_mappings import', 'from model_mappings import')
source = source.replace('from .iterable import', 'from iterable import') source = source.replace('from .iterable import', 'from iterable import')
@ -604,7 +604,7 @@ class TestHardwareDetectionNoTorch:
code = textwrap.dedent(f"""\ code = textwrap.dedent(f"""\
import sys import sys
sys.path.insert(0, {str(sandbox_dir)!r}) sys.path.insert(0, {str(sandbox_dir)!r})
source = open({str(HARDWARE_PY)!r}).read() source = open({str(HARDWARE_PY)!r}, encoding = "utf-8").read()
ns = {{'__name__': '__test__'}} ns = {{'__name__': '__test__'}}
exec(source, ns) exec(source, ns)
device = ns['detect_hardware']() device = ns['detect_hardware']()
@ -624,7 +624,7 @@ class TestHardwareDetectionNoTorch:
code = textwrap.dedent(f"""\ code = textwrap.dedent(f"""\
import sys import sys
sys.path.insert(0, {str(sandbox_dir)!r}) sys.path.insert(0, {str(sandbox_dir)!r})
source = open({str(HARDWARE_PY)!r}).read() source = open({str(HARDWARE_PY)!r}, encoding = "utf-8").read()
ns = {{'__name__': '__test__'}} ns = {{'__name__': '__test__'}}
exec(source, ns) exec(source, ns)
versions = ns['get_package_versions']() versions = ns['get_package_versions']()
@ -651,7 +651,7 @@ class TestHardwareDetectionNoTorch:
code = textwrap.dedent(f"""\ code = textwrap.dedent(f"""\
import sys import sys
sys.path.insert(0, {str(sandbox_dir)!r}) sys.path.insert(0, {str(sandbox_dir)!r})
source = open({str(hw_sandbox / 'hardware.py')!r}).read() source = open({str(hw_sandbox / 'hardware.py')!r}, encoding = "utf-8").read()
ns = {{'__name__': '__test__'}} ns = {{'__name__': '__test__'}}
exec(source, ns) exec(source, ns)
assert callable(ns['detect_hardware']) assert callable(ns['detect_hardware'])

2
tests/python/test_fast_language_model_text_only.py
View file

@ -14,7 +14,7 @@ UTILS_PATH = REPO_ROOT / "unsloth" / "models" / "_utils.py"
def _source(path): def _source(path):
return path.read_text() return path.read_text(encoding = "utf-8")
def _class_method(tree, class_name, method_name): def _class_method(tree, class_name, method_name):

2
tests/python/test_fast_model_config_passthrough.py
View file

@ -12,7 +12,7 @@ LLAMA_PATH = REPO_ROOT / "unsloth" / "models" / "llama.py"
def _source(path): def _source(path):
return path.read_text() return path.read_text(encoding = "utf-8")
def _class_method(tree, class_name, method_name): def _class_method(tree, class_name, method_name):

6
tests/python/test_gpu_init_ldconfig_guard.py
View file

@ -17,13 +17,13 @@ def _find_geteuid_guard(tree: ast.AST):
def test_gpu_init_has_geteuid_guard(): def test_gpu_init_has_geteuid_guard():
tree = ast.parse(GPU_INIT.read_text()) tree = ast.parse(GPU_INIT.read_text(encoding = "utf-8"))
guard = _find_geteuid_guard(tree) guard = _find_geteuid_guard(tree)
assert guard is not None, "_gpu_init.py must guard ldconfig recovery on os.geteuid()" assert guard is not None, "_gpu_init.py must guard ldconfig recovery on os.geteuid()"
def test_ldconfig_calls_only_inside_geteuid_guard(): def test_ldconfig_calls_only_inside_geteuid_guard():
src = GPU_INIT.read_text() src = GPU_INIT.read_text(encoding = "utf-8")
tree = ast.parse(src) tree = ast.parse(src)
guard = _find_geteuid_guard(tree) guard = _find_geteuid_guard(tree)
assert guard is not None assert guard is not None
@ -39,6 +39,6 @@ def test_ldconfig_calls_only_inside_geteuid_guard():
def test_non_root_branch_warns_when_bnb_present(): def test_non_root_branch_warns_when_bnb_present():
src = GPU_INIT.read_text() src = GPU_INIT.read_text(encoding = "utf-8")
assert "elif bnb is not None" in src assert "elif bnb is not None" in src
assert "sudo ldconfig" in src assert "sudo ldconfig" in src

2
tests/python/test_grpo_ddp_model_config.py
View file

@ -9,7 +9,7 @@ SOURCE_PATH = os.path.join(REPO_ROOT, "unsloth", "models", "rl_replacements.py")
def _read_source() -> str: def _read_source() -> str:
with open(SOURCE_PATH, "r") as fh: with open(SOURCE_PATH, "r", encoding = "utf-8") as fh:
return fh.read() return fh.read()

2
tests/python/test_orpo_processor_text_tokenizer.py
View file

@ -10,7 +10,7 @@ RL_PATH = os.path.join(REPO_ROOT, "unsloth", "models", "rl_replacements.py")
def _load_orpo_rewriter(name = "orpo_trainer_text_tokenizer"): def _load_orpo_rewriter(name = "orpo_trainer_text_tokenizer"):
src = open(RL_PATH).read() src = open(RL_PATH, encoding = "utf-8").read()
tree = ast.parse(src) tree = ast.parse(src)
ns = {"re": re} ns = {"re": re}
# Materialise sibling module-level _-prefixed assignments the rewriter may reference. # Materialise sibling module-level _-prefixed assignments the rewriter may reference.

2
tests/python/test_pad_token_fix.py
View file

@ -21,7 +21,7 @@ WANTED = {
def _load_pad_helpers(): def _load_pad_helpers():
"""Exec only the pad-token helpers with a stub logger (no heavy imports).""" """Exec only the pad-token helpers with a stub logger (no heavy imports)."""
tree = ast.parse(open(TOK_PATH).read()) tree = ast.parse(open(TOK_PATH, encoding = "utf-8").read())
nodes = [] nodes = []
for node in tree.body: for node in tree.body:
if isinstance(node, ast.Assign): if isinstance(node, ast.Assign):

18
tests/python/test_studio_import_no_torch.py
View file

@ -148,7 +148,7 @@ class TestDataCollatorsNoTorchVenv:
loggers = types.ModuleType('loggers') loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read()) exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
print("OK: exec succeeded") print("OK: exec succeeded")
""") """)
result = subprocess.run( result = subprocess.run(
@ -168,7 +168,7 @@ class TestDataCollatorsNoTorchVenv:
loggers = types.ModuleType('loggers') loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read()) exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
obj = DataCollatorSpeechSeq2SeqWithPadding(processor=None) obj = DataCollatorSpeechSeq2SeqWithPadding(processor=None)
assert obj.processor is None, "processor should be None" assert obj.processor is None, "processor should be None"
print("OK: DataCollatorSpeechSeq2SeqWithPadding instantiated") print("OK: DataCollatorSpeechSeq2SeqWithPadding instantiated")
@ -190,7 +190,7 @@ class TestDataCollatorsNoTorchVenv:
loggers = types.ModuleType('loggers') loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read()) exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
obj = DeepSeekOCRDataCollator(processor=None) obj = DeepSeekOCRDataCollator(processor=None)
assert obj.processor is None, "processor should be None" assert obj.processor is None, "processor should be None"
assert obj.max_length == 2048, "default max_length should be 2048" assert obj.max_length == 2048, "default max_length should be 2048"
@ -212,7 +212,7 @@ class TestDataCollatorsNoTorchVenv:
loggers = types.ModuleType('loggers') loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers sys.modules['loggers'] = loggers
exec(open({str(DATA_COLLATORS)!r}).read()) exec(open({str(DATA_COLLATORS)!r}, encoding = "utf-8").read())
obj = VLMDataCollator(processor=None) obj = VLMDataCollator(processor=None)
assert obj.processor is None assert obj.processor is None
assert obj.mask_input_tokens is True, "default mask_input_tokens should be True" assert obj.mask_input_tokens is True, "default mask_input_tokens should be True"
@ -259,7 +259,7 @@ class TestChatTemplatesNoTorchVenv:
sys.modules['iterable'] = iterable sys.modules['iterable'] = iterable
# Read and transform the source: replace relative imports with absolute # Read and transform the source: replace relative imports with absolute
source = open({str(CHAT_TEMPLATES)!r}).read() source = open({str(CHAT_TEMPLATES)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import') source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .model_mappings import', 'from model_mappings import') source = source.replace('from .model_mappings import', 'from model_mappings import')
source = source.replace('from .iterable import', 'from iterable import') source = source.replace('from .iterable import', 'from iterable import')
@ -305,7 +305,7 @@ class TestChatTemplatesNoTorchVenv:
sys.modules['iterable'] = iterable sys.modules['iterable'] = iterable
ns = {{}} ns = {{}}
source = open({str(CHAT_TEMPLATES)!r}).read() source = open({str(CHAT_TEMPLATES)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import') source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .model_mappings import', 'from model_mappings import') source = source.replace('from .model_mappings import', 'from model_mappings import')
source = source.replace('from .iterable import', 'from iterable import') source = source.replace('from .iterable import', 'from iterable import')
@ -402,7 +402,7 @@ class TestFormatConversionNoTorchVenv:
sys.modules['utils.hardware'] = hardware_mod sys.modules['utils.hardware'] = hardware_mod
# Read and exec format_conversion.py # Read and exec format_conversion.py
source = open({str(FORMAT_CONVERSION)!r}).read() source = open({str(FORMAT_CONVERSION)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import') source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .iterable import', 'from iterable import') source = source.replace('from .iterable import', 'from iterable import')
ns = {{'__name__': '__test__'}} ns = {{'__name__': '__test__'}}
@ -463,7 +463,7 @@ class TestFormatConversionNoTorchVenv:
sys.modules['utils'] = utils_mod sys.modules['utils'] = utils_mod
sys.modules['utils.hardware'] = hardware_mod sys.modules['utils.hardware'] = hardware_mod
source = open({str(FORMAT_CONVERSION)!r}).read() source = open({str(FORMAT_CONVERSION)!r}, encoding = "utf-8").read()
source = source.replace('from .format_detection import', 'from format_detection import') source = source.replace('from .format_detection import', 'from format_detection import')
source = source.replace('from .iterable import', 'from iterable import') source = source.replace('from .iterable import', 'from iterable import')
ns = {{'__name__': '__test__'}} ns = {{'__name__': '__test__'}}
@ -517,7 +517,7 @@ class TestNegativeControls:
loggers = types.ModuleType('loggers') loggers = types.ModuleType('loggers')
loggers.get_logger = lambda n: None loggers.get_logger = lambda n: None
sys.modules['loggers'] = loggers sys.modules['loggers'] = loggers
exec(open({temp_file!r}).read()) exec(open({temp_file!r}, encoding = "utf-8").read())
""") """)
result = subprocess.run( result = subprocess.run(
[no_torch_venv, "-c", code], [no_torch_venv, "-c", code],

2
tests/python/test_v100_fullft_precision.py
View file

@ -29,7 +29,7 @@ RL_PY = Path(__file__).resolve().parents[2] / "unsloth" / "models" / "rl.py"
def _extract_mixed_precision_code() -> str: def _extract_mixed_precision_code() -> str:
lines = RL_PY.read_text().split("\n") lines = RL_PY.read_text(encoding = "utf-8").split("\n")
try: try:
start = next(i for i, l in enumerate(lines) if "mixed_precision = (" in l) start = next(i for i, l in enumerate(lines) if "mixed_precision = (" in l)
except StopIteration: except StopIteration:

2
tests/python/test_vision_lora_targeting.py
View file

@ -37,7 +37,7 @@ def test_vlm_lora_regex_respects_language_only_with_explicit_targets():
def test_fast_vision_model_wraps_explicit_targets_when_layer_filters_are_used(): def test_fast_vision_model_wraps_explicit_targets_when_layer_filters_are_used():
source = Path("unsloth/models/vision.py").read_text() source = Path("unsloth/models/vision.py").read_text(encoding = "utf-8")
assert "target_modules = get_peft_regex(" in source assert "target_modules = get_peft_regex(" in source
assert "target_modules = list(target_modules)" in source assert "target_modules = list(target_modules)" in source

4
tests/saving/test_fix_sentencepiece_gguf_robustness.py
View file

@ -82,7 +82,7 @@ def test_entry_with_non_int_id_is_skipped(tmp_path):
def test_save_py_except_clause_is_broad_exception(): def test_save_py_except_clause_is_broad_exception():
with open(_SAVE_PY) as f: with open(_SAVE_PY, encoding = "utf-8") as f:
tree = ast.parse(f.read()) tree = ast.parse(f.read())
for node in ast.walk(tree): for node in ast.walk(tree):
if isinstance(node, ast.FunctionDef) and node.name == "unsloth_save_pretrained_gguf": if isinstance(node, ast.FunctionDef) and node.name == "unsloth_save_pretrained_gguf":
@ -102,7 +102,7 @@ def test_save_py_except_clause_is_broad_exception():
def test_tokenizer_utils_uses_import_protobuf_fallback_pattern(): def test_tokenizer_utils_uses_import_protobuf_fallback_pattern():
with open(_TOK_PY) as f: with open(_TOK_PY, encoding = "utf-8") as f:
src = f.read() src = f.read()
tree = ast.parse(src) tree = ast.parse(src)
for node in ast.walk(tree): for node in ast.walk(tree):

10
tests/security/test_scan_packages.py
View file

@ -35,9 +35,11 @@ def test_fixture_bytes_are_deterministic(tmp_path):
rebuild_dir = tmp_path / "rebuild" rebuild_dir = tmp_path / "rebuild"
rebuild_dir.mkdir() rebuild_dir.mkdir()
# The build helper writes to its own dir; copy + patch HERE. # The build helper writes to its own dir; copy + patch HERE.
builder_src = (FIXTURES / "_build.py").read_text() builder_src = (FIXTURES / "_build.py").read_text(encoding = "utf-8")
rebuilt_helper = rebuild_dir / "_build.py" rebuilt_helper = rebuild_dir / "_build.py"
rebuilt_helper.write_text(builder_src) # builder_src came out of a checked-in file, so it carries whatever
# non-ASCII that file holds and cp1252 cannot encode it back out.
rebuilt_helper.write_text(builder_src, encoding = "utf-8")
# Run with SOURCE_DATE_EPOCH=0 and HERE override via a shim. # Run with SOURCE_DATE_EPOCH=0 and HERE override via a shim.
shim = rebuild_dir / "run.py" shim = rebuild_dir / "run.py"
shim.write_text( shim.write_text(
@ -1260,7 +1262,7 @@ def test_committed_baseline_suppresses_known_but_not_a_new_payload():
import json import json
baseline_path = REPO_ROOT / "scripts" / "scan_packages_baseline.json" baseline_path = REPO_ROOT / "scripts" / "scan_packages_baseline.json"
entries = json.loads(baseline_path.read_text())["entries"] entries = json.loads(baseline_path.read_text(encoding = "utf-8"))["entries"]
target = next( target = next(
e e
for e in entries for e in entries
@ -1296,7 +1298,7 @@ def test_committed_baseline_entries_all_carry_evidence_hash():
import json import json
baseline_path = REPO_ROOT / "scripts" / "scan_packages_baseline.json" baseline_path = REPO_ROOT / "scripts" / "scan_packages_baseline.json"
entries = json.loads(baseline_path.read_text())["entries"] entries = json.loads(baseline_path.read_text(encoding = "utf-8"))["entries"]
assert entries, "committed baseline should not be empty" assert entries, "committed baseline should not be empty"
missing = [ missing = [
f"{e['package']}:{e['file']}:{e['check']}" for e in entries if not e.get("evidence_hash") f"{e['package']}:{e['file']}:{e['check']}" for e in entries if not e.get("evidence_hash")

4
tests/studio/install/test_llama_pr_force_and_source.py
View file

@ -346,7 +346,7 @@ class TestSourcePatternsSh:
@pytest.fixture(autouse = True) @pytest.fixture(autouse = True)
def _load_source(self): def _load_source(self):
self.content = SETUP_SH.read_text() self.content = SETUP_SH.read_text(encoding = "utf-8")
def test_has_default_pr_force(self): def test_has_default_pr_force(self):
assert '_DEFAULT_LLAMA_PR_FORCE=""' in self.content assert '_DEFAULT_LLAMA_PR_FORCE=""' in self.content
@ -412,7 +412,7 @@ class TestSourcePatternsPs1:
@pytest.fixture(autouse = True) @pytest.fixture(autouse = True)
def _load_source(self): def _load_source(self):
self.content = SETUP_PS1.read_text() self.content = SETUP_PS1.read_text(encoding = "utf-8")
def test_has_default_pr_force(self): def test_has_default_pr_force(self):
assert '$DefaultLlamaPrForce = ""' in self.content assert '$DefaultLlamaPrForce = ""' in self.content

6
tests/studio/install/test_managed_node_runtime.py
View file

@ -125,7 +125,7 @@ def test_resolve_falls_back_to_managed_when_no_system(monkeypatch, tmp_path):
monkeypatch.setenv("UNSLOTH_STUDIO_HOME", str(tmp_path)) monkeypatch.setenv("UNSLOTH_STUDIO_HOME", str(tmp_path))
managed = nr.managed_node_binary() managed = nr.managed_node_binary()
managed.parent.mkdir(parents = True, exist_ok = True) managed.parent.mkdir(parents = True, exist_ok = True)
managed.write_text("#!/bin/sh\necho v24.17.0\n") managed.write_text("#!/bin/sh\necho v24.17.0\n", encoding = "utf-8")
monkeypatch.setattr(nr.shutil, "which", lambda name: None) monkeypatch.setattr(nr.shutil, "which", lambda name: None)
monkeypatch.setattr(nr, "_node_version_ok", lambda exe: str(exe) == str(managed)) monkeypatch.setattr(nr, "_node_version_ok", lambda exe: str(exe) == str(managed))
assert nr.resolve_node_executable() == str(managed) assert nr.resolve_node_executable() == str(managed)
@ -136,7 +136,7 @@ def test_resolve_prefers_managed_over_unsuitable_system(monkeypatch, tmp_path):
monkeypatch.setenv("UNSLOTH_STUDIO_HOME", str(tmp_path)) monkeypatch.setenv("UNSLOTH_STUDIO_HOME", str(tmp_path))
managed = nr.managed_node_binary() managed = nr.managed_node_binary()
managed.parent.mkdir(parents = True, exist_ok = True) managed.parent.mkdir(parents = True, exist_ok = True)
managed.write_text("fake") managed.write_text("fake", encoding = "utf-8")
monkeypatch.setattr(nr.shutil, "which", lambda name: "/old/node") monkeypatch.setattr(nr.shutil, "which", lambda name: "/old/node")
monkeypatch.setattr(nr, "_node_version_ok", lambda exe: str(exe) == str(managed)) monkeypatch.setattr(nr, "_node_version_ok", lambda exe: str(exe) == str(managed))
assert nr.resolve_node_executable() == str(managed) assert nr.resolve_node_executable() == str(managed)
@ -167,7 +167,7 @@ def test_negative_result_is_not_cached(monkeypatch, tmp_path):
managed = nr.managed_node_binary() managed = nr.managed_node_binary()
managed.parent.mkdir(parents = True, exist_ok = True) managed.parent.mkdir(parents = True, exist_ok = True)
managed.write_text("now-installed") managed.write_text("now-installed", encoding = "utf-8")
monkeypatch.setattr(nr, "_node_version_ok", lambda exe: str(exe) == str(managed)) monkeypatch.setattr(nr, "_node_version_ok", lambda exe: str(exe) == str(managed))
assert nr.resolve_node_executable() == str(managed) assert nr.resolve_node_executable() == str(managed)

46
tests/studio/install/test_pr4562_bugfixes.py
View file

@ -617,7 +617,7 @@ class TestSourceCodePatterns:
def test_setup_sh_no_rm_before_prereq_check(self): def test_setup_sh_no_rm_before_prereq_check(self):
"""rm -rf must appear AFTER cmake/git checks, not before.""" """rm -rf must appear AFTER cmake/git checks, not before."""
content = SETUP_SH.read_text() content = SETUP_SH.read_text(encoding = "utf-8")
# Anchor on the source-build cmake check block. # Anchor on the source-build cmake check block.
idx_block = content.find("command -v cmake") idx_block = content.find("command -v cmake")
assert idx_block != -1 assert idx_block != -1
@ -630,7 +630,7 @@ class TestSourceCodePatterns:
def test_setup_sh_clone_uses_branch_tag(self): def test_setup_sh_clone_uses_branch_tag(self):
"""git clone in source-build should use --branch via the clone args array.""" """git clone in source-build should use --branch via the clone args array."""
content = SETUP_SH.read_text() content = SETUP_SH.read_text(encoding = "utf-8")
assert "_CLONE_ARGS=(git clone --depth 1)" in content assert "_CLONE_ARGS=(git clone --depth 1)" in content
assert ( assert (
'_CLONE_ARGS+=(--branch "$_RESOLVED_SOURCE_REF")' in content '_CLONE_ARGS+=(--branch "$_RESOLVED_SOURCE_REF")' in content
@ -642,7 +642,7 @@ class TestSourceCodePatterns:
def test_setup_sh_source_build_uses_helper_latest_tag_only(self): def test_setup_sh_source_build_uses_helper_latest_tag_only(self):
"""Shell source fallback should only use helper latest-tag resolution.""" """Shell source fallback should only use helper latest-tag resolution."""
content = SETUP_SH.read_text() content = SETUP_SH.read_text(encoding = "utf-8")
assert "--resolve-source-build" not in content assert "--resolve-source-build" not in content
assert "--resolve-install-tag" not in content assert "--resolve-install-tag" not in content
assert '--resolve-llama-tag latest --published-repo "ggml-org/llama.cpp"' in content assert '--resolve-llama-tag latest --published-repo "ggml-org/llama.cpp"' in content
@ -653,7 +653,7 @@ class TestSourceCodePatterns:
def test_setup_sh_prebuilt_install_entrypoint(self): def test_setup_sh_prebuilt_install_entrypoint(self):
"""Shell prebuilt path uses the helper install entrypoint, not the old releases-latest flow.""" """Shell prebuilt path uses the helper install entrypoint, not the old releases-latest flow."""
content = SETUP_SH.read_text() content = SETUP_SH.read_text(encoding = "utf-8")
assert "--resolve-install-tag" not in content assert "--resolve-install-tag" not in content
assert "_HELPER_RELEASE_REPO}/releases/latest" not in content assert "_HELPER_RELEASE_REPO}/releases/latest" not in content
assert "ggml-org/llama.cpp/releases/latest" not in content assert "ggml-org/llama.cpp/releases/latest" not in content
@ -663,7 +663,7 @@ class TestSourceCodePatterns:
fork like every other host, so the release-repo decision is unconditional. fork like every other host, so the release-repo decision is unconditional.
Guards against a silent reintroduction of a ggml-org CPU routing branch. Guards against a silent reintroduction of a ggml-org CPU routing branch.
GPU usability detection (used for PyTorch / source decisions) must stay.""" GPU usability detection (used for PyTorch / source decisions) must stay."""
content = SETUP_SH.read_text() content = SETUP_SH.read_text(encoding = "utf-8")
assert '_HELPER_RELEASE_REPO="unslothai/llama.cpp"' in content assert '_HELPER_RELEASE_REPO="unslothai/llama.cpp"' in content
assert '_HELPER_RELEASE_REPO="ggml-org/llama.cpp"' not in content assert '_HELPER_RELEASE_REPO="ggml-org/llama.cpp"' not in content
# Usability gating (not routing) still distinguishes a hidden GPU. # Usability gating (not routing) still distinguishes a hidden GPU.
@ -676,14 +676,14 @@ class TestSourceCodePatterns:
def test_setup_sh_reports_installed_prebuilt_release(self): def test_setup_sh_reports_installed_prebuilt_release(self):
"""Shell wrapper should report the installed prebuilt release from metadata.""" """Shell wrapper should report the installed prebuilt release from metadata."""
content = SETUP_SH.read_text() content = SETUP_SH.read_text(encoding = "utf-8")
assert "UNSLOTH_PREBUILT_INFO.json" in content assert "UNSLOTH_PREBUILT_INFO.json" in content
assert "installed release:" in content assert "installed release:" in content
assert 'print_installed_llama_prebuilt_release "$LLAMA_CPP_DIR"' in content assert 'print_installed_llama_prebuilt_release "$LLAMA_CPP_DIR"' in content
def test_setup_sh_macos_arm64_uses_metal_flags(self): def test_setup_sh_macos_arm64_uses_metal_flags(self):
"""Apple Silicon source builds should explicitly enable Metal like upstream.""" """Apple Silicon source builds should explicitly enable Metal like upstream."""
content = SETUP_SH.read_text() content = SETUP_SH.read_text(encoding = "utf-8")
assert "_IS_MACOS_ARM64=true" in content assert "_IS_MACOS_ARM64=true" in content
assert 'if [ "$_IS_MACOS_ARM64" = true ]; then' in content assert 'if [ "$_IS_MACOS_ARM64" = true ]; then' in content
assert "-DGGML_METAL=ON" in content assert "-DGGML_METAL=ON" in content
@ -695,7 +695,7 @@ class TestSourceCodePatterns:
def test_setup_sh_macos_metal_configure_has_cpu_fallback(self): def test_setup_sh_macos_metal_configure_has_cpu_fallback(self):
"""GPU configure/build failure retries a CPU build. Stays label-agnostic """GPU configure/build failure retries a CPU build. Stays label-agnostic
(PR #5826 generalised the Metal-only wording via $_FB_LABEL).""" (PR #5826 generalised the Metal-only wording via $_FB_LABEL)."""
content = SETUP_SH.read_text() content = SETUP_SH.read_text(encoding = "utf-8")
assert "_TRY_METAL_CPU_FALLBACK=true" in content assert "_TRY_METAL_CPU_FALLBACK=true" in content
assert 'configure failed; retrying CPU build..." "$C_WARN"' in content assert 'configure failed; retrying CPU build..." "$C_WARN"' in content
assert 'build failed; retrying CPU build..." "$C_WARN"' in content assert 'build failed; retrying CPU build..." "$C_WARN"' in content
@ -714,7 +714,7 @@ class TestSourceCodePatterns:
"""PR #5826: a fresh CUDA toolkit's host-compiler whitelist lags distro gcc/clang """PR #5826: a fresh CUDA toolkit's host-compiler whitelist lags distro gcc/clang
(nvcc "#error -- unsupported GNU version"). setup.sh exports (nvcc "#error -- unsupported GNU version"). setup.sh exports
NVCC_PREPEND_FLAGS=-allow-unsupported-compiler via env, not CMAKE_ARGS (word-splitting safety).""" NVCC_PREPEND_FLAGS=-allow-unsupported-compiler via env, not CMAKE_ARGS (word-splitting safety)."""
content = SETUP_SH.read_text() content = SETUP_SH.read_text(encoding = "utf-8")
assert "-allow-unsupported-compiler" in content assert "-allow-unsupported-compiler" in content
# Via NVCC_PREPEND_FLAGS (covers the configure-time probe too), not CMAKE_ARGS. # Via NVCC_PREPEND_FLAGS (covers the configure-time probe too), not CMAKE_ARGS.
assert "export NVCC_PREPEND_FLAGS=" in content assert "export NVCC_PREPEND_FLAGS=" in content
@ -726,7 +726,7 @@ class TestSourceCodePatterns:
def test_setup_ps1_exports_allow_unsupported_compiler(self): def test_setup_ps1_exports_allow_unsupported_compiler(self):
"""Windows parity for PR #5826: CUDA toolkit whitelist lags MSVC. setup.ps1 sets """Windows parity for PR #5826: CUDA toolkit whitelist lags MSVC. setup.ps1 sets
NVCC_PREPEND_FLAGS=-allow-unsupported-compiler in the CUDA branch via env, out of $CmakeArgs.""" NVCC_PREPEND_FLAGS=-allow-unsupported-compiler in the CUDA branch via env, out of $CmakeArgs."""
content = SETUP_PS1.read_text() content = SETUP_PS1.read_text(encoding = "utf-8")
assert "-allow-unsupported-compiler" in content assert "-allow-unsupported-compiler" in content
# Via process env, not $CmakeArgs, so it reaches both the configure probe and `cmake --build`. # Via process env, not $CmakeArgs, so it reaches both the configure probe and `cmake --build`.
assert "$env:NVCC_PREPEND_FLAGS" in content assert "$env:NVCC_PREPEND_FLAGS" in content
@ -763,7 +763,7 @@ class TestSourceCodePatterns:
def test_setup_sh_does_not_enable_metal_for_intel_macos(self): def test_setup_sh_does_not_enable_metal_for_intel_macos(self):
"""Intel macOS should stay on the existing non-Metal path in this patch.""" """Intel macOS should stay on the existing non-Metal path in this patch."""
content = SETUP_SH.read_text() content = SETUP_SH.read_text(encoding = "utf-8")
assert 'if [ "$_IS_MACOS_ARM64" = true ]; then' in content assert 'if [ "$_IS_MACOS_ARM64" = true ]; then' in content
assert ( assert (
'Darwin" ] && { [ "$_HOST_MACHINE" = "arm64" ] || [ "$_HOST_MACHINE" = "aarch64" ]; }' 'Darwin" ] && { [ "$_HOST_MACHINE" = "arm64" ] || [ "$_HOST_MACHINE" = "aarch64" ]; }'
@ -778,20 +778,20 @@ class TestSourceCodePatterns:
def test_setup_ps1_uses_checkout_b(self): def test_setup_ps1_uses_checkout_b(self):
"""PS1 should use checkout -B, not checkout --force FETCH_HEAD.""" """PS1 should use checkout -B, not checkout --force FETCH_HEAD."""
content = SETUP_PS1.read_text() content = SETUP_PS1.read_text(encoding = "utf-8")
assert "checkout -B unsloth-llama-build" in content assert "checkout -B unsloth-llama-build" in content
assert "checkout --force FETCH_HEAD" not in content assert "checkout --force FETCH_HEAD" not in content
def test_setup_ps1_clone_uses_branch_tag(self): def test_setup_ps1_clone_uses_branch_tag(self):
"""PS1 clone should use --branch with the resolved tag.""" """PS1 clone should use --branch with the resolved tag."""
content = SETUP_PS1.read_text() content = SETUP_PS1.read_text(encoding = "utf-8")
assert "--branch" in content and "$ResolvedSourceRef" in content assert "--branch" in content and "$ResolvedSourceRef" in content
# The old commented-out clone line should be gone. # The old commented-out clone line should be gone.
assert "# git clone --depth 1 --branch" not in content assert "# git clone --depth 1 --branch" not in content
def test_setup_ps1_no_git_pull(self): def test_setup_ps1_no_git_pull(self):
"""PS1 should use fetch, not pull (which fails in detached HEAD).""" """PS1 should use fetch, not pull (which fails in detached HEAD)."""
content = SETUP_PS1.read_text() content = SETUP_PS1.read_text(encoding = "utf-8")
# No "git pull" in the source-build section (only valid on a branch). # No "git pull" in the source-build section (only valid on a branch).
lines = content.splitlines() lines = content.splitlines()
for i, line in enumerate(lines): for i, line in enumerate(lines):
@ -800,18 +800,18 @@ class TestSourceCodePatterns:
# Allowed elsewhere; fail only in the llama.cpp build section. # Allowed elsewhere; fail only in the llama.cpp build section.
context = "\n".join(lines[max(0, i - 5) : i + 5]) context = "\n".join(lines[max(0, i - 5) : i + 5])
if "LlamaCppDir" in context: if "LlamaCppDir" in context:
pytest.fail(f"Found 'git pull' in llama.cpp build section at line {i+1}") pytest.fail(f"Found 'git pull' in llama.cpp build section at line {i + 1}")
def test_setup_ps1_prebuilt_install_entrypoint(self): def test_setup_ps1_prebuilt_install_entrypoint(self):
"""PS1 prebuilt path uses the helper install entrypoint, not the old releases-latest flow.""" """PS1 prebuilt path uses the helper install entrypoint, not the old releases-latest flow."""
content = SETUP_PS1.read_text() content = SETUP_PS1.read_text(encoding = "utf-8")
assert "--resolve-install-tag" not in content assert "--resolve-install-tag" not in content
assert "$HelperReleaseRepo/releases/latest" not in content assert "$HelperReleaseRepo/releases/latest" not in content
assert "ggml-org/llama.cpp/releases/latest" not in content assert "ggml-org/llama.cpp/releases/latest" not in content
def test_setup_ps1_reports_installed_prebuilt_release(self): def test_setup_ps1_reports_installed_prebuilt_release(self):
"""PS1 wrapper should report the installed prebuilt release from metadata.""" """PS1 wrapper should report the installed prebuilt release from metadata."""
content = SETUP_PS1.read_text() content = SETUP_PS1.read_text(encoding = "utf-8")
assert "Get-InstalledLlamaPrebuiltRelease" in content assert "Get-InstalledLlamaPrebuiltRelease" in content
assert "UNSLOTH_PREBUILT_INFO.json" in content assert "UNSLOTH_PREBUILT_INFO.json" in content
assert "installed release:" in content assert "installed release:" in content
@ -822,7 +822,7 @@ class TestSourceCodePatterns:
def test_setup_ps1_source_build_uses_helper_latest_tag_only(self): def test_setup_ps1_source_build_uses_helper_latest_tag_only(self):
"""PS1 source fallback should only use helper latest-tag resolution.""" """PS1 source fallback should only use helper latest-tag resolution."""
content = SETUP_PS1.read_text() content = SETUP_PS1.read_text(encoding = "utf-8")
assert "--resolve-source-build" not in content assert "--resolve-source-build" not in content
assert "--resolve-install-tag" not in content assert "--resolve-install-tag" not in content
assert ( assert (
@ -835,7 +835,7 @@ class TestSourceCodePatterns:
def test_setup_ps1_prebuilt_install_disables_native_error_abort(self): def test_setup_ps1_prebuilt_install_disables_native_error_abort(self):
"""PS1 prebuilt install should not abort setup on helper stderr.""" """PS1 prebuilt install should not abort setup on helper stderr."""
content = SETUP_PS1.read_text() content = SETUP_PS1.read_text(encoding = "utf-8")
install_idx = content.index("& python @prebuiltArgs 2>&1") install_idx = content.index("& python @prebuiltArgs 2>&1")
block = content[max(0, install_idx - 800) : install_idx + 800] block = content[max(0, install_idx - 800) : install_idx + 800]
assert "$PSNativeCommandUseErrorActionPreference = $false" in block assert "$PSNativeCommandUseErrorActionPreference = $false" in block
@ -844,7 +844,7 @@ class TestSourceCodePatterns:
def test_setup_ps1_helper_disables_error_action_abort(self): def test_setup_ps1_helper_disables_error_action_abort(self):
"""Helper resolution should suppress terminating NativeCommandError on PS 5.1.""" """Helper resolution should suppress terminating NativeCommandError on PS 5.1."""
content = SETUP_PS1.read_text() content = SETUP_PS1.read_text(encoding = "utf-8")
helper_idx = content.index("function Invoke-LlamaHelper") helper_idx = content.index("function Invoke-LlamaHelper")
block = content[helper_idx : helper_idx + 2200] block = content[helper_idx : helper_idx + 2200]
assert "$previousErrorActionPreference = $ErrorActionPreference" in block assert "$previousErrorActionPreference = $ErrorActionPreference" in block
@ -853,19 +853,19 @@ class TestSourceCodePatterns:
def test_setup_ps1_uses_local_tempfile_helper(self): def test_setup_ps1_uses_local_tempfile_helper(self):
"""PS1 should not depend on New-TemporaryFile being available anywhere.""" """PS1 should not depend on New-TemporaryFile being available anywhere."""
content = SETUP_PS1.read_text() content = SETUP_PS1.read_text(encoding = "utf-8")
assert "function New-UnslothTemporaryFile" in content assert "function New-UnslothTemporaryFile" in content
assert "$resolveErrorLog = New-TemporaryFile" not in content assert "$resolveErrorLog = New-TemporaryFile" not in content
def test_setup_ps1_find_nvcc_uses_version_sort_for_latest_toolkit(self): def test_setup_ps1_find_nvcc_uses_version_sort_for_latest_toolkit(self):
"""The unconstrained nvcc fallback should not sort toolkit dirs lexicographically.""" """The unconstrained nvcc fallback should not sort toolkit dirs lexicographically."""
content = SETUP_PS1.read_text() content = SETUP_PS1.read_text(encoding = "utf-8")
assert "Sort-Object Name | Select-Object -Last 1" not in content assert "Sort-Object Name | Select-Object -Last 1" not in content
assert "Sort-Object { [version]($_.Name -replace '^v','') } -Descending" in content assert "Sort-Object { [version]($_.Name -replace '^v','') } -Descending" in content
def test_binary_env_linux_has_binary_parent(self): def test_binary_env_linux_has_binary_parent(self):
"""The Linux branch of binary_env should include binary_path.parent.""" """The Linux branch of binary_env should include binary_path.parent."""
content = MODULE_PATH.read_text() content = MODULE_PATH.read_text(encoding = "utf-8")
in_func = False in_func = False
in_linux = False in_linux = False
found = False found = False

6
tests/studio/load_freeze/test_load_orchestrator.py
View file

@ -441,7 +441,7 @@ def test_load_model_caches_audio_type_inside_serial_load_lock():
"""Audio-type detection must run inside load_model under _serial_load_lock, """Audio-type detection must run inside load_model under _serial_load_lock,
else a concurrent /load can replace the backend mid-probe (review on #5669).""" else a concurrent /load can replace the backend mid-probe (review on #5669)."""
f = _REPO_ROOT / "studio" / "backend" / "core" / "inference" / "llama_cpp.py" f = _REPO_ROOT / "studio" / "backend" / "core" / "inference" / "llama_cpp.py"
text = f.read_text() text = f.read_text(encoding = "utf-8")
assert ( assert (
"with self._serial_load_lock" in text "with self._serial_load_lock" in text
), "LlamaCppBackend.load_model must hold self._serial_load_lock" ), "LlamaCppBackend.load_model must hold self._serial_load_lock"
@ -462,7 +462,7 @@ def test_routes_inference_reads_cached_audio_type_not_calls_detect():
"""routes/inference.py must read cached _audio_type/_is_audio, not call """routes/inference.py must read cached _audio_type/_is_audio, not call
detect_audio_type / init_audio_codec directly (both moved into load_model).""" detect_audio_type / init_audio_codec directly (both moved into load_model)."""
f = _REPO_ROOT / "studio" / "backend" / "routes" / "inference.py" f = _REPO_ROOT / "studio" / "backend" / "routes" / "inference.py"
text = f.read_text() text = f.read_text(encoding = "utf-8")
assert "llama_backend.detect_audio_type(" not in text, ( assert "llama_backend.detect_audio_type(" not in text, (
"routes/inference.py should not call detect_audio_type directly; " "routes/inference.py should not call detect_audio_type directly; "
"load_model already cached it under the lock." "load_model already cached it under the lock."
@ -485,7 +485,7 @@ def test_no_other_async_route_calls_detect_audio_type_unwrapped():
# function helper is excluded below. # function helper is excluded below.
pattern = re.compile(r"\b\w+\.detect_audio_type\s*\(") pattern = re.compile(r"\b\w+\.detect_audio_type\s*\(")
for path in routes_dir.rglob("*.py"): for path in routes_dir.rglob("*.py"):
for i, line in enumerate(path.read_text().splitlines(), start = 1): for i, line in enumerate(path.read_text(encoding = "utf-8").splitlines(), start = 1):
m = pattern.search(line) m = pattern.search(line)
if not m: if not m:
continue continue

10
tests/studio/playwright_chat_ime_i18n.py
View file

@ -241,10 +241,12 @@ with sync_playwright() as p:
# Source-level guard: grep the unmounted edit/compare composers' JSX for dir="auto". # Source-level guard: grep the unmounted edit/compare composers' JSX for dir="auto".
_repo_root = Path(__file__).resolve().parents[2] _repo_root = Path(__file__).resolve().parents[2]
_thread_src = ( _thread_src = (_repo_root / "studio/frontend/src/components/assistant-ui/thread.tsx").read_text(
_repo_root / "studio/frontend/src/components/assistant-ui/thread.tsx" encoding = "utf-8"
).read_text() )
_shared_src = (_repo_root / "studio/frontend/src/features/chat/shared-composer.tsx").read_text() _shared_src = (_repo_root / "studio/frontend/src/features/chat/shared-composer.tsx").read_text(
encoding = "utf-8"
)
_edit_idx = _thread_src.find("aui-edit-composer-input") _edit_idx = _thread_src.find("aui-edit-composer-input")
if _edit_idx == -1 or 'dir="auto"' not in _thread_src[_edit_idx : _edit_idx + 600]: if _edit_idx == -1 or 'dir="auto"' not in _thread_src[_edit_idx : _edit_idx + 600]:
soft_fail('edit composer source is missing dir="auto"') soft_fail('edit composer source is missing dir="auto"')

2
tests/studio/playwright_chat_ui.py
View file

@ -98,7 +98,7 @@ def expected_default_model():
/ "defaults.py" / "defaults.py"
) )
try: try:
tree = ast.parse(defaults_path.read_text()) tree = ast.parse(defaults_path.read_text(encoding = "utf-8"))
except Exception as exc: except Exception as exc:
fail(f"could not read {defaults_path}: {exc}") fail(f"could not read {defaults_path}: {exc}")
models = None models = None

2
tests/studio/studio_api_smoke.py
View file

@ -142,7 +142,7 @@ except Exception as exc:
# GET / cross-origin must NOT leak the bootstrap password in the served HTML. # GET / cross-origin must NOT leak the bootstrap password in the served HTML.
boot_path = AUTH_DIR / ".bootstrap_password" boot_path = AUTH_DIR / ".bootstrap_password"
if boot_path.exists(): if boot_path.exists():
bootstrap_pw = boot_path.read_text().strip() bootstrap_pw = boot_path.read_text(encoding = "utf-8").strip()
if bootstrap_pw: if bootstrap_pw:
req = urllib.request.Request( req = urllib.request.Request(
f"{BASE}/", f"{BASE}/",

28
tests/studio/test_auth_form_input_count.py
View file

@ -51,7 +51,7 @@ def _conditional_extent(src: str) -> tuple[int, int]:
def test_hasbootstrappassword_constant_is_derived_from_bootstrap_window_value(): def test_hasbootstrappassword_constant_is_derived_from_bootstrap_window_value():
"""The guard must read from window.__UNSLOTH_BOOTSTRAP__, matching the backend's """The guard must read from window.__UNSLOTH_BOOTSTRAP__, matching the backend's
bootstrap-injection contract in studio/backend/main.py::_inject_bootstrap.""" bootstrap-injection contract in studio/backend/main.py::_inject_bootstrap."""
src = AUTH_FORM.read_text() src = AUTH_FORM.read_text(encoding = "utf-8")
assert "const hasBootstrapPassword = Boolean(window.__UNSLOTH_BOOTSTRAP__?.password);" in src, ( assert "const hasBootstrapPassword = Boolean(window.__UNSLOTH_BOOTSTRAP__?.password);" in src, (
"hasBootstrapPassword constant missing or its derivation drifted; " "hasBootstrapPassword constant missing or its derivation drifted; "
"this is the gate that hides the Current password input on first boot" "this is the gate that hides the Current password input on first boot"
@ -61,7 +61,7 @@ def test_hasbootstrappassword_constant_is_derived_from_bootstrap_window_value():
def test_exactly_one_hasBootstrapPassword_conditional_exists(): def test_exactly_one_hasBootstrapPassword_conditional_exists():
"""Only one `!hasBootstrapPassword` JSX check is allowed; a second would split """Only one `!hasBootstrapPassword` JSX check is allowed; a second would split
rendering into branches and likely hide or duplicate the New / Confirm inputs.""" rendering into branches and likely hide or duplicate the New / Confirm inputs."""
src = AUTH_FORM.read_text() src = AUTH_FORM.read_text(encoding = "utf-8")
count = src.count("!hasBootstrapPassword") count = src.count("!hasBootstrapPassword")
assert count == 1, ( assert count == 1, (
f"expected exactly one !hasBootstrapPassword usage, found {count}; " f"expected exactly one !hasBootstrapPassword usage, found {count}; "
@ -72,7 +72,7 @@ def test_exactly_one_hasBootstrapPassword_conditional_exists():
def test_current_password_input_is_inside_the_hasBootstrapPassword_conditional(): def test_current_password_input_is_inside_the_hasBootstrapPassword_conditional():
"""`id="current-password"` must sit inside `{!hasBootstrapPassword && (...)}`, """`id="current-password"` must sit inside `{!hasBootstrapPassword && (...)}`,
else it renders on first boot too, regressing the pre-#5490 UX that PR #5545 restores.""" else it renders on first boot too, regressing the pre-#5490 UX that PR #5545 restores."""
src = AUTH_FORM.read_text() src = AUTH_FORM.read_text(encoding = "utf-8")
s, e = _conditional_extent(src) s, e = _conditional_extent(src)
idx = src.find('id="current-password"') idx = src.find('id="current-password"')
assert idx != -1, "the Current password input was removed entirely" assert idx != -1, "the Current password input was removed entirely"
@ -86,7 +86,7 @@ def test_current_password_input_is_inside_the_hasBootstrapPassword_conditional()
def test_new_password_input_is_outside_the_hasBootstrapPassword_conditional(): def test_new_password_input_is_outside_the_hasBootstrapPassword_conditional():
"""`id="new-password"` must sit outside `{!hasBootstrapPassword && (...)}`, """`id="new-password"` must sit outside `{!hasBootstrapPassword && (...)}`,
else it disappears on admin-forced resets, regressing PR #5490.""" else it disappears on admin-forced resets, regressing PR #5490."""
src = AUTH_FORM.read_text() src = AUTH_FORM.read_text(encoding = "utf-8")
s, e = _conditional_extent(src) s, e = _conditional_extent(src)
idx = src.find('id="new-password"') idx = src.find('id="new-password"')
assert idx != -1, "the New password input was removed entirely" assert idx != -1, "the New password input was removed entirely"
@ -99,7 +99,7 @@ def test_new_password_input_is_outside_the_hasBootstrapPassword_conditional():
def test_confirm_password_input_is_outside_the_hasBootstrapPassword_conditional(): def test_confirm_password_input_is_outside_the_hasBootstrapPassword_conditional():
"""Same as New password, for `id="confirm-password"`.""" """Same as New password, for `id="confirm-password"`."""
src = AUTH_FORM.read_text() src = AUTH_FORM.read_text(encoding = "utf-8")
s, e = _conditional_extent(src) s, e = _conditional_extent(src)
idx = src.find('id="confirm-password"') idx = src.find('id="confirm-password"')
assert idx != -1, "the Confirm password input was removed entirely" assert idx != -1, "the Confirm password input was removed entirely"
@ -114,7 +114,7 @@ def test_change_password_jsx_declares_exactly_three_password_inputs():
"""The change-password JSX block (`{!isLoginMode && (...)}`) must declare exactly """The change-password JSX block (`{!isLoginMode && (...)}`) must declare exactly
current/new/confirm; a fourth would break the 2-input first-boot contract (the current/new/confirm; a fourth would break the 2-input first-boot contract (the
conditional only hides Current).""" conditional only hides Current)."""
src = AUTH_FORM.read_text() src = AUTH_FORM.read_text(encoding = "utf-8")
start = src.find("{!isLoginMode && (") start = src.find("{!isLoginMode && (")
assert start != -1, ( assert start != -1, (
"the change-password JSX subtree marker {!isLoginMode && (...)} " "the change-password JSX subtree marker {!isLoginMode && (...)} "
@ -147,7 +147,7 @@ def test_change_password_jsx_declares_exactly_three_password_inputs():
def test_login_jsx_declares_exactly_one_password_input(): def test_login_jsx_declares_exactly_one_password_input():
"""The login JSX block (`isLoginMode && (...)`) must declare exactly one password """The login JSX block (`isLoginMode && (...)`) must declare exactly one password
input (the bootstrap password pasted from the CLI); a second breaks the per-mode matrix.""" input (the bootstrap password pasted from the CLI); a second breaks the per-mode matrix."""
src = AUTH_FORM.read_text() src = AUTH_FORM.read_text(encoding = "utf-8")
start = src.find("{isLoginMode && (") start = src.find("{isLoginMode && (")
assert start != -1, "the login JSX subtree marker is missing" assert start != -1, "the login JSX subtree marker is missing"
depth = 1 depth = 1
@ -163,18 +163,20 @@ def test_login_jsx_declares_exactly_one_password_input():
ids = re.findall(r'id="([a-z-]+)"', subtree) ids = re.findall(r'id="([a-z-]+)"', subtree)
# Lock the count, not the spelling, so a rename does not falsely fail. # Lock the count, not the spelling, so a rename does not falsely fail.
pw_ids = [x for x in ids if "password" in x] pw_ids = [x for x in ids if "password" in x]
assert len(pw_ids) == 1, ( assert (
f"login JSX must declare exactly one password-typed input; " f"found {pw_ids!r}" len(pw_ids) == 1
) ), f"login JSX must declare exactly one password-typed input; found {pw_ids!r}"
def test_auth_flow_routes_do_not_mount_global_settings(): def test_auth_flow_routes_do_not_mount_global_settings():
root = (FRONTEND / "app/routes/__root.tsx").read_text() root = (FRONTEND / "app/routes/__root.tsx").read_text(encoding = "utf-8")
assert "{!isAuthFlowRoute && <SettingsDialog />}" in root assert "{!isAuthFlowRoute && <SettingsDialog />}" in root
assert "useSettingsDialogStore.getState().closeDialog();" in root assert "useSettingsDialogStore.getState().closeDialog();" in root
assert "if (isAuthFlowRoute) return;" in root assert "if (isAuthFlowRoute) return;" in root
for route in ("login", "change-password", "onboarding"): for route in ("login", "change-password", "onboarding"):
assert "isAuthFlow: true" in (FRONTEND / f"app/routes/{route}.tsx").read_text() assert "isAuthFlow: true" in (FRONTEND / f"app/routes/{route}.tsx").read_text(
encoding = "utf-8"
)
def test_auth_redirect_targets_are_idempotent_and_concurrent(tmp_path: Path): def test_auth_redirect_targets_are_idempotent_and_concurrent(tmp_path: Path):
@ -190,7 +192,7 @@ def test_auth_redirect_targets_are_idempotent_and_concurrent(tmp_path: Path):
pytest.skip("node --experimental-strip-types not available") pytest.skip("node --experimental-strip-types not available")
source = ( source = (
AUTH_API.read_text() AUTH_API.read_text(encoding = "utf-8")
.replace('from "@/lib/api-base"', 'from "./stubs.mjs"') .replace('from "@/lib/api-base"', 'from "./stubs.mjs"')
.replace('from "./session"', 'from "./stubs.mjs"') .replace('from "./session"', 'from "./stubs.mjs"')
) )

2
tests/studio/test_cancel_atomicity.py
View file

@ -9,7 +9,7 @@ from pathlib import Path
SOURCE_PATH = Path(__file__).resolve().parents[2] / "studio" / "backend" / "routes" / "inference.py" SOURCE_PATH = Path(__file__).resolve().parents[2] / "studio" / "backend" / "routes" / "inference.py"
_SRC = SOURCE_PATH.read_text() _SRC = SOURCE_PATH.read_text(encoding = "utf-8")
_TREE = ast.parse(_SRC) _TREE = ast.parse(_SRC)

12
tests/studio/test_cancel_id_wiring.py
View file

@ -13,10 +13,14 @@ from pathlib import Path
WORKSPACE = Path(__file__).resolve().parents[2] WORKSPACE = Path(__file__).resolve().parents[2]
MODELS_SRC = (WORKSPACE / "studio/backend/models/inference.py").read_text() MODELS_SRC = (WORKSPACE / "studio/backend/models/inference.py").read_text(encoding = "utf-8")
ROUTES_SRC = (WORKSPACE / "studio/backend/routes/inference.py").read_text() ROUTES_SRC = (WORKSPACE / "studio/backend/routes/inference.py").read_text(encoding = "utf-8")
ADAPTER_SRC = (WORKSPACE / "studio/frontend/src/features/chat/api/chat-adapter.ts").read_text() ADAPTER_SRC = (WORKSPACE / "studio/frontend/src/features/chat/api/chat-adapter.ts").read_text(
API_TYPES_SRC = (WORKSPACE / "studio/frontend/src/features/chat/types/api.ts").read_text() encoding = "utf-8"
)
API_TYPES_SRC = (WORKSPACE / "studio/frontend/src/features/chat/types/api.ts").read_text(
encoding = "utf-8"
)
def _find_class(tree: ast.AST, name: str) -> ast.ClassDef | None: def _find_class(tree: ast.AST, name: str) -> ast.ClassDef | None:

8
tests/studio/test_chat_preset_builtin_invariants.py
View file

@ -40,13 +40,15 @@ def _require_node():
def _ensure_harness(): def _ensure_harness():
TEMP.mkdir(parents = True, exist_ok = True) TEMP.mkdir(parents = True, exist_ok = True)
(TEMP / "register.mjs").write_text( (TEMP / "register.mjs").write_text(
"import { register } from 'node:module';\nregister('./loader.mjs', import.meta.url);\n" "import { register } from 'node:module';\nregister('./loader.mjs', import.meta.url);\n",
encoding = "utf-8",
) )
(TEMP / "loader.mjs").write_text( (TEMP / "loader.mjs").write_text(
"export function resolve(specifier, context, next) {\n" "export function resolve(specifier, context, next) {\n"
" if (specifier.endsWith('/types/runtime')) return next(specifier + '.ts', context);\n" " if (specifier.endsWith('/types/runtime')) return next(specifier + '.ts', context);\n"
" return next(specifier, context);\n" " return next(specifier, context);\n"
"}\n" "}\n",
encoding = "utf-8",
) )
@ -54,7 +56,7 @@ def _run(script: str):
_require_node() _require_node()
_ensure_harness() _ensure_harness()
script_path = TEMP / "run.mts" script_path = TEMP / "run.mts"
script_path.write_text(script) script_path.write_text(script, encoding = "utf-8")
env = dict(os.environ, NODE_NO_WARNINGS = "1") env = dict(os.environ, NODE_NO_WARNINGS = "1")
result = subprocess.run( result = subprocess.run(
[ [

4
tests/studio/test_chat_prompt_variables.py
View file

@ -6,7 +6,9 @@ from pathlib import Path
WORKSPACE = Path(__file__).resolve().parents[2] WORKSPACE = Path(__file__).resolve().parents[2]
ADAPTER_SRC = (WORKSPACE / "studio/frontend/src/features/chat/api/chat-adapter.ts").read_text() ADAPTER_SRC = (WORKSPACE / "studio/frontend/src/features/chat/api/chat-adapter.ts").read_text(
encoding = "utf-8"
)
def _function_source(name: str) -> str: def _function_source(name: str) -> str:

20
tests/studio/test_chat_response_details_ui_contract.py
View file

@ -20,14 +20,14 @@ CHAT_TAB_TSX = REPO / "studio/frontend/src/features/settings/tabs/chat-tab.tsx"
def test_assistant_more_menu_exposes_response_details_action(): def test_assistant_more_menu_exposes_response_details_action():
src = THREAD_TSX.read_text() src = THREAD_TSX.read_text(encoding = "utf-8")
assert "MessageResponseDetailsSheet" in src assert "MessageResponseDetailsSheet" in src
assert "See response details" in src assert "See response details" in src
assert "setDetailsOpen(true)" in src assert "setDetailsOpen(true)" in src
def test_response_details_sheet_uses_unsloth_sheet_and_key_sections(): def test_response_details_sheet_uses_unsloth_sheet_and_key_sections():
src = DETAILS_TSX.read_text() src = DETAILS_TSX.read_text(encoding = "utf-8")
assert "SheetContent" in src assert "SheetContent" in src
assert "Response details" in src assert "Response details" in src
assert "MessageResponseModelBadge" in src assert "MessageResponseModelBadge" in src
@ -45,17 +45,17 @@ def test_response_details_sheet_uses_unsloth_sheet_and_key_sections():
def test_response_model_badge_is_user_configurable_and_rendered_once_per_message(): def test_response_model_badge_is_user_configurable_and_rendered_once_per_message():
prefs_src = CHAT_PREFS_TS.read_text() prefs_src = CHAT_PREFS_TS.read_text(encoding = "utf-8")
chat_tab_src = CHAT_TAB_TSX.read_text() chat_tab_src = CHAT_TAB_TSX.read_text(encoding = "utf-8")
thread_src = THREAD_TSX.read_text() thread_src = THREAD_TSX.read_text(encoding = "utf-8")
reasoning_src = REASONING_TSX.read_text() reasoning_src = REASONING_TSX.read_text(encoding = "utf-8")
assert "showResponseModel: boolean" in prefs_src assert "showResponseModel: boolean" in prefs_src
assert "showResponseModel: false" in prefs_src assert "showResponseModel: false" in prefs_src
assert "showResponseModel: saved?.showResponseModel ?? false" in prefs_src assert "showResponseModel: saved?.showResponseModel ?? false" in prefs_src
assert "Show response model" in chat_tab_src assert "Show response model" in chat_tab_src
assert "setShowResponseModel" in chat_tab_src assert "setShowResponseModel" in chat_tab_src
details_src = DETAILS_TSX.read_text() details_src = DETAILS_TSX.read_text(encoding = "utf-8")
assert ( assert (
"aui-response-model-badge pointer-events-none relative inline-flex min-h-5" in details_src "aui-response-model-badge pointer-events-none relative inline-flex min-h-5" in details_src
) )
@ -76,7 +76,7 @@ def test_response_model_badge_is_user_configurable_and_rendered_once_per_message
def test_reasoning_keeps_streaming_height_cap_through_automatic_collapse(): def test_reasoning_keeps_streaming_height_cap_through_automatic_collapse():
src = REASONING_TSX.read_text() src = REASONING_TSX.read_text(encoding = "utf-8")
assert "const [retainStreamingHeight, setRetainStreamingHeight]" in src assert "const [retainStreamingHeight, setRetainStreamingHeight]" in src
assert "setRetainStreamingHeight(false)" in src assert "setRetainStreamingHeight(false)" in src
@ -91,7 +91,7 @@ def test_reasoning_clears_manual_open_on_a_new_stream():
isOpen is `(streaming && !dismissed) || manualOpen` and manualOpen is only isOpen is `(streaming && !dismissed) || manualOpen` and manualOpen is only
settable while idle, so the new-stream reset has to clear it too. settable while idle, so the new-stream reset has to clear it too.
""" """
src = REASONING_TSX.read_text() src = REASONING_TSX.read_text(encoding = "utf-8")
marker = "setDismissedWhileStreaming(false)" marker = "setDismissedWhileStreaming(false)"
start = src.find(marker) start = src.find(marker)
@ -101,7 +101,7 @@ def test_reasoning_clears_manual_open_on_a_new_stream():
def test_response_details_metadata_is_persisted_without_backend_schema_change(): def test_response_details_metadata_is_persisted_without_backend_schema_change():
src = ADAPTER_TS.read_text() src = ADAPTER_TS.read_text(encoding = "utf-8")
assert "interface ResponseDetailsMetadata" in src assert "interface ResponseDetailsMetadata" in src
assert "buildResponseDetails" in src assert "buildResponseDetails" in src
assert "responseDetails: buildResponseDetails(finishedAt)" in src assert "responseDetails: buildResponseDetails(finishedAt)" in src

14
tests/studio/test_chat_title_generation.py
View file

@ -41,7 +41,7 @@ def _balanced_block(src: str, anchor: str) -> str:
def test_title_model_prompt_targets_conversation_topic(): def test_title_model_prompt_targets_conversation_topic():
block = _source_until( block = _source_until(
RUNTIME_TSX.read_text(), RUNTIME_TSX.read_text(encoding = "utf-8"),
"async function generateTitleWithModel", "async function generateTitleWithModel",
"\nconst inflightTitleByKey", "\nconst inflightTitleByKey",
) )
@ -54,7 +54,7 @@ def test_title_model_prompt_targets_conversation_topic():
def test_title_model_payload_includes_optional_assistant_reply(): def test_title_model_payload_includes_optional_assistant_reply():
block = _source_until( block = _source_until(
RUNTIME_TSX.read_text(), RUNTIME_TSX.read_text(encoding = "utf-8"),
"async function generateTitleWithModel", "async function generateTitleWithModel",
"\nconst inflightTitleByKey", "\nconst inflightTitleByKey",
) )
@ -71,7 +71,7 @@ def test_title_model_payload_includes_optional_assistant_reply():
def test_generate_title_passes_first_assistant_reply_after_first_user(): def test_generate_title_passes_first_assistant_reply_after_first_user():
block = _balanced_block( block = _balanced_block(
RUNTIME_TSX.read_text(), RUNTIME_TSX.read_text(encoding = "utf-8"),
"async generateTitle(remoteId", "async generateTitle(remoteId",
) )
@ -84,7 +84,7 @@ def test_generate_title_passes_first_assistant_reply_after_first_user():
def test_tool_call_only_first_assistant_still_uses_first_user_message(): def test_tool_call_only_first_assistant_still_uses_first_user_message():
source = RUNTIME_TSX.read_text() source = RUNTIME_TSX.read_text(encoding = "utf-8")
extract_block = " ".join(_balanced_block(source, "function extractTextParts").split()) extract_block = " ".join(_balanced_block(source, "function extractTextParts").split())
generate_block = " ".join(_balanced_block(source, "async generateTitle(remoteId").split()) generate_block = " ".join(_balanced_block(source, "async generateTitle(remoteId").split())
@ -104,7 +104,7 @@ def test_tool_call_only_first_assistant_still_uses_first_user_message():
def test_auto_title_disabled_uses_deterministic_user_text_fallback(): def test_auto_title_disabled_uses_deterministic_user_text_fallback():
block = _balanced_block( block = _balanced_block(
RUNTIME_TSX.read_text(), RUNTIME_TSX.read_text(encoding = "utf-8"),
"async generateTitle(remoteId", "async generateTitle(remoteId",
) )
auto_title_off = _balanced_block(block, "if (!autoTitle)") auto_title_off = _balanced_block(block, "if (!autoTitle)")
@ -114,7 +114,7 @@ def test_auto_title_disabled_uses_deterministic_user_text_fallback():
def test_model_failure_still_falls_back_to_user_text(): def test_model_failure_still_falls_back_to_user_text():
source = RUNTIME_TSX.read_text() source = RUNTIME_TSX.read_text(encoding = "utf-8")
model_block = _source_until( model_block = _source_until(
source, source,
"async function generateTitleWithModel", "async function generateTitleWithModel",
@ -130,7 +130,7 @@ def test_model_failure_still_falls_back_to_user_text():
def test_title_normalizer_still_enforces_output_constraints(): def test_title_normalizer_still_enforces_output_constraints():
block = _source_until( block = _source_until(
RUNTIME_TSX.read_text(), RUNTIME_TSX.read_text(encoding = "utf-8"),
"async function generateTitleWithModel", "async function generateTitleWithModel",
"\nconst inflightTitleByKey", "\nconst inflightTitleByKey",
) )

4
tests/studio/test_cli_run_alias.py
View file

@ -17,7 +17,7 @@ def _module_calls(source: str):
def test_top_level_run_alias_registered(): def test_top_level_run_alias_registered():
"""`app.command("run", ...)` must be invoked with studio_run as its target.""" """`app.command("run", ...)` must be invoked with studio_run as its target."""
source = _CLI_INIT.read_text() source = _CLI_INIT.read_text(encoding = "utf-8")
# Find ``app.command("run", ...)`` call -- the decorator-call form. # Find ``app.command("run", ...)`` call -- the decorator-call form.
found_decorator_call = False found_decorator_call = False
@ -46,7 +46,7 @@ def test_top_level_run_alias_registered():
def test_studio_run_imported_for_alias(): def test_studio_run_imported_for_alias():
"""The alias must wire up to the studio.run function, not redefine it.""" """The alias must wire up to the studio.run function, not redefine it."""
source = _CLI_INIT.read_text() source = _CLI_INIT.read_text(encoding = "utf-8")
tree = ast.parse(source) tree = ast.parse(source)
has_import = False has_import = False
for node in ast.walk(tree): for node in ast.walk(tree):

13
tests/studio/test_cli_studio_defaults.py
View file

@ -48,20 +48,19 @@ def _find_typer_option_default(source: str, func_name: str, long_option: str):
def test_studio_default_host_is_loopback(): def test_studio_default_host_is_loopback():
"""`unsloth studio` (studio_default) --host default must be 127.0.0.1.""" """`unsloth studio` (studio_default) --host default must be 127.0.0.1."""
source = _STUDIO_CMD_PY.read_text() source = _STUDIO_CMD_PY.read_text(encoding = "utf-8")
host_default = _find_typer_option_default(source, "studio_default", "--host") host_default = _find_typer_option_default(source, "studio_default", "--host")
assert ( assert (
host_default is not None host_default is not None
), "Could not find --host typer.Option default in studio_default()" ), "Could not find --host typer.Option default in studio_default()"
assert host_default == "127.0.0.1", ( assert (
f"studio_default() --host default must be '127.0.0.1' (loopback) " host_default == "127.0.0.1"
f"but got '{host_default}'." ), f"studio_default() --host default must be '127.0.0.1' (loopback) but got '{host_default}'."
)
def test_studio_run_host_is_loopback(): def test_studio_run_host_is_loopback():
"""`unsloth studio run` --host default must be 127.0.0.1.""" """`unsloth studio run` --host default must be 127.0.0.1."""
source = _STUDIO_CMD_PY.read_text() source = _STUDIO_CMD_PY.read_text(encoding = "utf-8")
host_default = _find_typer_option_default(source, "run", "--host") host_default = _find_typer_option_default(source, "run", "--host")
assert host_default is not None, "Could not find --host typer.Option default in run()" assert host_default is not None, "Could not find --host typer.Option default in run()"
assert host_default == "127.0.0.1", ( assert host_default == "127.0.0.1", (
@ -71,7 +70,7 @@ def test_studio_run_host_is_loopback():
def test_dns_pinning_opt_out_is_registered_safe_by_default(): def test_dns_pinning_opt_out_is_registered_safe_by_default():
source = _STUDIO_CMD_PY.read_text() source = _STUDIO_CMD_PY.read_text(encoding = "utf-8")
for func_name in ("studio_default", "run"): for func_name in ("studio_default", "run"):
default = _find_typer_option_default(source, func_name, "--disable-dns-pinning") default = _find_typer_option_default(source, func_name, "--disable-dns-pinning")
assert default is False, f"{func_name} must keep DNS pinning enabled by default" assert default is False, f"{func_name} must keep DNS pinning enabled by default"

42
tests/studio/test_composer_rtl_bidi_attribute.py
View file

@ -26,22 +26,22 @@ def _block_around(
def test_main_composer_has_dir_auto(): def test_main_composer_has_dir_auto():
# PR #5784 turned the attribute into a JSX conditional; anchor on the inner # PR #5784 turned the attribute into a JSX conditional; anchor on the inner
# "Message input" literal, which survives both spellings. # "Message input" literal, which survives both spellings.
block = _block_around(THREAD_TSX.read_text(), '"Message input"') block = _block_around(THREAD_TSX.read_text(encoding = "utf-8"), '"Message input"')
assert 'dir="auto"' in block, 'main composer is missing dir="auto"' assert 'dir="auto"' in block, 'main composer is missing dir="auto"'
def test_edit_composer_has_dir_auto(): def test_edit_composer_has_dir_auto():
block = _block_around(THREAD_TSX.read_text(), "aui-edit-composer-input") block = _block_around(THREAD_TSX.read_text(encoding = "utf-8"), "aui-edit-composer-input")
assert 'dir="auto"' in block, 'edit composer is missing dir="auto"' assert 'dir="auto"' in block, 'edit composer is missing dir="auto"'
def test_compare_composer_has_dir_auto(): def test_compare_composer_has_dir_auto():
block = _block_around(SHARED_TSX.read_text(), "Send to both models") block = _block_around(SHARED_TSX.read_text(encoding = "utf-8"), "Send to both models")
assert 'dir="auto"' in block, 'compare composer is missing dir="auto"' assert 'dir="auto"' in block, 'compare composer is missing dir="auto"'
def test_ime_workflow_step_does_not_set_studio_old_pw(): def test_ime_workflow_step_does_not_set_studio_old_pw():
yml = WORKFLOW_YML.read_text() yml = WORKFLOW_YML.read_text(encoding = "utf-8")
drive_idx = yml.find("Drive IME + multilingual paste regression") drive_idx = yml.find("Drive IME + multilingual paste regression")
assert drive_idx != -1, "IME drive step not found in workflow" assert drive_idx != -1, "IME drive step not found in workflow"
next_step_idx = yml.find("- name:", drive_idx + 1) next_step_idx = yml.find("- name:", drive_idx + 1)
@ -53,7 +53,7 @@ def test_ime_workflow_step_does_not_set_studio_old_pw():
def test_ime_pass_password_step_does_not_export_old_pw(): def test_ime_pass_password_step_does_not_export_old_pw():
yml = WORKFLOW_YML.read_text() yml = WORKFLOW_YML.read_text(encoding = "utf-8")
pass_idx = yml.find("Pass bootstrap pw for IME / i18n test") pass_idx = yml.find("Pass bootstrap pw for IME / i18n test")
assert pass_idx != -1, "IME password setup step not found" assert pass_idx != -1, "IME password setup step not found"
next_step_idx = yml.find("- name:", pass_idx + 1) next_step_idx = yml.find("- name:", pass_idx + 1)
@ -65,7 +65,7 @@ def test_ime_pass_password_step_does_not_export_old_pw():
def test_ime_playwright_script_does_not_read_studio_old_pw(): def test_ime_playwright_script_does_not_read_studio_old_pw():
src = IME_PY.read_text() src = IME_PY.read_text(encoding = "utf-8")
code_only = re.sub(r'""".*?"""', "", src, flags = re.DOTALL) code_only = re.sub(r'""".*?"""', "", src, flags = re.DOTALL)
assert ( assert (
"STUDIO_OLD_PW" not in code_only "STUDIO_OLD_PW" not in code_only
@ -76,7 +76,7 @@ def test_ime_playwright_script_does_not_read_studio_old_pw():
def test_main_composer_has_stuck_compositionend_watchdog(): def test_main_composer_has_stuck_compositionend_watchdog():
"""Issue #5546: WSL Chrome never emits compositionend after IME commit, so the """Issue #5546: WSL Chrome never emits compositionend after IME commit, so the
composer needs a watchdog releasing the composing flag or Send stays disabled.""" composer needs a watchdog releasing the composing flag or Send stays disabled."""
src = THREAD_TSX.read_text() src = THREAD_TSX.read_text(encoding = "utf-8")
assert ( assert (
"IME_STUCK_TIMEOUT_MS" in src "IME_STUCK_TIMEOUT_MS" in src
), "main composer is missing the stuck-compositionend watchdog (issue #5546)" ), "main composer is missing the stuck-compositionend watchdog (issue #5546)"
@ -87,7 +87,7 @@ def test_main_composer_has_stuck_compositionend_watchdog():
def test_compare_composer_has_stuck_compositionend_watchdog(): def test_compare_composer_has_stuck_compositionend_watchdog():
src = SHARED_TSX.read_text() src = SHARED_TSX.read_text(encoding = "utf-8")
assert ( assert (
"IME_STUCK_TIMEOUT_MS" in src "IME_STUCK_TIMEOUT_MS" in src
), "compare composer is missing the stuck-compositionend watchdog (issue #5546)" ), "compare composer is missing the stuck-compositionend watchdog (issue #5546)"
@ -97,7 +97,7 @@ def test_compare_composer_has_stuck_compositionend_watchdog():
def test_main_composer_keydown_repins_composing_during_ime(): def test_main_composer_keydown_repins_composing_during_ime():
"""Issue #5546: the keydown IME gate must re-pin composingRef so a follow-up """Issue #5546: the keydown IME gate must re-pin composingRef so a follow-up
Enter does not submit preedit text after the watchdog clears it.""" Enter does not submit preedit text after the watchdog clears it."""
src = THREAD_TSX.read_text() src = THREAD_TSX.read_text(encoding = "utf-8")
assert "onKeyDown" in src, "main composer is missing onKeyDown IME gate" assert "onKeyDown" in src, "main composer is missing onKeyDown IME gate"
assert "e.nativeEvent.isComposing" in src and "keyCode === 229" in src, ( assert "e.nativeEvent.isComposing" in src and "keyCode === 229" in src, (
"main composer keydown gate must check both nativeEvent.isComposing " "main composer keydown gate must check both nativeEvent.isComposing "
@ -108,7 +108,7 @@ def test_main_composer_keydown_repins_composing_during_ime():
def test_compare_composer_keydown_repins_composing_during_ime(): def test_compare_composer_keydown_repins_composing_during_ime():
"""Compare composer onKeyDown re-pins composingRef on IME keypress so a """Compare composer onKeyDown re-pins composingRef on IME keypress so a
follow-up click-Send during the watchdog window does not slip preedit text.""" follow-up click-Send during the watchdog window does not slip preedit text."""
src = SHARED_TSX.read_text() src = SHARED_TSX.read_text(encoding = "utf-8")
assert "composingRef.current = true" in src, ( assert "composingRef.current = true" in src, (
"compare composer keydown gate must re-pin composingRef when the " "compare composer keydown gate must re-pin composingRef when the "
"browser still considers the IME active" "browser still considers the IME active"
@ -142,7 +142,7 @@ def _extract_block(
def test_main_composer_keydown_rearms_watchdog(): def test_main_composer_keydown_rearms_watchdog():
"""After keydown re-pins composingRef the watchdog must re-arm, else the """After keydown re-pins composingRef the watchdog must re-arm, else the
WSL+Chrome no-compositionend path locks Send after any IME keypress (#5546).""" WSL+Chrome no-compositionend path locks Send after any IME keypress (#5546)."""
src = THREAD_TSX.read_text() src = THREAD_TSX.read_text(encoding = "utf-8")
block = _extract_block(src, "const onKeyDown = useCallback") block = _extract_block(src, "const onKeyDown = useCallback")
assert "refreshStuckTimer" in block, ( assert "refreshStuckTimer" in block, (
"main composer keydown gate must call refreshStuckTimer after " "main composer keydown gate must call refreshStuckTimer after "
@ -159,12 +159,11 @@ def test_main_composer_keydown_rearms_watchdog():
def test_compare_composer_keydown_rearms_watchdog(): def test_compare_composer_keydown_rearms_watchdog():
"""Same re-arm contract for the compare-mode composer.""" """Same re-arm contract for the compare-mode composer."""
src = SHARED_TSX.read_text() src = SHARED_TSX.read_text(encoding = "utf-8")
block = _extract_block(src, "function onKeyDown", opener = "{", closer = "}") block = _extract_block(src, "function onKeyDown", opener = "{", closer = "}")
assert "refreshStuckImeTimer" in block, ( assert (
"compare composer keydown gate must call refreshStuckImeTimer " "refreshStuckImeTimer" in block
"after re-pinning composingRef" ), "compare composer keydown gate must call refreshStuckImeTimer after re-pinning composingRef"
)
def _assert_enter_guard_before_immediate_recovery(block: str, refresh_call: str) -> None: def _assert_enter_guard_before_immediate_recovery(block: str, refresh_call: str) -> None:
@ -177,10 +176,9 @@ def _assert_enter_guard_before_immediate_recovery(block: str, refresh_call: str)
"composingRef; candidate-confirming Enter must not submit" "composingRef; candidate-confirming Enter must not submit"
) )
guard_block = block[enter_idx:recovery_idx] guard_block = block[enter_idx:recovery_idx]
assert "preventDefault()" in guard_block, ( assert (
"Enter while composingRef is stuck must prevent the same key from " "preventDefault()" in guard_block
"falling through to submit" ), "Enter while composingRef is stuck must prevent the same key from falling through to submit"
)
assert ( assert (
refresh_call in guard_block refresh_call in guard_block
), "Enter while composingRef is stuck must keep the watchdog armed" ), "Enter while composingRef is stuck must keep the watchdog armed"
@ -190,12 +188,12 @@ def _assert_enter_guard_before_immediate_recovery(block: str, refresh_call: str)
def test_main_composer_stuck_enter_does_not_clear_before_submit(): def test_main_composer_stuck_enter_does_not_clear_before_submit():
src = THREAD_TSX.read_text() src = THREAD_TSX.read_text(encoding = "utf-8")
block = _extract_block(src, "const onKeyDown = useCallback") block = _extract_block(src, "const onKeyDown = useCallback")
_assert_enter_guard_before_immediate_recovery(block, "refreshStuckTimer") _assert_enter_guard_before_immediate_recovery(block, "refreshStuckTimer")
def test_compare_composer_stuck_enter_does_not_clear_before_submit(): def test_compare_composer_stuck_enter_does_not_clear_before_submit():
src = SHARED_TSX.read_text() src = SHARED_TSX.read_text(encoding = "utf-8")
block = _extract_block(src, "function onKeyDown", opener = "{", closer = "}") block = _extract_block(src, "function onKeyDown", opener = "{", closer = "}")
_assert_enter_guard_before_immediate_recovery(block, "refreshStuckImeTimer") _assert_enter_guard_before_immediate_recovery(block, "refreshStuckImeTimer")

12
tests/studio/test_export_output_path_contract.py
View file

@ -32,7 +32,7 @@ def _return_tuple_arity(fn):
def test_export_methods_return_three_tuple_annotation(): def test_export_methods_return_three_tuple_annotation():
tree = ast.parse(EXPORT.read_text()) tree = ast.parse(EXPORT.read_text(encoding = "utf-8"))
for fn_name in EXPORT_FNS: for fn_name in EXPORT_FNS:
fn = _find_method(tree, "ExportBackend", fn_name) fn = _find_method(tree, "ExportBackend", fn_name)
assert fn is not None, f"missing ExportBackend.{fn_name}" assert fn is not None, f"missing ExportBackend.{fn_name}"
@ -46,7 +46,7 @@ def test_export_methods_return_three_tuple_annotation():
def test_export_methods_return_three_element_tuples(): def test_export_methods_return_three_element_tuples():
tree = ast.parse(EXPORT.read_text()) tree = ast.parse(EXPORT.read_text(encoding = "utf-8"))
for fn_name in EXPORT_FNS: for fn_name in EXPORT_FNS:
fn = _find_method(tree, "ExportBackend", fn_name) fn = _find_method(tree, "ExportBackend", fn_name)
assert fn is not None assert fn is not None
@ -57,7 +57,7 @@ def test_export_methods_return_three_element_tuples():
def test_local_save_assigns_output_path(): def test_local_save_assigns_output_path():
tree = ast.parse(EXPORT.read_text()) tree = ast.parse(EXPORT.read_text(encoding = "utf-8"))
for fn_name in EXPORT_FNS: for fn_name in EXPORT_FNS:
fn = _find_method(tree, "ExportBackend", fn_name) fn = _find_method(tree, "ExportBackend", fn_name)
assert fn is not None assert fn is not None
@ -74,7 +74,7 @@ def test_local_save_assigns_output_path():
def test_gpu_save_method_bound_for_hub_only(): def test_gpu_save_method_bound_for_hub_only():
tree = ast.parse(EXPORT.read_text()) tree = ast.parse(EXPORT.read_text(encoding = "utf-8"))
fn = _find_method(tree, "ExportBackend", "export_merged_model") fn = _find_method(tree, "ExportBackend", "export_merged_model")
assert fn is not None assert fn is not None
found_pre_save_method = False found_pre_save_method = False
@ -103,7 +103,7 @@ def test_gpu_save_method_bound_for_hub_only():
def test_mlx_hub_only_uses_temp_directory(): def test_mlx_hub_only_uses_temp_directory():
src = EXPORT.read_text() src = EXPORT.read_text(encoding = "utf-8")
assert ( assert (
src.count("tempfile.TemporaryDirectory") >= 3 src.count("tempfile.TemporaryDirectory") >= 3
), "expected TemporaryDirectory in merged, base, and lora hub-push paths" ), "expected TemporaryDirectory in merged, base, and lora hub-push paths"
@ -111,7 +111,7 @@ def test_mlx_hub_only_uses_temp_directory():
def test_is_mlx_imported_from_unsloth(): def test_is_mlx_imported_from_unsloth():
src = EXPORT.read_text() src = EXPORT.read_text(encoding = "utf-8")
assert "from unsloth import" in src assert "from unsloth import" in src
head = src.split("class ExportBackend")[0] head = src.split("class ExportBackend")[0]
assert "_IS_MLX" in head assert "_IS_MLX" in head

18
tests/studio/test_frontend_dep_removal.py
View file

@ -53,8 +53,7 @@ CASES: list[Case] = [
), ),
Case( Case(
"C3", "C3",
"removing katex is safe: streamdown/math, mermaid, " "removing katex is safe: streamdown/math, mermaid, rehype-katex all keep it at top level",
"rehype-katex all keep it at top level",
["katex"], ["katex"],
"PASS", "PASS",
[], [],
@ -69,8 +68,7 @@ CASES: list[Case] = [
), ),
Case( Case(
"C6", "C6",
"removing @radix-ui/react-slot is safe: pulled by " "removing @radix-ui/react-slot is safe: pulled by radix-ui umbrella + @assistant-ui/react",
"radix-ui umbrella + @assistant-ui/react",
["@radix-ui/react-slot"], ["@radix-ui/react-slot"],
"PASS", "PASS",
[], [],
@ -852,7 +850,7 @@ ADV_CASES: list[AdvCase] = [
"A12", "A12",
"JSDoc @import of removed pkg should FAIL", "JSDoc @import of removed pkg should FAIL",
"adv12.ts", "adv12.ts",
'/** @type {import("__adv_only_pkg_l__").Foo} */\n' "const x = null;\n", '/** @type {import("__adv_only_pkg_l__").Foo} */\nconst x = null;\n',
"__adv_only_pkg_l__", "__adv_only_pkg_l__",
"FAIL", "FAIL",
["__adv_only_pkg_l__"], ["__adv_only_pkg_l__"],
@ -1047,7 +1045,7 @@ PKG_FIELD_CASES: list[PkgFieldCase] = [
def run_pkg_field_cases() -> int: def run_pkg_field_cases() -> int:
head_pkg = json.loads(HEAD_PKG.read_text()) head_pkg = json.loads(HEAD_PKG.read_text(encoding = "utf-8"))
passed = 0 passed = 0
for pc in PKG_FIELD_CASES: for pc in PKG_FIELD_CASES:
synth_head = json.loads(json.dumps(head_pkg)) synth_head = json.loads(json.dumps(head_pkg))
@ -1110,13 +1108,13 @@ def run_pkg_field_cases() -> int:
def run_adversarial_cases() -> int: def run_adversarial_cases() -> int:
ADVERSARIAL_TMP_DIR.mkdir(parents = True, exist_ok = True) ADVERSARIAL_TMP_DIR.mkdir(parents = True, exist_ok = True)
head_pkg = json.loads(HEAD_PKG.read_text()) head_pkg = json.loads(HEAD_PKG.read_text(encoding = "utf-8"))
passed = 0 passed = 0
for ac in ADV_CASES: for ac in ADV_CASES:
# Drop the synthetic file. # Drop the synthetic file.
fpath = ADVERSARIAL_TMP_DIR / ac.filename fpath = ADVERSARIAL_TMP_DIR / ac.filename
try: try:
fpath.write_text(ac.content) fpath.write_text(ac.content, encoding = "utf-8")
# Base adds the target pkg; real head lacks it, so the script # Base adds the target pkg; real head lacks it, so the script
# treats it as removed and scans the repo (now with our file). # treats it as removed and scans the repo (now with our file).
synth_base = json.loads(json.dumps(head_pkg)) synth_base = json.loads(json.dumps(head_pkg))
@ -1259,7 +1257,7 @@ ENUM_CASES: list[EnumCase] = [
def run_enum_cases() -> int: def run_enum_cases() -> int:
head_pkg = json.loads(HEAD_PKG.read_text()) head_pkg = json.loads(HEAD_PKG.read_text(encoding = "utf-8"))
passed = 0 passed = 0
ADVERSARIAL_TMP_DIR.mkdir(parents = True, exist_ok = True) ADVERSARIAL_TMP_DIR.mkdir(parents = True, exist_ok = True)
for ec in ENUM_CASES: for ec in ENUM_CASES:
@ -1508,7 +1506,7 @@ def run_wrapper_cases() -> int:
def main() -> int: def main() -> int:
head_pkg = json.loads(HEAD_PKG.read_text()) head_pkg = json.loads(HEAD_PKG.read_text(encoding = "utf-8"))
print(f"Running {len(CASES)} edge cases against {SCRIPT.relative_to(REPO)}") print(f"Running {len(CASES)} edge cases against {SCRIPT.relative_to(REPO)}")
print() print()
results: list[tuple[Case, bool, str]] = [] results: list[tuple[Case, bool, str]] = []

2
tests/studio/test_is_mlx_dispatch_gate.py
View file

@ -27,7 +27,7 @@ UNSLOTH_INIT = REPO_ROOT / "unsloth" / "__init__.py"
def test_is_mlx_gate_uses_three_required_predicates(): def test_is_mlx_gate_uses_three_required_predicates():
"""_IS_MLX must AND Darwin+arm64+importable-mlx; dropping any breaks dispatch.""" """_IS_MLX must AND Darwin+arm64+importable-mlx; dropping any breaks dispatch."""
tree = ast.parse(UNSLOTH_INIT.read_text()) tree = ast.parse(UNSLOTH_INIT.read_text(encoding = "utf-8"))
target = None target = None
for node in ast.walk(tree): for node in ast.walk(tree):

2
tests/studio/test_llama_cpp_wall_clock_cap.py
View file

@ -14,7 +14,7 @@ SOURCE_PATH = (
/ "inference" / "inference"
/ "llama_cpp.py" / "llama_cpp.py"
) )
SRC = SOURCE_PATH.read_text() SRC = SOURCE_PATH.read_text(encoding = "utf-8")
TREE = ast.parse(SRC) TREE = ast.parse(SRC)

14
tests/studio/test_mlx_training_worker_behaviors.py
View file

@ -15,7 +15,7 @@ def _find_func(tree, name):
def test_run_mlx_training_passes_token_to_from_pretrained(): def test_run_mlx_training_passes_token_to_from_pretrained():
tree = ast.parse(WORKER.read_text()) tree = ast.parse(WORKER.read_text(encoding = "utf-8"))
fn = _find_func(tree, "_run_mlx_training") fn = _find_func(tree, "_run_mlx_training")
assert fn is not None assert fn is not None
found = False found = False
@ -36,7 +36,7 @@ def test_run_mlx_training_passes_token_to_from_pretrained():
def test_wandb_init_strips_secret_keys(): def test_wandb_init_strips_secret_keys():
src = WORKER.read_text() src = WORKER.read_text(encoding = "utf-8")
assert "_wandb_sensitive" in src, "expected a sensitive-key set near wandb.init" assert "_wandb_sensitive" in src, "expected a sensitive-key set near wandb.init"
assert '"hf_token"' in src and '"wandb_token"' in src assert '"hf_token"' in src and '"wandb_token"' in src
assert ( assert (
@ -45,26 +45,26 @@ def test_wandb_init_strips_secret_keys():
def test_local_dataset_loader_uses_load_dataset_path(): def test_local_dataset_loader_uses_load_dataset_path():
src = WORKER.read_text() src = WORKER.read_text(encoding = "utf-8")
assert "_resolve_mlx_local_dataset_files" in src assert "_resolve_mlx_local_dataset_files" in src
assert "_mlx_local_dataset_loader_for_files" in src assert "_mlx_local_dataset_loader_for_files" in src
assert "data_files = all_files" in src or "data_files=all_files" in src assert "data_files = all_files" in src or "data_files=all_files" in src
def test_send_aliases_status_message_to_message(): def test_send_aliases_status_message_to_message():
src = WORKER.read_text() src = WORKER.read_text(encoding = "utf-8")
assert 'kwargs["message"] = sm' in src or 'kwargs["message"]=sm' in src assert 'kwargs["message"] = sm' in src or 'kwargs["message"]=sm' in src
def test_slice_uses_inclusive_end_and_handles_zero(): def test_slice_uses_inclusive_end_and_handles_zero():
src = WORKER.read_text() src = WORKER.read_text(encoding = "utf-8")
assert "min(end + 1, len(ds))" in src or "min(end+1, len(ds))" in src assert "min(end + 1, len(ds))" in src or "min(end+1, len(ds))" in src
assert "slice_start if slice_start is not None else 0" in src assert "slice_start if slice_start is not None else 0" in src
assert "slice_end if slice_end is not None else len(ds) - 1" in src assert "slice_end if slice_end is not None else len(ds) - 1" in src
def test_poll_stop_returns_on_broken_pipe(): def test_poll_stop_returns_on_broken_pipe():
src = WORKER.read_text() src = WORKER.read_text(encoding = "utf-8")
assert "except (EOFError, OSError)" in src assert "except (EOFError, OSError)" in src
lines = src.splitlines() lines = src.splitlines()
for i, line in enumerate(lines): for i, line in enumerate(lines):
@ -83,7 +83,7 @@ def test_poll_stop_returns_on_broken_pipe():
def test_unsloth_zoo_mlx_imports_have_friendly_error(): def test_unsloth_zoo_mlx_imports_have_friendly_error():
src = WORKER.read_text() src = WORKER.read_text(encoding = "utf-8")
assert "from unsloth_zoo.mlx.loader import FastMLXModel" in src assert "from unsloth_zoo.mlx.loader import FastMLXModel" in src
assert "from unsloth_zoo.mlx.trainer import" in src assert "from unsloth_zoo.mlx.trainer import" in src
assert "raise ImportError" in src assert "raise ImportError" in src

2
tests/studio/test_model_picker_contracts.py
View file

@ -23,7 +23,7 @@ FRONTEND = WORKDIR / "studio" / "frontend" / "src"
def _read(rel: str) -> str: def _read(rel: str) -> str:
path = FRONTEND / rel path = FRONTEND / rel
assert path.exists(), f"missing source file: {path}" assert path.exists(), f"missing source file: {path}"
return path.read_text() return path.read_text(encoding = "utf-8")
def test_models_api_sends_token_via_header_not_query(): def test_models_api_sends_token_via_header_not_query():

2
tests/studio/test_studio_gguf_export_script_pin.py
View file

@ -12,7 +12,7 @@ from pathlib import Path
SOURCE_PATH = ( SOURCE_PATH = (
Path(__file__).resolve().parents[2] / "studio" / "backend" / "core" / "export" / "export.py" Path(__file__).resolve().parents[2] / "studio" / "backend" / "core" / "export" / "export.py"
) )
SRC = SOURCE_PATH.read_text() SRC = SOURCE_PATH.read_text(encoding = "utf-8")
TREE = ast.parse(SRC) TREE = ast.parse(SRC)

2
tests/studio/test_studio_text_descender_clipping.py
View file

@ -24,7 +24,7 @@ APP_SIDEBAR = WORKDIR / "studio" / "frontend" / "src" / "components" / "app-side
def _read(path: Path) -> str: def _read(path: Path) -> str:
assert path.exists(), f"missing source file: {path}" assert path.exists(), f"missing source file: {path}"
return path.read_text() return path.read_text(encoding = "utf-8")
def test_model_selector_trigger_label_uses_leading_tight(): def test_model_selector_trigger_label_uses_leading_tight():

2
tests/test_fast_generate_slow_guard.py
View file

@ -13,7 +13,7 @@ UTILS = os.path.join(HERE, "unsloth", "models", "_utils.py")
def _load_factory(): def _load_factory():
src = open(UTILS).read() src = open(UTILS, encoding = "utf-8").read()
for node in ast.parse(src).body: for node in ast.parse(src).body:
if isinstance(node, ast.FunctionDef) and node.name == "make_fast_generate_wrapper": if isinstance(node, ast.FunctionDef) and node.name == "make_fast_generate_wrapper":
ns = {"functools": functools} ns = {"functools": functools}

4
tests/test_fp8_device_context.py
View file

@ -78,7 +78,7 @@ class _LaunchVisitor(ast.NodeVisitor):
def _load_device_context_helper(fake_torch: _FakeTorch): def _load_device_context_helper(fake_torch: _FakeTorch):
source = FP8_SOURCE.read_text() source = FP8_SOURCE.read_text(encoding = "utf-8")
tree = ast.parse(source) tree = ast.parse(source)
for node in tree.body: for node in tree.body:
if isinstance(node, ast.FunctionDef) and node.name == "_fp8_triton_device_context": if isinstance(node, ast.FunctionDef) and node.name == "_fp8_triton_device_context":
@ -144,7 +144,7 @@ def test_fp8_device_context_is_noop_for_non_cuda_tensor() -> None:
def test_fp8_triton_launches_enter_tensor_device_context() -> None: def test_fp8_triton_launches_enter_tensor_device_context() -> None:
tree = ast.parse(FP8_SOURCE.read_text()) tree = ast.parse(FP8_SOURCE.read_text(encoding = "utf-8"))
function_names = {node.name for node in ast.walk(tree) if isinstance(node, ast.FunctionDef)} function_names = {node.name for node in ast.walk(tree) if isinstance(node, ast.FunctionDef)}
assert "_fp8_triton_device_context" in function_names assert "_fp8_triton_device_context" in function_names

2
tests/test_gemma4_chat_template.py
View file

@ -14,7 +14,7 @@ CHAT_TEMPLATES_PATH = os.path.join(
def _extract_template(name): def _extract_template(name):
src = open(CHAT_TEMPLATES_PATH).read() src = open(CHAT_TEMPLATES_PATH, encoding = "utf-8").read()
pattern = rf'{re.escape(name)}\s*=\s*\\\n"""(.*?)"""' pattern = rf'{re.escape(name)}\s*=\s*\\\n"""(.*?)"""'
m = re.search(pattern, src, flags = re.DOTALL) m = re.search(pattern, src, flags = re.DOTALL)
assert m, f"Could not extract {name} from chat_templates.py" assert m, f"Could not extract {name} from chat_templates.py"

2
tests/test_gemma_2b_mapper_key.py
View file

@ -18,7 +18,7 @@ MAPPER_PATH = os.path.join(os.path.dirname(__file__), os.pardir, "unsloth", "mod
def _load_mappers(): def _load_mappers():
with open(MAPPER_PATH) as f: with open(MAPPER_PATH, encoding = "utf-8") as f:
source = f.read() source = f.read()
namespace = {} namespace = {}
exec(compile(source, MAPPER_PATH, "exec"), namespace) exec(compile(source, MAPPER_PATH, "exec"), namespace)

2
tests/test_generate_kwarg_gate.py
View file

@ -9,7 +9,7 @@ VISION = os.path.join(HERE, "unsloth", "models", "vision.py")
def _load_helper(): def _load_helper():
src = open(VISION).read() src = open(VISION, encoding = "utf-8").read()
mod = ast.parse(src) mod = ast.parse(src)
for node in mod.body: for node in mod.body:
if isinstance(node, ast.FunctionDef) and node.name == "_unsloth_generate_accepts_kwarg": if isinstance(node, ast.FunctionDef) and node.name == "_unsloth_generate_accepts_kwarg":

6
tests/test_gradient_checkpointing_restore.py
View file

@ -28,8 +28,8 @@ import re
from pathlib import Path from pathlib import Path
_ROOT = Path(__file__).resolve().parent.parent / "unsloth" / "models" _ROOT = Path(__file__).resolve().parent.parent / "unsloth" / "models"
_RL = (_ROOT / "rl.py").read_text() _RL = (_ROOT / "rl.py").read_text(encoding = "utf-8")
_RL_REPLACEMENTS = (_ROOT / "rl_replacements.py").read_text() _RL_REPLACEMENTS = (_ROOT / "rl_replacements.py").read_text(encoding = "utf-8")
# The single-line ternary form used at the trainer call sites: # The single-line ternary form used at the trainer call sites:
# <obj>._unsloth_gradient_checkpointing if hasattr(<obj>, '...') else getattr(<args>, 'gradient_checkpointing', True) # <obj>._unsloth_gradient_checkpointing if hasattr(<obj>, '...') else getattr(<args>, 'gradient_checkpointing', True)
@ -162,7 +162,7 @@ def test_recording_sites_are_real_module_code():
# string. Assert it's present at the choke point (patch_peft_model, so loaded adapters # string. Assert it's present at the choke point (patch_peft_model, so loaded adapters
# are covered) and at the pre-wrapped pass-through, both of which bypass the old # are covered) and at the pre-wrapped pass-through, both of which bypass the old
# get_peft_model-only recording. # get_peft_model-only recording.
llama = (_ROOT / "llama.py").read_text() llama = (_ROOT / "llama.py").read_text(encoding = "utf-8")
tree = ast.parse(llama) tree = ast.parse(llama)
def assigns_marker(node): def assigns_marker(node):

2
tests/test_import_fixes_drift.py
View file

@ -704,7 +704,7 @@ def test_accelerate_find_device_skips_empty_logits():
def test_accelerate_patch_wired_into_gpu_init(): def test_accelerate_patch_wired_into_gpu_init():
"""The patch must be installed at startup, not only importable.""" """The patch must be installed at startup, not only importable."""
source = Path(__file__).resolve().parent.parent / "unsloth" / "_gpu_init.py" source = Path(__file__).resolve().parent.parent / "unsloth" / "_gpu_init.py"
source = source.read_text() source = source.read_text(encoding = "utf-8")
assert "patch_accelerate_recursively_apply()" in source, ( assert "patch_accelerate_recursively_apply()" in source, (
"DRIFT DETECTED: patch_accelerate_recursively_apply is defined but " "DRIFT DETECTED: patch_accelerate_recursively_apply is defined but "
"never called in _gpu_init.py, so real imports never install it." "never called in _gpu_init.py, so real imports never install it."

2
tests/test_loader_glob_skip.py
View file

@ -116,7 +116,7 @@ class TestLoaderSourceHasGuard(unittest.TestCase):
loader_path = os.path.join( loader_path = os.path.join(
os.path.dirname(__file__), os.pardir, "unsloth", "models", "loader.py" os.path.dirname(__file__), os.pardir, "unsloth", "models", "loader.py"
) )
with open(loader_path) as f: with open(loader_path, encoding = "utf-8") as f:
source = f.read() source = f.read()
lines = source.splitlines() lines = source.splitlines()

2
tests/test_multi_image_grpo_chunking.py
View file

@ -12,7 +12,7 @@ SOURCE_PATH = os.path.join(REPO_ROOT, "unsloth", "models", "rl_replacements.py")
def _read_source() -> str: def _read_source() -> str:
with open(SOURCE_PATH, "r") as fh: with open(SOURCE_PATH, "r", encoding = "utf-8") as fh:
return fh.read() return fh.read()

2
tests/test_offload_embedding_hooks.py
View file

@ -11,7 +11,7 @@ VISION = os.path.join(HERE, "unsloth", "models", "vision.py")
def _load_installer(): def _load_installer():
src = open(VISION).read() src = open(VISION, encoding = "utf-8").read()
mod = ast.parse(src) mod = ast.parse(src)
for node in mod.body: for node in mod.body:
if isinstance(node, ast.FunctionDef) and node.name == "_install_offload_embedding_hooks": if isinstance(node, ast.FunctionDef) and node.name == "_install_offload_embedding_hooks":

2
tests/test_offload_tied_guard.py
View file

@ -11,7 +11,7 @@ VISION = os.path.join(HERE, "unsloth", "models", "vision.py")
def _load_fn(): def _load_fn():
src = open(VISION).read() src = open(VISION, encoding = "utf-8").read()
mod = ast.parse(src) mod = ast.parse(src)
for node in mod.body: for node in mod.body:
if isinstance(node, ast.FunctionDef) and node.name == "_embeddings_are_tied": if isinstance(node, ast.FunctionDef) and node.name == "_embeddings_are_tied":

1252
tests/test_source_read_encoding.py Normal file
View file

File diff suppressed because it is too large Load diff

89
tests/test_studio_install_workspace_guard.py
View file

@ -15,16 +15,13 @@ SETUP_SH = REPO_ROOT / "studio" / "setup.sh"
# Stubs for helpers the extracted guard block calls; mv-based replacement reproduces the venv-gone # Stubs for helpers the extracted guard block calls; mv-based replacement reproduces the venv-gone
# effect without the full rollback machinery. # effect without the full rollback machinery.
_INSTALL_GUARD_STUBS = ( _INSTALL_GUARD_STUBS = (
"substep() { :; }\n" 'substep() { :; }\n_start_studio_venv_replacement() {\n mv -- "$1" "$1.replaced"\n}\n'
"_start_studio_venv_replacement() {\n"
' mv -- "$1" "$1.replaced"\n'
"}\n"
) )
def _extract_install_sh_guard_block() -> str: def _extract_install_sh_guard_block() -> str:
"""Extract install.sh's venv guard block (up to the first elif) as a self-contained snippet.""" """Extract install.sh's venv guard block (up to the first elif) as a self-contained snippet."""
src = INSTALL_SH.read_text() src = INSTALL_SH.read_text(encoding = "utf-8")
m = re.search( m = re.search(
r'(if \[ -x "\$VENV_DIR/bin/python" \]; then\n.*?)elif \[ "\$_STUDIO_HOME_REDIRECT" != "env"', r'(if \[ -x "\$VENV_DIR/bin/python" \]; then\n.*?)elif \[ "\$_STUDIO_HOME_REDIRECT" != "env"',
src, src,
@ -119,7 +116,7 @@ def test_default_mode_skips_sentinel_check(tmp_path):
def test_install_ps1_has_matching_env_mode_guard(): def test_install_ps1_has_matching_env_mode_guard():
src = INSTALL_PS1.read_text() src = INSTALL_PS1.read_text(encoding = "utf-8")
block_start = src.index("if (Test-Path -LiteralPath $VenvPython)") block_start = src.index("if (Test-Path -LiteralPath $VenvPython)")
block = src[block_start : block_start + 2000] block = src[block_start : block_start + 2000]
assert ( assert (
@ -131,7 +128,7 @@ def test_install_ps1_has_matching_env_mode_guard():
def test_setup_ps1_has_writability_probe(): def test_setup_ps1_has_writability_probe():
src = SETUP_PS1.read_text() src = SETUP_PS1.read_text(encoding = "utf-8")
idx = src.index("if (Test-Path -LiteralPath $_studioOverride -PathType Container)") idx = src.index("if (Test-Path -LiteralPath $_studioOverride -PathType Container)")
block = src[idx : idx + 2000] block = src[idx : idx + 2000]
assert ( assert (
@ -193,7 +190,7 @@ def test_env_mode_passes_when_bin_unsloth_is_a_symlink(tmp_path):
def test_install_ps1_sentinel_uses_pathtype_leaf(): def test_install_ps1_sentinel_uses_pathtype_leaf():
"""Remove-Item $VenvDir gate must use -PathType Leaf so a sentinel-path directory cannot satisfy it.""" """Remove-Item $VenvDir gate must use -PathType Leaf so a sentinel-path directory cannot satisfy it."""
src = INSTALL_PS1.read_text() src = INSTALL_PS1.read_text(encoding = "utf-8")
block_start = src.index("if (Test-Path -LiteralPath $VenvPython)") block_start = src.index("if (Test-Path -LiteralPath $VenvPython)")
block = src[block_start : block_start + 2000] block = src[block_start : block_start + 2000]
assert ( assert (
@ -206,7 +203,7 @@ def test_install_ps1_sentinel_uses_pathtype_leaf():
def test_setup_ps1_stale_venv_has_env_mode_guard(): def test_setup_ps1_stale_venv_has_env_mode_guard():
"""setup.ps1 stale-venv branch must gate Remove-Item $VenvDir on a custom-root Unsloth sentinel.""" """setup.ps1 stale-venv branch must gate Remove-Item $VenvDir on a custom-root Unsloth sentinel."""
src = SETUP_PS1.read_text() src = SETUP_PS1.read_text(encoding = "utf-8")
idx = src.index("Stale venv detected") idx = src.index("Stale venv detected")
block = src[idx : idx + 1500] block = src[idx : idx + 1500]
assert ( assert (
@ -226,7 +223,7 @@ def test_setup_ps1_stale_venv_has_env_mode_guard():
def test_setup_sh_prebuilt_llama_cpp_has_ownership_guard(): def test_setup_sh_prebuilt_llama_cpp_has_ownership_guard():
"""setup.sh prebuilt llama.cpp path must _assert_studio_owned_or_absent before install_llama_prebuilt.py.""" """setup.sh prebuilt llama.cpp path must _assert_studio_owned_or_absent before install_llama_prebuilt.py."""
src = SETUP_SH.read_text() src = SETUP_SH.read_text(encoding = "utf-8")
idx = src.index("installing prebuilt llama.cpp...") idx = src.index("installing prebuilt llama.cpp...")
block = src[idx : idx + 2000] block = src[idx : idx + 2000]
assert ( assert (
@ -240,7 +237,7 @@ def test_setup_sh_prebuilt_llama_cpp_has_ownership_guard():
def test_setup_ps1_prebuilt_llama_cpp_has_ownership_guard(): def test_setup_ps1_prebuilt_llama_cpp_has_ownership_guard():
"""setup.ps1 prebuilt llama.cpp path must Assert-StudioOwnedOrAbsent before install_llama_prebuilt.py.""" """setup.ps1 prebuilt llama.cpp path must Assert-StudioOwnedOrAbsent before install_llama_prebuilt.py."""
src = SETUP_PS1.read_text() src = SETUP_PS1.read_text(encoding = "utf-8")
idx = src.index("installing prebuilt llama.cpp bundle (preferred path)") idx = src.index("installing prebuilt llama.cpp bundle (preferred path)")
block = src[idx : idx + 2000] block = src[idx : idx + 2000]
assert ( assert (
@ -266,9 +263,9 @@ def test_env_mode_passes_when_venv_marker_present(tmp_path):
"""install.sh env-mode guard must accept the in-VENV .unsloth-studio-owned marker as a sentinel.""" """install.sh env-mode guard must accept the in-VENV .unsloth-studio-owned marker as a sentinel."""
studio_home = tmp_path / "ws" studio_home = tmp_path / "ws"
res = _run_install_guard(studio_home, redirect = "env", create_venv_marker = True) res = _run_install_guard(studio_home, redirect = "env", create_venv_marker = True)
assert res.returncode == 0, ( assert (
f"in-VENV marker must allow cleanup; " f"stdout={res.stdout!r} stderr={res.stderr!r}" res.returncode == 0
) ), f"in-VENV marker must allow cleanup; stdout={res.stdout!r} stderr={res.stderr!r}"
assert "RESULT=ok" in res.stdout assert "RESULT=ok" in res.stdout
assert not (studio_home / "unsloth_studio").exists() assert not (studio_home / "unsloth_studio").exists()
@ -318,16 +315,15 @@ def test_env_mode_blocks_when_bin_unsloth_is_broken_symlink(tmp_path):
text = True, text = True,
capture_output = True, capture_output = True,
) )
assert res.returncode != 0, ( assert (
"broken symlink at bin/unsloth must NOT pass; " res.returncode != 0
f"stdout={res.stdout!r} stderr={res.stderr!r}" ), f"broken symlink at bin/unsloth must NOT pass; stdout={res.stdout!r} stderr={res.stderr!r}"
)
assert (venv / "important.txt").is_file() assert (venv / "important.txt").is_file()
def test_install_sh_writes_venv_marker_after_uv_venv(): def test_install_sh_writes_venv_marker_after_uv_venv():
"""install.sh must write .unsloth-studio-owned into $VENV_DIR right after `uv venv` succeeds.""" """install.sh must write .unsloth-studio-owned into $VENV_DIR right after `uv venv` succeeds."""
src = INSTALL_SH.read_text() src = INSTALL_SH.read_text(encoding = "utf-8")
create_idx = src.index('run_install_cmd "create venv" uv venv "$VENV_DIR"') create_idx = src.index('run_install_cmd "create venv" uv venv "$VENV_DIR"')
tail = src[create_idx : create_idx + 600] tail = src[create_idx : create_idx + 600]
assert ( assert (
@ -337,7 +333,7 @@ def test_install_sh_writes_venv_marker_after_uv_venv():
def test_install_ps1_writes_venv_marker_after_uv_venv(): def test_install_ps1_writes_venv_marker_after_uv_venv():
"""install.ps1 must write .unsloth-studio-owned into $VenvDir after `uv venv` succeeds.""" """install.ps1 must write .unsloth-studio-owned into $VenvDir after `uv venv` succeeds."""
src = INSTALL_PS1.read_text() src = INSTALL_PS1.read_text(encoding = "utf-8")
venv_create = src.index("uv venv $VenvDir --python") venv_create = src.index("uv venv $VenvDir --python")
tail = src[venv_create : venv_create + 1500] tail = src[venv_create : venv_create + 1500]
assert ( assert (
@ -347,7 +343,7 @@ def test_install_ps1_writes_venv_marker_after_uv_venv():
def test_install_ps1_guard_accepts_venv_marker(): def test_install_ps1_guard_accepts_venv_marker():
"""install.ps1 env-mode guard must accept the in-VENV .unsloth-studio-owned marker as a sentinel.""" """install.ps1 env-mode guard must accept the in-VENV .unsloth-studio-owned marker as a sentinel."""
src = INSTALL_PS1.read_text() src = INSTALL_PS1.read_text(encoding = "utf-8")
block_start = src.index("if (Test-Path -LiteralPath $VenvPython)") block_start = src.index("if (Test-Path -LiteralPath $VenvPython)")
block = src[block_start : block_start + 2000] block = src[block_start : block_start + 2000]
assert ( assert (
@ -357,7 +353,7 @@ def test_install_ps1_guard_accepts_venv_marker():
def test_setup_helpers_gate_on_canonical_custom_root(): def test_setup_helpers_gate_on_canonical_custom_root():
"""setup.sh/setup.ps1 ownership guards must gate on a canonical custom-vs-legacy root comparison.""" """setup.sh/setup.ps1 ownership guards must gate on a canonical custom-vs-legacy root comparison."""
sh_src = SETUP_SH.read_text() sh_src = SETUP_SH.read_text(encoding = "utf-8")
sh_idx = sh_src.index("_assert_studio_owned_or_absent() {") sh_idx = sh_src.index("_assert_studio_owned_or_absent() {")
sh_func = sh_src[sh_idx : sh_idx + 600] sh_func = sh_src[sh_idx : sh_idx + 600]
assert ( assert (
@ -369,7 +365,7 @@ def test_setup_helpers_gate_on_canonical_custom_root():
and "_STUDIO_HOME_IS_CUSTOM=" in sh_src and "_STUDIO_HOME_IS_CUSTOM=" in sh_src
), "setup.sh must compute the canonical custom-root flag" ), "setup.sh must compute the canonical custom-root flag"
ps_src = SETUP_PS1.read_text() ps_src = SETUP_PS1.read_text(encoding = "utf-8")
ps_idx = ps_src.index("function Assert-StudioOwnedOrAbsent") ps_idx = ps_src.index("function Assert-StudioOwnedOrAbsent")
ps_func = ps_src[ps_idx : ps_idx + 800] ps_func = ps_src[ps_idx : ps_idx + 800]
assert ( assert (
@ -382,7 +378,7 @@ def test_setup_helpers_gate_on_canonical_custom_root():
def test_setup_ps1_inplace_git_sync_marks_studio_owned(): def test_setup_ps1_inplace_git_sync_marks_studio_owned():
"""setup.ps1 in-place git-sync branch must Mark-StudioOwned after a successful sync.""" """setup.ps1 in-place git-sync branch must Mark-StudioOwned after a successful sync."""
src = SETUP_PS1.read_text() src = SETUP_PS1.read_text(encoding = "utf-8")
inplace_idx = src.index('Test-Path -LiteralPath (Join-Path $LlamaCppDir ".git")') inplace_idx = src.index('Test-Path -LiteralPath (Join-Path $LlamaCppDir ".git")')
# The in-place branch ends just before the temp-dir clone branch. # The in-place branch ends just before the temp-dir clone branch.
clone_idx = src.index("Cloning llama.cpp @", inplace_idx) clone_idx = src.index("Cloning llama.cpp @", inplace_idx)
@ -397,7 +393,7 @@ def test_setup_ps1_inplace_git_sync_marks_studio_owned():
def test_setup_ps1_inplace_git_sync_asserts_studio_owned_before_mutation(): def test_setup_ps1_inplace_git_sync_asserts_studio_owned_before_mutation():
"""setup.ps1 in-place git-sync must Assert-StudioOwnedOrAbsent before any destructive git op.""" """setup.ps1 in-place git-sync must Assert-StudioOwnedOrAbsent before any destructive git op."""
src = SETUP_PS1.read_text() src = SETUP_PS1.read_text(encoding = "utf-8")
inplace_idx = src.index('Test-Path -LiteralPath (Join-Path $LlamaCppDir ".git")') inplace_idx = src.index('Test-Path -LiteralPath (Join-Path $LlamaCppDir ".git")')
clone_idx = src.index("Cloning llama.cpp @", inplace_idx) clone_idx = src.index("Cloning llama.cpp @", inplace_idx)
inplace_block = src[inplace_idx:clone_idx] inplace_block = src[inplace_idx:clone_idx]
@ -410,7 +406,7 @@ def test_setup_ps1_inplace_git_sync_asserts_studio_owned_before_mutation():
def _extract_check_health_function() -> str: def _extract_check_health_function() -> str:
src = INSTALL_SH.read_text() src = INSTALL_SH.read_text(encoding = "utf-8")
fn_start = src.index("_check_health() {") fn_start = src.index("_check_health() {")
fn_end = src.index("\n}\n", fn_start) + 2 fn_end = src.index("\n}\n", fn_start) + 2
return src[fn_start:fn_end] return src[fn_start:fn_end]
@ -498,7 +494,7 @@ def test_check_health_handles_arbitrary_id_token():
def test_install_ps1_test_studio_health_verifies_studio_root_id(): def test_install_ps1_test_studio_health_verifies_studio_root_id():
"""install.ps1 Test-StudioHealth must compare studio_root_id against baked $_ExpectedStudioRootId.""" """install.ps1 Test-StudioHealth must compare studio_root_id against baked $_ExpectedStudioRootId."""
src = INSTALL_PS1.read_text() src = INSTALL_PS1.read_text(encoding = "utf-8")
fn_start = src.index("function Test-StudioHealth") fn_start = src.index("function Test-StudioHealth")
fn_end = src.index("\n}\n", fn_start) + 2 fn_end = src.index("\n}\n", fn_start) + 2
fn = src[fn_start:fn_end] fn = src[fn_start:fn_end]
@ -510,7 +506,7 @@ def test_install_ps1_test_studio_health_verifies_studio_root_id():
def test_install_ps1_bakes_studio_root_id_into_launcher(): def test_install_ps1_bakes_studio_root_id_into_launcher():
"""install.ps1 must persist a CSPRNG id at share/studio_install_id and bake it as $_ExpectedStudioRootId.""" """install.ps1 must persist a CSPRNG id at share/studio_install_id and bake it as $_ExpectedStudioRootId."""
src = INSTALL_PS1.read_text() src = INSTALL_PS1.read_text(encoding = "utf-8")
assert "$_studioRootId" in src, "install.ps1 must compute $_studioRootId for the launcher" assert "$_studioRootId" in src, "install.ps1 must compute $_studioRootId for the launcher"
assert ( assert (
'"share"' in src and "studio_install_id" in src '"share"' in src and "studio_install_id" in src
@ -526,7 +522,7 @@ def test_install_ps1_bakes_studio_root_id_into_launcher():
def test_health_endpoint_exposes_studio_root_id_not_raw_path(): def test_health_endpoint_exposes_studio_root_id_not_raw_path():
"""/api/health must expose studio_root_id (hex digest), NOT the raw path (info disclosure on -H 0.0.0.0).""" """/api/health must expose studio_root_id (hex digest), NOT the raw path (info disclosure on -H 0.0.0.0)."""
main_py = REPO_ROOT / "studio" / "backend" / "main.py" main_py = REPO_ROOT / "studio" / "backend" / "main.py"
src = main_py.read_text() src = main_py.read_text(encoding = "utf-8")
health_idx = src.index('@app.get("/api/health")') health_idx = src.index('@app.get("/api/health")')
# Slice up to the next top-level @app. so a growing body stays in scope. # Slice up to the next top-level @app. so a growing body stays in scope.
next_app_idx = src.find("\n@app.", health_idx + 1) next_app_idx = src.find("\n@app.", health_idx + 1)
@ -542,7 +538,7 @@ def test_health_endpoint_exposes_studio_root_id_not_raw_path():
def test_install_sh_bakes_studio_root_id_into_launcher(): def test_install_sh_bakes_studio_root_id_into_launcher():
"""install.sh must persist the id at share/studio_install_id and bake it into the launcher for ALL modes.""" """install.sh must persist the id at share/studio_install_id and bake it into the launcher for ALL modes."""
src = INSTALL_SH.read_text() src = INSTALL_SH.read_text(encoding = "utf-8")
assert ( assert (
"_css_studio_root_id" in src "_css_studio_root_id" in src
), "install.sh must compute _css_studio_root_id for the launcher" ), "install.sh must compute _css_studio_root_id for the launcher"
@ -568,8 +564,10 @@ def test_tauri_preflight_scrubs_studio_home_env():
preflight_root / "preflight.rs", preflight_root / "preflight.rs",
*(preflight_root / "preflight").glob("*.rs"), *(preflight_root / "preflight").glob("*.rs"),
] ]
preflight = "\n".join(p.read_text() for p in preflight_paths if p.exists()) preflight = "\n".join(p.read_text(encoding = "utf-8") for p in preflight_paths if p.exists())
commands = (REPO_ROOT / "studio" / "src-tauri" / "src" / "commands.rs").read_text() commands = (REPO_ROOT / "studio" / "src-tauri" / "src" / "commands.rs").read_text(
encoding = "utf-8"
)
# Expect 2 scrubs in preflight (run_cli_probe + probe_cli_capability), 1 in commands. # Expect 2 scrubs in preflight (run_cli_probe + probe_cli_capability), 1 in commands.
assert ( assert (
preflight.count('cmd.env_remove("UNSLOTH_STUDIO_HOME")') >= 2 preflight.count('cmd.env_remove("UNSLOTH_STUDIO_HOME")') >= 2
@ -587,7 +585,7 @@ def test_tauri_preflight_scrubs_studio_home_env():
def test_install_sh_shim_uses_atomic_replace(): def test_install_sh_shim_uses_atomic_replace():
"""install.sh shim install must use ln -sfn for atomic replace (rm+ln left a missing-shim window).""" """install.sh shim install must use ln -sfn for atomic replace (rm+ln left a missing-shim window)."""
src = INSTALL_SH.read_text() src = INSTALL_SH.read_text(encoding = "utf-8")
shim_idx = src.index('_shim_path="$_LOCAL_BIN/unsloth"') shim_idx = src.index('_shim_path="$_LOCAL_BIN/unsloth"')
block = src[shim_idx : shim_idx + 1500] block = src[shim_idx : shim_idx + 1500]
assert ( assert (
@ -600,7 +598,7 @@ def test_install_sh_shim_uses_atomic_replace():
def test_install_sh_create_shortcuts_seeds_id_from_csprng_with_python_fallback(tmp_path): def test_install_sh_create_shortcuts_seeds_id_from_csprng_with_python_fallback(tmp_path):
"""_create_shortcuts seeds ids from /dev/urandom (python3 secrets fallback) and is re-run idempotent.""" """_create_shortcuts seeds ids from /dev/urandom (python3 secrets fallback) and is re-run idempotent."""
src = INSTALL_SH.read_text() src = INSTALL_SH.read_text(encoding = "utf-8")
fn_start = src.index('_css_data_dir="$DATA_DIR"') fn_start = src.index('_css_data_dir="$DATA_DIR"')
block = src[fn_start : fn_start + 3000] block = src[fn_start : fn_start + 3000]
urandom_idx = block.index("od -An -N32 -tx1 /dev/urandom") urandom_idx = block.index("od -An -N32 -tx1 /dev/urandom")
@ -645,7 +643,7 @@ def test_install_sh_create_shortcuts_seeds_id_from_csprng_with_python_fallback(t
def test_install_sh_create_shortcuts_fails_fast_when_no_entropy(): def test_install_sh_create_shortcuts_fails_fast_when_no_entropy():
"""With no entropy source, _create_shortcuts must `return 1` not bake an empty studio_root_id.""" """With no entropy source, _create_shortcuts must `return 1` not bake an empty studio_root_id."""
src = INSTALL_SH.read_text() src = INSTALL_SH.read_text(encoding = "utf-8")
fn_start = src.index('_css_data_dir="$DATA_DIR"') fn_start = src.index('_css_data_dir="$DATA_DIR"')
block = src[fn_start : fn_start + 3000] block = src[fn_start : fn_start + 3000]
assert ( assert (
@ -661,7 +659,7 @@ def test_install_sh_create_shortcuts_fails_fast_when_no_entropy():
def test_install_sh_bakes_installed_is_env_mode_flag_in_launcher(): def test_install_sh_bakes_installed_is_env_mode_flag_in_launcher():
"""install.sh must bake the install-time mode into the launcher so a sourced studio.conf can't flip it.""" """install.sh must bake the install-time mode into the launcher so a sourced studio.conf can't flip it."""
src = INSTALL_SH.read_text() src = INSTALL_SH.read_text(encoding = "utf-8")
assert ( assert (
"_INSTALLED_IS_ENV_MODE='@@INSTALLED_IS_ENV_MODE@@'" in src "_INSTALLED_IS_ENV_MODE='@@INSTALLED_IS_ENV_MODE@@'" in src
), "launcher heredoc must declare _INSTALLED_IS_ENV_MODE='@@INSTALLED_IS_ENV_MODE@@'" ), "launcher heredoc must declare _INSTALLED_IS_ENV_MODE='@@INSTALLED_IS_ENV_MODE@@'"
@ -676,7 +674,7 @@ def test_install_sh_bakes_installed_is_env_mode_flag_in_launcher():
def test_install_sh_launcher_gates_port_file_on_baked_flag_not_runtime_env(): def test_install_sh_launcher_gates_port_file_on_baked_flag_not_runtime_env():
"""Launcher PORT_FILE/LOCK_DIR must gate on baked $_INSTALLED_IS_ENV_MODE, not runtime $UNSLOTH_STUDIO_HOME.""" """Launcher PORT_FILE/LOCK_DIR must gate on baked $_INSTALLED_IS_ENV_MODE, not runtime $UNSLOTH_STUDIO_HOME."""
src = INSTALL_SH.read_text() src = INSTALL_SH.read_text(encoding = "utf-8")
heredoc_start = src.index("cat > \"$_css_launcher\" << 'LAUNCHER_EOF'") heredoc_start = src.index("cat > \"$_css_launcher\" << 'LAUNCHER_EOF'")
heredoc_end = src.index("LAUNCHER_EOF\n", heredoc_start) heredoc_end = src.index("LAUNCHER_EOF\n", heredoc_start)
heredoc = src[heredoc_start:heredoc_end] heredoc = src[heredoc_start:heredoc_end]
@ -724,7 +722,7 @@ def test_install_sh_launcher_gates_port_file_on_baked_flag_not_runtime_env():
def test_main_py_studio_root_id_caches_at_module_load(): def test_main_py_studio_root_id_caches_at_module_load():
"""_studio_root_id() must read the id once at module load and reuse it (no per-poll FS/hash work).""" """_studio_root_id() must read the id once at module load and reuse it (no per-poll FS/hash work)."""
main_py = (REPO_ROOT / "studio" / "backend" / "main.py").read_text() main_py = (REPO_ROOT / "studio" / "backend" / "main.py").read_text(encoding = "utf-8")
assert ( assert (
"_STUDIO_ROOT_ID_CACHE: str = _read_studio_install_id()" in main_py "_STUDIO_ROOT_ID_CACHE: str = _read_studio_install_id()" in main_py
), "main.py must populate _STUDIO_ROOT_ID_CACHE from _read_studio_install_id() at module load" ), "main.py must populate _STUDIO_ROOT_ID_CACHE from _read_studio_install_id() at module load"
@ -785,7 +783,7 @@ def test_llama_cpp_search_roots_handles_studio_root_oserror():
holds the handler so the two never disagree on which root is legacy.""" holds the handler so the two never disagree on which root is legacy."""
llama_cpp = ( llama_cpp = (
REPO_ROOT / "studio" / "backend" / "core" / "inference" / "llama_cpp.py" REPO_ROOT / "studio" / "backend" / "core" / "inference" / "llama_cpp.py"
).read_text() ).read_text(encoding = "utf-8")
def _method_body(name: str) -> str: def _method_body(name: str) -> str:
# Whole method body (def to next sibling def) so the check survives growth. # Whole method body (def to next sibling def) so the check survives growth.
@ -830,7 +828,7 @@ def test_install_sh_install_id_survives_symlinked_studio_home(tmp_path):
def test_install_sh_substitutes_root_id_before_data_dir(): def test_install_sh_substitutes_root_id_before_data_dir():
"""sed must bake the non-user-controlled placeholders before @@DATA_DIR@@ so a crafted $DATA_DIR isn't mutated.""" """sed must bake the non-user-controlled placeholders before @@DATA_DIR@@ so a crafted $DATA_DIR isn't mutated."""
src = INSTALL_SH.read_text() src = INSTALL_SH.read_text(encoding = "utf-8")
root_id_idx = src.index("s|@@STUDIO_ROOT_ID@@|$_css_studio_root_id|g") root_id_idx = src.index("s|@@STUDIO_ROOT_ID@@|$_css_studio_root_id|g")
env_mode_idx = src.index("s|@@INSTALLED_IS_ENV_MODE@@|$_css_is_env_mode|g") env_mode_idx = src.index("s|@@INSTALLED_IS_ENV_MODE@@|$_css_is_env_mode|g")
data_dir_idx = src.index("s|@@DATA_DIR@@|$_sed_safe|g") data_dir_idx = src.index("s|@@DATA_DIR@@|$_sed_safe|g")
@ -845,13 +843,15 @@ def test_install_sh_substitutes_root_id_before_data_dir():
def test_install_sh_root_id_pass_does_not_mutate_user_data_dir(tmp_path): def test_install_sh_root_id_pass_does_not_mutate_user_data_dir(tmp_path):
"""A $DATA_DIR containing the literal @@STUDIO_ROOT_ID@@ must survive the placeholder-first sed passes.""" """A $DATA_DIR containing the literal @@STUDIO_ROOT_ID@@ must survive the placeholder-first sed passes."""
src = INSTALL_SH.read_text() src = INSTALL_SH.read_text(encoding = "utf-8")
heredoc_start = src.index("cat > \"$_css_launcher\" << 'LAUNCHER_EOF'") heredoc_start = src.index("cat > \"$_css_launcher\" << 'LAUNCHER_EOF'")
heredoc_body_start = src.index("\n", heredoc_start) + 1 heredoc_body_start = src.index("\n", heredoc_start) + 1
heredoc_body_end = src.index("LAUNCHER_EOF\n", heredoc_start) heredoc_body_end = src.index("LAUNCHER_EOF\n", heredoc_start)
template = src[heredoc_body_start:heredoc_body_end] template = src[heredoc_body_start:heredoc_body_end]
launcher_path = tmp_path / "launch.sh" launcher_path = tmp_path / "launch.sh"
launcher_path.write_text(template) # template comes out of install.sh, so it carries whatever non-ASCII that
# file holds and cp1252 cannot encode it back out.
launcher_path.write_text(template, encoding = "utf-8")
# sed order: root-id first, then data-dir. # sed order: root-id first, then data-dir.
weird_data_dir = "/tmp/with-@@STUDIO_ROOT_ID@@/share" weird_data_dir = "/tmp/with-@@STUDIO_ROOT_ID@@/share"
root_id = "deadbeef" * 8 root_id = "deadbeef" * 8
@ -866,7 +866,8 @@ sed "s|@@DATA_DIR@@|$_sed_safe|g" "{launcher_path}" > "{launcher_path}.tmp" \\
&& mv "{launcher_path}.tmp" "{launcher_path}" && mv "{launcher_path}.tmp" "{launcher_path}"
""" """
subprocess.run(["bash", "-c", script], check = True) subprocess.run(["bash", "-c", script], check = True)
final = launcher_path.read_text() # written as utf-8 just above, and the template carries U+2500.
final = launcher_path.read_text(encoding = "utf-8")
assert ( assert (
f"DATA_DIR='{weird_data_dir}'" in final f"DATA_DIR='{weird_data_dir}'" in final
), f"DATA_DIR must be preserved verbatim (no @@STUDIO_ROOT_ID@@ mutation); got: {final[:500]}" ), f"DATA_DIR must be preserved verbatim (no @@STUDIO_ROOT_ID@@ mutation); got: {final[:500]}"
@ -877,7 +878,7 @@ sed "s|@@DATA_DIR@@|$_sed_safe|g" "{launcher_path}" > "{launcher_path}.tmp" \\
def test_install_ps1_install_id_file_layout_matches_backend_read_path(): def test_install_ps1_install_id_file_layout_matches_backend_read_path():
"""install.ps1 must write the id at share/studio_install_id where the backend reads it, idempotently.""" """install.ps1 must write the id at share/studio_install_id where the backend reads it, idempotently."""
src = INSTALL_PS1.read_text() src = INSTALL_PS1.read_text(encoding = "utf-8")
id_idx = src.index('$_studioIdDir = Join-Path $StudioHome "share"') id_idx = src.index('$_studioIdDir = Join-Path $StudioHome "share"')
context = src[id_idx : id_idx + 1500] context = src[id_idx : id_idx + 1500]
assert ( assert (

4
tests/test_studio_root_resilience.py
View file

@ -64,7 +64,7 @@ def test_kill_orphan_catches_oserror_from_studio_root():
"""Cleanup must not crash when studio_root() raises. _kill_orphaned_servers """Cleanup must not crash when studio_root() raises. _kill_orphaned_servers
resolves the install root through the shared _resolved_studio_root_and_is_legacy() resolves the install root through the shared _resolved_studio_root_and_is_legacy()
classifier, which swallows (ImportError, OSError, ValueError) on the probe.""" classifier, which swallows (ImportError, OSError, ValueError) on the probe."""
src = LLAMA_CPP.read_text() src = LLAMA_CPP.read_text(encoding = "utf-8")
# Cleanup delegates to the shared classifier rather than importing studio_root inline. # Cleanup delegates to the shared classifier rather than importing studio_root inline.
assert "LlamaCppBackend._resolved_studio_root_and_is_legacy()" in _method_body( assert "LlamaCppBackend._resolved_studio_root_and_is_legacy()" in _method_body(
src, "_kill_orphaned_servers" src, "_kill_orphaned_servers"
@ -85,7 +85,7 @@ def _exec_search_roots_block(
"""Run _find_llama_server_binary's search_roots derivation -- plus the shared """Run _find_llama_server_binary's search_roots derivation -- plus the shared
_resolved_studio_root_and_is_legacy() classifier it delegates to -- with a _resolved_studio_root_and_is_legacy() classifier it delegates to -- with a
controlled studio_root() and resolve(), without importing the heavy module.""" controlled studio_root() and resolve(), without importing the heavy module."""
src = LLAMA_CPP.read_text() src = LLAMA_CPP.read_text(encoding = "utf-8")
# Shared root classifier (holds the defensive try/except for studio_root()). # Shared root classifier (holds the defensive try/except for studio_root()).
# End the slice at the next sibling def/decorator at the same indent rather # End the slice at the next sibling def/decorator at the same indent rather
# than the literal "@staticmethod" string, so a future docstring mentioning a # than the literal "@staticmethod" string, so a future docstring mentioning a

2
tests/test_tool_mask_zoo_compat.py
View file

@ -15,7 +15,7 @@ RL_REPLACEMENTS_SOURCE_PATH = os.path.join(REPO_ROOT, "unsloth", "models", "rl_r
def _read(path: str) -> str: def _read(path: str) -> str:
with open(path, "r") as fh: with open(path, "r", encoding = "utf-8") as fh:
return fh.read() return fh.read()

4
tests/utils/test_prepare_inputs_leftpad.py
View file

@ -46,7 +46,7 @@ WIRED_MODEL_FILES = [
def _load_function(): def _load_function():
tree = ast.parse(LLAMA_PY.read_text()) tree = ast.parse(LLAMA_PY.read_text(encoding = "utf-8"))
for node in ast.walk(tree): for node in ast.walk(tree):
if isinstance(node, ast.FunctionDef) and node.name == FUNC_NAME: if isinstance(node, ast.FunctionDef) and node.name == FUNC_NAME:
return node return node
@ -207,7 +207,7 @@ def test_model_families_stay_wired_to_shared_prepare_inputs():
path = REPO_ROOT / "unsloth" / "models" / fname path = REPO_ROOT / "unsloth" / "models" / fname
if not path.exists(): if not path.exists():
continue continue
if "fix_prepare_inputs_for_generation(" not in path.read_text(): if "fix_prepare_inputs_for_generation(" not in path.read_text(encoding = "utf-8"):
missing.append(fname) missing.append(fname)
assert not missing, ( assert not missing, (
"these model files no longer call fix_prepare_inputs_for_generation, " "these model files no longer call fix_prepare_inputs_for_generation, "

6
tests/utils/test_rope_scaling_drift.py
View file

@ -52,7 +52,7 @@ MAX_POS = 131072
def _load_class_init(): def _load_class_init():
tree = ast.parse(LLAMA_PY.read_text()) tree = ast.parse(LLAMA_PY.read_text(encoding = "utf-8"))
for node in ast.walk(tree): for node in ast.walk(tree):
if isinstance(node, ast.ClassDef) and node.name == CLASS_NAME: if isinstance(node, ast.ClassDef) and node.name == CLASS_NAME:
for sub in node.body: for sub in node.body:
@ -96,7 +96,7 @@ def _iter_names_and_calls(node):
def _find_method(source_path, class_name, method_name): def _find_method(source_path, class_name, method_name):
for node in ast.walk(ast.parse(source_path.read_text())): for node in ast.walk(ast.parse(source_path.read_text(encoding = "utf-8"))):
if isinstance(node, ast.ClassDef) and node.name == class_name: if isinstance(node, ast.ClassDef) and node.name == class_name:
for sub in node.body: for sub in node.body:
if isinstance(sub, ast.FunctionDef) and sub.name == method_name: if isinstance(sub, ast.FunctionDef) and sub.name == method_name:
@ -105,7 +105,7 @@ def _find_method(source_path, class_name, method_name):
def _find_function(source_path, function_name): def _find_function(source_path, function_name):
for node in ast.walk(ast.parse(source_path.read_text())): for node in ast.walk(ast.parse(source_path.read_text(encoding = "utf-8"))):
if isinstance(node, ast.FunctionDef) and node.name == function_name: if isinstance(node, ast.FunctionDef) and node.name == function_name:
return node return node
return None return None

27
unsloth_cli/__init__.py
View file

@ -4,6 +4,28 @@
import os as _os import os as _os
import sys as _sys import sys as _sys
# Are we the `unsloth` console script, rather than a library import? Both the
# stream guard below and the `-np<N>` rewrite further down are entry-point
# behaviour and must not reach into a host application that imports us.
_entry_base = _os.path.basename(_sys.argv[0]).lower() if _sys.argv else ""
_is_entry_point = _entry_base in {"unsloth", "unsloth.exe"}
# Typer renders help via rich, whose box characters cp1252 and cp437 cannot encode,
# so `unsloth --help` dies once stdout is a pipe or a file. Windows gets UTF-8, as
# unsloth/__init__ already does; elsewhere the caller's encoding is kept and only
# the error handler is relaxed, so an explicit PYTHONIOENCODING still picks the
# bytes and only loses unencodable glyphs. Before typer, which binds the stream.
if _is_entry_point:
_to_utf8 = _sys.platform == "win32"
for _name in ("stdout", "stderr"):
_stream = getattr(_sys, _name, None)
try:
if "utf" not in (_stream.encoding or "").lower():
_stream.reconfigure(encoding = "utf-8" if _to_utf8 else None, errors = "replace")
except Exception:
pass
del _name, _stream, _to_utf8
import typer import typer
from importlib.metadata import version as package_version, PackageNotFoundError from importlib.metadata import version as package_version, PackageNotFoundError
@ -22,10 +44,9 @@ from unsloth_cli.commands.studio import (
# Canonicalise `-np<N>` only under the `unsloth` console-script; # Canonicalise `-np<N>` only under the `unsloth` console-script;
# third-party scripts that import unsloth_cli keep their argv intact. # third-party scripts that import unsloth_cli keep their argv intact.
_entry_base = _os.path.basename(_sys.argv[0]).lower() if _sys.argv else "" if _is_entry_point:
if _entry_base in {"unsloth", "unsloth.exe"}:
_expand_attached_np_short() _expand_attached_np_short()
del _entry_base del _entry_base, _is_entry_point
def show_version(value: bool): def show_version(value: bool):

6
unsloth_cli/tests/test_start.py
View file

@ -587,7 +587,9 @@ def test_write_codex_config_profile(tmp_path, monkeypatch):
assert catalog["models"][0]["supports_reasoning_summary_parameter"] is False assert catalog["models"][0]["supports_reasoning_summary_parameter"] is False
assert catalog["models"][0]["supports_parallel_tool_calls"] is False assert catalog["models"][0]["supports_parallel_tool_calls"] is False
assert catalog["models"][0]["base_instructions"] == start._CODEX_FALLBACK_PROMPT.read_text() assert catalog["models"][0]["base_instructions"] == start._CODEX_FALLBACK_PROMPT.read_text(
encoding = "utf-8"
)
config = _parse_toml((tmp_path / "config.toml").read_text()) config = _parse_toml((tmp_path / "config.toml").read_text())
assert config["model_providers"]["unsloth_api"]["env_key"] == "UNSLOTH_STUDIO_AUTH_TOKEN" assert config["model_providers"]["unsloth_api"]["env_key"] == "UNSLOTH_STUDIO_AUTH_TOKEN"
@ -631,7 +633,7 @@ def test_write_codex_subagent_bridge_keeps_parent_credentials_out(tmp_path, monk
tmp_path, tmp_path,
yolo = False, yolo = False,
) )
assert json.loads(path.read_text()) == { assert json.loads(path.read_text(encoding = "utf-8")) == {
"api_key": "private-token", "api_key": "private-token",
"codex_home": str(tmp_path / "child"), "codex_home": str(tmp_path / "child"),
"bypass_permissions": False, "bypass_permissions": False,