unsloth/unsloth
Tenith Hasintha 739ebeea82
Fix/issue 3667 vicuna template (#5357)
* Fix: Add missing utf-8 encoding to text-mode file operations

Fixes #2795 by explicitly adding encoding='utf-8' to open() calls. This prevents UnicodeDecodeError on Windows with non-UTF-8 system locales when processing files containing UTF-8 characters.

* fix(chat_templates): escape apostrophe in vicuna default system message

The unescaped apostrophe in 'user's' broke Jinja2 template parsing
when _change_system_message() substituted the default message into the
template string via re.sub. Escape it with \' to match the existing
vicuna_old pattern.

Fixes #3667

---------

Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
2026-05-14 18:49:47 +04:00
..
dataprep Fix raw text paragraph break normalization (#4884) 2026-04-09 04:45:43 -07:00
kernels Fix: Add missing utf-8 encoding to text-mode file operations (#5356) 2026-05-14 18:15:27 +04:00
models Fix: Add missing utf-8 encoding to text-mode file operations (#5356) 2026-05-14 18:15:27 +04:00
optimizers feat: Implement Q-GaLore optimizer and custom embedding learning rate… (#4511) 2026-03-25 01:03:10 -07:00
registry Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks" 2025-12-01 07:24:58 -08:00
utils Add check to disable xformers on newer GPUs (#4342) 2026-03-16 22:42:38 -07:00
__init__.py MLX training support for Studio on Apple Silicon (#5340) 2026-05-14 05:24:20 -07:00
_auto_install.py Add PyTorch 2.10 and xformers 0.0.34 support (#3985) 2026-02-05 05:56:26 -08:00
_gpu_init.py chore: trim verbose comments added in PR #5416 (commit 12295c1f) (#5418) 2026-05-14 04:25:23 -07:00
chat_templates.py Fix/issue 3667 vicuna template (#5357) 2026-05-14 18:49:47 +04:00
device_type.py MLX training support for Studio on Apple Silicon (#5340) 2026-05-14 05:24:20 -07:00
import_fixes.py import_fixes + drift detectors: cover transformers 5.x drift (#5423) 2026-05-14 05:14:21 -07:00
ollama_template_mappers.py Re-apply #4939: updated models template mappers (#4950) 2026-04-15 07:52:12 -07:00
save.py Fix: Add missing utf-8 encoding to text-mode file operations (#5356) 2026-05-14 18:15:27 +04:00
tokenizer_utils.py fix: 3 patch_* helpers — fast_lora import, sft_trainer Union, openenv OSError (#5319) 2026-05-07 00:12:09 -07:00
trainer.py Trim trainer.py import-fix comment to one line 2026-05-06 07:22:56 +00:00