ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Parse a .json dataset file as one JSON document instead of line-by-line (#7422)

Browse source 
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
This commit is contained in:
Vineeth Sai Varikuntla 2026-07-28 17:40:43 -07:00 • committed by GitHub
commit 7ac75c6572
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 162 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

3
.github/workflows/consolidated-tests-ci.yml vendored
View file

@ -372,7 +372,8 @@ jobs:
tests/python/test_fast_language_model_text_only.py \
tests/test_bad_mappings_redirect.py \
tests/test_prefetch_snapshot_scope.py \
tests/test_gemma_2b_mapper_key.py
tests/test_gemma_2b_mapper_key.py \
tests/test_raw_text_json_loading.py
# test_run_attention_flash_varlen_receives_window_and_softcap was deselected
# until attention_dispatch.py predefined flash_attn_varlen_func as None; it
# monkeypatches that name, so it no longer needs flash_attn on this runner.