ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
3,291 commits 391 branches 58 tags 304 MiB
Commit graph

6 commits

Author SHA1 Message Date
pre-commit-ci[bot]
3ac4f3c213 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-08 11:35:21 +00:00
Daniel Han
24bbe8a97a Fix bugs and add improvements to RawTextDataLoader 
- Fix test file: use return_tokenized instead of return_tensors
- Fix test file: use text_dataset instead of undefined dataset variable
- Move parameter validation to constructor (fail fast on invalid params)
- Add labels field in tokenized output for causal LM training
- Add empty file handling with clear error message
- Add tests for constructor validation and labels field
 2026-01-08 11:35:00 +00:00
pre-commit-ci[bot]
2ee16e7ee2 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2025-11-20 13:09:08 +00:00
vangmay
e57b1c73fd Make the chunk function efficient 2025-11-20 21:08:33 +08:00
pre-commit-ci[bot]
69e8967742 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2025-11-20 12:51:18 +00:00
vangmay
e473c0dbdf Write simple test 2025-11-18 22:36:38 +08:00