ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
3,305 commits 391 branches 58 tags 304 MiB
Commit graph

6 commits

Author SHA1 Message Date
pre-commit-ci[bot]
8c506a27c4 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-08 11:35:21 +00:00
Daniel Han
56ebc94c94 Fix bugs and add improvements to RawTextDataLoader 
- Fix test file: use return_tokenized instead of return_tensors
- Fix test file: use text_dataset instead of undefined dataset variable
- Move parameter validation to constructor (fail fast on invalid params)
- Add labels field in tokenized output for causal LM training
- Add empty file handling with clear error message
- Add tests for constructor validation and labels field
 2026-01-08 11:35:00 +00:00
pre-commit-ci[bot]
73ed28c79c [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2025-11-20 13:09:08 +00:00
vangmay
a12eefa75a Make the chunk function efficient 2025-11-20 21:08:33 +08:00
pre-commit-ci[bot]
ffc68beec0 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2025-11-20 12:51:18 +00:00
vangmay
5017d9748b Write simple test 2025-11-18 22:36:38 +08:00