Aditya Ghai
|
08bc291300
|
Direct windows support for unsloth (#1841)
* Direct Windows Support(main)
* Update pyproject.toml
* Update README.md
Added the suggested changes to README
|
2025-02-27 20:25:46 -08:00 |
|
Daniel Han
|
4570c8b41e
|
Bug Fixes (#1774)
* Update rl.py
* Update tokenizer_utils.py
* Auto patching
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update rl.py
* Update tokenizer_utils.py
* Update rl.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update tokenizer_utils.py
* Update rl.py
* Update rl.py
* Update rl.py
* max seq length
* Update rl.py
* Update rl.py
* Patching
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* NEFTune
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Extra replacements
* Update rl_replacements.py
* Update rl.py
* extra RL replacements
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update _utils.py
* Update loader_utils.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* autocast
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update pyproject.toml
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* GRPO optimized
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Selective Log softmax
* Fix GRPO bsz
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Fix TRL
* Metrics GRPO
* Update rl_replacements.py
* Update rl_replacements.py
* No compile
* Update rl.py
* Remove docs
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* llama-quantize on WINDOWS WSL error fix - edit save.py (gguf saving breaks) (#1649)
* edit save.py to fix gguf saving breaks.
* add check for .exe or not exe file extension for linux and windows
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* unsloth_num_chunks
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py (#1754)
Fix typo in comment: know -> now.
This was printed when running the Llama3.1_(8B)-GRPO.ipynb example notebook, so I'd expect others to run into it as well.
* Optional logits
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* fix an import error (#1767)
* fix an import error
* Delete .gitignore
* Update loader.py
* Update save.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* SamplingParams
* Convert mask to float (#1762)
* [Windows Support] Add latest `xformers` wheels to pyproject.toml (#1753)
* Add latest xformers
* Add a couple of lines to docs
* vLLMSamplingParams
* Update __init__.py
* default num_chunks == -1
* Versioning
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update _utils.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update pyproject.toml
* Update pyproject.toml
---------
Co-authored-by: Gennadii Manzhos <105049664+everythingisc00l@users.noreply.github.com>
Co-authored-by: Seth Weidman <seth@sethweidman.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Ben <6579034+versipellis@users.noreply.github.com>
|
2025-02-20 09:20:49 -08:00 |
|
Daniel Han
|
a45a08f91b
|
Memory Efficient GRPO (#1773)
* Update __init__.py
* Update loader.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Better TRL handling
* Update rl.py
* Update tokenizer_utils.py
* Auto patching
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update rl.py
* Update tokenizer_utils.py
* Update rl.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update tokenizer_utils.py
* Update rl.py
* Update rl.py
* Update rl.py
* max seq length
* Update rl.py
* Update rl.py
* Patching
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* NEFTune
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Extra replacements
* Update rl_replacements.py
* Update rl.py
* extra RL replacements
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update _utils.py
* Update loader_utils.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* autocast
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update pyproject.toml
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* GRPO optimized
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Selective Log softmax
* Fix GRPO bsz
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Fix TRL
* Metrics GRPO
* Update rl_replacements.py
* Update rl_replacements.py
* No compile
* Update rl.py
* Remove docs
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* llama-quantize on WINDOWS WSL error fix - edit save.py (gguf saving breaks) (#1649)
* edit save.py to fix gguf saving breaks.
* add check for .exe or not exe file extension for linux and windows
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* unsloth_num_chunks
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py (#1754)
Fix typo in comment: know -> now.
This was printed when running the Llama3.1_(8B)-GRPO.ipynb example notebook, so I'd expect others to run into it as well.
* Optional logits
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* fix an import error (#1767)
* fix an import error
* Delete .gitignore
* Update loader.py
* Update save.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* SamplingParams
* Convert mask to float (#1762)
* [Windows Support] Add latest `xformers` wheels to pyproject.toml (#1753)
* Add latest xformers
* Add a couple of lines to docs
* vLLMSamplingParams
* Update __init__.py
* default num_chunks == -1
* Versioning
---------
Co-authored-by: Gennadii Manzhos <105049664+everythingisc00l@users.noreply.github.com>
Co-authored-by: Seth Weidman <seth@sethweidman.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Ben <6579034+versipellis@users.noreply.github.com>
|
2025-02-20 04:23:28 -08:00 |
|
Daniel Han
|
c66350a48a
|
Memory efficient GRPO, DPO etc (#1716)
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Faster inference?
* Update llama.py
* Update llama.py
* Update utils.py
* Update llama.py
* Update llama.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update mapper.py
* Fast Inference via vLLM
* Update llama.py
* Update llama.py
* Update utils.py
* Create rl.py
* PatchRL
* Update rl.py
* Update rl.py
* Update rl.py
* PatchRLStatistics
* Update rl.py
* Update rl.py
* Update rl.py
* Update utils.py
* Update utils.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* RL metrics
* Update rl.py
* RL metrics
* Update __init__.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update chat_templates.py
* Update mapper.py
* Fp8 cache
* Update llama.py
* Update llama.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update __init__.py
* Update loader.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Better TRL handling
* Update rl.py
* Update tokenizer_utils.py
* Auto patching
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update rl.py
* Update tokenizer_utils.py
* Update rl.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update tokenizer_utils.py
* Update rl.py
* Update rl.py
* Update rl.py
* max seq length
* Update rl.py
* Update rl.py
* Patching
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* NEFTune
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Extra replacements
* Update rl_replacements.py
* Update rl.py
* extra RL replacements
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update _utils.py
* Update loader_utils.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* autocast
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update pyproject.toml
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* GRPO optimized
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Selective Log softmax
* Fix GRPO bsz
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Fix TRL
|
2025-02-15 01:19:39 -08:00 |
|
Daniel Han
|
26f8d8580b
|
Update pyproject.toml
|
2025-02-13 15:13:55 -08:00 |
|
Daniel Han
|
95fb1d699d
|
Fix bugs (#1701)
* Phi 4
* Update llama.py
* Torch.Cuda Is Available Condition and Warning (#1545)
* check for torch.cuda and triton if available
on my machine(mac m3) the cuda were not available
* Update pyproject.toml
* Update __init__.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update mistral.py
* Update mistral.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Fix
* Bug fixes
* Update mapper.py
* Add dropout to granite to match HF's implementation (#1557)
Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>
* Update llama.py
* Update llama.py
* Bug fixes
* fix: flash_attn_detection_error (#1556)
* fix: flash_attn_detection_error
* Update _utils.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update mapper.py
* Update gemma.py
* Update gemma.py
* Update gemma.py
* Update gemma.py
* dim fix
* Update _utils.py
* Torch 2.6 support
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Faster inference?
* Update llama.py
* Update llama.py
* Update utils.py
* Update llama.py
* Update llama.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update mapper.py
* Fast Inference via vLLM
* Update llama.py
* Update llama.py
* Update utils.py
* Create rl.py
* PatchRL
* Update rl.py
* Update rl.py
* Update rl.py
* PatchRLStatistics
* Update rl.py
* Update rl.py
* Update rl.py
* Update utils.py
* Update utils.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* RL metrics
* Update rl.py
* RL metrics
* Update __init__.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update chat_templates.py
* Update mapper.py
* Fp8 cache
* Update llama.py
* Update llama.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update __init__.py
* Update loader.py
* Update rl.py
* Update rl.py
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Better TRL handling
* Update rl.py
* Update tokenizer_utils.py
* Auto patching
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update rl.py
* Update tokenizer_utils.py
* Update rl.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update tokenizer_utils.py
* Update rl.py
* Update rl.py
* Update rl.py
* max seq length
* Update rl.py
* Update rl.py
* Patching
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* NEFTune
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Extra replacements
* Update rl_replacements.py
* Update rl.py
* extra RL replacements
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update _utils.py
* Update loader_utils.py
* Update rl.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* autocast
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update pyproject.toml
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update rl_replacements.py
* Update llama.py
* Update _utils.py
---------
Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>
Co-authored-by: AminWhat <88392440+aminwhat@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: Zhe Zhang <2631992879@qq.com>
|
2025-02-13 14:50:44 -08:00 |
|
Daniel Han
|
c8e4d4b767
|
Update
|
2025-02-06 03:23:54 -08:00 |
|
Daniel Han
|
e288d96272
|
Update pyproject.toml
|
2025-02-06 02:44:23 -08:00 |
|
Daniel Han
|
144190bd06
|
GRPO, vLLM, Bug Fixes, Reinforcement Learning (#1620)
* use exact model name
* Update save.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* print
* Update _utils.py
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update vision.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update loader.py
* accurate_accumulation
* Update loader.py
* Update loader.py
* Update _utils.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update pyproject.toml
* Update __init__.py
* Update pyproject.toml
* Update __init__.py
* Update __init__.py
* Fix Triton heuristics
https://github.com/triton-lang/triton/issues/5224
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Xformers
* Update loader.py
* Update loader.py
* Rewind
* Update _utils.py
* Update _utils.py
* requires grad
* Update loader.py
* Update _utils.py
* Update loader.py
* changing model to base_model if peft model is already used
* Improve debugging experience (#1512)
* Create CONTRIBUTING.md (#1472)
Creating contributing guidelines
* Update CONTRIBUTING.md
improved sentence
* Improve logging control in `unsloth_compile_transformers` by conditionally redirecting stdout based on UNSLOTH_DISABLE_LOGGER environment variable
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>
* Update loader.py
* Update llama.py
* Update llama.py
* Revert "Update llama.py"
This reverts commit a8edd0931a.
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Auto change is_bfloat16_supported
* Update llama.py
* Force data-type
* Update llama.py
* All attention refactor fix (#1491)
* change initilization of n_heads, n_kv_heads, hidden_size in llama.py
* do the same for cohere, mistral, gemma2, granite
* do the same for flexattention,cohere, mistral, granite
* Update llama.py
* Update llama.py
* Update granite to work with latest post_patch methods (#1502)
* Update granite to work with latest post_patch methods
* Pass position_embeddings for granite even if transformers<4.47
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Minor fixes for granite models (#1503)
* Update granite.py
Grab residual multiplier directly from layer
* Update llama.py
Version should read >= 4.47.1 as that is the version requiring the changes
* Update granite.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* support modelscope models and datasets (#1481)
* support modelscope
* change modelscope args
* remove useless import
* remove useless import
* fix
* wip
* fix
* remove useless code
* add readme
* add some comments
* change print to raise error
* update comment
* Update loader.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Merge branch 'main' into nightly
* Phi 4
* Update llama.py
* Torch.Cuda Is Available Condition and Warning (#1545)
* check for torch.cuda and triton if available
on my machine(mac m3) the cuda were not available
* Update pyproject.toml
* Update __init__.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update mistral.py
* Update mistral.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Fix
* Bug fixes
* Update mapper.py
* Add dropout to granite to match HF's implementation (#1557)
Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>
* Update llama.py
* Update llama.py
* Bug fixes
* fix: flash_attn_detection_error (#1556)
* fix: flash_attn_detection_error
* Update _utils.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update mapper.py
* Update gemma.py
* Update gemma.py
* Update gemma.py
* Update gemma.py
* dim fix
* Update _utils.py
* Torch 2.6 support
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Faster inference?
* Update llama.py
* Update llama.py
* Update utils.py
* Update llama.py
* Update llama.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update mapper.py
* Fast Inference via vLLM
* Update llama.py
* Update llama.py
* Update utils.py
* Create rl.py
* PatchRL
* Update rl.py
* Update rl.py
* Update rl.py
* PatchRLStatistics
* Update rl.py
* Update rl.py
* Update rl.py
* Update utils.py
* Update utils.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* RL metrics
* Update rl.py
* RL metrics
* Update __init__.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update chat_templates.py
* Update mapper.py
* Fp8 cache
* Update llama.py
* Update llama.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update __init__.py
* Update loader.py
---------
Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>
Co-authored-by: Itsuro Tajima <tajima@georepublic.de>
Co-authored-by: Muhammad Osama <muhammadosama1994@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>
Co-authored-by: Kareem <81531392+KareemMusleh@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: Z <coffeevampirebusiness@gmail.com>
Co-authored-by: tastelikefeet <58414341+tastelikefeet@users.noreply.github.com>
Co-authored-by: AminWhat <88392440+aminwhat@users.noreply.github.com>
Co-authored-by: Zhe Zhang <2631992879@qq.com>
|
2025-02-06 02:41:12 -08:00 |
|
Daniel Han
|
f90bd4ec49
|
Fix Mistral, Qwen (#1565)
* use exact model name
* Update save.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* print
* Update _utils.py
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update vision.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update loader.py
* accurate_accumulation
* Update loader.py
* Update loader.py
* Update _utils.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update pyproject.toml
* Update __init__.py
* Update pyproject.toml
* Update __init__.py
* Update __init__.py
* Fix Triton heuristics
https://github.com/triton-lang/triton/issues/5224
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Xformers
* Update loader.py
* Update loader.py
* Rewind
* Update _utils.py
* Update _utils.py
* requires grad
* Update loader.py
* Update _utils.py
* Update loader.py
* changing model to base_model if peft model is already used
* Improve debugging experience (#1512)
* Create CONTRIBUTING.md (#1472)
Creating contributing guidelines
* Update CONTRIBUTING.md
improved sentence
* Improve logging control in `unsloth_compile_transformers` by conditionally redirecting stdout based on UNSLOTH_DISABLE_LOGGER environment variable
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>
* Update loader.py
* Update llama.py
* Update llama.py
* Revert "Update llama.py"
This reverts commit a8edd0931a.
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Auto change is_bfloat16_supported
* Update llama.py
* Force data-type
* Update llama.py
* All attention refactor fix (#1491)
* change initilization of n_heads, n_kv_heads, hidden_size in llama.py
* do the same for cohere, mistral, gemma2, granite
* do the same for flexattention,cohere, mistral, granite
* Update llama.py
* Update llama.py
* Update granite to work with latest post_patch methods (#1502)
* Update granite to work with latest post_patch methods
* Pass position_embeddings for granite even if transformers<4.47
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Minor fixes for granite models (#1503)
* Update granite.py
Grab residual multiplier directly from layer
* Update llama.py
Version should read >= 4.47.1 as that is the version requiring the changes
* Update granite.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* support modelscope models and datasets (#1481)
* support modelscope
* change modelscope args
* remove useless import
* remove useless import
* fix
* wip
* fix
* remove useless code
* add readme
* add some comments
* change print to raise error
* update comment
* Update loader.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Merge branch 'main' into nightly
* Phi 4
* Update llama.py
* Torch.Cuda Is Available Condition and Warning (#1545)
* check for torch.cuda and triton if available
on my machine(mac m3) the cuda were not available
* Update pyproject.toml
* Update __init__.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update mistral.py
* Update mistral.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Fix
* Bug fixes
* Update mapper.py
* Add dropout to granite to match HF's implementation (#1557)
Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>
* Update llama.py
* Update llama.py
* Bug fixes
* fix: flash_attn_detection_error (#1556)
* fix: flash_attn_detection_error
* Update _utils.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
---------
Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>
Co-authored-by: Itsuro Tajima <tajima@georepublic.de>
Co-authored-by: Muhammad Osama <muhammadosama1994@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>
Co-authored-by: Kareem <81531392+KareemMusleh@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: Z <coffeevampirebusiness@gmail.com>
Co-authored-by: tastelikefeet <58414341+tastelikefeet@users.noreply.github.com>
Co-authored-by: AminWhat <88392440+aminwhat@users.noreply.github.com>
Co-authored-by: Zhe Zhang <2631992879@qq.com>
|
2025-01-20 01:27:24 -08:00 |
|
Daniel Han
|
e42bd98706
|
Update Unsloth-Zoo
|
2025-01-08 16:46:04 -08:00 |
|
Daniel Han
|
d90aefea98
|
Update pyproject.toml
|
2025-01-07 04:29:09 -08:00 |
|
Daniel Han
|
63782ea3af
|
Bug fixes (#1516)
* use exact model name
* Update save.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* print
* Update _utils.py
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update vision.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update loader.py
* accurate_accumulation
* Update loader.py
* Update loader.py
* Update _utils.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update pyproject.toml
* Update __init__.py
* Update pyproject.toml
* Update __init__.py
* Update __init__.py
* Fix Triton heuristics
https://github.com/triton-lang/triton/issues/5224
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Xformers
* Update loader.py
* Update loader.py
* Rewind
* Update _utils.py
* Update _utils.py
* requires grad
* Update loader.py
* Update _utils.py
* Update loader.py
* changing model to base_model if peft model is already used
* Improve debugging experience (#1512)
* Create CONTRIBUTING.md (#1472)
Creating contributing guidelines
* Update CONTRIBUTING.md
improved sentence
* Improve logging control in `unsloth_compile_transformers` by conditionally redirecting stdout based on UNSLOTH_DISABLE_LOGGER environment variable
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>
* Update loader.py
* Update llama.py
* Update llama.py
* Revert "Update llama.py"
This reverts commit b7ddf962d2.
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Auto change is_bfloat16_supported
* Update llama.py
* Force data-type
* Update llama.py
* All attention refactor fix (#1491)
* change initilization of n_heads, n_kv_heads, hidden_size in llama.py
* do the same for cohere, mistral, gemma2, granite
* do the same for flexattention,cohere, mistral, granite
* Update llama.py
* Update llama.py
* Update granite to work with latest post_patch methods (#1502)
* Update granite to work with latest post_patch methods
* Pass position_embeddings for granite even if transformers<4.47
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Minor fixes for granite models (#1503)
* Update granite.py
Grab residual multiplier directly from layer
* Update llama.py
Version should read >= 4.47.1 as that is the version requiring the changes
* Update granite.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* support modelscope models and datasets (#1481)
* support modelscope
* change modelscope args
* remove useless import
* remove useless import
* fix
* wip
* fix
* remove useless code
* add readme
* add some comments
* change print to raise error
* update comment
* Update loader.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
---------
Co-authored-by: Itsuro Tajima <tajima@georepublic.de>
Co-authored-by: Muhammad Osama <muhammadosama1994@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>
Co-authored-by: Kareem <81531392+KareemMusleh@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: Z <coffeevampirebusiness@gmail.com>
Co-authored-by: tastelikefeet <58414341+tastelikefeet@users.noreply.github.com>
|
2025-01-07 04:23:14 -08:00 |
|
Daniel Han
|
e254125954
|
Bug fixes (#1484)
* Update save.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* print
* Update _utils.py
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update vision.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update loader.py
* accurate_accumulation
* Update loader.py
* Update loader.py
* Update _utils.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update pyproject.toml
|
2024-12-29 03:57:31 -08:00 |
|
Daniel Han
|
da4741bbef
|
Update pyproject.toml
|
2024-12-26 04:12:46 -08:00 |
|
Daniel Han
|
47f210922d
|
Update pyproject.toml
|
2024-12-26 04:04:23 -08:00 |
|
Daniel Han
|
30bb143a87
|
Update pyproject.toml
|
2024-12-26 03:26:01 -08:00 |
|
Daniel Han
|
258d8135e7
|
Update pyproject.toml
|
2024-12-24 03:35:04 -08:00 |
|
Daniel Han
|
fd2295d6a6
|
Update pyproject.toml
|
2024-12-24 01:55:04 -08:00 |
|
Daniel Han
|
77ca6e2dca
|
Triton windows
|
2024-12-24 00:17:33 -08:00 |
|
Daniel Han
|
f97f49b08d
|
Update pyproject.toml
|
2024-12-24 00:08:49 -08:00 |
|
Daniel Han
|
660df7b9ab
|
Update pyproject.toml
|
2024-12-24 00:08:03 -08:00 |
|
Daniel Han
|
61147bfbad
|
Update pyproject.toml
|
2024-12-24 00:07:37 -08:00 |
|
Daniel Han
|
6e6e65bd46
|
Update pyproject.toml
|
2024-12-24 00:02:30 -08:00 |
|
Daniel Han
|
e6d50c8839
|
Update pyproject.toml
|
2024-12-23 22:49:16 -08:00 |
|
Daniel Han
|
5323157160
|
Update pyproject.toml
|
2024-12-23 22:47:20 -08:00 |
|
Daniel Han
|
fba4cf7fb4
|
Update pyproject.toml
|
2024-12-23 22:47:11 -08:00 |
|
Daniel Han
|
753b954702
|
dependencies
|
2024-12-23 22:27:35 -08:00 |
|
Daniel Han
|
c939334570
|
Update pyproject.toml
|
2024-12-23 21:53:43 -08:00 |
|
Daniel Han
|
01b256b4cf
|
Update pyproject.toml
|
2024-12-23 21:52:50 -08:00 |
|
Daniel Han
|
4cc97b23cc
|
Version
|
2024-12-23 21:43:32 -08:00 |
|
Daniel Han
|
2f969fa137
|
Update pyproject.toml
|
2024-12-23 21:16:30 -08:00 |
|
Daniel Han
|
25c5b0524d
|
Update pyproject.toml
|
2024-12-23 21:14:01 -08:00 |
|
Daniel Han
|
69dc1ad694
|
Version
|
2024-12-23 21:07:51 -08:00 |
|
Daniel Han
|
e8ae4727d0
|
Update pyproject.toml
|
2024-12-21 01:24:09 -08:00 |
|
Daniel Han
|
58bcf66d27
|
Temp fix
|
2024-12-20 03:07:04 -08:00 |
|
Daniel Han
|
8144766f78
|
Update pyproject.toml
|
2024-11-26 03:29:59 -08:00 |
|
Daniel Han
|
4fe74c93ee
|
Update pyproject.toml
|
2024-11-21 17:46:22 -08:00 |
|
Daniel Han
|
15268ba184
|
Bug fixes (#1245)
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165)
* chore: update chat_templates.py (#1166)
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180)
* Fix DPO, ORPO (#1177)
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165)
* chore: update chat_templates.py (#1166)
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* donot upcast lm_head and embeddings to float32 (#1186)
* Cleanup upcast logs (#1188)
* Fix/phi-longrope (#1193)
* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding
* Typo
* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache
* Update llama.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update transformers
* Unk token issues
* Update _utils.py
* Fix pad token
* Update llama.py
* Typo
* ignored labels
* Revert "ignored labels"
This reverts commit 9d07be077b.
* More patching
* Update _utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Feat/all tmp (#1219)
* Update save.py
Check whether path is in /tmp dir for Kaggle environment
* Update save.py
Move temporary_location to /tmp in Kaggle
* Enhance Kaggle environment support in save and tokenizer utilities
---------
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
* Bug fixes
* Update pyproject.toml
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Tied weights
* Revert "Tied weights"
This reverts commit 8090b7c01a.
* Tied weights
* Utils
* CE Loss patching
* Update __init__.py
* Update __init__.py
* Patching
* Update cross_entropy_loss.py
* CE Loss
* Update _utils.py
* Update _utils.py
* CE Loss
* Update _utils.py
* Update _utils.py
* Layernorm
* Update _utils.py
* Update _utils.py
* Post patch
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* typing
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* int64
* Update _utils.py
* Update cross_entropy_loss.py
* constexpr
* constexpr
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* CE
* Update cross_entropy_loss.py
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update utils.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* typing
* Update rope_embedding.py
* types
* Disable compiling
* Update _utils.py
* Update _utils.py
* Forward hook
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update llama.py
* Update llama.py
* Update _utils.py
* Update pyproject.toml
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
|
2024-11-05 13:29:37 -08:00 |
|
Daniel Han
|
9d5f58224d
|
Update pyproject.toml
|
2024-10-26 18:05:55 -07:00 |
|
Daniel Han
|
e7ede2f7db
|
Torch 2.5
|
2024-10-26 18:03:15 -07:00 |
|
Daniel Han
|
4c85177719
|
Many bug fixes (#1162)
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165)
* chore: update chat_templates.py (#1166)
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
|
2024-10-23 03:14:57 -07:00 |
|
Daniel Han
|
d2a032e117
|
Gradient Accumulation Fix (#1134)
* Unsloth Zoo
* Update trainer.py
* Update trainer.py
* Update cross_entropy_loss.py
* n_items
* Update llama.py
* kwargs
* Remove extraneous f prefixes (#1133)
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
* Update __init__.py
---------
Co-authored-by: Emil Sadek <esadek@hotmail.com>
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
|
2024-10-14 19:17:35 -07:00 |
|
Daniel Han
|
e130e748f0
|
Reload
|
2024-10-05 17:21:48 -07:00 |
|
Daniel Han
|
248c27d205
|
Fix merges (#1079)
* Layernorm
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Update layernorm.py
* Patch layernorm
* Update layernorm.py
* RMS Layernorm
* Update rms_layernorm.py
* Causal LM
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update layernorm.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* Llama 3.2
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update llama.py
* Update vision.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update loader.py
* Update loader.py
* Update loader.py
* Dependencies
* Update pyproject.toml
* Update _utils.py
|
2024-09-30 03:03:01 -07:00 |
|
Daniel Han
|
b314837622
|
Update pyproject.toml
|
2024-09-27 01:36:45 -07:00 |
|
Daniel Han
|
930d2ad1a8
|
Update pyproject.toml
|
2024-09-25 23:47:15 -07:00 |
|
Daniel Han
|
5b345ec757
|
Update pyproject.toml
|
2024-09-25 23:13:49 -07:00 |
|
Daniel Han
|
1397f2e1ab
|
Torch 2.4, Xformers>0.0.27, TRL>0.9, Python 3.12 + bug fixes (#902)
* Update pyproject.toml
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update _utils.py
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* fix_tokenizer
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update save.py
* Update save.py
* Update save.py
* Update save.py
* Update save.py
* Update loader.py
* Update pyproject.toml
* Update _utils.py
* Update gemma2.py
* Update gemma2.py
* Update _utils.py
* gemma 2 mask
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update _utils.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update _utils.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Torch 2.4 Xformers 0.0.27post2
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Gemma 2 fixes
* Update gemma2.py
* Update llama.py
* Update llama.py
* Update save.py
* Update save.py
|
2024-08-10 19:59:40 -07:00 |
|
Daniel Han
|
734e478605
|
Fix tokenizers (#887)
* Update pyproject.toml
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update _utils.py
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* fix_tokenizer
* Update tokenizer_utils.py
* Update tokenizer_utils.py
|
2024-08-06 20:24:44 -07:00 |
|