Daniel Han
|
d1f3b6c1c4
|
Create Run.png
|
2024-07-23 13:14:21 -07:00 |
|
Daniel Han
|
cc11b78861
|
Update llama.py
|
2024-07-23 12:28:12 -07:00 |
|
Daniel Han
|
4dd4ad2104
|
Update llama.py
|
2024-07-23 12:27:46 -07:00 |
|
Daniel Han
|
caa4028287
|
Update _utils.py
|
2024-07-23 12:25:24 -07:00 |
|
Daniel Han
|
777453967f
|
Update loader.py
|
2024-07-23 12:12:29 -07:00 |
|
Daniel Han
|
824511e265
|
Update README.md
|
2024-07-23 12:07:27 -07:00 |
|
Daniel Han
|
22968a2134
|
Update README.md
|
2024-07-23 11:51:08 -07:00 |
|
Daniel Han
|
c86b13d465
|
Llama 3.1 (#797)
* Llama 3.1
* Update _utils.py
* Llama 3.1
* Update _utils.py
* Update llama.py
* Update llama.py
* hack for rotary
* patch RoPE
* refix rope
* Update _utils.py
* Update llama.py
* Llama 3.1 check
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
|
2024-07-23 11:40:49 -07:00 |
|
Daniel Han
|
e41cc4093c
|
Update tokenizer_utils.py
|
2024-07-20 13:25:59 -07:00 |
|
Daniel Han
|
5f496efdb4
|
Update tokenizer_utils.py
|
2024-07-20 13:22:36 -07:00 |
|
Daniel Han
|
ba515ec92d
|
Update llama.py
|
2024-07-20 12:47:36 -07:00 |
|
Daniel Han
|
72e1b03544
|
Update llama.py
|
2024-07-20 11:53:32 -07:00 |
|
Daniel Han
|
ea250d7b3b
|
Merge branch 'main' into nightly
|
2024-07-20 09:47:22 -07:00 |
|
Daniel Han
|
182ab7e0cb
|
Update mistral.py
|
2024-07-19 09:32:27 -07:00 |
|
Daniel Han
|
565a5a3894
|
Fix Gemma
|
2024-07-19 09:27:18 -07:00 |
|
Daniel Han
|
568bfddfc3
|
Update README.md
|
2024-07-19 03:05:15 -07:00 |
|
Daniel Han
|
39885a4ddc
|
Update README.md
|
2024-07-19 03:03:50 -07:00 |
|
Daniel Han
|
dd201f57b1
|
Nightly (#784)
* Update __init__.py
* dynamic RoPE
* Update mistral.py
* Update llama.py
* Update tokenizer_utils.py
* Update mistral.py
* Update llama.py
* Update __init__.py
* Update flex_attention.py
* Update llama.py
* Update llama.py
* Mistral Nemo
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
|
2024-07-19 01:39:08 -07:00 |
|
Daniel Han
|
4a432fa5bc
|
Merge branch 'main' into nightly
|
2024-07-19 01:38:47 -07:00 |
|
Daniel Han
|
8ee997cac2
|
Update tokenizer_utils.py
|
2024-07-19 01:29:52 -07:00 |
|
Daniel Han
|
81e1bedf0a
|
Nightly (#783)
* Update __init__.py
* dynamic RoPE
* Update mistral.py
* Update llama.py
* Update tokenizer_utils.py
* Update mistral.py
* Update llama.py
* Update __init__.py
* Update flex_attention.py
* Update llama.py
* Update llama.py
* Mistral Nemo
* Update tokenizer_utils.py
* Update tokenizer_utils.py
|
2024-07-19 01:24:46 -07:00 |
|
Daniel Han
|
ad3d38ad4d
|
Update tokenizer_utils.py
|
2024-07-19 01:06:41 -07:00 |
|
Daniel Han
|
10c13545c3
|
Update tokenizer_utils.py
|
2024-07-19 01:03:51 -07:00 |
|
Daniel Han
|
7cc2c6f563
|
Merge branch 'main' into nightly
|
2024-07-19 00:59:48 -07:00 |
|
Daniel Han
|
009de39800
|
Update tokenizer_utils.py
|
2024-07-19 00:41:35 -07:00 |
|
Daniel Han
|
6c3573990c
|
Mistral Nemo (#782)
* Update __init__.py
* dynamic RoPE
* Update mistral.py
* Update llama.py
* Update tokenizer_utils.py
* Update mistral.py
* Update llama.py
* Update __init__.py
* Update flex_attention.py
* Update llama.py
* Update llama.py
* Mistral Nemo
|
2024-07-19 00:14:24 -07:00 |
|
Daniel Han
|
c1d3493704
|
Mistral Nemo
|
2024-07-18 22:53:23 -07:00 |
|
Daniel Han
|
1524504624
|
Update llama.py
|
2024-07-18 22:07:23 -07:00 |
|
Daniel Han
|
477793753f
|
Update llama.py
|
2024-07-18 21:57:33 -07:00 |
|
Daniel Han
|
d824ef8982
|
Merge branch 'main' into nightly
|
2024-07-18 21:55:10 -07:00 |
|
Daniel Han
|
e4598d01b2
|
Fix bugs (#779)
* Update __init__.py
* dynamic RoPE
* Update mistral.py
* Update llama.py
* Update tokenizer_utils.py
* Update mistral.py
* Update llama.py
* Update __init__.py
* Update flex_attention.py
|
2024-07-18 18:19:24 -07:00 |
|
Daniel Han
|
2a251ec594
|
Update flex_attention.py
|
2024-07-18 18:18:09 -07:00 |
|
Daniel Han
|
ed56977a8c
|
Update __init__.py
|
2024-07-18 14:43:03 -07:00 |
|
Daniel Han
|
983c2b601a
|
Update llama.py
|
2024-07-18 14:31:35 -07:00 |
|
Daniel Han
|
fbf6cc747e
|
Update mistral.py
|
2024-07-18 13:33:13 -07:00 |
|
Daniel Han
|
40aeb2629e
|
Update tokenizer_utils.py
|
2024-07-18 13:25:30 -07:00 |
|
Daniel Han
|
a56b2d45c9
|
Update llama.py
|
2024-07-18 12:33:22 -07:00 |
|
Daniel Han
|
6c8618c754
|
Update mistral.py
|
2024-07-18 12:08:49 -07:00 |
|
Daniel Han
|
54dfb1a9e1
|
dynamic RoPE
|
2024-07-18 12:06:38 -07:00 |
|
Daniel Han
|
7eb34655a5
|
Update __init__.py
|
2024-07-18 11:07:32 -07:00 |
|
Daniel Han
|
aed740060a
|
Update pyproject.toml
|
2024-07-18 10:59:09 -07:00 |
|
Daniel Han
|
2cf7f0e7cc
|
Update __init__.py
|
2024-07-18 10:58:12 -07:00 |
|
Daniel Han
|
ccd0c8ea9b
|
Mistral Nemo 12b (#777)
* Update gemma2.py
* Update llama.py
* Update llama.py
* Update gemma2.py
* init
* Update gemma2.py
* Update gemma2.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* All RoPE Scaling support
* cleanup
* Update llama.py
* Update llama.py
* Update _utils.py
* Update _utils.py
* exec
* exec
* Attention_Module
* attention_module
* imports
* exec
* Update llama.py
* Update llama.py
* boolean mask
* revert masking
* Update llama.py
* Update save.py
* Update llama.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update utils.py
* retry
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update _utils.py
* Update _utils.py
* Update gemma2.py
* Update chat_templates.py
* Gemma 2 Ollama support
* Update llama.py
* Update llama.py
* error handling
* Update _utils.py
* Update _utils.py
* Stats for debugging
* Update _utils.py
* Update _utils.py
* Debugging
* Update tokenizer_utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Check exec, eval
* Update _utils.py
* Update _utils.py
* Images
* Bug fixes
* Update pyproject.toml
* Bug fixes
* Update _utils.py
* Update _utils.py
* Deprecation fix
* Update chat_templates.py
* Now permitting use of pre-installed llama.cpp (#763)
* Now permitting use of pre-installed llama.cpp
* Update save.py
---------
Co-authored-by: Giuseppe Strafforello <giuseppe.strafforello@titantechnologies.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update save.py
* Deprecation & compile
* typo
* Update chat_templates.py
* Update chat_templates.py
* train_on_responses_only
* Update llama.py
* Update llama.py
* Update save.py
* Update gemma2.py
* Flex Attention
* typos
* Update _utils.py
* Update llama.py
* Update __init__.py
* Update flex_attention.py
* Update llama.py
* Update llama.py
* emulation
* Update __init__.py
* Update rope_embedding.py
* Update flex_attention.py
* Update flex_attention.py
* Update rope_embedding.py
* libdevice
* triton_tanh
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* score
* Update llama.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update llama.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Flex Attention removal
* upload tensorboard training stats to hub if available (#773)
* causal_mask
* Update llama.py
* Update llama.py
* Update flex_attention.py
* Update _utils.py
* Update mapper.py
* Update _utils.py
---------
Co-authored-by: pepistrafforello <pepi.strafforello@gmail.com>
Co-authored-by: Giuseppe Strafforello <giuseppe.strafforello@titantechnologies.com>
Co-authored-by: Sébastien De Greef <sebdg@binarycompany.com>
|
2024-07-18 10:51:10 -07:00 |
|
Daniel Han
|
0f2e484f39
|
Chat templates
|
2024-07-15 14:36:44 -07:00 |
|
Daniel Han
|
e32fc24088
|
Train on responses only (#770)
* Update gemma2.py
* Update llama.py
* Update llama.py
* Update gemma2.py
* init
* Update gemma2.py
* Update gemma2.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* All RoPE Scaling support
* cleanup
* Update llama.py
* Update llama.py
* Update _utils.py
* Update _utils.py
* exec
* exec
* Attention_Module
* attention_module
* imports
* exec
* Update llama.py
* Update llama.py
* boolean mask
* revert masking
* Update llama.py
* Update save.py
* Update llama.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update utils.py
* retry
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update _utils.py
* Update _utils.py
* Update gemma2.py
* Update chat_templates.py
* Gemma 2 Ollama support
* Update llama.py
* Update llama.py
* error handling
* Update _utils.py
* Update _utils.py
* Stats for debugging
* Update _utils.py
* Update _utils.py
* Debugging
* Update tokenizer_utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Check exec, eval
* Update _utils.py
* Update _utils.py
* Images
* Bug fixes
* Update pyproject.toml
* Bug fixes
* Update _utils.py
* Update _utils.py
* Deprecation fix
* Update chat_templates.py
* Now permitting use of pre-installed llama.cpp (#763)
* Now permitting use of pre-installed llama.cpp
* Update save.py
---------
Co-authored-by: Giuseppe Strafforello <giuseppe.strafforello@titantechnologies.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update save.py
* Deprecation & compile
* typo
* Update chat_templates.py
* Update chat_templates.py
* train_on_responses_only
* Update llama.py
* Update llama.py
* Update save.py
* Update gemma2.py
---------
Co-authored-by: pepistrafforello <pepi.strafforello@gmail.com>
Co-authored-by: Giuseppe Strafforello <giuseppe.strafforello@titantechnologies.com>
|
2024-07-14 22:41:04 -07:00 |
|
Daniel Han
|
92dce38e8b
|
Many bug fixes (#754)
* Update gemma2.py
* Update llama.py
* Update llama.py
* Update gemma2.py
* init
* Update gemma2.py
* Update gemma2.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* All RoPE Scaling support
* cleanup
* Update llama.py
* Update llama.py
* Update _utils.py
* Update _utils.py
* exec
* exec
* Attention_Module
* attention_module
* imports
* exec
* Update llama.py
* Update llama.py
* boolean mask
* revert masking
* Update llama.py
* Update save.py
* Update llama.py
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update utils.py
* retry
* Update gemma2.py
* Update gemma2.py
* Update gemma2.py
* Update _utils.py
* Update _utils.py
* Update gemma2.py
* Update chat_templates.py
* Gemma 2 Ollama support
* Update llama.py
* Update llama.py
* error handling
* Update _utils.py
* Update _utils.py
* Stats for debugging
* Update _utils.py
* Update _utils.py
* Debugging
* Update tokenizer_utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Check exec, eval
* Update _utils.py
* Update _utils.py
* Images
* Bug fixes
* Update pyproject.toml
* Bug fixes
* Update _utils.py
* Update _utils.py
|
2024-07-10 01:59:06 -07:00 |
|
Daniel Han
|
f1dc078c26
|
Update llama.py
|
2024-07-08 10:44:19 -07:00 |
|
Daniel Han
|
4ae30315e3
|
Update llama.py
|
2024-07-08 10:38:49 -07:00 |
|
Daniel Han
|
ea330bdc73
|
Update _utils.py
|
2024-07-08 10:01:20 -07:00 |
|
Daniel Han
|
36488b5867
|
Update llama.py
|
2024-07-07 15:46:36 -07:00 |
|