Squashed commit of the following:

commit 61d45c60db
Merge: f024ce2 13a7cbf
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 17 22:12:44 2024 +1100

    Merge branch 'main' into nightly

commit f024ce2821
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 17 22:12:18 2024 +1100

    Update __init__.py

commit d38cf5387c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 17 22:09:23 2024 +1100

    Update fast_lora.py

commit 9c35a2c4b0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 17 22:08:33 2024 +1100

    Update pyproject.toml

commit 6edc35f686
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 17 20:18:00 2024 +1100

    Update fast_lora.py

commit 2a9d4fb947
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 17 20:10:30 2024 +1100

    Bugs

commit 14717be070
Merge: 5c24a3b cb12179
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 17 20:10:26 2024 +1100

    Merge branch 'main' into nightly

commit 5c24a3bc2e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 17 02:46:35 2024 +1100

    Update README.md

commit fd729a7131
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 17 02:44:58 2024 +1100

    Update README.md

commit 7e9f092e9f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 16 23:17:35 2024 +1100

    Update save.py

commit e3efca8778
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 16 22:36:57 2024 +1100

    Update save.py

commit d58fa31e0c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 16 22:31:02 2024 +1100

    Update save.py

commit 64d954bd13
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 16 20:32:12 2024 +1100

    Update save.py

commit 815202f832
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 16 20:03:51 2024 +1100

    GGUF

commit 338b2c928b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 16 04:40:21 2024 +1100

    Update README.md

commit f342425c1e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 16 04:34:51 2024 +1100

    Update README.md

commit cef733a420
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 16 04:10:54 2024 +1100

    Update fast_lora.py

commit e5bcab2a74
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 16 03:43:16 2024 +1100

    Update fast_lora.py

commit 80cfe132f6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 16 03:38:13 2024 +1100

    Fix bugs

commit d8e98be90d
Merge: 51c2484 1e61cdb
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 16 00:14:02 2024 +1100

    Merge branch 'main' into nightly

commit 51c2484ffd
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Mar 15 22:35:44 2024 +1100

    Update rope_embedding.py

commit 3e93a78794
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Mar 15 20:14:56 2024 +1100

    Update rope_embedding.py

commit 82d80e9e98
Merge: 718a6a1 9ac4ed6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Mar 15 19:50:20 2024 +1100

    Merge branch 'main' into nightly

commit 718a6a1c84
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Mar 15 19:50:03 2024 +1100

    Update pyproject.toml

commit 385c6d44a8
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Mar 15 19:49:09 2024 +1100

    Update pyproject.toml

commit 9c9ede4680
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Mar 15 19:48:39 2024 +1100

    Update pyproject.toml

commit 9d6c9c9ebc
Merge: a12e4ea 42076f6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Mar 15 05:07:09 2024 +1100

    Merge branch 'main' into nightly

commit a12e4ea3e0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Mar 15 04:44:54 2024 +1100

    Update chat_templates.py

commit c3e0e518d9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Mar 15 04:30:03 2024 +1100

    Update chat_templates.py

commit fe0e2d7baa
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Mar 15 03:50:32 2024 +1100

    Update chat_templates.py

commit 5dfe582c96
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Mar 15 03:37:37 2024 +1100

    Update chat_templates.py

commit ec73a776ec
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Mar 15 03:34:32 2024 +1100

    Update chat_templates.py

commit 5c4241a240
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Mar 14 20:27:20 2024 +1100

    Update pyproject.toml

commit 6841a303d9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Mar 14 20:20:16 2024 +1100

    Update pyproject.toml

commit 341b5f46e6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Mar 14 20:12:54 2024 +1100

    Update pyproject.toml

commit 6809846464
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Mar 14 20:12:02 2024 +1100

    Update pyproject.toml

commit 77edfb10b7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Mar 14 20:11:24 2024 +1100

    Update pyproject.toml

commit 0ed19ec170
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Mar 14 20:10:10 2024 +1100

    Update pyproject.toml

commit 1f4c625f2a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Mar 14 20:06:05 2024 +1100

    Update pyproject.toml

commit 8fa0aab61c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Mar 14 20:04:01 2024 +1100

    Update pyproject.toml

commit eb61377632
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Mar 14 19:02:29 2024 +1100

    Fix Colab

commit d6ac9b56c1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Mar 12 20:12:55 2024 +1100

    upcasting

commit b7c3190e97
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Mar 12 00:46:57 2024 +1100

    Update pyproject.toml

commit 4d77c32425
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 23:28:03 2024 +1100

    Update pyproject.toml

commit 98573e8618
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 23:21:27 2024 +1100

    kaggle new

commit 4a794dd880
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 20:28:09 2024 +1100

    Update pyproject.toml

commit a0c18c9880
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 20:05:06 2024 +1100

    Update save.py

commit 684eaae9ea
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 19:52:18 2024 +1100

    GGUF incorrect

commit 029d588603
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 19:13:42 2024 +1100

    Update save.py

commit 252a38a2df
Merge: 63b1f58 dd72d9f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 19:13:29 2024 +1100

    Merge branch 'main' into nightly

commit 63b1f5879e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 04:05:43 2024 +1100

    Update llama.py

commit cb0d937469
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 03:53:23 2024 +1100

    Account for DoRA

commit 2e87755a5c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 03:19:20 2024 +1100

    Update llama.py

commit 93d88ad68d
Merge: aba595d e375c58
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 03:15:30 2024 +1100

    Merge branch 'main' into nightly

commit aba595de9c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 03:07:44 2024 +1100

    Update llama.py

commit 133097d995
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 02:25:10 2024 +1100

    Update save.py

commit b5d3d63df5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 02:15:33 2024 +1100

    Update save.py

commit 8be91b050f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 11 02:10:51 2024 +1100

    Update chat_templates.py

commit 23b7a5764c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 19:32:02 2024 +1100

    Update fast_lora.py

commit c1728a9904
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 19:11:00 2024 +1100

    Update fast_lora.py

commit c192ce3ed4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 19:09:10 2024 +1100

    Update fast_lora.py

commit 7e3abd19ba
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 18:36:35 2024 +1100

    Update fast_lora.py

commit c1f3e70394
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 18:33:52 2024 +1100

    Update fast_lora.py

commit 08da057f04
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 18:09:58 2024 +1100

    Update save.py

commit 1e8922af2b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 17:42:41 2024 +1100

    Revert

commit 74fc5caa60
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 16:20:20 2024 +1100

    Accuracy

commit 35c6d776c4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 14:13:53 2024 +1100

    Update save.py

commit 6d2bc97117
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 14:08:38 2024 +1100

    Update llama.py

commit baf8e4c0a8
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 13:11:42 2024 +1100

    Update llama.py

commit c0d9516255
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 13:10:03 2024 +1100

    Update llama.py

commit 91877f5506
Merge: f887080 a2215fd
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 13:08:23 2024 +1100

    Merge branch 'main' into nightly

commit f887080a6d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 04:31:40 2024 +1100

    Tokenizer overwritten

commit 1c1461ae09
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 04:08:02 2024 +1100

    Update loader.py

commit 14f063819a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 03:54:18 2024 +1100

    model_name

commit daba749ee1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 03:48:05 2024 +1100

    Update llama.py

commit 457b7ba6d6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 02:57:41 2024 +1100

    Update chat_templates.py

commit 5c9629f5fe
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 10 02:55:07 2024 +1100

    Update save.py

commit e6c3cdfc2d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 9 23:22:14 2024 +1100

    Update save.py

commit f0a3c05b07
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 9 23:19:02 2024 +1100

    Update save.py

commit 58d1f1e03c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 9 20:05:52 2024 +1100

    Update chat_templates.py

commit 9b3dd3e9f8
Merge: b321ada a0cc0d1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Mar 8 19:40:25 2024 +1100

    Merge branch 'main' into nightly

commit b321adac2c
Merge: 03232df d862d05
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Mar 7 04:32:51 2024 +1100

    Merge branch 'main' into nightly

commit 03232dff4e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Mar 7 02:48:15 2024 +1100

    Update chat_templates.py

commit e67c6b4ec9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Mar 7 02:05:38 2024 +1100

    Fix warning

commit 5a7f52819e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Mar 6 19:21:44 2024 +1100

    Update rms_layernorm.py

commit 837ba610cf
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Mar 6 18:18:07 2024 +1100

    RoPE and Gemma precision

commit 1e41fa0c8c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Mar 6 05:07:37 2024 +1100

    Update save.py

commit 333d3d9a51
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Mar 6 04:52:59 2024 +1100

    Update gemma.py

commit 85c052d8fb
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Mar 6 04:48:26 2024 +1100

    sqrt

commit ed1aa0007a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Mar 6 04:24:10 2024 +1100

    Update gemma.py

commit be81c07d43
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Mar 6 04:14:11 2024 +1100

    Gemma precision

commit 5a693a107d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Mar 5 23:58:00 2024 +1100

    Layernorms

commit e0a2463738
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Mar 5 20:07:44 2024 +1100

    Update pyproject.toml

commit 160320b3d9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Mar 5 18:55:50 2024 +1100

    Update gemma.py

commit 9f7f205f67
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Mar 5 18:27:11 2024 +1100

    Update rms_layernorm.py

commit 43e710b064
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Mar 5 18:21:59 2024 +1100

    Fix Gemma merging

commit 3137392ccf
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Mar 5 02:41:56 2024 +1100

    Update gemma.py

commit fe35b127fc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Mar 5 02:40:20 2024 +1100

    Update gemma.py

commit 5fae81b628
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Mar 5 02:22:31 2024 +1100

    Update gemma.py

commit 961813b35f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Mar 5 01:45:08 2024 +1100

    Update gemma.py

commit c027dacb18
Merge: cb193f7 b4fe3cd
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 4 16:19:13 2024 +1100

    Merge branch 'main' into nightly

commit cb193f7e0a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 4 16:14:50 2024 +1100

    Update gemma.py

commit 440c29273f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 4 16:11:12 2024 +1100

    Update rms_layernorm.py

commit c31b27b7dc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 4 16:04:29 2024 +1100

    Update rms_layernorm.py

commit 6fa081ae77
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Mar 4 16:00:35 2024 +1100

    Update rms_layernorm.py

commit aa2fb63048
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 3 19:32:18 2024 +1100

    Update gemma.py

commit ac23e4b612
Merge: 1fea4ff 528d5fb
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 3 19:29:41 2024 +1100

    Merge branch 'main' into nightly

commit 1fea4ffcf2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 3 18:07:55 2024 +1100

    Update geglu.py

commit 245fe4716c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 3 03:39:36 2024 +1100

    Update _utils.py

commit e27523cd5e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 3 03:39:10 2024 +1100

    Update __init__.py

commit 65fbde6cd9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 3 03:37:43 2024 +1100

    Update __init__.py

commit 9a2e791e6c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 3 03:35:21 2024 +1100

    Update llama.py

commit 786885c38c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 3 03:17:59 2024 +1100

    Approx gelu

commit 1c7f0d21ee
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Mar 3 02:24:39 2024 +1100

    Update geglu.py

commit 393e53b016
Merge: c88ab10 dbba69b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 2 18:29:02 2024 +1100

    Merge branch 'main' into nightly

commit c88ab10a5c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 2 18:28:39 2024 +1100

    Approx gelu

commit e032445694
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 2 02:39:11 2024 +1100

    Update pyproject.toml

commit c970a2b3be
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Mar 2 02:38:40 2024 +1100

    Small fixes

commit db87262625
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Mar 1 04:16:25 2024 +1100

    Update pyproject.toml

commit d44bbf5f2e
Merge: 0866020 d0c15bb
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 29 00:17:47 2024 +1100

    Merge branch 'nightly' of https://github.com/unslothai/unsloth into nightly

commit 0866020037
Merge: 54b26c0 1fe11ba
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 29 00:17:12 2024 +1100

    Merge branch 'main' into nightly

commit d0c15bb508
Author: Daniel Han <danielhanchen@gmail.com>
Date:   Thu Feb 29 00:17:03 2024 +1100

    Hotfix - fix DoRA, Gemma prompt template (#202) (#203)

    * Update save.py

    * saving

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update save.py

    * Update __init__.py

    * Update save.py

    * Update save.py

    * Update save.py

    * save

    * trainer

    * spaces

    * original

    * Gemma

    * Update pyproject.toml

    * Update mapper.py

    * Update fast_lora.py

    * FastGemmaModel

    * model_type

    * Update llama.py

    * Update llama.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update llama.py

    * Update llama.py

    * Update fast_lora.py

    * Update llama.py

    * Update llama.py

    * Update cross_entropy_loss.py

    * Update llama.py

    * Update llama.py

    * gemma

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update fast_lora.py

    * Update fast_lora.py

    * Fast CE Loss

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * CE

    * Update llama.py

    * Update llama.py

    * Update cross_entropy_loss.py

    * Update geglu.py

    * Update cross_entropy_loss.py

    * revert

    * Update llama.py

    * Update llama.py

    * norm

    * Update gemma.py

    * Update gemma.py

    * position_ids

    * Update gemma.py

    * Update gemma.py

    * pos

    * Update llama.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update cross_entropy_loss.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update llama.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update llama.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * revert

    * revert

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update llama.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update cross_entropy_loss.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * rope

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * llama

    * Update llama.py

    * gemma

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update gemma.py

    * Update save.py

    * RoPE

    * Update llama.py

    * Update llama.py

    * Update llama.py

    * Update gemma.py

    * correct_dtype

    * Update gemma.py

    * Update cross_entropy_loss.py

    * Update cross_entropy_loss.py

    * Chat Templates

    * Update README.md

    * Update README.md

    * Update llama.py

    * DoRA

    * Update _utils.py

    * Update chat_templates.py

commit 54b26c0466
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 29 00:16:19 2024 +1100

    Update llama.py

commit 074aa737ce
Merge: b1892b5 f946bed
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 29 00:14:42 2024 +1100

    Merge branch 'main' into nightly

commit b1892b5511
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 29 00:09:00 2024 +1100

    Update chat_templates.py

commit 072dc0c447
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 29 00:02:40 2024 +1100

    Update _utils.py

commit d37f284eaa
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 28 23:59:49 2024 +1100

    DoRA

commit 967fed83c7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 28 22:02:18 2024 +1100

    Update llama.py

commit a4cd4a6e41
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 27 01:40:45 2024 +1100

    Update README.md

commit 90a5c2c121
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 27 01:39:52 2024 +1100

    Update README.md

commit 4b7df80abd
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 27 00:16:57 2024 +1100

    Chat Templates

commit 54ff6eb169
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 17:27:42 2024 +1100

    Update cross_entropy_loss.py

commit 7cf69dd9f2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 16:19:39 2024 +1100

    Update cross_entropy_loss.py

commit 1f450cd6ee
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 14:44:22 2024 +1100

    Update gemma.py

commit 7a5db6758c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 14:39:43 2024 +1100

    correct_dtype

commit cf16811ab9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 14:38:27 2024 +1100

    Update gemma.py

commit 637d01643c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 14:37:48 2024 +1100

    Update llama.py

commit 3d73560df3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 14:37:09 2024 +1100

    Update llama.py

commit 0d24acf0f5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 14:36:39 2024 +1100

    Update llama.py

commit 1aeca6f00c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 14:35:44 2024 +1100

    RoPE

commit 4ec1b216e0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 14:15:37 2024 +1100

    Update save.py

commit 11c9ea4d1b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 05:06:04 2024 +1100

    Update gemma.py

commit 24e9d0af28
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 05:05:41 2024 +1100

    Update gemma.py

commit 0b693df646
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 04:37:12 2024 +1100

    Update gemma.py

commit ae336e9663
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 04:36:10 2024 +1100

    Update gemma.py

commit 1d53b366a0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 04:32:49 2024 +1100

    Update gemma.py

commit b1864710c5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 04:30:29 2024 +1100

    Update gemma.py

commit 686d9fa4c4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 04:27:55 2024 +1100

    Update gemma.py

commit 40169f9fd3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 04:23:38 2024 +1100

    Update gemma.py

commit b0b38f770b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 04:20:19 2024 +1100

    Update gemma.py

commit 13e06a062b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:56:34 2024 +1100

    Update gemma.py

commit fd5389a485
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:55:15 2024 +1100

    Update cross_entropy_loss.py

commit a5abe39ded
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:53:49 2024 +1100

    Update cross_entropy_loss.py

commit 78c96ef83c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:50:34 2024 +1100

    Update cross_entropy_loss.py

commit 88cfe5eeb2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:48:16 2024 +1100

    Update cross_entropy_loss.py

commit dba4c0355d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:46:18 2024 +1100

    Update cross_entropy_loss.py

commit 199460d5ef
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:43:22 2024 +1100

    Update cross_entropy_loss.py

commit 13718b9847
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:36:38 2024 +1100

    Update cross_entropy_loss.py

commit 48736a0755
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:35:45 2024 +1100

    Update cross_entropy_loss.py

commit c1dbf6710e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:29:19 2024 +1100

    Update cross_entropy_loss.py

commit b55d7ad4b2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:29:11 2024 +1100

    Update cross_entropy_loss.py

commit bc3ff0f3a6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:28:22 2024 +1100

    Update cross_entropy_loss.py

commit 7bbce701c8
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:26:42 2024 +1100

    Update cross_entropy_loss.py

commit 9c0b3cd431
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:19:26 2024 +1100

    Update cross_entropy_loss.py

commit e696487e84
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:18:01 2024 +1100

    Update cross_entropy_loss.py

commit c675220373
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:16:31 2024 +1100

    Update cross_entropy_loss.py

commit bb6b6f238e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:15:19 2024 +1100

    Update cross_entropy_loss.py

commit 096544eff4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:14:04 2024 +1100

    Update cross_entropy_loss.py

commit b5eff3bfbc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:12:23 2024 +1100

    Update cross_entropy_loss.py

commit fdaa9493be
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 03:09:37 2024 +1100

    Update cross_entropy_loss.py

commit 8259b16a5d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 26 02:49:02 2024 +1100

    gemma

commit df8034d185
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:48:34 2024 +1100

    Update llama.py

commit db2b387b22
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:47:45 2024 +1100

    llama

commit 36d29078cd
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:35:39 2024 +1100

    Update gemma.py

commit 4cc14b2074
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:30:59 2024 +1100

    Update gemma.py

commit bc428f4e7b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:29:01 2024 +1100

    Update gemma.py

commit 7c958ab9e7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:25:57 2024 +1100

    Update gemma.py

commit d530f95135
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:25:15 2024 +1100

    Update gemma.py

commit 6ad44835d2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:22:35 2024 +1100

    Update gemma.py

commit 46723124f7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:21:03 2024 +1100

    Update gemma.py

commit ad1ce483db
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:20:54 2024 +1100

    Update gemma.py

commit 47d4a33f41
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:20:09 2024 +1100

    Update gemma.py

commit 6b531eff93
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:18:23 2024 +1100

    Update gemma.py

commit 147d129772
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:17:16 2024 +1100

    Update gemma.py

commit 884fadc744
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:14:57 2024 +1100

    Update gemma.py

commit 9d648cbccf
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:14:38 2024 +1100

    Update gemma.py

commit 6238f16625
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:08:04 2024 +1100

    Update gemma.py

commit 9a20dff5d2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 04:07:47 2024 +1100

    Update gemma.py

commit 94833602da
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:55:44 2024 +1100

    Update gemma.py

commit 40c244a7bc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:53:40 2024 +1100

    Update gemma.py

commit 0e1578dedb
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:51:32 2024 +1100

    Update gemma.py

commit 33a72ba122
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:50:59 2024 +1100

    Update gemma.py

commit 890d73e6d0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:49:55 2024 +1100

    Update gemma.py

commit 33eeb7add2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:44:22 2024 +1100

    Update gemma.py

commit 096a4192fb
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:39:40 2024 +1100

    Update gemma.py

commit f270f377ab
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:39:31 2024 +1100

    Update gemma.py

commit 765e54fe55
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:38:18 2024 +1100

    Update gemma.py

commit 29721a86d1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:37:30 2024 +1100

    Update gemma.py

commit 208e2c1189
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:32:23 2024 +1100

    Update gemma.py

commit aff1db7a4c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:22:24 2024 +1100

    Update gemma.py

commit 98903e72c2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:21:37 2024 +1100

    Update gemma.py

commit 20da5547f5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:19:34 2024 +1100

    Update gemma.py

commit 20e9ca2fac
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:16:55 2024 +1100

    Update gemma.py

commit 842b310767
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:16:34 2024 +1100

    Update gemma.py

commit 11067cf849
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:14:02 2024 +1100

    Update gemma.py

commit 6725dc93f1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:13:18 2024 +1100

    Update gemma.py

commit baf11e0e34
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:12:07 2024 +1100

    Update gemma.py

commit e0ada344ac
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:11:20 2024 +1100

    Update gemma.py

commit 39a58f7d2e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:07:20 2024 +1100

    Update gemma.py

commit c96e8c28f8
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:03:31 2024 +1100

    Update gemma.py

commit 10d03d8315
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:03:23 2024 +1100

    Update gemma.py

commit 45a33bae1c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:01:56 2024 +1100

    Update gemma.py

commit 77fea80a1b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:01:30 2024 +1100

    Update gemma.py

commit e3517f0248
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 03:00:22 2024 +1100

    Update gemma.py

commit 03c7f211a6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:49:14 2024 +1100

    Update gemma.py

commit 105a0325f1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:42:56 2024 +1100

    Update gemma.py

commit 9d9e38a74a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:41:24 2024 +1100

    Update gemma.py

commit 49a124243e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:37:48 2024 +1100

    Update gemma.py

commit 5c31fd1e5b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:34:18 2024 +1100

    Update gemma.py

commit c0f6b6c0cc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:31:22 2024 +1100

    Update gemma.py

commit 911745377a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:30:06 2024 +1100

    Update gemma.py

commit f73698859e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:28:01 2024 +1100

    Update gemma.py

commit e56eb3f406
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:26:57 2024 +1100

    Update gemma.py

commit 045bce2fa5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:25:39 2024 +1100

    Update gemma.py

commit a847719499
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:18:43 2024 +1100

    Update gemma.py

commit c058d1aa96
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:17:45 2024 +1100

    Update gemma.py

commit dfb2e250f6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:17:12 2024 +1100

    Update gemma.py

commit f2fca2d6c1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:10:35 2024 +1100

    Update gemma.py

commit ae473e0f2b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:09:08 2024 +1100

    rope

commit ce2d74732d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:07:29 2024 +1100

    Update llama.py

commit ed5042aa07
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:04:11 2024 +1100

    Update llama.py

commit 42840647d0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:02:59 2024 +1100

    Update llama.py

commit ecd21b375b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 02:02:02 2024 +1100

    Update llama.py

commit 5bf8cacebd
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 01:58:33 2024 +1100

    Update gemma.py

commit 66a4380d08
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 01:50:42 2024 +1100

    Update gemma.py

commit b85b45e7a0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 01:49:39 2024 +1100

    Update gemma.py

commit 72870a1ed3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 01:48:02 2024 +1100

    Update gemma.py

commit 7b903368ab
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 01:45:44 2024 +1100

    Update gemma.py

commit 0c2d7e503b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 01:39:25 2024 +1100

    Update cross_entropy_loss.py

commit 9ae5abc83f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 01:12:44 2024 +1100

    Update gemma.py

commit 3907ea9003
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 01:10:12 2024 +1100

    Update gemma.py

commit c4c8558a90
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 01:04:09 2024 +1100

    Update gemma.py

commit e0e96ef5ce
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 01:03:04 2024 +1100

    Update gemma.py

commit df22d0cb3a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 00:46:58 2024 +1100

    Update gemma.py

commit b9d9aa1b07
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 25 00:44:44 2024 +1100

    Update gemma.py

commit 785760ba0d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 22:43:00 2024 +1100

    Update gemma.py

commit 103b1cdbed
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 22:42:09 2024 +1100

    Update gemma.py

commit b482ce13e9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 22:40:46 2024 +1100

    Update gemma.py

commit 7be57334cb
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 22:39:28 2024 +1100

    Update gemma.py

commit 2242bdf0a3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 22:30:14 2024 +1100

    Update gemma.py

commit a066c348f2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 22:24:43 2024 +1100

    Update llama.py

commit b3d7e61608
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 20:20:37 2024 +1100

    Update gemma.py

commit 5b5652de36
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 20:17:24 2024 +1100

    Update gemma.py

commit 0f5cc839f3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 20:15:25 2024 +1100

    Update gemma.py

commit 6011a490c6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 20:13:24 2024 +1100

    Update gemma.py

commit 6d94cf88a7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 20:11:23 2024 +1100

    Update gemma.py

commit c706447fd4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 20:04:31 2024 +1100

    Update gemma.py

commit ed3f139a9c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 19:55:22 2024 +1100

    Update gemma.py

commit b7ba95857f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 19:47:10 2024 +1100

    Update gemma.py

commit f0b21f9b7b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 19:45:20 2024 +1100

    Update gemma.py

commit 762790d6a9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 19:35:03 2024 +1100

    Update gemma.py

commit b174e54507
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 19:33:04 2024 +1100

    Update gemma.py

commit 47c6feaf89
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 19:12:52 2024 +1100

    Update gemma.py

commit b2b658cbee
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 19:10:36 2024 +1100

    Update gemma.py

commit 407205dd87
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 18:58:31 2024 +1100

    Update gemma.py

commit 82e45c02a5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 18:25:25 2024 +1100

    revert

commit 94201755db
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 18:17:38 2024 +1100

    revert

commit d5e625b674
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 18:04:40 2024 +1100

    Update cross_entropy_loss.py

commit ba19344fb9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 18:03:45 2024 +1100

    Update cross_entropy_loss.py

commit 7916872408
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:48:19 2024 +1100

    Update llama.py

commit 49129bc66c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:43:19 2024 +1100

    Update gemma.py

commit ebbd4756d4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:41:53 2024 +1100

    Update gemma.py

commit 68e280405c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:40:21 2024 +1100

    Update gemma.py

commit 9af090f6ea
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:32:04 2024 +1100

    Update gemma.py

commit 61add47f37
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:31:10 2024 +1100

    Update gemma.py

commit de96285442
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:29:51 2024 +1100

    Update gemma.py

commit 0abbcdc15a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:29:09 2024 +1100

    Update gemma.py

commit 17036a6e54
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:27:49 2024 +1100

    Update llama.py

commit 8b7de591c1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:26:19 2024 +1100

    Update gemma.py

commit 041aa7d909
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:18:09 2024 +1100

    Update gemma.py

commit 71483e6c8d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:17:26 2024 +1100

    Update gemma.py

commit 8cb12078f9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:15:44 2024 +1100

    Update gemma.py

commit 23e6ebf14d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:13:40 2024 +1100

    Update gemma.py

commit a6752f3f16
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:12:00 2024 +1100

    Update gemma.py

commit 9c565580fb
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:09:03 2024 +1100

    Update gemma.py

commit cd479b4374
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:05:59 2024 +1100

    Update cross_entropy_loss.py

commit 0080357a88
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:03:54 2024 +1100

    Update gemma.py

commit e8de606be9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:02:54 2024 +1100

    Update gemma.py

commit 35adcbf83c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 17:02:14 2024 +1100

    Update gemma.py

commit 7e33aa2520
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 16:59:24 2024 +1100

    Update gemma.py

commit 4c7d21e41b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 16:57:16 2024 +1100

    Update gemma.py

commit 3feae56451
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 16:55:21 2024 +1100

    Update gemma.py

commit c060bad7ce
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 16:47:39 2024 +1100

    Update gemma.py

commit 8ffaf5f109
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 15:44:10 2024 +1100

    Update gemma.py

commit be098122f1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 15:42:12 2024 +1100

    Update llama.py

commit 8a618c60ed
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 14:29:15 2024 +1100

    pos

commit ef235c3508
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 14:27:39 2024 +1100

    Update gemma.py

commit a48adb0a83
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 14:27:31 2024 +1100

    Update gemma.py

commit 8920cda9a7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 14:13:31 2024 +1100

    position_ids

commit a1eab801fa
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 14:06:48 2024 +1100

    Update gemma.py

commit c07aff5188
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 14:05:18 2024 +1100

    Update gemma.py

commit 4c6e122caa
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 14:02:26 2024 +1100

    norm

commit 5c5bb53241
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 13:59:30 2024 +1100

    Update llama.py

commit 0e1826dd8c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 13:55:37 2024 +1100

    Update llama.py

commit 893b5dfe2b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 13:54:00 2024 +1100

    revert

commit 9ab87f3648
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 13:33:05 2024 +1100

    Update cross_entropy_loss.py

commit d2dc658077
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 13:22:17 2024 +1100

    Update geglu.py

commit 1fcbd61f76
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 03:40:56 2024 +1100

    Update cross_entropy_loss.py

commit fadcb311c3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 03:40:43 2024 +1100

    Update llama.py

commit 1a2a10d028
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 03:38:25 2024 +1100

    Update llama.py

commit 17a1a855e0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 03:33:46 2024 +1100

    CE

commit 097629108f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 03:30:43 2024 +1100

    Update llama.py

commit 30cc4ffd67
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 03:27:21 2024 +1100

    Update llama.py

commit 20227ba5c7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 03:14:29 2024 +1100

    Update llama.py

commit 73a8616f99
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 03:03:00 2024 +1100

    Update llama.py

commit bd4fd22f34
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 02:58:05 2024 +1100

    Update cross_entropy_loss.py

commit 2814f02087
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 02:33:28 2024 +1100

    Update cross_entropy_loss.py

commit dd321363f2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 02:30:49 2024 +1100

    Update cross_entropy_loss.py

commit 574b2f787f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 02:29:33 2024 +1100

    Update cross_entropy_loss.py

commit 6184f770e7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 02:10:38 2024 +1100

    Update cross_entropy_loss.py

commit f5f3d6794e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 02:08:26 2024 +1100

    Update cross_entropy_loss.py

commit 88bf684c61
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 02:06:11 2024 +1100

    Update cross_entropy_loss.py

commit 603c71c7f0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 24 02:02:11 2024 +1100

    Fast CE Loss

commit 25339b71f7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 18:12:15 2024 +1100

    Update fast_lora.py

commit ebfc8f8a55
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 17:57:42 2024 +1100

    Update fast_lora.py

commit 6ab5eb75f4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 17:55:58 2024 +1100

    Update llama.py

commit f999bcd52d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 17:46:33 2024 +1100

    Update llama.py

commit 13d0cee4f6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 17:37:17 2024 +1100

    Update llama.py

commit b66f6dbfd3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 17:28:29 2024 +1100

    Update llama.py

commit 5866b08f71
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 04:11:08 2024 +1100

    gemma

commit dd478be2fa
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 03:59:09 2024 +1100

    Update llama.py

commit 2ff33fb270
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 03:52:24 2024 +1100

    Update llama.py

commit 4c9f366688
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 03:27:42 2024 +1100

    Update cross_entropy_loss.py

commit cfebc8d979
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 03:27:03 2024 +1100

    Update llama.py

commit 4b009aad33
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 03:16:50 2024 +1100

    Update llama.py

commit 6f340f5417
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 02:38:16 2024 +1100

    Update fast_lora.py

commit ff27a824fa
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 02:33:37 2024 +1100

    Update llama.py

commit 5d728270d3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 02:28:05 2024 +1100

    Update llama.py

commit e743e58cb6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 02:23:25 2024 +1100

    Update gemma.py

commit 879bdd2efb
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 02:20:26 2024 +1100

    Update gemma.py

commit 9052a85f92
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 02:19:06 2024 +1100

    Update gemma.py

commit bad295a413
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 02:15:12 2024 +1100

    Update llama.py

commit b30d2502cc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 02:12:02 2024 +1100

    Update llama.py

commit 11233cb3d4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 02:10:11 2024 +1100

    model_type

commit 76de9c1fe3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 02:05:31 2024 +1100

    FastGemmaModel

commit c0d32de795
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 02:02:39 2024 +1100

    Update fast_lora.py

commit bd2fa264c9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 01:57:34 2024 +1100

    Update mapper.py

commit 9a1b28d691
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 01:55:25 2024 +1100

    Update pyproject.toml

commit 0beaf18908
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 23 01:50:51 2024 +1100

    Gemma

commit 0659d90d95
Merge: 7372768 3e4c5a3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 21 17:49:57 2024 +1100

    Merge branch 'main' into nightly

commit 7372768d14
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 21 03:29:49 2024 +1100

    original

commit 10d9d56434
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 21 03:25:46 2024 +1100

    spaces

commit edd03f66fb
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 21 03:14:27 2024 +1100

    trainer

commit 028ee5ca06
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 21 02:05:33 2024 +1100

    save

commit 917f791ab7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 21 00:53:17 2024 +1100

    Update save.py

commit bf3e10b26e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 21 00:48:18 2024 +1100

    Update save.py

commit d266332141
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 23:28:37 2024 +1100

    Update save.py

commit 4d1e575047
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 20:01:33 2024 +1100

    Update __init__.py

commit 6aac6c4be8
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 19:50:29 2024 +1100

    Update save.py

commit 83d906a2c0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 18:35:11 2024 +1100

    Update save.py

commit 5d88ffefd5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 18:00:51 2024 +1100

    Update save.py

commit 4c9be6d057
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 17:12:18 2024 +1100

    Update save.py

commit d3eac595a1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 17:11:33 2024 +1100

    Update save.py

commit 632705b9b0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 17:00:19 2024 +1100

    Update save.py

commit 8f60bf57be
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 15:59:38 2024 +1100

    Update save.py

commit 164fa807fe
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 04:44:17 2024 +1100

    Update save.py

commit 11e04a7057
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 04:42:47 2024 +1100

    Update save.py

commit d2e76580ae
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 04:41:40 2024 +1100

    Update save.py

commit 3e3dc37640
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 04:41:34 2024 +1100

    Update save.py

commit d9751e6553
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 04:35:07 2024 +1100

    Update save.py

commit 44659a05a3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 04:13:58 2024 +1100

    Update save.py

commit 34998914ac
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 03:35:19 2024 +1100

    saving

commit 3728b8e876
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 02:19:36 2024 +1100

    Update save.py

commit c20bb30710
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 02:07:01 2024 +1100

    Update save.py

commit 2c207b4989
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 20 00:10:04 2024 +1100

    llama.cpp bugs

commit 0ffd7b46f3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 19 19:39:47 2024 +1100

    linking

commit ad0c2bcd87
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 19 13:08:03 2024 +1100

    Update save.py

commit 0c1b71ac63
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 19 12:51:25 2024 +1100

    Update save.py

commit 14db89f4fc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 19 04:13:40 2024 +1100

    Update save.py

commit e31071f368
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 19 04:01:27 2024 +1100

    Update save.py

commit aa45208f5a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 19 03:39:09 2024 +1100

    Update save.py

commit 28de27dd8d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 19 03:22:21 2024 +1100

    PeftModel token + saving

commit 99cdf0bd5a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 19 02:45:54 2024 +1100

    Update save.py

commit 71555c7678
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 19 02:37:43 2024 +1100

    Update save.py

commit 89d2418cd7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 23:35:18 2024 +1100

    Update save.py

commit 3e89d4e7ec
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 23:31:24 2024 +1100

    Update save.py

commit 457c044473
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 23:25:36 2024 +1100

    install

commit 22378e95cc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 23:12:37 2024 +1100

    Update pyproject.toml

commit 3d789cb09e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 22:59:37 2024 +1100

    Update save.py

commit 31a62c52d3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 22:54:38 2024 +1100

    trainer

commit 870edf3534
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 22:52:51 2024 +1100

    Update save.py

commit b6a6e90b7a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 22:03:22 2024 +1100

    Update save.py

commit 02b7b7f3f2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 20:28:56 2024 +1100

    Update save.py

commit 53f6a07b92
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 20:22:52 2024 +1100

    Update save.py

commit 7ee8243e8b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 20:19:28 2024 +1100

    Update save.py

commit 94b5c58c2e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 20:11:51 2024 +1100

    Update save.py

commit c6ad5f97a3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 20:09:11 2024 +1100

    Update loader.py

commit cc2764cddd
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 19:54:34 2024 +1100

    Update save.py

commit 0af3a1b143
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 19:44:59 2024 +1100

    Update save.py

commit ef8abf4c6e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 19:25:56 2024 +1100

    apache

commit 164b950cad
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 19:24:35 2024 +1100

    spaces

commit 7aedc92637
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 19:21:02 2024 +1100

    slashes

commit 7573ee2c22
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 19:15:41 2024 +1100

    slash

commit e51a381305
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 19:08:04 2024 +1100

    globals

commit c660879cdc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 18:30:03 2024 +1100

    spaces

commit dcbbab3c22
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 18:25:19 2024 +1100

    spaces

commit 5edae1cd79
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 18:21:54 2024 +1100

    readme

commit 22f1f52513
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 18:18:12 2024 +1100

    Update llama.py

commit 2e65e63678
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 18:09:38 2024 +1100

    saving bugs

commit c9a524a99a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 18 04:15:20 2024 +1100

    Bugs

commit 6d3a3b4286
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 15 19:24:07 2024 +1100

    Fix RoPE precision issues

commit 84b37f8548
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 15 03:53:58 2024 +1100

    Update mapper.py

commit bd4a701e7a
Merge: 629c39d a030e80
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 15 03:35:47 2024 +1100

    Merge branch 'main' into nightly

commit 629c39d2e5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 14 23:17:38 2024 +1100

    Update mistral.py

commit 227c26ca44
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 14 23:15:45 2024 +1100

    Update llama.py

commit b660132380
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 14 23:11:06 2024 +1100

    Saving, LlamaRotaryEmbedding issues

commit 91a3c43468
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 14 17:58:41 2024 +1100

    Update chat_templates.py

commit efbb1e6049
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 14 17:56:14 2024 +1100

    patch tokenizer

commit 5f5910ffee
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 14 17:45:02 2024 +1100

    Update chat_templates.py

commit d40a12852e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 14 17:31:33 2024 +1100

    Update chat_templates.py

commit 7c713cb58c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 14 17:27:03 2024 +1100

    Update chat_templates.py

commit b28a383d1c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 14 04:20:41 2024 +1100

    Update chat_templates.py

commit 4f20e20e28
Merge: 2cdf43d 7c46209
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 14 03:25:40 2024 +1100

    Merge branch 'main' into nightly

commit 2cdf43d8b7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 12 04:28:41 2024 +1100

    Chat Templates

commit acd635aa0d
Merge: b7c5296 e091bca
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 11 16:42:37 2024 +1100

    Merge branch 'main' into nightly

commit b7c52963ad
Merge: 868fb27 ba3fa46
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 9 03:52:36 2024 +1100

    Merge branch 'main' into nightly

commit 868fb27e11
Merge: 601dc9e 7e7f5f3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 8 16:50:19 2024 +1100

    Merge branch 'main' into nightly

commit 601dc9ec4b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 8 03:39:45 2024 +1100

    Update llama.py

commit 31de486f1c
Merge: 81128a4 12b19e3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 8 03:39:19 2024 +1100

    revert

commit 81128a4504
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 8 03:00:53 2024 +1100

    Update llama.py

commit 998097394a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 8 02:57:09 2024 +1100

    Update llama.py

commit 277ca9eecf
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 8 02:52:43 2024 +1100

    Update llama.py

commit d6ab9c92d7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 8 02:49:26 2024 +1100

    Update llama.py

commit e094914b0e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 8 02:30:49 2024 +1100

    Update llama.py

commit 9a54a6f05e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 20:15:19 2024 +1100

    Update llama.py

commit e94647c2dc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 19:25:46 2024 +1100

    Update llama.py

commit e8593b7103
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 18:25:08 2024 +1100

    Update llama.py

commit 1065936ecb
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 18:21:36 2024 +1100

    Update llama.py

commit 60b47f6130
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 18:18:43 2024 +1100

    Update llama.py

commit e43e819a57
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 18:02:58 2024 +1100

    Update llama.py

commit 1e77ab25b6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 17:43:51 2024 +1100

    Update llama.py

commit b36e0bfc89
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 17:34:59 2024 +1100

    Update llama.py

commit ab0a3c9976
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 04:40:19 2024 +1100

    Update mistral.py

commit 2b346dc498
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 03:45:07 2024 +1100

    Update save.py

commit 17a6b12ee3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 03:41:32 2024 +1100

    Update save.py

commit 7da7afcc78
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 03:20:55 2024 +1100

    __version__

commit 9e2b00e167
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 03:19:33 2024 +1100

    __version__

commit 9c3849fc1d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 03:18:02 2024 +1100

    Update pyproject.toml

commit bfb3ea7179
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 02:57:54 2024 +1100

    Update save.py

commit 213cfee903
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 02:57:24 2024 +1100

    Update save.py

commit 8b52dc027e
Merge: 8e9d9c3 1d393f7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 02:50:35 2024 +1100

    Merge branch 'main' into nightly

commit 8e9d9c38b1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Feb 7 01:48:31 2024 +1100

    SWA inference

commit d0b1144423
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 6 18:53:32 2024 +1100

    Fix llm_int8_skip_modules

commit 39a2a7c57d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 6 18:50:48 2024 +1100

    Fix SWA inference

commit aa0427f898
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 6 02:11:06 2024 +1100

    Update save.py

commit 262289d5a5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 6 01:54:22 2024 +1100

    Update save.py

commit 53ca91f744
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Feb 6 01:30:48 2024 +1100

    Update save.py

commit e487abd94a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 5 19:07:00 2024 +1100

    Update save.py

commit 797a87a3e3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 5 18:53:27 2024 +1100

    Update save.py

commit e9031ceabe
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 5 18:48:40 2024 +1100

    mistral swa

commit 33192d6dc8
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 5 18:05:25 2024 +1100

    Update save.py

commit 03ed3a83d5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 5 17:38:01 2024 +1100

    Torch 2.2.0

commit a50daa19d8
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 5 17:32:43 2024 +1100

    Update save.py

commit d69cef9f23
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 5 17:16:20 2024 +1100

    Update save.py

commit 1750b13a63
Merge: 63ed23a 35f2ab4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Feb 5 02:29:56 2024 +1100

    Merge branch 'main' into nightly

commit 63ed23ae98
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 14:02:26 2024 +1100

    Update utils.py

commit 990068b977
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 13:50:21 2024 +1100

    Update utils.py

commit 6ab4019e4a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 04:07:47 2024 +1100

    Update llama.py

commit 7ab3426608
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:57:29 2024 +1100

    Update llama.py

commit 201d90c3ba
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:54:12 2024 +1100

    Update llama.py

commit 00242d5f89
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:52:08 2024 +1100

    Update llama.py

commit 3bfb0ebf9d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:50:31 2024 +1100

    Update llama.py

commit 71899d7060
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:48:18 2024 +1100

    Update llama.py

commit 9cd2517949
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:46:30 2024 +1100

    Update llama.py

commit 24431b4347
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:45:25 2024 +1100

    Update llama.py

commit 63816fc119
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:42:41 2024 +1100

    Update llama.py

commit 7166b11599
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:41:15 2024 +1100

    Update llama.py

commit d867b9bbdf
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:27:15 2024 +1100

    Update llama.py

commit 9a5ebef148
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:23:26 2024 +1100

    Update llama.py

commit 65270cec60
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:13:42 2024 +1100

    Update llama.py

commit 80fa8e93c9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:12:07 2024 +1100

    Update llama.py

commit 36b400e598
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:08:49 2024 +1100

    Update llama.py

commit 4a5d3b1de7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 03:05:49 2024 +1100

    Update llama.py

commit 88a695da3a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 02:49:51 2024 +1100

    Update llama.py

commit ac9bc79251
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 02:42:30 2024 +1100

    Update llama.py

commit 8c8685eeef
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 02:40:30 2024 +1100

    Update llama.py

commit 607dfa1d0e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 02:35:17 2024 +1100

    Update llama.py

commit 54802ecbb9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 02:32:21 2024 +1100

    New version

commit 711e5c0922
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 02:19:42 2024 +1100

    attention_mask

commit 74d7fc65c6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 02:03:23 2024 +1100

    SDPA

commit fcb884643b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 02:02:51 2024 +1100

    Update llama.py

commit aa032fc80d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 01:46:04 2024 +1100

    Update mistral.py

commit 31578f2010
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Feb 4 01:42:46 2024 +1100

    fast inference

commit 257cd7d531
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 23:22:21 2024 +1100

    Update llama.py

commit e500b785f7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 23:12:16 2024 +1100

    Update llama.py

commit cf0fae9a55
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 23:01:56 2024 +1100

    Update llama.py

commit 381b991c45
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 22:42:11 2024 +1100

    Update llama.py

commit 665908eb70
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 22:28:20 2024 +1100

    Update llama.py

commit 5534f8a8f1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 20:22:42 2024 +1100

    more temp matrices

commit 68db1c7af7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 19:44:36 2024 +1100

    fast inference again

commit d76f583349
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 19:30:22 2024 +1100

    Update mistral.py

commit 9225dd6708
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 19:19:13 2024 +1100

    Update llama.py

commit 8270821269
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 19:08:42 2024 +1100

    Update llama.py

commit 1c6e1f18b4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 18:32:18 2024 +1100

    Update llama.py

commit dd03abedd6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 18:20:06 2024 +1100

    Update llama.py

commit 522f6dbfd2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 18:07:55 2024 +1100

    Update llama.py

commit ad357dec89
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 03:52:25 2024 +1100

    fast inference + saving config.json

commit a78d6fba7e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 03:34:02 2024 +1100

    Update llama.py

commit fa3d23406e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 03:10:39 2024 +1100

    Update utils.py

commit 8e3f0296be
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 02:55:22 2024 +1100

    Update utils.py

commit 404e177adb
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 02:30:47 2024 +1100

    Update utils.py

commit a81d193830
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 02:20:00 2024 +1100

    Update utils.py

commit 4436a1099d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 01:50:48 2024 +1100

    Update llama.py

commit a15ffc95e0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 01:48:03 2024 +1100

    Update llama.py

commit 0be46406d8
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 01:39:39 2024 +1100

    Update llama.py

commit e911047188
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 01:35:45 2024 +1100

    Update llama.py

commit c934c16e21
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 01:29:50 2024 +1100

    Update llama.py

commit 705bbba5c5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Feb 3 01:22:53 2024 +1100

    Update llama.py

commit 9c2bed35b9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 23:45:02 2024 +1100

    Update llama.py

commit ea9b4eea0c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 23:40:52 2024 +1100

    Update llama.py

commit c6ad936f88
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 23:32:58 2024 +1100

    Update llama.py

commit 03df291110
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 23:32:36 2024 +1100

    Update llama.py

commit dc2740416b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 22:26:47 2024 +1100

    Update llama.py

commit b33c92d1bd
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 22:22:42 2024 +1100

    Update llama.py

commit 923c6bad14
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 22:20:45 2024 +1100

    Update llama.py

commit b3703926ca
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 22:17:02 2024 +1100

    Update llama.py

commit 1b274212e1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 20:35:00 2024 +1100

    Update llama.py

commit 9df19aeb9f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 20:32:21 2024 +1100

    past_key_values

commit 82eea75cde
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 20:29:35 2024 +1100

    torch compile

commit e748078095
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 20:24:44 2024 +1100

    Update llama.py

commit 9146fa443d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 20:24:28 2024 +1100

    Update llama.py

commit a5a123db08
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 20:20:14 2024 +1100

    Update llama.py

commit 82ead808d5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 20:19:15 2024 +1100

    Update llama.py

commit 1497521e5c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 20:06:52 2024 +1100

    Update mistral.py

commit c3c454a729
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 20:06:31 2024 +1100

    Update llama.py

commit f299c9cde1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 19:44:52 2024 +1100

    Update llama.py

commit 0e1d67d4e1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 19:07:41 2024 +1100

    fast inference

commit 168ded977e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 18:28:05 2024 +1100

    Update llama.py

commit bf441055ca
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 18:14:31 2024 +1100

    faster inference

commit 7c2b04254c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 16:59:02 2024 +1100

    Update llama.py

commit ca99d7c194
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 16:46:30 2024 +1100

    Update mistral.py

commit 40e8848c4e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 15:56:56 2024 +1100

    Update llama.py

commit 6cc9835021
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 13:52:12 2024 +1100

    Update llama.py

commit 7ad1a1fa62
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 04:18:34 2024 +1100

    Update llama.py

commit 0b661a23b9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Feb 2 03:55:36 2024 +1100

    Update llama.py

commit da64d3403c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 23:54:01 2024 +1100

    Update llama.py

commit abc47836dc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 23:41:32 2024 +1100

    Update llama.py

commit f231c4f395
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 23:41:23 2024 +1100

    Update llama.py

commit 5f3c51b394
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 23:41:11 2024 +1100

    Update llama.py

commit e0ea238256
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 23:40:36 2024 +1100

    Update llama.py

commit 73f63d6884
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 23:38:07 2024 +1100

    Update llama.py

commit e4b5e38800
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 23:17:53 2024 +1100

    inference

commit 334c5ed1f0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 22:48:54 2024 +1100

    Update llama.py

commit cd39f6108f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 22:34:33 2024 +1100

    lm_head

commit 24c4c37b7c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 22:21:35 2024 +1100

    revert

commit e791db9b06
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 20:12:59 2024 +1100

    Update llama.py

commit 1793a16c8f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 20:03:54 2024 +1100

    faster inference

commit 19fb50e244
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 19:56:54 2024 +1100

    Update utils.py

commit b83cea7cb4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 19:56:06 2024 +1100

    Update llama.py

commit 8920cafbe7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 19:46:38 2024 +1100

    inference

commit 38b59825b1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 19:36:36 2024 +1100

    Update llama.py

commit e8ec80a4c2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 19:19:44 2024 +1100

    Update llama.py

commit a2cb7a113b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 19:07:30 2024 +1100

    Update llama.py

commit a5ee70b63f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 18:34:43 2024 +1100

    Update llama.py

commit e2f0dd8683
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 18:16:46 2024 +1100

    Update llama.py

commit 3f0ddf0c7c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 18:16:15 2024 +1100

    Update llama.py

commit e90b3bf192
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 17:59:21 2024 +1100

    Update llama.py

commit 57044509ad
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 17:44:22 2024 +1100

    Update llama.py

commit cf4b58eeb6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 17:20:02 2024 +1100

    inference

commit 648c79ec1b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 03:47:51 2024 +1100

    faster inference

commit 20f19391e6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 03:05:02 2024 +1100

    Update mistral.py

commit e2f72fe52f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 03:04:46 2024 +1100

    Revert

commit 329f80ac4c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 02:47:57 2024 +1100

    Update llama.py

commit 713a95ca0e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Feb 1 02:46:26 2024 +1100

    Inference

commit acbdef7ff5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Jan 31 20:16:08 2024 +1100

    padding

commit 0dc26ed98a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Jan 31 20:06:28 2024 +1100

    Update llama.py

commit 9f31254539
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Jan 31 20:02:21 2024 +1100

    Fix SDPA

commit 7227de48cf
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Jan 31 19:48:33 2024 +1100

    Update llama.py

commit 5edad55b5d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Jan 31 19:44:19 2024 +1100

    Update llama.py

commit c928c57aa9
Merge: 5da0555 051a73b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Jan 31 19:38:27 2024 +1100

    Merge branch 'main' into nightly

commit 5da05558a0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Jan 31 03:50:47 2024 +1100

    past_key_value

commit d347db0944
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Jan 31 03:46:11 2024 +1100

    Update loader.py

commit c0edaa46db
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Jan 31 03:40:55 2024 +1100

    revert inference

commit 55fe6052ca
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Jan 31 03:36:40 2024 +1100

    if past_key_value is not None and q_len == 1:

commit 248887b51c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Jan 31 03:35:24 2024 +1100

    LlamaAttention_fast_forward_inference

commit c68a3bc9ec
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Jan 31 03:28:30 2024 +1100

    Update loader.py

commit 44168377f3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Wed Jan 31 01:32:56 2024 +1100

    Update rope_embedding.py

commit 270df81b60
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 30 17:33:08 2024 +1100

    Remove fast path

commit b8f665bf22
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 30 17:26:45 2024 +1100

    fast lm_head

commit ed5a653ecf
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 30 04:16:39 2024 +1100

    Update mistral.py

commit e0bad0eec5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 30 04:10:14 2024 +1100

    Fix inference

commit 71725aeea5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 30 03:23:42 2024 +1100

    Update __init__.py

commit 3ddda6f492
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 30 03:08:46 2024 +1100

    Update mistral.py

commit 6f74c98fbc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 23:19:43 2024 +1100

    Update utils.py

commit a7bfeec919
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 23:14:51 2024 +1100

    Update utils.py

commit 7c87d60bc1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 23:07:40 2024 +1100

    Update utils.py

commit bb364204cc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 20:09:59 2024 +1100

    Update llama.py

commit 4700d51916
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 19:55:40 2024 +1100

    Fast inference repatch

commit 58cabcbc06
Merge: 01b8162 0562464
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 17:59:34 2024 +1100

    Merge branch 'main' into nightly

commit 01b8162244
Merge: 5cfea20 206a9b6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 17:49:57 2024 +1100

    Merge branch 'main' into nightly

commit 5cfea20129
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 17:35:19 2024 +1100

    Update llama.py

commit 25a88ea003
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 16:52:04 2024 +1100

    Update llama.py

commit 03ca52dc94
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 03:43:39 2024 +1100

    saving

commit 01e5c305f9
Merge: e10e488 8faf469
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 03:43:19 2024 +1100

    Merge branch 'main' into nightly

commit e10e48893b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 03:43:17 2024 +1100

    Update save.py

commit 5bd916b91b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 03:43:05 2024 +1100

    Update mistral.py

commit 11ba2c520b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 03:34:27 2024 +1100

    Mistral patch

commit 498dfb8acc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 02:42:08 2024 +1100

    print

commit a3d2b9b778
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 02:15:52 2024 +1100

    Update save.py

commit 9e00cc287a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 00:59:29 2024 +1100

    Update save.py

commit 460de24ea2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 00:57:11 2024 +1100

    Update save.py

commit b060d7b621
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 00:56:44 2024 +1100

    Update save.py

commit 74b69775c2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 00:51:57 2024 +1100

    Update save.py

commit fef0589d6e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 00:51:45 2024 +1100

    Update save.py

commit 9dec4b3f08
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 29 00:49:31 2024 +1100

    Update save.py

commit ee0bf6fe66
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 20:07:17 2024 +1100

    Update save.py

commit c69c166b82
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 18:12:11 2024 +1100

    patch_saving_functions

commit ac02ba6c38
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 18:08:52 2024 +1100

    Update save.py

commit 20e524a596
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 18:04:35 2024 +1100

    Update save.py

commit 788e695180
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 17:48:34 2024 +1100

    Patch saving

commit 31222ced74
Merge: 893aab0 1ecc018
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 17:48:28 2024 +1100

    Merge branch 'main' into nightly

commit 893aab0e57
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 17:20:49 2024 +1100

    Update dpo.py

commit d5c852e711
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 16:47:33 2024 +1100

    Update llama.py

commit ddb48efd33
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 16:43:10 2024 +1100

    Update llama.py

commit 5fe166d32e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 16:40:59 2024 +1100

    Update llama.py

commit 2d64e0a904
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 16:37:21 2024 +1100

    Update mistral.py

commit 2f73cb4049
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 16:31:01 2024 +1100

    Update llama.py

commit 6aa46ffff6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 15:13:54 2024 +1100

    Update llama.py

commit ee6f5096ed
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 13:53:06 2024 +1100

    attention mask

commit f1b0fd0848
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 13:37:38 2024 +1100

    Update mistral.py

commit 7663a32753
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 04:29:52 2024 +1100

    Update save.py

commit a158003625
Merge: 36829b7 cd32ba7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 04:29:35 2024 +1100

    Merge branch 'nightly' of https://github.com/unslothai/unsloth into nightly

commit 36829b7ec4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 04:27:10 2024 +1100

    Update save.py

commit 917ce15861
Merge: 166f8c8 89daa0e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 04:20:15 2024 +1100

    Merge branch 'main' into nightly

commit 166f8c812e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 04:18:19 2024 +1100

    attention mask

commit 6c7f0dbcb4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 04:08:19 2024 +1100

    Update llama.py

commit c836ed7f3d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 04:08:03 2024 +1100

    Update mistral.py

commit 12d57e5308
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 03:59:23 2024 +1100

    labels

commit 4c5ebcc960
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 03:56:54 2024 +1100

    Update llama.py

commit 6a027a8292
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 03:55:23 2024 +1100

    Update llama.py

commit 9f9739cbac
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 03:49:54 2024 +1100

    attention_mask

commit 2bd77e7277
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 02:34:03 2024 +1100

    Update fast_lora.py

commit a1e5aca7cc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 01:27:46 2024 +1100

    Update fast_lora.py

commit a2f705d65b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 01:17:27 2024 +1100

    Update fast_lora.py

commit d01ba458df
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 28 00:45:55 2024 +1100

    Update fast_lora.py

commit 6fa0635971
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 23:20:41 2024 +1100

    Update fast_lora.py

commit e094af81a1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 20:39:19 2024 +1100

    Update fast_lora.py

commit c74e1af85c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 20:37:43 2024 +1100

    Update fast_lora.py

commit 363ffba1c2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 20:17:08 2024 +1100

    Update fast_lora.py

commit 510c85f412
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 20:04:55 2024 +1100

    Update swiglu.py

commit 35daafdd6e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 19:53:57 2024 +1100

    Update fast_lora.py

commit 6201f7681f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 19:38:28 2024 +1100

    Update fast_lora.py

commit 86a1c9788b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 19:29:53 2024 +1100

    Update fast_lora.py

commit 8e0e4ccdc6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 19:18:59 2024 +1100

    Update fast_lora.py

commit 2e4c59deaf
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 19:05:02 2024 +1100

    Update swiglu.py

commit 85e87d9ba2
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 19:03:04 2024 +1100

    Swiglu

commit 83b6937285
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 18:21:29 2024 +1100

    Update fast_lora.py

commit 3d3e7f5b42
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 18:15:34 2024 +1100

    Update fast_lora.py

commit d3f3b6fc49
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 18:00:41 2024 +1100

    Update fast_lora.py

commit e77d7c069f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 18:00:22 2024 +1100

    Update fast_lora.py

commit f7d11d10f8
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 17:47:54 2024 +1100

    Update fast_lora.py

commit 8ed03f5f45
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 17:38:30 2024 +1100

    Update pyproject.toml

commit af65cb0d3d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:50:47 2024 +1100

    Works?

commit fb5333726a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:49:36 2024 +1100

    Update llama.py

commit a59ec7903c
Merge: 704e36a 87a7ef1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:48:40 2024 +1100

    Merge branch 'main' into nightly

commit 704e36a64e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:48:34 2024 +1100

    Revert "Update llama.py"

    This reverts commit a208ec46e0.

commit a208ec46e0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:48:03 2024 +1100

    Update llama.py

commit bd2ff90817
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:47:42 2024 +1100

    Update llama.py

commit a3d892a15b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:47:01 2024 +1100

    Update llama.py

commit b89599a3f7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:46:47 2024 +1100

    Update llama.py

commit baeea64917
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:20:58 2024 +1100

    Update save.py

commit ecdbb28dcb
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:19:59 2024 +1100

    Update save.py

commit 393341f211
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:18:25 2024 +1100

    Update swiglu.py

commit 47babc780a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:16:38 2024 +1100

    Update fast_lora.py

commit 9edc309f59
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:15:37 2024 +1100

    Update llama.py

commit a27ac6175e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 04:03:47 2024 +1100

    Update utils.py

commit 7fb64e0e31
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 03:58:21 2024 +1100

    Update fast_lora.py

commit e3bd0bb74f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 03:51:53 2024 +1100

    Update save.py

commit ba847f5411
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 03:38:33 2024 +1100

    Update fast_lora.py

commit c57495df6f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 03:38:19 2024 +1100

    Update fast_lora.py

commit d379bb8cf5
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 03:36:27 2024 +1100

    Update fast_lora.py

commit 421ed33b42
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 03:23:50 2024 +1100

    Update fast_lora.py

commit 83ceb11cc9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 03:12:53 2024 +1100

    Update fast_lora.py

commit f3da0d2e4a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 02:52:54 2024 +1100

    Update fast_lora.py

commit 4ae3ad3bce
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 02:34:21 2024 +1100

    Update fast_lora.py

commit c74d3dd3b9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 02:22:15 2024 +1100

    Update fast_lora.py

commit 0a1aa98d3c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 02:19:00 2024 +1100

    Update fast_lora.py

commit f7760719f0
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 01:28:03 2024 +1100

    Update fast_lora.py

commit 38bff800b6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sat Jan 27 01:16:31 2024 +1100

    Update fast_lora.py

commit 485d54fd70
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 23:48:17 2024 +1100

    Update fast_lora.py

commit ce08a15cee
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 23:33:17 2024 +1100

    Update fast_lora.py

commit b4492eb29c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 23:14:44 2024 +1100

    Update fast_lora.py

commit 3865c8cd68
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 23:06:06 2024 +1100

    Update fast_lora.py

commit e9e34f1d4d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 22:29:38 2024 +1100

    Update fast_lora.py

commit d96f665510
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 22:21:54 2024 +1100

    Update swiglu.py

commit f8aa20d2db
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 22:09:32 2024 +1100

    Update fast_lora.py

commit ae0d9380c1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 22:07:10 2024 +1100

    Update swiglu.py

commit 694da2dba3
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 20:06:52 2024 +1100

    Update fast_lora.py

commit 4f573494da
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 20:04:51 2024 +1100

    Update fast_lora.py

commit cb06ce849b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 19:41:48 2024 +1100

    Update fast_lora.py

commit e0a36b356b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 19:30:18 2024 +1100

    Update fast_lora.py

commit c4f0de58a9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 19:21:00 2024 +1100

    Update fast_lora.py

commit a0a409d5af
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 19:20:53 2024 +1100

    Update fast_lora.py

commit 97658f9a74
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 19:12:08 2024 +1100

    Update fast_lora.py

commit 979de52220
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 19:00:22 2024 +1100

    Update fast_lora.py

commit 39d251757f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 18:19:36 2024 +1100

    Update fast_lora.py

commit 77365cc740
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 18:11:43 2024 +1100

    Update llama.py

commit 4bfbccec4b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 18:04:59 2024 +1100

    Update fast_lora.py

commit 56dffca23f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 17:44:10 2024 +1100

    Update llama.py

commit 259097bc61
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 17:36:40 2024 +1100

    Update fast_lora.py

commit b8de6b6a9b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 17:25:36 2024 +1100

    Update fast_lora.py

commit 796aa4d0ef
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 17:13:09 2024 +1100

    Update fast_lora.py

commit 9ecb3bc869
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 17:04:33 2024 +1100

    Update fast_lora.py

commit 7bed11a8ba
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 16:43:50 2024 +1100

    Update fast_lora.py

commit 317bbc57fe
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 16:16:32 2024 +1100

    Update fast_lora.py

commit c4ac728d4c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 16:08:10 2024 +1100

    Update fast_lora.py

commit ab44945336
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 14:06:25 2024 +1100

    Update fast_lora.py

commit 84772de40f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 13:59:57 2024 +1100

    Update fast_lora.py

commit 653f1036ae
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 03:49:48 2024 +1100

    Update fast_lora.py

commit 46ec8bbc3d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 03:15:23 2024 +1100

    Repatch

commit 99eeebf72a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 02:41:15 2024 +1100

    Update swiglu.py

commit ae156d9154
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 02:07:35 2024 +1100

    Update llama.py

commit 29945bd8c1
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Fri Jan 26 01:33:23 2024 +1100

    Update llama.py

commit 194093e375
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Jan 25 23:35:56 2024 +1100

    remove patching

commit 4f60055f9e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Jan 25 23:25:17 2024 +1100

    Update fast_lora.py

commit 2d25facc7b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Jan 25 23:14:37 2024 +1100

    Update fast_lora.py

commit 380f2fd6e4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Jan 25 19:05:43 2024 +1100

    Fix saving and bnb-4bit

commit 0474451a6a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Jan 25 03:55:26 2024 +1100

    Update pyproject.toml

commit 5a5d34b8ae
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Jan 25 03:47:23 2024 +1100

    Update mapper.py

commit bbf5ef65af
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Jan 25 03:46:23 2024 +1100

    Graceful FA2 error + torch 2.1.1

commit 5ea888c578
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Thu Jan 25 02:29:37 2024 +1100

    Update to transformers 4.37

commit d2f1521a49
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 23:23:05 2024 +1100

    incorrect inference

commit f186fe9bc9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 19:44:45 2024 +1100

    Update mistral.py

commit e184b06c4b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 19:28:17 2024 +1100

    Update mistral.py

commit b314500993
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 19:23:58 2024 +1100

    q_len issue

commit d6e85d7731
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 19:17:42 2024 +1100

    q_len == 1

commit 5d9e68181e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 19:11:17 2024 +1100

    hidden_states

commit f311a8efcc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 18:04:20 2024 +1100

    Update llama.py

commit eeee6333ec
Merge: 20d8f22 a833f40
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 18:04:10 2024 +1100

    Merge branch 'main' into nightly

commit 20d8f223d7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 03:47:28 2024 +1100

    Fast LoRA saving

commit 1bb1c3c2b9
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 03:36:50 2024 +1100

    LoRA

commit 24c7a67556
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 02:44:52 2024 +1100

    Update llama.py

commit d87ef86991
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 02:22:37 2024 +1100

    Update llama.py

commit a77c448939
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 02:18:37 2024 +1100

    Update llama.py

commit 5260ec2a0a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 02:13:01 2024 +1100

    Update llama.py

commit 716e03fe1b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 02:09:35 2024 +1100

    Update llama.py

commit 00f50876f7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 02:03:25 2024 +1100

    Update llama.py

commit e5b5333137
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 02:00:10 2024 +1100

    Update llama.py

commit 9a5062e6c4
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 01:44:10 2024 +1100

    Update llama.py

commit 1289ae825b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 01:41:20 2024 +1100

    RoPE

commit 7e4140ebfc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 01:37:36 2024 +1100

    Update llama.py

commit 1ba28d8e26
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 01:33:51 2024 +1100

    Update llama.py

commit 3e1b244d5e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 01:29:03 2024 +1100

    Fast inference RoPE

commit 8647f0e86e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 01:06:34 2024 +1100

    Update llama.py

commit 085a8e944a
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Tue Jan 23 00:32:57 2024 +1100

    inference

commit f41a437540
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 22 22:34:33 2024 +1100

    Update utils.py

commit 31f0c9c08d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 22 22:31:38 2024 +1100

    Update utils.py

commit 4220f6a6dc
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 22 22:26:55 2024 +1100

    No print

commit e1cbc9e423
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 22 22:21:29 2024 +1100

    Update utils.py

commit 050c61bc0b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 22 20:04:27 2024 +1100

    Update utils.py

commit 7c3b647ba7
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 22 20:01:59 2024 +1100

    fast_linear_forward

commit 8c613e851f
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 22 19:36:17 2024 +1100

    Apache 2

commit a18f982e67
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 22 19:31:34 2024 +1100

    Max sequence lengths

commit e38485feaa
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 22 18:18:23 2024 +1100

    Mistral correct RoPE scaling

commit 278de9f375
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 22 04:33:44 2024 +1100

    Update llama.py

commit 0666589ace
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 22 04:28:24 2024 +1100

    Update save.py

commit e34393f975
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 22 04:25:37 2024 +1100

    Update llama.py

commit 2a3d4f3a8d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 22 04:18:42 2024 +1100

    fast inference

commit a5ab4dc21a
Merge: 8828ece b370c9c
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Mon Jan 22 01:58:00 2024 +1100

    Merge branch 'main' into nightly

commit 8828eceb5b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 22:18:53 2024 +1100

    Update llama.py

commit fcde58859b
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 20:06:14 2024 +1100

    Update llama.py

commit 2f80890578
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 19:21:56 2024 +1100

    Update llama.py

commit 7be801ff8d
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 19:10:48 2024 +1100

    Update llama.py

commit 92512e670e
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 19:01:45 2024 +1100

    Update llama.py

commit 5c927e4930
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 19:01:28 2024 +1100

    Update mistral.py

commit fe2bc30987
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 18:52:25 2024 +1100

    Update llama.py

commit ac99a47a45
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 18:52:15 2024 +1100

    Update llama.py

commit 5bf108ebda
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 18:15:55 2024 +1100

    Update llama.py

commit b591f33a37
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 18:06:44 2024 +1100

    Update llama.py

commit da7b4f59ee
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 17:32:37 2024 +1100

    Update save.py

commit 196ab974d6
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 16:53:58 2024 +1100

    Update llama.py

commit cbc1c69e29
Author: Daniel Han-Chen <danielhanchen@gmail.com>
Date:   Sun Jan 21 16:22:28 2024 +1100

    faster saving & inference

unsloth/__init__.py

@@ -53,8 +53,6 @@ if (major_torch < 2):
 elif (major_torch == 2) and (minor_torch < 2):
     # Disable expandable_segments
     del os.environ["PYTORCH_CUDA_ALLOC_CONF"]
-    # Must reimport Pytorch!
-    importlib.reload(torch)
 pass